Methodology

Federal awards — USAspending.gov

The official federal award database (contracts, grants, loans, and subawards), mandated by the DATA Act. We download bulk archive ZIP files from files.usaspending.gov/award_data_archive/, convert them to compressed Parquet, and record the exact file name, URL, and SHA-256 hash of every file. Current scope: Department of Defense agencies, FY2017 onward. Update cadence: monthly.

DoD budget justification books (“J-books”)

The detailed budget submissions the Pentagon sends to Congress each spring, published at comptroller.defense.gov. These cover every R&D program (R-2 exhibits) and procurement budget line (P-40 exhibits) with program narratives, project-level cost tables, and congressional justifications.

The Pentagon's budget system embeds its own database inside these PDFs: the full structured XML is attached inside the PDF itself. We extract that XML directly. For the rare document that lacks an XML attachment, we use a deterministic PDF parser as a fallback, with an accuracy gate requiring ≥98% numeric-field agreement against XML-backed ground truth. We also download the official R-1 and P-1 Excel rollups, which serve as independent control totals. Update cadence: annual.

Improper-payment estimates — paymentaccuracy.gov

Agencies are legally required to estimate and report payment-error rates. The dollar exposure figure we show per agency is derived by multiplying the published rate by the published outlay figure. The federal government reported approximately $186 billion in improper payments in FY2025. This is a derived estimate and is labeled as such. Update cadence: annual.

GAO high-risk list — gao.gov/high-risk-list

The GAO's biennial list of federal programs at high risk for fraud, waste, or mismanagement. Update cadence: biennial.

Senate lobbying disclosures — lda.senate.gov

The Senate Lobbying Disclosure Act database (lda.senate.gov/api/v1) contains filings for 2025 and prior years, each with a permanent UUID, registrant, client company, dollar amounts, agencies lobbied, and issue text that frequently names specific programs. We have linked LDA client names to our company-family database: 32,780 program mentions across 245 programs connect filings to budget lines. Lobbying income and expenditure by year are shown alongside federal obligations received — influence is presented side by side with outcomes, never as a causal claim.

State checkbooks — California and Connecticut (pilot)

California's Open Fi$Cal and Connecticut's OpenCheckbook publish transaction-level government spending. We aggregate by department, spending category, and fiscal year, and use Census Bureau population estimates (NST-EST series) for per-capita comparisons. The category mappings that bridge both states' classification systems are published alongside the data.

California data covers FY2025 only — CA Open Fi$Cal updates on a lag; prior years not yet ingested. why →

Reconciliation

Every figure extracted from a J-book clears two arithmetic checks. Check A: project-level amounts within an exhibit must sum to the program-element total in that same exhibit (tolerance: ±$0.001M). Check B: that program-element total must match the corresponding row in the official R-1 or P-1 Excel rollup. Failures do not get published — they go to a human review queue.

Zero-absent rule

When a program has no funding for a given fiscal year, the official rollup workbook simply omits the row. Our reconciliation code recognizes this so zero-funded programs are not incorrectly flagged as failures.

Coverage gate

At least 99% of R-1 program elements must have either extracted detail or an explicit gap record. Silent holes are a build failure.

Provenance spot-check

Each build randomly samples 50 served facts and mechanically verifies that the cited source document exists on disk, its SHA-256 matches the download manifest, and the XML element path resolves to a real node. A number whose citation chain breaks does not render.

Per-build automated checks

197 automated test functions across 42 test modules, plus phase-level verification gates, plus 21 dbt data-model assertions run on every build. The analyst-agent evaluation set (45 question-answer pairs) requires ≥41 correct answers and 100% citation resolution before shipping.

Citation tiers

Every rendered figure is in one of three states. Cited (underlined, clickable): a fact_id resolves to a source document in our citation index — J-book PDF page-and-bbox, or a specific workbook cell. XML-path chip: a zero-dollar budget line that exists in the structured XML but has no corresponding citation row — the XML element path is displayed. Citation tier pending (⁂): figure is from a dataset (USAspending, LDA filings, trajectory workbooks) for which row-level citation linkage is not yet complete — methodology work in progress.

Company families — registry fact vs. name inference

High confidence (registry fact): SAM.gov records a common registered parent name for the subsidiaries. Medium confidence (name inference): slightly different legal-name variants normalize to the same string. Both tiers appear on screen; the method is always disclosed.

Budget-to-contract links

Connecting a budget program element to the contracts that funded it is an inference. We use three tiers. High: the award's federal account code matches the budget line's appropriation, and program-title keywords overlap substantially. Medium: account matches and the contracting sub-agency matches the budget organization. Low: only the account matches. Low-tier links are useful for exploration but are not evidence of a program-to-program connection.

Derived figures are labeled derived

Any figure computed from published rates or published subtotals — rather than directly reported in a source document — is labeled as derived wherever it appears.

Year-over-Year Swings (yoy_swing)

Programs where FY2025 total is ≥ $50M and the absolute percentage change to FY2026 is ≥ 50%. Budget figures come from the fct_budget_trajectory mart (trajectory pivot of FY2025 and FY2026 enacted/requested budget workbook lines). The figure shown is the percentage change; the delta in dollar terms is the cited trajectory figure.

Zeroed in FY2026 (zeroed_fy2026)

Programs that had a positive FY2025 total but show zero or null in FY2026. The figure shown is the last known FY2025 amount. No FY2025 floor — any positive amount qualifies. Programs may be cancelled, transferred, or restructured into another line item.

Award Concentration Shifts (concentration_shift)

Programs whose Herfindahl-Hirschman Index (HHI), computed from high-confidence award transactions grouped by fiscal year, is non-trivial. The HHI floor is $5M in matched obligations. HHI = sum(share² × 10,000) where share = family_obligation / total_obligation; only positive obligations are included (negative/recoupment flows are excluded). An HHI above 2,500 indicates near-monopoly concentration; above 1,500 is moderately concentrated.

New Defense Contractors (new_entrant)

Entity families whose first award year in the DoD transaction data is FY2024 or later and whose cumulative positive obligations exceed $1M. “First award year” is determined from the USAspending award archive (FY2017 onward). Entities that received their first award before FY2017 may appear as new entrants due to archive coverage limits — treat as a weak signal. The figure shown is total cumulative obligations (USD).