Converting XBRL Financials to JSON for Backtesting

Every public US company files financial data to the SEC in XBRL — a tagged XML format designed for machines. In theory, that means you can grab Apple's revenue history, Tesla's cash flows, or every regional bank's loan book directly from EDGAR, for free, in a structured form.

In practice, anyone who has tried to feed raw XBRL into a backtester has discovered the data is a minefield: thousands of overlapping concept names, dimensional facts that look duplicated but are not, fiscal calendars that drift across years, and a missing fourth quarter that has to be derived. This guide walks through what the format actually contains, the parsing pipeline that turns it into usable JSON, and a practical workflow for backtesting against point-in-time fundamentals.

What XBRL actually is

XBRL (eXtensible Business Reporting Language) is XML with a heavy domain layer on top. A 10-K filing typically ships several documents: an instance file with the actual numbers, a taxonomy schema that defines the concepts, and link bases that describe how concepts relate (presentation, calculation, definition, label). The SEC mandates the us-gaap taxonomy plus an issuer-specific extension taxonomy where companies coin their own concepts.

Each fact in the instance document carries a concept name, a value, a unit, a context (period and entity), an optional set of dimensions (axis → member pairs), a precision/decimals attribute, and a language for narrative facts. A single 10-K can contain six to ten thousand facts.

<us-gaap:Revenues
    contextRef="FD2024Q4YTD"
    unitRef="USD"
    decimals="-6"
    id="fact-001">383285000000</us-gaap:Revenues>

<context id="FD2024Q4YTD">
  <entity><identifier scheme="...">0000320193</identifier></entity>
  <period>
    <startDate>2023-10-01</startDate>
    <endDate>2024-09-28</endDate>
  </period>
</context>

That single fact tells you Apple reported $383.285B in revenue for the fiscal year ending September 28, 2024, with rounding to the nearest million. To make use of that programmatically, you need to dereference the context, parse the dates, normalize the unit, and decide whether this concept (us-gaap:Revenues) maps to your internal revenue field. Now do that for every concept in every filing for every company.

Five problems no one warns you about

Once you start parsing real filings at scale, the same five issues come up repeatedly. Each one looks like a quirk; collectively they are the reason “just download EDGAR” is not a viable backtesting strategy.

1. Concept proliferation

Revenue is not a single concept. The us-gaap taxonomy alone contains Revenues, SalesRevenueNet, SalesRevenueGoodsNet, SalesRevenueServicesNet, RevenueFromContractWithCustomerExcludingAssessedTax, and a dozen more. Different companies, and the same company across years, pick different concepts. Some report a single top-line; others report two or three component concepts that sum to revenue.

Issuer-specific extension taxonomies make it worse. Tesla has reported revenue through tsla:AutomotiveSalesRevenue and similar custom concepts. Without a mapping layer, a naive parser will silently miss data for entire companies.

2. Dimensional facts

Many facts carry dimensions: an axis paired with a member. us-gaap:Revenues with us-gaap:StatementBusinessSegmentsAxis = tsla:AutomotiveSegmentMember is segment-level revenue. The same concept without dimensions, in the same filing, is consolidated revenue. They are not duplicates — one rolls up into the other.

For a top-of-statement backtest you want only non-dimensional facts. For a segment-aware strategy you want exactly the dimensional ones. Get this wrong and you'll silently double-count or attribute consolidated metrics to a single segment.

3. Period classification (instant vs duration)

Balance sheet items are instants (a value at a point in time). Income statement and cash flow items are durations (a value over a period). The XBRL context tells you which, but the period itself can be ambiguous: a 10-Q for Q3 contains a Q3-only duration and a year-to-date (9-month) cumulative duration for the same concept. They look identical except for their start dates.

Classifying every fact by period type is non-negotiable. If you load all duration facts as “quarterly”, your Q3 revenue will be triple-counted because the year-to-date cumulative is also in there.

4. Fiscal calendars

About a third of S&P 500 companies do not use a December fiscal year. Apple ends in late September; Costco in late August; Walmart in late January. Many use a 52/53-week calendar that drifts by a few days each year, so the actual fiscal year-end might be September 28 one year and September 27 the next.

Naively grouping facts by calendar year breaks any cross-company comparison. You need fiscal-year-aware classification: for a company with an August fiscal year-end, a fact ending September 2025 belongs to fiscal year 2026, not 2025. And the 52/53-week drift means you need a small grace window around the nominal month boundary.

5. The missing Q4

This one trips up almost every first-time XBRL parser. Q4 is not directly reported. The 10-K contains the full-year FY duration, and the three 10-Qs contain Q1, Q2, and Q3. To get a discrete Q4 income statement, you have to subtract.

Q4 = FY (10-K) - 9M (Q3 10-Q)
   = FY - H1 - Q3                  (fallback if 9M is missing)
   = FY - Q1 - Q2 - Q3             (fallback if H1 is missing too)

This sounds simple until you handle restatements, late-filed 10-Q/As, or mid-year fiscal changes. Reconstruction has to be deterministic and idempotent — backtests need stable history, not values that flip every time a company files an amendment.

Parsing XBRL with Arelle

The de facto open-source XBRL processor is Arelle. It is mature, taxonomy-aware, used by regulators, and can run as a CLI or as a long-running web service. Arelle handles the things you do not want to reimplement: schema resolution, taxonomy caching, label resolution across labelLink files, and dimension validation.

# Spawn Arelle as a local web server
arelleCmdLine --webserver localhost:8080

# Fetch facts as JSON for a specific filing instance
URL="https://www.sec.gov/Archives/edgar/data/320193/.../aapl-20240928.htm"
ENC=$(node -e "console.log(encodeURIComponent(process.argv[1]))" "$URL")
curl "http://localhost:8080/rest/xbrl/$ENC/facts?media=json&factListCols=Name,contextRef,unitRef,Dec,Lang,Value,Period,Dimensions"

The response is a JSON array, one entry per fact, with resolved start/end dates instead of raw context references. From there it is “just” an iteration problem: filter, classify, normalize, store.

Filtering before you store anything

Not every fact is worth keeping. For financial filings the rules we apply are roughly:

• Drop language-tagged narrative facts (risk-factor text, accounting-policy paragraphs).
• Drop facts without a decimals attribute — usually unit-less metadata.
• For top-line statements, keep non-dimensional numeric facts only; route dimensional facts to a separate segments table.
• Skip prior-year comparative balance-sheet snapshots that appear inside a current filing — they are the same numbers already stored against the prior filing.

“As-reported” vs normalized: a real distinction

Financial data vendors generally pick one of two postures, and the choice has direct backtesting consequences.

Backtests need the second posture. You are simulating what an algorithm would have decided with the data available on a given historical date. Adjusted numbers from a future restatement are look-ahead bias by another name.

StockFit exposes both views. /api/financials/as-reported returns the full raw fact set with its original concept names, dimensions, and source filing reference. /api/financials/income-statement, /api/financials/balance-sheet, and /api/financials/cash-flow-statement return the curated, canonical view — same underlying values, but with stable field names so your code does not break when an issuer switches concepts between filings.

A stable contract for code: curated facts

The bridge between the messy XBRL world and clean JSON is a mapping layer: each company's set of reported concepts is mapped onto a stable list of canonical fields (revenue, costOfRevenue, operatingIncome, cashAndEquivalents, and so on). Two principles make this layer trustworthy:

1. Mapping is per-company, not global. Tesla's mapping for revenue may pull from tsla:AutomotiveSalesRevenue + tsla:ServicesAndOtherRevenue; Apple's pulls from a single us-gaap:RevenueFromContractWithCustomerExcludingAssessedTax. A one-size-fits-all mapping silently loses data.
2. Mappings are versioned and re-derivable. When a company changes the concepts it reports under (which they do, especially around accounting standard changes like ASC 606), the mapping updates and historical periods are re-mapped against the new definition. Backtests must be reproducible across runs.

We also derive a small set of fields from related facts when the company does not report them directly. grossProfit is computed from revenue - costOfRevenue; workingCapital from currentAssets - currentLiabilities; ebit from a chain of fallbacks (operatingIncome + otherExpenses, then netIncome + interestExpense + incomeTaxExpense). The derived fields are flagged in the response so a backtest can choose to use only directly-reported values if needed.

Reconstructing Q4 deterministically

Reconstruction is a read-time operation, not a write-time one. Storing a synthesized Q4 in the same table as native quarters means a future restatement of either the 10-K or the Q3 10-Q makes your Q4 stale until the next ingest. Computing Q4 at query time keeps it consistent with whatever the most recent FY and 9M values actually are.

The reconstruction is the chain shown earlier: prefer FY - 9M, fall back to FY - H1 - Q3, fall back to FY - Q1 - Q2 - Q3. Reconstruction happens after the curated mapping, so it works on canonical fields rather than on raw concept names — which means it stays correct across companies whose 9M facts come from different XBRL concepts.

For balance sheet items there is no reconstruction needed: instants from the 10-K already represent fiscal year-end snapshots, and instants from a 10-Q are quarter-end snapshots.

Point-in-time semantics

Every backtest needs an honest answer to: “what did I know on date D?” For SEC fundamentals, the answer is governed by the filing date, not the period-end date. A 10-K for fiscal year ending December 31, 2024 might not be filed until late February 2025. Using its values to make a January 2025 trade is look-ahead.

Two practical rules:

• Filter by filed timestamp. Every fact stored from a filing should carry both its period_end and the filing's acceptedDateTime from EDGAR. Backtests should query WHERE filed_at ≤ backtest_date.
• Account for restatements. If a company files a 10-K/A in March 2026 amending FY2024, the original 10-K's values were what an algorithm running in February 2025 would have seen. Either store both versions and select by filed_at, or accept the inaccuracy and document it.

End-to-end: a backtesting workflow

Here is the shape of a typical workflow using the StockFit API. The example builds a simple quality-and-value screen rebalanced quarterly, but the same pattern works for any fundamentals strategy.

1. Pull the universe of symbols

curl -H "Authorization: Bearer $STOCKFIT_KEY" \
  "https://api.stockfit.io/v1/api/symbols?exchange=NASDAQ&pageSize=1000"

2. For each symbol, fetch a curated income statement

curl -H "Authorization: Bearer $STOCKFIT_KEY" \
  "https://api.stockfit.io/v1/api/financials/income-statement?\
symbol=AAPL&period=quarter&limit=4"

The response is a flat array, one entry per fiscal period. period is the period_end date; facts is a map of canonical field names. Missing values come back as the literal string "N/A" so a key always exists. Quarter alignment is already fiscal-year-aware — below, Apple's December period is fiscal Q1 2026 because Apple's fiscal year ends in late September.

[
  {
    "period": "2025-12-26",
    "facts": {
      "revenue":         143756000000,
      "costOfRevenue":    74525000000,
      "grossProfit":      69231000000,
      "operatingIncome":  50852000000,
      "netIncome":        42097000000,
      "eps":              2.85,
      "ebitda":           54066000000,
      "interestExpense":  "N/A"
    }
  },
  {
    "period": "2025-09-26",
    "facts": {
      "revenue":         102466000000,
      "costOfRevenue":    54125000000,
      "grossProfit":      48341000000,
      "operatingIncome":  32427000000,
      "netIncome":        27466000000,
      "eps":              1.85,
      "ebitda":           35554000000,
      "interestExpense":  "N/A"
    }
  }
]

3. Combine with balance sheet and cash flow

# Same shape, swap the path
curl -H "Authorization: Bearer $STOCKFIT_KEY" \
  "https://api.stockfit.io/v1/api/financials/balance-sheet?symbol=AAPL&period=quarter&limit=40"

curl -H "Authorization: Bearer $STOCKFIT_KEY" \
  "https://api.stockfit.io/v1/api/financials/cash-flow-statement?symbol=AAPL&period=quarter&limit=40"

4. Join with /api/filings for point-in-time semantics

The financials endpoints return period_end, not the date the filing was accepted by the SEC. To filter by what an algorithm could have seen on a given backtest date, fetch the corresponding filings and join on periodEndDate → dateFiled.

curl -H "Authorization: Bearer $STOCKFIT_KEY" \
  "https://api.stockfit.io/v1/api/filings?symbol=AAPL&type=10-Q&pageSize=20"

{
  "page": 1, "pageSize": 3, "totalPages": 33, "totalResults": 99,
  "data": [
    {
      "type":            "10-Q",
      "accessionNumber": "0000320193-26-000006",
      "url":             "https://www.sec.gov/Archives/edgar/data/320193/.../aapl-20251227.htm",
      "dateFiled":       "2026-01-30",
      "periodEndDate":   "2025-12-27",
      "xbrl":            true,
      "amendment":       false
    }
  ]
}

Note that periodEndDate from /api/filings can be one or two days off from the financials period field for 52/53-week filers like Apple (the financials endpoint normalizes to a fiscal-quarter-end; the filing itself uses the literal end-of-period date from the cover page). Either round to the nearest week or use a tolerance when joining.

5. Build a point-in-time panel

const KEY = process.env.STOCKFIT_KEY;
const BASE = 'https://api.stockfit.io/v1/api';
const headers = { Authorization: `Bearer ${KEY}` };

const get = async (path, params) => {
  const url = new URL(BASE + path);
  for (const [k, v] of Object.entries(params)) url.searchParams.set(k, v);
  const r = await fetch(url, { headers });
  if (!r.ok) throw new Error(`${r.status} ${url}`);
  return r.json();
};

// Load curated quarterly income statement; coerce "N/A" → null
async function loadFinancials(symbol) {
  const rows = await get('/financials/income-statement', {
    symbol, period: 'quarter', limit: 40,
  });
  return rows.map(({ period, facts }) => ({
    symbol, period,
    ...Object.fromEntries(
      Object.entries(facts).map(([k, v]) => [k, v === 'N/A' ? null : v])
    ),
  }));
}

// Load 10-Q + 10-K filings — that's where dateFiled lives
async function loadFilings(symbol) {
  const [q, k] = await Promise.all([
    get('/filings', { symbol, type: '10-Q', pageSize: 100 }),
    get('/filings', { symbol, type: '10-K', pageSize: 50 }),
  ]);
  return [...q.data, ...k.data].map(f => ({
    periodEnd: new Date(f.periodEndDate),
    dateFiled: new Date(f.dateFiled),
    accessionNumber: f.accessionNumber,
  }));
}

// Tolerant join: match each financial period to nearest filing period_end within 7 days
function joinPanel(fins, fils) {
  const TOLERANCE_MS = 7 * 24 * 3600 * 1000;
  return fins.map(row => {
    const periodMs = new Date(row.period).getTime();
    let best = null, bestDiff = Infinity;
    for (const f of fils) {
      const diff = Math.abs(f.periodEnd.getTime() - periodMs);
      if (diff < bestDiff && diff <= TOLERANCE_MS) { best = f; bestDiff = diff; }
    }
    return { ...row, dateFiled: best?.dateFiled ?? null,
             accessionNumber: best?.accessionNumber ?? null };
  });
}

const universe = ['AAPL', 'MSFT', 'GOOG', 'NVDA'];
const panel = (await Promise.all(universe.map(async s => {
  const [fins, fils] = await Promise.all([loadFinancials(s), loadFilings(s)]);
  return joinPanel(fins, fils);
}))).flat();

// As-of query: what did we know on 2024-06-30?
const asof = new Date('2024-06-30');
const visible = panel.filter(r => r.dateFiled && r.dateFiled <= asof);

// Trailing-twelve-month revenue per symbol (latest 4 visible quarters)
const bySymbol = new Map();
for (const r of visible.sort((a, b) => a.dateFiled - b.dateFiled)) {
  if (!bySymbol.has(r.symbol)) bySymbol.set(r.symbol, []);
  bySymbol.get(r.symbol).push(r);
}
const ttm = [...bySymbol].map(([symbol, rows]) => ({
  symbol,
  ttmRevenue: rows.slice(-4).reduce((s, r) => s + (r.revenue ?? 0), 0),
}));
console.table(ttm);

The point-in-time filter on dateFiled is what makes the result honest. The 7-day tolerance in the join handles the 52/53-week period_end discrepancy. Run with Node 18+ (native fetch) and top-level await in an ESM module.

6. Walk forward across rebalance dates

// Generate quarterly rebalance dates (2018-Q1 through 2025-Q1)
function quarterlyDates(start, end) {
  const out = [];
  for (let d = new Date(start); d <= end; d.setMonth(d.getMonth() + 3)) {
    out.push(new Date(d));
  }
  return out;
}

const rebalanceDates = quarterlyDates(new Date('2018-01-01'), new Date('2025-01-01'));
const positions = [];

for (const d of rebalanceDates) {
  const visible = panel.filter(r => r.dateFiled && r.dateFiled <= d);
  // Latest visible quarter per symbol
  const latest = new Map();
  for (const r of visible.sort((a, b) => a.dateFiled - b.dateFiled)) {
    latest.set(r.symbol, r);
  }
  // Toy ranking: highest gross margin
  const ranked = [...latest.values()]
    .filter(r => r.revenue && r.grossProfit)
    .map(r => ({ ...r, margin: r.grossProfit / r.revenue }))
    .sort((a, b) => b.margin - a.margin)
    .slice(0, 20)
    .map(r => r.symbol);
  positions.push({ date: d.toISOString().slice(0, 10), longs: ranked });
}

// Hand off to your favourite backtester
console.log(positions[0]);

Wrapping up

XBRL is structured data with a long tail of edge cases. The work between “XML on EDGAR” and “clean JSON your strategy can iterate over” is real and ongoing — new accounting standards, new issuer extensions, new corporate actions. If you'd rather not maintain that pipeline yourself, the StockFit API exposes both the raw fact set and a curated, point-in-time, fiscal-year-aware JSON view of every US public company.

• /api/financials/as-reported — full raw fact set, traceable to the source filing.
• /api/financials/income-statement, /balance-sheet, /cash-flow-statement — curated, canonical fields keyed by fiscal-period-end. Join with /api/filings on periodEndDate for point-in-time filtering.
• /api/financials/key-metrics, /growth, /scores — derived ratios and quality signals computed off the same underlying facts.
• /api/filings, /api/filings/document, /api/filings/item — filing metadata and section extraction when a strategy needs to read the underlying 10-K or 10-Q text.

Full reference is at api.stockfit.io/docs. The free tier is enough to prototype most backtesting workflows end-to-end. If you build something interesting, send a note to info@stockfit.io — we read everything.

Related reading on this site: Insights — AI-generated company deep dives built using exactly the workflow described above.