I Fed It Structured Data Instead of Raw Text
Only 15% of periodic-report text was being read; it ate 55% of LLM input and the result was cover-page restatements. Swapping raw text for six confirmed figures from the disclosure API cut input to 1/40 and put numbers in the summaries
The disclosure-summary service takes corporate disclosures, summarizes them with an LLM, and ships them out as a weekly feed. While counting tokens for a move of the LLM calls from my own GPU to the subscription CLI, per-call overhead came down a lot but the body text itself didn’t budge. Instead of hunting for more to cut, I went and looked at what the model was actually being fed.
What I set out to do
The evening before, I started with a calculation. If I moved the disclosure-summary service’s LLM calls from my own GPU to the subscription CLI, how many tokens would it eat? I was half thinking of selling the hardware secondhand. Per-call overhead I had cut from 33,090 tokens to 248, but the body itself hadn’t shrunk.
Seems like the tokens can’t be cut as much as I expected, and it’s not easy to run on a subscription — is that right? But I do wonder whether I’m actually using it for what it’s worth.
So the question changed. Not what to cut, but what am I analyzing right now.
How I asked
I had it measure the last 30 days of production calls by category.
| Category | Monthly calls | Share of input | Average length |
|---|---|---|---|
| Periodic reports | 2,413 | 55.3% | 23,835 chars |
| Other (noise) | 6,573 | 34.4% | 5,442 chars |
| Meaningful events | 3,007 | 10.3% | 3,561 chars |
90% of LLM cost was going to disclosures the product classifies as noise. Annual, half-year and quarterly reports ate more than half the input, and what they contributed to the weekly feed was 32 items.
Where it broke
Periodic-report raw text has a median of 158,089 characters. The maximum was 1.24 million. Model input was being clipped to the first 16,000 characters and the last 8,000, so only 15% of the text got read. 99.3% of periodic reports were pushing up against that limit.
The problem is where the cut falls. The front is the table of contents, the company overview, and the list of subsidiaries. Revenue and operating profit sit in the middle of the document, so they drop out entirely. The result was a cover-page restatement. A financial holding company’s half-year report summary was “disclosure of the status of 412 consolidated subsidiaries,” and a public utility’s was “management status and business report for the first half of 2026.” Not wrong, but zero information.
I asked, “Is dropping them from summaries entirely an option? Wouldn’t people like periodic-report summaries?” The answer was that the intuition was right. The problem was that we weren’t producing that summary right now.
How I fixed it
Keep using the LLM, but change what it’s fed. The disclosure API (OpenDART) has confirmed financial figures, free with the same key. Six accounts — revenue, operating profit, net income, total assets, total liabilities, total equity — plus the rate of change against the prior period, computed deterministically in code into 400-600 characters, and that goes into the model. The swap point is one line in the submission logic; async submission, polling and fact-checking stay as they are.
It was done in one PR, but there were traps. 24 pre-review agents verified against the real API and fixed 17 issues; 5 rounds of code review fixed 5 more. Income-statement items have to use the cumulative amount. The current-period figure in a half-year report is three months’ worth, so results look halved. The meaning of “prior period” differs per table. For the income statement it’s the same period last year; for the balance sheet it’s the last fiscal year-end. Truncating amounts turns 179 million won into “100 million won,” shaving off 44%. And since the lookup key is only company, year and report type, the API always returns the latest corrected version. If the receipt number in the response differs from the target, it falls back to the raw-text path.
The result: per-item input went from 24,000 characters to about 400, and total LLM input fell 54%. That afternoon I re-analyzed 6,830 of the existing 8,429 periodic reports in 1 hour 42 minutes, and the financial-path success rate was 99.6%. One holding company’s summary changed from “operations centered on the investment division and IT services” to “revenue ₩122.7033T, operating profit ₩1.8185T, −0.6% and −24.1% respectively.”
The night before there had been one more incident of a similar kind. I was running a backfill of 13,437 items to find the original link from correction filings, and at the 86% mark all 598 failures were attachment corrections. I had put in a self-check defense: a page is valid only if the viewer’s correction-chain list contains the page itself. But an attachment correction is an attachment submission, not a body, so it wasn’t in that list. When failures cluster in a single type, it’s the code, not the data. I widened the check to the attachment list, recovered 634, and the final count was 13,436/13,437.
Lessons
- Don’t throw raw text at an LLM. If there’s a structured source, pull it with code and feed only the numbers. Input becomes 1/40 and accuracy goes up.
- Before cutting cost, classify where the cost goes. Half was going to documents classified as noise. Changing the feed mattered more than tightening the clamp.
- Truncated input produces a plausible answer, not a wrong one. “Status of 412 subsidiaries” doesn’t get caught as an error. You only know by reading samples with your own eyes.
- “Prior period” and “current period” in the disclosure API mean different things per table. Spell out cumulative vs. current quarter, same period last year vs. last fiscal year-end, line by line.
- When failures cluster in a single type, it’s a class bug. When all 598 were attachment corrections, opening a single actual page was all it took.