Airfares change several times a day, exist for every combination of date, carrier and fare class, and are worthless the moment they are stale. Nobody can collect all of that. The engineering is in deciding, continuously, which slice of a ninety-four-thousand-route network is worth money today — and the whole network is swept every day regardless, so nothing new goes unseen.
The source is named because it is public; the client is not. Every figure on this page is a measured production number the client agreed to publish. Have a source of your own? Send it over, whatever its size — you get an answer within 24 hours.
fare quotes since 2024
a year
route coverage
quotes a day
Not more rows. Every one of these projects started with somebody who already had data and could not use it for the decision in front of them.
The problem
The client runs revenue management for a mid-size carrier and needs to see competitor fares on their own network several times a day. Their previous supplier delivered a daily file covering the top routes, which meant the routes under competitive attack — usually not the top ones — were the ones they could not see.
Fixed on day one
Every one of these is measured continuously and reported on the same dashboard the client watches. A commitment nobody measures is a sentence in a proposal.
None of these is solved by better headers or a bigger proxy pool. Each needs a different piece of engineering, and working out which one you are actually facing is most of the job.
The combination space is effectively unbounded and almost all of it is worthless. Collecting uniformly produces a large, expensive dataset that misses the movements that matter.
What we doA demand and volatility model drives sampling: routes under competitive pressure and dates inside the booking curve get intraday attention, quiet routes get a daily touch.
A morning fare can be wrong by lunchtime on a contested route, and a daily file makes the client's analysts systematically late to every fare war.
What we doIntraday refresh tiers with promotion and demotion driven by observed movement, so a route that starts moving is escalated within a cycle rather than at the next planning review.
Fare class, baggage inclusion, change restrictions and residency rules all determine whether two prices are actually comparable, and a headline number hides all of it.
What we doRestrictions and inclusions are captured per quote, so the client's model compares like with like rather than the cheapest visible number.
At a billion fare quotes a year, a careless request path turns a viable product into an infrastructure bill.
What we doCost per million quotes was fixed before the build and reported weekly, and the sampling model is tuned against it continuously rather than at review time.
The architecture is not interesting; every extraction system has a queue, a fetcher and a parser. These are the decisions that made this one work where the last one did not.
Sampling density follows where fares actually move: close-in dates on contested routes get frequent attention, far dates on stable routes get very little. This is the whole difference between an affordable network view and an unaffordable one.
A route that starts moving is escalated to intraday within a cycle and demoted again when it settles, without anybody filing a request.
Fare class, baggage and change rules travel with the quote. Comparing headline prices across carriers with different inclusions is the most common way airfare data misleads.
Quotes land on a stream as they are reconciled. A four-times-daily refresh delivered as a nightly file is a daily file with extra steps.
The sampling model is adjusted continuously against cost per million quotes, which sits on the same dashboard as coverage and freshness.
Production figures, not a benchmark run. Coverage is recounted daily against an independent sample of the live source rather than asserted, which is why the numbers are not round.
Daily profile
One search returns a fare ladder, not a price — about twenty-two quotes — so 9.6 million requests a day, 111 a second, produce 210 million quotes: 6.3 billion a month, 77 billion a year, 150 billion since 2024. Fewer than fifteen per cent of routes sit on the intraday tier at any moment and the set changes daily. Refreshing all ninety-four thousand four times a day would cost roughly five times as much and tell the client nothing extra.
Headline
These four sit in the support agreement. When one of them drifts outside its band, we are alerted within fifteen minutes and fixing it is routine work under the monthly arrangement, not a change request.
The columns are the client's own numbers from before the rebuild and the measured ones from production today. The left column is the part most vendors would rather not put on a page.
| Metric | Before | Today |
|---|---|---|
Network visibility |
Top 400 routes | All 94 000 |
Refresh on contested routes |
Daily | Four times a day |
Comparability |
Headline price | Fare class and restrictions retained |
Time to see a fare war start |
A day or more | Within a refresh cycle |
The routes the client could not previously see were, predictably, the ones a competitor had chosen to attack. That is not a coincidence — it is how route selection works.
One case file per source, each opening the same way: what the client needed, what the source put in the way, how the pipeline is built and what the numbers are today.
340M listings watched, sold prices captured
Read the case →
84M listings, eight markets, eight currencies
Read the case →
64M offers, competition at the offer level
Read the case →
1 950 stores, promotions caught the day they turn
Read the case →
2 800 stores, loyalty and shelf priced separately
Read the case →
Start here
Send the source, the fields you need and roughly how often. You get a straight answer within 24 hours: whether it can be done, what makes it hard, what coverage is achievable and roughly what it costs to build and to run. Whatever the size.
Direct
NDA before technical detail, as always. If it isn't our kind of work, we say so in the first reply.