Send us the source
Brief 01 What was hard 02 How it works 03 Numbers 04 Outcome 05 All cases → Send us the source

Atoms / Data extraction / Cases / Expedia

Travel — running since 2024

Expedia — package pricing, decomposed.

A package price is deliberately not the sum of its parts, and that difference is the entire product for anybody selling travel pricing intelligence. Collecting it means asking for the same trip several ways and reconciling the answers.

The source is named because it is public; the client is not. Every figure on this page is a measured production number the client agreed to publish. Have a source of your own? Send it over, whatever its size — you get an answer within 24 hours.

Case fileexpediaIn production
SourceExpedia
ClientTravel pricing intelligence
Running since2024
DeliverySnowflake, daily
Markets24
Product types3
Reach → Read → Reconcile → DeliverRecounted daily
30B

itineraries priced since 2024

17B

a year

98.1%

measured coverage

46M

itineraries a day

SourceExpedia
VerticalTravel
Running since2024 — without a rebuild
The brief01 / 05

What the client actually needed.

Not more rows. Every one of these projects started with somebody who already had data and could not use it for the decision in front of them.

The problem

Where it started

The client sells competitive pricing data to hotels and airlines, who want to know what a customer actually pays and how much of that reaches the supplier. The package discount is where that question lives, and it cannot be read off a page — it only exists as the gap between how the same trip is priced in different combinations.

Fixed on day one

What we committed to

Hotel, flight and package priced for the same itinerary, same sessionEvery sample
Component decomposition retained rather than a single totalRequired
Markets and points of sale kept distinct, not merged24 markets
Cost per priced itinerary fixed before the buildCapped

Every one of these is measured continuously and reported on the same dashboard the client watches. A commitment nobody measures is a sentence in a proposal.

What was hard02 / 05

Four things that beat the previous attempt.

None of these is solved by better headers or a bigger proxy pool. Each needs a different piece of engineering, and working out which one you are actually facing is most of the job.

01 — CombinatoricsCombinatorics

The same trip, several ways

Answering the client's question means pricing one itinerary as a hotel, as a flight, and as a package — three times the work for one data point, and the interesting number is the difference.

What we doItineraries are sampled against a demand model rather than enumerated, and the three pricings are executed within one session so the comparison is genuinely like for like.

02 — SessionSession

Comparability across the triple

If the three pricings come from different sessions or moments, the difference between them is noise rather than the package discount.

What we doThe triple is collected as an atomic unit with a shared session and a bounded time window. A triple that cannot be completed is discarded rather than delivered partially.

03 — Point of salePoint of sale

Market is not country

Prices vary by point of sale, currency and residency in ways that do not follow national borders, and merging them produces averages nobody transacts at.

What we doPoint of sale is an explicit dimension. Nothing is merged across it, and the client's model decides what to compare rather than inheriting a decision made in the pipeline.

04 — CostCost

Three prices for one row

At three pricings per data point the unit cost is what decides how many markets and how much of the calendar the client can afford.

What we doCost per priced itinerary was fixed before the build. Sampling density, session reuse and caching of stable components were all designed against that number.

How it works03 / 05

Five decisions the pipeline is built on.

The architecture is not interesting; every extraction system has a queue, a fetcher and a parser. These are the decisions that made this one work where the last one did not.

01

Sample itineraries, do not enumerate them

Which routes, dates and stay lengths get priced comes from a demand model built with the client. Enumeration is affordable for nobody and mostly prices trips nobody takes.

02

Collect the triple atomically

Hotel, flight and package for one itinerary are priced inside one session within a bounded window. An incomplete triple is dropped, because a partial comparison is worse than none.

03

Keep point of sale explicit

Nothing is averaged across points of sale inside the pipeline. Comparability decisions belong to the client's model, not to a collector's convenience.

04

Cache what does not move

Property attributes, route structures and policies change slowly; prices do not. Splitting them and refreshing on separate clocks is most of the cost reduction.

05

Report cost per itinerary

Unit cost sits on the same dashboard as coverage. At three pricings per row it is the constraint that decides how far the product can reach.

The numbers04 / 05

What it does on an ordinary day.

Production figures, not a benchmark run. Coverage is recounted daily against an independent sample of the live source rather than asserted, which is why the numbers are not round.

Daily profile

Where the volume goes

Itineraries priced a day46M
Requests a day12M
Complete triples a day15M
Cached components reused61%
Triples discarded as incomplete2.6%

Every usable data point costs three pricings, so 15 million complete triples a day means 46 million itineraries priced and 12 million requests — 139 a second. That is 1.4 billion itineraries a month, 17 billion a year and 30 billion since 2024, yielding 5.5 billion complete triples a year. The multiplier is why the sampling model, not the collector, is the expensive part of this system.

Headline

The four that are contractual

Priced since 2024
30B
Coverage
98.1 %
Triples a year
5.5B
Cost / itinerary
−48 %

These four sit in the support agreement. When one of them drifts outside its band, we are alerted within fifteen minutes and fixing it is routine work under the monthly arrangement, not a change request.

What changed05 / 05

Before, and after.

The columns are the client's own numbers from before the rebuild and the measured ones from production today. The left column is the part most vendors would rather not put on a page.

MetricBeforeToday

Package discount

Estimated from separate feeds

Measured on matched triples

Point of sale

Merged into country

Explicit dimension

Markets covered

4

24

Cost per priced itinerary

Baseline

52% lower

The client had been inferring the package discount by subtracting two unrelated feeds. The error in that estimate turned out to be larger than the discount itself.

Start here

Have a source of your own? Two lines are enough.

Send the source, the fields you need and roughly how often. You get a straight answer within 24 hours: whether it can be done, what makes it hard, what coverage is achievable and roughly what it costs to build and to run. Whatever the size.

Direct

NDA before technical detail, as always. If it isn't our kind of work, we say so in the first reply.