Send us the source
Brief 01 What was hard 02 How it works 03 Numbers 04 Outcome 05 All cases → Send us the source

Atoms / Data extraction / Cases / Airbnb

Travel — running since 2023

Airbnb — supply, calendars and the price nobody quotes.

Short-let revenue management lives or dies on two things: how much comparable supply exists in a neighbourhood, and what that supply actually costs a guest once cleaning and fees are added. Neither is a field you can read; both have to be assembled.

The source is named because it is public; the client is not. Every figure on this page is a measured production number the client agreed to publish. Have a source of your own? Send it over, whatever its size — you get an answer within 24 hours.

Case fileairbnbIn production
SourceAirbnb
ClientShort-let revenue management
Running since2023
DeliveryBigQuery, daily
Listings7.4M
Markets240 cities
Reach → Read → Reconcile → DeliverRecounted daily
230B

calendar-nights since 2023

77B

a year

97.2%

measured coverage

7.4M

listings tracked

SourceAirbnb
VerticalTravel
Running since2023 — without a rebuild
The brief01 / 05

What the client actually needed.

Not more rows. Every one of these projects started with somebody who already had data and could not use it for the decision in front of them.

The problem

Where it started

The client prices short-let inventory for property managers. Their competitive picture came from a sample of listings collected once a week, which meant a manager pricing a weekend in a busy neighbourhood was doing it against a picture assembled before the neighbourhood filled up. What they needed was the whole comparable set, with calendars, refreshed daily.

Fixed on day one

What we committed to

Comparable supply in each market, not a sample97% floor
Availability calendars for the booking window that matters365 days
Landed price including cleaning and fees, at realistic party sizesEvery listing
Cost per market held flat as markets are addedDesign input

Every one of these is measured continuously and reported on the same dashboard the client watches. A commitment nobody measures is a sentence in a proposal.

What was hard02 / 05

Four things that beat the previous attempt.

None of these is solved by better headers or a bigger proxy pool. Each needs a different piece of engineering, and working out which one you are actually facing is most of the job.

01 — GeographyGeography

Supply is a neighbourhood question

Comparability is local — three streets over is a different market — so a city-level sample tells a manager almost nothing about their own block.

What we doMarkets are collected by geographic tiling sized to observed price variance, dense where prices change over short distances and sparse where they do not.

02 — CalendarsCalendars

Availability is 365 fields

A listing's value is in its calendar, and the calendar is not one number. Collecting a year of availability for millions of listings is the expensive half of the problem.

What we doCalendar depth is tiered by how far ahead the client's managers actually price: the near window is refreshed daily, the far window weekly, and the split is measured against how often far dates change.

03 — FeesFees

Cleaning is most of a short stay

On a two-night stay a cleaning fee can be a third of the total. A nightly rate without it is not comparable to anything.

What we doLanded price is computed at the party sizes and stay lengths the client models, with every fee component stored separately so managers can see where the money is.

04 — ChurnChurn

Listings appear and vanish

Short-let supply turns over fast, and a static listing set decays into fiction within a quarter.

What we doSupply discovery runs continuously alongside refresh, so new listings enter the picture within a day and delistings are recorded as events rather than as silence.

How it works03 / 05

Five decisions the pipeline is built on.

The architecture is not interesting; every extraction system has a queue, a fetcher and a parser. These are the decisions that made this one work where the last one did not.

01

Tile by observed variance

Tile density comes from measured price variance across distance, not from a fixed grid. Dense neighbourhoods get dense tiling; a quiet suburb does not, which is where the cost saving lives.

02

Tier the calendar window

The near booking window is refreshed daily and the far window weekly, with the boundary set by how often far dates actually move. Collecting 365 days uniformly is the naive plan that makes this unaffordable.

03

Compute the landed price

Fees are captured as components and the landed total is computed at the client's modelled party sizes. A nightly rate on its own is the number most competitors publish and nobody can use.

04

Discover continuously

New supply is found by ongoing traversal rather than by a periodic rebuild of the listing set, so a neighbourhood filling up is visible within a day.

05

Hold cost flat per market

Adding a market must not add cost per market. Tiling, tiering and discovery are all tuned against a per-market budget that is reported weekly.

The numbers04 / 05

What it does on an ordinary day.

Production figures, not a benchmark run. Coverage is recounted daily against an independent sample of the live source rather than asserted, which is why the numbers are not round.

Daily profile

Where the volume goes

Calendar-nights a day210M
Requests a day3.4M
Listings tracked7.4M
Listings refreshed a day2.4M
New listings discovered a day18k

One calendar request returns about sixty nights, so 3.4 million requests — 39 a second — produce 210 million night-values a day: 6.3 billion a month, 77 billion a year, 230 billion since 2023. The near window is refreshed daily and the far window weekly; collecting all 365 nights for all 7.4 million listings every day would need nine times the traffic and change nothing a revenue manager acts on.

Headline

The four that are contractual

Nights since 2023
230B
Coverage
97.2 %
Listings
7.4M
Calendar depth
365 days

These four sit in the support agreement. When one of them drifts outside its band, we are alerted within fifteen minutes and fixing it is routine work under the monthly arrangement, not a change request.

What changed05 / 05

Before, and after.

The columns are the client's own numbers from before the rebuild and the measured ones from production today. The left column is the part most vendors would rather not put on a page.

MetricBeforeToday

Comparable set

A weekly sample

The whole neighbourhood, daily

Price basis

Nightly rate

Landed total at modelled party sizes

Calendar visibility

None

365 days, tiered refresh

Cost of a new market

Linear

Flat, budget-driven

Managers stopped arguing with the tool the month the landed price appeared. A nightly rate that nobody pays is the fastest way to lose a user's trust.

Start here

Have a source of your own? Two lines are enough.

Send the source, the fields you need and roughly how often. You get a straight answer within 24 hours: whether it can be done, what makes it hard, what coverage is achievable and roughly what it costs to build and to run. Whatever the size.

Direct

NDA before technical detail, as always. If it isn't our kind of work, we say so in the first reply.