For a platform selling to agents, the interesting entity is not the property — it is the agent. Which listings are theirs, how fast they move, which brokerage they sit under this month, and which of them just lost a listing to a competitor down the road.
The source is named because it is public; the client is not. Every figure on this page is a measured production number the client agreed to publish. Have a source of your own? Send it over, whatever its size — you get an answer within 24 hours.
records to date
a year
measured coverage
agent records maintained
Not more rows. Every one of these projects started with somebody who already had data and could not use it for the decision in front of them.
The problem
The client sells prospecting tools to real-estate agents. Their value proposition depends on knowing, faster than the agent does, that a listing changed hands or an agent moved brokerage. They had listing data and no reliable agent graph, so the product could describe the market but never the people in it.
Fixed on day one
Every one of these is measured continuously and reported on the same dashboard the client watches. A commitment nobody measures is a sentence in a proposal.
None of these is solved by better headers or a bigger proxy pool. Each needs a different piece of engineering, and working out which one you are actually facing is most of the job.
An agent changes brokerage and, from a naive collector's point of view, disappears and is replaced by a stranger with the same name. The event the client sells on is exactly the one that breaks identity.
What we doA persistent agent identity maintained across licence identifiers, contact details and listing overlap, so a brokerage move is recorded as a transition on one record rather than a death and a birth.
Listing reassignment between agents happens without ceremony, and a daily snapshot cannot say when. For a prospecting product, the timing is the value.
What we doAttribution is checked hourly for the segment the client cares about, and every change is emitted as a timestamped event rather than inferred from a diff.
Offices, teams and franchises nest, and flattening them destroys the territory analysis the product is built on.
What we doThe brokerage hierarchy is maintained as a graph with its own history, so a team splitting or an office being absorbed is a recorded event.
Agent records are personal data even when professionally published, and the collection scope has to be defensible rather than merely technically possible.
What we doScope is limited to professionally published business contact information, documented per field, with retention and deletion controls designed in and the client's lawful basis recorded before the build.
The architecture is not interesting; every extraction system has a queue, a fetcher and a parser. These are the decisions that made this one work where the last one did not.
Agents are persistent entities with a history. The alternative — matching by name each night — fails precisely on the transitions the client sells.
Reassignments, brokerage moves and status changes are emitted with timestamps as they are observed. A diff between two nightly files cannot tell you when something happened, and when is the product.
Offices, teams and franchises nest and change. Flattening them is easy and destroys the territory logic the client's customers actually buy.
Fields are enumerated and justified, retention is set per field, and deletion is implemented rather than promised. This is documented before anything is collected.
The client's active territories are refreshed hourly and the rest daily. Uniform hourly refresh of a national agent graph is affordable for nobody.
Production figures, not a benchmark run. Coverage is recounted daily against an independent sample of the live source rather than asserted, which is why the numbers are not round.
Daily profile
Sixteen million records a day is 480 million a month, 5.8 billion a year and 17 billion since 2023, from 4.6 million requests at 53 a second. Ninety identity corrections a week is a healthy number: zero would mean the matching is too loose and quietly merging different people, which is the failure nobody catches until a customer complains.
Headline
These four sit in the support agreement. When one of them drifts outside its band, we are alerted within fifteen minutes and fixing it is routine work under the monthly arrangement, not a change request.
The columns are the client's own numbers from before the rebuild and the measured ones from production today. The left column is the part most vendors would rather not put on a page.
| Metric | Before | Today |
|---|---|---|
Agent graph |
None — names in a column | Persistent identities with history |
Time to see a listing change hands |
Next day, at best | Within the hour |
Brokerage structure |
Flat list | Maintained hierarchy |
Privacy position |
Undocumented | Scoped and documented per field |
The prospecting product's core claim — you hear about it before your competitor does — only became true when identity survived the move.
One case file per source, each opening the same way: what the client needed, what the source put in the way, how the pipeline is built and what the numbers are today.
7.4M listings, 210M calendar-nights a day
Read the case →
46M itineraries priced a day, decomposed
Read the case →
94k routes, 210M fare quotes a day
Read the case →
340M listings watched, sold prices captured
Read the case →
84M listings, eight markets, eight currencies
Read the case →
Start here
Send the source, the fields you need and roughly how often. You get a straight answer within 24 hours: whether it can be done, what makes it hard, what coverage is achievable and roughly what it costs to build and to run. Whatever the size.
Direct
NDA before technical detail, as always. If it isn't our kind of work, we say so in the first reply.