proxies for travel and airfare scraping
a fare is not one number. it changes with the country you appear to be in, the currency, and sometimes the device. collecting it accurately means querying from inside each market.
why this needs proxies
airlines and travel sites localise prices by geography and currency, and defend their fare data by blocking datacenter ranges and fingerprinting automated clients. the price you scrape from one office ip is the price for that one place, and often not a price a real customer was ever shown. residential addresses in the origin market return the fare that market actually sees.
which proxies to use
what it costs
estimate: about 1.2 mb per query, covering a route search and its results as html or json with assets. calendar and multi-city searches that return large result sets run higher.
| fare queries | bandwidth | residential | residential lite | mobile |
|---|---|---|---|---|
| 10,000 | 12 gb | $23.40 | $3.60 | $16.80 |
| 100,000 | 117 gb | $187.20 | $29.25 | $140.40 |
| 1,000,000 | 1,172 gb | $1523.60 | $234.40 | $1172.00 |
volume rates are applied automatically, so the larger rows already reflect the cheaper tier.
how to set it up
faq
- why do airfares change by location?
- airlines and travel agencies price by point of sale, currency and demand signals tied to where the request comes from. two people searching the same route from different countries routinely see different fares.
- residential or datacenter for airline sites?
- residential. airline and travel sites block datacenter ranges hard and fingerprint aggressively, so datacenter addresses rarely return a usable fare. residential in the origin market is what works.
- how often can i poll a route?
- the limit is the target, not the proxy. spread queries across enough addresses that no single one polls faster than a person plausibly would, and rotate.