Skip to main content
You have a table with holes in it: companies without an industry, products without a current price, contacts without a location. The data exists on the open web. Collecting it at any volume from one IP does not work.

​
Why this needs proxies at all

Enrichment is many small requests to the same few domains - exactly the shape rate limiters are built to catch. From a single IP you get maybe a few hundred rows before the responses turn into 429s and blocks. Spreading the requests across residential IPs turns the same job into ordinary traffic.

​
The setup

One proxy per source, so you can see per-source cost and stop one without touching the others:
I am enriching 40,000 company records from three sources. Create a proxy for each, name them after the source, cap each at 25 GB a month, and give me the connection strings.
Three separate resources rather than one shared proxy is the point: when the registry source starts blocking, you rotate that one and the other two keep running.
Ask the agent to write the run into the description - “Q3 enrichment run” - and you will still know what these were for when you find them next quarter.

​
Rotating versus sticky

Enrichment is the case where the default behaviour is what you want. Without a session parameter, each request gets a fresh IP - which is exactly right for independent lookups. Reach for a sticky session only when a single record needs several requests in sequence: a search, then a detail page, then a sub-page. Then hold the IP for that record and drop it:
Use the record ID as the session ID and a short TTL. See Session Control.

​
Watching the cost while it runs

Enrichment jobs have a habit of costing more than the estimate. Ask mid-run:
How much traffic and money have the three enrichment proxies used today?
The job boards proxy is using three times the others. That is worth knowing on day one rather than at the invoice - usually it means a source is returning full pages where you only need a fragment.

​
Country matters more than you think

Many sources return different data by visitor location: local pricing, regional availability, translated fields. If your records are European, enriching them through US exit IPs quietly gives you the wrong values. Set the country in the username per batch:
One proxy covers every country you need - the country is a username parameter, not a property of the resource.

​
When the run is done

The Q3 enrichment run is finished. Deactivate the three enrich_ proxies.
Deactivation stops the traffic and keeps the resource, so the next run reuses the same names and the same history. The agent will not delete them unless you explicitly ask.

​
Next steps

Price Monitoring

The same shape, run on a schedule instead of once.

Provisioning & Cost Control

Keeping a long job inside a budget.