2extract Developer Documentation

An agent with a browsing tool works fine until it meets a site that refuses cloud IP ranges. Then it gets a block page, reports back that the page was empty, and you spend an afternoon working out why. The fix is to route the agent’s fetches through a residential IP. With the MCP server, the agent can set that up itself.

The problem

Most agent runtimes fetch from a datacenter: your CI runner, a cloud function, a hosted sandbox. A large share of commercial sites treat those ranges as automated traffic and respond with a block, a CAPTCHA, or a stripped-down page. What makes this expensive is that the agent usually cannot tell. It receives HTTP 200 and some HTML, and reports what it read.

The setup

Ask once, at the start of the session:

Before you fetch anything, create a residential proxy for browsing, cap it at 5 GB a month, and route your requests through it.

The agent creates the resource, applies the cap, and hands back a connection string it can use for the rest of the run.

listProxyPlans        ⌐ Pay-As-You-Go · residential
getAccountBalance     ⌐ $149.95 available
createProxyResource   ⌐ agent_browsing
                        "General web access for research sessions"
setProxyLimits        ⌐ 5 GB per month

http://2xt-customer-K3nP7qLm2Xv-proxy-agent_browsing:s7Kd92mQ@proxy.2extract.net:5555

One proxy is usually enough. Add a country to the username when a site is region-locked - see Geo Targeting.

Sessions matter more than you expect

A site that tolerates one request will often block the same client when the IP changes mid-flow. Anything with a login, a cart, or pagination needs the IP to stay put. Add a session parameter to the username and the exit IP holds:

...-proxy-agent_browsing-session-run42-time-10:...

session is any identifier you choose; time is how many minutes to hold it. Full parameter reference under Session Control.

Give each agent run its own session ID. Two runs sharing one ID share one IP, and the rate limits that come with it.

Keeping it contained

An agent with web access and a proxy can reach anything, and it will keep going until something stops it. Cap the spend. Ask for a monthly ceiling when the proxy is created:

Create a browsing proxy and cap it at 5 GB a month.

A runaway loop then stops at the cap instead of at your balance. This is the limit that actually holds - it is enforced on our side, not asked of the model. Give the run its own proxy. One proxy per agent run, rather than one shared across everything, means you can see exactly what that run consumed and deactivate it on its own when the job is done.

Wiring it into a fetch

Once the agent has the string, it goes wherever your runtime takes proxy settings:

import requests

PROXY = "http://2xt-customer-K3nP7qLm2Xv-proxy-agent_browsing-session-run42-time-10:s7Kd92mQ@proxy.2extract.net:5555"

def fetch(url: str) -> str:
    r = requests.get(url, proxies={"http": PROXY, "https": PROXY}, timeout=30)
    r.raise_for_status()
    return r.text

import axios from "axios";
import { HttpsProxyAgent } from "https-proxy-agent";

const PROXY = "http://2xt-customer-K3nP7qLm2Xv-proxy-agent_browsing-session-run42-time-10:s7Kd92mQ@proxy.2extract.net:5555";
const agent = new HttpsProxyAgent(PROXY);

export async function fetch(url) {
  const { data } = await axios.get(url, { httpsAgent: agent, timeout: 30000 });
  return data;
}

For a headless browser instead of an HTTP client, see the Puppeteer and Playwright guides.

When this is the wrong tool

Be honest about the boundary. A residential IP changes where the request appears to come from. It does not:

solve CAPTCHAs
make a headless browser look like a real one
get you past a login you do not have

If you are getting CAPTCHAs rather than blocks, the fingerprint is the problem, not the IP - start with Blocked or CAPTCHA.

Introduction

MCP Server

MCP Clients

Use Cases

For AI Agents

Agent Web Access

The problem

The setup

Sessions matter more than you expect

Keeping it contained

Wiring it into a fetch

When this is the wrong tool

Next steps

Data Enrichment

Provisioning & Cost Control

​The problem

​The setup

​Sessions matter more than you expect

​Keeping it contained

​Wiring it into a fetch

​When this is the wrong tool

​Next steps

Data Enrichment

Provisioning & Cost Control

The problem

The setup

Sessions matter more than you expect

Keeping it contained

Wiring it into a fetch

When this is the wrong tool

Next steps