Skip to navigation
Configure the browser

Trim Network Traffic

View as Markdown

Residential proxy bytes are usually the most expensive line on a scraping bill, and most of those bytes are ads, trackers, and images your automation never looks at. Turn them off at the runtime and measure the difference on the run itself.

import { Bctrl } from "@bctrl/sdk";
import { chromium } from "playwright";
const bctrl = new Bctrl({ apiKey: process.env.BCTRL_API_KEY! });
const runtime = await bctrl.runtimes.create({
  type: "browser",
  name: "lean-scraper",
  config: {
    proxy: { type: "managed-rotating", country: "US" },
    networkTraffic: {
      blockAds: true,
      blockTrackers: true,
      blockResourceTypes: ["image", "media", "font"], // pages you read, not look at
      urlBlocklist: ["https://metrics.acme-cdn.com"],
    },
  },
});
const { connection, runId } = await bctrl.runtimes.start(runtime.id);
const browser = await chromium.connectOverCDP(connection.cdpUrl);
const context = browser.contexts()[0] ?? (await browser.newContext());
const page = context.pages()[0] ?? (await context.newPage());
for (const url of urlsToScrape) {
  await page.goto(url);
  // ...extract what you need...
}
await browser.close();
await bctrl.runtimes.stop(runtime.id);
// The receipt: what the run actually consumed.
const usage = await bctrl.runs.usage(runId);
console.log(usage.proxyBytes, usage.runtimeSeconds, usage.creditsUsed);

Run it once with networkTraffic and once without, on the same URL list, and compare proxyBytes - on media-heavy sites the cut is usually dramatic.

📸 Content TODO: do that comparison on a real site list and put the two proxyBytes numbers here as a small before/after table - a concrete “X% fewer proxy bytes” beats the adjective.

The knobs

  • blockAds / blockTrackers - filter-list blocking, the safe default for almost everything.
  • blockResourceTypes - drop whole resource classes (image, media, font, ping, beacon, prefetch, texttrack). Don’t block image if a flow needs to see the page - vision agents and screenshot steps included.
  • urlAllowlist / urlBlocklist - surgical control when one host is the problem (or the only thing allowed).
  • saver - preset data-saver levels (light / medium / high, or none to disable) when you’d rather not hand-pick.

Blocking also speeds pages up - fewer requests through the proxy means faster loads, which compounds across a long crawl.

Next