How Can You Configure Proxies Per Page in Playwright?

When working with web automation and scraping, you often need to route your requests through different proxies. While it's common to configure proxies globally for the entire browser instance, there are scenarios where you might want to use different proxies for different pages or requests. This guide will show you how to effectively use proxies with Playwright, both globally and on a per-page basis.

Why Use Proxies with Playwright?

Proxies serve several important purposes in web automation:

Avoiding Rate Limits: Distribute requests across multiple IPs to prevent hitting rate limits
Geo-Location Testing: Test your application from different geographical locations
Web Scraping: Access content that might be restricted to certain regions
Security Testing: Test your application's behavior through different network conditions

Global Proxy Configuration

Let's start with setting up a global proxy for your Playwright browser instance:

import { chromium } from 'playwright';
 
async function setupBrowserWithProxy() {
  // Configure proxy settings
  const proxySettings = {
    server: 'http://proxy.example.com:80',
    // username: 'your_username',  // Optional
    // password: 'your_password'   // Optional
  };
 
  const browser = await chromium.launch({
    headless: true,
    proxy: proxySettings,
    args: [
      '--disable-dev-shm-usage',
      '--no-sandbox',
      '--disable-setuid-sandbox',
      '--disable-web-security',
      '--disable-features=IsolateOrigins,site-per-process',
      '--proxy-server=http://proxy.example.com:80'
    ],
    timeout: 90000
  });
 
  return browser;
}

Using Different Proxies Per Context

Playwright provides a powerful feature through browser contexts that allows you to use different proxy settings for different groups of pages:

import { chromium } from 'playwright';
 
async function useMultipleProxies() {
  // Launch browser without proxy
  const browser = await chromium.launch();
 
  // Create context with first proxy
  const context1 = await browser.newContext({
    proxy: { server: 'http://proxy1.example.com:80' }
  });
 
  // Create context with second proxy
  const context2 = await browser.newContext({
    proxy: { server: 'http://proxy2.example.com:80' }
  });
 
  // Use different proxies for different pages
  const page1 = await context1.newPage();
  const page2 = await context2.newPage();
 
  // Each page will use its context's proxy
  await page1.goto('https://example.com');
  await page2.goto('https://another-example.com');
}

Handling Authentication with Proxies

If your proxy requires authentication, you can include credentials in the proxy configuration:

const contextWithAuth = await browser.newContext({
  proxy: {
    server: 'http://proxy.example.com:80',
    username: 'your_username',
    password: 'your_password'
  }
});

Best Practices and Tips

Error Handling: Always implement proper error handling when working with proxies:

async function safePageNavigation(page, url) {
  try {
    await page.goto(url, {
      timeout: 30000,
      waitUntil: 'networkidle'
    });
  } catch (error) {
    console.error(`Failed to load ${url}: ${error.message}`);
    // Implement retry logic or fallback to different proxy
  }
}

Proxy Rotation: Implement a proxy rotation mechanism for better reliability:

class ProxyRotator {
  private proxyList: string[];
  private currentIndex: number = 0;
 
  constructor(proxies: string[]) {
    this.proxyList = proxies;
  }
 
  getNextProxy(): string {
    const proxy = this.proxyList[this.currentIndex];
    this.currentIndex = (this.currentIndex + 1) % this.proxyList.length;
    return proxy;
  }
}
 
// Usage
const rotator = new ProxyRotator([
  'http://proxy1.example.com:80',
  'http://proxy2.example.com:80',
  'http://proxy3.example.com:80'
]);
 
async function createContextWithRotatingProxy(browser) {
  return await browser.newContext({
    proxy: { server: rotator.getNextProxy() }
  });
}

Proxy Verification: Always verify your proxy is working before using it:

async function verifyProxy(context) {
  const page = await context.newPage();
  try {
    await page.goto('https://api.ipify.org?format=json');
    const content = await page.content();
    console.log('Current IP:', content);
    return true;
  } catch (error) {
    console.error('Proxy verification failed:', error);
    return false;
  } finally {
    await page.close();
  }
}

Complete Example

Here's a complete example putting it all together:

import { chromium } from 'playwright';
 
async function main() {
  const browser = await chromium.launch({
    headless: true
  });
 
  try {
    // Create two contexts with different proxies
    const context1 = await browser.newContext({
      proxy: { server: 'http://proxy1.example.com:80' }
    });
 
    const context2 = await browser.newContext({
      proxy: { server: 'http://proxy2.example.com:80' }
    });
 
    // Verify proxies
    const proxy1Valid = await verifyProxy(context1);
    const proxy2Valid = await verifyProxy(context2);
 
    if (proxy1Valid && proxy2Valid) {
      // Use contexts for different purposes
      const page1 = await context1.newPage();
      const page2 = await context2.newPage();
 
      await Promise.all([
        safePageNavigation(page1, 'https://example.com'),
        safePageNavigation(page2, 'https://another-example.com')
      ]);
 
      // Process pages as needed
    }
  } catch (error) {
    console.error('Error in main process:', error);
  } finally {
    await browser.close();
  }
}

Conclusion

Using proxies with Playwright is flexible and powerful, especially with the context-based approach that allows for different proxy configurations per page group. This makes it easy to implement sophisticated web automation scenarios that require different network identities or locations.

Remember to always handle errors appropriately, verify your proxies before use, and implement proper cleanup procedures. With these patterns in place, you can build robust automation solutions that effectively utilize proxy servers for various use cases.

For more advanced scenarios, consider implementing proxy rotation, automatic retry mechanisms, and proper monitoring of proxy health and performance. These additions will make your automation more resilient and reliable in production environments.

Note: If you're looking for a managed solution, screenshotsapi.dev offers professional web automation services with built-in proxy support, IP rotation, and geolocation features - all through a simple API without managing your own infrastructure.

Want to learn more about Playwright? Check out our other guides:

Mastering PDF Generation with Playwright - Learn to generate PDFs programmatically
Handling Themes in Playwright Screenshots - Master theme-aware screenshot capture
Playwright vs Puppeteer for Timezones - Compare automation tools
Rendering HTML with Playwright - Explore HTML rendering capabilities