How to Use Selenium Proxy for Browser Automation

Alex Anie

Posted On: September 3, 2025

24 Min

Table of Contents

Proxies are dedicated servers that can be used to bypass restricted content or block IP addresses between a user and a website. When using a Selenium proxy with Python and Selenium WebDriver, you can route your requests through various IP addresses, helping you avoid bot detection, bypass rate limits, and access geo-restricted content. Using Selenium proxy enables a scalable, reliable scraping system.

Overview

What Is Selenium Proxy?

Selenium proxy allows you to route browser traffic through different IP addresses, helping automation scripts avoid detection, bypass rate limits, and access geo-restricted content. When integrated with Python and Selenium WebDriver, proxies enable scalable, reliable, and privacy-conscious browser automation.

Key Features of Selenium Proxy

  • Bypass Restrictions: Access geo-blocked or limited websites.
  • Avoid IP Blocking: Rotate proxies to prevent detection during scraping or testing.
  • Enhanced Privacy: Mask your real IP to simulate traffic from different regions.
  • Seamless Integration: Works with Python Selenium WebDriver locally or on cloud Selenium Grids.

Steps to Set Up Selenium Proxy

  • Install Dependencies: Add Selenium, WebDriver Manager, and dotenv to your Python environment.
  • Configure Proxy Settings: Define proxy server, port, and authentication if needed.
  • Apply Browser Options: Pass proxy details into Chrome or Firefox WebDriver options.
  • Launch Browser with Proxy: Start a Selenium WebDriver session routed through the proxy.
  • Navigate and Test: Open the target website to verify traffic routing.
  • Handle Exceptions: Catch proxy errors like timeouts or connection failures.
  • Close Browser: End the session to free system resources.

What Is a Selenium Proxy?

A Selenium proxy (or proxy server) acts as an intermediary between your automation script and the target server. When Selenium sends a request, the proxy forwards it to the destination server, receives the response, and passes it back. This setup masks your actual IP address, making the request appear as though it’s coming from a different location.

By rotating IP addresses through a Selenium proxy, you can send multiple simultaneous requests without being easily detected or rate-limited. This is especially helpful for accessing geographically restricted content, bypassing IP bans, and maintaining anonymity while scraping or automating website interactions.

When integrated with Selenium, a well-configured Selenium proxy not only reduces the risk of being blocked but also simulates user activity from various regions and ensures scalable, reliable automation testing.

To fully grasp how a Selenium proxy integrates into browser automation, it’s important to understand what is Selenium WebDriver, since WebDriver is the backbone that manages browser interactions.

Key Features of Selenium Proxy

When working with Selenium for browser automation, proxies open up a wide range of possibilities beyond just sending requests.

Here are some of the most valuable features you can leverage with Selenium proxy:

  • Bypass Restrictions: Access geo-blocked or region-specific websites by routing Selenium sessions through different servers. Perfect for testing localized content.
  • Avoid IP Blocking: Rotate proxies to mimic real traffic and prevent detection during scraping or large-scale automation.
  • Enhanced Privacy: Mask your real IP and simulate requests from multiple regions for safer testing and experimentation.
  • Seamless Integration: Easily configure proxies with Selenium WebDriver, locally or on cloud Selenium Grids, for scalable testing.

Setting Up Selenium Proxy

To set up a proxy in Selenium, the first step is to choose the programming language and web driver you want to work with.

Selenium supports multiple languages, including Python, Java, JavaScript (Node.js), C#, Ruby, and Kotlin, as well as different browser drivers such as Chrome, Firefox, and Edge.

Depending on the WebDriver you choose, the configuration and setup vary, and require a different configuration to run.

To get started, you can use Python for scripting and Chrome WebDriver for browser automation to integrate proxy configurations into Selenium.

This combination is widely used due to its simplicity, flexibility, and strong community support, making it ideal for demonstrating how to integrate proxy configurations into your Selenium setup.

Installing Required Packages

Before getting started, ensure that Python is installed on your computer. You can verify this by running the following command in your terminal:

This should return the version number of Python currently installed. If Python is not installed, you can download and install it from the official Python website.

Once Python is set up, it’s time to install the necessary packages for our project. Open your terminal in your preferred text editor. If you’re using Visual Studio Code, press Ctrl + J to open the terminal panel.

Run the following command to install the required dependencies:

You can also install colorama to enable colored output in the terminal. This is helpful for debugging and enhanced readability:

After installation, you can confirm that all packages are properly installed by running:

Output:

package-version

This confirms that your development environment is correctly set up and ready for Selenium automation with proxy support.

Basic Proxy Configuration

To start configuring with Selenium proxy, you first need a list of proxies to work with. One reliable source for free public proxies is free-proxy-list.net, which provides proxies from various countries and regions.

Free proxies often have limitations. Many are unreliable because they’re heavily used, slow, or incompatible with certain platforms. That’s why it’s a good practice to test them first before use.

free-proxy-list

On the website, click the “Get Raw List” button to copy a list of proxies. These proxies will be in the format IP:PORT.

Note: While free proxies are useful for testing, they are often unreliable. For production workloads, use premium proxy providers such as Bright Data, Oxylabs, or ScraperAPI that offer stable IP pools and compliance guarantees.

Next, create a new Python file named proxy_utils.py. Inside it, define a variable called proxy_list and paste your copied proxies as a list. Then, add the following Python code:

The Python script above tests a list of HTTP proxies to determine which one is usable, i.e., successfully returns a 200 OK status when accessing a given URL.

The find_working_proxy() function accepts three parameters:

  • test_url: The URL to test the proxy against.
  • proxies: The list of proxies to loop through.
  • timeout: The maximum number of seconds to wait for each response.

The function loops through each proxy in the list and sends a request through it. If a request returns a 200 status, the proxy is considered working and is returned. If none of the proxies succeed, the function returns None.

This script is particularly useful when working with public proxies, as it automates the process of selecting a working IP.

You can then integrate the working proxy into your Selenium Python scripts for automation, reducing request errors and improving reliability in your browser automation tasks.

Running Selenium Proxy

In the previous section on configuring Selenium proxy, you built a proxy configuration system using the find_working_proxy() function. This function loops through a list of free proxies and returns the first one that successfully connects (i.e., returns a 200 OK status).

Now, to implement a real-world scenario, you will scrape a list of datasets through a working proxy. This demonstrates how Selenium proxy can help mask your real location while performing automation tasks and shows how to connect a proxy to a Selenium test.

For this example, you will use the LambdaTest Playground site as the target for scraping and testing your Selenium proxy setup.

Code Implementation:

Create a new Python file and name it scrape_blog.py, and type out the code below.

Code Walkthrough:

  1. Import Modules: The script starts by importing all necessary modules: Selenium libraries for browser automation, webdriver-manager to manage Selenium ChromeDriver, and proxy_utils to use the find_working_proxy() function created earlier.

  2. Define the Target URL: A variable test_url is set to the blog home page on LambdaTest’s demo site:

  3. Find a Working Proxy: The script calls find_working_proxy(test_url) to get a live, working proxy. If no working proxy is found, the script exits:

  4. Configure Selenium with the Proxy: If a working proxy is found, it’s passed to Chrome via the –proxy-server flag using Options(). The script also:
    • Enables headless mode.
    • Disables automation detection features.
  5. Scrape Blog Carousel Data: Using the Selenium WebDriver, the script loads the blog homepage. With WebDriverWait, it waits for the blog carousel to load and then extracts data like:
    • Blog title
    • Author name
    • Number of comments
    • Number of views

    These are stored as dictionaries in the all_cards list.

  6. Print the Extracted Data: After collecting the blog post data, the script loops through each blog card and neatly prints the extracted information to the terminal:

  7. Clean Up: The browser session is closed using driver.quit() to free up system resources. This example demonstrates how you can integrate a working proxy with Selenium to scrape data anonymously.

    8. Test Execution:
    Now, type the command below to run the script:

    selenum-proxy-output

    Running Selenium proxy locally can strain your system, limit parallel testing, and make it difficult to manage multiple browsers, OS versions, or unstable proxies. These issues can slow down automation and cause frequent test failures.

    Cloud testing platforms provide a scalable and reliable way to run tests. With robust infrastructure, dependable proxies, and pre-configured browser environments, you can execute proxy-enabled tests in parallel across different browsers and operating systems.

    This makes it easier to monitor executions in real time, test across multiple geolocations, and avoid the hassle of complex local setups and maintenance. One such platform that offers all these capabilities is LambdaTest.

    Running Selenium Proxy on Cloud

    To scale proxy-based tests across different browsers and OS, A useful platform for this is LambdaTest, a GenAI-native test execution platform that provides a Selenium Grid online with 3,000+ browser, OS combinations. This setup lets you run proxy-enabled tests in parallel, monitor executions in real-time, and eliminate the need for local infrastructure.

    Note: Proxies in Selenium are most commonly used in QA testing scenarios such as geo-testing, performance monitoring, and restricted network simulation. If using proxies for scraping, always comply with site terms and laws.

    Here, to implement Selenium proxy on LambdaTest, all you need to do is move your Selenium test to the cloud and configure your code for scalable, cloud-based execution.

    To get started with Selenium proxy on LambdaTest, follow the steps given below:

    1. Create a New File: Create a Python file named track_product.py and paste your test code.
    2. Set Up Credentials: Get your Username and Access Key from the LambdaTest dashboard under Account Settings > Password & Security. Save them in a .env file to keep them secure and avoid public exposure.
    3. Update the Selenium Grid URL: Use the credentials in your script when connecting to LambdaTest’s Selenium Grid:
    4. Configure Capabilities: Define key automation testing parameters, such as browser, version, operating system, and additional test settings, to ensure your tests run in the exact environment you need.

      Example for Chrome:

    You can generate the required Selenium proxy capabilities using the LambdaTest Automation Capabilities Generator.

    Code Implementation:

    Code Walkthrough:

    1. Load Environment Variables: Load LambdaTest credentials from the .env file using load_dotenv().
    2. Get Credentials: Retrieve LT_USERNAME and LT_ACCESS_KEY using os.getenv().
    3. Set Grid URL: Construct the LambdaTest Selenium Grid URL with your credentials.
    4. Define Target URL: Specify the product page to scrape.
    5. Find Working Proxy: Use find_working_proxy(test_url) to select a proxy that successfully connects.
    6. Configure Chrome Options: Set Chrome options, including proxy server, disabling automation detection, and headless mode if needed.
    7. Set LambdaTest Capabilities: Define browser, version, OS, project, build, and additional test settings.
    8. Create Remote WebDriver: Start a Selenium session on LambdaTest with the configured options and capabilities.
    9. Open Target Page: Navigate to the product page using driver.get().
    10. Wait for Elements: Use WebDriverWait to ensure the product elements are loaded before scraping.
    11. Scrape Product Details: Extract product name, price, and image URL using Selenium locators.
    12. Print Results: Display the scraped product information along with the proxy used.
    13. Clean Up: Close the Selenium session using driver.quit() to free cloud resources.

    Test Execution:
    Once your environment is configured, run the test script using:

    Terminal Output:
    proxy-output

    LambdaTest Dashboard Output:

    Lambdatest-output

    For more details, check out the documentation on running Selenium Tests Behind The Proxy.

    Using Selenium Proxy with Authentication

    Small-scale websites can often be tested or scraped using free random proxies. However, when dealing with large-scale platforms like Amazon, which frequently update their DOM structure and dynamically render content, free proxies are often insufficient.

    For these complex sites, you’ll need a premium proxy service that offers reliable IPs and supports proxy authentication (such as username and password).

    Most commercial proxy services provide credentials like:

    • Proxy host
    • Proxy port
    • Username
    • Password
    • (Sometimes an API key or session token)

    Here, you will learn how to set up an authenticated Selenium proxy. You’ll see how to send a request through a proxy server that requires credentials and verify that the proxy is working by retrieving your IP and location from a target site.

    Code Implementation:
    Create a new Python file as auth_proxy.py, and type the following code below

    Code Walkthrough:

    1. Implement Proxy Authentication: Since Chrome does not natively support proxy authentication via command-line flags, implement the code using a temporary Chrome extension that injects the proxy credentials.
    2. Embed Credentials: Include the username and password in a background script within the extension.
    3. Launch Chrome with Proxy: Start Chrome in headless mode with the extension and proxy configured.
    4. Test Proxy Connection: Visit a test page like https://ipinfo.io/json to confirm that the proxy is working.
    5. Extract IP and Location: Use Selenium to extract the IP address and location details from the page.
    6. Clean Up: Remove the temporary extension ZIP after the test is complete.

    Note: Always store proxy credentials securely in environment variables or secret managers (like AWS Secrets Manager or Vault) instead of hardcoding them into scripts

    Text Execution:

    Use the following command in your terminal to print the output.

    Output:

    When using premium proxy services, Selenium proxy setup instructions may vary slightly depending on the provider. Always refer to the proxy provider’s documentation for details.

    Implement a Rotating Selenium Proxy

    When scraping large and dynamic platforms like Amazon or Alibaba, it’s common to encounter strict anti-bot mechanisms. These websites can easily detect automated scraping tools like Selenium, especially if multiple requests originate from the same IP address.

    If your script repeatedly hits a website with a single proxy IP, it risks being rate-limited, flagged, or completely blocked.

    Using a rotating Selenium proxy helps you avoid detection by randomly switching between multiple proxy IPs during the script’s execution. This simulates requests from different users across various locations, making your automation look more natural and harder to detect.

    In this example, you’ll use a list of public proxies. Each time your script runs, a random proxy is selected from the list to make a request. You can use https://ipinfo.io/json

    Code Implementation:

    Create a file named proxy_geo_checker.py and type the following code:

    Code Walkthrough:

    1. Import Modules: Load Selenium, Chrome WebDriver, Colorama, and random for proxy selection.
    2. Select Proxy: Randomly pick one IP from the proxy_list using random.choice().
    3. Configure Browser: Launch Chrome with the selected proxy in headless mode and disable automation detection.
    4. Open Target URL: Navigate to https://ipinfo.io/json to check the proxy.
    5. Extract Data: Read the JSON response to get IP address, city, country, ISP, and geolocation.
    6. Clean Up: Close the browser session with driver.quit().

    Test Execution:

    Type the command below in the terminal to run the script.

    Output:

    The output shows where the Selenium proxy appears to be located, helping verify that the rotating proxy is working as intended.

    Handling Selenium Proxy Errors

    Working with a Selenium proxy comes with its own set of challenges, such as network instability, poor proxy quality, and configuration mismatches. These issues can quickly lead to failed tests, blocked IPs, or inconsistent results.

    To avoid common pitfalls and maximize performance, it’s essential to understand how to properly handle proxy-related issues and implement optimization strategies.

    Here are some common Selenium proxy errors you might face and effective ways to solve them:

    • Problem: When faced with net::ERR_PROXY_CONNECTION_FAILED, it’s due to an invalid proxy or a proxy server being down.
      Solution: Validate proxies before starting Selenium; skip or retry invalid ones.
    • Problem: When faced with net::ERR_TUNNEL_CONNECTION_FAILED, it’s because the proxy cannot tunnel HTTPS traffic.
      Solution: Ensure the proxy supports HTTPS.
    • Problem: SSL Handshake Failed occurs due to a broken SSL certificate or an incompatible protocol.
      Solution: Switch to a different HTTPS proxy or update Selenium/WebDriver versions.
    • Problem: Timeouts and long page loads happen when the proxy server is slow or overloaded.
      Solution: Use high-quality or paid proxies; implement timeouts and fallback proxies.
    • Problem: Blocked or CAPTCHA pages occur when the proxy IP is blacklisted or flagged as suspicious.
      Solution: Rotate proxies frequently; combine with user-agent spoofing or CAPTCHA solvers.

    Apart from errors related to Selenium proxy, there are many other common exceptions in Selenium that you might encounter.

    Best Practices of Selenium Proxy

    For effective Selenium testing, a Selenium proxy helps route traffic securely and manage requests without triggering site blocks. By validating proxies, preferring HTTPS support, and rotating them, you can maintain consistent and reliable test execution.

    • Use HTTPS Proxies: Prefer HTTPS proxies for better security, compatibility, and ease of use.
    • Test Proxy Before Use: Always validate that the proxy works with your target site to avoid runtime failures. Implement a health-check script to ensure proxies return valid HTTP status codes (e.g., 200 OK).
    • Rotate Proxies: Use rotating proxies for large-scale scraping or automation to prevent IP bans and rate-limiting. Rotate through a list of proxies periodically or per request batch.
    • Check Latency and Speed: Select proxies with low latency to ensure your Selenium automation runs efficiently.
    • Avoid Free Proxies for Critical Tasks: Free proxies can be unreliable, slow, or blocked; consider premium proxies for important tasks.
    • Use Proxy Authentication When Needed: For secure or paid proxies, integrate the username and password correctly in your Selenium proxy setup.
    • Limit Requests per Proxy: Avoid sending too many requests from a single proxy to reduce the risk of detection.
    • Integrate with Headless Browsers: Ensure your Selenium proxy setup works with headless mode if your automation requires it. Headless browsers consume fewer resources but may trigger anti-bot measures; combine with proxy and browser fingerprinting tweaks.
    • Use Exponential Backoff for Retries: If a proxy fails, retry the operation with gradually increasing delays to reduce server stress.
    • Log Proxy Usage: Keep logs of working/failing proxies, response times, and proxy activity to troubleshoot issues and monitor performance.
    • Consider Geolocation Needs: Choose proxies from regions relevant to your testing or scraping requirements.

    Conclusion

    A Selenium proxy is an essential tool for large-scale scraping and testing, helping you bypass restrictions, avoid rate limits, and simulate real-user behavior across different locations. From simple HTTP setups to rotating and authenticated proxies, each method enhances the reliability of your automation.

    By combining Selenium with effective proxy strategies and best practices, you can build more resilient, efficient, and scalable automation scripts. Whether you’re testing locally, handling authentication, or deploying to the cloud with platforms like LambdaTest, mastering Selenium proxy integration gives you a strong edge in modern web automation.

    Frequently Asked Questions (FAQs)

    How do I test if a proxy is working in Selenium?

    You can test a proxy by directing your Selenium-driven browser to a site like httpbin.org/ip or whatismyipaddress.com. These sites return the IP address seen by the server. If the IP shown matches your proxy instead of your real one, then the proxy is working. Logging this result is a reliable way to verify proxy functionality.

    How do I set up a proxy in WebDriver for Chrome?

    In Selenium, you can configure Chrome with a proxy by passing the –proxy-server argument through ChromeOptions. For example, you can specify chrome_options.add_argument(“–proxy-server=http://ip:port”) before initializing the driver. This ensures that all traffic from the browser session is routed through the given proxy. The same approach can be adapted for SOCKS and HTTPS proxies.

    How do proxies, Selenium, and Python work together in web automation?

    Selenium acts as the automation tool that controls the browser, while Python provides the programming logic to orchestrate tasks and handle data. A proxy sits between your automated browser and the target website, masking your real IP or simulating access from a different location. Together, these components enable you to access restricted content or reduce the risk of detection when scraping or testing. Each layer complements the other to make automation more flexible and reliable.

    Can I use different proxies for each Selenium session?

    Yes, you can use a unique proxy for every Selenium session. Doing so reduces the chance of being flagged or blocked by websites since requests appear to come from multiple users or regions. It’s a common practice in large-scale scraping and testing scenarios. By rotating proxies, you add an extra layer of anonymity and resilience to your automation.

    Is proxy authentication necessary?

    In many cases, yes, especially when using premium or paid proxies. Authentication usually requires a username and password, which adds security and ensures the proxy is not misused. Free proxies often don’t need authentication, but are typically less stable and more prone to downtime. Using authenticated proxies is generally safer for long-term projects.

    How can I rotate proxies automatically in Selenium?

    You can implement proxy rotation by maintaining a list of proxies or fetching them from a proxy API. Before launching each new Selenium session, you randomly assign a proxy from this list. This way, every browser instance connects through a different IP, reducing patterns that might trigger detection. Some developers also integrate proxy rotation libraries or services for easier management.

    What types of proxies are supported in Selenium?

    Selenium supports common proxy types such as HTTP, HTTPS, and SOCKS. The specific configuration depends on the browser you are automating. For Chrome and Firefox, you can set these proxies using browser options or profiles. This flexibility allows you to choose the right proxy type for your use case.

    Will using a proxy with Selenium make web scraping undetectable?

    No, proxies alone do not guarantee stealth. While they can mask your IP, websites use multiple detection techniques like browser fingerprinting, unusual request patterns, or bot detection tools. To increase your chances of avoiding detection, you should also rotate user agents, manage cookies, and mimic human-like browsing behavior. Proxies are just one part of a broader anti-detection strategy.

    Is it legal to use proxies with Selenium for scraping?

    Legality depends on the target site and how you use the data. Many websites restrict automated scraping in their Terms of Service or block bots via robots.txt. While proxies can help you bypass restrictions technically, you should always consider the legal and ethical implications. To stay safe, only scrape data you are legally allowed to access.

    Citations

    Alex Anie

    A Front End Technical Writer. I write blogs on Front End Technologies, such as HTML, CSS, JavaScript. When I am not coding, I enjoy watching tutorials and listening to twitter spaces. I live in Lagos, Nigeria

    See author's profile

Author

A Front End Technical Writer. I write blogs on Front End Technologies, such as HTML, CSS, JavaScript. When I am not coding, I enjoy watching tutorials and listening to twitter spaces. I live in Lagos, Nigeria

Blogs: 19

linkedintwitter