Puppeteer Real Browser

This package prevents Puppeteer from being detected as a bot in services like Cloudflare and allows you to pass captchas without any problems. It behaves like a real browser. If you are only interested in Cloudflare WAF, please check this repo: https://github.com/zfcsoftware/cf-clearance-scraper

Installation

If you are using a Linux operating system, xvfb must be installed for the library to work correctly.

npm i puppeteer-real-browser

if you are using linux:

sudo apt-get install xvfb

Include

CommonJS

const { connect } = require('puppeteer-real-browser');

const start = async () => {
    const { page, browser } = await connect()
}

Module

import { connect } from 'puppeteer-real-browser'

const { page, browser } = await connect()

Usage

const { connect } = require("puppeteer-real-browser")

async function test() {

    const { browser, page } = await connect({

        headless: false,

        args: [],

        customConfig: {},

        turnstile: true,

        connectOption: {},

        disableXvfb: false,
        ignoreAllFlags: false
        // proxy:{
        //     host:'<proxy-host>',
        //     port:'<proxy-port>',
        //     username:'<proxy-username>',
        //     password:'<proxy-password>'
        // }

    })
    await page.goto('<url>')

}

test()

headless: The default value is false. Values such as “new”, true, “shell” can also be sent, but it works most stable when false is used.

args: If there is an additional flag you want to add when starting Chromium, you can send it with this string. Supported flags: https://github.com/GoogleChrome/chrome-launcher/blob/main/docs/chrome-flags-for-tools.md

customConfig: https://github.com/GoogleChrome/chrome-launcher The browser is initialized with this library. What you send with this object is added as a direct initialization argument. You should use the initialization values in this repo. You should set the userDataDir option here and if you want to specify a custom chrome path, you should set it with the chromePath value.

turnstile: Cloudflare Turnstile automatically clicks on Captchas if set to true

connectOption: The variables you send when connecting to chromium created with puppeteer.connect are added

disableXvfb: In Linux, when headless is false, a virtual screen is created and the browser is run there. You can set this value to true if you want to see the browser.

ignoreAllFlags If true, all initialization arguments are overridden. This includes the let's get started page that appears on the first load.

How to Install Puppeteer-extra Plugins?

Some plugins, such as puppeteer-extra-plugin-anonymize-ua, may cause you to be detected. You can use the plugin installation test in the library's test file to see if it will cause you to be detected.

The following is an example of installing a plugin. You can install other plugins in the same way as this example.

npm i puppeteer-extra-plugin-click-and-wait

const test = require('node:test');
const assert = require('node:assert');
const { connect } = require('puppeteer-real-browser');

test('Puppeteer Extra Plugin', async () => {
    const { page, browser } = await connect({
        args: ["--start-maximized"],
        turnstile: true,
        headless: false,
        // disableXvfb: true,
        customConfig: {},
        connectOption: {
            defaultViewport: null
        },
        plugins: [
            require('puppeteer-extra-plugin-click-and-wait')()
        ]
    })
    await page.goto("https://google.com", { waitUntil: "domcontentloaded" })
    await page.clickAndWaitForNavigation('body')
    await browser.close()
})

Docker

You can use the Dockerfile file in the main directory to use this library with docker. It has been tested with docker on Ubuntu server operating systems.

To run a test, you can follow these steps

git clone https://github.com/zfcsoftware/puppeteer-real-browser

cd puppeteer-real-browser

docker build -t puppeteer-real-browser-project .

docker run puppeteer-real-browser-project