Puppeteer Support for Firefox

Puppeteer controls a browser... from the outside... like a puppeteer controls a puppet. Other tools like Cypress (and ironically the very first version of Selenium 20 years ago) drive the browser from the inside using JavaScript. But we abandoned that "inside out" approach in later versions of Selenium because of the limitations imposed by the browser JS security sandbox. Cypress is still trying to make it work and I wish them luck.

You could probably figure out how to connect Llama to Puppeteer. (If no one has done it, yet, that would be an awesome project.)

Is it possible to now use Puppeteer from inside the browser?

Talking about WebDriver (BiDi) in general rather than Puppeteer specifically, it depends what exactly you mean.

Classic WebDriver is a HTTP-based protocol. WebDriver BiDi uses websockets (although other transports are a possibility for the future). Script running inside the browser can create HTTP connections and create websockets connections, so you can create a web page that implements a WebDriver or WebDriver BiDi client. But of course you need to have a browser to connect to, and that needs to be configured to actually allow connections from your host; for obvious security reasons that's not allowed by default.

This sounds a bit obscure, but it can be useful. Firefox devtools is implemented in HTML+JS in the browser (like the rest of the Firefox UI), and can connect to a different Firefox instance (e.g. for debugging mobile Firefox from desktop). The default runner for web-platform-tests drives the browser from the outside (typically) using WebDriver, but it also provides an API so the in-browser tests can access some WebDriver commands.

Yes. I'm not aware of any documentation walking one through it though.

There is a extension api that exposes a CDP connection [1][2]

You can create a Puppeteer.Browser given a CDP connection.

You can bundle Puppeteer in a browser (we do this in Lighthouse/Chrome DevTools[3]).

These two things is probably enough to get it working, though it may be limited to the active tab.

[1] https://chromedevtools.github.io/devtools-protocol/#:~:text=...

[2] https://stackoverflow.com/a/55284340/24042444

[3] https://source.chromium.org/chromium/chromium/src/+/main:thi...

webdriver bidi info -https://www.youtube.com/watch?v=6oXic6dcn9w

local scraping howto - https://www.freecodecamp.org/news/web-scraping-in-javascript...

local LLM framework - https://ollama.com/

Unironically, ask ChatGPT (or your favorite LLM) to create a hello world WebDriver or Puppeteer script (and installation instructions) and go from there.

I’d go with puppeteer for your use case as it’s the easier option to set up browser automation with. But it’s not like you can really go wrong with playwright or selenium either.

Playwright only really gets better than puppeteer if you’re doing actual website testing of a website you’re building which is where it shines.

Selenium is awesome, and probably has more guide/info available but it’s also harder to get into.

Use playwright's code generator that turns turn page interactions into code.

https://playwright.dev/python/docs/codegen-intro

I am an active user of both Selenium and Puppeteer/Pyppeteer. I use them because it's what I learned and they still work great, and explicitly because it's not Microsoft.

Is there any reason why I should consider Selenium or Puppeteer?

I'm not a heavy user of these tools, but I've dabbled in this space.

I think Playwright is far ahead as far as features and robustness go compared to alternatives. Firefox has been supported for a long time, as well as other features mentioned in this announcement like network interception and preload scripts. CDP in general is much more mature than WebDriver BiDi. Playwright also has a more modern API, with official bindings in several languages.

One benefit of WebDriver BiDi is that it's in process of becoming a W3C standard, which might lead to wider adoption eventually.

But today, I don't see a reason to use anything other than Playwright. Happy to read alternative opinions, though.

Maybe you don't want to live in a world where Microsoft owns everything (again)?

I think Playwright depends on forking the browsers to support the features they need, so that may be less stable than using a standard explicitly supported by the browsers, and/or more representative of realistic browser use.

It works for me with stock Chromium and Chrome on Linux. But for Firefox, i apparently need a custom patched build, which isn't available for the distro i run, so i haven't confirmed that.

IIRC, you can use the system installed browser, but need to know the executable path when launching. I remember it being a bit of a pain to do, but have done it.

Guidepup looks like it's a decent stab in that direction: https://www.guidepup.dev/

Only Windows and MacOS though, which is a problem for build pipelines. I too would very much like the page descriptions and the accessibility inputs to be the primary way of driving a page. It would make accessible access the default, rather than something you have to argue for.

It depends on what you’re testing. Much of a typical page is visual noise that is invisible to the accessibility tree but is often still something you’ll want tests for. It’s also not uncommon for accessible ui paths to differ from regular ones via invisible screen-reader only content, eg in a complex dropdown list. So you can end up with a situation where you test that accessible path works but not regular clicks!

If you really want gold standard screen reader testing, there’s no substitute for testing with actual screen readers. Each uses the accessibility tree in its own way. Remember also that each browser has its own accessibility tree.

This is not always possible as some machines don't support network namespaces, but it's a perfectly valid solution. But this solution is Linux-only, do BSD OSes like MacOS support UID and NET namespaces?

Great, I stand corrected! I still don't know how they convince firefox/chromium to use a pipe as a websocket transport layer.

If you open up the code Playwright codebase you will discover that it is literally Puppeteer with the copyright message header in the base files belonging to Google. It is a fork.

I also wonder the same. Playwright is so good. I simply don't have flaky tests even when dealing with features that are playwrights' fault.

I used to have so many issues with Selenium and so only used it in must have situations defaulting to capybara to run out specs.

There used to be an extension for Firefox called "IE Tab for Firefox" that used the IE rendering engine inside a Firefox tab, for sites that only worked in IE.

There are some browsers that support multiple rendering engines out of the box, like Maxthon (Blink + Trident) and Lunascape (Blink + Gecko + Trident).

Indeed, Firefox uses PDF.js which I've found to produce really good results.

have you actually done any web scrapping at scale? The problem is never the web automation. It's bypassing IP blacklist, rate limits, capcha etc, and a hosted service can provide solutions for those:

Proxies included..., Auto Captcha Solving, Advanced Stealth Mode

Other than that, like everything else, a hosted service is always an option and not contradict with you being able to host that service directly, they're just for different sets of constrains.