What is a headless browser?

What is a headless browser?

The innate value of code is its ability to automate work on behalf of people. Traditionally, that means connecting many APIs together, processing data, and presenting it in a beautiful interface. Code uses other code to get things done.

Over the last decade, software has undergone a massive shift from desktop to web applications. People now complete their work, handle their finances, and run their businesses through web browsers. As more of our daily tasks move online, the desire to automate web-based workflows becomes increasingly crucial. But what if you want to automate something that doesn't expose an API? What if you need to interact with an application that hasn't been built to be used by code?

Simply fetching HTML data from the website often isn't enough. Modern websites use JavaScript to load data on the client side and control interactive elements - if you only retrieve the HTML, you miss crucial parts of how the application actually works.

Instead, what if you could use software the same way people do? Going to the website, clicking buttons, submitting forms, and performing all the actions a person would do. But instead of being controlled by a human, it's controlled by code.

This is the concept of a headless browser.

How does a headless browser work?

A headless browser is a browser designed to be controlled by code. It's functionally identical to the web browser you use on your machine, but can run on a server. It's "headless" because there's no graphical user interface (GUI), but it still loads and interacts with pages exactly like a regular web browser does.

But if there's no GUI, how does code control a headless browser? The answer lies in browser protocols - standardized ways for code to communicate with and control browsers. The most widely used is the Chrome DevTools Protocol (CDP), which provides low-level control over browser operations.

These protocols are low-level, meaning they provide a lot of control over the browser, but they are also complex and difficult to work with. It's like building software using machine code.

That's why developers typically use browser automation frameworks - tools that provide simple, abstracted interfaces for controlling browsers.

Browser automation frameworks

Browser automation frameworks make it easy to write code that controls browsers. They handle the complexity of protocol communication and provide familiar programming interfaces. Here are the most popular options:

Playwright

• Created by Microsoft

• Modern, promise-based API

• Originally focused on testing web applications

Stagehand

• Created by Browserbase

• A superset of Playwright, with AI features for self-healing automations

• Specialized for browser automation

Puppeteer

• Developed by Google’s Chrome team

• Deep integration with CDP

• Excellent for PDF generation and screenshots

Selenium

• Oldest automation framework

• Uses WebDriver protocol - a legacy protocol for browser automation

• Large ecosystem of tools and plugins

Using headless browsers

Headless browsers unlock new possibilities for developers. By enabling code to interact with any website the same way people do, they allow you to build integrations without being limited by available APIs.

Getting started with headless browsers is straightforward. With just a few lines of code, you can automate basic web interactions locally:

const { chromium } = require('playwright');

async function example() { 
  const browser = await chromium.launch(); 
  const page = await browser.newPage(); 
  await page.goto('https://example.com'); 
  await page.click('#submit-button'); 
}

However, running headless browsers in production presents significant challenges.

You need to:

• Manage browser instances and resources

• Handle concurrent sessions

• Deal with browser crashes and cleanup

• Scale infrastructure as demand grows

• Monitor performance and reliability

This is where Browserbase comes in. We provide a reliable, scalable infrastructure for running headless browsers in production, letting you focus on building your automation logic rather than managing browser infrastructure.