How to turn any webpage into clean markdown for your LLM

TL;DR: Your agent doesn't need raw HTML, it needs the content. Fetch Extract turns any URL into clean markdown or structured JSON in one HTTP call, so you skip the browser, the HTML parsing, and the wasted tokens. Set format: "markdown" for readable text or format: "json" with a schema for structured fields.

Feeding a webpage to an LLM sounds simple until you look at what a page actually contains. A single article carries nav bars, cookie banners, ad slots, script tags, inline styles, and tracking pixels wrapped around the few paragraphs you wanted. Pass that raw HTML to a model and you pay for every token of markup, and the model still has to guess which parts are the content.

The fix is to convert the page to markdown before it ever reaches the model. Markdown keeps the structure a model reads well, headings, lists, links, and code blocks, and drops the markup a model doesn't need. This post shows how to do that conversion in one API call, and when to reach for structured JSON instead.

Why raw HTML is the wrong input for an LLM

Raw HTML is expensive and noisy. A content page is often ten to twenty times larger in bytes than the text a reader sees, and most of that weight is markup the model has to wade through. You pay for those tokens on every call, and the surrounding tags dilute the signal your prompt depends on.

Markdown solves both problems at once. It is compact, so a page costs far fewer tokens, and it preserves the semantic structure models were trained on. A heading stays a heading, a list stays a list, a link keeps its text and target. The model spends its context on meaning instead of parsing div soup.

How do you convert a webpage to markdown?

Send the URL to Fetch with format: "markdown". Browserbase retrieves the page and returns markdown in the content field. There is no browser session to manage and no HTML parser to write.

import Browserbase from "@browserbasehq/sdk";

const bb = new Browserbase({ apiKey: process.env.BROWSERBASE_API_KEY! });

const response = await bb.fetchAPI.create({
  url: "https://www.browserbase.com/",
  format: "markdown",
});

console.log(response.content);

The response envelope is the same across every format. What changes is the value in content: with format: "markdown" you get markdown generated from the fetched page, ready to drop into a prompt.

How do you get structured JSON instead of markdown?

When you want specific fields rather than the whole page, set format: "json" and pass a JSON Schema describing the shape you want. Browserbase extracts the page into that structure and returns it in content as an object.

import Browserbase from "@browserbasehq/sdk";

const bb = new Browserbase({ apiKey: process.env.BROWSERBASE_API_KEY! });

const response = await bb.fetchAPI.create({
  url: "https://www.browserbase.com/",
  format: "json",
  schema: {
    type: "object",
    properties: {
      title: { type: "string" },
      summary: { type: "string" },
    },
    required: ["title"],
  },
});

console.log(response.content);

The schema field is only valid when format is json. Send a schema with raw or markdown and the request is rejected. Reach for JSON when a downstream step needs typed fields, and for markdown when you want the readable page to hand a model.

Markdown or JSON: which should you use?

Use markdown when you want to hand a model the whole readable page, and JSON when a downstream step needs specific typed fields. Raw is for the rare case where you want the exact upstream bytes untouched.

NeedFormatWhat you get back
Feed a whole page to an LLMmarkdownReadable markdown, headings and lists intact
Pull specific typed fieldsjsonA structured object matching your schema
The exact upstream bytesrawThe original response body, unchanged

Fetch without a format flag returns raw content and is the cheapest option. markdown and json run a conversion (Fetch Extract) and are priced separately.

What are the limits to know?

Fetch retrieves pages over HTTP and does not run JavaScript, so a page that renders its content client-side won't expose that content to a plain fetch. It also caps responses at 5 MB, times out after 60 seconds, and can't convert PDF responses to markdown or JSON.

When you hit one of those, fall back to a full browser session, which renders JavaScript and handles long page loads. A common pattern is to try Fetch first and catch the 502 for a page over the size limit.

import Browserbase from "@browserbasehq/sdk";
import { chromium } from "playwright-core";

const bb = new Browserbase({ apiKey: process.env.BROWSERBASE_API_KEY! });

try {
  const response = await bb.fetchAPI.create({
    url: "https://httpbin.org/",
  });
  console.log(response.content);
} catch (err: any) {
  if (err?.statusCode === 502) {
    // Fall back to a full browser session for large pages
    const session = await bb.sessions.create();
    const browser = await chromium.connectOverCDP(session.connectUrl);
    const page = browser.contexts()[0].pages()[0];
    await page.goto("https://httpbin.org/");
    const content = await page.content();
    console.log(content);
    await browser.close();
  }
}

How does this fit an agent pipeline?

Markdown conversion is the middle step of a three-part pattern. Search finds the sources when your agent doesn't yet have a URL. Fetch reads those pages fast and cheap, and converts them to markdown or JSON for the model. A browser session handles the pages that need a login, real interaction, or JavaScript rendering.

Most reading work never needs a browser at all, which is the point of doing it over HTTP. Start with Fetch for anything you only need to read, and reserve full sessions for the pages that genuinely require them. For a deeper breakdown of which primitive fits which job, see which Browserbase API to use.

Frequently Asked Questions

What is the difference between Fetch and Fetch Extract?

Fetch without a format flag returns the page content directly and is the cheapest option. Fetch Extract is what runs when you set the format to markdown or json, converting the page before returning it. That conversion is priced separately.

Do I need a browser session to convert a page to markdown?

No. Fetch retrieves the page over HTTP and returns markdown in one call, with no session to create or tear down. Use a browser session only when a page needs JavaScript rendering, a login, or real interaction.

Why convert HTML to markdown before sending it to an LLM?

Raw HTML is far larger than the text a reader sees and buries the content in markup, so you pay for wasted tokens and dilute the signal. Markdown is compact and keeps the structure models read well, headings, lists, and links, so the model spends its context on meaning.

Can Fetch convert a PDF to markdown?

No. Fetch can't convert PDF responses to markdown or structured JSON. For PDF content, use a browser session instead.

What happens if the page is larger than 5 MB?

Fetch returns a 502 error when the response body exceeds 5 MB. Catch it and fall back to a browser session, which handles large pages and long loads.

Take one URL you're currently feeding to a model as raw HTML and run it through Fetch with format: "markdown" instead. Compare the token count and the answer quality. The Fetch docs cover every option, including proxies, redirects, and JSON extraction.