PixelRAG is an open-source visual Retrieval-Augmented Generation system developed by UC Berkeley SkyLab and BAIR. It indexes documents, PDFs, and web pages as high-resolution screenshot tiles rather than parsed plain text, allowing vision-language models to reason over tables, charts, diagrams, and layout hierarchy.

How does pixelbrowse give Claude Code eyes?

pixelbrowse is a plugin that captures a web page screenshot via the pixelshot CLI tool (Playwright/CDP) and feeds the rendered visual tiles directly to Claude's vision context, enabling accurate reading of infographics, pricing tables, and UI layouts.

PIXEL_RAG_VISUAL.exe

<- Back to Resources

VISUAL RAG & MULTIMODAL AI

PixelRAG: Visual Document Search & Giving AI Coding Agents Eyes ⚡

Why web screenshots beat plain text for Retrieval-Augmented Generation: preserving tables, layout, and infographics across 8.28M documents with UC Berkeley's PixelRAG.

AEO SUMMARY Visual Knowledge Retrieval

Traditional text-based RAG strips away CSS, layout structure, tables, and charts during HTML-to-text parsing. PixelRAG (UC Berkeley SkyLab / BAIR) renders documents directly to screenshot tiles and executes vector retrieval over the images themselves. This preserves 100% of spatial formatting, allowing vision models (Claude 3.5 Sonnet, GPT-4o) to accurately answer questions about complex diagrams, financial tables, and UI states.

AGENT TOOLING TIP Give Claude Eyes with pixelbrowse

Install pixelshot on your machine and use the pixelbrowse plugin for Claude Code. Instead of feeding messy raw HTML to your agent, Claude screenshots the URL and visually reads the interface, boosting web scraping accuracy by over 40%.

Quickstart & API Usage

# Install CLI tool
pip install pixelrag

# Render any web page or PDF to screenshot tiles
pixelshot https://en.wikipedia.org/wiki/Artificial_intelligence --output ./tiles

# Search the hosted 8.28M Wikipedia visual index
curl -X POST https://api.pixelrag.ai/search \
  -H "Content-Type: application/json" \
  -d '{"queries": [{"text": "What is the transformer architecture?"}], "n_docs": 5}'

MASTER PROMPT

1-Click Visual Document Analysis & Extraction Prompt

Paste this prompt into Claude or GPT-4o alongside screenshot tiles of any complex table, dashboard, or financial report:

PIXEL_RAG_VISUAL_EXTRACTOR

Act as an expert Multimodal Document Intelligence and Visual Data Extraction Specialist utilizing the PixelRAG framework. Analyze the attached screenshot tiles:

1. SPATIAL & LAYOUT RECONSTRUCTION:
- Identify all tables, charts, callout boxes, and hierarchical headers visible in the image.
- Map out the exact columns, headers, and row data without dropping formatting.

2. VISUAL INSIGHT SYNTHESIS:
- Extract all key metrics, percentages, data trends, and footnotes.
- Explain what the visual diagram/infographic conveys that raw text would miss.

3. STRUCTURED DATA OUTPUT:
- Convert the extracted tabular data into a clean, copyable Markdown table and JSON object.

GITHUB REPO PixelRAG

View on GitHub ->

Visual document RAG and pixelbrowse framework from StarTrail-org / UC Berkeley.

Prerequisites: Python 3.10+, PyTorch, Playwright / Selenium

# 1. Clone the PixelRAG repository
git clone https://github.com/StarTrail-org/PixelRAG.git
cd PixelRAG

# 2. Install dependencies
pip install -r requirements.txt

# 3. Run visual document retrieval pipeline
python pixelrag_demo.py --doc sample.pdf --query "Extract total ARR and EBITDA from table"

⚡ BUILDING MULTIMODAL AI AGENTS?

I help founders build automated GTM systems, vision-enabled AI workflows, and scalable revenue engines.

Book a Growth Strategy Consultation ->