My AI agents needed a browser, a git server, and a way to not make me lose my mind managing all of it.
I've been building my own AI agents that live inside Docker containers, instead of just grabbing something like Open Claw or Hermes off the shelf.
Partly because I wanted to actually understand what's happening under the hood, and partly because the off-the-shelf options felt like a black box I'd have to fight with eventually anyway.
Once you start down this path, three problems show up almost immediately: how does the agent see and click around a web page, where do its code changes go for review, and how do you keep track of five containers without losing your mind.
Here's what I figured out for each one.
Problem 1: giving an agent eyes (and hands) in a browser
This was the trickiest part, so let's start here since it's the one that actually matters most.
My agent needed to do two different things: click through a web app it built to test the UI, and scrape content from sites that flat out refuse headless browsers. Turns out these aren't quite the same problem, and the solution had to cover both.
Option A — fully headless, no display at all. Playwright (or Puppeteer) can launch Chromium with zero GUI, no Xvfb, nothing. This is the lightest setup — just a Dockerfile with the Playwright MCP server installed and a port exposed. The catch: I can't watch what's happening. If something breaks, I'm stuck reading screenshots after the fact instead of seeing it live.
Option B — headed browser with Xvfb. This gives the browser a real window to render into using a virtual framebuffer, even though there's no physical monitor. Good middle ground, but on its own I still can't see the screen without VNC layered on top.
Option C — headed browser with Xvfb + noVNC. Same virtual display as Option B, but now there's a web-based VNC viewer on top, so I can open a browser tab and literally watch the agent click around in real time. This is what made the "test the UI by clicking through it" use case actually work — I could sit there and watch Chrome open, navigate, and interact with my app exactly like a person would.
Here's the part I didn't expect going in: "headless" doesn't mean the page doesn't render. The rendering engine runs fully either way — headless just means there's no visible window. So for most scraping tasks, even sites with heavy JavaScript, headless mode is fine on its own.
The problem only shows up with sites that specifically detect and block headless browsers. For those, a headed instance (Xvfb, with or without noVNC) is what actually gets through, since it presents like a real browser session.
My recommendation: run both, but as separate concerns. Default everything to headless Playwright MCP, since it's lightest and works for 90% of what the agent does.
Then keep a second container (or mode) with Xvfb + noVNC on standby specifically for: (1) UI testing where I want to watch the agent click through my own apps, and (2) the handful of stubborn sites that block headless sessions.
Trying to make one setup do both jobs all the time just adds overhead you don't need most of the time.
One more decision worth mentioning: whether the browser container also has full OS-level tools (like a Kali-based image with nmap, sqlmap, etc.) or whether it's scoped to just the browser.
Unless your agent genuinely needs OS-level access for something beyond browsing, keep it scoped to the browser. Less surface area, less to think about if something goes wrong.
Problem 2: where do code changes actually go
My agent uses git to push changes for review, which meant I needed somewhere for those pushes to land.
GitHub was the obvious default, but I started wondering if I should just self-host instead, since I already have a Linux server doing not much else.
The case for self-hosting (Gitea):
- Full control — code never leaves my own box
- No GitHub Actions minutes bill, no rate limits, no surprise policy changes
- It's free. Genuinely free, no asterisk, MIT licensed
- Packages neatly into Docker, so it fits right into the same infrastructure as everything else
The case against:
- I'm now the sysadmin. Backups, updates, security patches — all me
- Less plug-and-play with third-party tools that assume github.com
- Some upfront setup if I want HTTPS and a real domain instead of just an IP and port
When self-hosting actually makes sense: if your agents are pushing to a private server anyway (which mine are, since they live in Docker containers on my own infrastructure), self-hosting Gitea fits naturally — there's no real benefit to bouncing code out to a third party just to bounce it back in.
It's also the right call if you're on a home network / homelab setup, since you can skip HTTPS and domain config entirely and just hit http://server-ip:3000.
Where it stops making sense is if you want outside contributors, broad third-party integrations, or you just don't want to be the one who fixes it at 11pm if it goes down.
For my setup specifically — agent runs locally, pushes to a server I already control, no outside collaborators — self-hosting Gitea will be a good options for me.
It will run as a single Docker Compose stack (Gitea + Postgres), needs barely any resources, and supports Gitea Actions if I ever want CI/CD that's compatible with existing GitHub Actions workflow files.
If you want the absolute lightest option, Gogs runs on 256MB of RAM with no CI/CD baked in.
If you care more about open-source governance than staying close to the for-profit-owned upstream, Forgejo is a drop-in community fork of Gitea — same workflow, different priorities behind the scenes.
Problem 3: actually managing all these containers
Once you've got a browser container, a Gitea container, and however many agent containers, you need a way to see what's running without docker ps-ing your life away. That's where Portainer came in.
Setup is genuinely just a few steps:
- Create a
docker-compose.yml:
services:
portainer:
image: portainer/portainer-ce:latest
container_name: portainer
restart: always
ports:
- "9443:9443"
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- portainer_data:/data
volumes:
portainer_data:
-
Run
docker compose up -d -
Go to
https://your-server-ip:9443, click through the self-signed cert warning (expected, not a bug), and create an admin account — you've got about 5 minutes before it times out, so don't wander off mid-setup like I did the first time -
Click "Get Started" to connect to your local Docker socket, and you're in
A couple things worth flagging: mounting /var/run/docker.sock gives Portainer full control over your Docker daemon, so don't expose this to anyone you don't trust.
And if you want your volume data on a specific path instead of a Docker-managed one, swap the named volume for an environment variable pointing at a real folder — just make sure the .env file sits next to your compose file or it won't pick it up.
From here, every container — the browser agent, Gitea, the agent containers themselves — shows up in one dashboard. Logs, restarts, resource usage, all in one place instead of SSH-ing in and squinting at terminal output.
Where this leaves things
Browser access turned out to be the real design decision in this whole project — everything else (Gitea, Portainer) was more straightforward "pick the tool, follow the steps" territory.
If you're building something similar, I'd start with headless and only add the Xvfb/noVNC layer once you actually hit a site or a UI test that needs it.
No point running a heavier setup for problems you don't have yet.
Anyone else running their own agent infrastructure instead of using a packaged option? Curious what's tripped you up.