Kubernetes, Whyyyyyy!!!!

This one’s for anyone who ever thought,“How hard could self-hosted Kubernetes really be?”…and then promptly fell down the VPN tunnels of despair, container runtime configs, and Calico black magic.

Welcome to my personal chaos circus, where the clowns are headless Selenium scrapers and the ringmaster is kubelet. 🎪

🕷️ Why I Even Did This to Myself

Kosmopulse—the geospatial search engine I’ve been building—is powered by dozens of scraping bots, enrichment services, and a PostgreSQL backend. And I wanted full control:

• No cloud vendor lock-in

• Custom VPN mesh between nodes

• Maximum flexibility for RAM-hungry bots

• 100% observability from my Windows PC

So naturally, I went with bare-metal Kubernetes across:

• My local WSL2-based Ubuntu node (control plane)

• Two remote VPS worker nodes, all communicating via WireGuard VPN

A terrible idea?Yes.Would I do it again?Also yes.

💥 Pain

• WSL2 IPs change every reboot → had to pin down kubeadm API bind and re-wire DNS hacks using wireguard.

• VPS networking is weird → had to fight ufw/nftables until packets agreed to behave.

• Calico said “No routes for you” more times than I’d like to admit

• iptables-persistent… isn't

• Testing pod internet? “Just run a curl pod,” they said. Turns out that’s a privilege.

⚙️ Why It Was Worth It (Kind Of)

Once it finally worked, I got:

✅ Distributed, resilient scraper bots.

✅ Easy restarts when a bot dies (read: daily).

✅ Real-time logs from all nodes, from my laptop.

✅ Lens access like a Kubernetes god.

✅ A private API surface—no cloud egress fees, no exposure

If you're building scrapers, you might not need Kubernetes.But if you’re building 20+ custom bot network with different runtimes, memory needs, and failure modes, you kind of do.

Just… budget a few weeks for networking therapy.

— That one dev who thought hosting Kubernetes from WSL2 could be done in a weekend.

Ps. Also added 4 news sources Bankokpost, Techcrunch, Arstechnica, Taipei andTimes.