Agent architecture · Digital Orukami Running
Fox (PUMBA)
Since I started messing around with local models, the ideas have not stopped. The
joke is that the breakthrough usually lands around 12:30 AM, and the digital caffeine
high that follows keeps me up until 3 AM with class at 9:30. Fox (PUMBA) is the
project that grew out of that: it started as a tiny chat agent I ran off a cheap GPU
from a local garage sale, proof to myself that a local model did not need to feed the
corporate expansion of AI consumerism to be useful. It grew from there, more data,
then the entire ECHO landing page and club history, then a second GPU so it could
toggle between two brains instead of one. This past weekend the v2 build of Fox got
upgraded, which is why I want to give the project some context before getting into the
new hardware for v3.
Fox is a local inference AI agent I design and build as the ECHO Club's guide,
running fully on self hosted hardware so nothing depends on a cloud API. The core is a
dependency light Python relay that serves a local Ollama model over a loopback
endpoint with a strict honesty doctrine: when no model is reachable it says so plainly
and never mocks a response. It offers a two brain toggle, a fast 3B model for quick
answers and a deeper 8B for harder ones, retrieval over a curated club knowledge base
via tool calling, an append only message log, and a custom HUD style comms dashboard
with a compose stage, live status, and a transcript record. The agent has an identity
of its own; in the hero screenshot it chooses its own nickname, Fox. The current work
is a voice capability: not off the shelf text to speech but a composed, expressive
voice language built from a small alphabet of sounds, tuned on an inflection bench and
streamed live. The project spans agent architecture, local model deployment,
retrieval, and voice synthesis.
This past weekend the rig behind Fox took a real step forward. The target is a
workstation the team nicknamed unitzero, built around a Threadripper TRX50 platform
running Linux, aiming for 72 GB of pooled VRAM across three Intel Arc Pro B60 GPUs.
Two of the three cards are in and already pooling memory across PCIe to run a 32
billion parameter Qwen3 model, quantized, that will not fit on either card alone: a
single card load fails and the two card pooled load succeeds. On this prototype path,
llama.cpp's Vulkan backend, generation runs around 10 to 11 tokens per second, with a
faster Ollama compatible serving stack planned but not yet installed. Fox's relay only
speaks Ollama's API shape, so rather than rewrite Fox I built a small translator that
converts the rig's API shape into the one Fox already expects, and I proved it through
the live dashboard itself: a real message sent through Fox's UI got a real reply back
from the new rig. The honest, charming part is that when I asked Fox about its new
hardware, it said it did not know, because its own persona files still describe the old
hardware and only the model backend had changed, the honesty doctrine working exactly
as intended. The third card, a faster production grade serving stack, and tensor
parallel validation across all three cards are next. This is real and working, not
finished.
Local inferencePythonOllamaRAGVoice