ESP32 microcontroller LLM runs TinyStories offline in reported demo
Developer SlvDev reportedly fit a 28.9 million-parameter model on an $8 to $10 ESP32-S3, but its use is limited to simple text generation.
By Renata Fuchs · Policy Reporter
· 3 min read
Developer SlvDev has reportedly run a 28.9-million-parameter language model offline on an ESP32-S3 board costing roughly $8 to $10. The ESP32 microcontroller LLM demonstration is a notable exercise in model-memory engineering, not evidence that low-cost embedded hardware can host a useful general chatbot, coding tool or agent.
The model is TinyStories, a small model originally developed by Microsoft Research to generate short stories. Better Stack said in a July 31 video description that it reproduced the training and flashing process from a bare board and tested prompts. The Register reported that SlvDev achieved 9.88 tokens per second, a developer-reported result rather than an independently published benchmark.
The hardware details vary slightly by board configuration and reporting. Better Stack described an $8 ESP32-S3 with 512 KB of RAM. The Register described an ESP32-S3 setup with 520 KB of SRAM, 8 MB of PSRAM and flash-storage options reaching 16 MB. Neither report provides a full independent code audit, model artifact review or benchmark methodology.
How did an ESP32 microcontroller LLM fit on the chip?
At 16-bit precision, TinyStories would need about 60 MB to hold its weights, according to The Register. That is far beyond the memory available on the board. SlvDev reportedly first used quantization, reducing the precision of model weights and cutting the weight footprint by 75% to 14.9 MB. The trade-off is lower numerical precision and potentially lower model quality.
The second step was per-layer embeddings, a technique The Register described as borrowed from Google’s Gemma model family. Rather than retaining all weights in working memory, the implementation reportedly placed about 25 million parameters, or roughly 12 MB, in flash storage. The output head, embeddings and KV cache remained in PSRAM, while activations used the chip’s 520 KB of SRAM. The result was an active-memory requirement of about 2 MB, The Register reported.
That division is the point of the demonstration. Flash is much slower than SRAM or DRAM, so moving weights there can hurt inference speed. Per-layer placement limits that penalty when the stored weights are not accessed continuously. The reported throughput suggests the approach can produce text at a workable pace for a narrow model, though the available material does not establish reproducible performance across boards or prompts.
What can the TinyStories model do in practice?
Based on the reporting, it can generate short, reasonably coherent stories. That is a constrained task and a long way from running a capable assistant. The Register said the model was not suitable on this evidence for general chat, code generation or agent workloads.
For operators building local AI features, the practical signal is that compression and memory placement can push inference into far smaller devices than conventional deployments. The boundary remains model capability: devices with more memory and compute, including Raspberry Pi systems and smartphones, can accommodate more useful local models. This ESP32-S3 result shows what can be made to fit, not a deployment template for broad language-model workloads.
This story draws on original reporting from The Register.
