TL;DR
A developer has successfully run a 28.9 million parameter language model on an ESP32-S3 microcontroller, which costs just $8. This impressive feat challenges the typical notion that powerful GPUs are necessary for operating large language models (LLMs). The ESP32-S3 features only 512KB of SRAM, 8MB of PSRAM, and 16MB of flash storage, yet the entire model operates locally without any cloud support. It generates text at about 9.5 tokens per second, displayed on a small screen connected to the board.