28.9M-parameter LLM runs locally on ESP32-S3 at 9 tokens/s - CNX Software

Slava S. (slvDev) has optimized a 28.9M-parameter LLM running locally on an ESP32-S3 development board at around 9 tokens/s while generating text, or more

Raccontata dacnx-software.com