Classifying Text Locally
Calling an LLM API is the default answer to “classify this text”. This series asks when that default is wrong. Each post runs the same kind of benchmark on public datasets and answers one practical question:
- Do you need an LLM at all, or does a small local model do the job?
- How small can that model get before accuracy breaks?
- With no labels, can an LLM teacher label the data for a small student?
- Do a few human labels on top of the teacher’s labels help?
Read them in order, or jump to the question you have.
- 1 Nov 2026 Do You Need an LLM for That? Classifying Text Locally
Three tasks, four approaches, one laptop: when prompting an LLM beats a small local classifier, and when it does not. - 16 Nov 2026 How Small Can a Text Classifier Be?
Fine-tuned encoders and decision models from 17M to 4B parameters: what each costs on a laptop, with labels and without. - 21 Nov 2026 No Labels? Borrow Them: Distilling a Classifier Locally
Rent an LLM teacher for a few cents to label your data, keep a 150M student. How much of the teacher survives, and what does it cost?