A lot of local-AI writing assumes you have a 4090. Most people do not. A meaningful share of developers are on a laptop with 4 to 6 GB of VRAM, and the question they actually have is whether any of this works for them.
What works
- Writing unit tests for an existing file. Pattern-heavy, easy to verify, and a 3B handles it.
- Single-file edits with clear instructions — add a parameter, extract a function, handle an error case.
- Explaining code you are reading, if you use a general model rather than a coder.
- Boilerplate: config, scaffolding, repetitive type definitions.
What does not
- Refactors spanning several files. The model loses coherence and you spend longer reviewing than you would writing.
- Anything needing a long context. Small models with a large window still lose the middle.
- Subtle debugging. It will confidently propose the wrong cause.
Where the wall is
The wall is not the parameter count, it is memory pressure. Once weights spill from VRAM into system RAM, tokens per second collapse and the tool stops feeling interactive. This is why Osmium rates each model as fitting, tight, or too big rather than just listing sizes — 'tight' is the configuration that will disappoint you.
The practical advice
On 4 GB, run Qwen2.5 Coder 3B, use it for the well-specified tasks above, and stay in Ask mode when you want explanation. That is a genuinely useful assistant for nothing per month. When you hit the wall, you will know exactly which wall it is — and adding a cloud plan for the hard 10% is then an informed decision rather than a default.