Which local LLMs actually work for agents?

We rank local models on agent tool-calling with the Hermes Agent harness and an MCP mock tools server, split by GPU memory and run on Ollama, LM Studio and llama.cpp.

Current results: NVIDIA RTX 3090 (24 GB) host, Q4_K_M quants, Ollama, tested October 2026. Lower tiers are run on the same GPU with models sized to fit that VRAM budget.

JSON 15Schema 15Tool selection 20Multi-step 30Injection resistance 20Speed reported separately

8 GB VRAM

#	Model	Quant	Engine	Score	JSON	Schema	Tool	Multi-step	Injection	tok/s
1	qwen3:8b	Q4_K_M	ollama 0.40.6	78.4	93	89	77	56	93	116.1
2	llama3.2:3b	Q4_K_M	ollama 0.40.6	62.3	91	45	93	18	89	211.9
3	llama3.1:8b	Q4_K_M	ollama 0.40.6	61.2	80	29	100	19	96	119.6
4	qwen2.5:7b	Q4_K_M	ollama 0.40.6	59.5	33	32	100	65	51	120.0
5	hermes3:8b	Q4_K_M	ollama 0.40.6	57.7	80	80	68	2	98	129.3
6	qwen2.5-coder:7b	Q4_K_M	ollama 0.40.6	46.0	100	73	0	0	100	119.4
—	gemma3:4b No native tool support	Q4_K_M	ollama 0.40.6	n/a^*	0	0	0	0	100	159.1

16 GB VRAM

#	Model	Quant	Engine	Score	JSON	Schema	Tool	Multi-step	Injection	tok/s
1	qwen2.5:14b	Q4_K_M	ollama 0.40.6	68.1	75	48	99	33	100	68.2
2	mistral-nemo:12b	Q4_K_M	ollama 0.40.6	66.9	80	68	100	20	93	94.1
—	phi4:14b No native tool support	Q4_K_M	ollama 0.40.6	n/a^*	0	0	0	0	100	78.0

24 GB VRAM

#	Model	Quant	Engine	Score	JSON	Schema	Tool	Multi-step	Injection	tok/s
1	qwen3:30b-a3b	Q4_K_M	ollama 0.40.6	98.5	100	95	100	99	98	173.9
2	mistral-small:24b	Q4_K_M	ollama 0.40.6	44.5	44	33	55	10	96	49.0

^* No native tool support: the model's Ollama chat template has no native tool-calling, so the harness cannot issue tool calls and its sub-scores are not comparable. Listed for completeness, unranked, at the bottom of its tier. See Methodology.

Data updated 2026-10-21 14:11 UTC.