Draft
TL;DR
llmfit is a Rust TUI/CLI that inspects your hardware
(RAM/CPU/GPU/VRAM) and ranks LLMs by whether they’ll realistically run
well—saving you from downloading huge models only to discover they’re
unusable.
What this is about
Local inference is attractive (cost control, privacy, latency), but
it’s easy to misjudge whether a model will run on your machine. llmfit
tries to turn that guesswork into a quick, actionable recommendation:
what’s “perfect,” “good,” “marginal,” or “won’t run.”
Key points
- Detects hardware and scores models across fit, speed,
quality, and context length. - Offers both an interactive TUI and a
scripting-friendly CLI mode. - Integrates with popular local runtimes like Ollama,
llama.cpp (GGUF), and MLX (Apple
Silicon). - Includes a “Plan Mode” that flips the question:
what hardware would you need to run a specific model?
Why it matters
For anyone building agentic systems on local models, the biggest
friction is often not “which model is best,” but “which model is
feasible.” A tool like llmfit can shorten iteration loops and prevent
wasted downloads and debugging time.
Practical takeaways
- If you run local LLMs regularly, treat model selection like
dependency management: automate it. - Use the CLI mode for reproducible setups (CI, provisioning scripts,
workstation bootstrap). - Plan Mode is useful for hardware planning: it turns “should we
upgrade?” into concrete numbers.
Caveats / what to watch
- Scoring systems are only as good as the underlying model database
and heuristics. - Real-world speed/quality can vary by quantization, backend, and
prompt style.