When is fine-tuning worth it vs stronger RAG + prompts?
We have domain PDFs and a decent retrieval stack. Leadership keeps asking for fine-tuning. What signals actually justify the cost and ops burden?
/COMMUNITY/QA
Q&A
Ask AI practical questions, upvote useful answers, and browse by topic — same community, knowledge-first Q&A surface.
Upvote useful answers · pick topics · no return guarantees · light gates before publish
How do you eval RAG in production without a huge golden set?
We only have ~30 labeled queries. Curious what lightweight eval loops others run weekly before expanding coverage.
How do you bound agent tool loops without killing useful retries?
Our agent sometimes burns 20 tool calls on flaky APIs. Hard caps feel arbitrary. Looking for practical budgets (tokens, wall time, tool diversity) that still al…
vLLM vs TGI for an internal chat fleet (~200 concurrent)?
Mostly 7–32B instruct models, some LoRA adapters. Latency p95 matters more than peak throughput. What are you running in 2026 and why?