Evaluating an enterprise chatbot
A claim of "92% accuracy" means almost nothing unless you state which question set it was measured on and what counted as correct.
“Should we use RAG or fine-tune” comes up in nearly every generative AI conversation. The framing assumes the two are substitutes; in practice they address different problems.
1. How RAG works {#rag}
RAG has two steps. Retrieval finds the passages most relevant to the question. Generation places those passages in context and asks the model to answer strictly from them.
The most important strength is not accuracy but traceability. Every answer can point at the document and page it came from. For enterprises, that is usually a precondition for going live at all.
2. What fine-tuning solves {#finetune}
Fine-tuning teaches a behaviour: answer style, output format, handling of a specialised request type. It is an inefficient way to load knowledge, since every document change means retraining.
3. Decision framework {#decision}
Start with RAG for anything involving internal knowledge. Only consider fine-tuning with measured evidence that RAG is insufficient, and where the gap is behavioural rather than informational.
Across BigAI deployments, roughly 90% of internal assistant use cases were solved end to end by RAG combined with improving the source documents themselves.