RAG or fine-tuning: which to use so that an AI knows your company

Published on 2026-11-16 by ArduBot

Practical comparison between RAG, fine-tuning, prompts, and APIs when manuals, policies, style, or private information change.

“I want to train ChatGPT with my documents” is a common phrase, but it often describes a search problem, not a training problem. The right decision depends on what changes, what the model needs to remember, and what accuracy the answer requires.

When RAG makes sense

If the information changes frequently — prices, policies, datasheets, manuals — it is usually reasonable to retrieve it at the time of answering. RAG searches for relevant fragments and presents them as context. It allows updating documents without retraining the model and can show source citations.

That does not guarantee error-free results. A search engine that selects the wrong manual or a model that ignores an exception can fail just the same.

When to consider fine-tuning

Fine-tuning can be useful for adapting behavior or format patterns in a repetitive task with enough well-prepared examples, subject to the provider's capabilities and policies. It is not a living document base nor does it replace an updatable source of truth. Introducing a new commercial policy should not require retraining every time.

If the goal is for the AI to respond with a particular JSON format, sometimes instructions, structured output, and validation are enough. It is not necessary to assume that fine-tuning is always needed.

When neither option is ideal

Question: “How many units of product 547 are left in stock right now?”. That transactional data needs to query an authorized inventory. Neither RAG indexed an hour ago nor a model trained months ago can guarantee an up-to-date figure.

Question: “What is the storage policy for hazardous products?”. Here it does make sense to retrieve the current manual, review conditions, and cite the section.

Quick decision matrix

| Need | Initial approach | |---|---| | Manuals and policies that change | RAG | | Current status of orders and stock | API or transactional database | | Rigid response format | Instructions and validation | | Specialized behavior with examples | Evaluate fine-tuning | | Private information from multiple clients | Retrieval with strict authorization |

A real project can combine several options. What matters is identifying which part of the problem each one solves.

What to ask before budgeting

How many sources exist? Do they change daily? Do documents need to be cited? Can clients be mixed? What happens if no information appears? How much does an incorrect answer cost? These questions help more than choosing technology by fashion.

For a proof of concept, use 20 representative questions and compare a static prompt against document retrieval. Measure accuracy, citations, maintenance, and latency, not just text fluency.

Opportunity for an automatic product

The interesting need for ArduMaker is not “automatically training AIs”, but creating and maintaining a knowledge source queryable via API. That could serve bots, SaaS and agencies without forcing them to build an index from scratch. It is an idea under evaluation, not a feature already launched.

If you are deciding between RAG and training for a project, share your need at ArduMaker.

References: RAG Concepts in Microsoft · RAG explained in ArduMaker.

  • RAG
  • fine-tuning
  • LLM
  • IA empresas
  • documentos

Keep reading

All articles · VPS plans