Short answer: most Australian SMBs need RAG, not fine-tuning. RAG (retrieval-augmented generation) lets an AI answer questions from your documents — your policies, product specs, price lists and past jobs — and it can quote its sources. Fine-tuning retrains a model on your examples to change how it writes, not what it knows. If your information changes regularly and you need accurate, citable answers, that’s RAG. Fine-tuning is the right call in a much narrower set of cases, and most of what business owners think they need it for — tone of voice, formatting — is handled by good prompt design instead.
The two terms get thrown around interchangeably by vendors, which makes quotes hard to compare and projects easy to over-spec. This guide explains both in plain English, gives you a decision framework, and lays out the real cost difference.
What RAG actually is, in plain English
RAG is a way of giving a general-purpose AI model (like GPT or Claude) a reference library to check before it answers.
Here’s the flow:
- Your documents — PDFs, policies, manuals, website pages, spreadsheets — are chopped into small chunks and indexed, a bit like a searchable filing system.
- When someone asks a question, the system retrieves the handful of chunks most relevant to that question.
- The AI reads those chunks and writes an answer based on them, usually with a citation pointing back to the source document.
The key point: the model itself never changes. It stays the same off-the-shelf model everyone else uses. Your business knowledge lives in the retrieval layer, completely separate from the model.
That separation is what makes RAG so practical for small businesses:
- Update a document, update the answers. Change your pricing PDF today and the system answers from the new version tomorrow. No retraining, no downtime.
- Answers come with receipts. Because the AI is working from retrieved text, it can show the user (or your staff) exactly which document and paragraph it relied on. That matters a lot for compliance, quotes and anything a customer might dispute.
- It stays in its lane. A well-built RAG system answers from your material or says “I don’t know” — instead of guessing, which is the default behaviour of a raw model.
Our RAG Knowledge Systems service builds exactly this: your documents, indexed and wired into a chatbot or internal assistant, with citations turned on — from $9,500, delivered in 4–6 weeks.
What fine-tuning actually is
Fine-tuning takes a pre-trained model and trains it further on your own examples — typically hundreds to thousands of input/output pairs — so its behaviour shifts.
What fine-tuning is genuinely good at:
- Consistent format. If you need every response structured a very specific way (say, a fixed quote template or a particular JSON shape), fine-tuning bakes that in.
- Specialised style. It can learn a writing voice from examples when prompts alone aren’t getting you there.
- Domain shorthand. Models fine-tuned on, say, medical or legal transcripts get better at the phrasing and structure of that domain.
What fine-tuning is bad at — and this is where most SMB projects go wrong — is facts. Fine-tuning is not a reliable way to teach a model your knowledge. It doesn’t store your documents in a retrievable way, it can’t cite sources, and when your price list changes you have to retrain. A fine-tuned model asked about something it saw once in training data will often produce a confident, wrong, uncorrectable answer.
There’s also an operational reality: fine-tuning means you’re now maintaining a custom model — training runs, evaluation sets, version control, and retraining every time your material changes. That’s a machine learning operations habit most 10-person businesses don’t want and don’t need.
The decision framework
Forget the jargon. Two questions settle it for almost every SMB use case.
Question 1: Does your underlying information change?
If your answers depend on documents that get updated — pricing, policies, stock, procedures, product specs — you need those documents outside the model, where you can edit them. That’s RAG.
If the knowledge is stable and universal (the grammar of a language, the rules of a game, a fixed classification scheme), it can live inside the model.
Question 2: Do you need to know where an answer came from?
If a customer, an auditor, or your own staff will ever ask “says who?”, you need citations. Only RAG gives you that, because only RAG reads from actual documents at answer time.
Run your case through those two questions:
- Changes often + needs citations → RAG. This is most SMB scenarios: customer support bots, internal policy assistants, quoting helpers.
- Stable knowledge + specific tone or format → fine-tuning. Narrower than people assume.
- Both → RAG first, fine-tune later if a genuine format or style problem remains. They can be combined, but RAG is almost always the foundation.
- Neither clearly → you probably just need a well-prompted chatbot, not either technique. More on that below.
Cost and complexity compared
Here’s the honest side-by-side, using our published pricing for the RAG row and industry-typical figures (estimates, not quotes) for the fine-tuning row:
| RAG | Fine-tuning | |
|---|---|---|
| What it changes | What the AI knows | How the AI behaves |
| Typical SMB build cost | From $9,500 (4–6 weeks) with us; industry range roughly $8k–$25k | Roughly $15k–$50k+ once data prep and evaluation are counted |
| Cost when your info changes | Near zero — swap the document | A retraining run, each time |
| Can cite sources | Yes | No |
| Risk of confident wrong answers | Low (answers grounded in retrieved text) | Higher (facts baked into weights, not checkable) |
| Data you need to start | Your existing documents | Hundreds–thousands of curated example pairs |
| Ongoing maintenance | Document hygiene | Model versioning, evals, retraining |
The row that surprises people is “cost when your info changes”. A RAG system treats a policy update as a file upload. A fine-tuned model treats it as a project. Over two or three years, that difference usually dwarfs the build cost.
”But I want it to sound like us”
This is the most common reason SMB owners ask about fine-tuning, and it’s mostly a misunderstanding.
Tone of voice — friendly vs formal, short vs detailed, “no worries” vs “certainly” — is controlled by the system prompt: the standing instructions the model reads before every conversation. A well-written system prompt with a few example responses gets you 90–95% of the way to a consistent brand voice, at zero training cost, and you can tweak it in an afternoon.
Fine-tuning earns its fee for tone only when:
- responses must follow a rigid, complex structure every single time, and
- you’ve already tried prompt engineering with examples and it still drifts.
That’s rare in SMB land. In practice, when we scope a custom AI chatbot build (from $6,500, fixed fee, 3–5 weeks), conversation and tone design is part of the job — no fine-tuning required. Fine-tune for facts is a mistake; fine-tune for tone is usually overkill.
The verdict for Australian SMBs
If you run a trade business, a clinic, a professional services firm, a wholesaler — the pattern is almost always the same. You have a pile of documents that change, a team that answers the same questions repeatedly, and customers who expect correct answers they can trust. That’s a textbook RAG case: index your material, put a chatbot or internal assistant in front of it, keep the citations on.
Fine-tuning becomes relevant later, if at all — when you’ve outgrown prompts for a very specific format, or you’re operating at a volume where shaving tokens off each call matters. That’s a scale problem, and a nice one to have.
Start with the boring, correct foundation. A RAG knowledge system built on your actual documents does more for accuracy and trust than any amount of model training, and it gets cheaper to run every time you update a file instead of every time you retrain a model.
Which option are you?
- Your answers live in documents that change, and people need to trust them? That’s RAG — from $9,500, live in 4–6 weeks.
- You need a customer-facing bot with your tone, channels and handover rules, but nothing exotic? That’s a well-prompted custom chatbot — fixed fee from $6,500, no fine-tuning needed.
- Genuinely unsure whether your case is RAG, fine-tuning, or neither? That’s exactly what our free 20-minute scoping call is for — we’ll tell you which one fits, and if the honest answer is “you don’t need either yet”, you’ll hear that too.