What Is RAG, and How Does AI Find Answers in Your Documents?

01 / ARTIFICIAL INTELLIGENCE / DEEP EXPLAINER

Retrieve first, then generate

The original research contrasts knowledge in parameters with explicit updateable memory.RAG does not make a model omniscient; it narrows the evidence available at answering time.

02 / ARTIFICIAL INTELLIGENCE / DEEP EXPLAINER

Documents are split before they can be found

PDFs, pages and manuals are extracted, divided into chunks, embedded and indexed.Chunks too large mix topics; chunks too small sever conditions from exceptions.

One of the earliest and most consequential RAG decisions is not model choice.It is deciding where one meaningful piece of evidence begins and ends.

03 / ARTIFICIAL INTELLIGENCE / DEEP EXPLAINER

Similarity is not always relevance

Vector search finds semantic similarity, not always what a question requires.Names, dates, negations and identifiers may need keyword or structured retrieval.

The answer passage may use different words from the query.Mature systems combine methods, preserve metadata and rerank results instead of accepting the nearest vectors.

04 / ARTIFICIAL INTELLIGENCE / DEEP EXPLAINER

The error may happen before the model speaks

If retrieval selects the wrong policy version, the model can answer faithfully from bad evidence.A citation makes the failure look grounded.

Evaluation must separate retrieval recall from answer faithfulness: first test whether necessary evidence was found, then whether the answer stayed within it.

05 / ARTIFICIAL INTELLIGENCE / DEEP EXPLAINER

Easy updates do not mean easy governance

An external index is easier to update than weights, but duplicates, permissions, deletion, effective dates and version conflicts move into retrieval.A former employee should not recover old confidential material because semantic search finds it.

Access control must filter retrieval before evidence reaches the model.

06 / ARTIFICIAL INTELLIGENCE / DEEP EXPLAINER

The best RAG system can refuse to answer

When passages are insufficient, sources conflict or the question lies outside the collection, a good system asks for clarification or declines.RAG's value is not making AI answer everything.

It restricts where an answer may come from and keeps that route inspectable.

07 / ARTIFICIAL INTELLIGENCE / DEEP EXPLAINER

Evaluation cannot stop at whether the answer looks right

Evaluation needs four layers: was the right document retrieved, ranked where the model could use it, answered faithfully, and cited at the claim level?An answer can look right because the model guessed from internal knowledge while retrieval failed.

The hidden failure appears when the corpus changes.

08 / ARTIFICIAL INTELLIGENCE / DEEP EXPLAINER

Tables, scanned PDFs and Cantonese change the retrieval problem

Enterprise content contains tables, headers, scanned pages, two-column layouts and mixed languages.Broken extraction cannot be repaired by perfect embeddings.

Hong Kong data adds written Chinese, Cantonese, English abbreviations and aliases.Much RAG engineering happens before generation: OCR, cleaning, versioning and chunk boundaries.

09 / ARTIFICIAL INTELLIGENCE / DEEP EXPLAINER

RAG is a data product before it is an AI feature

The visible answer is the last stage of a pipeline whose quality was often decided earlier—during ingestion, extraction, permissions, chunking and versioning.Improving the generator cannot recover a missing page or a broken table.

This is why impressive demos can deteriorate after a real organisation connects years of untidy files.The durable advantage comes from maintaining the evidence layer, not merely swapping in a newer language model.