Building a CV Analyzer That Doesn't Hallucinate

When we started building Cvaluate, the obvious approach was to hand a CV to a large language model and ask for feedback. The first prototypes were impressive — fluent paragraphs, confident tone, suggestions that sounded like a career coach. They were also wrong. The model would praise a skill you never listed, rewrite a bullet with a metric you never claimed, or criticise a gap that did not exist. That is preventing LLM hallucination in practice: not a research curiosity, but a product requirement for anyone trusting AI with their job search.

Why hallucination is worse on CVs than in chat

In a general chatbot, a fabricated detail is annoying. On a CV analyser, it is actively harmful. A candidate might paste an invented bullet into an application. A false weakness might send them rewriting a section that was already strong. Job seekers are often anxious and time-poor — they will act on specific advice if it looks authoritative.

The failure mode is subtle. Models do not usually invent entire CVs from scratch. They embellish: turning "managed projects" into "managed 12 projects", adding "stakeholder management" because the role sounds senior, or suggesting you quantify a result you only described qualitatively. Each invention is small; together they erode trust.

Extraction first, evaluation second

Our architecture follows a pattern we describe more fully in why we use a two-step AI pipeline: extract, then evaluate. Step one turns the uploaded document into structured data — roles, dates, bullets, skills, education. Step two scores and rewrites against that structure, not against the model's memory of what CVs usually look like.

Separating the steps matters because evaluation models are optimised for judgement and language. Left alone, they will fill gaps. By forcing them to work from a fixed extraction artefact, we narrow the input surface and make it easier to verify that every output line traces back to source text.

JSON schema as a contract

Unstructured prose is impossible to validate automatically. We require the evaluation step to return structured output via JSON schema — scores as numbers, weaknesses as arrays, rewrites paired with original bullet identifiers. If the response is not valid JSON, or if a required field is missing, we reject it and retry with a tighter prompt.

Schema enforcement gives us three practical wins:

• Type safety — a score cannot silently become "quite good"; it must be a number in range.
• Completeness checks — we know if the model skipped the requirements matrix or strengths list.
• Grounding hooks — rewrite objects include a reference to the source bullet hash or index, so we can verify the original text exists before showing the suggestion.

This is the same principle behind reliable API integrations everywhere: define the shape of acceptable output, then fail fast when the model drifts.

Grounding every claim in extracted text

Grounding LLM output means the model may only assert facts present in the extraction layer. Prompt instructions spell this out, but we also validate programmatically. If a weakness mentions "no cloud experience" but the extracted skills include AWS, we flag the response. If a rewrite quotes a job title that does not appear in the work history block, we discard it.

Grounding is especially important for job-description matching. Without it, a model might claim you lack a keyword that appears on page two of your CV because it skimmed the summary. Our requirements matrix ties each row to evidence spans in the extracted text — matched or missing, not guessed.

What grounded rewrites look like

Here is the difference between generic AI feedback and a grounded rewrite. Generic output might say:

Weak: "Add more metrics to your project management bullet."

A grounded rewrite preserves your facts and improves phrasing:

Before: Coordinated delivery across three workstreams for a platform migration.

After: Coordinated delivery across three workstreams for a platform migration, completing cutover two weeks ahead of schedule.

The "after" version only adds specificity the model can justify from context you provided — or it flags that you should supply a metric rather than inventing one. That discipline is central to how we designed prompts for useful feedback.

Validation, retries, and graceful degradation

Even with schemas and grounding, models occasionally produce edge-case failures — malformed JSON, out-of-range scores, or a rewrite that references the wrong section. Our pipeline treats those as transport errors, not user-facing results. We retry with lower temperature and stricter instructions. If validation still fails after a small number of attempts, we return partial results with a clear notice rather than silently shipping bad data.

Reliability engineering for LLMs looks more like data pipelines than like traditional unit tests. You monitor rejection rates, log failure categories, and tighten prompts when a new CV layout pattern breaks extraction. In the CVs we analyse, parsing quality upstream is often the root cause of downstream hallucination pressure — which is why PDF extraction is its own engineering battle.

What we deliberately do not do

We do not ask the model to guess missing dates. We do not fabricate achievements to make a CV "stronger". We do not present confidence scores that imply statistical precision we do not have. Those choices sometimes make feedback feel less flashy than a generic chatbot — good. Job seekers need accurate mirrors, not fiction.

For a broader view of how AI fits into hiring technology — and where it still falls short — read our guide to AI in hiring. When you want to see the pipeline on your own document, run your CV through Cvaluate's free analysis and check whether every rewrite points back to text you actually wrote.