LLM interview questions, organised by what they test
LLM interview questions are best prepared by theme rather than by volume. Nine themes cover almost all of them: model fundamentals, inference and serving, retrieval, agents and tool use, evaluation, reliability and safety, cost and latency, production debugging, and project judgment. Interviewers rarely score the first answer — they score the follow-ups, which is why memorised responses collapse and reasoning from a real system holds up.
This is a map of what gets asked and why, not a question bank. Each theme links onward to the deeper guide where one exists.
Why question banks stop working after the first follow-up
A memorised answer is a single move in a conversation that lasts five. An interviewer asks how you would evaluate a summarisation feature, you produce a fluent paragraph about eval sets, and then they ask what your eval set contained, how you labelled it, how many examples, who disagreed, and what the metric did when you changed the prompt. The script runs out immediately.
This is not a trick. It is how experienced interviewers distinguish someone who has read about a practice from someone who has run it. The reliable defence is to anchor every theme below to one system you actually worked on, so the follow-ups have somewhere real to go.
It also means volume is the wrong target. Being able to recognise three hundred questions is worth less than being able to go four levels deep on nine themes.
The nine themes
For each theme: what interviewers are probing, and the follow-up that usually separates candidates.
- 1. Model fundamentals
- Probing: whether you understand what a model can and cannot be relied on to do — context limits, tokenisation effects, determinism, and why a temperature change is not a quality strategy. The separating follow-up is usually "why does that happen?" rather than "what is it called?".
- 2. Inference and serving
- Probing: token accounting, prompt and prefix caching, batching, streaming, context window management, and hosted versus self-hosted trade-offs. The separating follow-up is what actually dominates your p95 today.
- 3. Retrieval and RAG
- Probing: ingestion, chunking, embeddings, hybrid search, reranking, and context construction. The separating follow-up is what you do when the top results are irrelevant, and how you detect that in production.
- 4. Agents and tool use
- Probing: state, tool interface design, orchestration, termination, idempotency and recovery. The separating follow-up is how the agent stops when it is going wrong.
- 5. Evaluation
- Probing: golden sets, task metrics, judge models used carefully, human review, offline against online. The separating follow-up is a regression your evaluation actually caught, and what it cost to build.
- 6. Reliability and safety
- Probing: grounding, refusal behaviour, guardrails, prompt injection through retrieved or tool-supplied content, and the false-positive cost of every filter you add.
- 7. Cost and latency
- Probing: where money and time are spent, and which lever you pull first. The separating follow-up is whether you measured before optimising.
- 8. Production debugging
- Probing: instrumentation, tracing a single bad output back to its cause, and distinguishing a retrieval failure from a prompt failure from a model change.
- 9. Project judgment
- Probing: what you chose not to build, when you decided a model was the wrong tool, and what you would do differently. This theme appears in every round, not only the behavioral one.
What to ask your recruiter
Before you prepare anything, send one short email: how many rounds, what each one is called, how long each runs, whether any of them is a take-home, and whether an AI assistant is allowed in the coding round. Recruiters answer this routinely. Anything they confirm outranks every pattern on this page, because it is first-party evidence about your loop rather than a general tendency across companies.
Which themes appear in which round
A rough allocation. Use it to decide where to spend preparation once your recruiter has confirmed the round names.
| Round | Dominant themes | Usually absent |
|---|---|---|
| LLM / AI system design | Serving, retrieval, evaluation, reliability, cost. | Model internals, trivia about recent releases. |
| Evaluation round | Evaluation, reliability, production debugging. | Architecture internals. |
| Practical coding | Production debugging, tool use, sometimes retrieval. | Cost modelling, safety policy. |
| Project deep dive | Project judgment, debugging, evaluation, cost. | Nothing, in practice — deep dives roam. |
| Behavioral | Project judgment. | Everything technical, though examples should still be concrete. |
A structure that survives follow-ups
Use it for any theme. It signals experience because it is the order in which the work actually happens.
- State the decision you are being asked to make, in one sentence, so you and the interviewer are solving the same problem.
- Name the constraint that decides it: latency, cost, privacy, data availability, or risk.
- Give your answer and the trade-off you accepted, explicitly. An answer with no cost attached reads as untested.
- Say how you would know you were wrong — the metric, the trace, or the eval case.
- Ground it in one real instance from your own work, including a number if you have one and a clear "I do not remember" if you do not.
Study now, review, skip
- Themes 3, 5 and 8: retrieval, evaluation, production debugging. These carry the most weight in most loops.
- One real system you can use as the anchor for every theme.
- Theme 7 with real numbers from that system.
- Themes 2 and 4: serving and agents, unless the role centres on them, in which case they move up.
- Theme 6, especially injection through retrieved content.
- Memorising architecture internals beyond what you need to reason about behaviour.
- Long question lists. Depth on nine themes beats recognition of three hundred prompts.
- Anything claiming to be a specific company's exact questions.
Common failure modes
- Fluent first answers with nothing behind them, which the second follow-up exposes.
- Naming tools and frameworks instead of describing decisions and trade-offs.
- Using a public benchmark as a stand-in for a task-specific evaluation.
- Claiming numbers from memory that do not survive arithmetic.
- Answering a judgment question with a technique, when the interviewer asked what you would decide.
- Never saying "I do not know", which reads as less trustworthy rather than more competent.
Common questions
- Is there a list of the most common LLM interview questions?
- Lists exist, but they age badly and none of them is specific to your loop. The themes above are stable; the phrasings change constantly. Preparing the reasoning behind a theme transfers to any phrasing of it.
- Do interviewers ask about transformer architecture?
- In product engineering loops it is uncommon, and usually only at the level of what the architecture implies for behaviour. Research and inference-infrastructure roles are different, and the job description normally makes that obvious.
- How technical do the answers need to be?
- Specific rather than technical. A concrete account of one system, with the constraint that decided each choice, consistently outperforms a general explanation of best practice.
- What if I am asked something I have never done?
- Say that, then reason out loud from the nearest thing you have done. Interviewers score the reasoning. Improvised experience is detected quickly and costs far more than the admission would have.
Related interview guides
- RAG interview questions: retrieval, evals and failure modes
Ingestion through reranking to context construction, centred on production trade-offs.
- LLM evaluation interview: evals, judges and regressions
Golden sets, task metrics, judge models, online monitoring, and agent evaluation.
- LLM system design interview, decoded
The LLM-specific cut: retrieval architecture, prompting and orchestration, token budgets, failure handling.
- LLM engineer interview: rounds and prep
Serving, retrieval, agents, evaluation, and the latency and cost judgment this title is screened on.
How every ranking on this site is reasoned about, including what counts as evidence: read the methodology.
Map your actual interview.
This page is the general pattern. Paste the job you are actually interviewing for and the wording your recruiter used, and the map ranks the rounds you are likely to face, explains why each is ranked where it is, and tells you what to skip.
See a complete Full Interview Map example