Interview Bank
Agent and LLM engineering questions, with real answers.
How do you choose a chunking strategy for RAG?
Chunk on semantic boundaries rather than fixed sizes, keep enough context per chunk to stand alone, and overlap to avoid splitting answers.
How do you choose which model to use for a task?
Test candidates on your own evaluation set, weigh cost and latency against measured success rate, and re-check when models change.
How do you detect an agent failing silently?
Verify the end state independently of the agent's claim, since a confident report of success is not evidence of success.
How do you evaluate an agent?
Define task level success criteria you can check programmatically, build a fixed test set, and measure end state rather than trajectory.
How do you handle failures in an agent system?
Distinguish tool failures from reasoning failures, feed errors back as context, cap retries, and design for human recovery.
How do you reduce agent latency?
Cut the number of sequential model calls, stream output, parallelise independent tool calls, and cache the prefix.
How do you stop a prompt change from breaking something else?
Version prompts in git, keep a fixed regression set, run it on every change, and check the cases you were not trying to fix.
How does function calling actually work under the hood?
The model emits structured text matching a schema you provided; your code executes it and returns the result as a new message.
How would you design a permission system for an agent?
Default deny for consequential actions, scope credentials narrowly, make prompts meaningful, and isolate rather than rely on pattern matching.
How would you reduce the cost of running an agent?
Shrink context, cache the stable prefix, route easy work to cheaper models, and cut the number of loop iterations.
Walk through the agent loop step by step
Prompt, model decides, tool executes, result returns, repeat until done, with a stopping condition and a step limit.
What are the pitfalls of using an LLM as a judge?
Position bias, verbosity bias, self preference, and poor calibration, all of which require validating the judge against human labels.
What happens when a task needs more context than the window allows?
You compress, retrieve selectively, decompose into subtasks, or isolate work in subagents that return summaries.
What happens when an agent has too many tools?
Selection accuracy falls, context cost rises on every request, and similar tools get confused with each other.
What is a context window and why does it constrain agent design?
The fixed token budget a model attends to in one request, holding everything the agent currently knows.
What is an AI agent, and how does it differ from a chatbot?
An agent runs a model in a loop with tool access and acts on the results; a chatbot returns text for a human to act on.
What is MCP and what problem does it solve?
An open protocol standardising how agents connect to external tools, turning an N times M integration problem into N plus M.
What is prompt caching and how do you design for it?
Providers cache an unchanged context prefix and charge less for it, so you keep stable content at the front and volatile content at the end.
What is prompt injection and why can it not simply be fixed?
Untrusted content entering context is indistinguishable from instructions, because both are just tokens with no separate control channel.
What is RAG and when do you need it?
Retrieving relevant documents and putting them in context before generating, used when the model needs knowledge it was not trained on.
What is the difference between context and memory?
Context is what the model sees in this request; memory is external storage plus a retrieval strategy for putting things back into it.
What is the difference between MCP and plain function calling?
Function calling is the model capability; MCP is a protocol standardising how tool providers and agents connect so tools are portable.
What is the ReAct pattern?
Interleaving reasoning and acting, so the model thinks before each tool call and observes the result before the next one.
What is the risk of installing a skill or MCP server from a stranger?
Both are executable in effect, one as instructions the agent follows and one as code running with your privileges.
What makes a tool definition good or bad for an agent?
A clear name, a description stating when to use it, few parameters, and error messages that tell the model how to recover.
When is a multi agent system better than a single agent?
Rarely. It helps for genuine context isolation or parallel independent subtasks, and hurts through coordination overhead everywhere else.
When should you not use an agent?
When the task is deterministic, when errors are costly and unverifiable, or when a single model call would do.
When would you fine tune instead of using RAG?
Fine tune to change behaviour, format, or style; retrieve to supply knowledge. Most problems people bring to fine tuning are context problems.
Why combine keyword search with vector search?
Embeddings capture meaning but miss exact terms; keyword search nails identifiers and rare words but misses paraphrase. Together they cover each other.
Why does agent quality degrade over a long session?
The window fills and gets compressed, attention spreads unevenly over long contexts, and early instructions lose weight against recent noise.