134 AI Essentials interview questions with answers
134 AI Essentials interview questions with detailed answers, sorted from basic to advanced. Free to practice, with progress saved automatically.
All AI Essentials interview questions
What is a Large Language Model (LLM) and how does it work?
What are the components of the Transformer architecture?
What is tokenization? How does BPE (Byte Pair Encoding) work?
How do self-attention and Query/Key/Value work?
What is the context window and why does it matter?
What are Temperature, Top-p, and Top-k in LLMs?
What is the KV Cache and why does it speed up inference?
How do zero-shot, one-shot, and few-shot prompting differ?
What is Chain-of-Thought (CoT) prompting and when to use it?
How does ReAct (Reasoning + Acting) prompting work?
What is Prompt Injection? How to defend against it?
How do you get reliable structured (JSON) output from LLMs in production?
What is prompt chaining? How does it differ from agents and when to use it?
What is LLM jailbreaking? Common techniques and defenses?
What is RAG (Retrieval-Augmented Generation) and why does it matter?
What are the components of a full RAG pipeline?
What are chunking strategies in RAG? How to choose chunk size?
What is hybrid search and why is it better than pure vector search?
What is re-ranking in RAG? When should you use it?
RAG hallucinates even with correct context. How do you debug and fix it?
How do Agentic RAG and Graph RAG differ from standard RAG?
Query transformation in RAG: HyDE, query decomposition, step-back prompting?
How do you evaluate RAG end-to-end? How do RAGAS metrics work?
What is an AI Agent? How does it differ from a simple LLM call?
How does tool/function calling work? How to design good tools?
What is MCP (Model Context Protocol)?
Agent memory: how do short-term and long-term differ?
Your agent loops infinitely or burns too many tokens. How do you fix it?
Plan-and-Execute vs ReAct agent pattern. When to use which?
What is a multi-agent system? Common orchestration patterns?
Human-in-the-loop (HITL) for AI agents: when is it needed and how to design it?
What is fine-tuning? When should you fine-tune instead of RAG/prompting?
How does LoRA (Low-Rank Adaptation) work and why is it effective?
How does QLoRA differ from LoRA? How does it enable fine-tuning 65B on a single 48GB GPU?
What is RLHF? How does the LLM alignment pipeline work?
What is catastrophic forgetting in fine-tuning? How to prevent it?
Preparing a dataset for LLM fine-tuning: format, size, quality?
What are embeddings? How do models convert text to vectors?
How do vector DBs differ from traditional DBs? How do they do fast similarity search?
Metadata filtering in vector DBs: pre-filter vs post-filter vs hybrid?
Embedding drift when switching embedding models. How to handle migration?
How does LLMOps differ from traditional MLOps?
What is quantization in LLMs? How do INT8, INT4, FP16 differ?
LLM costs are too high. Strategies to cut production costs without losing quality?
What are LLM guardrails? How to implement input/output guardrails?
Streaming LLM responses: how to implement and benefits?
Prompt versioning, A/B testing, rollback in production LLM?
What is LLM-as-a-judge? Pros and cons?
How do you detect and measure hallucination in LLM outputs?
What is LLM red teaming? Process and testing frameworks?
What is multi-modal AI? How do CLIP and vision-language models work?
How do you handle PII in LLM input/output (GDPR, data privacy)?
Design an LLM-powered customer support chatbot. What's the architecture?
Design an AI gateway (LLM proxy) managing multi-provider across the org?
How to pick a GPU for LLM inference? Memory, bandwidth, compute.
Continuous batching vs static batching. Why does it boost LLM throughput 10–20x?
What is speculative decoding? Why does it cut inference latency 2–3x?
Tensor vs Pipeline vs Data parallelism: when to use which?
Monitoring LLM inference in production: TTFT, ITL, GPU util, key metrics?
Self-hosted LLM vs API-based inference: when, cost, and challenges?
Implement a basic RAG pipeline (embedding + vector search + generation).
Implement retry with exponential backoff for LLM API calls (rate limit, timeout).
Implement semantic caching for LLM queries (cache responses for similar queries).
Implement prompt injection detection for user input (multi-layer).
AI Engineer vs ML Engineer: differences, skill set, when each is needed?
Boss asks "what's the ROI of this AI feature?" — how do you answer and measure?
Explain LLM limitations to non-technical stakeholders ("why isn't AI 100% accurate?")
Self-consistency and Tree-of-Thought: when to use for better reasoning?
Multi-modal RAG: handling PDFs with tables, images, charts in the knowledge base?
How does DPO (Direct Preference Optimization) work? Why is it replacing RLHF?
AI bias: types, detection, and mitigation?
Speech-to-Text (Whisper) and Text-to-Speech: architecture and use cases?
Document understanding: extract structured data from complex-layout PDFs?
Major benchmark suites: MMLU, HumanEval, GSM8K — what they measure and limitations?
Implement an AI agent with tool use (calculator + web search) from scratch?
How do you debug a poorly performing RAG system (wrong answers, retrieval miss)?
Design an AI code review system at scale for a 1000+ engineer org?
What makes a good prompt? Principles for writing effective prompts?
How do system / user / assistant message roles differ? What is a chat template?
How do base models differ from instruct/chat models?
When is few-shot prompting enough, and when should you fine-tune? How do you pick few-shot examples?
How is LLM API cost billed per token? Why does conversation cost grow each turn?
What criteria pick an embedding model for RAG?
What is perplexity? What does it measure and what are its limits?
RAG vs a long-context window: when to choose which?
What is knowledge distillation? Why does it yield a small yet strong model?
How do LangChain, LangGraph, and LlamaIndex differ? When to pick which?
How do you evaluate an LLM application? Offline vs online evaluation, golden dataset?
What are reasoning models (o-series, extended thinking)? What is test-time compute?
Synthetic data for fine-tuning/evaluating LLMs: how to generate it and what are the risks?
A data flywheel for LLM apps: how do you collect user feedback to improve over time?
Which metrics measure retrieval quality? How do Recall@k, MRR, and NDCG differ?
How does constrained decoding (structured output) work at the token level? How is it different from just prompting for JSON?
How does indirect prompt injection work via RAG/tool results? How do you design defense-in-depth for a production app?
How does evaluating an AI agent differ from evaluating a single LLM call? How do you measure trajectory?
Multi-tenant RAG: how do you ensure each user only retrieves documents they are allowed to see?
Keeping a RAG knowledge base fresh: how do you manage incremental indexing and deletion?
Handling Vietnamese in an LLM/RAG pipeline: what to watch for in tokenizers, embeddings, and Unicode normalization?
Choosing an LLM provider/model for a product serving Vietnamese users: what factors matter?
Self-hosting open models (Qwen, Llama, Vietnamese-adapted models like PhoGPT/Vistral) vs calling commercial APIs: when is it worth it?
Designing a customer-support chatbot on Zalo OA / Facebook Messenger with an LLM: webhook flow, sessions, and human handoff?
How does prompt caching work? What are the cache-hit conditions and how do you structure prompts to exploit it?
How should you chunk a long document that contains headings, tables and code?
Chunks lose context once separated from their source document. How does contextual retrieval fix this?
Small chunks retrieve precisely but lack context; large chunks are the opposite. How does small-to-big retrieval solve this?
Hybrid search with BM25 + vectors: how do you merge the two rankings, and why not just add the scores?
How do bi-encoders and cross-encoders differ, and why is a reranker not used for the search step itself?
How many chunks should go into the prompt? Does raising top-k to 50 improve answers?
What metadata should you store in the vector DB at ingest time? Why is text plus vector not enough?
Choosing an embedding model for your domain: are public leaderboards enough? How do you measure it yourself?
What do you do when the model calls a tool with bad arguments or the tool throws? How do you design the error feedback loop?
How do you force the model to call a tool instead of answering directly? What are parallel tool calls?
An agent calls tools with side effects (create order, send email, refund). How do you design so retries do not duplicate the effect?
A conversation outgrows the context window. How do sliding windows and progressive summarization differ, and which do you choose?
The model's JSON gets truncated or comes wrapped in extra prose. How do you handle that in production?
Users upload documents into your RAG system, and a document may hide instructions targeting the model. What do you do at ingest time?
What biases affect LLM-as-judge scoring, and how do you calibrate so the numbers mean something?
Your LLM provider fails or rate-limits at peak hours. How do you design fallback without a surprise drop in quality?
To stream an LLM answer to the browser, do you pick SSE or WebSocket?
In a Next.js app, where do you put the LLM call? Why not call it straight from a client component?
While waiting for the model, what should the UI show so the app does not feel frozen?
When should you NOT use an LLM and write ordinary code instead?
The user hits Stop mid-answer. How do you cancel so billing actually stops?
The provider errors or drops after half the answer has streamed. How do you design timeouts and retries?
Design a fallback to another model/provider when the primary fails. What are the traps?
Many users hit the feature at once and the provider returns 429. How do you design queueing and rate limiting?
How do you measure and cap token spend per user? Where do you enforce it?
How do you store conversation history so the UI can replay it fully while you still control what goes to the model?
How do you stop abuse of an AI feature: spam, deliberate token burning, or people using your app as a free LLM?
Customer data contains personal information but the product calls a foreign provider API. How do you handle it before sending?
Changing a prompt requires a redeploy, so rollback is slow. How do you design prompts as versioned configuration?
A/B testing two prompt versions on real users: how do you split, what do you measure, and how long until you can conclude?
Measuring AI feature quality in production: what do you log, how do you sample, and how do you turn real traffic into an eval set?
The AI feature runs on serverless and long answers exceed the function time limit. How do you handle it?
