Generative AI and LLM Quiz

Question 1

Why did transformers replace RNNs for most NLP tasks?

Accepted Answer

Self-attention lets every token weigh every other token in one parallel step, instead of pushing information down a recurrent chain, so long-range dependencies survive and training parallelises. The cost is attention quadratic in sequence length.

Question 2

What are the tradeoffs in RAG chunk size?

Accepted Answer

Small chunks retrieve precisely but strip the context that qualifies a fact; large ones keep context but dilute the embedding and spend tokens. Start near a few hundred tokens with some overlap, split on document structure, then tune against an eval set.

Question 3

Why is a bigger context window not always the answer?

Accepted Answer

Tokens cost money and latency, and accuracy sags for material buried mid-prompt. Retrieving the 2k tokens that matter usually beats pasting 200k. Long windows are for genuinely long documents; prompt caching makes a stable prefix cheaper, not free.

Question 4

What is the KV cache, and why does it cap concurrency?

Accepted Answer

Attention keys and values for the prefix, cached per layer, so each new token attends over saved state instead of recomputing the sequence. The cache grows with batch size, sequence length and layers, so GPU memory rather than compute limits throughput.

Question 5

Prompt, system prompt, tools: what does each do?

Accepted Answer

The system prompt sets durable role, rules and output format; the user prompt carries the request; tool definitions are schemas the model can call so your code fetches data or acts. The model never runs a tool itself, your application does.

Question 6

When do you use RAG instead of fine-tuning?

Accepted Answer

RAG when the model lacks knowledge: facts that change, are private, or need a citation. Fine-tuning when it lacks behaviour: a format, a tone, a narrow task. An index is cheaper to update than weights, so reach for RAG first.

Question 7

What is an embedding?

Accepted Answer

A fixed-length vector for text, an image or audio, placed so similar meanings land near each other, which turns search into a nearest-neighbour lookup. Vectors only compare within one model, and closeness is not relevance, hence rerankers.

Question 8

How does BERT differ from a GPT-style model?

Accepted Answer

BERT is an encoder: bidirectional, trained by masking tokens, so every position sees both sides, which suits classification and embeddings but cannot generate. GPT is a causal decoder trained to predict the next token, which is what enables generation.

Question 9

How do you evaluate a RAG system?

Accepted Answer

Retrieval and generation separately. Retrieval: recall@k and nDCG on labelled question-document pairs. Generation: faithfulness (every claim supported by the context), relevance and citations. Most bad answers are retrieval failures.

Question 10

What actually reduces hallucinations?

Accepted Answer

Grounding, not scolding: retrieval with citations, permission to answer that it does not know, tool calls for facts and arithmetic, and a verification pass. The model optimises for the likeliest next token, not for truth, so instructions alone will not.

About the generative ai and llm quiz

More quizzes

Generative AI and LLM Quiz

Keyboard shortcuts

About the generative ai and llm quiz

More quizzes