NVIDIA GenAI LLM Associate Course Notes

Large language models generate text by turning input text into tokens, representing those tokens as vectors, and predicting likely next tokens from the surrounding context. The exam expects more than a product name: understand that generation is probabilistic, context-dependent, and limited by the information available in the prompt, retrieved content, or learned parameters.

• Do not treat a context window as permanent model knowledge; it is only the working token span available for the current request.

• If the question asks how an LLM creates text, look for autoregressive next-token generation from prompt context.

Prompt engineering is the practice of shaping model behavior through instructions, context, examples, constraints, and output format requirements. A strong prompt tells the model the task, the audience, the evidence to use, and the response shape. Vague prompts usually produce vague answers because the model must infer missing requirements.

• Do not confuse prompt formatting with factual grounding; a clean JSON response can still be unsupported.

• If the goal is consistent structure, choose output format instructions or schema guidance.

Retrieval-augmented generation separates knowledge lookup from language generation. The retriever finds relevant source passages, and the LLM uses those passages to produce an answer. Choose RAG when answers depend on current, private, or large document collections that should not be memorized into model weights.

• Do not skip document parsing and chunking quality; weak chunks produce weak retrieval even with a strong generator.

• If the prompt depends on enterprise documents, choose a RAG pipeline with ingestion, embeddings, vector search, reranking, and grounded generation.

Inference serving is the production side of an LLM system: accepting requests, running the model, and returning outputs within latency, throughput, cost, and reliability targets. The model is only one part of serving; batching, caching, memory, hardware, and API behavior determine whether the system works under real traffic.

• Do not describe GPU acceleration as the whole deployment answer; serving quality also depends on batching, cache behavior, APIs, and validation.

• If the item names prefill, decode, KV cache, batching, or quantization, it is testing inference optimization rather than prompt design.

Responsible LLM design starts by identifying what can go wrong: unsafe instructions, toxic output, privacy leakage, biased behavior, unsupported claims, intellectual property exposure, and misuse. Safety is not a final checkbox; it has to be designed into prompts, retrieval, model behavior, deployment controls, and monitoring.

• Do not rely on one safety filter for every risk; input controls, output controls, retrieval controls, monitoring, and human review solve different failures.

• If risky input appears before model execution, choose input guardrails; if risky generated text appears after execution, choose output guardrails.

Evaluation begins with a dataset that represents the real tasks the system must handle. Include ordinary cases, difficult edge cases, refusal cases, retrieval-heavy cases, and examples that previously failed. A small polished demo set is not enough because it hides regressions and overstates readiness.

• Do not use a general benchmark as proof that a domain-specific assistant is ready for production.

• If the question asks whether a change broke known behavior, choose regression testing.