NVIDIA GenAI LLM Associate Course Notes

This section establishes the foundational knowledge required to understand how large language models (LLMs) operate and when they are appropriate for various tasks. It covers core concepts like tokenization, attention mechanisms, and the limitations of LLMs, emphasizing the crucial distinction between context and training data.

• **Context vs. Training Data:** Context influences the current request; training data permanently updates the model’s weights. A single prompt does not retrain the model.

• Prioritize understanding the limitations of LLMs, especially regarding context window capacity and the potential for hallucinations.

The exam tests techniques to guide and adapt large language models (LLMs) to achieve desired outputs. Prompting and adaptation are critical for tailoring models to specific tasks and domains, maximizing their utility, and mitigating potential risks. Effective prompting involves crafting clear instructions and examples, while adaptation leverages fine-tuning and parameter-efficient methods to imbue models with specialized knowledge and behavior.

• **Prompt Tuning vs. LoRA:** Prompt tuning adapts a small set of *virtual* tokens, while LoRA adapts a subset of the model's *parameters*. Prompt tuning keeps the base model frozen, while LoRA modifies the model's weights.

• Prioritize techniques that are most efficient and cost-effective for the given scenario. Full fine-tuning is often overkill.

The exam tests Retrieval-Augmented Generation (RAG), a critical technique for grounding LLMs in specific knowledge bases. RAG combines the generative power of LLMs with the ability to retrieve relevant information, improving accuracy, reducing hallucinations, and enabling citations. Understanding RAG is essential for building reliable and trustworthy AI applications, especially in regulated or knowledge-intensive domains.

• Distinguish between dense vector search (semantic similarity) and sparse retrieval (keyword matching). Hybrid approaches combine both.

• Pay close attention to the specific requirements of the scenario – is accuracy paramount, or is speed more important?

The exam tests deploying and serving large language models (LLMs) using NVIDIA technologies. Efficient and scalable inference is critical for real-world applications, and understanding the tools and techniques to optimize performance is essential.

• **NIM vs. Triton:** NIM provides a packaged solution for supported models, while Triton is a more general-purpose inference server.

• Understand the purpose of each NVIDIA technology and when to use them.

The exam tests the critical aspects of deploying generative AI responsibly, covering safety, governance, and ethical considerations. It's not just about building powerful models; it's about ensuring they are used safely, fairly, and in compliance with relevant regulations and organizational policies. Expect scenario-based questions that require you to select the most appropriate controls and processes to mitigate risks associated with generative AI applications.

• **Input vs. Output Guardrails:** Input guardrails prevent *harmful requests*, while output guardrails prevent *harmful responses*. They address different points of vulnerability.

• Prioritize proactive safety measures over reactive ones. Prevention is better than cure.

The exam tests the critical process of evaluating generative AI LLM models and RAG systems, from initial experimentation to production deployment. Effective evaluation ensures models meet performance, quality, and safety requirements, minimizing risks and maximizing user satisfaction. It's not just about accuracy; it's about understanding how a model behaves in real-world scenarios and establishing a repeatable process for ongoing monitoring and improvement.

• **Shadow Deployment vs. Canary Release:** Shadow deployments run *in addition* to the production model, while canary releases *replace* a small portion of production traffic.

• Prioritize questions involving RAG systems – they are frequently tested.