AI-300 Course Notes

An Azure Machine Learning workspace is the project boundary for MLOps assets. It holds or organizes compute, environments, jobs, models, endpoints, data assets, datastores, notebooks, and connections to supporting Azure resources. AI-300 questions often start here because most other choices, such as compute targets or datasets, only make sense after the workspace exists.

• Choosing a compute instance for scalable training jobs when the scenario needs an autoscaling compute cluster.

• Use compute instances for interactive notebooks and compute clusters for repeatable training jobs.

Model lifecycle work begins during development, not at deployment. Azure ML supports shared notebooks, environments, data assets, compute, jobs, and experiment records so teams can collaborate without losing reproducibility. Notebook collaboration is useful for exploration, but production MLOps requires the notebook logic to become tracked code, repeatable jobs, and reviewable pipelines.

• Treating notebooks as the production lifecycle instead of converting important work into tracked jobs and pipelines.

• MLflow tracking is the clue for run comparison, parameters, metrics, artifacts, and lineage.

GenAI Ops starts with a Foundry project and an authenticated client. The Foundry Projects SDK gives developers a programmatic way to connect to project resources and endpoints, while the default credential chain can use developer credentials locally and managed identity in Azure. The exam distinction is that production code should avoid stored keys and use Entra-backed authentication where possible.

• Hard-coding keys in GenAI client code instead of using managed identity or the default credential chain.

• Foundry Projects SDK and endpoints point to programmatic GenAI project operations.

Quality assurance for generative AI relies on repeatable evaluation runs. A useful evaluation run fixes the test set, prompt version, model or agent configuration, retrieval settings, and metric definitions so results can be compared over time. Groundedness checks whether output is supported by supplied context, while quality metrics such as relevance, coherence, fluency, and task success show whether the answer is useful enough for the scenario.

• Using production telemetry as a substitute for controlled evaluation runs.

• Evaluation runs compare prompt, model, retrieval, or agent configurations under controlled tests.

Performance tuning starts with understanding the content pipeline. Azure Content Understanding can analyze documents, images, audio, and video into structured, organized, searchable data. Analyzer tuning matters when the default extraction or schema does not capture the fields, entities, or structure the GenAI workflow needs. Better extraction can improve downstream retrieval before any prompt or model change is attempted.

• Fine-tuning the model when the real issue is poor retrieval, bad analyzer output, or weak chunk selection.

• Content Understanding and analyzer tuning improve the structured data available to downstream retrieval.