IBM AI Engineering Course Notes

Foundation models are pre-trained AI models capable of generating text, translating languages, and answering questions. These models form the basis for adapting to specific tasks. Prompt Lab provides a hands-on environment to experiment with these models and understand their capabilities.

• Do not treat prompt tuning as full fine-tuning; prompt tuning learns virtual prompt tokens while the base model weights remain unchanged.

• If the question mentions deterministic output, look for greedy decoding or temperature 0.0.

Evaluating prompt templates within IBM watsonx.aie relies on systematic measurement using metrics like BLEU and ROUGE. This process begins with establishing reference answers – high-quality responses used as benchmarks – and mapping prompt variables to corresponding columns within test datasets. Accurate variable mapping is crucial for repeatable and reliable evaluation results.

• Do not score prompt quality without reference answers or a mapped evaluation dataset.

• If the item asks for repeatable prompt evaluation, look for reference answers and variable-to-column mapping.

AutoAI automates the creation and deployment of machine learning models by exploring algorithms and hyperparameters using a specified data source. A key aspect is utilizing a holdout dataset to prevent overfitting and ensure unbiased model evaluation. AutoAI incorporates advanced imputation techniques to refine the data, creating a strong foundation for model training. The resulting model is then deployed from a Deployment Space, leveraging a client repository store for versioning and management.

• Do not train and evaluate on the same data; holdout data is used to detect whether the AutoAI pipeline generalizes.

• If AutoAI must be reviewed or customized, look for saving the generated pipeline as a notebook.

Deployment Spaces in IBM Watson provide a secure and controlled environment for deploying and managing AI models. These spaces are fundamental to the MLOps workflow, enabling efficient execution and monitoring of deployed models. The watsonx.ai Runtime SDK is a crucial tool for interacting with these deployed models programmatically, primarily through Python.

• Do not confuse a project with a deployment space; projects develop assets, while spaces hold assets prepared for runtime deployment.

• If the asset is still in a project, promote or import it into a deployment space before deployment.

Efficient retrieval is fundamental to IBM AI Engineering, enabling AI applications to access and utilize relevant information. The system transforms text into numerical vectors, allowing for quick identification of similar information. Retrieval-Augmented Generation (RAG) combines vector indexes and embedding models to ground AI responses in retrieved context, improving accuracy and relevance, particularly when augmenting the generation process with context from a knowledge base. This approach leverages a vector search strategy, considering data size and query complexity, to deliver the right information at the right time.

• Do not confuse semantic search with keyword matching; vector indexes compare embeddings by meaning.

• If the question mentions grounded answers from documents, look for RAG with a vector index.

Effective governance for IBM Watson AI models begins with establishing a framework – a structured approach encompassing policies, processes, and controls to manage risks and ensure responsible AI development and deployment. This framework proactively identifies and mitigates potential biases within models, utilizing techniques like Bias Mitigation and the Disparate Impact Ratio to guarantee fairness.

• Do not use a single accuracy score as a governance answer when the question asks about fairness, drift, explainability, or lifecycle evidence.

• If the question asks why a model decision happened, look for explainability or SHAP.

Controlling access to IBM Watsonx AI projects is fundamental for ensuring data security and responsible AI development. Project roles, specifically the distinction between Admin and Editor, dictate the level of control a user has over assets and configurations. Understanding these roles is crucial for operational effectiveness and preventing unauthorized modifications.

• Do not assume project visibility grants credential use; connection assets can be visible while credentials remain restricted.

• If the question involves creating deployment spaces or managing access, check whether Admin-level permission is required.

Combined Governance in IBM AI Engineering establishes controls and processes to manage risks and ensure the responsible use of AI solutions. This approach centers on leveraging technologies like watsonx.governance to monitor and evaluate prompt templates throughout their lifecycle, specifically tracking prompt templates and assessing their quality against defined guidelines. Prompt-template evaluation is a core component, utilizing metrics and thresholds to guarantee desired AI behavior.

• Do not use a single accuracy score as a governance answer when the question asks about fairness, drift, explainability, or lifecycle evidence.

• If the question asks why a model decision happened, look for explainability or SHAP.

The IBM-AIE-04 SDK provides a Python interface for interacting with the watsonx.ai Runtime, enabling developers to build applications that leverage AI models. The core component is the APIClient, which handles authentication and formatting requests for services like `generate_text_stream`. This abstraction simplifies development and deployment, allowing focus on application logic. Developers utilize the SDK to generate text, targeting the appropriate environment using either a Project ID or Space ID.

• Do not call generation APIs without setting the correct projectId or spaceId context.

• If the code streams model output, look for generate_text_stream.

AI engineers improve machine learning model deployments within the IBM watsonx platform by directing tensor computations to devices like GPUs. The `.to(device)` method facilitates this movement, accelerating computations by assigning input tensors to the GPU before initiating model inference. This approach is crucial for leveraging the parallel processing capabilities of GPUs to accelerate machine learning workloads.

• Do not place only the model on the GPU; input tensors must be moved to the same device.

• If PyTorch inference is slow, check device placement for both the model and tensors.

Big data projects often leverage Apache Spark for processing large datasets. Spark SQL provides a way to manipulate data using SQL-like syntax, while the Tokenizer component breaks down unstructured text into individual tokens, preparing it for machine learning models. ML Pipelines orchestrate the entire process, from data transformation to model training and evaluation, ensuring data consistency and reproducibility. These tools are crucial for building scalable and maintainable AI solutions.

• Do not fit preprocessing on the full dataset before train/test separation; that leaks information from evaluation data.

• If the prompt names concat or concat_ws, think Spark SQL string construction.

Retrieval-Augmented Generation (RAG) leverages vector indexes to enhance response quality. The process begins by dividing documents into manageable chunks, prioritizing structural boundaries like double newlines and paragraph breaks. This creates a vector index, enabling efficient similarity searches based on user queries. The system then uses similarity metrics, such as cosine similarity or Euclidean distance, to compare query and document vectors, determining relevance.

• Do not treat chunking, embeddings, and reranking as interchangeable; each changes a different part of retrieval quality.

• If retrieval accuracy is weak, check chunking, embedding model choice, similarity metric, and reranking.