DP-3028: Implement Generative AI engineering with Azure Databricks
Length: 1 Day(s) Cost:$895 + GST
= Scheduled class
= Guaranteed to run
= Fully booked
Click on the date to book online
Please wait as we are loading the schedules...
| LOCATION | March | April | May | June |
|---|---|---|---|---|
| Auckland | ||||
| Hamilton | ||||
| Christchurch | ||||
| Wellington | ||||
| Virtual Class |
Generative Artificial Intelligence (AI) engineering with Azure Databricks uses the platform's capabilities to explore, fine-tune, evaluate, and integrate advanced language models. By using Apache Spark's scalability and Azure Databricks' collaborative environment, you can design complex AI systems.
This course is best suited to Data Scientists
Before attending this course, students should have familiarity with fundamental AI concepts and Azure Databricks.
After completing this course, students will be able to:
- Describe Large Language Models (LLMs)
- Use LLMs for Natural Language Processing (NLP) tasks
- Set up a RAG workflow
- Improve model accuracy by reranking your search results
- Identify the need for multi-stage reasoning systems
- Implement multi-stage reasoning with libraries like LangChain, LlamaIndex, Haystack, and the DSPy framework
- Understand when to use fine-tuning
- Compare LLM and traditional ML evaluations
- Describe the responsible AI principles for implementation of language models
- Implement key security tooling for language models
Get started with language models in Azure Databricks
- Describe Generative AI.
- Describe Large Language Models (LLMs).
- Identify key components of LLM applications.
- Use LLMs for Natural Language Processing (NLP) tasks.
- Lab: Explore language models
Implement Retrieval Augmented Generation (RAG) with Azure Databricks
- Set up a RAG workflow.
- Prepare your data for RAG.
- Retrieve relevant documents with vector search.
- Improve model accuracy by reranking your search results.
- Lab: Set up RAG
Implement multi-stage reasoning in Azure Databricks
- Identify the need for multi-stage reasoning systems.
- Describe a multi-stage reasoning workflow.
- Implement multi-stage reasoning with libraries like LangChain, LlamaIndex, Haystack, and the DSPy framework.
- Lab: Implement multi-stage reasoning with LangChain
Fine-tune language models with Azure Databricks
- Understand when to use fine-tuning.
- Prepare your data for fine-tuning.
- Fine-tune an Azure OpenAI model.
- Lab: Fine-tune an Azure OpenAI model
Evaluate language models with Azure Databricks
- Compare LLM and traditional ML evaluations.
- Describe the relationship between LLM evaluation and evaluation of entire AI systems.
- Describe generic LLM evaluation metrics like accuracy, perplexity, and toxicity.
- Describe LLM-as-a-judge for evaluation.
- Lab: Evaluate an Azure OpenAI model