arrow_back
Before Starting
Readme
About This Course
Course Curriculum | Pre-requisite
Structured Notes
Download Notes | LLMOps
Lectures
Introduction to LLMOps and its Lifecycle
Tools Stack and MLOps vs LLMOps
Building the RAG Application
RAG Evaluation and its Concepts
What is Golden Dataset and related concepts
Synthesize Golden Dataset using DeepEval
RAG Metrics Part 1
RAG Metrics Part 2
RAG Triad, Custom Metrics(GEval) and Evaluate API
Prompt Management using Langfuse
Prompt Versioning and Prompt Caching in Langfuse
Setting up Parameters Config and Validation using Pydantic
Building the Experiment Tracking Pipeline
Concept of Regression Testing
Building Regression Testing Pipeline and Introduction to Guardrails
Guardrails through Guardrails AI
Adversarial Attacks and Building Custom Guardrails
Implementing Guardrails on RAG Application
Application Serving through Fast API
Implementing CI Pipeline using GitHub Actions
Concepts of Docker for LLMOps
Implementing CD Pipeline and Manual Deployment to EC2
Automated Deployment using AWS Code Deploy
Tracing Application using Langfuse
Implementing Observability and Building Dashboards using Langfuse
Performing Online Evals over Deployed Application
Infrastructure Monitoring using Prometheus
Building Dashboard around Infrastructure Metrics using Grafana
Project | Agentic Application built with LangGraph
LLMOps Agent Project
Prompt Management
Agent Evals Part 1
Agent Evals Part 2
Experiment Tracking Pipeline
Guardrails over Agentic App Part 1
Guardrails over Agentic App Part 2
Serving the Agent App
Building CI Pipeline and Regression Testing
Tracing your Agentic Application
Package your App and Build CD Pipeline
Application Deployment on AWS
Running Online Evals
Monitoring Infrastructure through Prometheus and Grafana
Feedback & Certificate
Feedback Form