Master LLM evaluation and testing: LLM-as-a-judge, golden datasets, RAG and agent evals, CI/CD regression testing and production monitoring. 2026 course.
The course’s skills. The same ones appear on your Certificate of Completion, so you can show what you covered.
A premium, complete and advanced course on evaluating and testing LLM-powered systems, updated for 2026. You will learn why evals are the hardest and most important discipline in AI engineering, the full evaluation taxonomy (offline vs online, reference-based vs reference-free), why classic metrics such as BLEU, ROUGE and exact match fail on open-ended text, and how embedding-based metrics help and mislead. You will master LLM-as-a-judge (rubric design, position/verbosity/self-preference bias, calibration against humans), building golden datasets with edge cases and slices, and the real 2026 toolchain: promptfoo, LangSmith, Langfuse, Braintrust, DeepEval, Ragas and pytest. The course goes deep on evaluating RAG (retrieval and generation), evaluating agents (task completion and trajectory), safety and guardrail evals, red-teaming, regression testing prompts in CI/CD, online A/B testing, quality-drift monitoring, human annotation and the real cost of evaluation. Includes runnable code, production patterns, a strong focus on data privacy and GDPR for eval data, and a comprehensive final assessment. Educational content, not legal advice.
For developers, engineers and technical professionals who build or integrate AI solutions. It belongs to AI Engineering, on the IT Pro track.
Advanced level: assumes hands-on experience with AI in the course’s field.
10 modules · 30 lessons · about 25 hours · 370 quiz questions
Modules and lessons in order, plus the link to the preview of lesson 1 — so you can decide at your own pace.
You read the lesson, ask the AI Professor right on the page and check what you remember.
Why LLM Evaluation Is Hard — and Non-Negotiable
Most engineering teams can ship a web service they can reason about: the same input yields the same output, a failing assertion is unambiguous, and a green test suite means the behavior is locked in. LLM-powered systems break every one of those assumptions.
Written lessons, with examples and exercises. Your progress is saved automatically, so you can pick up where you left off.
Select a passage and it explains it, gives you an example or tells you why it matters. You can type or talk to it, in English.
Every lesson has its own quiz, with explanations for the answers: 370 questions across the course. The professor also summarizes the lesson and prepares review flashcards.
Once you complete the lessons and reach the minimum quiz score, you issue it from your account, with a unique number, publicly verifiable.
The AI Professor is included in your subscription, per-course access included, with a monthly fair-use limit. It is an AI system, not a human teacher, and its answers may contain errors; details in the Terms. See how the AI Professor works.
The Certificate of Completion for “LLM Evaluation and Testing: Shipping Reliable AI”: a private attestation with a unique number and a QR code. A recruiter confirms it is authentic on the public verification page, with no account and no cost.
Public verification page, QR code and cryptographic fingerprint, with no account for the person checking.
Add it to your CV and LinkedIn profile; a recruiter confirms authenticity with one click.
The course, the tier you earned (Pass, Merit or Distinction), the serial number and the issue date, plus the percentage of lessons completed.
A private attestation of course completion. It is NOT a diploma or a state-recognised qualification under Romanian Laws 198/2023, 199/2023 or Government Ordinance 129/2000; employers may take it into account at their own discretion. Certification policy.
Billed monthly, in EUR. You can stop the renewal anytime from your account, and access starts right after payment. Prices are totals, with VAT included at the rate of your country.
LLM Evaluation and Testing: Shipping Reliable AI
about €4 per hour of course
Developers, data engineers, architects. Code and production.
€16 per month for each of the 25 courses
The whole catalog, both tracks side by side.
€10 per month for each of the 50 courses
Subscriptions renew automatically; you can stop the renewal anytime from your account, and access stays until the end of the period already paid for. As digital content with immediate access, see the withdrawal conditions.
About the course, the price, the AI Professor and the certificate.
A premium, complete and advanced course on evaluating and testing LLM-powered systems, updated for 2026. You will learn why evals are the hardest and most important discipline in AI engineering, the full evaluation taxonomy (offline vs online, reference-based vs reference-free), why classic metrics such as BLEU, ROUGE and exact match fail on open-ended text, and how embedding-based metrics help and mislead. You will master LLM-as-a-judge (rubric design, position/verbosity/self-preference bias, calibration against humans), building golden datasets with edge cases and slices, and the real 2026 toolchain: promptfoo, LangSmith, Langfuse, Braintrust, DeepEval, Ragas and pytest. The course goes deep on evaluating RAG (retrieval and generation), evaluating agents (task completion and trajectory), safety and guardrail evals, red-teaming, regression testing prompts in CI/CD, online A/B testing, quality-drift monitoring, human annotation and the real cost of evaluation. Includes runnable code, production patterns, a strong focus on data privacy and GDPR for eval data, and a comprehensive final assessment. Educational content, not legal advice.
The course has 10 modules and 30 lessons, with 370 quiz questions. The full curriculum, module by module, is shown on the course page.
Yes. The course is Advanced level and assumes hands-on experience with AI in its field.
The estimated duration is about 25 hours, across 30 lessons. You learn at your own pace, and your progress is saved automatically at every lesson, so you can pick up where you left off.
Access to this course alone costs €99 per month. The course is also included in the IT Pro bundle, €399 per month for all 25 IT Pro courses, and in All Access, €499 per month for the whole international catalog.
Plans are billed monthly, in EUR. Every price is a total, with VAT included at the rate of your country. Compare the plans.
Yes. You can read a free preview of lesson 1, “Why LLM Evaluation Is Hard — and Non-Negotiable”, without an account. The preview shows you how the lesson is written; the rest of the lesson and the other lessons open after payment.
The AI Professor is built into every lesson and starts from the lesson you have open: it explains the passages you select, gives you examples, summarizes the lesson, tests you with a quiz and prepares review flashcards. You can type or talk to it, in English.
It is included in your subscription, per-course access included, with a monthly fair-use limit. It is an AI system, not a human teacher, and its answers may contain errors; details in the Terms.
When you meet the completion thresholds shown on the platform (lessons completed and the minimum quiz score), you can issue your Certificate of Completion for “LLM Evaluation and Testing: Shipping Reliable AI” from your account, with a unique number, publicly verifiable on the site.
It is a private attestation of course completion, issued by GALIAN SOFTWARE DEVELOPMENT SRL. It is NOT a diploma or a state-recognised qualification under Romanian Laws 198/2023, 199/2023 or Government Ordinance 129/2000; employers may take it into account at their own discretion. Details in the Certification policy.
Yes. Course access and the bundles are subscriptions that renew automatically at the end of each monthly billing period. You can see the next renewal date in your account at any time.
You can stop the renewal at any time, from your account or by email, with no other formality. Access stays active until the end of the period already paid for, and no pro-rata refund is made for a period already started, except in the cases provided by law. Your progress stays saved in your account. Details in the Terms & Conditions, section 5.5.
Yes. Company plans give a team access to the courses, with a dashboard where you follow each member’s progress and manage seats centrally. See the plans for companies.
All of them are included in the IT Pro bundle, at about €16 per course, or you can take them separately, at €99 a month each.
IT Pro: AI Engineering
Build MCP servers in Python and TypeScript: protocol architecture, OAuth 2.1, security, Docker and Kubernetes, CI/CD to production. Hands-on course for 2026.
IT Pro: AI Engineering
Engineer context and memory for AI agents: context anatomy, memory layers, retrieval, compaction, multi-agent systems and cost optimization. For 2026.
€99/monthVAT included
Buy the course Context Engineering and Memory for AI Agents: Beyond PromptingIT Pro: AI Engineering
Master RAG end to end: embeddings, chunking, vector databases, hybrid search, re-ranking, Agentic RAG and production scaling. Hands-on course for 2026.
€99/monthVAT included
Buy the course RAG: Retrieval-Augmented Generation in Practice“LLM Evaluation and Testing: Shipping Reliable AI”: 30 lessons and about 25 hours of content, with your progress saved at every lesson.
Just this course
€99/month
VAT included · stop the renewal anytime from your account
Buy the course Or all 25 IT Pro courses: €399/monthChatGPT, OpenAI, Claude, Anthropic, Gemini, Google, Microsoft 365 Copilot, Cursor, Midjourney and the other product and company names mentioned are trademarks or trade names of their respective owners. Cursuri-AI.ro is an independent training provider and is not affiliated with, sponsored by or endorsed by any of them.