Microsoft.Extensions.AI.Evaluation - Samples

Microsoft.Extensions.AI.Evaluation is a set of .NET libraries (defined in the following NuGet packages) that provide the tooling necessary to evaluate the quality and efficacy of LLM responses in your intelligent applications.

Microsoft.Extensions.AI.Evaluation: Defines core abstractions and types for supporting evaluation.
Microsoft.Extensions.AI.Evaluation.Quality: Contains evaluators that can be used to evaluate the quality of AI responses in your projects including Relevance, Truth, Completeness, Fluency, Coherence, Retrieval, Equivalence and Groundedness.
Microsoft.Extensions.AI.Evaluation.Safety: Contains a set of evaluators that are built atop the Azure AI Foundry Evaluation service that can be used to evaluate the content safety of AI responses in your projects including Protected Material, Groundedness Pro, Ungrounded Attributes, Hate and Unfairness, Self Harm, Violence, Sexual, Code Vulnerability and Indirect Attack.
Microsoft.Extensions.AI.Evaluation.Reporting: Contains support for caching LLM responses, storing the results of evaluations and generating reports from that data.
Microsoft.Extensions.AI.Evaluation.Reporting.Azure: Supports the Microsoft.Extensions.AI.Evaluation.Reporting library with an implementation for caching LLM responses and storing the evaluation results in an Azure Storage container.
Microsoft.Extensions.AI.Evaluation.Console: A command line dotnet tool for generating reports and managing evaluation data.

The Microsoft.Extensions.AI.Evaluation libraries are built on top of core AI abstractions defined in the Microsoft.Extensions.AI libraries. Samples for Microsoft.Extensions.AI are available here.

Samples

Name	Description
API Usage Examples	Examples that demonstrate the various APIs that are included in the above NuGet packages.

Learn More

Check out the following .NET Live talk on YouTube:

Also check out the following blog posts:

Documentation for the libraries is available on Microsoft Learn.

Feedback & Contributing

We welcome your feedback and contributions! Head to the dotnet/extensions repo to provide feedback or contribute to the project. Existing issues for the Microsoft.Extensions.AI.Evaluation libraries can be viewed here. We also welcome feedback and contributions for the samples in the current repository.