If you think you have an interesting eval, please open a pull request with your contribution. OpenAI staff actively review these evals when considering improvements to upcoming models.

Sep 5, 2025 · Eval Recipes provides a benchmarking harness for evaluating AI agents on real-world tasks in isolated Docker containers. We have a few sample tasks ranging from creating …

Adding new rows to the table below with eval results, given new models and new system prompts. This repository is NOT intended as a replacement for https://github.com/openai/evals, which is …

With Evals, we aim to make it as simple as possible to build an eval while writing as little code as possible. To get started, we recommend that you follow these steps in order:

EvalPlus is a rigorous evaluation framework for LLM4Code, with: HumanEval+: 80x more tests than the original HumanEval! MBPP+: 35x more tests than the original MBPP! EvalPerf: …

Redefine agentic evaluation in enterprise AI. By simulating realistic scenarios, these evals enables rigorous, multi-dimensional assessment of LLM-powered productivity agents. - …

Python library for Evaluation. Contribute to instructlab/eval development by creating an account on GitHub.

Overview Eval is an open source platform designed to revolutionize the way companies assess technical candidates. By leveraging real-world open source issues, the platform provides a …

VoQA Benchmark is a comprehensive benchmark for Visual-only Question Answering (VoQA) that provides a unified evaluation framework for both open-source and closed-source models. …

Framework for evaluating Large Language Models (LLMs) trained on code, based on the HumanEval benchmark. Supports automated testing and performance analysis. - …