Kiosk

View Tool Evaluations Only

Focus solely on evaluation metrics and benchmark details without any model distractions. Access comprehensive tool performance insights and comparisons easily.

SWE-bench
Software Engineering Agent Benchmark
SOFTWARE ENGINEERING
REAL GITHUB ISSUES
CODE TESTING
Evaluates agents on real-world software engineering tasks using actual GitHub issues. Features full repository context, Docker test environments, and code modification testing. Multiple variants including Lite, Verified, and Multimodal versions.
SWE-bench
ToolSandbox
Stateful Tool Use Evaluation
STATEFUL
INTERACTIVE
TOOL DEPENDENCIES
Interactive evaluation benchmark for LLM tool use with stateful execution, implicit state dependencies, and built-in user simulator. Challenges include State Dependency, Canonicalization, and Insufficient Information scenarios.
Apple
AgentBench
Multi-dimensional Agent Evaluation
MULTI-ENVIRONMENT
DECISION MAKING
REASONING
Comprehensive benchmark with 8 distinct environments assessing LLM reasoning and decision-making as agents. Reveals significant performance gaps between commercial and open-source models, with weaknesses in long-term reasoning and instruction following.
THUDM
τ-Bench (Tau-Bench)
Real-world Agent Reliability Testing
AGENT RELIABILITY
REAL-WORLD SCENARIOS
MULTI-TURN DIALOGUE
Evaluates AI agents in realistic scenarios with dynamic user and tool interaction. Features stateful evaluation and reliability testing with pass^k metric. Even GPT-4o achieves <50% success rate, dropping to ~25% on reliability tests.
Sierra AI
Berkeley Function Calling Leaderboard (BFCL)
The Gold Standard for Tool Calling Evaluation
TOOL CALLING
REAL-WORLD APIS
MULTI-TURN
AST EVALUATION
Comprehensive evaluation of LLM function calling capabilities with 2,000+ test cases across Python, Java, JavaScript, REST APIs, and SQL. Features multi-turn interactions, real-world API execution, and AST evaluation. Currently in V3 with multi-step function calling support.
UC Berkeley