Provider Comparator via Concentrate API
Free6 GitHub stars
Agent ToolAgnosticFile System
Overview
This CLI tool allows users to compare two LLM providers using the same prompts and a third LLM as a judge. It generates an HTML leaderboard report, making it ideal for developers and researchers evaluating AI models.