ai-evals
Projects tagged with ai-evals on GitHub.
283 projects
skillspec
RustSkillSpec makes agent skills followable, testable, and provable with Doctor risk reports, guided imports, structured contracts, and alignment proof.
waku-agent
PythonWaku Waku! Waku Agent is a local-first AI agent harness you actually own, including loop, memory, eval, all in code built to stay legible as it grows.
bindwidth
JavaScriptEvidence-aware on-prem LLM inference sizing and TCO calculator
agent-apprenticeship
PythonThe living ecosystem where AI agents complete tasks through workflow loops, improve through iterative execution, are evaluated by mentor agents or humans in the loop, and turn completed work into reusable work experience and data to improve future agents.
agent-apprenticeship
PythonThe living ecosystem where AI agents complete tasks through workflow loops, improve through iterative execution, are evaluated by mentor agents or humans in the loop, and turn completed work into reusable work experience and data to improve future agents.
DeepSpec
PythonDeepSpec: a full-stack codebase for training and evaluating speculative decoding algorithms
forsy-trace-skill
PythonOpen skill for capturing AI agent work as structured traces.
llm-space
TypeScriptA desktop app to prototype agent ideas, inspect every harness step, replay failures, and evaluate performance, all in one place. Local-first, cloud-ready for managed agents.
PerceptionBench
PythonPerceptionBench: Evaluating Atomic Visual Perception in Multimodal Large Language Models
canivibecodeit
JavaScriptFind out which subscriptions are one prompt away from free. caniuse.com, but for replacing SaaS with one AI prompt.
maka
TypeScriptApache Maka (Incubating) is a local-first AI agent workspace. Model messages, tool calls, tool results, permission decisions, and termination events are recorded as an append-only log.
better-harness
JavaScriptAn open-source Harness Engineering platform for coding agents—define harnesses as code, run controlled experiments, inspect evidence, and compare outcomes.
FrontierAgent
Python🧩 FrontierAgent, our agent framework, open-sourced alongside it — native command-line TUI, ReAct and Agent Team modes, one command on macOS and Linux, no preinstall, no hard Docker dependency.
RealReplicaBench
HTMLCommerceAgentBench: Benchmarking Long-Horizon Agents in High-Fidelity, Stateful, and Reproducible Replicas of Real Online Services
system-prompts-and-models-of-ai-tools
FULL Augment Code, Claude Code, Cluely, CodeBuddy, Comet, Cursor, Devin AI, Junie, Kiro, Leap.new, Lovable, Manus, NotionAI, Orchids.app, Perplexity, Poke, Qoder, Replit, Same.dev, Trae, Traycer AI, VSCode Agent, Warp.dev, Windsurf, Xcode, Z.ai Code, Dia & v0. (And other Open Sourced) System Prompts, Internal Tools & AI Models
generative-ai-for-beginners
Jupyter Notebook21 Lessons, Get Started Building with Generative AI
ai-copywriter
PythonAn AI copywriter that uses real copywriting skills + real marketing knowledge with human tone.
ai-data-extractor
PythonFree open-source extractor for AI coding assistant chat histories. Supports Claude Code, Cursor, Windsurf, Aider, Cline/Roo Code, and more.
ai-trains-ai
PythonRL-training an AI agent to RL-train AI agents.
ai-sdk
GoGrafana AI SDK for Go — streaming, tool-calling AI backends that speak fluent @ai-sdk/react
AI-Engineering-Lab
Jupyter NotebookA free, self-paced 24-week AI engineering course: Python, machine learning, LLMs, RAG, fine-tuning, agents and MCP, Azure and Vertex and Bedrock, and Databricks. 43 runnable notebooks, one continuous case study. MIT licensed, no signup. By Zorost Intelligence AI Lab.
openclaw
TypeScriptYour own personal AI assistant. Any OS. Any Platform. The lobster way. 🦞
n8n
TypeScriptFair-code workflow automation platform with native AI capabilities. Combine visual building with custom code, self-host or cloud, 400+ integrations.
JavaGuide
JavaScriptJava 面试 & 后端通用面试指南,覆盖计算机基础、数据库、分布式、高并发、系统设计与 AI 应用开发
dify
TypeScriptBuild Agentic workflows, RAG pipelines, with rich AI model and tool support on one collaborative workspace. Deploy on cloud, VPC, or self-hosted, so teams move from prototype to production without rebuilding the stack.
langflow
PythonLangflow is a powerful tool for building and deploying AI-powered agents and workflows.
open-webui
PythonUser-friendly AI Interface (Supports Ollama, OpenAI API, ...)
awesome-llm-apps
Python100+ AI Agents, Agent Skills and RAG Apps - Free and Open Source.
MoneyPrinterTurbo
Python利用 AI 大模型和自动化工作流,根据主题或关键词一键生成高清短视频。Generate HD short videos from a topic or keyword with an automated AI workflow.
browser-use
Python🌐 Make websites accessible for AI agents. Automate tasks online with ease.
supabase
TypeScriptThe Postgres development platform. Supabase gives you a dedicated Postgres database to build your web, mobile, and AI applications.
gemini-cli
TypeScriptAn open-source AI agent that brings the power of Gemini directly into your terminal.
pi
TypeScriptAI agent toolkit: unified LLM API, agent loop, TUI, coding agent CLI
claude-mem
JavaScriptPersistent Context Across Sessions for Every Agent – Captures everything your agent does during sessions, compresses it with AI, and injects relevant context back into future sessions. Works with Claude Code, OpenClaw, Codex, Gemini, Hermes, Copilot, OpenCode + More
worldmonitor
TypeScriptReal-time global intelligence dashboard. AI-powered news aggregation, geopolitical monitoring, and infrastructure tracking in a unified situational awareness interface
LocalAI
GoLocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.
deepseek-reasonix
GoDeepSeek-native AI coding agent for your terminal. Engineered around prefix-cache stability — leave it running.
ego-lite
JavaScriptThe fastest browser for AI agents to run browser automation, built for sharing your logged-in browser state with your AI agents, like Codex or Claude Code, without disturbing you. Zero cost, zero config.
PixelRAG
Pythonhttps://arxiv.org/abs/2606.28344. The end of web parsing. The beginning of scalable pixel-native search. link: https://pixelrag.ai/
SparkyFitness
TypeScriptSparkyFitness: Built for Families. Powered by AI. Track food, fitness, water, and health — together.
genoffice
TypeScriptFree, open-source AI office suite for macOS, Windows & Linux — Word (.docx), Excel (.xlsx), PowerPoint (.pptx), PDF and Markdown editing with built-in AI agents.
openbot
TypeScriptOpen-source AI coworkers that each get a computer of their own: a browser, files and tools, with every action decided before it happens and recorded after. Bring any AG-UI agent.
open-science
TypeScriptOpen Science is an open-source, local-first, model-agnostic AI research workbench for reproducible scientific research on macOS, Windows, and Linux.
unlazy
JavaScriptAnti-laziness skill for AI agents. Core: the Depth Tree method, which splits a task N layers deep and gives every leaf the full time budget of the whole task, so effort multiplies with depth. Grounded in 2025-2026 research on model laziness, underthinking and premature completion.
penecho
JavaScriptThink with AI beyond the chat box. A shared canvas for handwriting, equations, diagrams, and spatial reasoning.
OptMem
PythonPermanent memory for AI agents. A 426-token prompt, a script, plug and play.
OpenChatCut
TypeScriptOpen-source, local-first conversational AI video editor with a professional multi-track timeline, Agent Skills, MCP integration, and Remotion rendering.
numbat
GoVisibility into AI agent activity on endpoints, with on-device detection, optional pre-action blocking, and forensic reconstruction.