evaluate

Multi-tier framework for evaluating AI agent skills with quality gates, semantic overlap detection, synthetic evaluation dataset generation, and live agent evaluation that measures how skills affect agent behavior.

Comprehensive NLP Evaluation System

Easiest way of fine-tuning HuggingFace video classification models

한국어 STT 출력의 CER, WER, CRR, 키워드·개체명·코퍼스 평가를 제공하는 Python 패키지

Safe and fast evaluation of untrusted user-supplied python expressions

한국어 벤치마크 평가 코드 통합본(?)

Transformers training in a supercomputer with the 🤗 Stack and Slurm

aigc evals

[SG2RL@ICCV 2023] Code for our paper "Haystack: A Panoptic Scene Graph Dataset to Evaluate Rare Predicate Classes"

PromptPotter: AI optimization engine for automated prompt engineering (APE). Searches and tunes prompts and parameters against your data using a generate-evaluate-critique cycle. Improve AI outputs, automate prompt optimization, and fine-tune your pipeline.

Python implementation of Shunting-yard Algorithm to evaluate logical tag expressions

Local-first fraud OS: JSON evaluate packs, graph hops, receipts, and an analyst desk on Compose. Source-available Elastic License 2.0 (not OSS).

Occupational Bias in Open-Source Pretrained Large Language Models: Analyzing Polarity towards Creative and Technical Professions

Parse infix boolean expressions to RPN, evaluate and generate truth tables.

Question Answering Datasets evaluation scripts and leaderboards

"EvArgs" is a Python module designed for value assignment, easy expression parsing, and type casting. It validates values based on defined rules and offers flexible configuration along with custom validation methods.

(Meta-) evaluation tools for the TREC AutoJudge framework.

Clean crawling of websites to keep relevant infos only

Generate infinite images for most model's training or validation

Distributed, extensible framework for testing, routing, and validating LLM-based agents.