NVIDIA/SkillEvaluator
v0.9.0Curated ai tools / ai-evals / testing repository (NVIDIA/SkillEvaluator) tracked via conditional GitHub .atom polling.
SkillEvaluator is a multi-tier evaluation framework and CI/CD quality gate from NVIDIA for testing AI agent skills and plugins before production rollout. It solves the challenge of silent agent regressions by running security scans, semantic overlap detection, synthetic dataset generation, and live agent benchmarks to measure real behavioral impact.
Build and review better evaluation datasets. Declare deter. Drop-in upgrade. Migration: None.
Key Release Highlights (v0.9.0)
- Build and review better evaluation datasets. Declare deterministic checks for script execution, workspace files, and final responses. Generate datasets grounded in skill ins...
- Track skill improvements over time. Compare retained skill versions with
--compare-toandcompare-versions, browse run history, and reuse a compatible no-skill baseline ... - Control evaluation spending. Set token, judge-call, attempt, and priced-cost budgets. Reports retain partial results when limits stop a run. Token usage is shown per case an...
- Understand the evidence behind each result. Reports show met, unmet, unchecked, and inapplicable expectations; dataset coverage and sufficiency; skill activation; failed-arm...
Verified safe drop-in upgrade for skillevaluator in pyproject.toml.
npm install skillevaluator@0.9.0Ecosystem Compatibility Matrix & Verified Graph Warnings
Zero-Noise Historical Release Timeline (1)
Format: [Core Change] + [Required Action] + [Migration Effort]Build and review better evaluation datasets. Declare deter. Drop-in upgrade. Migration: None.
Upgrade & Migration Q&A (NVIDIA/SkillEvaluator v0.9.0)
Direct Answers for Engineers & AI AgentsIs NVIDIA/SkillEvaluator v0.9.0 safe to upgrade in production?
NVIDIA/SkillEvaluator v0.9.0 is classified as COMPATIBLE (MINOR release). Build and review better evaluation datasets. Declare deter. Drop-in upgrade. Migration: None.
What breaking changes or compatibility risks exist in NVIDIA/SkillEvaluator v0.9.0?
No breaking API changes or peer dependency conflicts are recorded for NVIDIA/SkillEvaluator v0.9.0 in the Reposen compatibility graph.
What is the bundle size impact of skillevaluator v0.9.0?
Measured bundle size telemetry is currently unavailable for skillevaluator v0.9.0; verify build output locally before deploying.
How do I upgrade skillevaluator to v0.9.0?
Upgrade skillevaluator to v0.9.0 by running "pip install skillevaluator==0.9.0" and applying the migration steps: npm install skillevaluator@0.9.0
Related Engineering Blueprints (1)
The Infrastructure Race: How Tech Giants Are Standardizing AI Agent Architecture
Architectural analysis of Alibaba open-code-review, Tencent LoopForge, and NVIDIA SkillEvaluator: hybrid pipelines, human approval gates, and CI/CD skill evals.