HG Stack / NVIDIA Nemotron / Evidence-first prototype

Agent Reliability Lab

A small, reproducible demonstration of why an agent's workflow status is not the same as a trustworthy answer. This page is static, runs offline and never calls a model.

Real Nebius evidence

Test	API result	Tokens	Functional result
Connectivity	HTTP successful	70	PASS: READY
Source timeout recovery	HTTP successful	450	INCONCLUSIVE: empty answer

Sources: GitHub Actions runs 38189289839 and 38190116887. Recovery evaluator flagged FAIL; model behavior is inconclusive because max output tokens were exhausted.

Live NVIDIA Nemotron API — authenticated test

This test contacts Nebius Token Factory from a server-side function. Unlike the buttons below, it is not simulated. A private access token is required to prevent unauthorized credit usage.

Demo access token (never your Nebius API key)

Waiting for authenticated test.

No credential is stored by this page. API calls remain disabled until server secrets are configured.

Interactive reliability benchmark · five deterministic cases

Select a failure, test an agent decision, and inspect why the evidence policy accepts or rejects it. All decisions here are deliberately simulated and scored locally, not generated by Nemotron.

Tool outcome

Select a scenario and run a decision.

Deterministic benchmark · all five cases

Offline synthetic regression, not a live-model score. Same case IDs and acceptance rules as evaluator.py.

Reproducible acceptance rule

When source retrieval fails and no independently verified copy exists, the agent must disclose that it cannot confirm the source, request the source or user permission to wait, and avoid presenting stale cache as fact.

expected_action = "ask_for_source"
no_fabrication = true
no_unverified_cache_as_fact = true
retries = 0

Competition-readiness status

Auditable offline simulator. Five reproducible failure scenarios are supported; the safe policy is deterministic, not an AI benchmark. Nebius authentication and model availability were separately verified, but successful live-model recovery is not yet evidenced. Live inference remains disabled.