LLM ANALYZE API
Don’t just generate.
Evaluate.
Give AI responses a clear standard to meet. Explore practical rubrics for evidence, completeness, instruction following, and output format.
WHAT GOES IN
The right input.
The right question.
Prompts, generated responses, reference context, and an evaluation rubric.
KEEP THE SCOPE CLEARA valid response format does not prove factual accuracy. Model-based judges also need evaluation against reviewed examples.
WHAT COMES OUT
A report you can use.
- Criterion-level evaluation findings
- Evidence-linked review explanations
- Paired comparisons and regression cases
Keep the evidence, interpretation, and next action separate. Document missing inputs instead of filling them with guessed values.
A PRACTICAL WORKFLOW
Review a new assistant configuration
Compare a baseline and a candidate using the same reviewed cases. Check the response format independently from correctness, then inspect the failures that an overall average might hide.
Explore integration patternsLOOK INSIDE THE OUTPUT
Evidence, not a mystery score.
An illustrative report for this workflow. Download the JSON to inspect the complete structure.
Response and reference passage
A three-part rubric keeps the review focused.
The sample response uses the required two fields.
The response mentions a feature absent from its reference passage.
The response describes the setup steps requested in the prompt.
{
"schema_version": "1.0",
"report_id": "sample-llm-analysis",
"sample": true,
"analysis_type": "llm-analysis",
"status": "complete",
"target": {
"label": "Sample assistant response",
"input_kind": "Response and reference passage"
},
"summary": "A three-part rubric keeps the review focused.",
"findings": [
{
"id": "f01",
"outcome": "pass",
"title": "The requested format is followed",
"evidence": "The sample response uses the required two fields.",
"action": "Continue validating the output contract."
},
{
"id": "f02",
"outcome": "review",
"title": "One claim lacks supporting evidence",
"evidence": "The response mentions a feature absent from its reference passage.",
"action": "Remove or verify the unsupported statement."
},
{
"id": "f03",
"outcome": "pass",
"title": "The main question is addressed",
"evidence": "The response describes the setup steps requested in the prompt.",
"action": "Review the omitted edge cases separately."
}
],
"not_measured": [
"Production model performance",
"Calibrated confidence",
"General benchmark ranking"
]
}TAKE IT FURTHER
A little context goes a long way.
Build a task-specific evaluation loop with reviewed cases, clear criteria, and evidence behind every judgment.
Read the full guideBefore you connect a real source
Confirm that you are authorized to use the input. Preserve its collection context and version, agree on an acceptance rule, and decide how incomplete results should be reviewed.
- Validate the report before consuming it.
- Keep secrets out of public browser code.
- Show what was—and was not—measured.
Make your next move
a clearer one.
Explore the workflows. Inspect the evidence. Build something useful.