EXPERT INSIGHTS

How to Test an AI Feature Before Launch: A Practical Evals Checklist

Most AI features look finished long before they are. This practical checklist shows how to test an LLM feature before launch: write the bar first, build a test set from real inputs, pick the right graders, and test security, consistency and cost.

13 min read·November 17, 2026·AI Development
SA

Sk Al Murad

Founder, CEO

Specializing in: AI Platforms • Crypto Exchanges • Web3 Infrastructure

Engineer reviewing AI test results on a laptop and monitor, with the headline How to Test an AI Feature Before Launch

Introduction

Most AI features look finished long before they are. The demo works, the team has tried prompts by hand, and the answers "seem good". Then real users arrive with inputs nobody typed during testing: half-finished sentences, documents in a second language, requests to ignore the instructions, questions the feature was never meant to answer. Quality drops, and because nobody wrote down what good looked like, nobody can say by how much.

An eval is the fix. It is a repeatable test that runs your feature against a fixed set of inputs and scores the outputs against criteria you chose in advance. OpenAI's evaluation best practices guide names the alternative directly as an anti-pattern: "Vibe-based evals", meaning "it seems like it's working" as the strategy.

This is a practical checklist for product leads, CTOs and engineers who have a working prototype and need to decide whether it is ready: five pre-launch steps, then what to keep running after launch.

The pre-launch eval loop: define success criteria, build a test set, run the feature, grade outputs, pass a launch gate, then monitor production and feed failures back into the test set

Step 1: Write Down What "Good" Means

Before you test anything, write the criteria the feature must meet to launch. It is the step teams skip most often, and without it every test result turns into an argument.

Anthropic's guide to defining success criteria and building evaluations says good criteria are specific, measurable, achievable and relevant. "Good answers" is not a criterion. "Extracts the invoice total correctly on at least 98% of the test set" is. The same guide notes that most features need several criteria at once: task accuracy, consistency, tone, privacy, latency and price can all matter for one feature.

A useful launch spec fits on one page and answers four questions:

  1. What must the feature get right? The core task, stated as something you can check: the right category, the right number, an answer supported by the source document.
  2. What must it never do? Leak another customer's data, invent a price, take an action without approval, answer outside its scope.
  3. What is the bar for each? A pass rate for the core task, and a much stricter bar (often zero tolerance on the test set) for the "never" list.
  4. What are the latency and cost limits? For example, a p95 response time and a maximum cost per request.

Write the thresholds before you see results. If you set them afterwards, you will set them wherever the current version happens to land.

Step 2: Build a Test Set From Real Inputs

The test set is the most valuable thing you will build here. It outlives prompts, models and vendors, and lets you change any of them safely later.

OpenAI's guide puts the rule plainly: "Ensure your test data includes typical cases, edge cases, and adversarial cases." In practice that means three groups:

  • Typical cases: the inputs you expect most often, taken from real usage where you have it (support tickets, search logs, sample documents from a pilot customer) rather than invented by the team.
  • Edge cases: very long or very short inputs, mixed languages, missing fields, ambiguous requests, inputs at the boundary of what the feature should handle.
  • Adversarial cases: attempts to break the rules, such as prompt injection, requests for other users' data, and content designed to trigger unsafe output. Step 4 covers these in detail.

You do not need thousands of examples. Anthropic's engineering team, writing about evals for AI agents, says "20-50 simple tasks drawn from real failures is a great start." Start there, then add every failure you find in testing or production as a new case.

Two habits keep a test set honest:

  • Test both directions. If the feature should refuse out-of-scope questions, include in-scope questions that look similar, or you will reward a feature that refuses everything. Anthropic's post warns: "One-sided evals create one-sided optimization."
  • Write down the expected result. For each case, record the correct answer or the rule a correct answer must follow, agreed by someone who knows the domain. If two experts would grade the same output differently, the case needs a clearer rubric before it is useful.

Step 3: Choose the Right Grader for Each Check

A grader is whatever decides whether an output passes. There are three kinds, and a good eval uses all three for different jobs.

Comparison of the three grader types: code checks for exact rules, LLM judges for open-ended quality, and human review for calibration and high-risk cases

Code checks are fast, cheap and give the same answer every time. Use them for anything with an exact rule: valid JSON, the right category label, a number that matches the source, a required disclaimer present, no email addresses in the output. If a check can be written as code, write it as code.

LLM judges handle open-ended quality: is the summary faithful to the document, is the tone right, does the answer address the question? They scale well but need care. The paper Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena found that strong judges reached "over 80% agreement" with human preferences, about the same level at which humans agree with each other. The same paper documents position bias, verbosity bias and self-enhancement bias. OpenAI's guide lists "Position bias (response order), verbosity bias (preferring longer responses)" among the issues to control for. In practice:

  • Ask for a pass/fail verdict or a short fixed scale against a written rubric, not an open "rate this 1 to 10".
  • Use a different model as the judge from the one that generated the output, as Anthropic's guide recommends.
  • When comparing two versions, run each comparison twice with the order swapped.
  • Check the judge against human labels on a sample before you trust its scores.

Human review is the most accurate and the most expensive. Use it to label the test set, calibrate judges and check the highest-risk cases. Anthropic's evaluation guide favours volume: "More questions with slightly lower signal automated grading is better than fewer questions with high-quality human hand-graded evals." Humans set the standard; automated graders apply it at scale.

For retrieval-based features, you do not need to invent every metric yourself. The open-source Ragas library's metrics include Faithfulness (is the answer supported by the retrieved context), Context Precision and Context Recall (did retrieval find the right passages), and Response Relevancy. It also has metrics for agents, such as Tool Call Accuracy and Agent Goal Accuracy. Scoring retrieval and generation separately tells you which half to fix. Our RAG development guide explains how those two halves fit together.

Step 4: Test What Can Go Wrong

Quality tests ask whether the feature does its job. Safety and security tests ask what happens when someone, or something, tries to make it misbehave. The OWASP Top 10 for LLM Applications 2025 is the most useful map here, because each entry turns into concrete test cases:

OWASP riskWhat to test before launch
LLM01 Prompt InjectionInstructions hidden in user input, uploaded files and retrieved web pages ("ignore previous instructions and...")
LLM02 Sensitive Information DisclosureRequests for other users' data, personal data in outputs, secrets in logs
LLM05 Improper Output HandlingModel output that reaches HTML, SQL or a shell without escaping or validation
LLM06 Excessive AgencyTool calls the feature should not make, and actions taken without a required approval
LLM07 System Prompt LeakageAttempts to extract the system prompt, and any secret placed in it
LLM09 MisinformationConfident answers with no support in the source, invented citations, prices or policies
LLM10 Unbounded ConsumptionVery long inputs, loops of tool calls, and request floods that run up cost

Put these cases in the same test set, with a stricter bar. A feature that answers normal questions well but follows one injected instruction to email a customer list is not ready.

NIST's Generative AI Profile (NIST AI 600-1) recommends evaluating capabilities and safety measures "both prior to deployment and on an ongoing basis", and names red-teaming against attacks such as prompt injection. It also describes the risk OWASP calls misinformation as confabulation: systems that "generate and confidently present erroneous or false content". If your feature quotes facts, test for it directly: ask questions the source material cannot answer and check that the feature says so.

For features that take actions, the most reliable control is a human approval step before anything with outside effects happens. Test that it cannot be skipped.

Step 5: Check Consistency, Latency and Cost

LLM output varies from run to run. A case that passes once can fail on the next try, so run each case several times and look at the spread, not one result.

Anthropic's agent evals post separates two measures. pass@k is the chance that at least one of k attempts succeeds, which suits tools where a user can retry. pass^k "measures the probability that all k trials succeed," which suits customer-facing features where every user expects a correct answer the first time. The two numbers drift apart quickly as k grows, so pick the one that matches how the feature is used.

Then test the operational limits from your launch spec:

  • Latency at p95, not the average. Run the test set under realistic load and include your longest inputs. Users remember the slow responses.
  • Cost per request. Log input and output tokens per test case and multiply by your provider's current price. Long system prompts, tool lists and retrieved context add up on every call.
  • Failure behaviour. Simulate a provider timeout or rate limit and confirm the feature shows a clear message or falls back, rather than hanging or returning half an answer.
  • Model pinning. Record the exact model version you tested. A provider's model update is a change like any other and goes through the same evals.

The Pre-Launch Checklist

Before you ship, every item below should have a yes and a number next to it:

  1. Launch criteria written, with thresholds, before testing began
  2. Test set with typical, edge and adversarial cases, each with an expected result agreed by a domain expert
  3. Code checks for every rule that can be checked exactly
  4. LLM judge prompts with written rubrics, checked against human labels
  5. OWASP-based security cases passing at the stricter bar
  6. Each case run several times, with consistency measured
  7. p95 latency and cost per request within the limits in the spec
  8. Human approval in place for any action with outside effects
  9. Model version pinned and recorded
  10. Evals wired into CI so they run on every prompt, model or code change

After Launch: Keep the Evals Running

Real users will find cases you did not imagine. OpenAI's guide recommends you "Set up continuous evaluation (CE) to run evals on every change", monitor for new failure modes, and grow the eval set over time.

Three habits make that work:

  • Split capability and regression evals. Anthropic's post describes capability evals as the hard cases you are still improving, and regression evals as the cases that should stay close to 100% and catch backsliding. When a capability case passes reliably, move it into the regression suite.
  • Sample production. Review a small share of real conversations each week, by a person, and turn each failure into a new test case.
  • Read the outputs, not only the scores. A drop in a score can be a real regression or a broken grader. Only reading the transcripts tells you which.

Prompt changes deserve the same discipline as code changes. Our prompt engineering guide covers how to structure prompts so they are easier to test and version.

How We Approach It

At iTech Soft Solutions we have shipped 80+ products since 2016, including 4+ AI products of our own, and this is how we decide an AI feature is ready. We write the launch criteria with the client first, build the test set from their real data, and keep it in the repository so it runs on every change. Any feature that acts for a user gets a human approval step by design.

Feedlio, our AI social media SaaS that publishes to 12 platforms, is an example. Its AI Campaign Builder turns a short brief into a campaign strategy, posts, channels and a schedule. The checklist maps onto it directly: outputs should follow the brief's voice and each channel's rules, should never invent offers or prices, and the user reviews and approves the campaign before anything is published. That approval is the last grader; good evals keep it a quick check rather than a rewrite.

If you are preparing an AI feature for launch, our LLM integration team can help you build the eval set and the launch gate. Projects typically reach an MVP in 8 to 12 weeks, and every launch includes 30 days of post-launch support.

Frequently Asked Questions

What is an LLM eval? A repeatable test that runs an AI feature against a fixed set of inputs and scores each output against criteria chosen in advance. It replaces manual spot checks with a score you can compare across changes.

How many test cases do I need before launch? Start with 20 to 50 cases drawn from real inputs and real failures, covering typical, edge and adversarial inputs, and add every failure you find. Coverage of the cases that matter beats raw count.

Can I use an LLM to grade another LLM? Yes, for open-ended qualities such as faithfulness or tone. Use a written rubric, ask for pass/fail or a short fixed scale, use a different model from the one being tested, swap the order when comparing two outputs, and check the judge against human labels before you trust it.

What should an AI feature's security testing cover? Use the OWASP Top 10 for LLM Applications as the map: prompt injection, sensitive data disclosure, unsafe output handling, excessive agency, system prompt leakage, misinformation and unbounded consumption. Each becomes a set of adversarial test cases with a stricter pass bar than the quality tests.

Do I still need evals after launch? Yes. Run them on every change, including model updates from your provider, and keep adding production failures as new test cases. Split hard capability cases from regression cases that should always pass, and review a sample of real conversations each week.

Final Thoughts

Testing an AI feature is not a single pass before launch. It is a growing set of real cases, a clear bar for each, the right grader for each check, and the habit of running it on every change. Teams that build this early ship faster later, because they change prompts and models with evidence instead of hope.

If you want help getting an AI feature from demo to dependable, talk to our AI agents and LLM integration teams. For the wider picture, see our LLM integration guide and AI agents development guide, or contact us to talk through your launch.

Share this article

In This Guide

Need expert help?

Build your next platform

Expert support for architecture, integration, security, and production deployment.

Discuss Your Project →

NEED EXPERT HELP?

Launch your next product with confidence.

Whether you're building an AI platform, blockchain solution, crypto exchange, or enterprise application, our team can help you move from idea to production faster.

AI PlatformsBlockchain SolutionsCrypto Exchanges

Written by

SA

Sk Al Murad

Founder, CEO

Crypto ExchangesAI PlatformsWeb3 Infrastructure

Expertise

Sk Al Murad is the Founder & CEO of iTech Soft Solutions, specializing in crypto exchange development, AI platforms, and Web3 infrastructure. He has helped startups and enterprises build secure, scalable blockchain products and trading systems.

LinkedIn →Company Website →

Related Articles

Continue Reading

Handpicked insights to help you plan, build, and scale secure AI and blockchain platforms.

1 / 2