You can evaluate immigration AI on a small subset of matters before expanding across the firm. Define the preparation steps being tested, use representative files, and measure the work remaining after the system produces its output. Track staff time, factual corrections, evidence traceability and attorney review separately.
A polished demonstration answers whether software can produce something useful. A pilot should answer whether your team can use that output reliably in its actual workflow.
Start with one workflow and a clear finish line
“Test AI” is too broad to evaluate.
Choose a specific preparation workflow, such as producing a draft support letter and associated forms from an existing matter file. Write down what the system will receive, what it should return and what your team must still complete.
Define the finish line as an output your attorney can review against the firm’s agreed checklist. Generating a document is an intermediate step.
Keep filing outside the pilot unless submission is explicitly included in the approved scope, with clear human authorization.
Choose representative matters
An initial evaluation can use three carefully selected matters:
- A routine matter with complete, consistent inputs.
- A matter containing missing or conflicting information.
- A matter with a variation your firm regularly encounters.
Three matters are a starting sample, not proof of performance across a practice. Expand the evaluation if outcomes vary or the sample does not reflect your caseload.
Previously completed matters can help establish expected outputs. Review data handling and permissions before sharing files, and avoid exposing unnecessary client information.
Do not compare a straightforward matter in one product with an unusually complex matter in another. Use equivalent inputs and the same scope wherever possible.
Establish the baseline before testing
Record how the team currently completes the same preparation work.
Separate active work from elapsed time. A case waiting overnight for a document has a long turnaround, but that waiting period is not eight hours of staff effort.
Capture time spent on:
- Organizing and checking inputs.
- Entering information and preparing drafts.
- Correcting facts, formatting and evidence references.
- Reviewing the output.
- Returning documents and updating the case record.
Before testing, use our firm-template preparation checklist to separate reusable language from case-specific facts.
Record implementation and template setup separately. Those costs matter, but mixing them into ordinary case work makes both harder to understand. For broader software selection, use the immigration AI buyer guide.
Use a scorecard for every matter
Agree on acceptance criteria before reviewing the results.
| Measure | What to record |
|---|---|
| Input preparation | Staff minutes spent making the file usable |
| Draft preparation | Active staff time and system processing time, separately |
| Factual accuracy | Incorrect, missing or unsupported statements and fields |
| Conflicting information | Whether discrepancies were surfaced for review |
| Evidence traceability | Whether reviewers could locate support for material statements |
| Attorney review | Review minutes and substantive changes required |
| Output usability | Template, formatting and document-organization corrections |
| Case-system handoff | Uploads, downloads, re-entry and record updates still required |
| Exceptions | Failures, manual recovery and unresolved questions |
Avoid combining everything into one numerical score. Record what needed correction, why it mattered and how much work the correction required. A fast result with an unresolved factual problem should remain visible beside the time result.
Classify corrections before counting them
Use the same categories for each pilot matter. Agree on examples with the supervising attorney before the first test, then apply those definitions to the baseline process and the AI-assisted process.
| Category | Example | What to record |
|---|---|---|
| Incorrect fact | A draft uses a different employer address from the source selected for the test | Expected value, actual value, source and affected outputs |
| Missing required content | A section included in the agreed preparation scope is absent | The requirement in the test checklist and what is missing |
| Unsupported statement | A letter asserts experience that the supplied evidence does not establish | Statement, missing support and reviewer decision |
| Unresolved conflict | Two sources disagree and the draft silently chooses one | Both sources, the difference and who must resolve it |
| Assembly or transfer defect | A returned packet is missing a page or the wrong version was saved | Expected output, observed output and recovery work |
| Editorial preference | The attorney prefers a shorter introduction without changing its meaning | Preference and edit time, separately from factual defects |
These are suggested working categories. Whether an issue is substantive depends on the matter. A change to a few words can be more consequential than a full page of style edits. The attorney decides which issues block acceptance and what further review is necessary.
If a source fact was missing from the test inputs, record that input limitation. Do not label the output correct merely because the software could not have known the answer. Record whether it left the question open, requested clarification or invented a value. An appropriate missing-information flag is a different result from an unsupported assertion.
Keep one log entry per underlying issue
Record the affected locations under the same issue rather than inflating the count whenever a wrong value repeats.
For example, a fictional matter contains one incorrect company address repeated in a form, support letter and index. Log one underlying factual issue affecting three outputs. Check all three locations when resolving it. This separates the source of the problem from the amount of cleanup it caused.
| Field | What to enter |
|---|---|
| Matter and output version | The fictional or approved test matter and draft being reviewed |
| Issue ID and category | A reference for this problem and its classification |
| Expected result | The agreed checklist item or selected source, including location |
| Observed result | What the draft or saved output actually contains |
| Affected locations | Each form field, paragraph or packet item needing attention |
| Review decision | Blocking issue, correction needed, preference or unresolved question |
| Effort | Minutes investigating, correcting and rechecking, without counting the same time twice |
| Resolution | What changed, who checked it and whether it recurred on rerun |
Keep sensitive source details within the firm's approved system. A separate pilot spreadsheet does not need copies of passports or other client documents to identify an issue.
Report counts with a denominator and scope
Prefer “two of five test matters needed factual corrections” to “the system is 60% accurate”. The first statement describes what the team observed. The second implies a defined accuracy test that a small matter sample usually does not provide.
If checking individual fields, state how many fields were checked, how they were selected and which outputs were included. A check of names and addresses cannot establish the quality of legal analysis. Do not combine field checks and narrative review into one accuracy percentage.
Track unresolved issues separately from resolved ones. A draft that was eventually corrected can be useful, but it should not be recorded as correct on the first pass. Likewise, a second run on the same matter is a rerun, not another independent case in the sample.
Work through a fictional pilot result
Suppose five test matters produce the following findings:
- Two matters contain factual issues requiring correction.
- One of those issues repeats in three outputs.
- All five receive style edits.
- One matter still has an unresolved evidence question at the end of the test.
Report those observations separately. Do not count every style edit as a factual failure or treat the corrected versions as proof that the first drafts were error-free. Preserve the blocking question even if the total preparation time improved.
The figures illustrate reporting only. They are not Infinity results, a recommended sample size or evidence of reliability across a practice.
At the pilot review, compare the correction categories, active effort and remaining questions with the baseline. Investigate material unsupported statements, unresolved identity conflicts or lost review control before expanding. Use the immigration review QA workflow for the wider review handoff.
Test what happens when the inputs change
A routine file may show how the system handles expected work. Exceptions show what your team must manage.
In an authorized test environment, introduce a controlled change, such as a corrected address or a conflicting date. Check whether the system:
- Identifies the affected information.
- Flags the conflict rather than silently choosing an answer.
- Updates the relevant output.
- Makes the change understandable to the reviewer.
Confirm what happens to earlier versions. Your team should know which output is current and which still requires review.
These are evaluation questions, not claims that every product supports the same behavior.
Follow the work back into the case-management system
An integration label does not explain the full handoff.
Ask the vendor to demonstrate where the workflow starts, which records it reads, which documents it returns and which updates require staff action.
Watch the actual process. Count the downloads, uploads, copy-and-paste steps and application changes that remain.
Some products import documents and return completed work. Others support additional configured actions. Verify the exact behavior available for your system, permissions and case type. See the MyCase workflow for an integration-specific example. For destination-side checks, use the CMS handoff checklist.
Calculate value from the complete workflow
Use this calculation for each matter:
Net staff time saved = baseline active staff time − pilot active staff time
Pilot time should include input preparation, operation, corrections, review and final handoff.
For an illustrative example, suppose the current process takes 100 active staff minutes. The pilot takes 15 minutes to prepare inputs, 10 minutes to operate, 20 minutes to correct and 25 minutes to review and return the work. Total active time is 70 minutes, leaving 30 minutes saved.
These numbers illustrate the method. They are not an Infinity benchmark.
Compare the value of measured time saved with software usage charges and implementation costs. Time released creates capacity; it becomes a cash saving only when an actual expense falls.
Decide whether to expand, adjust or stop
Expand when representative matters meet the agreed quality requirements, staff effort improves and the remaining work is clear.
Adjust the scope when the system helps with one preparation stage but leaves another stage largely manual.
Pause when corrections erase the benefit, important exceptions remain unexplained or the team cannot verify the output confidently.
Document the reason for the decision. A useful pilot can reveal that a product fits one workflow but needs further testing elsewhere. For a related process, see the immigration intake QA workflow.
Frequently asked questions
Can we pilot immigration AI on only a few matters?
Yes. Define a limited scope and select representative files. A small sample can reveal workflow problems, but it does not establish reliability across every case type.
How long should the pilot run?
Long enough to complete the agreed preparation and review steps and investigate exceptions. A drafting test and a seasonal intake test may require different evaluation periods.
Should we measure only drafting speed?
No. Include input preparation, corrections, attorney review and the handoff back into the case record.
Should every attorney edit count as an AI error?
No. Separate incorrect facts, omissions, unsupported statements and workflow defects from editorial preferences. Record the time spent on both. Classification helps explain the work remaining without treating style preferences as factual failures.
How should we count one mistake repeated across several documents?
Log one underlying issue and list each affected location. Report the number of issues and their spread separately, then verify every affected output after correction.
Can a small pilot establish an accuracy percentage?
A pilot can report defined checks within its sample. State the denominator, selection method, case types and outputs reviewed. Avoid turning a few completed matters into a general accuracy claim for the product or practice.
Does a successful pilot remove attorney review?
No. The evaluation should establish how the software supports review and where legal judgment and final authorization remain.
Evaluate Infinity against your workflow
When evaluating Infinity, agree on the supported preparation steps, inputs, outputs and integration requirements for your firm’s configuration. Use the same scorecard to identify what it prepares and what remains with your team.
Book a case-preparation workflow review