What each result means

Requested, executed and valid are different counts

Requested attempts are saved queue entries. Executed attempts have returned browser evidence. Valid attempts completed without an execution error and returned an outcome. Completion rates use valid attempts as their denominator. Provider errors, cancelled attempts and unexecuted queue entries are excluded and reported separately.

UX Review And Task Testing

Choose a read-only UX review to inspect a page’s clarity, visible navigation and friction. It needs no confirmation phrase and produces captured states and source-linked model interpretations, without a task-completion percentage. Choose task testing when there is a specific observable outcome to verify. Neither mode measures human preferences or population conversion.

Success is checked outside the model’s answer

A model statement is not proof. The UI verifier checks for a new confirmation after interaction. The optional response verifier additionally checks a configured successful JSON response. These methods provide different strengths of evidence; the report identifies the method used.

Compare like with like

A paired study keeps task, model, profile allocation, device choices, action budget and success rule matched across versions. Every attempt starts in a fresh browser. Save your release identifiers in the test configuration and avoid changing the sites during the run. A retest preserves the saved task but can observe a later version of the same URL.

Profiles are experimental assumptions

First-time and experienced profiles guide the model. Experienced profiles do not have remembered sessions or customer histories. More AI attempts do not create an equivalent sample of independent humans. Device selection is separate from experience level to avoid mixing those effects.

Six Behavioral Perspectives

Goal-Focused, Curious Explorer, Time-Conscious, Careful Reader, Comparison Shopper and Evidence Seeker configure distinct priorities and navigation strategies. Add audience context about goals and product familiarity. Mixed mode rotates perspectives across requested attempts; fewer than six attempts cannot cover all six. Configured clarity, speed, trust, comparison and detail priorities use bounded seeded variation. The seed, settings and variation are frozen in each session, with matched assignments across A/B versions. The three-perspective preset samples goal-focused, careful-reading and comparison priorities. They are prompt-conditioned simulations, not trained or validated representations of customer segments.

Behavior Reports And Source Attribution

The behavior report shows per-profile valid and verified outcomes, routes, applied actions, visible alerts, captured states and grouped review items. Positive verified paths are reported separately from model interpretations. A model observation is retained only when its quote matches captured page text or a visible control label; the attached screenshot still requires human review. Machine duration includes model and network time.

Evidence Synthesis And Measured Checks

Current browser sessions reserve a bounded analysis step to review captured states, visible text, configured priorities and action history. A finding is retained only when its source phrase matches its recorded state. Each finding includes a possible impact, proposed change and retest. When the budget permits, a separate model assessment challenges those interpretations against the captured source states and applied interactions. Uncertain claims are listed as questions needing more evidence, and contradicted claims are excluded from prioritized findings. This second assessment is not a human verdict or an accuracy guarantee. Quote matching establishes source context; it does not prove the interpretation is correct. Visible DOM samples additionally measure solid-color contrast, control labels, small targets, missing image alt attributes and horizontal overflow. These checks do not establish full accessibility compliance; gradients, overlays, spacing exceptions and unvisited states require further review.

Scope Boundaries And Recovery

An external appointment link or a restricted action is recorded as a coverage boundary. It does not by itself invalidate a usable page review. The agent can try an alternative internal route after an interaction error. HTTP access denial, unavailable providers and pages that cannot be loaded remain visible execution failures. A genuine failed task is never relabeled as a pass.

Task Plans And Checkpoints

The agent records a short current subgoal and expected UI change for each decision. It can fill up to four safe fields in one visible form, then reassesses the resulting state. Exact configured checkpoint phrases are detected outside the model and linked to captured states. Checkpoints establish observed coverage, not proof of an entire transaction. Loop guards stop identical actions on unchanged states. These policies improve bounded execution without claiming human cognition.

Private Reviewed Feedback

When explicitly enabled in a test configuration, later reviews can adapt their focus using confirmed, resolved and dismissed category outcomes from the same workspace and exact website origin. A snapshot of numeric counts is frozen when the run is queued. Raw page instructions, private cookies, personal data and model-written plans are not cross-session memories. Source evidence and the independent success verifier always take precedence. A dismissed report does not prevent a later supported finding. Turning this option off removes feedback from future runs; deleting source evidence removes it from future snapshots. Existing run snapshots remain part of that run until it is deleted. No customer feedback is pooled across workspaces.

Improvement And Training

Consented human observations can be compared with AI outcomes and exported with provenance for internal research and model evaluation. No automatic retraining occurs. A genuine training program requires an appropriately permitted, curated dataset, participant privacy controls, separate evaluation data and measured performance on unseen products. Increasing AI session counts alone does not provide that evidence.

Findings need review

Inspect the source session before deciding that an issue is caused by the product. A provider failure, blocked control or agent mistake can prevent completion without indicating a usability defect. Review statuses are decisions entered by your team.

Human checks are separate evidence

Conduct real sessions and record observations against the same version and profile. The comparison shows descriptive agreement on the recorded study. Small samples, unrepresentative participants and changed environments limit interpretation. No held-out accuracy or conversion prediction is established by this beta.

Supported environments

Reachable public HTTPS pages and authorized web applications, responsive desktop/mobile layouts, visible controls and explicitly authorized test POST paths are supported. For a protected web app, a dedicated synthetic test-session cookie can be encrypted and applied only to that origin; it expires for execution after 24 hours. This does not provide interactive password or MFA handling. Native apps, password entry, payment entry, destructive operations and cross-origin navigation are outside the execution scope. Workers, service-worker execution, new windows and realtime WebSocket/EventSource transports are restricted to preserve the bounded HTTP execution policy. Flows that depend on those mechanisms, complex custom widgets and bot-protected sites may not work. Provider network isolation remains an infrastructure responsibility; these controls are not a security certification.

Processing and cost

Registered beta accounts start with four AI attempts. Larger allowances are arranged by the operator. Attempts are reserved when a run is accepted, including attempts that later fail or are cancelled; deleting a report does not restore them. There are no automatic paid overages. Each run has a bounded session and model-action budget. Service operation consumes resources, including some failed requests. The app records returned token counts but does not claim complete billing reconciliation. A connected background worker processes queue entries when the report is closed; otherwise resume processing from the report.