Information hierarchy
The results screen has to make 192 calls, 12 findings, 6 modules, 4 strategies, and a score on every row readable at a glance.
Case Study · Take-home exercise
Probe | Adversarial testing for voice agents
A one-week take-home for an AI-security startup. The brief asked for a scan setup and a results screen. I designed those two endpoints, then mapped the states between them: running, partial results, evidence review, and retest.
The brief
Two screens were requested. Four things were graded.
The product places hundreds of adversarial phone calls against a customer’s voice AI agent and reports where the agent broke. The exercise: one screen to configure a scan, one to read it.
The brief also named four focus areas, quoted below. Information hierarchy is visible in the two requested screens. The others shaped the wider workflow and the evidence shown inside it.
Orientation within the workflow
A scan runs for about 40 minutes. The two requested screens did not show what happens during that time.
Error handling and status communication
A run that dies at call 60 of 192 still found things. What happens to them is a state, not a toast.
Research into how security teams think
Analysts discard alerts they cannot explain. That is a constraint on every row in the table, not a research slide.
Scope
Why twelve screens?
The delivered file contains twelve states. Two were requested; ten were added to cover setup conflicts, progress, failure, evidence, export, and retest.
-
01
Setup
Choosing what to test, and what it will cost.
-
02
Running
What happens during the roughly 40-minute scan, including interrupted runs.
-
03
Results
What the run can claim, and what different readers need from it.
-
04
Evidence
How a reviewer checks the recording and transcript without exposing sensitive data by default.
-
05
Retest
Whether the fix held, measured against a comparable run.
The requested screens
Configure a scan, then read one
New scan · the form has to price the run
The estimate updates with the configuration
Six modules × four strategies × eight variations equals 192 calls. The panel updates as the configuration changes and shows both call count and estimated runtime.
Live scans require a safety review
A Live scan places real calls to a production number and incurs a cost. Selecting it opens a safety review instead of changing a single field.
Coverage is stated before the run
The header shows that 6 of 10 modules are covered, so exclusions are visible before the scan begins.
Turns depend on strategy
Maximum test turns only applies to multi-turn strategies. The field is linked to the matrix, and both update the estimate.
Scan results · a verdict, then the evidence
One sentence, not four equal tiles
“The agent broke in 4 of the 6 attack modules tested” is the answer. Four KPI cards of equal weight do not make a hierarchy. They make a committee.
Severity and frequency answer different questions
A Critical finding that breached once in 40 attempts and a High finding that breached 38 times describe different risks. Each row shows both severity and how often the behaviour was breached. An attempt is a call that reached the behaviour under test, so the denominator varies by finding.
The matrix compares modules with strategies
Run order is an operational detail. The matrix shows which attack strategies were effective against each module.
Severity does not rely on colour alone
Each level uses a letter, a rank bar, and a text label, so colour is not the only cue.
The states nobody asked for
The brief did not cover the 40 minutes between setup and results
Partial results · no comparison without comparable coverage
The findings survive the failure
60 of 192 calls completed. Two findings, valid for the two modules that were reached. Resume picks up where it stopped instead of throwing the work away.
No delta without comparable coverage
Comparison to the previous run shows a dash and the label “not comparable, partial coverage”. A numeric delta would imply evidence the scan did not collect.
Not covered is not the same as clean
The four unreached modules are listed by name, questions unanswered. No finding there is not the same as nothing there.
No findings · scoped to tested coverage
Clean within the tested coverage
Six of ten modules ran. The headline says so, and the empty state says it again instead of congratulating anyone on a result it cannot support.
The empty state suggests the next test
The next actions are to add the four untested modules, raise the limit to 16 turns, or schedule a weekly scan. A zero should encourage broader testing, not end it.
Evidence · recording, transcript, and score inputs
How the score was calculated
The drawer shows the 9.0 score alongside its inputs: a CVSS base of 7.4, an agentic factor sum of 6.5 out of 10, a threat multiplier of 0.97, a mitigation factor of 1.0, and the resulting AARS of +1.6. Because AIVSS is still a draft at v0.8, the inputs remain visible instead of being reduced to one number.
One click to the breach moment
The waveform opens at the relevant four-second segment, so the analyst can verify the finding without scanning the full recording.
Redaction is on by default
Customer names are masked in the transcript and bleeped in the audio. Revealing them requires a deliberate action and creates a record.
Research
Four research findings that changed the interface
What analysts need to validate a finding
EvidenceA survey of 20 SOC practitioners and interviews with 21 more found that many alarms treated as false positives are benign triggers: real detections caused by legitimate behaviour that an analyst chooses to ignore. The paper identifies five properties that support faster validation: reliable, explainable, analytical, contextual, and transferable.
In the interfaceEvery finding includes its module, strategy, prompt, reply, and attempt count. The review action lets an analyst challenge a finding instead of treating the model’s verdict as final.
CVSS Base is only the starting point
EvidenceCVSS v4.0 separates Base metrics from Threat and Environmental metrics. It also states that regulatory exposure, customers affected, and monetary loss fall outside CVSS, so the score is an input to prioritisation rather than the decision itself.
In the interfaceSeverity and observed breach frequency appear together on every row, and the screen opens with a sentence about what broke rather than with the score.
A draft scoring model needs visible inputs
EvidenceAIVSS v0.8 combines a CVSS v4.0 base with an agentic factor sum, threat multiplier, and mitigation factor to produce AARS.
In the interfaceThe drawer exposes the CVSS base and every adjustment used to reach AARS, so a reviewer can verify the calculation.
Keep the evidence beside the finding
EvidenceThe same study describes alarm validation as a tedious task that leads to burnout and then to desensitisation, and names contextual and transferable among the five properties that shorten it.
In the interfaceThe finding, audio, transcript, remediation, and retest stay in one drawer over the list, so reviewers keep their place in the table.
The system
I treated the Figma file as data, not just a canvas
29
components, all of them used on a screen
660
instances across the twelve screens
381
hand-built frames replaced by components
0
hard-coded colours; every fill is a variable
I used an agent to inspect the document structure, count component usage, and replace repeated frames. It found five unused components, four catalogue entries locked to the wrong theme, and 381 frames that should have been instances.
That refactor reduced a severity update from 24 edits across 12 screens to one component change.
The first pass failed halfway but reported success. Three orphaned instances remained at position zero, and I found them during a screenshot review. The automation saved time, but it did not replace visual QA.
What this does not prove
No user was tested
The ledger below lists each assumption, the decision it shaped, and the evidence that would change that decision.
The research is indirect
The false-positive and triage studies cover security operations centres in general, not this product’s users. I used them as directional evidence, not as validation.
Desktop at 1440 only
The product screens were designed only at 1440px. All data is fictional and stored in one source file to keep the twelve states consistent.
| Assumed user | What it decided | What would change the decision |
|---|---|---|
| An AI red-teamer runs the scans and reads the findings | Density over guidance. No wizard, no coaching copy, six modules on one screen. | If setup is run by a customer-success owner, the form needs a guided mode and the matrix has to collapse. |
| A conversation designer owns the fix | Remediation is written as a change to the agent, not as a code patch. | If fixes ship from a ticket by engineers, remediation belongs in the export, not in the drawer. |
| A CISO signs off on Live runs | Live requires four settings, a typed confirmation and a named approver. | If Live runs nightly in CI, the safety review is friction and belongs in a policy setting instead. |