We ran our own audit. It said “proves a click.”
Last week we published a skill that tells you what your agent's approval step proves: nothing, a click, or the action. Before publishing it we pointed it at the example in our own repository, the one we hand to prospects. Biometric-signed receipt, verified offline, no receipt no deploy. We expected the good verdict.
It said proves a click. Six properties, four of them Fail. The report is published verbatim in the plugin repo, and this post walks through it, because an audit tool whose authors hide their own result is not worth installing.
The gate
The example is a deploy service. An agent asks for approval to deploy; a human approves on their phone with Face ID; the device signs a hash of the plan; the platform countersigns; the deploy service verifies both signatures against a published key set and compares the plan hash before it acts. Every one of those pieces worked. The audit's controls confirmed them: unsigned receipts refused, denied receipts refused, swapped region refused, in-process replay refused.
Then it ran nine probes. All nine passed, in the attacker's favour.
The scorecard
| # | Property | Result | Status |
|---|---|---|---|
| 1 | Binding | Fail | confirmed |
| 2 | Rendering | Fail | asserted |
| 3 | Channel authenticity | Pass | confirmed, two caveats |
| 4 | Freshness and single use | Fail | confirmed |
| 5 | Approver anchoring | Unknown | relying-party half confirmed |
| 6 | Evidence | Fail | confirmed |
Row 3 passed, so the human was real. Rows 1, 4 and 6 failed, so what they approved, whether it ran once, and what we could show afterwards were not established. That is the definition of a click.
Finding 1: we hashed a sentence
The device signed a hash of the plan. The plan was
"Deploy web to us-east-1".
That bound two words. The deploy request carried five fields: service, region, image, replicas, commit. Three of them were not in the sentence, so a receipt the human signed for the sentence authorised any image, any replica count, any commit.
✔ PROBE binding: fields the plan template does not mention are unbound (image, replicas)
Two more probes came free with the sentence. The template
interpolated the body fields with a JavaScript template string, and
an array prints as its joined elements, so
["web"] passed the hash check as
web and reached the executor as an
array. And the delimiter “ to ” could appear inside a value: request
approval for service api to prod,
region us-east-1, then deploy service
api to region
prod to us-east-1. One receipt, two
different deploys, same hash.
The root cause was one level down. The platform already accepted a structured plan object and hashed it with canonical JSON. The SDK had no field to send one. So the example did what the SDK allowed and put the plan in the description.
Finding 2: we logged the wrong name
A receipt carries two identities: a subject that the platform asserts, and the list of approvers whose devices actually signed. The example logged the subject. The probe minted a receipt where the subject was Alice and the device signature was Mallory's. The service deployed and logged “approved by Alice”.
Finding 3: single use lasted until the next restart
The example used the in-memory replay store. Use a receipt, restart
the service, present the receipt again inside its ten-minute window:
deployed twice on one approval. The store has a guard that refuses to
start outside development, but it keys on
NODE_ENV=production, and the example
never set it.
Finding 4: anyone enrolled could approve
The service had no notion of who may approve a production deploy. Any identity with an enrolled device produced an acceptable receipt. The agent chose which human to ask. And the agent's API key could invite new humans to the tenant, so an agent with an inbox tool could, in principle, enrol its own approver. That last step needs the live platform's email verification, so the audit marked it asserted rather than confirmed, following its own rule about never probing live systems. It still went in the report.
Two things the audit found that are not in the example
The report also surfaced two protocol-level caveats, which is the part we were least pleased to read and the part most worth publishing.
The device signature covers the plan hash but not the decision bit. Approve and deny produce the same device signature; the platform's countersignature carries the result. So a party holding the platform key could re-label a denial as an approval, and the relying party would accept it.
And the verifier always enforces expiry. A receipt is
third-party-verifiable evidence for ten minutes. After that an auditor
gets RECEIPT_EXPIRED, not a proof,
unless there is an audit mode that checks signatures and the hash
while skipping expiry and replay. There was not.
What we changed
The fixes landed in one commit across the example, the SDK and the verifier.
The plan is a struct, validated before it is hashed. Every field the executor acts on, and nothing else:
export function deployPlan(p: DeployParams) {
return {
service: p.service,
region: p.region,
image: p.image, // digest-pinned, never a tag
replicas: p.replicas,
commit: p.commit,
};
}
The parser rejects unknown fields, non-string values, mutable image
tags and replica counts outside 1 to 100, and it runs before anything
is hashed, so an array can no longer coerce into the string it prints
as. The SDK gained a plan passthrough
so the agent sends the same object the service rebuilds.
Policy reads the device-signed list. The allowed-approvers list is enforced on the device-signed approvers, never on the asserted subject. The audit entry records both, kept apart, so a mismatch is visible rather than hidden. The service refuses to start without a policy.
Single use survives restarts. A new zero-dependency replay store on Node's built-in SQLite, so the example has no native build step and no excuse.
The operator keeps a record. Each deploy writes an audit entry with the validated params, the device-signed approvers, the asserted subject, the receipt id and the request id. The receipt itself can be fetched again while the platform holds it.
The tests
Every probe became a test with the assertion flipped. The suite has 19 tests and runs with locally minted receipts against a local key set; nothing talks to the platform.
| Attack | Before | After |
|---|---|---|
| Approve one image digest, deploy another | deployed | 403 PLAN_HASH_MISMATCH |
| Receipt for the sentence-shaped plan | deployed | 403 PLAN_HASH_MISMATCH |
| Array for service, string for replicas, extra field, :latest tag | deployed | 400 INVALID_DEPLOY_PARAMS |
| Use a receipt, restart, use it again | deployed twice | 409 REPLAY |
| Receipt from a device not in the policy | deployed | 403 APPROVER_NOT_ALLOWED |
| sub says the allowed approver, device says someone else | deployed, mis-logged | 403 |
| Start with no approver policy | started | refuses to start |
What it proves now
Rows 1, 4 and 5 flip. Row 2 flips with them, because the device renders every field of the struct it signs. Row 6 is now operator-grade.
On the strict rule it is still not proves the action, and we are not going to round up. The two protocol caveats are open: the decision bit is not device-signed, and there is no audit mode after expiry. Both are wire-format changes across the platform, two apps and the verifier. They are on the backlog with the report attached. When they ship we will re-run the audit, blind, and publish that too.
What the audit changed about the audit
The report ended with a section of feedback on the skill itself, and most of it went in.
- The verdict rule now says what an Unknown row does: it counts as Fail and is always marked asserted.
- Property 3 asks who signs the decision bit and who signs the action hash, separately.
- The report has a place for controls, the tests that show what the gate does refuse, because they are what the verdict rests on.
- There is a probe recipe for signed-receipt vendors: mint with their fixtures, re-label an envelope, restart the process.
- Remediation rung 5 is written as retention and re-verifiability when a receipt vendor is already in use, so it stops reading as an advertisement.
Run it on yours
/plugin marketplace add humanauth-ai/hitl-skills
/plugin install hitl@humanauth Then ask what your approval step proves. If the answer is a click and you need the action, the report tells you which rung to climb first, and the first four rungs never mention us.