Blog · 2026-11-14

We ran our own audit. It said “proves a click.”

Last week we published a skill that tells you what your agent's approval step proves: nothing, a click, or the action. Before publishing it we pointed it at the example in our own repository, the one we hand to prospects. Biometric-signed receipt, verified offline, no receipt no deploy. We expected the good verdict.

It said proves a click. Six properties, four of them Fail. The report is published verbatim in the plugin repo, and this post walks through it, because an audit tool whose authors hide their own result is not worth installing.

The gate

The example is a deploy service. An agent asks for approval to deploy; a human approves on their phone with Face ID; the device signs a hash of the plan; the platform countersigns; the deploy service verifies both signatures against a published key set and compares the plan hash before it acts. Every one of those pieces worked. The audit's controls confirmed them: unsigned receipts refused, denied receipts refused, swapped region refused, in-process replay refused.

Then it ran nine probes. All nine passed, in the attacker's favour.

The scorecard

# Property Result Status
1 Binding Fail confirmed
2 Rendering Fail asserted
3 Channel authenticity Pass confirmed, two caveats
4 Freshness and single use Fail confirmed
5 Approver anchoring Unknown relying-party half confirmed
6 Evidence Fail confirmed

Row 3 passed, so the human was real. Rows 1, 4 and 6 failed, so what they approved, whether it ran once, and what we could show afterwards were not established. That is the definition of a click.

Finding 1: we hashed a sentence

The device signed a hash of the plan. The plan was "Deploy web to us-east-1".

That bound two words. The deploy request carried five fields: service, region, image, replicas, commit. Three of them were not in the sentence, so a receipt the human signed for the sentence authorised any image, any replica count, any commit.

✔ PROBE binding: fields the plan template does not mention are unbound (image, replicas)

Two more probes came free with the sentence. The template interpolated the body fields with a JavaScript template string, and an array prints as its joined elements, so ["web"] passed the hash check as web and reached the executor as an array. And the delimiter “ to ” could appear inside a value: request approval for service api to prod, region us-east-1, then deploy service api to region prod to us-east-1. One receipt, two different deploys, same hash.

The root cause was one level down. The platform already accepted a structured plan object and hashed it with canonical JSON. The SDK had no field to send one. So the example did what the SDK allowed and put the plan in the description.

Finding 2: we logged the wrong name

A receipt carries two identities: a subject that the platform asserts, and the list of approvers whose devices actually signed. The example logged the subject. The probe minted a receipt where the subject was Alice and the device signature was Mallory's. The service deployed and logged “approved by Alice”.

Finding 3: single use lasted until the next restart

The example used the in-memory replay store. Use a receipt, restart the service, present the receipt again inside its ten-minute window: deployed twice on one approval. The store has a guard that refuses to start outside development, but it keys on NODE_ENV=production, and the example never set it.

Finding 4: anyone enrolled could approve

The service had no notion of who may approve a production deploy. Any identity with an enrolled device produced an acceptable receipt. The agent chose which human to ask. And the agent's API key could invite new humans to the tenant, so an agent with an inbox tool could, in principle, enrol its own approver. That last step needs the live platform's email verification, so the audit marked it asserted rather than confirmed, following its own rule about never probing live systems. It still went in the report.

Two things the audit found that are not in the example

The report also surfaced two protocol-level caveats, which is the part we were least pleased to read and the part most worth publishing.

The device signature covers the plan hash but not the decision bit. Approve and deny produce the same device signature; the platform's countersignature carries the result. So a party holding the platform key could re-label a denial as an approval, and the relying party would accept it.

And the verifier always enforces expiry. A receipt is third-party-verifiable evidence for ten minutes. After that an auditor gets RECEIPT_EXPIRED, not a proof, unless there is an audit mode that checks signatures and the hash while skipping expiry and replay. There was not.

What we changed

The fixes landed in one commit across the example, the SDK and the verifier.

The plan is a struct, validated before it is hashed. Every field the executor acts on, and nothing else:

export function deployPlan(p: DeployParams) {
  return {
    service: p.service,
    region: p.region,
    image: p.image,      // digest-pinned, never a tag
    replicas: p.replicas,
    commit: p.commit,
  };
}

The parser rejects unknown fields, non-string values, mutable image tags and replica counts outside 1 to 100, and it runs before anything is hashed, so an array can no longer coerce into the string it prints as. The SDK gained a plan passthrough so the agent sends the same object the service rebuilds.

Policy reads the device-signed list. The allowed-approvers list is enforced on the device-signed approvers, never on the asserted subject. The audit entry records both, kept apart, so a mismatch is visible rather than hidden. The service refuses to start without a policy.

Single use survives restarts. A new zero-dependency replay store on Node's built-in SQLite, so the example has no native build step and no excuse.

The operator keeps a record. Each deploy writes an audit entry with the validated params, the device-signed approvers, the asserted subject, the receipt id and the request id. The receipt itself can be fetched again while the platform holds it.

The tests

Every probe became a test with the assertion flipped. The suite has 19 tests and runs with locally minted receipts against a local key set; nothing talks to the platform.

Attack Before After
Approve one image digest, deploy another deployed 403 PLAN_HASH_MISMATCH
Receipt for the sentence-shaped plan deployed 403 PLAN_HASH_MISMATCH
Array for service, string for replicas, extra field, :latest tag deployed 400 INVALID_DEPLOY_PARAMS
Use a receipt, restart, use it again deployed twice 409 REPLAY
Receipt from a device not in the policy deployed 403 APPROVER_NOT_ALLOWED
sub says the allowed approver, device says someone else deployed, mis-logged 403
Start with no approver policy started refuses to start

What it proves now

Rows 1, 4 and 5 flip. Row 2 flips with them, because the device renders every field of the struct it signs. Row 6 is now operator-grade.

On the strict rule it is still not proves the action, and we are not going to round up. The two protocol caveats are open: the decision bit is not device-signed, and there is no audit mode after expiry. Both are wire-format changes across the platform, two apps and the verifier. They are on the backlog with the report attached. When they ship we will re-run the audit, blind, and publish that too.

What the audit changed about the audit

The report ended with a section of feedback on the skill itself, and most of it went in.

Run it on yours

/plugin marketplace add humanauth-ai/hitl-skills
/plugin install hitl@humanauth

Then ask what your approval step proves. If the answer is a click and you need the action, the report tells you which rung to climb first, and the first four rungs never mention us.