CI verification for agent deliverables Module

Vaks PM · Integration guide · Signed CI results · July 2026

What you will end up with. When an AI agent submits a deliverable and claims its build passed, Vaks PM no longer takes its word for it — a signed result from your CI system decides. The agent's own verdict is discarded in favour of the signed one. This closes the gap where an agent could mark its own work as passing.

What it does — and what it does not

An agent that produces work also reports whether it verified that work. Left alone, that is self-attestation: the agent grades its own homework. CI verification replaces the agent's claim with independent, cryptographically signed evidence from the system that actually ran the build.

Vaks PM never reaches out. It has no outbound access to your CI. Instead your CI pushes a signed result to a webhook, and Vaks PM verifies the signature. Nothing here opens an outbound path from your instance.
The signed verdict overrides the agent. When a deliverable declares the ci_signed method, the status the agent reported is ignored; the matched CI result is used instead, both when the deliverable is submitted and again at the review gate.
It corroborates, it does not fetch. Vaks PM holds no build logs and runs nothing. It only checks whether a signed pass/fail arrived for the right commit.
Every rejection is silent to the caller. The endpoint always answers 202, even for a bad signature or an unknown tenant — deliberately, so it cannot be used to probe your instance or trigger retry storms. The only place a rejection shows is the connector's status line and the audit log.

Prerequisites

Side	What you need
Your CI	GitHub Actions, or any CI able to POST a JSON body with an HMAC signature. It must be able to reach your Vaks PM instance — this is an inbound call, so a fully private instance needs the CI to be on the same network or reachable through your own gateway.
Vaks PM	The `agent:manage` permission — an organization administrator, or a user holding the AI administrator grant. This is the same permission that governs agents themselves, so whoever runs your AI programme can set it up without being a full administrator.
A shared secret	A string you generate, set on both sides. It never travels; it is only used to sign and verify.

No licence needed, and agents cannot touch it. CI verification is not licence-gated. And because agent:manage can never be held by an agent, no agent can create or alter the connector that judges its own work.

Step 1 — Create the connector in Vaks PM

Open Admin → Integrations → CI verification and press Add a connector. Choose the signer type:

GitHub Actions — it reads GitHub's native webhook events and its X-Hub-Signature-256 header.
Generic CI (HMAC) — for anything else; it reads a flat JSON body and an X-Vaks-CI-Signature header.

Fill in:

Field	What to put
Label	Free text, for the connector list.
Shared HMAC secret	The secret you will also set on the CI webhook. Encrypted at rest, never shown again — leave blank when editing to keep the stored one.
Enabled (accept incoming results)	Tick it. A disabled connector rejects every result as if it did not exist.

Save, then press Copy URL on the connector's card. That is the address your CI posts to:

https://<your-host>/api/v1/webhooks/ci/<provider>

where <provider> is github or generic. Only one connector of each type exists per organization.

Post to the tenant's own hostname. The organization is identified from the host in the URL, not from anything in the body. An IP address or a shared hostname resolves to no tenant and the result is dropped — you would see 202 and nothing would happen. If you run several organizations, each has its own connector, secret and URL.

Step 2 — In GitHub

Add a repository (or organization) webhook: Settings → Webhooks → Add webhook.

Payload URL: the URL you copied, ending in /github.
Content type: application/json. This matters — any other content type means the raw body is never captured and the signature will never match.
Secret: the exact shared secret from step 1.
Events: choose individual events and select Workflow runs, Check suites, or Statuses — whichever your pipeline emits. Vaks PM reads all three.

GitHub signs the body with X-Hub-Signature-256 automatically once the secret is set; there is nothing to configure on the signature itself. In-progress runs and ping events are accepted and ignored — only a completed run carries a verdict.

Step 2b — A generic CI signer

If you are not on GitHub, have your pipeline POST a flat JSON body to the /generic URL, signed with HMAC-SHA256 of the raw body, in the X-Vaks-CI-Signature header as sha256=<hex>.

# Body
{ "commitSha": "<full-sha>", "conclusion": "success", "runUrl": "https://ci.example.com/run/123" }

# Header
X-Vaks-CI-Signature: sha256=<hmac-sha256 hex of the raw body, keyed with the shared secret>

A minimal signer, in shell:

BODY='{"commitSha":"'"$GIT_SHA"'","conclusion":"'"$RESULT"'"}'
SIG=$(printf '%s' "$BODY" | openssl dgst -sha256 -hmac "$SECRET" -hex | sed 's/^.* //')
curl -sS -X POST "$VAKS_CI_URL" \
  -H 'Content-Type: application/json' \
  -H "X-Vaks-CI-Signature: sha256=$SIG" \
  --data-raw "$BODY"

What counts as a pass. Exactly one thing: conclusion equal to success (case-insensitive). Every other value — failure, cancelled, timed_out, anything — is a fail. The field may also be named status or state, and the commit may be commitSha, commit_sha or sha. A missing commit or a null verdict means the result is ignored, not failed.

Step 3 — Require it on a project

A connector on its own only records signed results. For a result to actually gate a deliverable, two things line up:

The deliverable declares the ci_signed verification method — the agent does this when it submits, and includes the commit it built.
The project's verification policy calls for that level of proof. This is set on the project's AI tab, where you also choose between report only (the gate observes and records but does not block) and enforce (the gate can hold or reject the deliverable).

Start in report only to see corroboration flowing without affecting reviews, then switch to enforce once you trust the signal.

It never deadlocks. If a project demands signed CI but has no mechanism able to provide it, the requirement escalates to human review rather than blocking forever. A deliverable is never stuck waiting for proof that cannot arrive.

Step 4 — Verify the setup

Trigger a CI run so a signed result is sent, then check the connector's card in Admin → Integrations → CI verification:

Last event shows a recent timestamp and no error — the result arrived and the signature verified.
A Last event rejected pill with a reason means the signature failed or the connector is misconfigured. This is the only visible signal, since the endpoint always answers 202.

Then run the real path once: have an agent submit a deliverable with the matching commit, and confirm in the review screen that the CI verdict is shown alongside it — provider, conclusion and a link to the run.

How results are matched to a deliverable

Understanding this removes most of the confusion:

Matching is by commit SHA, exactly. The result is paired to a deliverable whose commit equals the one signed. A short SHA against a full SHA will not match — send the full one.
A signed result is buffered for 24 hours. The CI can sign before the deliverable even exists; that is the point. But if the deliverable is reviewed more than 24 hours after the run, the buffered result has expired and the gate falls back to waiting.
Nothing is consumed. A deliverable can be re-evaluated any number of times against the same buffered result; the most recent non-expired result for the commit wins.
The verdict is re-checked at review. A result that arrives after the agent submitted is still picked up when the reviewer opens the gate — the frozen submitted status is never trusted.

An agent that lies gets caught and demoted. If an agent declared a pass, a human then requests changes, and the signed verdict was below the required bar, the agent's trust level is automatically lowered one notch and a critical audit entry records the mismatch. Over-attestation has a cost.

Troubleshooting

The endpoint answers 202 no matter what, so the connector card and the audit log are your only windows. Rejection reasons appear on the card's Last event line.

Symptom	Cause & fix
Nothing arrives, card never updates	Wrong hostname (posted to an IP or shared host → no tenant), or the wrong provider path. Confirm the URL ends in `/github` or `/generic` and uses the tenant's own domain.
Last event rejected · signature	The secret differs between the two sides, or — the classic — the content type is not `application/json`, so the raw body was never captured. Re-set the secret and fix the content type.
Last event rejected · no connector	The connector is disabled, has no secret, or its stored secret became unreadable after an encryption-key rotation. Re-enter the secret and enable it.
Results arrive but the gate stays pending	The commit does not match: short vs full SHA, or the deliverable carries a different commit. It can also mean the review happened more than 24 hours after the run — the buffer expired.
GitHub shows the webhook delivered, Vaks PM ignored it	The event was a `ping` or an in-progress run — accepted but not a verdict. Only completed runs carry a conclusion.

What is logged

Under Admin → Security & Compliance → Audit log: creating, editing and deleting a connector at critical severity; each accepted signed result as ci_result.received; each rejection as ci_webhook.rejected with the reason. When a deliverable is evaluated, the entry records whether CI corroborated it, the conclusion, and — if an agent over-attested — the trust downgrade. The shared secret is never written to the log.

Related: agent directories for the identities being governed · outbound webhooks, the reverse direction · all integrations.