AI-assisted delivery guardrails
This page lists the controls that apply to contributions to STOA when an AI assistant writes or reviews part of the change. It only describes what can be read in the public stoa and stoa-docs repositories: contribution rules, git hooks, GitHub Actions workflows, published architecture decisions. Each statement cites the file it comes from; the links are pinned to the inspected revisions in the source revisions section.
An AI-assisted change follows the same path as any other change: branch, pull request, checks on the pull request revision, maintainer review, squash merge, release. The delivery process page describes that path and the component CI workflows. This page focuses on the controls themselves and on what they do and do not prove.
Reading rulesβ
- An architecture decision record explains why a control exists. The workflow file at a given revision defines what runs. The job log of a given run shows what ran. Keep the three apart when assessing a change.
- Branch protection, required-check lists and repository variables are repository settings, not files. The workflow files document the expected values; the settings themselves must be verified on the repository.
- Several controls are advisory or warning-only at the inspected revision. They are listed as such below. A green run is not evidence that an advisory control passed.
Rules that apply to every commitβ
| Rule | Where it is defined | How it is enforced |
|---|---|---|
| Developer Certificate of Origin sign-off | CONTRIBUTING.md (DCO section), CLA.md, DCO text | Required by the contribution rules for every commit (git commit -s). The same requirement is stated in the docs repository contributing rules. |
| Conventional commit messages | CONTRIBUTING.md, commitlint.config.js | Validated locally by the commit-msg hook. Allowed types are listed in the config; scopes are a warning-level rule. |
| Lint on staged files | pre-commit hook | Runs lint-staged before each commit. |
| English for everything committed or published | CONTRIBUTING.md (Language section) | Applies to pull request titles, commit messages, comments, documentation and release notes. |
| Signed commits | required-checks.yml, check-signed-commits.py | Human commits authored at or after the baseline date set in the script are expected to carry a verified signature; bot and service-account commits are exempt. The job runs with continue-on-error, so it is warning-only at this revision. |
| Maintainer review | CODEOWNERS, GOVERNANCE.md | Every path requires review by the maintainers team; workflows, authentication, security, charts, deployment and documentation paths are listed explicitly. Governance requires one maintainer approval and a passing CI before merge, and an architecture decision record for architecture changes. |
| Pull request checklist | PULL_REQUEST_TEMPLATE.md | Declares the type of change, breaking changes, how it was tested, and a quality checklist (tests pass, no new lint warnings, no secrets, commit format). |
| Vulnerability reporting | SECURITY.md | Private reporting channel, acknowledgement and disclosure process, safe-harbor statement. Not a pull request check, but the reference for security findings raised by any reviewer, human or automated. |
Automated checks on every pull requestβ
| Check | Workflow | What it does | Blocking at this revision |
|---|---|---|---|
| CI Gate | ci-gate.yml, aggregation script | Detects changed components, calls their CI workflows and the security workflow, aggregates results, enforces container image pinning. See the delivery process for the per-component checks. | Yes for selected components; unselected components must be skipped. |
| Image Pinning Guard | required-checks.yml, check_image_pinning.py | Requires pinned container images in changed files. | Yes. |
| License Compliance | required-checks.yml, check_licences_offline.py | Offline license check of the dependency tree, with a report artifact. The mode comes from the LICENSES_MODE repository variable; the default warn never fails the job. | Only when the variable is set to deny or enforce. |
| SBOM Generation | required-checks.yml | Produces CycloneDX and SPDX software bills of materials with Trivy and uploads them as artifacts. | Yes (job must complete). |
| Regression Test Guard | regression-guard.yml | Detects fix pull requests and requires an added or modified regression test. Documentation and configuration-only fixes are exempt; the skip-regression label is an explicit override. | Yes, with the documented exemptions. |
| Security Scan | security-scan.yml | Called by the CI Gate on pull requests, and also on pushes to main and daily: SAST for Rust (Clippy), Python (Bandit, medium severity and confidence) and JavaScript (ESLint security plugin, two rules as errors; the zero-warning report step is non-blocking); dependency audits (cargo audit, pip-audit, npm audit); secret scan with Gitleaks on the pull request commit range using .gitleaks.toml; OpSec scan with opsec-scan.sh for infrastructure addresses, financial data, client references, default passwords and key material patterns; Trivy container scan reporting critical and high findings. | SAST, secret and OpSec scans fail the job. The container scan job and several dependency-audit steps carry continue-on-error with a tracked reason in the file. |
| Dependency Review | dependency-review.yml | Fails on newly introduced dependencies with high severity vulnerabilities and denies GPL-3.0, AGPL-3.0 and SSPL-1.0 licenses. | Yes. |
| Purged paths guard | purged-paths-guard.yml, purged-paths-guard.sh | Fails if a path removed from history reappears in the tree or in the commit range. The paths are stored as SHA-256 digests only. | Yes. |
| AI Context Lint | ai-context-lint.yml, ai_context_lint.py | On changes to CLAUDE.md, AGENTS.md, README.md or .claude/, blocks stale markers (retired components, nonexistent paths, outdated stack versions) that an LLM-driven edit could reintroduce. | Yes, on the listed paths. |
| OpenSSF Scorecard | scorecard.yml | Runs on pushes to main and on a schedule; results go to the repository security tab. | Not a pull request check. |
| Dependency updates | dependabot.yml | Weekly grouped minor and patch updates with a chore(deps) prefix; major updates are ignored for the configured ecosystems. | Not a check; produces pull requests that go through the checks above. |
The required-checks.yml header records which jobs the branch protection treats as required at the time of writing in that file. Confirm the list on the repository settings before relying on it.
Controls specific to AI-assisted changesβ
Advisory AI review of pull requestsβ
ADR-065 establishes an advisory-only automated review, the STOA PR Guardian. The workflow, the skill and the policy pack define it:
- Runs when a pull request is opened, marked ready for review or updated. Skipped for draft pull requests, bot authors, pull requests from forks, the
skip-guardianandwiplabels, and when theDISABLE_PR_GUARDIANrepository variable is set. - Diffs above 1000 lines are not reviewed; the workflow posts a notice that human review is required.
- Three axes: architecture decision compliance, security regression, AI code smell. The verdict is binary (GO or NO-GO) with a confidence level. Any security flag promotes the verdict to NO-GO.
- Hard rules in the skill: never approve, never request changes through the review API, comment only, English comments, summary comment starting with a fixed marker so it can be replaced idempotently.
- The agent runs with a restricted tool set (read, glob, grep, shell), a turn limit and a model pinned in the workflow. The run is
continue-on-error: a failure is reported as a comment, not hidden, and does not block the pull request.
The Guardian does not replace maintainer review. It never approves, and the CODEOWNERS rule still applies.
Multi-stage review and the stage 3 code reviewβ
ADR-061 describes a three-stage review: ticket relevance, plan validation, then code review before merge. The stage 3 code review is implemented by council-gate.yml and council-review.sh:
- Four axes evaluated independently on the diff (conformance, debt, attack surface, contract impact), aggregated into a score; the script approves at 8.0 or above and returns a rework exit code below.
- The workflow job is
continue-on-error, documented in the file as shadow mode: results are posted as a pull request comment and do not block the merge. - Skipped for the Dependabot, Claude and GitHub Actions bot accounts and for pull requests from forks; disabled by the
COUNCIL_S3_ENABLEDrepository variable; a missing API key turns the job into a no-op. - The script runs a Gitleaks pre-flight and refuses to send a diff that contains a detected secret.
AI-driven implementation from issuesβ
claude-issue-to-pr.yml lets a repository member hand an issue to an AI agent that opens a pull request. The controls in the file are:
- Stages chained by explicit human commands: ticket relevance review, then a
/gocomment, then plan validation, then a/go-plancomment, then implementation. The/goand/go-plancomments are honored only from users whose association is owner, member or collaborator. Issues labeledship-fast-pathby stage 1 (ship-mode changes estimated at five points or fewer) skip plan validation, and the workflow posts the/go-plancomment itself, as recorded in ADR-061. - A
DISABLE_L1_IMPLEMENTrepository variable disables real-event triggers; manual dispatch runs exactly one stage for controlled replays. - Each stage has a turn limit and a job timeout.
- The resulting pull request goes through the pull request checks listed above. Note that the Guardian and the stage 3 review skip bot-authored pull requests, so those axes fall back to maintainer review for such pull requests.
AI context contractβ
ADR-062, which partially supersedes ADR-030, defines the instruction files that an AI session loads: a root CLAUDE.md plus per-component files, limited to binary rules, with detailed protocols kept in on-demand documents. The rules relevant to delivery in the root file at the inspected revision include:
- A red CI on
mainis the absolute priority; nothing merges and nothing starts until it is green. - The pre-push quality gate is mandatory;
--no-verifyis forbidden outside an explicit emergency. - Pull requests above 300 changed lines must be split.
- A
fixwithout a regression test is blocked; tests are written before code for features and fixes; the API coverage floor is 70 percent and never decreases. - Merges are squash merges with branch deletion; no direct push to
main; force pushes, hard resets and deletion of third-party branches require asking first. - Secrets are never hardcoded; Kubernetes secret manifests use a placeholder;
.env*,*.pem,*.key,*.tfvarsand*credentials*files are never committed. - Credentials are never reset, changed or rotated by the assistant; destructive infrastructure operations require explicit confirmation.
AI Context Lint guards these files against stale content, as listed in the previous section.
Session hooks and restricted subagentsβ
.claude/settings.json wires shell hooks that run inside an AI session before or after tool calls. At the inspected revision the pre-tool hooks include:
- pre-edit-protect-infra.sh: blocks edits to protected infrastructure paths without explicit confirmation.
- pre-edit-no-k8s-secrets.sh: blocks writing values that look like real credentials into Kubernetes Secret manifests or Docker configuration.
- pre-push-quality-gate.sh: before a
git push, detects the changed components and runs their checks, including the OpSec scan; kill-switches are documented at the top of the file.
Subagent definitions in .claude/agents/ declare allowed and disallowed tools: for example the security reviewer cannot write or edit files, and the documentation writer has no shell access.
These hooks run only inside a local AI session that uses this configuration. They are a first line of defense for the operator, not a server-side control; the pull request workflows remain the enforced layer.
AI-written user interface changesβ
ADR-060 decides a four-layer validation for user interface changes produced by AI agents: seeded test data, accessibility-tree assertions, visual regression baselines and an accessibility gate. The corresponding workflows are e2e-a11y-gate.yml and e2e-visual-regression.yml; read their triggers at the revision being assessed, since they are not part of the CI Gate aggregation.
Documentation repository checksβ
Changes to this documentation site go through their own checks in the stoa-docs repository:
- ADR Lint with adr_lint.py: unique ADR numbers, header matching the filename, and a set of blocking consistency rules; missing status or date lines are warnings.
- Docs Audit: a client-name audit over the documentation and a content-compliance scan of blog articles for unsourced comparative or regulatory claims, on pull requests and pushes to
main. - docusaurus.config.ts: broken links, broken anchors and broken markdown links fail the build.
- The same DCO sign-off and conventional commit rules as the main repository, stated in the docs CONTRIBUTING.md.
Limits of the evidenceβ
- Advisory controls (Guardian, stage 3 review) never approve and never block. Maintainer approval under CODEOWNERS and GOVERNANCE.md is the approval step.
- Warning-only controls at this revision: signed-commit verification, license compliance in the default mode, several dependency-audit steps. Read the job summary, not only the job status.
- Repository variables named in the workflows (
DISABLE_PR_GUARDIAN,COUNCIL_S3_ENABLED,DISABLE_L1_IMPLEMENT,LICENSES_MODE) change behavior without a code change. Their values are not visible in the repository. - Nothing on this page is a certification or an attestation. It is a map of where each control is defined so that a reviewer can verify it for a given commit.
Verifying a given changeβ
- Record the pull request, its head commit and the merged commit.
- Open the workflow files at that commit and list the jobs they define for a pull request event.
- Compare that list with the checks reported on the pull request; note any job marked
continue-on-erroror skipped by a label or variable. - For AI-assisted changes, look for the Guardian summary comment and the stage 3 comment, and read the sign-off and signature status of each commit.
- Keep the commit, the run links and the maintainer approval together in the review record.
Source revisionsβ
Links are pinned to the inspected revisions. For another revision, inspect the same paths at that commit.