Writ

Development

Build and test commands, how Writ is built, and the test plans that need a real host or GitLab.

If you change Writ itself, start here. Read AGENTS.md for the repository rules and Architecture for the package map before you change code. Writing style sets the voice, terms, and rules for documentation.

Commands

CommandWhat it does
just buildBuild bin/writ with CGO_ENABLED=0.
just ciRun every local gate in order and stop at the first that fails: check, e2e, e2e-import, e2e-import-hcl, e2e-aws-emulator, e2e-tofu-s3, verify-mask, policy-parity, test-serve, test-tfstate, test-tofu, website, check-nix-package, and check-nixos. Print one line for each gate with its time. Set TMPDIR to a new directory under /var/tmp unless TMPDIR is set. Needs everything the gates need.
just checkCheck that vendorHash matches the Go modules, then run gofmt, go vet, staticcheck, and go test -race. Run it before every commit.
just e2e-importPlan imports of hashicorp/aws 6.70.0 objects through the importer's provider client, in a namespace with no network. Needs unshare.
just e2e-import-hclTake objects that tofu made into OpenTofu stacks with writ import bulk, and require a tofu plan that only imports and writ import accept passing on it. Needs Nix, nomad, unshare, and ip.
just e2e-aws-emulatorCreate a launch template, a security group, an ALB, its listener, and a listener rule on AWS Emulator with tofu and hashicorp/aws 6.70.0, take them into an OpenTofu stack with writ import bulk, and require a tofu plan that shows exactly 5 imports and writ import accept passing on it. Needs Nix, unshare, ip, and hashicorp/aws 6.70.0 in OpenTofu's plugin cache, which just e2e-aws-emulator-providers fills. x86_64-linux only: elsewhere it skips.
just e2e-aws-emulator-providersFill OpenTofu's plugin cache, TF_PLUGIN_CACHE_DIR or writ-e2e/tofu-plugins in the user cache directory, with hashicorp/aws 6.70.0 from the OpenTofu registry, with every AWS_ variable unset. Needs Nix and the network.
just e2eRun the end-to-end test, with real tofu and hashicorp/random, local, and tls, in namespaces with only loopback: plans by commit under writ serve --dev judged by examples/policy, a commit that policy refuses, and approve, apply, plan again, and a stale plan under a team's writ serve with Postgres. Needs Nix, unshare with subordinate uids, and ip. Fills OpenTofu's plugin cache on first run.
just e2e-tofu-s3Run a team's writ serve on an OpenTofu stack whose state is in a bucket of a local S3 stand-in, versitygw, with an STS stand-in for the plan and apply roles: a plan writes nothing to the bucket, an approved apply writes the state and the index, and a tofu apply outside Writ shows as drift. Needs what just e2e needs.
just test-serveRun the tests of control, tfstate, server, ui, and awsrole with -race against a scratch Postgres, then the provider isolation tests in user and network namespaces. Needs Nix and unshare with subordinate uids.
just test-tfstateRun OpenTofu 1.13.9 against the http state backend of internal/tfstate: plan, apply, a plan's token refused on writes, a second apply refused while one holds the lock, and a stale write refused. Needs Nix, unshare, and ip.
just test-tofuRun internal/tofu against OpenTofu 1.13.9 with random, local, and tls, in namespaces, as writ and as a run uid. Needs Nix and unshare with subordinate uids.
just fuzz-oidcFuzz the OIDC token and key set parsers and the refusals of every flow, 2 minutes each by default: just fuzz-oidc 5m.
just lint-gitlab-ciLint the GitLab CI component and its example with yamllint.
just verify-maskPlan and apply generated OpenTofu stacks twice through writ serve --dev, with two canaries, and require that what is stored and served is the same in both runs and holds neither canary; then fuzz the mask. just verify-mask STACKS FUZZTIME bounds it, 6 stacks and 30s by default. Needs what just e2e needs.
just policy-parityRun conftest verify on examples/policy, then plan three OpenTofu stacks through writ serve --dev judged by examples/policy and again with tofu alone judged by conftest, and require the same verdicts and messages. Needs what just e2e needs.
just protoRegenerate internal/tfplugin5 and internal/tfplugin6 from proto/. Needs Nix for protoc.
just websiteBuild the landing page and the docs site from docs/ into website/out.
just check-nix-packageBuild the flake's writ package. When vendorHash is stale, print the right hash and the line to change. Needs Nix.
just update-nix-hashWrite the right vendorHash into deploy/nix/package.nix and the module checksums into deploy/nix/go-sum.sha256. Needs Nix.
just check-nixosEvaluate the flake's NixOS module, build the writ package its unit runs, and check a sample config with it. Needs Nix.
just release-dry-runBuild every release artifact into dist/ and publish nothing, then install from dist/. Needs Nix. release/README.md has the other release recipes.

just ci is the full gate: run it before you merge to main. Run lint-gitlab-ci when a change touches what it tests.

Hosted CI

GitHub Actions runs part of just ci on a GitHub-hosted Ubuntu runner, with Go and every other tool from the flake.

WorkflowWhenWhat it runs
.github/workflows/ci.ymlA push to main, a pull requestjust build; go vet with the build tags of the e2e, tofu, showcase, record, and failpoint tests; just check with Postgres 18 on PATH, so the control, server, tfstate, and ui tests also run against a scratch Postgres; and the control UI showcase test.
.github/workflows/pages.ymlA push to main that changes website/, docs/, or README.md, or a manual runThe build of just website, then a deploy of website/out to GitHub Pages at writ.mrkaran.dev.
.github/workflows/release.ymlA pushed tag v*The release, as release/README.md describes.

The runner has no user namespaces with subordinate uids, so these gates never run there: e2e, e2e-import, e2e-import-hcl, e2e-aws-emulator, e2e-tofu-s3, verify-mask, policy-parity, test-tfstate, test-tofu, and the provider isolation step of test-serve. In just check, the tests that need a user namespace skip and say so: the provider isolation tests and the runas tests. check-nix-package, check-nixos, and the fuzz recipes don't run either. Run just ci locally for all of them.

The integration tests in just check download hashicorp/random, local, and tls from registry.opentofu.org, so the CI job needs the registry to be reachable.

Go module changes and the Nix package

deploy/nix/package.nix pins vendorHash, the hash of the Go modules that the Nix build downloads. Nix finds that download by vendorHash alone. If go.mod or go.sum changes and vendorHash doesn't, nix build .#writ fails with "inconsistent vendoring" or a hash mismatch.

deploy/nix/go-sum.sha256 holds the SHA-256 of go.mod, go.sum, and flake.lock from when vendorHash was last computed. flake.lock is there because its nixpkgs supplies the Go that vendors the modules. just check compares these checksums with the files and fails on any difference. It doesn't run Nix.

After you change a Go dependency or update flake.lock, run just update-nix-hash and commit deploy/nix/package.nix and deploy/nix/go-sum.sha256 with the change. Only that recipe writes go-sum.sha256, so don't edit the file by hand.

Don't edit proto/tfplugin5.proto or proto/tfplugin6.proto. They are copied from OpenTofu under MPL-2.0.

No test runs apply against a cloud provider, and no test reaches a real cloud host.

How Writ is built

Writ is one Go module with no cgo. cmd/writ parses flags and calls internal/cli. Everything else is in internal/. Architecture has the package map, the path of one plan through the code, and the import rules that internal/archtest enforces on every just check.

The provider protocol

writ import bulk reads objects through its own client of the Terraform plugin protocol, which speaks gRPC in version 5 and version 6 behind one interface. Version 5 is in wide use. On registry.opentofu.org, all 525 releases of hashicorp/aws serve only version 5, and so do the current random, tls, and local.

Writ decodes each version with code generated from its own .proto file, because the two versions disagree on field numbers in the schema messages. A version 5 schema decoded with version 6 types loses the write_only flag and the deprecation messages, with no error.

State

OpenTofu owns a stack's state. A stack on backend "http" {} keeps it in writ serve's control database, sealed under its environment's key, and a stack on backend "s3" keeps it in its own bucket. The control database is SQLite through modernc.org/sqlite, a pure Go driver, or Postgres through pgx. How the control plane works describes both backends.

Tests

Unit tests run against real types. The control, server, tfstate, and ui tests use a real SQLite file, and a scratch Postgres when initdb is on PATH or WRIT_TEST_POSTGRES names a server.

just e2e, just e2e-tofu-s3, and just verify-mask build writ and run it through writ serve and writ serve --dev with OpenTofu stacks, and just e2e-import-hcl through writ import bulk.

The e2e tests that start hashicorp/aws run it in a namespace with no network and with mock credentials. just e2e-import-hcl gives the provider a stand-in of the AWS API that answers fixed replies, which pins exact responses.

just e2e-aws-emulator gives the provider AWS Emulator 1.0.385 instead, so real objects that tofu created are read back as the provider reads them. The flake's aws-emulator package, deploy/nix/aws-emulator.nix, is the vendor's prebuilt Debian package pinned by its SHA-256: a 187 MB download and 1 GB on disk. Only that recipe's nix shell puts it on PATH. The dev shell, nix flake check, and the writ package leave it out. Nix downloads it once, at build time. The test runs the emulator, tofu, and writ only in a user, mount, and network namespace with only loopback, so nothing in it can reach AWS or the internet. tofu there installs providers only from its plugin cache, read as a filesystem mirror, and never asks a registry, so fill the cache first with just e2e-aws-emulator-providers. In the namespace the test mounts an empty tmpfs over ~/.aws, ~/.config/gcloud, ~/.azure, and ~/.docker, binds an empty file over ~/.netrc, in the caller's home and in /root, and masks the files that the caller's AWS_SHARED_CREDENTIALS_FILE and AWS_CONFIG_FILE name. It requires that synthetic credential files in a scratch home, masked the same way, read as absent or empty. The emulator listens on 127.0.0.1, runs with --telemetry off and EMULATOR_TELEMETRY=off, and the test requires its log to say telemetry is disabled. The test removes every AWS_ variable of the caller before it starts anything, sets HOME to a scratch directory, and points the provider at the emulator's own keys and the CA of its self-signed certificate through AWS_SHARED_CREDENTIALS_FILE and AWS_CA_BUNDLE. AWS Emulator 1.0.385 gives a new VPC security group no default egress rule, which the provider revokes on create, so the test untaints the group and applies again when that's the apply's only error, read from tofu apply -json. Most of a run is the provider's wait for the ALB to become active.

Record a real token

The token fixtures in internal/oidc/testdata/tokens were signed in a test, in the claim shapes GitLab and GitHub document. A recorded token replaces the guess with fact. Each file holds a token, the key set that signed it, the clock and audience to check it with, and the check it should fail, "" for none. The test runs every file in the directory.

  1. In a sandbox project, add a job with id_tokens: { WRIT_ID_TOKEN: { aud: https://writ.example.com } } that writes $WRIT_ID_TOKEN to a protected artifact.
  2. Save the key set from https://gitlab.com/oauth/discovery/keys at the same time.
  3. Write a fixture with at set inside the token's life, and run go test ./internal/oidc.

A recorded token is expired by the time it's committed, and it names the sandbox project. Use a project that has nothing in it.

Test plan for a real GitLab project

Each item needs a GitLab project in the owner's sandbox. No test here contacts GitLab.

CheckHowPass when
The claims of a merge request jobRecord a token as aboveIt passes every check, and ref_protected is the string "true" or "false"
The values of user_access_levelRecord tokens of jobs that a developer and a maintainer startThe values are developer and maintainer
The claims of a manual jobRecord the token of a manual job that a second user startsuser_id and user_login identify the user who started it, which approvals rely on
Input interpolation in a job nameInclude the component with stack: network twice, with env: staging and env: prodThe pipeline has writ-plan-network-staging and writ-plan-network-prod
The plan jobRun a merge request pipeline against a writ serve the runner can reach, with the project added as a repositoryThe job, with no checkout, prints the plan of its commit and its id, and the audit log has its call with the job's user

The forge's test plan lists the checks of the forge API and the read token.

Test plan for a real GitLab forge

These checks need the owner's GitLab sandbox. Every test here runs against the stand-in in internal/forge/forgetest, which serves the shapes of GitLab's documents, read 2026-11-10. No test here contacts GitLab.

CheckHowPass when
read_api covers the tree and raw blob endpointsA project access token with Reporter and read_api only, writ repo addThe repository is added and its stacks are listed
A bot user's personal access token on GitLab.com FreeThe same, with the bot user as Reporter of the groupThe same
A service account's tokenThe same, with a GitLab service accountRecord its tier and whether it works
Keyset pagination of the treeA project with over 100 files under the pathEvery file is listed, in more than one page, and the Link header has page_token
The tree lists blob ids that hash to the contentAny projectNo file is refused as tampered
A renamed project keeps its rowRename the project, wait for a pollThe poll works, by the numeric id
A deleted tokenRevoke the token, wait for a pollThe row records the 401, and the message names the read token
A merge request job's shaRun the component's job in a merge request pipelineThe plan is of the merge request's head commit, and the forge has it
The merge base takes a full refA merge request job, whose ref_path is refs/heads/BRANCHThe plan's ref_check is on-ref; the stand-in takes full refs, and GitLab isn't verified to
A merged results pipelineTurn on merged results pipelinesRecord whether the token's sha is fetchable through the commits API
A fork's merge requestOpen one from a fork, with the pipeline in the parent projectRecord the token's project_id, and that the plan is refused when it's the fork's
A stack name with / in a job nameInclude the component with stack: apps/apiThe pipeline has the job writ-plan-apps/api-ENV

Test plan for a real host

The tests here run in user and network namespaces on a developer's machine, with an STS stand-in. These checks need root on a real host, or AWS, and nothing has run them.

CheckHowPass when
The unit's capabilities and the user switch under systemdDeploy the NixOS module with providerUids on an EC2 instance with an instance role, and start writ-serve--check in ExecStartPre prints ok on every isolation line, and both metadata lines say ok, not not checked
The metadata rule on EC2As root on that host, run setpriv --reuid 61000 --regid 61000 --clear-groups curl -m 2 http://169.254.169.254/latest/meta-data/, then the same curl as rootThe first times out, and the second answers
Postgres refuses a provider's uidAs root, setpriv --reuid 61000 --regid 61000 --clear-groups psql "postgres://writ@/writ?host=/run/postgresql"Peer authentication fails
A remote plan with a plan rolePlan an OpenTofu stack with backend "s3" in an environment whose plan_role is a real role of a sandbox accountThe plan passes, and CloudTrail records the AssumeRole with the session name writ-ci-REF and the source identity gitlab-4242-1849213
A trust policy without sts:SetSourceIdentityRemove it from the plan role's trust policy and planThe plan fails with denied and names the role

The website

The site in website/ holds no copy of the docs. website/content.manifest.ts says which file goes on which page, and just website fails when a selector no longer matches its source. website/README.md has the details.

This page is built from docs/development.md in the repository. It is also served as plain Markdown.

On this page