Terraform structure¶
All AWS resources are defined in infra/. Reusable modules hold the resources; each environment is a thin
composition of those modules with its own variables and its own state. Terraform >= 1.5 (CI and deploys pin
1.5.7) with the AWS provider >= 5.70, < 7.0.
1. Layout¶
infra/
bootstrap/ one-time: S3 state bucket + DynamoDB lock table (local state, applied once by the owner)
modules/
network/ VPC, subnets, route tables, NAT, S3 gateway endpoint, interface endpoints (AZs are a variable)
security/ security groups and their rules, KMS key
data/ RDS PostgreSQL 16, parameter group (TLS forced), application secrets
auth/ Cognito user pool, domain, app client for the ALB
edge/ ALB, target group, listeners, ACM certificate + Route 53 records, Cognito rule on the agent host
service/ one ECS service: task definition, Service Connect, task + execution roles, log group
ci/ GitHub OIDC provider and ECR repositories (shared), one deploy role per environment
envs/
develop/ main.tf composes the modules + ECS cluster and Service Connect namespace; terraform.tfvars
prod/ identical main.tf; different terraform.tfvars and state key
2. Modules¶
| Module | Main resources | Key inputs | Key outputs |
|---|---|---|---|
network |
VPC 10.0.0.0/16, 3 subnet tiers × 2 AZs, NAT (one or one per AZ), S3 gateway endpoint, interface endpoints for bedrock-runtime, secretsmanager, ecr.api, ecr.dkr, logs |
AZs, single NAT flag, AZs for interface endpoints | VPC and subnet IDs, endpoint SG, NAT IPs |
security |
ALB, frontend, backend, mock and RDS security groups and the rules between them; endpoint SG rules; KMS key | VPC ID, enable_mocks |
SG IDs, KMS key ARN |
data |
RDS instance, subnet group, parameter group (rds.force_ssl), RDS-managed master secret, generated checkpoint AES key and session HMAC key in Secrets Manager |
Instance class, Multi-AZ, deletion protection, KMS key | Address, DB name/user, secret ARNs |
auth |
Cognito user pool (admin-created users, optional TOTP MFA), hosted UI domain, app client for the ALB | Domain | Pool ARN, client ID, pool domain |
edge |
ALB (idle timeout 300 s), frontend target group (health check /api/healthz), docs target group (/) and, with a docs host, its public host-header rule, HTTP listener, and with a domain: ACM certificate validated in Route 53, alias records for the app and docs hosts, HTTPS listener, Cognito rule on /agent, /agent/*, /api/agent/* |
Domain, Route 53 zone, subnets, SG, Cognito settings | Target group ARN, DNS name, base_url |
service |
ECS service (circuit breaker with rollback), task definition, Service Connect (server or client only), task and execution roles, log group (30 days) | Image, port, CPU/memory, env vars, secrets, desired count, SG, optional target group | Service name |
ci |
GitHub OIDC provider and ECR repositories (created or looked up), deploy role trusted for the listed OIDC subjects | Repository, OIDC subjects, create_shared_resources, state bucket and lock table |
Deploy role ARN, ECR URLs |
The service module is used four times: frontend, backend, docs and mock. The mock instance is created only when
enable_mocks = true.
HTTPS and Cognito depend on a domain. The certificate, HTTPS listener, alias record and Cognito rule are
created only when domain_name is set, and the auth module only exists then. Without a domain the ALB serves
plain HTTP on port 80, so the stack can be planned and applied without one.
3. Environments¶
| Variable | develop | prod |
|---|---|---|
enable_mocks |
true |
false |
domain_name (customers) |
dev.app.onboardassist.click |
app.onboardassist.click |
agent_domain_name (agents, Cognito) |
dev.agent.onboardassist.click |
agent.onboardassist.click |
docs_domain_name |
dev.docs.onboardassist.click |
docs.onboardassist.click |
interface_endpoint_azs |
2a |
2a, 2c |
single_nat_gateway |
true (1 NAT) |
false (1 per AZ) |
desired_count |
1 | 2 |
db_instance_class, db_multi_az, db_deletion_protection |
db.t4g.micro, no, no |
db.t4g.small, yes, yes |
PARTNER_API_URL, IDENTITY_API_URL, CONTRACT_API_URL |
http://mock:8080/partner etc. (Service Connect) |
https://partner.invalid etc. (placeholders until the real systems exist) |
BEDROCK_ENDPOINT_URL |
unset (real Bedrock) | unset (real Bedrock) |
agent_dev_auth |
true (default) |
true (default) |
create_shared_ci_resources |
true: creates the OIDC provider and ECR repositories |
false: looks them up |
github_oidc_subjects |
ref:refs/heads/main |
environment:prod |
External-system addresses are set only in envs/. The backend image is identical in both. image_tag (the
commit SHA) and state_bucket_name are passed by the deploy workflows with -var.
4. State backend¶
infra/bootstrapcreates the state bucketonboarding-tfstate-<account id>(versioned, KMS-encrypted, TLS-only policy, public access blocked) and the DynamoDB lock tableonboarding-terraform-locks. Terraform 1.5 locks S3 state through DynamoDB (use_lockfileneeds 1.10). The owner has applied it once; its own state is local and git-ignored, since a stack cannot store its state in a bucket it creates.- Each environment uses a partial S3 backend: the key (
envs/develop/terraform.tfstate,envs/prod/terraform.tfstate), region, lock table and encryption are inversions.tf; the bucket name contains the account ID and is passed at init:terraform init -backend-config="bucket=<state bucket>". - Develop and prod never share state.
- Local
.terraform/and*.tfstate*files are ignored by git.
5. How it is run¶
| Where | Command |
|---|---|
PR and push to develop or main (CI) |
terraform fmt -recursive -check, then terraform init -backend=false and terraform validate for envs/develop, envs/prod and bootstrap. No plan is run in CI |
Push to develop (deploy) |
terraform apply for envs/develop with the new image_tag. Skipped until the deploy role variable is set |
Push to main (prod) |
terraform apply for envs/prod from deploy-prod.yml, behind approval. Skipped until AWS_DEPLOY_ROLE_ARN_PROD is set (not run for this submission) |
| Locally | cd infra/envs/develop && terraform init -backend-config="bucket=<state bucket>" && terraform plan -var image_tag=<sha> -var state_bucket_name=<state bucket> |
One-time setup¶
These steps come from the comments in infra/ and .github/workflows/. They are done once per AWS account by
someone with admin credentials; after that, deploys run from GitHub Actions.
- Bootstrap state (done):
cd infra/bootstrap && terraform init && terraform apply. Note the outputsstate_bucket_nameandlock_table_name. - Domain (done for develop): register the domain in Route 53, which creates the public hosted zone. Set
domain_nameinenvs/develop/terraform.tfvars. - State bucket name: replace
<ACCOUNT_ID>instate_bucket_namein bothterraform.tfvarsfiles, or pass-var state_bucket_name=...as the workflows do. - First develop apply, by an admin: the deploy role that GitHub assumes is created by
envs/developitself, so the first apply cannot come from the workflow. Runterraform init -backend-config="bucket=<state bucket>"ininfra/envs/develop. The ECS services need images that do not exist yet, so one way is to create the CI resources first:terraform apply -target=module.ci -var image_tag=<any> -var state_bucket_name=<state bucket>. This creates the GitHub OIDC provider, the ECR repositories and the deploy role. Readterraform output deploy_role_arn. - GitHub repository variables (Settings → Secrets and variables → Actions → Variables):
AWS_DEPLOY_ROLE_ARN_DEVELOP= the developdeploy_role_arn,TF_STATE_BUCKET= the bootstrapstate_bucket_name. UntilAWS_DEPLOY_ROLE_ARN_DEVELOPis set,deploy-develop.ymlskips its jobs. - First deploy: push to
develop(or rundeploy-developby hand ondevelop). It builds and pushes the images, applies the full develop stack, waits for the services, and smoke-tests<base_url>/healthz. - Agent accounts: the Cognito pool only allows admin-created users. Create each agent in the console or with
aws cognito-idp admin-create-user. - Prod, when it is time: create the GitHub Environment
prodwith required reviewers and the variableTF_STATE_BUCKET, and set the repository variableAWS_DEPLOY_ROLE_ARN_PROD(a repository variable, since the workflow checks it before entering the environment; until it is set, pushes tomainskip the deploy). The prod deploy role comes from the firstenvs/prodapply, again by an admin. Prod looks up the OIDC provider and ECR repositories that develop created, so develop must exist first.