Terraform on Google Cloud: the patterns that hold and the traps in the provider
The google provider has behaviours that will surprise you the first time — authoritative IAM resources that wipe bindings, projects that will not delete, APIs that must be enabled before anything works. Here is the setup we use.
Terraform on Google Cloud is pleasant right up until the afternoon you replace a project's entire IAM policy with a three-line resource. That specific accident has happened to enough teams that it deserves to be the first section.
The IAM resources, in order of danger
The google provider gives you three ways to grant a role, and they are not equivalent.
google_project_iam_member— additive. Grants one role to one member, leaves everything else alone. This is what you want almost always.google_project_iam_binding— authoritative for that role. It sets the complete list of members for one role and removes anyone not in your configuration.google_project_iam_policy— authoritative for the entire project. It replaces every binding on the project with exactly what you wrote.
The last one, applied to a project whose IAM was partly managed by hand or by another team, removes access for everyone you did not list. Including, occasionally, the service account running Terraform, which turns a mistake into a mistake you cannot fix with Terraform.
Use _member unless you specifically need authoritative behaviour, and if you do use _binding, use it only for roles you fully own. The same pattern repeats for buckets, service accounts and every other IAM surface in the provider, with the same three suffixes.
Project and API bootstrapping
Two things bite in the first hour:
APIs must be enabled before resources exist, and enabling an API is itself eventually consistent. Use google_project_service with disable_on_destroy = false — otherwise destroying a workload disables an API that another project's resources were quietly relying on — and expect to need a dependency or a short wait on the first apply.
Projects created by Terraform keep the deletionProtection and lien behaviour you set. Liens are worth using on production projects: a lien makes project deletion fail until someone removes it deliberately, which is a good property for the resource that contains everything else.
Project IDs are globally unique and immutable. Use a naming convention with a random suffix from the start rather than discovering the collision in prod.
State and structure
The layout is the same principle as on AWS: one state per project per layer, remote in a GCS bucket with versioning and object versioning enabled, in the bootstrap project.
GCS backends handle locking natively, so there is no DynamoDB equivalent to set up. Enable bucket versioning before your first apply; recovering a corrupted state file is trivial with it and a reconstruction project without.
Cross-layer references go through data sources. Resist terraform_remote_state pointing at another team's bucket, for the same reason as everywhere else: it is a coupling that nothing makes visible.
Authentication, without keys
Terraform in CI authenticates via Workload Identity Federation, not a service account key. The pipeline's OIDC token is exchanged for a short-lived credential for a Terraform service account, with an attribute condition pinning it to your repository. Humans use gcloud auth application-default login plus impersonation of the same service account for plans, which means the plan runs with production permissions but leaves an audit trail carrying their identity.
This matters more on GCP than elsewhere because the service account key is such a convenient wrong answer — see why it is the credential that leaks.
Provider quirks worth knowing before they cost you a day
googleversusgoogle-beta. Many useful features are beta-only. Configure both providers and use the beta one per resource, rather than switching the whole configuration.- Labels and the provider's own defaults. The provider adds
goog-terraform-provisioned; if you diff labels in policy checks, allow for it. google_compute_instancerecreates on many changes. Check the plan for-/+before applying to anything stateful; machine type changes are in-place, but some disk and metadata changes are not.- Org policies are hierarchical and Terraform-managed ones can fight console-managed ones. Manage them in one place only.
- Service networking connections for private Cloud SQL are shared per VPC and are a classic source of "why does this apply hang" — they take several minutes and do not parallelise well.
The pipeline
Same shape everywhere: fmt, validate, tfsec or checkov, plan on pull request with a read-only identity, manual approval for production, apply from the saved plan. Add gcloud recommender output as a weekly report rather than a gate — recommendations are advice, and advice does not belong in a merge blocker. We set this up at the start of most cloud engagements, usually together with the folder and org policy structure.
What to do this week
Grep your configuration for google_project_iam_policy and _iam_binding. Every hit is a resource that can remove access it does not know about. Confirm each one is intentional, and convert the rest to _member.