gcp

---
name: gcp
description: Use for Google Cloud work. Triggers — GKE, Cloud Run, Cloud Functions, BigQuery, Cloud SQL, Spanner, Pub/Sub, Cloud Storage, Firestore, Vertex AI, Terraform on GCP, 'deploy to Google Cloud'.
---

# GCP (Google Cloud Platform)

## When to use
- Designing or reviewing GCP architectures
- IAM / Workload Identity design (least-privilege on GCP)
- Choosing compute: GCE vs GKE vs Cloud Run vs Cloud Functions vs App Engine
- BigQuery schema, partitioning, clustering, and cost control
- Cloud SQL / Spanner / Firestore / Bigtable selection and configuration
- Pub/Sub and Eventarc event-driven patterns
- Cloud Storage bucket design, IAM, lifecycle
- CI/CD with Cloud Build, Artifact Registry, Cloud Deploy
- Cost optimization and budget alerts
- Terraform / Deployment Manager IaC

## Workflow

1. **Classify workload type:** web serving, batch/analytics, streaming, ML training/inference, or data warehousing. Each drives a different primary GCP service combination.

2. **IAM — resource hierarchy first:**
   - GCP hierarchy: Organization → Folder → Project → Resource. Grant permissions at the lowest level possible.
   - Use folders to separate environments (prod, staging, dev) and business units.
   - Prefer predefined roles over basic roles (`Owner`/`Editor`/`Viewer`). Use custom roles for fine-grained needs.
   - Use Workload Identity for GKE pods and Cloud Run services — eliminates service account key files.
   - Service account keys are a security antipattern; avoid outside of local dev tooling.
   - Enable IAM Recommender and apply recommendations regularly.
   - Use VPC Service Controls for sensitive data projects (prevent data exfiltration).
   - Enable Organization Policy constraints: `compute.requireOsLogin`, `compute.vmExternalIpAccess`, `iam.disableServiceAccountKeyCreation`.

3. **Networking:**
   - Use Shared VPC for multi-project environments; one host project owns the network, service projects attach.
   - Cloud NAT for private GKE nodes / Cloud Run to reach the internet outbound without public IPs.
   - Private Google Access: GCE instances in private subnets can reach Google APIs without external IP.
   - Use Private Service Connect or VPC peering for managed service connectivity (Cloud SQL, Memorystore).
   - Cloud Armor: WAF and DDoS protection at the Global Load Balancer layer.
   - Use Google-managed SSL certificates on HTTPS load balancers.

4. **Compute selection:**
   - **Cloud Run:** preferred for stateless HTTP services and event-driven containers. Scales to zero. Supports Knative. Use min-instances to avoid cold starts. Supports gRPC and WebSockets (Cloud Run v2).
   - **GKE Autopilot:** managed Kubernetes without node management. Best for complex microservices needing K8s semantics. Workload Identity is default.
   - **GKE Standard:** when you need specific node machine types, GPU nodes, or fine-grained node management.
   - **Cloud Functions (2nd gen):** event-driven functions backed by Cloud Run. Use for lightweight Pub/Sub consumers, HTTP triggers, Eventarc events.
   - **Compute Engine (GCE):** VMs for stateful workloads, licensed software, custom kernels. Prefer N2D (AMD) or C3 (Intel) over N1 for price-performance. T2A (Arm/Ampere) for ~30% cost saving on compatible workloads.
   - **Batch:** managed job scheduling for HPC/ML batch workloads without GKE overhead.

5. **BigQuery — analytics and data warehouse:**
   - Partition tables by date/timestamp (`PARTITION BY DATE(_PARTITIONTIME)` or a column); required for tables > 1 TB.
   - Cluster on columns used in `WHERE` and `JOIN` (up to 4 columns) — reduces bytes scanned on top of partitioning.
   - Never use `SELECT *` in production queries; explicitly project columns to reduce scanned bytes.
   - Use `INFORMATION_SCHEMA.JOBS_BY_PROJECT` to audit query costs.
   - Use BigQuery Reservations (slot commitments) for predictable high-volume workloads; on-demand for bursty/ad-hoc.
   - Authorized views and row-level security for multi-tenant data sharing without copying datasets.
   - Prefer `MERGE` over `DELETE + INSERT` for upserts; reduces slot consumption.
   - Use BigQuery Omni for querying data in AWS S3 or Azure ADLS without moving it.
   - External tables (BigLake) for querying Parquet/CSV in Cloud Storage without ingestion cost.

6. **Cloud SQL / Spanner / Firestore:**
   - **Cloud SQL (PostgreSQL/MySQL):** managed relational DB. Enable HA (high availability with standby) for production. Use private IP + VPC peering. Set `max_connections` per instance size.
   - **Spanner:** globally distributed, horizontally scalable relational DB. Use when you need > 2 TB, global writes, or 99.999% SLA. Costly; size compute capacity in processing units.
   - **Firestore:** serverless document DB, native mobile/web SDK, offline sync. Good for user-facing apps, configuration, low-latency lookups. Not a replacement for BigQuery.
   - **Bigtable:** wide-column NoSQL for time-series, telemetry, and low-latency high-throughput workloads (>1 TB, millions of ops/s).

7. **Pub/Sub and Eventarc:**
   - Pub/Sub: at-least-once delivery. Set `ackDeadline` ≥ processing time + buffer. Always set a dead-letter topic.
   - Use push subscriptions for Cloud Run / Cloud Functions (HTTP endpoint receives messages); pull for batch workers.
   - Eventarc: routes events from GCP services (Cloud Storage, Audit Logs, Pub/Sub) to Cloud Run, Cloud Functions, GKE targets. Prefer over manual Pub/Sub plumbing for GCP service triggers.
   - Use Pub/Sub Lite for very high throughput at lower cost (zonal, no replication by default).

8. **Cloud Storage:**
   - Enable uniform bucket-level access (disables ACLs); manage access only through IAM.
   - Block public access by default; use signed URLs for time-limited public access.
   - Lifecycle rules: transition to Nearline (30 days), Coldline (90 days), Archive (365 days) for cost reduction.
   - Use CMEK (Cloud KMS) for regulated data. Default encryption is Google-managed.
   - Enable Object Versioning on critical buckets; set lifecycle to clean old versions.
   - Use signed URLs or Cloud CDN with a backend bucket for static asset serving.

9. **Cost controls:**
   - Set budget alerts at 50%, 90%, 100% of monthly budget with email + Pub/Sub notification.
   - Use committed use discounts (CUDs) for sustained GCE/GKE/Cloud SQL workloads (1- or 3-year).
   - BigQuery: use cost controls (`maximumBytesBilled`) in jobs; prefer slot reservations over on-demand for > 100 TB/month.
   - Enable Cloud Recommender for idle resources, right-sizing, and IAM over-provisioning.
   - Delete unused persistent disks, static IPs, and Cloud NAT rules (billed when unattached).
   - Use Cloud Storage nearline/coldline/archive for cold data vs Standard tier.

10. **IaC (Terraform recommended):**
    - Use `google` and `google-beta` Terraform providers. Pin provider versions.
    - Organize: one root module per environment, shared modules for reusable components (VPC, GKE cluster, Cloud SQL).
    - Use Workload Identity Federation in CI/CD (GitHub Actions, GitLab) instead of service account keys.
    - Use `terraform plan` artifact in CI; require human approval for `apply` in prod.
    - Store state in Cloud Storage backend with bucket versioning + locking via Cloud Storage.

## Standards

**Do:**
- Enable Cloud Audit Logs (Admin Activity + Data Access) on all projects.
- Enable Security Command Center (SCC) Premium for threat detection and compliance.
- Use Cloud Armor for all internet-facing HTTPS load balancers.
- Enable Binary Authorization on GKE to enforce signed container images.
- Use Artifact Registry (not Container Registry, which is deprecated) for container and package storage.
- Enable VPC Flow Logs on all subnets for network visibility.
- Tag all resources with `environment`, `team`, `service` labels for cost attribution.

**Do not:**
- Create service account keys for GKE workloads — use Workload Identity.
- Grant `roles/owner` or `roles/editor` to service accounts.
- Use `allUsers` or `allAuthenticatedUsers` on sensitive Cloud Storage buckets.
- Put Cloud SQL instances with public IP without Authorized Networks or Cloud SQL Auth Proxy.
- Use App Engine Standard for new projects (prefer Cloud Run for portability).
- Query BigQuery without partition filters on large partitioned tables (full scan costs).
- Expose GKE master endpoint publicly without authorized networks or private cluster.

## Common mistakes to avoid

- **Missing partition filter in BigQuery:** query against a 10 TB table without `WHERE _PARTITIONDATE = ...` scans all data — can cost hundreds of dollars per query.
- **Service account key files committed to source control:** rotate immediately, audit with Secret Scanner.
- **GKE nodes with public IPs:** use private cluster with Cloud NAT; public IPs are a direct attack surface.
- **Over-provisioning Spanner:** each processing unit is billed whether used or not; start small and monitor CPU utilization (target < 65% for regional, < 45% for multi-regional).
- **Cloud Run cold starts at scale:** set `min-instances: 1` for latency-sensitive services (costs ~$10/month for 1 vCPU instance).
- **Forgetting Cloud CDN for Cloud Run/GCS public content:** without CDN, egress charges accumulate at ~$0.12/GB.
- **Using default compute service account with Editor role:** the default SA has too many permissions; create dedicated SAs per service.

## Output format

- Architecture decisions → `docs/decisions/gcp-<topic>-<date>.md` (use `.claude/templates/decision-record.md`)
- Infrastructure specs → `docs/architecture/overview.md` (use `.claude/templates/architecture.md`)
- BigQuery schema → `docs/data-model/bigquery-<dataset>.md` (use `.claude/templates/data-model.md`)
- Cost model → `docs/specs/cost-model.md` (use `.claude/templates/cost-model.md`)
- Runbooks → `docs/runbooks/<service>.md` (use `.claude/templates/runbook.md`)

## Related checklists
- `.claude/checklists/architecture.md`
- `.claude/checklists/security.md`
- `.claude/checklists/devops.md`
- `.claude/checklists/production.md`
- `.claude/checklists/cost.md`
- `.claude/checklists/observability.md`

## Related agents
- `.claude/agents/core/solution-architect.md`
- `.claude/agents/engineering/cloud-architect.md`
- `.claude/agents/engineering/data-engineer.md`
- `.claude/agents/engineering/devops-engineer.md`
- `.claude/agents/quality/security-auditor.md`
- `.claude/agents/quality/production-readiness-auditor.md`

More agent context in muxammadmamajonov/dot-claude

74 other files this repository gives its agents, the first 60 shown.