Modern DevOps Engineering
Chapitre 7
07 — Google Cloud Platform (GCP)
07 — Google Cloud Platform (GCP)
Course: Google Cloud Platform (GCP)
1. Google Kubernetes Engine (GKE)
1.1 Standard vs Autopilot
| Feature | Standard | Autopilot |
|---|---|---|
| Node management | Manual | Fully managed |
| Node auto-repair | Configurable | Always enabled |
| Auto-scaling | Manual node pools | Automatic |
| Pricing | Per node | Per pod |
| Customization | Full (GPU, TPU, local SSDs) | Limited |
| Security | User-managed | Google-managed |
1.2 Cluster Creation
# Standard cluster
gcloud container clusters create production \
--region europe-west1 \
--num-nodes 3 \
--machine-type e2-standard-4 \
--network my-vpc \
--subnet my-subnet \
--enable-private-nodes \
--master-ipv4-cidr 172.16.0.0/28 \
--enable-ip-alias
# Autopilot cluster
gcloud container clusters create-auto autopilot-prod \
--region europe-west1 \
--network my-vpc \
--subnet my-subnet
1.3 Workload Identity
Workload Identity allows Kubernetes service accounts to impersonate GCP service accounts:
# Enable Workload Identity on cluster
gcloud container clusters update production \
--region europe-west1 \
--workload-pool=my-project.svc.id.goog
# Create GCP service account
gcloud iam service-accounts create myapp-sa \
--display-name="MyApp Service Account"
# Bind K8s SA → GCP SA
kubectl annotate serviceaccount myapp-k8s-sa \
--namespace default \
iam.gke.io/gcp-service-account=myapp-sa@my-project.iam.gserviceaccount.com
# Create IAM policy binding
gcloud iam service-accounts add-iam-policy-binding \
myapp-sa@my-project.iam.gserviceaccount.com \
--role roles/iam.workloadIdentityUser \
--member "serviceAccount:my-project.svc.id.goog[default/myapp-k8s-sa]"
2. Cloud Run
2.1 Service Deployment
Cloud Run runs stateless HTTP containers in a fully managed environment.
# Deploy a container
gcloud run deploy my-service \
--image gcr.io/my-project/myapp:latest \
--region europe-west1 \
--platform managed \
--allow-unauthenticated \
--memory 512Mi \
--cpu 1 \
--concurrency 80 \
--timeout 300 \
--set-env-vars "NODE_ENV=production,LOG_LEVEL=info" \
--set-secrets "API_KEY=api-key:latest" \
--service-account myapp-sa@my-project.iam.gserviceaccount.com \
--vpc-connector my-connector \
--vpc-egress private-ranges-only
2.2 Cloud Run YAML
apiVersion: serving.knative.dev/v1
kind: Service
metadata:
name: my-service
annotations:
run.googleapis.com/ingress: internal
run.googleapis.com/ingress-status: internal
spec:
template:
metadata:
annotations:
autoscaling.knative.dev/maxScale: "100"
autoscaling.knative.dev/minScale: "0"
run.googleapis.com/vpc-access-connector: projects/p/connectors/c
run.googleapis.com/execution-environment: gen2
spec:
containerConcurrency: 80
timeoutSeconds: 300
serviceAccountName: myapp-sa@my-project.iam.gserviceaccount.com
containers:
- image: gcr.io/my-project/myapp:latest
ports:
- containerPort: 8080
env:
- name: NODE_ENV
value: production
resources:
limits:
cpu: "1"
memory: 512Mi
3. BigQuery
3.1 Querying Data
BigQuery is a serverless, highly scalable data warehouse.
-- Standard SQL
SELECT
DATE(timestamp) as day,
COUNT(*) as requests,
APPROX_QUANTILES(latency, 100)[OFFSET(95)] as p95_latency
FROM `my-project.my_dataset.api_logs`
WHERE timestamp >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 7 DAY)
AND status >= 500
GROUP BY day
ORDER BY day DESC;
-- Create a table from query results
CREATE TABLE `my-project.my_dashboard.daily_errors`
AS
SELECT DATE(timestamp) as day, COUNT(*) as errors
FROM `my-project.my_dataset.api_logs`
WHERE status >= 500
GROUP BY day;
3.2 BigQuery Best Practices
- Partitioning: Partition by date for query cost reduction
- Clustering: Cluster by frequently-filtered columns
- Materialized views: Pre-compute aggregations
- Wildcard tables: Query multiple tables with
TABLE_DATE_RANGE - BI Engine: In-memory acceleration for dashboards
- Slot reservations: Guaranteed compute capacity
4. Cloud Storage
4.1 Bucket Types and Lifecycle
# Create bucket with uniform access
gsutil mb -l EUROPE-WEST1 \
-c STANDARD \
--uniform-bucket-level-access \
gs://myapp-data/
# Set lifecycle policy
cat > lifecycle.json <<EOF
{
"rule": [
{
"action": { "type": "SetStorageClass", "storageClass": "NEARLINE" },
"condition": { "age": 30, "matchesStorageClass": ["STANDARD"] }
},
{
"action": { "type": "SetStorageClass", "storageClass": "COLDLINE" },
"condition": { "age": 90 }
},
{
"action": { "type": "Delete" },
"condition": { "age": 365 }
}
]
}
EOF
gsutil lifecycle set lifecycle.json gs://myapp-data/
4.2 Object Versioning and Retention
# Enable versioning
gsutil versioning set on gs://myapp-data/
# Set retention policy (WORM)
gsutil retention set 365d gs://myapp-data/
# Enable encryption with CMEK
gsutil kms authorize -k projects/p/locations/europe-west1/keyRings/kr/cryptoKeys/ck
gsutil kms encryption -k projects/p/locations/europe-west1/keyRings/kr/cryptoKeys/ck gs://myapp-data/
5. VPC Networking
5.1 Shared VPC
Shared VPC allows provisioning subnets from a common host project:
# Host project setup
gcloud compute shared-vpc enable host-project \
--host-project host-project-id
# Attach service projects
gcloud compute shared-vpc associated-projects add \
--host-project host-project-id \
--service-project service-project-id
5.2 Cloud NAT
Cloud NAT enables outbound internet for private instances:
# Create Cloud NAT
gcloud compute routers create nat-router \
--network my-vpc \
--region europe-west1
gcloud compute routers nats create nat-config \
--router nat-router \
--region europe-west1 \
--auto-allocate-nat-external-ips \
--nat-all-subnet-ip-ranges
6. IAM
6.1 Roles and Conditions
# Predefined role
gcloud projects add-iam-policy-binding my-project \
--member "user:alice@example.com" \
--role "roles/container.clusterAdmin"
# Custom role
gcloud iam roles create custom.storageViewer \
--project my-project \
--title "Custom Storage Viewer" \
--permissions "storage.objects.get,storage.objects.list" \
--stage GA
# Conditional IAM
gcloud projects add-iam-policy-binding my-project \
--member "group:devops@example.com" \
--role "roles/compute.instanceAdmin" \
--condition "expression=resource.name.startsWith('projects/_/instances/dev-'),title=dev_only"
6.2 Service Accounts
# Create SA
gcloud iam service-accounts create myapp-sa \
--display-name "MyApp SA"
# Key management
gcloud iam service-accounts keys create myapp-sa-key.json \
--iam-account myapp-sa@my-project.iam.gserviceaccount.com
# Short-lived credentials
gcloud auth print-access-token \
--impersonate-service-account myapp-sa@my-project.iam.gserviceaccount.com
7. Cloud Build & Cloud Deploy
7.1 Cloud Build Configuration
# cloudbuild.yaml
steps:
- name: 'gcr.io/cloud-builders/docker'
args: ['build', '-t', 'europe-west1-docker.pkg.dev/$PROJECT_ID/my-repo/$_SERVICE_NAME:$COMMIT_SHA', '.']
- name: 'gcr.io/cloud-builders/docker'
args: ['push', 'europe-west1-docker.pkg.dev/$PROJECT_ID/my-repo/$_SERVICE_NAME:$COMMIT_SHA']
- name: 'gcr.io/google.com/cloudsdktool/cloud-sdk'
entrypoint: 'gcloud'
args:
- 'run'
- 'deploy'
- '$_SERVICE_NAME'
- '--image=europe-west1-docker.pkg.dev/$PROJECT_ID/my-repo/$_SERVICE_NAME:$COMMIT_SHA'
- '--region=europe-west1'
- '--platform=managed'
substitutions:
_SERVICE_NAME: myapp
options:
logging: CLOUD_LOGGING_ONLY
timeout: 1200s
7.2 Cloud Deploy Delivery Pipeline
# clouddeploy.yaml
apiVersion: deploy.cloud.google.com/v1
kind: DeliveryPipeline
metadata:
name: myapp-pipeline
serialPipeline:
stages:
- targetId: dev
profiles: [dev]
- targetId: staging
profiles: [staging]
- targetId: prod
profiles: [prod]
---
apiVersion: deploy.cloud.google.com/v1
kind: Target
metadata:
name: dev
description: Development cluster
run:
location: projects/my-project/locations/europe-west1
---
apiVersion: deploy.cloud.google.com/v1
kind: Target
metadata:
name: staging
description: Staging cluster
gke:
cluster: projects/my-project/locations/europe-west1/clusters/staging
---
apiVersion: deploy.cloud.google.com/v1
kind: Target
metadata:
name: prod
description: Production requires approval
requireApproval: true
gke:
cluster: projects/my-project/locations/europe-west1/clusters/prod
8. Cloud Monitoring
# Create uptime check
gcloud monitoring uptime create myapp-uptime \
--hostname "myapp.example.com" \
--resource-type "https://myapp.example.com/health" \
--period 5m \
--timeout 10s
# Create alert policy
gcloud alpha monitoring policies create \
--policy-from-file="alert-policy.yaml"
# Create dashboard
gcloud monitoring dashboards create \
--config-from-file="dashboard.json"
Summary
GCP excels in Kubernetes (GKE with Autopilot), serverless (Cloud Run), data analytics (BigQuery), and CI/CD (Cloud Build + Cloud Deploy). Shared VPC provides network isolation, Workload Identity securely connects Kubernetes to GCP services, and Cloud Monitoring provides observability.