Chapitre 15 : Platform Engineering

15.1 Introduction au Platform Engineering

15.1.1 Qu'est-ce que le Platform Engineering ?

Le Platform Engineering est la discipline qui consiste a concevoir, construire et maintenir une Internal Developer Platform (IDP). L'IDP est un ensemble d'outils, de services et de processus qui permet aux equipes de developpement de livrer des applications plus rapidement et de maniere autonome.

Probleme resolu : Sans platform engineering, chaque equipe de developpement cree sa propre infrastructure, son pipeline CI/CD, son monitoring... Ceci conduit a :

• Duplication des efforts
• Inconsistance entre les equipes
• Cout d'operation eleve
• Difficultes de gouvernance

Avec une IDP :

• Self-service pour les equipes
• Golden paths pre-definis
• Governance et securite integrees
• Reduction de la charge cognitive

15.1.2 Internal Developer Platform (IDP)

┌─────────────────────────────────────────┐
│         Developer Self-Service          │
│  (Backstage / Portal)                   │
├─────────────────────────────────────────┤
│           Golden Paths                  │
│  (Templates, Scaffolder)               │
├─────────────────────────────────────────┤
│       Infrastructure Abstraction        │
│  (Crossplane, Terraform Operator)       │
├─────────────────────────────────────────┤
│       Runtime / Orchestration           │
│  (Kubernetes, Service Mesh)            │
├─────────────────────────────────────────┤
│         Observability Platform          │
│  (Prometheus, Grafana, Loki, Tempo)    │
├─────────────────────────────────────────┤
│              Cloud Providers             │
│  (AWS, GCP, Azure)                     │
└─────────────────────────────────────────┘

15.1.3 Platform vs Produit

Une platform interne doit etre traitee comme un produit :

15.2 Backstage

15.2.1 Architecture

Backstage est une platform developer portal creee par Spotify, maintenant CNCF project :

┌──────────────────────────────────────────┐
│           Backstage App                  │
│  ┌─────────┐ ┌─────────┐ ┌─────────┐   │
│  │ Catalog │ │Scaffolder│ │  Tech   │   │
│  │  Plugin │ │  Plugin  │ │  Docs   │   │
│  └─────────┘ └─────────┘ └─────────┘   │
│  ┌─────────┐ ┌─────────┐ ┌─────────┐   │
│  │    ...  │ │  Search │ │   API   │   │
│  └─────────┘ └─────────┘ └─────────┘   │
├──────────────────────────────────────────┤
│          Plugin Backend                  │
│  (Catalog, Scaffolder, Search, Auth)    │
├──────────────────────────────────────────┤
│          Infrastructure                  │
│  (PostgreSQL, Elasticsearch, Redis)      │
└──────────────────────────────────────────┘

15.2.2 Installation

# Creer l'application Backstage
npx @backstage/create-app@latest

# Configurer les variables d'environnement
cat <<EOF > .env
POSTGRES_HOST=localhost
POSTGRES_PORT=5432
POSTGRES_USER=backstage
POSTGRES_PASSWORD=backstage
GITHUB_TOKEN=your_github_token
EOF

# Lancer Backstage
yarn dev

15.2.3 Software Catalog

Le software catalog est le cœur de Backstage :

# catalog-info.yaml
apiVersion: backstage.io/v1alpha1
kind: Component
metadata:
  name: myapp-api
  description: "API service for MyApp"
  annotations:
    github.com/project-slug: myorg/myapp-api
    backstage.io/techdocs-ref: dir:.
    prometheus.io/scrape: "true"
    grafana/alerting: "true"
spec:
  type: service
  lifecycle: production
  owner: team-alpha
  system: myapp-platform
  dependsOn:
    - component:default/myapp-database
  providesApis:
    - myapp-api
---
apiVersion: backstage.io/v1alpha1
kind: Component
metadata:
  name: myapp-database
  description: "PostgreSQL database for MyApp"
spec:
  type: database
  lifecycle: production
  owner: team-alpha
  system: myapp-platform
---
apiVersion: backstage.io/v1alpha1
kind: API
metadata:
  name: myapp-api
  description: "MyApp REST API"
spec:
  type: openapi
  lifecycle: production
  owner: team-alpha
  definition:
    $text: https://github.com/myorg/myapp-api/blob/main/openapi.yaml
---
apiVersion: backstage.io/v1alpha1
kind: Group
metadata:
  name: team-alpha
  description: "Team Alpha - Platform Engineering"
spec:
  type: team
  profile:
    displayName: Team Alpha
    email: team-alpha@company.com
  children: []
  members:
    - alice
    - bob
---
apiVersion: backstage.io/v1alpha1
kind: User
metadata:
  name: alice
spec:
  profile:
    displayName: Alice Johnson
    email: alice@company.com
  memberOf:
    - team-alpha

15.2.4 Software Templates (Scaffolder)

Les templates permettent aux developpeurs de creer des projets pre-configures :

# templates/nodejs-api-template.yaml
apiVersion: scaffolder.backstage.io/v1beta3
kind: Template
metadata:
  name: nodejs-api
  title: "Node.js API Service"
  description: "Create a new Node.js API with Express, tests, CI/CD"
  tags:
    - nodejs
    - express
    - api
spec:
  owner: platform-team
  type: service

  parameters:
    - title: "Service Details"
      required:
        - name
        - owner
      properties:
        name:
          title: "Service Name"
          type: string
          description: "Unique name of the service"
        owner:
          title: "Owner"
          type: string
          description: "Team owning this service"
          ui:field: OwnerPicker
        description:
          title: "Description"
          type: string
          description: "Description of the service"

  steps:
    - id: template
      name: "Generate Service"
      action: fetch:template
      input:
        url: ./skeleton
        values:
          name: ${{ parameters.name }}
          owner: ${{ parameters.owner }}
          description: ${{ parameters.description }}

    - id: publish
      name: "Publish to GitHub"
      action: publish:github
      input:
        repoUrl: github.com?owner=myorg&repo=${{ parameters.name }}
        defaultBranch: main

    - id: register
      name: "Register in Catalog"
      action: catalog:register
      input:
        repoContentsUrl: ${{ steps.publish.output.repoContentsUrl }}
        catalogInfoPath: /catalog-info.yaml

  output:
    links:
      - title: "Repository"
        url: ${{ steps.publish.output.remoteUrl }}
      - title: "Open in Catalog"
        icon: catalog
        entityRef: ${{ steps.register.output.entityRef }}

15.2.5 TechDocs

TechDocs est le systeme de documentation dans Backstage :

# mkdocs.yml
site_name: myapp-api
site_description: "MyApp API Documentation"
repo_url: https://github.com/myorg/myapp-api

nav:
  - Home: index.md
  - Getting Started: getting-started.md
  - Architecture: architecture.md
  - API Reference: api.md
  - Operations: operations.md
  - Runbook: runbook.md

plugins:
  - techdocs-core

15.3 Crossplane

15.3.1 Architecture

Crossplane est un framework open-source pour la gestion d'infrastructure :

Developer ──> Claim (RDS Instance) ──> Composition ──> Managed Resources
                                   │                        │
                              XR (Composite Resource)   AWS/GCP/Azure

Composants Crossplane :

• Provider : Plugin pour un cloud (AWS, GCP, Azure)
• Managed Resource : Ressource cloud (RDS, S3, VPC)
• Composite Resource (XR) : Ressource composee
• Composition : Definition du composite
• Claim : Demande d'infrastructure par le developpeur

15.3.2 Provider Configuration

# provider-aws.yaml
apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
  name: provider-aws
spec:
  package: xpkg.upbound.io/crossplane-contrib/provider-aws:v0.47.0
---
apiVersion: aws.upbound.io/v1beta1
kind: ProviderConfig
metadata:
  name: default
spec:
  credentials:
    source: Secret
    secretRef:
      namespace: crossplane-system
      name: aws-creds
      key: creds

15.3.3 Compositions

# composition-rds.yaml
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
  name: xrds.postgres.infra.example.org
spec:
  writeConnectionSecretsToNamespace: crossplane-system
  compositeTypeRef:
    apiVersion: infra.example.org/v1alpha1
    kind: XRDS
  resources:
    - name: rds-instance
      base:
        apiVersion: rds.aws.upbound.io/v1beta1
        kind: Instance
        spec:
          forProvider:
            engine: postgres
            engineVersion: "15"
            instanceClass: db.t3.small
            allocatedStorage: 20
            dbName: myapp
            username: admin
            passwordSecretRef:
              name: db-secret
              namespace: crossplane-system
      patches:
        - type: FromCompositeFieldPath
          fromFieldPath: spec.parameters.storageGB
          toFieldPath: spec.forProvider.allocatedStorage
        - type: FromCompositeFieldPath
          fromFieldPath: spec.parameters.instanceClass
          toFieldPath: spec.forProvider.instanceClass
        - type: ToCompositeFieldPath
          fromFieldPath: status.atProvider.endpoint
          toFieldPath: status.endpoint

    - name: db-subnet-group
      base:
        apiVersion: rds.aws.upbound.io/v1beta1
        kind: SubnetGroup
        spec:
          forProvider:
            subnetIds: []
      patches:
        - type: FromCompositeFieldPath
          fromFieldPath: spec.parameters.subnetIds
          toFieldPath: spec.forProvider.subnetIds

15.3.4 Composite Resource Definition (XRD)

# xrd-rds.yaml
apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
metadata:
  name: xrds.infra.example.org
spec:
  group: infra.example.org
  names:
    kind: XRDS
    plural: xrds
  claimNames:
    kind: RDSClaim
    plural: rdsclaims
  versions:
    - name: v1alpha1
      served: true
      referenceable: true
      schema:
        openAPIV3Schema:
          type: object
          properties:
            spec:
              type: object
              properties:
                parameters:
                  type: object
                  properties:
                    storageGB:
                      type: integer
                      default: 20
                    instanceClass:
                      type: string
                      default: db.t3.small
                    subnetIds:
                      type: array
                      items:
                        type: string
                  required:
                    - subnetIds

15.3.5 Claims (pour les developpeurs)

# claim-rds.yaml
apiVersion: infra.example.org/v1alpha1
kind: RDSClaim
metadata:
  name: myapp-database
  namespace: myapp
spec:
  parameters:
    storageGB: 50
    instanceClass: db.t3.medium
    subnetIds:
      - subnet-abc123
      - subnet-def456
  writeConnectionSecretToRef:
    name: db-connection

15.4 Golden Paths

15.4.1 Concept

Les golden paths sont des chemines pre-definies et valides pour les taches courantes :

Golden Path pour deployer un microservice :

1. Backstage Template : Creer le projet
2. GitHub Repository : Code + CI/CD
3. Crossplane Claim : RDS, Redis
4. ArgoCD Application : Deploiement
5. Grafana Dashboard : Monitoring
6. PagerDuty : On-call

15.4.2 Golden Path Template Backstage

# golden-path-microservice.yaml
apiVersion: scaffolder.backstage.io/v1beta3
kind: Template
metadata:
  name: golden-path-ms
  title: "Golden Path - Microservice"
  description: "Standard microservice with DB, monitoring, CI/CD"
spec:
  owner: platform-team
  type: service

  parameters:
    - title: "Microservice Configuration"
      required:
        - serviceName
        - team
        - database
      properties:
        serviceName:
          title: "Service Name"
          type: string
        team:
          title: "Team"
          type: string
          ui:field: OwnerPicker
        database:
          title: "Database Required"
          type: boolean
          default: true
        databaseSize:
          title: "Database Size"
          type: string
          enum: ["small", "medium", "large"]
          default: "small"

  steps:
    - id: scaffold
      name: "Scaffold Service"
      action: fetch:template
      input:
        url: ./skeleton
        values:
          serviceName: ${{ parameters.serviceName }}

    - id: create-repo
      name: "Create GitHub Repository"
      action: publish:github
      input:
        repoUrl: github.com?owner=${{ parameters.team }}&repo=${{ parameters.serviceName }}

    - id: deploy-database
      name: "Deploy Database"
      action: crossplane:claim
      input:
        claimTemplate: templates/rds-claim.yaml
        values:
          name: ${{ parameters.serviceName }}-db
          size: ${{ parameters.databaseSize }}

    - id: create-argocd-app
      name: "Create ArgoCD Application"
      action: argocd:create-application
      input:
        appName: ${{ parameters.serviceName }}
        repoUrl: https://github.com/${{ parameters.team }}/${{ parameters.serviceName }}
        namespace: ${{ parameters.team }}

15.5 Platform Teams

15.5.1 Structure d'equipe

Le modele "Team Topologies" recommande :

┌─────────────────────────────────────────────┐
│          Stream-aligned Teams               │
│  (Feature teams - livrent de la valeur)     │
├─────────────────────────────────────────────┤
│                   │                          │
│         ┌─────────┴─────────┐               │
│         │                   │               │
│   Enabling Team      Complicated             │
│   (Conseil,           Subsystem               │
│    formation)         Team                   │
│         │            (Expertise              │
│         │             technique)             │
│         └─────────┬─────────┘               │
│                   │                          │
├─────────────────────────────────────────────┤
│           Platform Team                      │
│  (IDP, Backstage, Crossplane, CI/CD)        │
└─────────────────────────────────────────────┘

15.5.2 Responsabilites de la Platform Team

platform_team_responsibilities:
  core_platform:
    - "Internal Developer Platform (Backstage)"
    - "CI/CD pipelines (GitHub Actions, GitLab CI)"
    - "GitOps (ArgoCD, Flux)"
    - "Artifact registry (Harbor, ECR)"
    
  infrastructure:
    - "Kubernetes clusters (EKS, GKE)"
    - "Cloud networking (VPC, DNS, TLS)"
    - "Database as a Service (Crossplane)"
    - "Service Mesh (Istio)"
    
  observability:
    - "Monitoring (Prometheus, Grafana)"
    - "Logging (Loki, ELK)"
    - "Tracing (Tempo, Jaeger)"
    - "Alerting (Alertmanager, PagerDuty)"
    
  security:
    - "Secrets management (Vault)"
    - "Policy as Code (Kyverno)"
    - "Image signing (Cosign)"
    - "Vulnerability scanning (Trivy)"
    
  developer_experience:
    - "Documentation (TechDocs)"
    - "Golden paths and templates"
    - "Self-service portals"
    - "Developer feedback loops"

15.6 Resume

• Le Platform Engineering construit une Internal Developer Platform (IDP)
• Backstage est le portail developpeur standard (CNCF)
• Le Software Catalog centralise la connaissance des services
• Les Templates Backstage automatisent la creation de projets
• Crossplane permet le self-service d'infrastructure
• Les Compositions Crossplane definissent des ressources composees
• Les Golden Paths sont des chemins valides pre-definis
• La Platform Team traite sa platform comme un produit
• L'ecosysteme CNCF fournit les briques technologiques