DevOps CI/CD Patterns

Core Principles

1. Automate Everything — Every step from linting to deployment should be codified in pipeline configuration. Manual steps introduce drift and human error.
2. Fast Feedback Loops — Developers should know within minutes whether their change broke something. Prioritize quick checks early in the pipeline.
3. Reproducibility — Builds must produce identical artifacts given the same inputs. Pin dependency versions, use immutable base images, and avoid relying on ambient state.
4. Fail Fast, Fail Loud — Place the cheapest and most likely-to-fail checks first. Notify the right people immediately on failure.
5. Security as a First-Class Concern — Secrets must never appear in logs, artifacts, or version control. Scope credentials to the narrowest permission set possible.

GitHub Actions Fundamentals

Workflows live in .github/workflows/ and are triggered by repository events. Structure jobs to maximize parallelism while respecting dependency ordering.

# .github/workflows/ci.yml
name: CI Pipeline
on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: npm
      - run: npm ci
      - run: npm run lint

  test:
    runs-on: ubuntu-latest
    needs: lint
    strategy:
      matrix:
        node-version: [18, 20, 22]
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: ${{ matrix.node-version }}
          cache: npm
      - run: npm ci
      - run: npm test

See github-actions-patterns for: workflow triggers, matrix builds, reusable workflows, and composite actions.

GitLab CI Patterns

GitLab CI uses a .gitlab-ci.yml file at the repository root. Stages define the execution order, and jobs within the same stage run in parallel by default.

# .gitlab-ci.yml
stages:
  - validate
  - test
  - build
  - deploy

variables:
  DOCKER_IMAGE: $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA

lint:
  stage: validate
  image: node:20-alpine
  cache:
    key: ${CI_COMMIT_REF_SLUG}-npm
    paths:
      - node_modules/
  script:
    - npm ci --prefer-offline
    - npm run lint
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH

See gitlab-ci-patterns for: stages, rules, artifacts, caching, environments, and include templates.

Pipeline Design

A well-designed pipeline follows the principle of progressive confidence: each stage increases certainty that the change is safe to deploy.

# Conceptual pipeline flow
# Stage 1: Validate (parallel) ─┬─ lint
#                                ├─ type-check
#                                └─ security-scan
# Stage 2: Test (parallel)    ─┬─ unit-tests
#                               └─ integration-tests
# Stage 3: Build              ── docker-build
# Stage 4: Deploy (sequential)─┬─ staging (auto)
#                               └─ production (manual gate)

# GitHub Actions fan-out / fan-in example
jobs:
  unit-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: make test-unit

  integration-tests:
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres:16
        env:
          POSTGRES_PASSWORD: test
    steps:
      - uses: actions/checkout@v4
      - run: make test-integration

  build:
    needs: [unit-tests, integration-tests]
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: docker build -t app:${{ github.sha }} .

See pipeline-design-patterns for: fan-out/fan-in, conditional stages, manual gates, and notifications.

Secrets Management

Never hardcode credentials. Use your platform's secrets store and scope access to the environments and branches that need them.

# GitHub Actions — environment-scoped secrets
jobs:
  deploy-production:
    runs-on: ubuntu-latest
    environment: production
    steps:
      - uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: ${{ secrets.AWS_ROLE_ARN }}
          aws-region: us-east-1
      - run: |
          aws ecs update-service \
            --cluster prod \
            --service api \
            --force-new-deployment

See secrets-management for: GitHub secrets, Vault integration, OIDC federation, and environment-scoped credentials.

Deployment Strategies

Choose a deployment strategy based on risk tolerance, rollback speed, and infrastructure capabilities.

# Kubernetes rolling update configuration
apiVersion: apps/v1
kind: Deployment
metadata:
  name: api-server
spec:
  replicas: 4
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
  template:
    spec:
      containers:
        - name: api
          image: registry.example.com/api:v2.3.1
          readinessProbe:
            httpGet:
              path: /healthz
              port: 8080
            initialDelaySeconds: 5
            periodSeconds: 10

See deployment-strategies for: blue-green, canary, rolling updates, feature flags, and rollback patterns.

Build Caching and Optimization

Caching transforms slow pipelines into fast ones. Cache dependency downloads, build artifacts, and Docker layers aggressively.

# GitHub Actions — multi-layer caching
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: npm

      - uses: actions/cache@v4
        with:
          path: ~/.cache/Cypress
          key: cypress-${{ runner.os }}-${{ hashFiles('package-lock.json') }}

      - run: npm ci --prefer-offline
      - run: npm run build

      - uses: docker/build-push-action@v5
        with:
          context: .
          cache-from: type=gha
          cache-to: type=gha,mode=max
          push: true
          tags: app:${{ github.sha }}

Anti-Patterns

Performance

• Parallelize independent jobs — Lint, type-check, and security scans have no dependencies on each other. Run them simultaneously.
• Cache aggressively — Dependency installation is often the slowest step. Cache node_modules, .m2, pip wheels, and Docker layers.
• Use path filters — Only trigger backend pipelines when backend files change. GitHub Actions paths: and GitLab changes: rules both support this.
• Choose right-sized runners — Larger runners cost more per minute but finish faster. Measure total cost (runner-minutes times price) to find the optimum.
• Minimize Docker context — Use .dockerignore to exclude test files, docs, and .git. Smaller contexts mean faster builds.
• Split test suites — Use test sharding or parallel test runners (Jest --shard, pytest-xdist) to distribute work across multiple jobs.
• Avoid redundant checkouts — Pass artifacts between jobs instead of checking out and rebuilding in every job.

source: DevOps CI/CD best practices — GitHub Actions docs, GitLab CI docs, Kubernetes deployment strategies, OWASP CI/CD security guidelines.