Infrastructure Engineering Skill

Comprehensive guide for modern infrastructure engineering covering DevOps practices, multi-cloud platforms (AWS, Azure, GCP, Cloudflare), FinOps cost optimization, and DevSecOps security practices.

When to Use This Skill

Use this skill when:

DevOps: Setting up CI/CD pipelines (GitHub Actions, GitLab CI, Jenkins), implementing GitOps workflows (ArgoCD, Flux) AWS: Deploying to EC2, Lambda, ECS, EKS, managing S3, RDS, using CloudFormation/CDK Azure: Working with Azure VMs, App Service, AKS, Azure Functions, Storage Accounts GCP: Managing Compute Engine, GKE, Cloud Run, Cloud Storage, App Engine Cloudflare: Deploying Workers, R2 storage, D1 databases, Pages applications Kubernetes: Managing clusters, deployments, services, ingress, Helm charts, operators Docker: Containerizing applications, multi-stage builds, Docker Compose, registries FinOps: Analyzing cloud costs, optimizing spend, reserved instances, spot instances, rightsizing DevSecOps: Security scanning (SAST/DAST), vulnerability management, secrets management, compliance IaC: Terraform, CloudFormation, Pulumi, configuration management Monitoring: Setting up observability, logging, metrics, alerting, distributed tracing Platform Selection Guide When to Use AWS

Best For:

General-purpose cloud computing at scale Mature ecosystem with 200+ services Enterprise workloads with compliance requirements Hybrid cloud with AWS Outposts Extensive third-party integrations Advanced networking and security controls

Key Services:

EC2 (virtual machines, flexible compute) Lambda (serverless functions, event-driven) ECS/EKS (container orchestration) S3 (object storage, industry standard) RDS (managed relational databases) DynamoDB (NoSQL, global tables) CloudFormation/CDK (infrastructure as code) IAM (identity and access management) VPC (virtual private cloud networking)

Cost Profile: Pay-as-you-go, reserved instances (up to 72% discount), savings plans, spot instances (up to 90% discount)

When to Use Azure

Best For:

Microsoft-centric organizations (.NET, Active Directory) Hybrid cloud scenarios (Azure Arc, Stack) Enterprise agreements with Microsoft Windows Server and SQL Server workloads Integration with Microsoft 365 and Dynamics Strong compliance certifications (90+ certifications)

Key Services:

Virtual Machines (Windows/Linux compute) App Service (PaaS for web apps) AKS (managed Kubernetes) Azure Functions (serverless compute) Storage Accounts (Blob, File, Queue, Table) SQL Database (managed SQL Server) Active Directory (identity management) ARM Templates/Bicep (infrastructure as code)

Cost Profile: Pay-as-you-go, reserved instances, Azure Hybrid Benefit for Windows/SQL Server licenses

When to Use Cloudflare

Best For:

Edge-first applications with global distribution Ultra-low latency requirements (<50ms) Static sites with serverless functions Zero egress cost scenarios (R2 storage) WebSocket/real-time applications (Durable Objects) AI/ML at the edge (Workers AI)

Key Products:

Workers (serverless functions) R2 (object storage, S3-compatible) D1 (SQLite database with global replication) KV (key-value store) Pages (static hosting + functions) Durable Objects (stateful compute) Browser Rendering (headless browser automation)

Cost Profile: Pay-per-request, generous free tier, zero egress fees

When to Use Kubernetes

Best For:

Container orchestration at scale Microservices architectures with 10+ services Multi-cloud and hybrid deployments Self-healing and auto-scaling workloads Complex deployment strategies (blue/green, canary) Service mesh architectures (Istio, Linkerd) Stateful applications with operators

Key Features:

Declarative configuration (YAML manifests) Automated rollouts and rollbacks Service discovery and load balancing Self-healing (restarts failed containers) Horizontal pod autoscaling Secret and configuration management Storage orchestration Batch job execution

Managed Options: EKS (AWS), AKS (Azure), GKE (GCP), managed k8s providers

Cost Profile: Cluster management fees + node costs (optimize with spot instances, cluster autoscaling)

When to Use Docker

Best For:

Local development consistency Microservices architectures Multi-language stack applications Traditional VPS/VM deployments Foundation for Kubernetes workloads CI/CD build environments Database containerization (dev/test)

Key Capabilities:

Application isolation and portability Multi-stage builds for optimization Docker Compose for multi-container apps Volume management for data persistence Network configuration and service discovery Cross-platform compatibility (amd64, arm64) BuildKit for improved build performance

Cost Profile: Infrastructure cost only (compute + storage), no orchestration overhead

When to Use Google Cloud

Best For:

Enterprise-scale applications Data analytics and ML pipelines (BigQuery, Vertex AI) Hybrid/multi-cloud deployments Kubernetes at scale (GKE) Managed databases (Cloud SQL, Firestore, Spanner) Complex IAM and compliance requirements

Key Services:

Compute Engine (VMs) GKE (managed Kubernetes) Cloud Run (containerized serverless) App Engine (PaaS) Cloud Storage (object storage) Cloud SQL (managed databases)

Cost Profile: Varied pricing, sustained use discounts, committed use contracts

Quick Start AWS Lambda Function

Install AWS CLI

curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip" unzip awscliv2.zip && sudo ./aws/install

Configure credentials

aws configure

Create Lambda function with SAM

sam init --runtime python3.11 sam build && sam deploy --guided

See: references/aws-lambda.md

AWS EKS Kubernetes Cluster

Install eksctl

brew install eksctl # or curl download

Create cluster

eksctl create cluster \ --name my-cluster \ --region us-west-2 \ --nodegroup-name standard-workers \ --node-type t3.medium \ --nodes 3 \ --nodes-min 1 \ --nodes-max 4

See: references/kubernetes-basics.md

Azure Deployment

Install Azure CLI

curl -L https://aka.ms/InstallAzureCli | bash

Login and create resources

az login az group create --name myResourceGroup --location eastus az webapp create --resource-group myResourceGroup \ --name myapp --runtime "NODE:18-lts"

See: references/azure-basics.md

Cloudflare Workers

Install Wrangler CLI

npm install -g wrangler

Create and deploy Worker

wrangler init my-worker cd my-worker wrangler deploy

See: references/cloudflare-workers-basics.md

Kubernetes Deployment

Create deployment

kubectl create deployment nginx --image=nginx:latest kubectl expose deployment nginx --port=80 --type=LoadBalancer

Apply from manifest

kubectl apply -f deployment.yaml

Check status

kubectl get pods,services,deployments

See: references/kubernetes-basics.md

Docker Container

Create Dockerfile

cat > Dockerfile <<EOF FROM node:20-alpine WORKDIR /app COPY package*.json ./ RUN npm ci --production COPY . . EXPOSE 3000 CMD ["node", "server.js"] EOF

Build and run

docker build -t myapp . docker run -p 3000:3000 myapp

See: references/docker-basics.md

Reference Navigation AWS (Amazon Web Services) aws-overview.md - AWS fundamentals, account setup, IAM basics aws-ec2.md - EC2 instances, AMIs, security groups, auto-scaling aws-lambda.md - Serverless functions, SAM, event sources, layers aws-ecs-eks.md - Container orchestration, ECS vs EKS, Fargate aws-s3-rds.md - S3 storage, RDS databases, backup strategies aws-cloudformation.md - Infrastructure as code, CDK, best practices aws-networking.md - VPC, subnets, security groups, load balancers Azure (Microsoft Azure) azure-basics.md - Azure fundamentals, subscriptions, resource groups azure-compute.md - VMs, App Service, AKS, Azure Functions azure-storage.md - Storage Accounts, Blob, Files, managed disks Cloudflare Platform cloudflare-platform.md - Edge computing overview, key components cloudflare-workers-basics.md - Getting started, handler types, basic patterns cloudflare-workers-advanced.md - Advanced patterns, performance, optimization cloudflare-workers-apis.md - Runtime APIs, bindings, integrations cloudflare-r2-storage.md - R2 object storage, S3 compatibility, best practices cloudflare-d1-kv.md - D1 SQLite database, KV store, use cases browser-rendering.md - Puppeteer/Playwright automation on Cloudflare Kubernetes & Container Orchestration kubernetes-basics.md - Core concepts, pods, deployments, services kubernetes-advanced.md - StatefulSets, operators, custom resources kubernetes-networking.md - Ingress, service mesh, network policies helm-charts.md - Package management, charts, repositories Docker Containerization docker-basics.md - Core concepts, Dockerfile, images, containers docker-compose.md - Multi-container apps, networking, volumes docker-security.md - Image scanning, secrets, best practices Google Cloud Platform gcloud-platform.md - GCP overview, gcloud CLI, authentication gcloud-services.md - Compute Engine, GKE, Cloud Run, App Engine CI/CD & GitOps cicd-github-actions.md - GitHub Actions workflows, runners, secrets cicd-gitlab.md - GitLab CI/CD pipelines, artifacts, caching gitops-argocd.md - ArgoCD setup, app of apps pattern, sync policies gitops-flux.md - Flux controllers, GitOps toolkit, multi-tenancy FinOps (Cost Optimization) finops-basics.md - Cost optimization principles, FinOps lifecycle finops-aws.md - AWS cost optimization, RI, savings plans, spot finops-azure.md - Azure cost management, reservations, hybrid benefit finops-gcp.md - GCP cost optimization, committed use, sustained use finops-tools.md - Cost analysis tools, Kubecost, CloudHealth, Infracost DevSecOps (Security) devsecops-basics.md - Security best practices, shift-left security devsecops-scanning.md - SAST, DAST, SCA, container scanning secrets-management.md - Vault, AWS Secrets Manager, sealed secrets compliance.md - SOC2, HIPAA, PCI-DSS, audit logging Infrastructure as Code terraform-basics.md - Terraform fundamentals, providers, state terraform-advanced.md - Modules, workspaces, remote state cloudformation-basics.md - CloudFormation templates, stacks, change sets Utilities & Scripts scripts/cloudflare-deploy.py - Automate Cloudflare Worker deployments scripts/docker-optimize.py - Analyze and optimize Dockerfiles scripts/cost-analyzer.py - Cloud cost analysis and reporting scripts/security-scanner.py - Automated security scanning Common Workflows Multi-Cloud Architecture

Edge Layer: Cloudflare Workers (global routing, caching)

Compute Layer: AWS ECS/Lambda or Azure App Service (application logic)

Data Layer: AWS RDS or Azure SQL (persistent storage)

CDN/Storage: Cloudflare R2 or AWS S3 (static assets)

Benefits: - Best-of-breed services per layer - Geographic redundancy - Cost optimization across providers

AWS ECS Deployment with CI/CD

GitHub Actions workflow

name: Deploy to ECS on: push jobs: deploy: - Build Docker image - Push to ECR - Update ECS task definition - Deploy to ECS service - Wait for deployment stabilization

Kubernetes GitOps with ArgoCD

Git repository structure

/apps /production - deployment.yaml - service.yaml - ingress.yaml /staging - deployment.yaml

ArgoCD syncs cluster state from Git

Changes: Git commit → ArgoCD detects → Auto-sync to cluster

Multi-Stage Docker Build

Build stage

FROM node:20-alpine AS build WORKDIR /app COPY package*.json ./ RUN npm ci COPY . . RUN npm run build

Production stage

FROM node:20-alpine WORKDIR /app COPY --from=build /app/dist ./dist COPY --from=build /app/node_modules ./node_modules USER node CMD ["node", "dist/server.js"]

FinOps Cost Optimization Workflow

1. Discovery: Identify untagged resources

2. Analysis: Right-size instances (CPU/memory utilization)

3. Optimization:

- Convert to reserved instances (predictable workloads)

- Use spot instances (fault-tolerant workloads)

- Schedule start/stop (dev environments)

4. Monitoring: Set budget alerts, track savings

5. Governance: Enforce tagging policies

DevSecOps Security Pipeline

1. Code Commit

2. SAST Scan: SonarQube, Semgrep (static code analysis)

3. Dependency Check: Snyk, Trivy (vulnerability scanning)

4. Build: Docker image

5. Container Scan: Trivy, Grype (image vulnerabilities)

6. DAST Scan: OWASP ZAP (runtime security testing)

7. Deploy: Only if all scans pass

8. Runtime Protection: Falco, AWS GuardDuty

Terraform Infrastructure Deployment

1. Write: Define infrastructure in .tf files

2. Init: terraform init (download providers)

3. Plan: terraform plan (preview changes)

4. Apply: terraform apply (create/update resources)

5. State: Store state in S3 with DynamoDB locking

6. Modules: Reuse common patterns across environments

Best Practices DevOps CI/CD: Automate testing and deployment, use feature flags for progressive rollouts GitOps: Declarative infrastructure, Git as single source of truth, automated sync Monitoring: Implement observability (logs, metrics, traces), set up alerting Incident Management: Runbooks, postmortems, blameless culture Automation: Infrastructure as code, configuration management, self-service platforms Security (DevSecOps) Shift Left: Security scanning early in pipeline (SAST, dependency checks) Secrets Management: Use Vault, AWS Secrets Manager, or sealed secrets (never in code/Git) Container Security: Run as non-root, minimal base images, regular scanning Network Security: Zero-trust architecture, service mesh, network policies Access Control: Least privilege IAM, MFA, temporary credentials Compliance: Audit logging, encryption at rest/transit, regular security reviews Runtime Protection: Security monitoring, intrusion detection, automated response Cost Optimization (FinOps) Tagging: Enforce resource tagging for cost allocation and tracking Rightsizing: Analyze utilization, downsize over-provisioned resources Reserved Capacity: Purchase RI/savings plans for predictable workloads (up to 72% discount) Spot/Preemptible: Use for fault-tolerant workloads (up to 90% discount) Scheduling: Auto-stop dev/test environments during off-hours Storage Optimization: Lifecycle policies, archive to cheaper tiers, delete orphaned resources Monitoring: Budget alerts, cost anomaly detection, chargeback/showback Governance: Approval workflows for expensive resources, quota management Kubernetes Resource Management: Set requests/limits, use horizontal pod autoscaling High Availability: Multi-zone clusters, pod disruption budgets, anti-affinity rules Security: RBAC, pod security policies, network policies, admission controllers Observability: Prometheus metrics, distributed tracing, centralized logging GitOps: ArgoCD/Flux for declarative deployments, automatic drift correction Performance Compute: Auto-scaling, load balancing, multi-region for low latency Caching: CDN, in-memory caching (Redis/Memcached), edge computing Storage: Choose appropriate tier (SSD vs HDD), enable caching, CDN for static assets Containers: Multi-stage builds, minimal images, layer caching Databases: Connection pooling, read replicas, query optimization, indexing Development Local Development: Docker Compose for consistent environments, dev containers Testing: Unit, integration, end-to-end tests in CI/CD pipeline Infrastructure as Code: Terraform/CloudFormation for repeatability Documentation: Architecture diagrams, runbooks, API documentation Version Control: Git for code and infrastructure, semantic versioning Decision Matrix Need Choose Compute Sub-50ms latency globally Cloudflare Workers Serverless functions (AWS ecosystem) AWS Lambda Serverless functions (Azure ecosystem) Azure Functions Containerized workloads (managed) AWS ECS/Fargate, Azure AKS, GCP Cloud Run Kubernetes at scale AWS EKS, Azure AKS, GCP GKE VMs with full control AWS EC2, Azure VMs, GCP Compute Engine Storage Object storage (S3-compatible) AWS S3, Cloudflare R2 (zero egress), Azure Blob Block storage for VMs AWS EBS, Azure Managed Disks, GCP Persistent Disk File storage (NFS/SMB) AWS EFS, Azure Files, GCP Filestore Database
Managed SQL (AWS) AWS RDS (PostgreSQL, MySQL, SQL Server) Managed SQL (Azure) Azure SQL Database Managed SQL (GCP) Cloud SQL NoSQL key-value AWS DynamoDB, Azure Cosmos DB, Cloudflare KV Global SQL (edge reads) Cloudflare D1, AWS Aurora Global CI/CD & GitOps
GitHub-integrated CI/CD GitHub Actions Self-hosted CI/CD GitLab CI/CD, Jenkins Kubernetes GitOps ArgoCD, Flux Cost Optimization
Predictable workloads Reserved Instances, Savings Plans Fault-tolerant workloads Spot Instances (AWS), Preemptible VMs (GCP) Dev/test environments Auto-scheduling, budget alerts Security
Secrets management HashiCorp Vault, AWS Secrets Manager, Azure Key Vault Container scanning Trivy, Snyk, AWS ECR scanning SAST/DAST SonarQube, Semgrep, OWASP ZAP Special Use Cases
Static site + edge functions Cloudflare Pages, AWS Amplify WebSocket/real-time Cloudflare Durable Objects, AWS API Gateway WebSocket ML/AI pipelines AWS SageMaker, GCP Vertex AI, Azure ML Browser automation Cloudflare Browser Rendering, AWS Lambda + Puppeteer Resources Cloud Providers AWS Docs: https://docs.aws.amazon.com Azure Docs: https://docs.microsoft.com/azure GCP Docs: https://cloud.google.com/docs Cloudflare Docs: https://developers.cloudflare.com Container & Orchestration Docker Docs: https://docs.docker.com Kubernetes Docs: https://kubernetes.io/docs Helm: https://helm.sh/docs CI/CD & GitOps GitHub Actions: https://docs.github.com/actions GitLab CI: https://docs.gitlab.com/ee/ci/ ArgoCD: https://argo-cd.readthedocs.io Flux: https://fluxcd.io/docs Infrastructure as Code Terraform: https://developer.hashicorp.com/terraform AWS CDK: https://docs.aws.amazon.com/cdk Pulumi: https://www.pulumi.com/docs Security & Compliance OWASP: https://owasp.org CIS Benchmarks: https://www.cisecurity.org/cis-benchmarks HashiCorp Vault: https://developer.hashicorp.com/vault FinOps & Cost Optimization FinOps Foundation: https://www.finops.org AWS Cost Optimization: https://aws.amazon.com/pricing/cost-optimization Kubecost: https://www.kubecost.com Implementation Checklist AWS Lambda Deployment Install AWS CLI and SAM CLI Configure AWS credentials (access key, secret key) Create Lambda function with SAM template Configure IAM role and policies Test locally with sam local invoke Deploy with sam deploy Set up CloudWatch monitoring and alarms AWS EKS Kubernetes Cluster Install kubectl, eksctl, aws-cli Configure AWS credentials Create EKS cluster with eksctl Configure kubectl context Install cluster autoscaler Set up Helm for package management Deploy applications with kubectl/Helm Configure ingress controller (ALB/NGINX) Azure Deployment Install Azure CLI Login with az login Create resource group Deploy App Service or AKS Configure continuous deployment Set up monitoring with Application Insights Kubernetes on Any Cloud Install kubectl and helm Connect to cluster (update kubeconfig) Create namespaces for environments Apply RBAC policies Deploy applications (deployments, services) Configure ingress for external access Set up monitoring (Prometheus, Grafana) Implement GitOps with ArgoCD/Flux CI/CD Pipeline (GitHub Actions) Create .github/workflows/deploy.yml Configure secrets (cloud credentials, API keys) Add build and test jobs Add container build and push to registry Add deployment job to cloud platform Set up branch protection rules Enable status checks and notifications FinOps Cost Optimization Implement resource tagging strategy Enable cost allocation tags Set up budget alerts Analyze resource utilization (CloudWatch, Azure Monitor) Identify rightsizing opportunities Purchase reserved instances for predictable workloads Configure auto-scaling and scheduling Regular cost reviews and optimization DevSecOps Security Add SAST scanning to CI/CD (SonarQube, Semgrep) Add dependency scanning (Snyk, Trivy) Implement container image scanning Set up secrets management (Vault, cloud provider) Configure security groups and network policies Enable audit logging Implement security monitoring and alerting Regular vulnerability assessments Cloudflare Workers Install Wrangler CLI Create Worker project Configure wrangler.toml (bindings, routes) Test locally with wrangler dev Deploy with wrangler deploy Docker Write Dockerfile with multi-stage builds Create .dockerignore file Test build locally Push to registry (ECR, ACR, GCR, Docker Hub) Deploy to target platform