The problem isn't tools, it's order
Most DevOps initiatives in mid-market companies fail the same way: a team installs Terraform, builds a beautiful IaC repo, then six months later realises the underlying servers are still configured by hand and nobody knows what state they're in. The repo describes a fiction.
This piece is the order we actually recommend in 2026 for a team that owns 5–100 servers, mixed cloud and on-premise. It's not the most ambitious sequence — it's the one that survives contact with reality.
Outline
- Start with config management (Ansible) before IaC (Terraform)
- Version-control everything: infra, configs, runbooks
- A minimum-viable CI/CD pipeline
- Secrets management without homemade vaults
- Observability from day one
- The pitfalls that kill small-team DevOps
- A 60-day automation roadmap
1. Start with config management before IaC
Terraform answers "what infrastructure exists?" Ansible (or its equivalents) answers "what is on each machine?". For an existing estate, the second question is the painful one — and the one with the highest payback. Walk the floor with Ansible:
- Inventory every server with one playbook.
- Apply a baseline role (users, SSH config, time sync, monitoring agent).
- Make
ansible-playbook --checkpart of every change. - Only then start pulling provisioning into Terraform.
A team that can run a single command and re-baseline its entire estate has more operational maturity than one with a shiny Terraform repo and untracked drift.
2. Version-control everything
Infrastructure code, configuration roles, runbooks, dashboards, alert definitions, even the on-call schedule template. One Git org, one CODEOWNERS file, mandatory pull requests for anything touching production. Keep the structure boring:
/infra # Terraform
/config # Ansible roles + inventories
/runbooks # Markdown, one per incident class
/observability # Dashboards-as-code, alert rules
If a change is interesting enough to make, it's interesting enough to review. This is the cheapest control plane you'll ever build.
3. A minimum-viable CI/CD pipeline
Forget eight-stage pipelines for a first version. The minimum viable pipeline:
- Build — compile, package, container image with a digest.
- Test — unit, integration, a security scan (e.g. Trivy on images).
- Deploy to staging — automatic on
main. - Deploy to production — manual approval, single click.
- Rollback — one button, tested monthly.
That last step is the difference between a CI/CD pipeline and a CI/CD aspiration. If you can't roll back without a meeting, you don't have a pipeline.
4. Secrets management without homemade vaults
Pick one of: HashiCorp Vault, Doppler, AWS Secrets Manager, Azure Key Vault, GCP Secret Manager. Connect Ansible, Terraform and CI to it. Rotate machine credentials automatically; rotate human credentials on offboarding. Never store secrets in env files in Git, even encrypted ones — the operational ergonomics always degrade and people commit cleartext eventually.
5. Observability from day one
You will need three things, in this order:
- Metrics (Prometheus + Grafana, or a hosted equivalent) — for capacity, saturation, error rates.
- Logs (Loki, Elastic, hosted) — for incident forensics.
- Traces (Tempo, Jaeger, hosted) — for anything user-facing.
Build dashboards before you build alerts. Build alerts only when you have a runbook for each one. An alert without a runbook is a pager that wakes someone for no clear action.
6. Pitfalls that kill small-team DevOps
| Pitfall | Why it kills |
|---|---|
| Over-engineering | Kubernetes for three services; a service mesh for two pods. |
| Snowflake servers | One critical host nobody touches, full of manual config. |
| No rollback testing | First real rollback is during an outage. |
| Tooling proliferation | Three secret stores, two IaC tools, two observability stacks. |
| Documentation in heads | Bus factor of one. |
The cure for each is boring: smaller surface, repeatable deploys, monthly drills, one tool per job, write it down.
7. A 60-day automation roadmap
- Week 1–2 — Inventory: one Ansible playbook describing the entire estate.
- Week 3–4 — Baseline role applied to all hosts. Drift report weekly.
- Week 5–6 — CI for the most-changed application. Build + test only.
- Week 7–8 — Deploy step added with manual approval. Rollback tested.
- Week 9–10 — Secrets centralised. Two riskiest credentials rotated.
- Week 11–12 — Metrics + logs in one place. Three actionable alerts with runbooks.
That's enough to change the operational character of a small team. Anything beyond week 12 is iteration on this base.
Recommended toolchain for a small team in 2026
- Config: Ansible
- IaC: Terraform (or OpenTofu)
- CI/CD: GitHub Actions or GitLab CI
- Containers: Docker; Kubernetes only if you need it
- Secrets: Vault, Doppler or your cloud's KMS
- Observability: Grafana stack (Prometheus + Loki + Tempo) or a hosted equivalent
This is unglamorous on purpose. Glamour is what kills DevOps in companies under 200 people.
Key takeaways
- Tame what exists with config management before describing the ideal with IaC.
- Version-control infrastructure, runbooks, dashboards and alert rules together.
- Minimum-viable CI/CD includes a tested rollback. Without that, it's aspiration.
- Centralise secrets early; observability is metrics → logs → traces in that order.
- Six weeks of disciplined work changes how a small team operates more than six months of tool shopping.
If you want a second opinion on your current DevOps roadmap or a fixed-scope automation kickoff, get in touch — you'll talk to an engineer, not a sales rep.