Matt Moore explores what it takes to secure your software supply chain, covering the difficult questions facing engineering teams around building or buying hardened images, maintenance, automation pitfalls, and supply chain attack risks.

Build vs. Buy: What it Really Takes to Harden Your Software Supply Chain

Author: Matt Moore

Securing the software supply chain has become one of the top priorities for modern engineering teams, especially in the wake of high-profile attacks like the SolarWinds breach. This article examines the true costs and complexities behind the decision to either build your own hardened container images or purchase supply chain security products.

Why Supply Chain Security Matters

Software build pipelines are now prime targets for attackers seeking to compromise organizations at scale.
Incidents such as the SolarWinds attack and a 2024 DockerHub breach have shown that tampering with build systems or image distribution channels can have wide-reaching and damaging effects.

The DIY (Build) Approach: More Than Meets the Eye

At the outset, building your own images may sound straightforward:

Start with a base image
Patch vulnerabilities
Add automation for ongoing updates

Hidden Challenges:

Upstream distributions of images are often out of date, so teams end up applying superficial, last-mile fixes.
Managing vulnerabilities, alerts, and regulatory requirements becomes an ongoing effort.
Statistics show 90% of codebases contain open source components lagging many versions behind.
Larger organizations may need whole teams dedicated to tracking security issues and maintaining compliance.

The Ongoing Burden of Maintenance

Developers on average spend ~6 hours/week just sourcing and updating open-source packages.
Keeping images patched, tested, and validated is a persistent engineering duty, not a one-time event.
Without dedicated ownership, even well-built images can become outdated liabilities.

Automation: Necessary but Not Sufficient

Automation tools can open pull requests and patch known vulnerabilities but can break if upstream changes introduce API or behavior shifts.
Example: A change in Kubernetes’ client-go library caused major refactoring for downstream projects when new context parameters were required.
Teams need deep understanding of automation, build tooling, and upstream communities to prevent automated updates from causing issues downstream.

Testing and Validation are Critical

Beyond unit tests, hardened images require full integration testing, sometimes in local Kubernetes clusters or actual cloud deployments.
Complex setups (e.g., images needing GPU resources) can slow down and complicate testing.
Without thorough, real-world tests, subtle incompatibilities may be missed, introducing risks into production.

Secure Distribution: The Last Mile

Even the most secure image is a risk if it’s distributed improperly.
Case in point: A compromised DockerHub token led to the distribution of a malicious Kong Ingress Controller image, highlighting the need for auditable and revocable access controls.

Conclusion: Hardening is a Practice, Not Just a Feature

Building your own hardened images provides more control but comes with substantial, ongoing maintenance costs and responsibilities.
Engineering organizations must honestly assess their capacity to support this practice over the long term.
Supply chain security isn’t something you can “set and forget”—it’s a continuous engineering commitment, especially as threats become more sophisticated.