A certain level of trust (and luck) is at the core of most open source projects - for better or worse. Owing to the fact that the copyright holder gives the rights for others to use, change, study, and distribute the software and its source code to additional parties and for any purpose. It can also be developed publicly. Open source software offers helpful tools that can drive innovation and collaboration, but securing them can be hard for multiple reasons.
Open Source: Charitable Contributions, Not Without Risk
A large range of people build open source software products. Some of them are managed by companies or non-profits, who also build and sell versions of the tools separately to gain revenue. However, many open-source projects rely heavily on dedicated volunteers who contribute their expertise, likely alongside other jobs. Sometimes, these individuals get some revenue from direct contributions via online donations from appreciative users of their software products, but often they are used without compensation. While their passion is commendable, the lack of funding combined with limited bandwidth for security checks can lead to difficulties in meticulously reviewing every code update submitted to the project. This creates a potential vulnerability that malicious actors could exploit.
This is exactly how the recent backdoor was implanted into XZ Utils. A known developer with a history of positive contributions to the open source project submitted a new contribution with malicious changes made to the code, establishing a backdoor.
This highlights the need for a more sustainable approach to open source security.
Open Source Backdoors: Not a New Risk
When it comes to open source software, this sort of risk is not necessarily new. However, it varies from some of the risks that face proprietary software. In saying that, if you look at most technologies today, virtually all of them have at least some open source software component within them, from the operating system to the packages in the coding language or package managers to basic utilities like ssh.
When internet giants like Google and Facebook make changes to code, they have huge amounts of instrumentation and automation in place to validate the impact of the changes, test them, and ensure they don’t introduce vulnerabilities. Now, this is not 100% prevention of bugs and vulnerabilities, but fundamentally, they deploy tried-and-true methods at a vast scale to greatly decrease the likelihood and risk of introducing a vulnerability. And then when vulnerabilities are introduced, as happens, they are promptly patched.
What Can We Learn from the XZ Utils Backdoor?
In the case of XZ Utils, we see the limit of small open source software products in their ability to effectively screen contributions for security issues and swiftly detect security issues once they have been introduced into their code that is already deployed.
The backdoor in XZ Utils went undiscovered for weeks until a dedicated engineer at Microsoft detected the tiniest of anomalies in SSH timing. By some measures, it was lucky the security issue was discovered as quickly as it was—but that’s what it was: luck.
In the past few years, Google has taken some important steps regarding open source software security, including offering free fuzzing for common open source software to find security vulnerabilities. These are critically important programs that help under-resourced open source projects achieve better security baselines.
Key Takeaway?
I’d love to see more of the large tech companies setting up these programs, and I’d love to see enterprise users of these open source software projects dedicate significant funding to teams with the sole purpose of securing the supply chain of software.