Resources Blog How is the Sonatype Safety Rating determined?

How is the Sonatype Safety Rating determined?

The Sonatype Safety Rating is generated by our experimental analysis tool and is an aggregate rating designed to estimate the likelihood of an open source project containing security vulnerabilities. You will see the rating of the project via Central and OSS Index. See more information and FAQs, here.

This tool leverages a variety of metrics, including the project's rate at which it updates vulnerable dependencies (also known as Mean Time to Update, or MTTU), as well as whether the project uses open source best practices, as measured by the OpenSSF's Security Scorecard.

The Security Scorecard assesses projects' practices like code review, signed releases, use of dependency update tools, and other similar measures, and produces a quantitative output. Further details about OpenSSF's Security Scorecard and the checks it runs can be found on its GitHub repository.

Sonatype's analysis tool combines these metrics and uses machine learning to output a scaled result that forms the basis for the Safety Rating of a project.  Projects are rated on a 1-10 scale, with 1 being the least safe and 10 being the safest. The more confident the model is that a project will not contain vulnerabilities, the higher the rating. The more confident the model is that the project will contain vulnerabilities, the lower it will rate the project. The model is based on empirical research conducted by the Sonatype Research Team, where we analyzed thousands of projects and determined a high correlation between the Safety Rating and the presence of vulnerabilities, with 88% of projects scoring below 5 having existing known vulnerabilities.

Curious to know more about the technical breakdown of our new metric? Read the full 8th Annual State of Software Supply Chain report.

Picture of Stephen Magill

Written by Stephen Magill

Stephen Magill is Vice President of Product Innovation at Sonatype. He’s the former CEO of MuseDev, a software company acquired by Sonatype, and is dedicated to helping developers write their best code through code quality automation.Stephen is a world-recognized expert on program analysis and was previously a principal scientist at Galois. Among his other accomplishments, he earned his Ph.D and M.S in CS from Carnegie Melon and serves on the University of Tulsa Industry Advisory Board.