Beyond the Numbers: CVSS Score Fails to Reflect Real-World Risks

An analysis, put together by security tools vendor JFrog, involved accessing the popular Common Vulnerability Scoring System (CVSS), an open industry standard framework for assessing the severity of security problems. The non-profit Forum of Incident Response and Security Teams (FIRST) manages the system, with the National Vulnerability Database (NVD) providing CVSS scores for confirmed vulnerabilities. JFrog’s analysis, which focused on accessing the impact of security bugs in open-source software, concluded that public CVSS impact metrics may oversimplify the risk posed by vulnerabilities because it lack context, among other factors.

JFrog analyzed the top 50 CVEs in 2022 and found a significant “discrepancy” between public severity ratings and the company’s assessments. In “most” cases, JFrog’s CVE severity rating was lower than the National Vulnerability Database (NVD) rating, indicating that these vulnerabilities were being overhyped. For instance, a buffer overrun in X.509 certificate verification (CVE-2022-3602) was initially considered a significant threat, but further investigation showed only marginal real-world impact. JFrog assigned a lower severity rating to 64% of the top 50 CVEs, while 90% received a lower or equal rating.

JFrog’s analysis reveals that NVD security ratings are often “underserved” because they fail to consider the complexity required to exploit them. Many of the vulnerabilities analyzed require specific environmental configurations or conditions to carry out a successful attack. JFrog also criticizes the lack of context when assigning CVE attack complexity metrics, citing the need to evaluate factors such as how software is deployed, the network environment, and whether a vulnerable API could parse untrusted data. As a result, the current system may set severity ratings too high or too low, which could impact decision-making and resource allocation.

JFrog also observed that 10 of the most prevalent vulnerabilities in 2022 impacting the enterprise tended to have low severity ratings and are regarded as a lower priority for enterprise IT teams and open source project maintainers – so remediation work is either delayed or (worse) entirely disregarded. If a bug is considered too small, developers may not create a patch, which JFrog says can only increase the number of affected systems over time. In contrast, if a CVSS rating is high, but the real-world impact is considered minuscule, the threat level could be faulted as misleading. Speaking to The Daily Swig, Shachar Menashe, senior director of security research at JFrog, said the best solution would be to update the CVSS standard to contain fields that would provide more context, such as exploitability in default configurations and whether or not there are context-dependent attack vectors.

Regarding potential improvements, Chris Gibson, the executive director of the non-profit Forum of Incident Response and Security Teams (FIRST) mentioned that CVSS v4.0 is in development and will enable product developers to provide supplementary urgency ratings, leading to a more accurate representation of the vulnerability’s urgency in their implementation. He also cautioned that the CVSS system has limitations, such as not considering contextual factors like the environment in which the vulnerability was found and its potential commercial or operational impact.