Commentary on Trend Micro’s Linux Threat Report 2021

On August 23, 2021, Trend Micro released a report titled “Linux Threat Report 2021 1H” by Magno Logan and Pawan Kinger. The report is based on Trend Micro’s Smart Protection Network (SPN) which they call “the data lake for all detections across all Trend Micro’s products“. Basically, every security product they make that detects vulnerabilities and reports them back to Trend Micro can and is used in their research like this, among other things. They further qualify that the report is based on additional data “collected data from honeypots, sensors, anonymized telemetry, and other backend services” and represents “the real-world prevalence of malware and vulnerability exploitation across enterprises” regardless of size or vertical.

Reports that talk about the most exploited vulnerabilities are compelling. They offer a short list of vulnerabilities that organizations can be absolutely sure they patched and offer no risk. Unfortunately, many of these reports have problems. I have written about some before including 2015 Verizon DBIR, 2016 Verizon DBIR, and Radware’s Top Web Exploits of 2020. I wish I had more time as I have seen other reports on exploit prevalence that had similar issues. In this case, Trend Micro’s report falls into at least one of the same traps that these prior reports have.

The first issue that pops out is the wording in the report that introduces a major point of confusion. In section two, titled “The Linux threat landscape: What are the top Linux threats?“, under the second heading titled “Vulnerabilities in Linux systems“, we get more details qualifying where the data came from to generate this part of the report:

“… we dissected IPS (Intrusion Prevention System) hits from Trend Micro Cloud One – Workload Security and sifted through over 50 million events, ignored false positives, eliminated test data, and layered data with available threat intel to draw some conclusions.”

Unfortunately, the next sentence immediately introduces some doubt and we don’t know how much doubt there is because they don’t qualify their error of marging:

“It should be noted that there can be a degree of error here due to the nature of the data and internet activity.”

If the margin for error is 1% in a dataset that large, not a big deal. If it is 10%, that can be problematic. If it is 50% then the report shouldn’t have even been written. We get to guess where that margin of error is apparently.

Now, for the section of the report that initially got my attention, we get to the top 15 vulnerabilities. I can’t finish that sentence because there is confusion:

If a list of vulnerabilities includes the top 15 that are “actively exploited” or “have a known proof of concept”, how do you even? Over 4,500 vulnerabilities in 2021 H1 have a public proof-of-concept or functional exploit. The next sentence clearly repeats the exact same thing. I can’t figure out how to explain that second part unless they are attempting to say “actively exploited and a public proof of concept” to distinguish from exploitation that is happening where the exploit is not actually published. That seems like a pretty big oversight given the nature of this section of the report. Further, it doesn’t qualify if the report is based on attempted exploitation that matches a signature or successful exploitation. After the table of vulnerabilities the report says “Table 1 shows the top vulnerabilities by volume of triggers.” which strongly suggests it is looking for exploit attempts. But that just leads to more questions like “if you see an attempt for that vulnerability but against a Windows server, does it count?

It gets even murkier looking at the table of the 15 vulnerabilities where one of them is listed as “N/A” for severity. That warrants digging into their list more closely and comparing the vulnerability information with that in VulnDB.

There are several observations to be made for this list:

  • CVE-2017-9805 is listed as ‘High’ severity suggesting they pulled at least some vulnerability data from the National Vulnerability Database. They score the vulnerability 8.1 (High) while VulnDB and CERT VU scores it 10.0. Looking at the original disclosure, there are no obvious qualifications that seem to justify an Access Complexity High (AC:H) rating.
  • Of the 430 vulnerabilities involving WordPress, base or plugins, that allow for remote code execution, why did only one make the list (CVE-2020-25213), and why that one? Given the amount of scanning for vulnerable WordPress installations I would expect more to be on the list. Hell, even the venerable CVE-2013-4338 given there are other CVE-2013s on the list.
  • The Atlassian Jira vulnerability is very curious given that it is a remote information disclosure issue and does not disclose sensitive information such as a password, that would result in further privilege escalation. Based on the logs of attrition.org over the last three months, there has been a single request for /secure/QueryComponent!Default.jspa. There have been five requests for /secure/QueryComponentRendererValue!Default.jspa (CVE-2020-36289) which is another information disclosure issue. There are also hundreds of information disclosure vulnerabilities that yield credentials which can be used to authenticate to an application to gain privileges. I would expect to see any one of those on the list before CVE-2020-14179.
  • Eclipse Jetty (CVE-2017-7657) is very curious to see on this list for several reasons. First, a four year old vulnerability that does not result in code execution. Second, there is a caveat for exploitation as explained in the Eclipse bug ticket: “was deployed behind an intermediary that imposed some authorization and that intermediary allowed arbitrarily large chunks to be passed on unchanged, then this flaw could be used to bypass the authorization imposed by the intermediary“. To see an HTTP request smuggling issue be that widely exploited over the thousands of other vulnerabilities that allow for a more serious impact in software found running on Linux is baffling. This strongly suggests the detection rule matching for that vulnerability is overly broad and triggers on exploit attempts for different issues.
  • The nginx vulnerability is listed as N/A which is curious. Looking at the associated NVD entry (CVE-2013-4547) we see they have a CVSSv2 score, but not a CVSSv3 score. That is due to it being a 2013 vulnerability and NVD not retroactively scoring all vulnerabilities. This, along with CVE-2017-9805 tells us that Trend Micro imported the scores from NVD but didn’t adjust for this one by using the CVSSv2 score, or developing their own CVSSv3 score. It seems weird to blindly use the CVSSv3 scores and have incomplete data when it is a simple correction to make.

Looking back to bullet #3, it’s interesting to compare the hits seen on our web server and then note that we also observed 10,659 requests for /wp-login.php in that same period. This, I think, illustrates a serious flaw in the methodology of this report. Most of the scanning we see for vulnerable WordPress instances is first looking for the presence of the software before attempting to exploit anything. Rather than throw hundreds of payloads for various flaws in the core software and vulnerable themes or plugins, it’s more efficient to check if the software is present first. Scan for the software to build a list of sites that are running WordPress before launching more significant attacks that may attract more attention.

As always, a real test to the veracity of this data would be for another firm that does large-scale monitoring of attacks to publish their own data, limited to the same approximate criteria as Trend Micro. That might explain bullet #4 at the very least.

Commentary on Radware’s Top Web Exploits of 2020

At the close of each year we see at least one article covering the top vulnerabilities / exploits from the prior year. This is usually written on the back of having large detection networks across the Internet that get a comprehensive view of exploitation. It’s a great way to get real intelligence for criminal hacking activity. Unfortunately, we often see a breakdown when it comes to conveying that information in a useful manner. I know there is an argument to be made that the companies releasing such blogs are primarily after PR, sure. But they also have an opportunity to help their clients and the rest of the world by ensuring the blogs contain more useful and actionable information.

For this commentary, I’ll examine Radware’s blog, “The Top Web Service Exploits in 2020” published December 23, 2020 and covered almost verbatim by Security Magazine on January 5, 2021. I don’t have a view into exploit activity itself, but I do have a good view into the vulnerability disclosure landscape that is a cornerstone of this commentary.

We’ll start by setting a few basic ideas for mutual understanding for any such blog. First, each exploit should be tied to a unique vulnerability or it should explain it is an exploit chain and clearly delineate each vulnerability in the chain or explain what it represents if not a pure vulnerability. Second, it should provide at least one external reference for each vulnerability; either a CVE ID, vendor advisory, or commonly accepted third-party advisory such as US-CERT or another similar body. This is what allows the reader to quickly determine if their organization has patched against the vulnerability or not. If I have to spend considerable time trying to determine which vulnerability is being described, many organizations may be at a complete loss trying to figure it out.

With that, let’s look at the top 10 exploited vulnerabilities in 2020, according to Radware, and try to figure out some additional information for perspective. I will also be very clear that Radware’s blog is extremely frustrating and not immediately helpful, instead requiring a lot of extra work. The fact that they only attributed three exploits to a CVE ID is a dismal commentary on the CVE ecosystem. This analysis of their analysis will server as a reminder that comprehensive vulnerability intelligence is the foundation of any good security program.


Service Exploit #1: /ws/v1/cluster/apps/new-application

Based on their description, this appears to match VulnDB 184750 “Apache Hadoop YARN ResourceManager REST API Request Handling Remote Command Execution“. The first thing of interest is it was disclosed on October 19, 2016 and does not have a CVE assignment over four years later. No wonder many organizations aren’t aware of this vulnerability and have not sought out their own remediation strategy.

Service Exploit #2: /manager/html

This is summarized as “Apache Tomcat Manager Application Upload Authenticated Code Execution” and goes on to describe it as “This module can be used to execute a payload on Apache Tomcat servers that have an exposed “manager” application. The payload is uploaded as a WAR archive containing a JSP application using a POST request against the /manager/html/upload component.

Despite this description, that does not cleanly map to any vulnerability in VulnDB. The closest matches are CVE-2017-12615 and CVE-2017-12617 which is an abstraction for different platforms, but fundamentally “Apache Tomcat HTTP PUT Method JSP File Upload Remote Code Execution“. On the surface this is a match with Apache Tomcat, JSP application, and POST request to achieve code execution. However, those two CVEs cover a JSP file upload, not a WAR archive, and do not mention the /manager/html/upload component. So we’re left wondering if the exploit described is simply a misconfiguration scenario (i.e. intended functionality not secured) or an actual disclosed vulnerability.

Service Exploit #3: /level/15/exec/-/sh/run/CR

Based on the description, this is a misconfiguration scenario where an administrator sets up a Cisco router with the HTTP admin interface enabled, but without password protection. This allows an attacker to use the legitimate functionality to run arbitrary commands.

Service Exploit #4: /admin/assets/js/views/login.js

Radware says this “resource belongs to Sangoma FreePBX code and it looks like the attackers are trying to detect vulnerable FreePBX servers and exploit one of the known vulnerabilities.” The first issue is that script doesn’t immediately track to a VulnDB entry based on titles, which reflect the script name typically. However, let’s consider the URL being seen: … login.js. Rather than attempting to exploit “one of the known vulnerabilities“, I would suggest instead they are trying default credentials. At least back around 2000, the tried-and-true default credentials of admin/admin were all you needed to access the interface.

This one is curious to me because presumably a company that was detecting exploit traffic and could see e.g. POST requests as demonstrated in Service Exploit #2, would also see that the attackers were trying the default credentials. So we’re left with Service Exploit #4 being of little help and only creating confusion over what is being exploited.

Service Exploit #5: /ftptest.cgi?loginuse=&loginpas=

Radware attributes this to “many cheap Wireless IP web cameras use the same genetic code based on the GoAhead code (the tiny, embedded web server).” This tracks cleanly with VulnDB 181032 “Axis Multiple Products axis-cgi/ftptest.cgi Multiple Parameters Remote Command Execution Weakness“. This is actually a fun rabbit hole as this disclosure originally comes from an audit of a AXIS A1001 Network Door Controller and exploitation of this issue requires privileged access to the management interface. With that in mind, we’re back to a default credential scenario that may be the actual issue. Back in 2001, defaults for Axis network cameras were covered by CVE-2001-1543.

[Update: Z Balazs points out that this finding is likely due to Persirai botnet activity and links to more information.]

Service Exploit #6: /service/extdirect

This is the only one of the ten exploits covered that they include a CVE ID for. CVE-2019-7238 maps to VulnDB 198437 “Nexus Repository Manager /service/extdirect Insufficient Access Control Request Handling Remote Code Execution“. But, is that really the right ID? If we look at CVE-2020-10204 we are given a very brief summary of “Sonatype Nexus Repository before 3.21.2 allows Remote Code Execution” and a link to the vendor advisory. However, VulnDB 226228 also maps to this and is summarized as “Nexus Repository Manager /service/extdirect Request Handling Remote Command Execution“. We immediately see the /service/extdirect from Radware’s finding in both titles. The vendor’s advisory does not include this endpoint though, but we find it in this exploit published on GitHub that tracks with the CVE-2020-10204 and we see it in a different exploit for CVE-2019-7238.

CVE-2019-7238 was fixed in Nexus Repository Manager version 3.15.0 and CVE-2020-10204 was fixed in version 3.21.2. Due to the vague vendor advisories it difficult to tell if this was a regression situation or something else. But, the CVE-2020-10204 vendor advisory gives us the interesting bit in the context of exploitation: “The vulnerability allows for an attacker with an administrative account on NXRM to execute arbitrary code by crafting a malicious request to NXRM.” That is an important distinction! So this is likely CVE-2019-7238 as Radware says, unless there are default credentials which would allow for exploiting CVE-2020-10204 as well.

Looking at the NVD entry for CVE-2020-10204 we also see that they scored this incorrectly for their CVSSv3 score, as ‘Privileges Required‘ should be ‘High‘, notLow‘ as they have it.

Service Exploit #7: /solr/admin/info/system?wt=json

For this one, we get an Apache Bug ID (SOLR-4882) and CVE-2013-6397 as references which is great. That said, it would be very helpful if Radware would link to these resources to make it easier for their readers.

Service Exploit #8: /vendor/phpunit/phpunit/src/Util/PHP/eval-stdin.php

This is the third exploit they match to an ID, CVE-2017-9841 and it was disclosed June 27, 2017. Another good reminder that software with disclosed vulnerabilities and vendor solutions are not being applied, causing many organizations to become low-hanging fruit in the exploit world.

One little nitpick is that the full path they include is likely not how this would manifest on a server. Everything after “src” would be the endpoint being scanned presumably: /Util/PHP/eval-stdin.php

Service Exploit #9: /hudson

With this, we run into another mess and rabbit hole. Radware summarizes this as “Hudson continuous integration tool – multiple vulnerabilities” and further describes Hudson as “a continuous integration tool written in Java, which runs in a servlet container, such as Apache Tomcat or the GlassFish application server. Over the years the project was replaced by Jenkins. The final release. 3.3.3 was on February 15, 2016. Today Hudson is no longer maintained and was announced as obsolete in February 2017.

Based on this description, this could be any one of at least 50 vulnerabilities going back to February, 2014, one of which does not have a CVE ID. 41 of these are in Jenkins software which is mentioned above.

Other Service Exploits

This is a curious conclusion to the “top 10” list, as it states “In addition to the new items that we covered in this list, we have also seen items that we already saw and covered in our previous blog Top 10 Web Service Exploits in 2019 such as /ctrlt/DeviceUpgrade_1, /TP/public/index.php and /nice%20ports%2C/Tri%6Eity.txt%2ebak.

That isn’t exactly a #10 on this list, rather a catch-all for “other stuff we saw including…“. The first listed tracks with VulnDB 170573 “Huawei HG532 Routers /ctrlt/DeviceUpgrade_1 NewStatusURL Element Remote Command Execution (Satori)” which is notable as it is used in Satori, a Mirai botnet variant.

The second tracks with VulnDB 194379 “ThinkPHP /public/index.php call_user_func_array() Function vars[1][] Parameter Remote Code Execution“. Note the different exploit path and we see it can actually be exploited via several endpoints according to analysis of the vulnerability by Knownsec 404 Team.

The third doesn’t immediately track with an entry in VulnDB. Radware gives us “/nice%20ports%2C/Tri%6Eity.txt%2ebak” which we can decode to a more friendly “/nice ports,/Trinity.txt.bak“. A quick Google for that request finds a blog from Dragos titled “Threat Hunting With Python Part 2: Detecting Nmap Behavior with Bro HTTP Logs” explaining this request:

The request for “/nice ports,/Trinity.txt.bak” comes from Nmap’s service detection routine testing how a server handles escape characters within a URI. The actual request is “GET /nice%20ports%2C/Tri%6Eity.txt%2ebak HTTP/1.0\r\n\r\n”.

So this isn’t an actual exploit, rather, it indicates that attackers are using the Nmap port scanner. This is a good reminder that “exploit scanning” doesn’t always cleanly map to a specific vulnerability.


Detecting exploitation is critical for every organization. Doesn’t matter if it is on-premises devices or a managed service-based detection. What is more critical is having comprehensive and timely vulnerability intelligence that can turn what you detect into actionable information. This is how you not only detect, but evaluate and remediate, assuming of course the vulnerability is known to the vendor or a mitigation can be enacted.

Response to Kenna Security’s Explanation of the DBIR Vulnerability Mess

[This was originally published on the OSVDB blog.]

Earlier this week, Michael Roytman of Kenna Security wrote a blog with more details about the vulnerability section of the Verizon DBIR report, partially in response to my last blog here questioning how some of the data was generated and the conclusions put forth. The one real criticism I will note, is that Roytman’s blog does not acknowledge or warn that the list of CVE IDs included in the DBIR had typos, causing the wrong IDs to be included. In the world of vulnerability databases, that unique ID is designed specifically to avoid such confusion. Carrying the wrong IDs undermines the integrity of the data being presented.

In addition to my comments, Roytman had a long call with Adrian Sanabria which led to generating a new set of data with a different scope. From the Kenna blog:

We had an excellent offline discussion in which he dove deeply into the assumptions of my work, asked thoughtful, deep questions in private, and together, we came up with a better metric for generating a top 10 vulnerabilities list. To address these issues, I scaled the total successful exploitation count for every vulnerability in 2015 by the number of observed occurrences of that vulnerability in Kenna’s aggregate dataset. Sifting through 265 million vulnerabilities gives us a top 10 list perhaps more in line with what was expected – but equally unexpected! The takeaway here is that datasets like the one explored in the DBIR might be noisy, might have false positives and the like, but carefully applied to your enterprise the additional context successful exploitation data lends to vulnerability management is priceless.

I won’t go into much detail in this blog, but will say that I disagree with the statement that severely flawed data can produce takeaways that are “priceless”. Organizations acting on these top 10 lists may be spending time and resources chasing vulnerabilities that do not impact them, or pose very little risk compared to other threats they face. That action most certainly has a price; one that can be enumerated to some degree due to the cost of the employee time required. With that in mind, let’s look at the methodology which is spelled out in more detail than the DBIR, before we consider the new top 10 list Roytman generated. First, his notes on the data examined:

The first is a convenience sample that includes 2,442,792 assets (defined as: workstations, servers, databases, ips, mobile devices, etc) and 264,912,235 vulnerabilities associated to those assets. The vulnerabilities are generated by 8 different scanners, they are: Beyond Security, Tripwire, McAfee VM, Qualys, Tenable, BeyondTrust, Rapid7, and OpenVAS . This dataset is used in determining remediation rates and the normalized open rate of vulnerabilities.

I am curious if it is normal practice to consider an IP address an asset, when the system that addresses the IP is largely considered the asset. Moving past that, one point that sticks in my mind is the tools that generate the data. From the list above, consider that Beyond Security claims to have “what is arguably the world’s most complete database of security vulnerabilities.” Click around their site and you see that “the AVDS database includes over 10,000 known vulnerabilities and the updates include discoveries by our own team and those discovered by corporate and private security teams around the world.” That is less than 25% of what CVE has, and less than 10% of what VulnDB has. They even show you that they only cover 200 CVE IDs for 2016, as compared to the 1,474 open 2016 CVEs.

They don’t specify which Tripwire product, include McAfee Vulnerability Manager which was declared End of Life in October last year, and don’t specify which Qualys product. So it is a start as far as explaining what tools generate the data, but still leaves a lot of guess-work.

Roytman describes the second data set used as:

The second is a convenience sample that includes 3,615,706,022 successful exploitation events which all take place in 2015 which come from Dell Secureworks’ Counter-Threat Unit and Alienvault’s Open Threat Exchange.

The third qualification, describing the methodology is perhaps the most important, and was lacking in the DBIR:

Please note the methodology of data collection: Successful Exploitation is defined as one successful technical exploitation of a vulnerability on one machine at a particular timestamp. The event is defined as: 1. An asset has a known CVE open. 2. An attack come in that matches the signature for that CVE on that asset and 3. One or more IOCs are detected/correlated post attack. It is not necessarily a loss of a data, or even root on a machine. It is just the successful use of whatever condition is outlined in a CVE.

I have reached out to Michael and requested a sampling of data for two of the CVE IDs on his new list, and he is going to do so. In the meantime, I had a discussion with several people more familiar with IDS than myself and asked about how they would detect attacks for CVE-2013-0229 and CVE-2001-0540 as an example. Sure, detecting a specific type of packet meant to trigger this is one thing, but what is the threshold for the IDS to say it is an attack, when exploitation requires “a large number of malformed Remote Desktop Protocol (RDP) requests“. Is there a specific number of packets for it to flag an attack in progress? If too low, it may be prone to a high number of false positives. If too high, it may not detect a successful exploitation of the issue. Which leads into the second part of the methodology, comparing it with “one or more IOCs [that] are detected/correlated post attack”. In this case, it would presumably be the targeted service not responding, which could be detected a number of ways (e.g. probing the port, seeing specific errors in the logs, noticing a given process not running). Once the service is down, is the IDS that isn’t aware of the host state still cataloging a single attack? Or is it generating alerts every X minutes it detects an attack ongoing? Hopefully the data Michael sends will help me better understand how that correlation is being made, as it represents a source of incredible bias for the resulting data analysis.

Moving on to Roytman’s new list using the above data and methodology, here is the top 10 list he sees in the data:

  1. 2015-03-05 – 2015-1637 – Microsoft Windows Secure Channel (Schannel) RSA Temporary Key Handling EXPORT_RSA Ciphers Downgrade MitM (FREAK)
  2. 2015-01-06 – 2015-0204 – OpenSSL RSA Temporary Key Handling EXPORT_RSA Ciphers Downgrade MitM (FREAK)
  3. 2014-04-07 – 2014-0160 – OpenSSL TLS Heartbeat Extension Packets Handling Out-of-bounds Read Remote Memory Disclosure (Heartbleed)
  4. 2012-03-13 – 2012-0152 – Microsoft Windows Remote Desktop Protocol Terminal Server RDP Packet Parsing Remote DoS
  5. 2009-05-16 – 2013-0229 – MiniUPnPd SSDP Handler minissdp.c ProcessSSDPRequest Function Malformed Input Handling Remote DoS
  6. 2002-02-12 – 2002-0012 – Multiple Vendor Malformed SNMP Trap Handling DoS
  7. 2002-02-12 – 2002-0013 – Multiple Vendor Malformed SNMP Message-Handling Remote DoS
  8. 2001-12-20 – 2001-0876 – Microsoft Windows Universal Plug and Play NOTIFY Directive URL Handling Remote Overflow
  9. 2001-12-20 – 2001-0877 – Microsoft Windows Universal Plug and Play NOTIFY Request Remote DoS
  10. 2001-07-25 – 2001-0540 – Microsoft Windows Terminal Server RDP Request Handling Memory Exhaustion Remote DoS

Roytman prefaces that list with a comment that the “top 10 list perhaps more in line with what was expected – but equally unexpected!” Indeed, that is certainly true. Expected? A high-profile vulnerability disclosed in 2014 that was seen being widely exploited then, and subsequent years, something sorely missing from the DBIR list even after the corrected CVE IDs were factored in. This is the type of vulnerability almost everyone expected to top the list due to how easy it was to exploit, and knowing that it had been used heavily. This speaks to the original point and reason for the first blog; all the data ‘science’ in the world that produces highly questionable results should not be taken as gospel. Even if the methodology was sound, it doesn’t mean the data being used was.

Unfortunately, Roytman’s revised list deviates quickly into the “equally unexpected“. The first DBIR report list had a single denial of service (DoS) attack on the list, which stood out as odd. The revised list bumped that number to two which was a bit odd. The most recent list expands that to six DoS attacks, which is highly questionable for one reason or another. First, you can question the data set and methodology leading to this conclusion, but let’s say that passes muster solely for the sake of argument. Second, you can question why so many denial of service attacks are on a top 10 most exploited vulnerabilities list, framed in a context that uses the word ‘compromise’, riding on the back of a report centered around data breaches. These attacks are not causing attackers to gain privileges or steal data. They are a nuisance most of the time, or potentially used in serious DoS attacks other times. It makes one question why DoS attacks weren’t dropped from the results completely, and disclaimed as such! Or, generate two lists; one with results based on raw data, DoS and everything, and a second list that focuses on vulnerabilities that may allow for privileges to be gained and a real compromise to happen.

I cannot stress this enough. Using a term like ‘Indicator of Compromise’ (IOC) in the context of DoS attacks is disingenuous and misleading. Going back to Roytman’s introduction into this section where he makes a comment about seeing the trees (referring to the classic metaphor), I find it ironic as that sums up the purpose of my blog. The DBIR was written as if they could only see the trees (data points), and not the forest (bigger picture), which is what many people took issue with.

One point that I overlooked on the original list, and still appears on the new list, is the presence of FREAK (twice even). Fortunately for me, Thomas Ptacek does a great job explaining why FREAK is likely on the list, but absolutely should not be. Using Roytman’s blog and data, he calculates that attackers would have spent $332,183,325 using Amazon EC2 to exploit FREAK. He continues by citing one of the researchers who discovered FREAK explaining one way that a high number of false positives are generated on that particular vulnerability. He goes on to drive the point home, quoting the researcher and commenting that it likely has not been exploited in the wild by an average attacker.

tqbf-freak

Roytman essentially dismisses all of this in his blog post while saying that I am correct, that “IDS alerts generate a ton of false positives, vulnerability scanners often don’t revisit signatures, CVE is not a complete list of vulnerability definitions. But those are just the trees, and we’ll get to them later.” Unfortunately, he doesn’t get to it later in a way that provides meaningful insight into the questions about the data and conclusions. Dan Guido wrote an excellent summary of why the DBIR vulnerability section has issues, and factors in Roytman’s latest blog, breaking it all down in a manner that highlights the flaws. Even with the revised list, it is still missing the US-CERT top 30 previously cited, the Microsoft data, and the recently disclosed ‘top PoC exploits distributed on social media‘. At some point, one would logically conclude these lists should have more overlap. One thing I would love to see from Verizon and Kenna is a detailed explanation of their methodology as it relates to detecting client-side exploits, that appear to be the defacto standard for infecting tens, maybe hundreds of thousands of hosts, every year to create botnets.

I want to look at this from one more perspective, because I think it beautifully highlights how vulnerability analysis is a moving target, but in this case for all the wrong reasons. While most vulnerability aggregators and analysts are constantly adapting to new variations of vulnerabilities, new sources of vulnerability information, and new players in the game with wildly different styles of disclosure, others that come along after the data is generated and do analysis frequently seem to lose perspective in my experience. I believe this is such a case and is best illustrated in what the DBIR top 10 looks like over three revisions in less than two weeks. Yes, I know Roytman’s list isn’t officially the DBIR list, but he generated the initial data and then opted to do a different form of analysis putting it forth as something that is more representative due to applying his analysis to a more limited dataset that he presumably trusts more (i.e. the Kenna aggregate dataset). One has to wonder if this was brought up to Verizon as a better way to approach the list, and if so, why was it rejected.

The following lists show the evolution of the CVEs that appear on the top 10, with strikeout denoting the typos between the original DBIR and Gabe’s clarification which is reflected in current downloads, underlining to show any denial of service attacks, and bold to show the new CVE IDs that appear with Roytman’s reworking.

DBIR      - DBIR Revised   - Roytman Blog
2015-1637 – 2015-1637     - 2015-1637
2015-0204 – 2015-0204     - 2015-0204
2012-1054 – 2002-1054     - 2014-0160
2011-0877 – 2001-0877     - 2012-0152
2003-0818 – 2003-0818     - 2013-0229
2002-0126 – 2002-0126     - 2002-0012
2002-0953 – 2002-0953     - 2002-0013
2001-0876 – 2001-0876     - 2001-0876
2001-0680 – 2001-0680     - 2001-0877
1999-1058 – 1999-1058     - 2001-0540

In my mail to Roytman asking for a sample data set, I suggested that it would be interesting to see him generate a list using his methodology, but remove any DoS attack (six of his ten), so the top list only included exploits that could achieve remote privileges of some sort. He replied to me with:

… and again, awesome idea. One of those all-too-simple in retrospect, damnit why didn’t I think of it earlier things.

I found this interesting thinking back to his use of the forest and the trees metaphor.

A Note on the Verizon DBIR 2016 Vulnerabilities Claims

[This was originally published on the OSVDB blog.]

[Updated 4/28/2016]

Verizon released their yearly Data Breach Investigations Report (DBIR) and it wasn’t too long before I started getting asked about their “Vulnerabilities” section (page 13). After bringing up some highly questionable points about last year’s report regarding vulnerabilities, several people felt that the report did not stand up to scrutiny. With a few questions leveled at me, I was curious if Verizon and partners learned from last year.

This year’s vulnerability data was provided by Kenna Security (formerly Risk I/O), and Verizon “also utilized vulnerability scan data provided by Beyond Trust, Qualys and Tripwire in support of this section.” So the data isn’t from a single vendor, but at least four vendors, giving the impression that the data should be well-rounded, and have less questions than last year.

From the report:

Secondly, attackers automate certain weaponized vulnerabilities and spray and pray them across the internet, sometimes yielding incredible success. The distribution is very similar to last year, with the top 10 vulnerabilities accounting for 85% of successful exploit traffic. While being aware of and fixing these mega-vulns is a solid first step, don’t forget that the other 15% consists of over 900 CVEs, which are also being actively exploited in the wild.

This is not encouraging, as they have 10 vulnerabilities that account for an incredible amount of traffic, and the footnoted list of CVE IDs suggests the same problems as last year. And just like last year, the report does not explain the methodology for detecting the vulnerabilities, does not include details about the generation of the statistics, and provides a loose definition of what “successfully exploited” means. Without more detail it is impossible for others to reproduce their results, and extremely difficult to explain or disclaim them as a third party reading the report. Going to the Kenna Security page about this report doesn’t really yield much clarity, but does highlight another potential flaw in the methodology:

Kenna’s Chief Data Scientist Michael Roytman was the primary author of this year’s “Vulnerabilities” chapter, analyzing a correlated threat data set that spans 200M+ successful exploitations across 500+ common vulnerabilities and exposures from over 20,000 enterprises in more than 150 countries.

It’s subtle, but notice they went through a data set that spans exploitations across “500+ common vulnerabilities and exposures”, also known as CVE. If the data is only looking for CVEs, then there is an incredibly large bias at play from the start, since they are missing at least half of the disclosed vulnerabilities. More importantly, this becomes a game of fractions that the industry is keen to overlook at every opportunity:

  • CVE represents approximately half of the disclosed vulnerabilities.
  • Vulnerability scanners and IPS/IDS don’t have signatures for all CVE IDs, so they look for some fraction of CVE.
  • Detection signatures are often flawed, leading to false positives and false negatives, meaning they are actually detecting a fraction of the CVE IDs they intend to.

Another crucial factor in how this data is generated is in the detection of the exploits. Of the four companies contributing data, one was founded in 2009 (Risk I/O / Kenna) and another in 2006 (Beyond Trust, in the context of this discussion). That leaves Qualys (founded 1999) and Tripwire (founded 1997), who are likely the sources of the signatures that detected the vulnerabilities. For those around in the late 90s, the vulnerability landscape was very different than today, and security products based on signatures back then are in some ways considered rudimentary compared to today. Over time, most security products do not revisit older signatures to improve them unless they have to, often due to customer demand. Newly formed companies basically never go back and write signatures for vulnerabilities from 1999. So it stands to reason that the detection of these issues are based on Qualys and/or Tripwire’s detection capabilities, and the signatures detecting these vulnerabilities are likely outdated and not as well-constructed as compared to their more recent signatures.

That leads us to ask, how many vulnerabilities are these companies really looking for? Where did the detection signatures originate and how accurate are they? While the DBIR does disclaim that the data used is a sample, they also admit “bias undoubtedly exists”. However, they don’t warn the reader of these extremely limiting caveats that put the entire data set into a perspective clearly showing strong bias. This, combined with the lack of detailed methodology for how these vulnerabilities are detected and correlated to measure ‘success’, ultimately means this data has little value other than for inclusion in pedestrian reports on vulnerabilities.

With that in mind, I can only go by what information is available. We’ll start with the concise list of the top 10 CVE IDs these four vulnerability intelligence providers say are being exploited the most, and Verizon labels as “successfully exploited”:

  1. 2015-03-05 – CVE-2015-1637 – Microsoft Windows Secure Channel (Schannel) RSA Temporary Key Handling EXPORT_RSA Ciphers Downgrade MitM (FREAK)
  2. 2015-01-06 – CVE-2015-0204 – OpenSSL RSA Temporary Key Handling EXPORT_RSA Ciphers Downgrade MitM (FREAK)
  3. 2012-02-25 – CVE-2012-1054 – Puppet k5login File Symlink File Overwrite Local Privilege Escalation
  4. 2011-07-19 – CVE-2011-0877 – Oracle Enterprise Manager Grid Control Instance Management Unspecified Remote Issue (2011-0877)
  5. 2004-02-10 – CVE-2003-0818 – Microsoft Windows ASN.1 Library (MSASN1.DLL) BER Encoding Handling Remote Integer Overflows
  6. 2002-01-15 – CVE-2002-0126 – BlackMoon FTP Server Multiple Command Remote Overflow
  7. 2001-12-26 – CVE-2002-0953 – PHPAddress globals.php LangCookie Parameter Remote File Inclusion
  8. 2001-12-20 – CVE-2001-0876 – Microsoft Windows Universal Plug and Play NOTIFY Directive URL Handling Remote Overflow
  9. 2001-04-13 – CVE-2001-0680 – QVT/Net / Term FTP Server LIST Command Traversal Remote File Access
  10. 1999-11-22 – CVE-1999-1058 – Vermillion FTPD Long CWD Command Handling Remote Overflow DoS

This list should raise serious red flags for anyone passingly familiar with vulnerabilities. Not only do we have very odd ‘top 10’ lists from last year and this year, but there is little overlap between them. How does 2015 show a top 10 list exploiting eight vulnerabilities with CVE identifiers between 1999 and 2002, meaning they had been exploited so much as many as thirteen years later, only to see them all drop off the list this year, to be replaced by new 15+ year old vulnerabilities? In addition to this oddity, there are more considerations leading to my top 10 list of questions about their list:

  1. How does a local vulnerability based on a symlink overwrite flaw (CVE-2012-1054) make it into a top 10 list of “85% of successful exploit traffic“?
  2. How does a local vulnerability in Puppet rank #3 on this list, given the install base of Puppet as compared to Adobe or Java?
  3. If they are detecting exploits on the wire, shouldn’t we see Java, Adobe Reader, and Adobe Flash somewhere on the list? The “Slow and steady—but how slow?” section even talks about time-to-exploit for Adobe.
  4. Why doesn’t this list remotely match US-CERT’s “Top 30 Targeted High Risk Vulnerabilities” that includes vulnerabilities back to 2006, but not a single one listed above?
  5. How does a vulnerability that by all accounts is so vague, that it has to be distinguished by the vendor issued CVE ID (CVE-2011-0877), have a signature and get exploited so much?
  6. How does a vulnerability in Oracle Enterprise Manager Grid Control show up as #4, when no Oracle Database vulnerabilities appear?
  7. How do you distinguish an FTP LIST command exploit from one vendor to another? (e.g. CVE-2005-2726, CVE-2002-0558, CVE-2001-0933, CVE-2001-0680) According to the one-liner methodology, this is done via pairing SIEM data, suggesting that BlackMoon and Vermillion are that popular today.
  8. Yet, how does a remote DoS in an Windows-based FTP program that doesn’t appear to have been distributed for a decade make it on this list? Are people really conducting targeted DoS attacks against this software?
  9. Is BlackMoon FTP Server really that prevalent to be exploited so often?
  10. Or is there a problem in generating this data, which would be more easily attributed to loose signatures detecting FTP attacks regardless of vendor?

Figure 12 in the report, which is described as “Count of CVEs exploited in 2015 by CVE publication date” is a curious thing to include, as the CVE publication date is very distinct from the vulnerability disclosure date. While a large percentage of CVE publication dates are within seven days of disclosure, many are not (e.g. CVE-2015-8852 disclosed 2015-03-23 and CVE publication on 2016-04-26). Enough to make this chart questionable as far as the insight it provides. Taking the data as presented, are they really saying that only ~ 73 vulnerabilities with a 2015 ID were successfully exploited in 2015 across “millions of successful real-world exploitations“? Given that 40 vulnerabilities were discovered in the wild, 33 of which have 2015 CVE IDs, that means that only ~40 other 2015 vulnerabilities were successfully exploited? If that is the takeaway, how is the security industry unable to stop the increasing wave of data breaches, the same kinds that led to this report? Something doesn’t add up here.

While people are cheering about the DBIR disclaiming there is sample bias (and not really enumerating it), they ignore the measurement bias, don’t speak to publication bias, don’t explain the attrition bias between 2015 and 2016, or potential chaining bias. As usual, the media is happy to glom onto such reports without asking any of these questions or providing critical analysis. As an industry, we need to keep challenging metrics and statistics to ensure they are not only accurate, but provide meaning that benefits us.


4/28/2016According to Gabe (@gdbassett), the list of CVEs in the DBIR is incorrect. He posted a new list of CVEs (mostly the same) via Twitter in a reply to Andreas Lindh who was surprised at the top ten list of vulnerabilities as well. Gabe also confirmed that afterwards they “compared the figure CVEs (listed above) against the raw data. After removing non-confirmed breaches, they match.He went on to link to another source showing “data” about one of the CVEs, that really doesn’t mean anything without more context. Meanwhile, Michael Roytman, who did the vulnerability section of the report confirmed that he/Kenna would be responding to this blog with one of their own.

I hate to harp on simple transposition style mistakes in a report, but given the severity behind using numeric identifiers for vulnerabilities, it seems like that should have been double and triple-checked. Even then, I don’t understand how someone familiar with vulnerabilities could see either list and not ask many of the same questions I did, or provide more information in the report to back the claims. That said, let’s look at Gabe’s new list of CVEs. Bold and links are used to highlight the new ones:

  1. 2015-03-05 – 2015-1637 – Microsoft Windows Secure Channel (Schannel) RSA Temporary Key Handling EXPORT_RSA Ciphers Downgrade MitM (FREAK)
  2. 2015-01-06 – 2015-0204 – OpenSSL RSA Temporary Key Handling EXPORT_RSA Ciphers Downgrade MitM (FREAK)
  3. 2004-02-10 – 2003-0818 – Microsoft Windows ASN.1 Library (MSASN1.DLL) BER Encoding Handling Remote Integer Overflows
  4. 2002-07-22 – 2002-1054 – Pablo FTP Server LIST Command Arbitrary Directory Listing Remote Information Disclosure
  5. 2002-01-15 – 2002-0126 – BlackMoon FTP Server Multiple Command Remote Overflow
  6. 2001-12-26 – 2002-0953 – PHPAddress globals.php LangCookie Parameter Remote File Inclusion
  7. 2001-12-20 – 2001-0877 – Microsoft Windows Universal Plug and Play NOTIFY Request Remote DoS
  8. 2001-12-20 – 2001-0876 – Microsoft Windows Universal Plug and Play NOTIFY Directive URL Handling Remote Overflow
  9. 2001-04-13 – 2001-0680 – QVT/Net / Term FTP Server LIST Command Traversal Remote File Access
  10. 1999-11-22 – 1999-1058 – Vermillion FTPD Long CWD Command Handling Remote Overflow DoS

The list gained another FTP server issue that doesn’t necessarily lead to privileges, and another remote denial of service attack, while losing the Puppet symlink (CVE-2012-1054) and the vague Oracle Enterprise Manager issue (CVE-2011-0877). All said and done, the list is just as confusing as before, perhaps more so. That gives us four FTP vulnerabilities, only one of which leads to remote code execution, and two denial of service attacks, that gain no real privileges for an attacker. As Andreas Lindh points out, that I failed to highlight, having a man-in-the-middle vulnerability occupy two spots on this list is also baffling in the context of the volume of attacks stated. Also note that with the addition of CVE-2002-1054 (Pablo FTP), there are now two vulnerabilities that appear on the DBIR 2015 and DBIR 2016 top ten CVE list.

Hopefully the forthcoming blog from Michael Roytman will shed some light on these issues.

A Note on the Verizon DBIR 2015, “Incident Counting”, and VDBs

[This was originally published on the OSVDB blog.]

Recently, the Verizon 2015 Data Breach Investigations Report (DBIR) was released to much fanfare as usual, prompting a variety of media outlets to analyze the analysis. A few days after the release, I caught a Tweet linking to a blog from Rory McCune that challenged one aspect of the report. On page 16 of the report, Verizon lists the top 10 CVE ID exploited.

dbir-top10-cve

The bottom of the table says “Top 10 CVEs Exploited” which can be interpreted many ways. The paragraph above qualifies it as the “ten CVEs account for almost 97% of the exploits observed in 2014“. This is problematic because neither fully explains what that means. Is this the top 10 detected from sensors around the world? Meaning the exploits were launched and detected, with no indication if they were successful? Or does this mean these are the top ten exploits that were launched and resulted in a successful compromise of some form?

This gets more confusing when you read McCune’s blog that lists out what those ten CVE correspond to. I’ll get to the one that drew his attention, but will start with a different one. CVE-1999-0517 is an identifier for “an SNMP community name is the default (e.g. public), null, or missing.” This is very curious CVE identifier related to SNMP as it is specific to a default string. Any device that has a default community name, but does not allow remote manipulation is a rather trivial information disclosure vulnerability. This begins to speak to what the chart means, as “exploit” is being used in a broad fashion and does not track to the usage many in our industry associate with it. McCune goes on to point another on their list, CVE-2001-0540 which is an identifier for a “memory leak in Terminal servers in Windows NT and Windows 2000 allows remote attackers to cause a denial of service (memory exhaustion) via a large number of malformed Remote Desktop Protocol (RDP) requests to port 3389“. This is considerably more specific, giving us affected operating systems and the ultimate impact; a denial of service. This brings us back to why this section is in the report when part (or all) of it has nothing to do with actual breaches. After these two, McCune points out numerous other issues that directly challenge how this data was generated.

Since Verizon does not explain their methodology in generating these numbers, we’re left with our best guesses and a small attempt at an explanation via Twitter, that leads to as many questions as answers. You can read the full Twitter chat between @raesene, @vzdbir, and @mroytman (RiskIO Data Scientist who provided the data to generate this section of the report). The two pages of ‘methodology’ Verizon provides in the report (pages 59-60) are too high-level to be useful for the section mentioned above. Even after the conversation, the ‘top 10 exploited’ is still highly suspicious and does not seem accurate. The final bit from Roytman may make sense in RiskIO’s world, but doesn’t to others in the vulnerability world.

That brings me to the first vulnerability McCune pointed out, that started a four hour rabbit hole journey for me last night. From his blog:

The best example of this problem is CVE-2002-1931 which gets listed at number nine. This is a Cross-Site Scripting issue in version 2.1.1 and 1.1.3 of a product called PHP Arena and specifically the pafileDB area of that product. Now I struggled to find out too much about that product because the site that used to host it http://www.phparena.net is now a gambling site (I presume that the domain name lapsed and was picked up as it got decent traffic). Searching via google for information, most of the results seemed to be from vulnerability databases(!) and using a google dork of inurl:pafiledb shows a total of 156 results, which seems low for one of the most exploited issues on the Internet.

This prompted me to look at CVE-2002-1931 and the corresponding OSVDB entry. Then I looked at the ‘pafiledb.php’ cross-site scripting issues in general, and immediately noticed problems:

pafiledb-before

This is a good testament to how far vulnerability databases (VDBs) have come over the years, and how our data earlier on is not so hot sometimes. Regardless of vulnerability age, I want database completeness and accuracy so I set out to fix it. What I thought would be a simple 15 minute fix took much longer. After reading the original disclosures of a few early paFileDB XSS issues, it became clear that the one visible script being tested was actually more complex, and calling additional PHP files. It also became quite clear that several databases, ours included, had mixed up references and done a poor job abstracting. Next step, take extensive notes on every disclosure including dates, exploit strings, versions affected, solution if available, and more. The rough notes for some, but not all of the issues to give a feel for what this entails:

pafiledb-notes1

Going back to this script being ‘more complex’, and to try to answer some questions about the disclosure, the next trick was to find a copy of paFileDB 3.1 or before to see what it entailed. This is harder than it sounds, given the age of the software and the fact the vendor site has been gone for seven years or more. With the archive finally in hand, it became more clear what the various ‘action’ parameters really meant.

pafiledb31

Going through each disclosure and following links to links, it also became obvious that every VDB missed the inclusion of some disclosures, and/or did not properly abstract them. To be fair, many VDBs have modified their abstraction rules over the years, including us. So using today’s standards along with yesterday’s data, we get a very different picture of those same XSS vulnerabilities:

pafiledb-after1

With this in mind, reconsider the Verizon report that says CVE-2002-1931 is a top 10 exploited vulnerability in 2014. That CVE is very specific, based on a 2002-10-20 disclosure that starts out referring to a vulnerability that we don’t have details on. Either it was publicly disclosed and the VDBs missed it, it was mentioned on the vendor site somewhere (and doesn’t appear to be there now, even with the great help of archive.org), or it was mentioned in private or restricted channels, but known to some. We’re left with what that Oct, 2002 post said:

Some of you may be familiar with Pafiledb provided by PHP arena. Well they just released a new version that fixed a problem with their counting of files. Along with that they said they fixed a possible security bug involving using Javascript as a search string. I checked it on my old version and it is infact there, so I updated to the new version so the bugs can be fixed and I checked it and it no longer works.

We know it is “pafiledb.php” only because that is the base script that in turn calls others. In reality, based on the other digging, it is very likely making a call to includes/search.php which contains the vulnerability. Without performing a code-level analysis, we can only include a technical note with our submission and move on. Continuing the train of thought, what data collection methods are being performed that assign that CVE to an attack that does not appear to be publicly disclosed? The vulnerability scanners from around that time, many of which are still used today, do not specifically test for and exploit that issue. Rather, they look for the additional three XSS disclosed in the same post further down. All three “pafiledb.php” exploits that call a specific action that correspond to /includes/rate.php, /includes/download.php, and /includes/email.php. It is logical that intrusion detection systems and vulnerability scanners would be looking for those three issues (likely lumped into one ID), but not the vague “search string” issue.

McCune’s observation and singling this vulnerability out is spot on. While he questioned the data in a different way, my method gives additional evidence that the ‘top 10’ is built on faulty data at best. I hope that this blog is both educational from the VDB side of things, and further encourages Verizon to be more forthcoming with their methodology for this data. As it stands, it simply isn’t trustworthy.

SANS Top 20 Report – Deja Vu

[This was originally published on the OSVDB blog.]

I previously blogged about the SANS Top 20 List in a pretty negative fashion. The list started off as the “Top 10 Vulnerabilities” and quickly expanded into the Top 20 Vulnerabilities. Even last year (2005), they were still calling it a “Top 20 Vulnerabilities” list when it clearly had become anything but that. This year, SANS finally wised up calling the list “SANS Top-20 Internet Security Attack Targets”. Yes, they are now listing the 20 most attacked ‘targets’, not ‘exploited vulnerabilities’. With this change, does the list regain some of the value it originally had and quickly lost? Let’s look at the list:

Operating Systems
W1. Internet Explorer
W2. Windows Libraries
W3. Microsoft Office
W4. Windows Services
W5. Windows Configuration Weaknesses
M1. Mac OS X
U1. UNIX Configuration Weaknesses
Cross-Platform Applications
C1 Web Applications
C2. Database Software
C3. P2P File Sharing Applications
C4 Instant Messaging
C5. Media Players
C6. DNS Servers
C7. Backup Software
C8. Security, Enterprise, and Directory Management Servers
Network Devices
N1. VoIP Servers and Phones
N2. Network and Other Devices Common Configuration Weaknesses
Security Policy and Personnel
H1. Excessive User Rights and Unauthorized Devices
H2. Users (Phishing/Spear Phishing)
Special Section
Z1. Zero Day Attacks and Prevention Strategies

So if you run Windows, Unix, or MacOS .. and/or have Web Applications, Database software, allow P2P file sharing, allow IM messaging, have media players (installed by default on most OSs), run DNS servers, run Backup Software, run Security/Enterprise/DM servers .. and/or use VoIP servers/phones or “network and other devices”.. and/or have weak policy governing user rights or don’t prohibit certain devices and you actually have users.. you have at least one of the “Top 20 Attack Targets”. Wow, is that ever so helpful. Oh, I forgot, failing all of that, “Zero Day Attacks” are a top 20 attack vector.

Hey SANS, could you make a more overly vague and general security list next time? Maybe for 2007 you could shorten it from the “Top 20” to the “Top 1” and just list “C1: Have a computer type device”. That would save your analysts a lot of time and be just as helpful to the masses. Seriously, ditch the list or go back to the basics.

SANS Top 20 Report Value

[This was originally published on the OSVDB blog.]

SANS has released their Top 20 Internet Security Vulnerabilities for 2005. Started in June 2000, “the SANS Institute and the National Infrastructure Protection Center (NIPC) at the FBI released a document summarizing the Ten Most Critical Internet Security Vulnerabilities”. The list was designed to help administrators tackle the most critical issues on their network, targeting the most often exploited vulnerabilities.

Looking back at the first report, the list covers fairly specific programs or services with known vulnerabilities. BIND (named), RPC services (rpc.ttdbserverd, rpc.cmsd, rpc.statd), IIS RDS, Sendmail, sadmind and mountd, unpassworded default accounts, IMAP/POP and SNMP are eight of the ten on the original list. Two of the items are more abstract, covering “CGI programs and application extensions (e.g., ColdFusion) installed on web servers” as well as “Global file sharing and inappropriate information sharing via NetBIOS [..] or UNIX NFS exports”. Overall, this list provides a good basis for mitigating vulnerabilities, which is arguable based on your definition of the word.

Wikipedia defines a vulnerability as “a weakness or other opening in a system”. Fortunately for me/us, CVE has already covered this issue with their Terminology page and I don’t have to expand on this in more depth. This definition is a fundamental distinction in the world of VDBs and important when evaluating anything (especially papers) on vulnerabilities. While SANS does not explicitly define ‘vulnerability’, the wording of the introduction leans toward a definition of a specific software flaw that can be used to gain access or elevated privileges, rather than the more broad definition of a weakness/opening (or ‘exposure‘ as CVE calls it).

Based on that observation, the original Top20 list kept to the point and formed a list of specific flaws in computer systems (8 of 10) as well as two exposures. Looking at the current list, one has to wonder if the current level of abstraction has made the list more like a beginner’s guide to securing your computer rather than a list of often exploited vulnerabilities. The 2005 list does not make a distinction between the use of the word vulnerability over the last five years, nor does it define the word. In future lists, I believe it is essential that SANS begin to do so and disclaim their work accordingly.

The first entry on the 2005 list is “Windows Services” (W1) which covers just about every major Microsoft Windows service imaginable, including MSDTC and COM+ Service, Print Spooler Service, Plug and Play Service, Server Message Block Service, Exchange SMTP Service, Message Queuing Service, License Logging Service, WINS Service, NNTP Service, NetDDE Service and Task Scheduler. This type of entry more qualifies as an exposure backed by dozens of vulnerabilities. The second and third entries are “Microsoft Internet Explorer” (W2), an entire Web Browser, and “Windows Libraries” (W3) which are most certainly general exposures, not specific vulnerabilities. Fourth, “Microsoft Office and Outlook Express” (W4) cover half a dozen programs including Word, Excel, PowerPoint and Access. Many of the vulnerabilities associated with these products may overlap with “Windows Libraries” as well. The fifth entry “Windows Configuration Weaknesses” (W5) is an abstract entry that can cover an enormous range of exposures that can leave a system open to attack. So far, 5 out of 5 Windows related entries are exposures, not vulnerabilities.

The second section covers Cross-Platform Applications which sets itself up to be a fairly high level entry. Backup software (C1), Anti-virus Software (C2), PHP-based Applications (C3), Database Software (C4), File Sharing Applications (C5), DNS Software (C6), Media Players (C7), Instant Messaging Applications (C8), Mozilla and Firefox Browsers (C9), and Other Cross-platform Applications (C10) make up this wide ranged list that covers an incredible amount of software, operating systems and protocols. It is easy to argue that this list covers well over 50% of the vulnerabilities reported in 2004 (over 4,500 disclosed, and over 6,200 in 2005), making one question the value of a “Top 20” list that covers thousands of vulnerabilities. Consider that “backup software” or “media players” is fairly high level and reaches a level of abstraction that matches beginner security literature. Then consider “other cross-platform applications” is a catch-all for almost everything else out there since most products can be installed on any operating system these days.

The third section starts out with a high level of abstraction and gets worse. First is UNIX Configuration Weaknesses (U1) which would cover a wide range of operating systems such as Linux (all flavors), FreeBSD, NetBSD, OpenBSD, Solaris, AIX, HP-UX, IRIX and dozens of other UNIX variants. This would also cover Mac OS X which is a UNIX based operating system, which is very curious given the second entry under this section. “Mac OS X” (U2) is an entire operating system, and this entry even covers some software that comes with it such as the Safari web browser. Considering Microsoft IE and Mozilla browsers got their own entries above, we begin to see that the list doesn’t even maintain a consistant level of abstraction.

The last sections covers “Top Vulnerabilities in Networking Products”. We start with “Cisco IOS and non-IOS Products” (N1) which covers any product made by Cisco. Given their high installation base, they are an obvious target for vulnerabilities but once again, listing an entire vendor which includes their thousands of products? Next it lists “Juniper, CheckPoint and Symantec Products” (N2) which cover all products by these three vendors. Reading the definition of this entry makes me wonder if they ran out of time when trying to finish the document. Vulnerabilities were announced in these products.. exploit code available for some.. duh? This applies to any software out there, even other vendors with a high installation base. Last on the list is “Cisco Devices Configuration Weaknesses” (N3) which seems to be redundant to N1. One entry for “all Cisco products” and a second entry for “misconfiguring all Cisco products”?!

In summary, the “The Twenty Most Critical Internet Security Vulnerabilities (Updated) ~ The Experts Consensus” covers what? Certainly not the Top 20 vulnerabilities as defined by many security practitioners and vulnerability databases. You can’t even argue the list contains 20 exposures when it lists entire operating systems (Mac OS X/U2), configuration weaknesses (Windows/W5, Unix/U1, Cisco/N3) as well as redundant entries (Cisco N1 and N3). So, what does this list really cover? Who is this list supposed to help exactly? Telling administrators to upgrade all software and verify all configurations seems like a shorter and sweeter way of saying the same thing.