The total number of publicly reported web application vulnerabilities has risen sharply, to the point where they have overtaken buffer overflows. This is probably due to ease of detection and exploitation of web vulnerabilities, combined with the proliferation of low-grade software applications. In 2005 and 2006, cross-site scripting (XSS) was number 1, and SQL injection was number 2. PHP remote file inclusion is number 3 in 2006; because it allows arbitrary code execution on a vulnerable server, this is a worrisome trend, although proper configuration is frequently enough to eliminate it.

Buffer overflows are still the number 1 issue as reported in operating system (OS) vendor advisories. XSS is still high in this category, at number 2 in 2005 and number 3 in 2006, although other web application vulnerabilities appear much less frequently.

Integer overflows, barely in the top 10 overall in the past few years, are in the top 3 for OS vendor advisories. This might indicate expert researcher interest in high-profile software.

There are noticeable differences in the types of vulnerabilities being reported in open and closed source OS vendor advisories. These merit further investigation because they might reflect important differences in development, research, and disclosure practices.

The data is inconclusive regarding whether there is a concrete improvement in overall software security. While there is a rise in "new" vulnerability classes, the raw numbers for older classes have not changed significantly. Further investigation is also required in this area.

The most basic data manipulations for these vulnerabilities are very simple to perform, e.g., "'" for SQL injection and "<script>alert('hi')</script>" for XSS. This makes it easy for beginning researchers to quickly test large amounts of software.

There is a plethora of freely available web applications. Much of the code is alpha or beta, written by inexperienced programmers with easy-to-learn languages such as PHP, and distributed on high-traffic sites. The applications might have a small or non-existent user base. Such software is often rife with easy-to-find vulnerabilities, and it is often a target for beginning researchers. The large number of these "fish-in-a-barrel" applications is probably a major contributor to the overall trends.

With XSS, every input has the potential to be an attack vector, which does not occur with other vulnerability types. This leaves more opportunity for a single mistake to occur in a program that otherwise protects against XSS. SQL injection also has many potential attack vectors.

Despite popular opinion that XSS is easily prevented, it has many subtleties and variants. Even solid applications can have flaws in them; consider non-standard browser behaviors that try to "fix" malformed HTML, which might slip by a filter that uses regular expressions. Finally, until early 2006, the PHP interpreter had a vulnerability in which it did not quote error messages, but many researchers only reported the surface-level "resultant" XSS instead of figuring out whether there was a different "primary" vulnerability that led to the error.

There is some evidence that over the past couple of years, web defacers have taken an interest in performing and publishing their own research. This is probably due to the ease of finding vulnerabilities, combined with the presence of high-risk problems such as PHP file inclusion, which can be used to remotely install powerful, easily-available backdoor code. Based on customer posts to numerous vendor forums, there is solid evidence that remote file inclusion is regularly used to compromise web servers, which also helps to explain its popularity.

For 2006, the top 5 vulnerability types are responsible for 57% of all CVEs. With over 35 vulnerability types used in this report, and dozens more as currently identified in CWE, this shows how most public reports concentrate only on a handful of vulnerability types.

PHP remote file inclusion (php-include) has been steadily gaining ground since 2001, enough so that it is number 3 at this point in 2006. See items (2) and (5) from the previous section for a possible explanation.

Over the years, there has been a noticeable decline in shell metacharacters, symbolic link following, and directory traversal. It is unclear whether software is actually improving with respect to these problems, or if they are not investigated as frequently.

Information leaks (infoleak) appear regularly. There are 2 main reasons for the prominence: "information leak" is a more general class than others (see CWE for more precise sub-categories), and when an error message includes a full path, that is usually categorized as an information leak, although it might be resultant from a separate primary vulnerability.

The inability to handle malformed inputs (dos-malform), which usually leads to a crash or hang, is also a general class. Malformed-input vulnerabilities have not been studied as closely as injection vulnerabilities, at least with respect to identifying the root cause of the problem. Also, many reports do not specify how an input is malformed. There are likely many cases in which a researcher accidentally triggers a more serious vulnerability but does not perform sufficient diagnosis to determine the primary issue. Finally, vendor reports might only identify an issue as being related to "malformed input," which obscures the primary cause.

As the percentage of buffer overflows has declined, there has been an increase in related vulnerability types, including integer overflows (int-overflow), signedness errors, and double frees (double-free). These are still very low-percentage, probably due to their relative newness and difficulty of detection compared to classic overflows. In addition, these newly emerging vulnerability types might be labeled as buffer overflows, since they often lead to buffer overflows, and the "buffer overflow" term is used interchangeably for attack, cause, and effect.

Other interesting web application vulnerabilities are webroot (storage of sensitive files under the web document root), form-field (web parameter tampering), upload of files with executable extensions (e.g., file.php.gif), eval injection, and Cross-Site Request Forgery (CSRF).

Integer overflows are heavily represented in OS vendor advisories, rising to number 2 so far in 2006, even though they represent a small percentage of vulnerabilities overall. This probably reflects growing interest by expert researchers in finding integer overflows, along with the tendency of expert researchers to evaluate widely deployed software. The affected software ranges widely, including the kernel, cryptographic modules, and multimedia file processors such as image viewers and music players. After 2004, many of the reported issues occur in libraries or common DLLs.

Buffer overflows are still #1. This is probably due to under-representation of web applications in OS advisories, relative to other CVEs. In addition, as related issues like integer overflows increase, they might be detected or reported as buffer overflows, since buffer overflows are frequently resultant from integer overflows.

XSS is still very common, even in OS advisories, and it appears with the same frequency as integer overflows in 2006. An informal analysis shows that the affected software includes web servers, web browsers, email clients, administrative interfaces, and Wiki/CMS.

With the exception of XSS, there is a wide gulf between web-related vulnerabilities in OS advisories and other issues. SQL injection is not even in the top 10 for OS advisories, and PHP remote file inclusion is practially nonexistent. Many other web-related vulnerabilities occupy the bottom of the chart. For SQL injection, it is possible that most OS-supported applications do not use databases, or aren't web accessible. SQL injection vulnerabilities are not web-specific, but it seems that they are rarely reported for non-web applications, so it is possible that this reflects some researcher bias.

Directory traversal and format string vulnerabilities are frequently reported at a higher rate in OS vendor advisories than elsewhere. The reason is unclear, because these vulnerabilities are not restricted to local attack vectors, so one might expect that they would also appear regularly in web applications. However, it is likely that researchers do not focus on format strings because they are rarely exploitable for code execution in languages other than C. In the case of PHP, many PHP functions are subject to both remote file inclusion and directory traversal, and it might be that only the file inclusion is publicly reported. (In fact, the overlap is so close that this sometimes causes difficulties with classification).

In 2006 so far, more than a quarter (27%) of the OS vendor advisories did not have sufficient details to actually classify the vulnerability (type "unk"). This is in sharp contrast to the non-OS issues, which comprise less than 8%. However, because of the data sets in question, the non-OS CVEs will include many non-coordinated disclosures that would, by their nature, require more details. The next table will demonstrate that it is not just closed source vendor advisories that omit sufficient details for vulnerability classification.

The "top 5" and "top 10" vulnerabilities in each year are a much smaller percentage of total vulnerabilities in OS vendor advisories than non-OS issues. For example, in 2005, the top 5 totaled 29.4% for OS issues, but 55% for non-OS. For OS issues, this suggests an increasing diversity in the kinds of vulnerabilities being reported, whereas for other issues, that diversity appears to be decreasing. However, this could be another reflection of the domination of web application vulnerabilities.

The percentage of "unknown" vulnerabilities - those that could not be classified due to lack of details - is significantly higher in closed source than open source advisories, and 45% so far for 2006. With such a wide discrepancy, it is difficult to know whether any of the remaining results in this section are significant. It should be noted that 10% of issues in open source advisories do not have enough details to classify the problem.

Buffer overflows are number 1 for both open and closed, with roughly the same percentage over the years.

Symbolic link vulnerabilities appear at a higher rate in open source than closed source, although this might be due to the non-Unix OSes in the data set. While Windows has "shortcuts" (.LNK) that are similar to Unix links, they appear very rarely in Microsoft advisories, or for Windows-based applications. It is not clear whether this is due to under-research or API/development differences. The author recalls that at least one Linux researcher appeared to concentrate on symbolic link issues in 2004 and 2005, so researcher bias might also be a factor.

Malformed-input vulnerabilities appear more frequently in closed source advisories than open source. This might be due to a lack of details in closed source advisories. If an advisory mentions a problem due to "malformed data," it might be assigned the dos-malform type. Another factor might be due to black box techniques. It seems likely that fuzzers and other tools would be used more frequently against closed source products than open source, but this is not known.

XSS vulnerabilities appear more frequently in open source advisories than closed, but this might be a reflection of vendor release policies for advisories. It seems that open source vendors are more likely to release advisories for smaller packages.

Format string vulnerabilities appear more frequently in open source. There are probably several factors. First, susceptible API library calls such as printf() are easily found in source code using crude methods, whereas binary reverse engineering techniques are not conducted by many researchers (this might also be an explanation for symbolic link issues). Second, many format string problems seem to occur in rarely-triggered error conditions, which makes them more difficult to test with black box methods.

Perhaps most surprising: it appears that, since 2003, the non-Unix closed source advisories have not mentioned any format strings. It is not clear why there would be such a radical difference, although it could be due to the lack of details in those advisories.

Integer overflows have been roughly the same rank for open and closed source. This is a curious similarity, since one might not expect open and closed source analysis techniques to be equally capable in finding these problems.

Another interesting example is in the use of default passwords. Over the years, very few open source vendor advisories have mentioned default passwords, whereas they appear with some regularity in closed source advisories. It is not clear whether this is a difference in shipping/configuration practices or vendor disclosure policies.

Shell metacharacter issues appear less frequently in non-Unix closed source than other closed source advisories. This result was found by a separate analysis; it is not evident in Table 4. This could be due to usage patterns of API functions such as CreateProcess() for Windows, and system() for Unix. This result is being reported because it is the most concrete example of how API functions might play a role in implementation-level vulnerabilities.

The vulnerability types could be tied to other CVE-normalized data, such as IDS, incident databases, or vulnerability scanning results. This could determine the types of vulnerabilities that are being actively exploited or detected in real-world enterprises.

More precise classification could be informative. Approximately 30% of CVEs have vulnerability types that cannot be described using the current classification scheme. Another 15% are "unknown" vulnerabilities whose disclosures do not have sufficient details to determine any vulnerability type, but this problem is unavoidable, since some vendors do not release these details.

A crude measure of researcher diversity might be possible by linking data to other vulnerability databases that record this information. This could be used to determine if the raw number of researchers is increasing (probably), how that rate is increasing relative to the number of vulnerabilities (unknown), and how many different bug types are found by the average researcher (probably fairly small). If such data is available, then a further breakdown could be performed based on professional researchers versus others.

More precise data sets could be identified, such as a cross-section of market leaders in various product categories, not just OS vendor advisories. CVE does not record this type of information.

Why aren't you giving out raw numbers for open vs. closed source?

Answer : we already said why. See paragraph 2 of the Table 4 analysis for a reminder.

Why are you releasing this report now, with incomplete 2006 data?

Answer : when MITRE mentioned the preliminary results at the Cyber Security Executive Summit on September 13, there was a lot more interest than we had originally anticipated. Subsequent discussion of the results might help us to provide a better report when 2006 is done.

How does this compare with the other summaries you've posted in the past? Why have the numbers and percentages changed for older years?

Answer : (1) we occasionally add CVEs for older issues, (2) some of the previously released summaries were cumulative instead of offering a year-by-year breakdown, and (3) eventually, as a new type of vulnerability is reported more frequently, the CVE project notices it enough to give it a name, or at least a type. Once we do that, we can go back and update the older CVEs that also had the issue. However, we often rely on keyword searches in CVE descriptions for doing these kinds of updates. The earliest reports of new vulnerability types probably don't get captured fully, because CVE descriptions frequently vary in the early days or months of a new vulnerability type. Most updates to these vulnerability trends trigger an informal review of the "other" vulnerabilities for the data set in order to update the type fields.

There are a lot more vulnerability types than what you've covered.

Answer : That's an observation, not a question. If a certain vulnerability type is not on the list, then it probably didn't appear frequently enough for the CVE project to track closely. There are several reasons: (1) the vulnerability type is selected from a large dropdown menu during CVE refinement, but also (2) our work in the Common Weakness Enumeration (CWE) is producing hundreds of vuln types, and we want that to become a little more stable before doing the next round of modifications to CVE data. Finally, (3) with approximately 3,500 vulnerabilities marked "other" or "not specified", it is cost-prohibitive to review each CVE when the set of categories is updated.

Why isn't my favorite web vulnerability here?

Answer : Many web vulnerabilities are difficult to classify because they are "multi-factor," i.e., they are composed of multiple bugs, weaknesses, and/or design limitations. Other web issues are really just specialized attacks that use other primary vulnerabilities. For example, most HTTP response splitting problems rely on CRLF injection, so they are classified under CRLF injection.

CWE, http://cwe.mitre.org

"Open Letter on the Interpretation of 'Vulnerability Statistics'"
Bugtraq, Full-Disclosure
January 5, 2006
http://lists.grok.org.uk/pipermail/full-disclosure/2006-January/041028.html