Re: Counting on CVEs
: I have just had a very concerning discussion about the usefulness of
: CVEs as a means to measure vulnerabilities today and the decay of its
: value if the trend continues. The discussion centered around the
: accuracy of the numbers of CVEs identified compared to those reported in
: the community as a whole. If we looked at just the CVE numbers, it
: appears that the numbers of vulnerabilities have been dropping since a
: high in 2008. This is a rather important error. As we all know, this is
: not accurate. Vulnerabilities have not been dropping, they are growing,
: not dropping by 30%.
I can't find it right off, but this came up several years back when
several people noticed the drop in vulnerability totals around 2008. After
additional examination of CVE, OSVDB, Secunia, and I believe BID, all four
databases showed roughly the same drop. That in turn lead to speculation
about *why* it was happening. I don't recall seeing anyone showing a 5
year trending of vulnerability counts, as seen through multiple VDBs, but
I would honestly request to see some rough numbers before pursuing this
line of discussion further.
: information between fielded products and between research analysis
: teams. As the numbers decline, it means we are forced to look elsewhere
: to provide the identification and communication that CVE provided in the
: past. More proprietary ids are becoming more the norm.
Is it cheaper for your team to manufacturer proprietary IDs instead of
looking to Secunia, ISS, BID, and OSVDB for external references? If the
answer is 'yes', why rely on CVE at all any more? If the answer is 'no',
then perhaps these other VDBs are doing some of the work for you, where
CVE is currently not.
: "If the vulnerabilities have dropped 30% since 2008, why do I need to be
: funding the security staff and efforts at the rate I am? I see that
: MITRE is reporting an overall drop each year since 2008 but yet you
: folks keep coming to me saying that the threats are much worse and we
: may be in the same relative situation we were when malware spiked a
: couple years past?"
That quote alone is a non sequitur really. I know that the conversations
are more detailed with these executives, but even as a summary, it is too
Just because the number of unique vulnerabilities tracked by a VDB drops,
does not mean the number of attackers drop. Further, it does not mean risk
has dropped. Even further, more companies are using custom web
applications, and no VDB tracks site-specific vulnerabilities (one current
OSF project wants to, but we simply don't have funding or volunteers to
make it happen). These are just a few issues of why an overall
vulnerability count may drop, but their need to hire you and any other
security company is stronger than ever. I certainly hope you can steer
them past this limited view re: CVE.
: For those that actively work in this environment, we know
: vulnerabilities have not dropped 30% since 2008. Quite the contrary, our
: internal numbers indicate an increasing trend similar to a 30% rise.
: Symantec has also reported a similar trend.
Could you provide links to the Symantec report? Again, I am just curious
about the ~ 2008 thing where most VDBs reported a drop.
: Another problem is the CVE format itself. There has been an active
: discussion about the format limitations as well. The CVE format is
: CVE-YYYY-NNNN. This means that currently we cannot have more than 10,000
: CVEs reported in a single year. At the rates we are seeing internally,
: we are already there.
As I said in a previous reply, OSVDB hit the 10k mark in 2006 due to our
: Then there are the limitations of CVE process in general. The focus is
: English only although some non-US vulnerabilities do get assigned from
: time to time. CVE does not support the international community and
: software written for non-English geo-centric areas are not included.
: While this has not been a concern for US-only software vendors, there is
: a great deal of regional software written for major emerging markets.
: None of those vulnerabilities are identified by a CVE.
This is very difficult to manage. Even if CVE could keep up with all
English-based disclosures from all sources (e.g., think of their team
being triple in size at the very least), trying to keep up with foreign on
top of it is another huge jump in resources. Carsten Eiram over at Secunia
could probably give some insight on this better than anyone. I have
noticed over the years that he appears to have several analysts that speak
other languages, as they frequently pull up vulnerabilities with no
: Given these constraints, it seems like it is time to figure out how to
: address the issues in a more of a creative way. We know the
: constraints. Is there something we can do to augment the MITRE staff in
: certain areas that would help? I can see the format issue being a
: rather easy one to attack but it is the coverage issue that is most
: concerning. Or we should ignore it and slowly let the value of CVE to
: the community and vendors decay?
If MITRE can't get funding to keep up (and catch up), looking to augment
the effort from external staff would certainly be interesting. However, I
would like to make two points on that.
1. Having non-MITRE resources work on CVE would require an entire overhaul
of the process and policies around the project. Further, MITRE would have
to divert some resources away from the already struggling CVE to
implement some form of training program. I know this because Steve
Christey and I 'traded roles' for *one* entry many years back. He gave me
a link to a vulnerability and said "write a CVE". I gave him a link to a
vulnerability and said "mangle the OSVDB entry to 100%". We both learned a
lot from that process, and we both walked away with newfound respect for
each other's process. Trying to train up a dozen people to meet their
standards would not be easy.
2. OSVDB has been around for some time now, and I have been involved with
it since 2004. We have been an 'open' database, where anyone could sign up
for an account and contribute to our entries. We provided an extensive
walk-through, near 24 hour support for a while should anyone have
questions, reminder help screens for entries, templates to help ensure the
text was readable, and more. Over the last 8 or so years, less than 1% of
our data has come from volunteers. Off hand, I would guess that the bottom
1,000 volunteers who signed up have contributed less than the #8 ranking
contributor (http://osvdb.org/contributors). In short, the idea of an open
community-driven database simply did not work. We tried a wide variety of
campaigns to encourage people to help ranging from a reward system to
internships to resume experience working for a 501(c)(3). In all of that
time, less than 5 have stuck with the project and become core
contributors. While we don't quite have the exposure of CVE, we are
certainly not 'unknown'. We've had the framework to do this for a long
time, and I have to think that if the community wanted it, they would have
supported us. I know dozens of researchers and penetration testers that
"don't have the time", but are overjoyed when I can create and mangle an
entry for a vulnerability that no VDB has, so they can provide an external
reference in their reports.
OSF / OSVDB