RE: Sources: Full and Partial Coverage
On Tue, 26 Jun 2012, Carsten Eiram wrote:
: > We're fairly ghetto, but OSVDB does a *lot* of source monitoring by hand.
: It takes a fair amount of manual labour to do it properly. Naturally, we
: don't sit in a browser visiting a huge list of sites every single day.
: We have robots monitoring mailing lists and web sites, checking for new
: discussions/content with certain keywords or new links.
Right. We have a weighted system based on the source, for priority in
checking the source. ICS-CERT and Adobe are 'priority 1' for example,
where low end software changelogs and bugtrackers are 'priority 9'.
Regardless, we rely on a person looking at the sources.
: > : 5. Have set searches for phrases that indicate important vulnerabilities
: > : ("overflow", "XSS", etc).
: That's one of the approaches we follow. Using that approach you, of
: course, need a solid list of keywords to ensure proper coverage. If you
: want to cover non-English sites you either need the same keywords in
: those languages as well or first run the monitored sites through a
: translation service e.g. Google Translate, hoping that it gets the
: translation right to trigger the keyword matches. It's a solid way to
: generate hits for further processing.
This is definitely a weakness for the automated parsing. Right now my
parser is only good for English and French. The list of keywords I believe
is robust. I had a solid list for several years, and then Steve Christey
contributed his list which almost doubled my own. It generates a
substantial amount of false positives, but I believe it is worth it as the
false negatives are likely much smaller.