Sunday 28 February 2016

Crowdsourced feeds from ThreatCrowd

Voting

Voting was added to ThreatCrowd recently, and I've been pleased to see a number of users regularly contributing votes.


These votes provide a useful source of malicious indicators, and so I've now put these into a feed in two files:

 https://www.threatcrowd.org/feeds/domains.txt
 https://www.threatcrowd.org/feeds/ips.txt
https://www.threatcrowd.org/feeds/hashes.txt

These feeds are not a substitute for the scale of auto-extracted command and control domains or the quality of some commercially provided feeds. But crowd-sourcing does go some way towards the quick sharing of threat intelligence between the community.

Updates
These files are updated once per hour, on the hour.

API
You can submit votes via the interface, or a simple API:

This will place a vote for "good.com" being non-malicious:
 https://www.threatcrowd.org/vote.php?vote=1&value=good.com

This will place a vote for "bad.com" being malicious:
 https://www.threatcrowd.org/vote.php?vote=0&value=bad.com

License
This data is available for free, and commercial use is allowed. It's licensed under http://www.apache.org/licenses/LICENSE-2.0
I make no guarantees to the quality of the data.

12 comments:

  1. Hi,

    I just added your IP Feed to FireHOL IP Lists, available at http://iplists.firehol.org/?ipset=threatcrowd

    ReplyDelete
  2. Thanks! Just looked at firehol - it's a great idea and I'd love to see it cover more of the commercial providers to give a quick idea of quality.

    ReplyDelete
  3. Me too! Let's see if the commercial ones are willing to be compared...

    ReplyDelete
    Replies
    1. True! Looks like someone compared a couple a while ago -> https://youtu.be/kstOKWL_OEk?t=18m12s

      Delete
    2. Interesting! This is what I found too: too few overlaps! It seems like the world is too big for anyone to cover the whole of it alone! My research shows that there are a few very interesting overlaps though: malware and abuse lists overlap to a great degree with proxies and anonymizers for example.

      Delete
  4. Hi,

    Your site SSL settings gives an error and can't load. Could you please check it?

    "ERR_SSL_VERSION_OR_CIPHER_MISMATCH"

    ReplyDelete
  5. Hi,

    Your site SSL settings gives an error and can't load. Could you please check it?

    "ERR_SSL_VERSION_OR_CIPHER_MISMATCH"

    ReplyDelete
  6. Two problems with the domains.txt starting around Nov1..
    listing "fi" - what is that, it's not a domain
    a few listings "*.exe" - those are not domains

    ReplyDelete
  7. Thanks! Sorry for the late reply. I've improved the validation.
    Still it's a bit hacky

    ReplyDelete
  8. How often are the list updated? It says every hour but there appears to be no changes.

    ReplyDelete
    Replies
    1. You can find its statistics here: http://iplists.firehol.org/?ipset=threatcrowd

      last time was updated Nov 16th 2016.

      Delete