RSS Feed Blog Parser to Cisco SecureX Threat Response Casebook [v2.0]

This is a sample script to parse the Cisco Talos blog (and other blogs!), check for Target Sightings and automatically add observables to Cisco Casebook. This enables Security Researchers and Threat Responders in a SOC to quickly see if the observables from Talos have been seen in their environment (by leveraging SecureX Threat Response (SxTR)).

For more information on how to use SxTR, please review this link: https://visibility.amp.cisco.com/#/help/introduction.
If you would like to see a demo of the script, please check out the video below:

https://youtu.be/cCe3y6XZqs0

Known issues

Currently there is no limit on the amount of obsevables (or the size of the blog post) to be parsed by the script. This might cause errors when running this with some blog posts (e.g. the Talos Threat Roundup). Version 3.0 will have this rate limiting built in to adhere to the 2000 character max that should be added to an investigation. You can find more info on that on the following link.

Release notes version 2.0

The ciscospark library has been updated to the newer webexteamssdk library.
The script now also removes all clean observables from the case to stop false positives. Often legitimate websites are added in a blog, but are not an observable associated directly with the malware campaign. This causes Target Sightings, without them being of much interest. Removing these from the investigation is also better for the performance of the script.
The script now also checks for Target Sightings. If there is a Sighting of a Target, the Webex Teams message and the Case description in Casebook will get a "HIGH PRIORITY" tag.
The script now has support for more RSS feeds. The FortiGuard and Unit42 RSS feeds have now been added as example (on top of the Talos RSS Feed).
The script will use the RSS feed "entry.link" to download the full blogpost, and does not just look at the "entry.description" of the RSS feed. The FortiGuard blog for example does not include the observables in their RSS feed, but only shows them on the actual original blog post.
Since the script has been expanded, it now can run longer than 10 minutes. This is actually the expiration time of the SxTR OAuth token. Therefore, every API call now retrieves a new OAuth token.
The script has generally been cleaned up.

Overview

The script leverages the Cisco Talos, FortiGuard and Unit42 blog RSS feeds (and/or other blogs) to retrieve all the current blogs.
It will then check if this is the first time the script has run:
- If the script is being run for the first time, it will parse through all blogs.
- If the script has run before, it will check if there was an update to the blog (using the “last_modified” element from RSS).
  - If there was an update -> parse all the new blogs.
  - If there was no update -> do nothing.
During the parsing of the blog, an attempt is made to remove False Positives, like hyperlinks to other webpages (e.g., Snort.org).
After this the SxTR API is used to retrieve all the observables from the cleaned blog.
In version 2.0 it now removes observables with a clean disposition (retrieved from SxTR API).
In version 2.0 it now also checks for Target Sightings. If there is a Sighting of a Target, the Webex Teams message and the Case description in Casebook will get a "HIGH PRIOIRTY" tag.
The last step is to create a SxTR Casebook with the retrieved observables. The title of the blog and the link to the blog will be added into the Case. Optionally, a Webex Teams message is sent to a room to update the Threat Responder.

Installation

Clone this repository or download the ZIP file.
Log in to https://visibility.amp.cisco.com/ with your Cisco Security credentials.
Make sure that you have Casebook enabled (+ the Casebook AMP, Threat Grid and Chrome widget, for extended functionality). Please find more information here: https://visibility.amp.cisco.com/#/help/casebooks.
Click on Modules.
Click on API Clients.
Click on Add API Credentials.
Give the API Credentials a name (e.g., Talos Blog Parser).
Select at least the Casebook and Private Intelligence checkboxes; however, to be sure, you can also click Select All.
Add an optional description if needed.
Click on Add New Client.
The Client ID and Client Secret are now shown to you. Do NOT click on close until you have copy-pasted these credentials to the config.json file in the repository.
It is possible to integrate the script with Webex Teams. In order to do that, an API Access Token and a Room ID need to be entered in the config.json file. Please retrieve your key from: https://developer.webex.com/docs/api/getting-started. Then create a dedicated Webex Teams space for these notifications and retrieve the Room ID from: https://developer.webex.com/docs/api/v1/rooms/list-rooms. Please be aware that the personal token from the getting started page only works for 12 hours. Please follow these steps to request a token per request: https://developer.webex.com/docs/integrations. This is roadmapped for v3.0 of the script.
Make sure that the config.json file looks like this (with the right keys and IDs filled in between the quotes):

{
    "client_id": "<your_client_id>",
    "client_secret": "<your_client_secret>",
    "last_etag": "",
    "last_modified": "",
    "webex_access_token": "<your_webex_access_token>",
    "webex_room_id": "<your_webex_room_id",
    "url_feeds": [
      {
          "feed_name": "[Talos RSS Feed]",
          "rss_url": "http://feeds.feedburner.com/feedburner/Talos",
          "last_etag": "",
          "last_modified": 0
      },
      ... 
}

You are now ready to execute the script. Go to a terminal and change directory to the folder that contains your rss_feed_to_casebook.py and config.json file.
Make sure you have the correct libraries installed by executing the requirements.txt file (use a Python virtual environment if preferred):

pip3 install -r requirements.txt

Now execute the rss_feed_to_casebook.py script:

python3.6 rss_feed_to_casebook.py

You are now done.

Notes and Road Map

Please feel free to use crontab to run the script every day. The script will handle this and create a new casebook only if a new blog is added. There is detailed information on how to use crontab here: https://pypi.org/project/python-crontab/.
Otherwise, you can also use a function I previously wrote, which is the intervalScheduler function in this script: https://github.com/chrivand/Firepower_O365_Feed_Parser/blob/VERSION-3.0/O365WebServiceParser.py.
This script works with the Talos, FortiGuard and Unit42 RSS feed, but potentially it will also work with other RSS feeds. You will need to add or change the url_feeds variable (in config.json) with another RSS feed. Also, you might need to clean the hyperlinks, etc., out of the blogs in a different way (even though I am doing this quite genericly).
I will keep updating this script and you can also do a pull request with an update.
Please open an "Issue" if there is something not working or if you have a feature request.
Currently the Webex Teams Authentication works with a temporary token. This will be improved with an official Webex Teams Integration (roadmapped for v3.0).

Use Case

The internet contains many free sources of threat intelligence that can be used in addition to Cisco Talos. Using the SecureX Threat Response API, it is possible to harvest this and discover internal security events. There is a big community out there that shares new "observables" (e.g. IP addresses, domains, file hashes, etc.) related to new cyber attacks and malware campaigns. It is very important nowadays to stay up to date with all of these threats that are posing all over the world. It is widely known that there are not enough resources to be found to fill up every Security Operation Center (i.e. SOC). Therefore, many organizations struggle with coping with the massive amount of new type of attacks.

The Cisco Talos blog is a perfect example of one of those free sources of threat intelligence that can be found on the internet. Cisco Talos is Cisco's threat intelligence research group, and posts their findings on their blogs. However, who has the time to read all of these blog posts, check all of their security tools for hits, and take action on them? What about all of the other threat intelligence research group?

The attached Code Exchange submission contains a sample script to search the Talos blog for threat/malware related observables. It then checks for "Target Sightings" and automatically adds observables to SecureX Casebook. If there are any hits on internal targets, it will add a "HIGH PRIORITY" tag to the Case. It will also add the link to the original blog as description in the Case for reference of the analyst. It will also send a Webex Teams alert to a configured Space (e.g. the Space used by a SOC). The same script can also parse other blogs (e.g. FortiGuard and Unit42), and this enables organizations to gather other vendor's threat intelligence as well. This allows security researchers and threat responders in a SOC to quickly see if the observables from threat intelligence blogs have been seen in their environment by leveraging SecureX Threat Response API.

Screenshot of the Webex Teams Space that is used by the SOC:

Business Summary

The internet contains many free sources of threat intelligence that can be used in addition to Talos. Using the SecureX Threat Response API, it is possible to harvest this and discover internal security events.