Twitter Search to Cisco SecureX Threat Response Casebook [v1.1]

This is a sample script to search the Twitter hashtag #opendir for threat/malware related observables, check for Target Sightings and automatically add observables to Cisco Casebook. This enables Security Researchers and Threat Responders in a SOC to quickly see if the observables from #opendir have been seen in their environment (by leveraging SecureX Threat Response (SxTR)). The #opendir hashtag is used by many threat intelligence researchers to post their findings on new threats.

For more information on how to use SxTR, please review this link: https://visibility.amp.cisco.com/#/help/introduction.

Known issues

No known issues.

Release notes version 1.1

The "twitter" library is being used to reach the Twitter standard search API.
The "webexteamssdk" library is beinbg used to post send alerts.
The script also removes all clean observables from the case to stop false positives. Often legitimate websites are added in a tweet, but are not an observable associated directly with the malware campaign. This causes Target Sightings, without them being of much interest. Removing these from the investigation is also better for the performance of the script.
The script now also checks for Target Sightings. If there is a Sighting of a Target, the Webex Teams message and the Case description in Casebook will get a "HIGH PRIORITY" tag.
The Webex Teams message contains a direct investigate link to cross launch to SecureX threat response.

Overview

The script leverages the #opendir Twitter hashtag to find tweets related to new malware campaigns.
It will then check if this is the first time the script has run:
- If the script is being run for the first time, it will parse through all most recent tweets (maximum 100, which usually goes back 3-5 days).
- If the script has run before, it will check if there are new tweets in #opndir (using the “since_id” element from the Twitter standard search API).
  - If there was an update -> parse all the new tweets.
  - If there was no update -> do nothing.
During the parsing of the tweets, the SxTR API is used to retrieve all the observables (using the Inspect API endpoint).
It then removes observables with a clean disposition (retrieved from SxTR API).
It also checks for Target Sightings. If there is a Sighting of a Target, the Webex Teams message and the Case description in Casebook will get a "HIGH PRIOIRTY" tag.
The last step is to create a SxTR Casebook with the retrieved observables. The username and the text of the tweet will be added into the Case. Optionally, a Webex Teams message is sent to a room to update the Threat Responder.

Installation

Clone this repository or download the ZIP file.
Log in to https://visibility.amp.cisco.com/ with your Cisco Security credentials.
Make sure that you have Casebook enabled (+ the Casebook AMP, Threat Grid and Chrome widget, for extended functionality). Please find more information here: https://visibility.amp.cisco.com/#/help/casebooks.
Click on Modules.
Click on API Clients.
Click on Add API Credentials.
Give the API Credentials a name (e.g., Twitter #opendir parser).
Select at least the Casebook and Private Intelligence checkboxes; however, to be sure, you can also click Select All.
Add an optional description if needed.
Click on Add New Client.
The Client ID and Client Secret are now shown to you. Do NOT click on close until you have copy-pasted these credentials to the config.json file in the repository.
It is possible to integrate the script with Webex Teams. In order to do that, an API Access Token and a Room ID need to be entered in the config.json file. Please retrieve your key from: https://developer.webex.com/docs/api/getting-started. Then create a dedicated Webex Teams space for these notifications and retrieve the Room ID from: https://developer.webex.com/docs/api/v1/rooms/list-rooms. Please be aware that the personal token from the getting started page only works for 12 hours. Please follow these steps to request a token per request: https://developer.webex.com/docs/integrations. This is roadmapped for v3.0 of the script.
Make sure you apply for a Twitter developer account. After you have been accepted, you can create a Twitter app, which creates the right credentials to perform API queries.
Make sure that the config.json file looks like this (with the right keys and IDs filled in between the quotes):

{
  "ctr": {
      "client_id": "<client-id-here>",
      "client_password": "<client-password-here>"
  },
  "webex": {
      "access_token": "<access-token-here",
      "room_id": "<room-id-here>"
  },
  "twitter": {
      "token": "<token-here>",
      "token_secret": "<token-secret-here>",
      "consumer_key": "<consumer-key-here>",
      "consumer_secret": "<consumer-secret-here",
      "since_id": 0
  }
}

You are now ready to execute the script. Go to a terminal and change directory to the folder that contains your twitter_search.py and config.json file.
Make sure you have the correct libraries installed by executing the requirements.txt file (using a Python virtual environment!):

python3.6 -m venv twitter_venv
source twitter_venv/bin/activate
pip install -r requirements.txt

Now execute the twitter_search.py script:

python3.6 twitter_search.py

You are now done.

Notes and Road Map

Please feel free to use crontab to run the script every day. The script will handle this and create a new casebook only if a new blog is added. There is detailed information on how to use crontab here: https://pypi.org/project/python-crontab/.
Otherwise, you can also use a function I previously wrote, which is the intervalScheduler function in this script: https://github.com/chrivand/Firepower_O365_Feed_Parser/blob/VERSION-3.0/O365WebServiceParser.py.
Technically this script can work with any hashtag, please change it on line 226 of twitter_search.py. In newer versions, I will add the possibility the scan multiple hashtags.
I will keep updating this script and you can also do a pull request with an update.
Please open an "Issue" if there is something not working or if you have a feature request.
Currently the Webex Teams Authentication works with a temporary token. This will be improved with an official Webex Teams Integration (roadmapped for v2.0).

Use Case

The internet contains many free sources of threat intelligence that can be used in addition to Cisco Talos. Using the SecureX Threat Response API, it is possible to harvest this and discover internal security events. There is a big community out there that shares new "observables" (e.g. IP addresses, domains, file hashes, etc.) related to new cyber attacks and malware campaigns. It is very important nowadays to stay up to date with all of these threats that are posing all over the world. It is widely known that there are not enough resources to be found to fill up every Security Operation Center (i.e. SOC). Therefore, many organizations struggle with coping with the massive amount of new type of attacks.

The #opendir Twitter hashtag is used by many threat intelligence researchers to post their findings on new threats. This is a perfect example of one of those free sources of threat intelligence that can be found on the internet. However, who has the time to read all of these Tweets, check all of their security tools for hits, and take action on them?

Screenshot of #opendir Twitter feed:

The attached Code Exchange submission contains a sample script to search the Twitter hashtag #opendir for threat/malware related observables. It then checks for "Target Sightings" and automatically adds observables to SecureX Casebook. If there are any hits on internal targets, it will add a "HIGH PRIORITY" tag to the Case. It will also add the original content of the Tweet as description in the Case. It will also send a Webex Teams alert to a configured Space (e.g. the Space used by a SOC). This enables security researchers and threat responders in a SOC to quickly see if the observables from #opendir have been seen in their environment by leveraging SecureX Threat Response API.

Screenshot of Threat Response:

Business Summary

The internet contains many free sources of threat intelligence that can be used in addition to Talos. Using the SecureX Threat Response API, it is possible to harvest this and discover internal security events.