Finally the new Automater release is out!

With the exception of my review of the Volatility Malware and Memory Forensics class yesterday, it has been a while since I have posted here. Time for me to get back into the swing of things. The best way to do so is with a new release to the tool that really launched code development projects on TekDefense.

Automater is a tool that I orginially created to automate the OSINT analysis of IP addresses. It quickly grew and became a tool to do analysis of IP Addresses, URLs, and Hashes. Unfortunately though, this was my first python project and I made a lot of mistakes, and as the project grew it bacame VERY hard for me to maintain. 

Luckily, a mentor and friend of mine (@jameshub3r) offered his time and expertise to do an enitre re-write of the code that would focus on a modular extensible framework. The new code hits the mark as far as that is concerned. The real power of Automater is how easy it is to modify what sources are checked and what data is taken from them without having to modify the python code. To modify sources simply open up the sites.xml file and modify away. I'll do another post later that goes into more detail there.

To view a bit more about installation and usage head over to the new Automater page.

You can download the code directly on Github. Remeber Automater is not a single file anymore, you need to download all of the files in the Automater repo to the same directory. To the first person that reports a valid bug to me, I'll send you a random game on Steam.

Here are a few screenshots to hold you over until you get it running.

Automater updates

So as many of you have may have noticed, I have updated Automater a few times over the last couple of months to address some specific issues and add some functionality. The changelog is as follows:

Changelog:

1.2.4

[+] Modifed Robtex data pull to match sites new formatting

[+] Added Virustotal search for the hash function

1.2.3

[+] Added HTTP Proxy support. Will pull OS default proxy settings.

[+] Modified some variables for consistency 

[+] Added comments

[-] Removed JoeBox from hash search

1.2.2

[+] Fixed FortiGuard rating https://github.com/1aN0rmus/TekDefense/issues/10

[+] Display help when no arguments are given https://github.com/1aN0rmus/TekDefense/issues/8

[+] Added Hash Search functionality https://github.com/1aN0rmus/TekDefense/issues/7

[+] Sources for Hash search are VxVault, ThreatExpert, JoeSandBox, and Minotaur

1.2.1

[+] Modified regex in Robtex function to pick up "A" records that were being missed.

[+] Alienvault reputation data added by guillermogrande.  Thank you!

1.2

[+] Changed output style to @ViolentPython style

[+] Fixed IPVoid and URLVoid result for new regexes

[+] Fixed form submit for IP's and URLs that were not previously scanned

So in short, it now has proxy support, pulls data from a few new places and will now take hashes as well. Don't worry we are not done with Automater though, I have a lot more planned.

Automater was the tool I wrote to learn basic python. As this was my first python project I made a lot of rookie mistakes. The code works and does what it is supposed to do, but it is sloppy and not optimized in the least. With that in mind, I plan to work on the next mjor release which will be a complete re-write of Automater from the ground up. Doing this should hopefully give us a more stable and extensible product.

See usage, installation, and download instructions at http://www.tekdefense.com/automater/

Automater 1.1 Release - IPVoid data fixed

Update: Automater gets its own project page http://www.tekdefense.com/automater/

Automator as most of you know, pulls IP and URL information from various sources in order to make analysis easier on the analyst.  Recently IPVoid changed up their site a bit, because of this I needed to make some modifications to Automater to get it to function appropriatley. To be specific I needed to modify the regular expressions to match the format of the new site.

I have some other changes that I will be making soon as well:

• Extra export options (csv, html)
• Malware domainlist checker
• Source engine selection 
• Re-write to utilize BeautifulSoup

If anyone has any other feature requests please let me know.

To see more about what Automater is and how it functions check out the tutorial.

1aN0rmus@TekDefense.com

Automater 1.0 - Passive IP and URL Analysis

Update: Automater gets its own project page http://www.tekdefense.com/automater/

Description:

Automater is an IP and URL analysis tool that I created to assist analyst in pulling data quickly and passively in mass.  This is the first stable release of the tool.

Features:

IPVoid: Pulls blacklist, ISP, and Geo Location

Robtex: Pulls DNS information. *A records only.

Fortiguard: Pulls URL Categorization

Unshotren.me: Will determine if the URL is shortened.  If it is it will display the final destination.

URLVoid - Pulls IP Address, blacklist, ISP, Geo Location, Domain creation date.

Installation:

 1. Download from github: 

wget https://github.com/1aN0rmus/TekDefense/archive/master.tar.gz

 2. Unzip the file:

tar -xvcf master.tar.gz 

 3. Make executable:

cd TekDefense-master

chmod +x *

 4. Now you are ready to run!

*Required Libraries: httplib2, re, sys, argparse, urllib, urllib2

Examples:

Display help information

./Automater.py -h

./Automater.py -t 188.95.52.162

./Automater.py -t securitytube.net

./Automater.py -e bit.ly/XDlV1q

./Automater.py -f hostsss

./Automater.py -f hostsss -o host.out

host.out

Video Demo:

Known Bugs:

• If the IP or URL has not been previously scanned at IPVoid or URLVoid, the script is supposed to submit the IP or URL and then pull results.  This seems to work most of the time, but on occasion it will not wait long enough to pull the appropriate result.  Running the command a second time will work though.
• Can not use the -e and -f switch together.
• URLs with http:// cannot be scanned.  Must take the http:// out for it to work.
• Please submit any other bugs to 1aN0rmus@tekdefense.com

Upcoming Features:

• For those who would like to be able to just query a specific engine or source such as robtex, we will be creating an option to do so.
• Check IP and/or URL against Malwaredomainlist
• Check IP and/or URL against malware sandboxes such as ThreatExpert.
• Summary report that will give statistics on the targets highlighting the known bad information such as blacklists and malicious URL categories.
• Please submit feature requests to 1aN0rmus@tekdefense.com