I have been spending my nights this week working on a script that will generate a web front end for the results that MASTIFF produces. It has been a blast creating so far, but I would really like to hear from the community what suggestions they have for it.
Automater is a IP and URL Analysis tool we created to help automate the analysis process. You can see a video of Automater in action in TekTip episode 15.
In this episode of TekTip we take a look at performing basic static analysis with MASTIFF. While that is the focus of this episode I wanted to delve into Maltrieve first.
Maltrieve is a fork of MWCrawler which you guys and gals may remember from a previous TekTip video. Maltrieve was created by Kyle Maxwell@KyleMaxwell. While it has the same basic function of MWCrawler which is downloading malware from various web resources, it works much faster and has more reliable web resources it pulls from. @KyleMaxwell is working to add thug integration as well.
Once downloaded you run maltrieve without any options, as seen below:
This will download the malware to a default directory of /tmp/malware
*Make sure this directory exists or change the path in the python script to match what you want
So, with Maltrieve done and a bunch of samples downloaded it is time to see the power of MASTIFF.
MASTIFF is an automated framework for static analysis created by Tyler Hudak @SecShoggath and was funded by the Cyber Fast Track DARPA program. Too bad Cyber Fast track is going away, there are so many awesome projects coming out of it right now.
What MASTIFF will do is it will analyze a file to determine the file type (pdf, zip, PE32) and based on that file type it will run the appropriate static analysis tools against the sample. The output for tools it runs are organized and packaged up with some key information also making its way to a sqllite database.
Some of the benefits of this framework are:
Easily Extensible: Built very modular so adding to the functionality is easy
Consistent: When you have a team of analyst working on malware it is important that everyone speak the same language. MASTIFF gives a consistent standard approach to static analysis.
Quick: Manual static analysis can take a long time. With MASTIFF I can run through hundreds of samples in minutes.
Documented: As a consequence of being a DARPA funded program the creator was forced to ensure that the framework was documented well. The documentation goes beyond the normal installation and usage covering workflow and methodology.
I do not cover installation in the video as it would take to long and be very boring but I will mention that installation is relatively easy. The only real pain is ensuring you have all of the third party tools installed. The pdf inside the archive for MASTIFF has great documentation to get you up and running. Once the dependencies and MASTIFF are installed though ensure that you modify the mastiff.conf file to reference the appropriate paths for where you installed the third party tools.
Now that the config is good and MASTIFF is installed you are ready to start analyzing malware. running mas.py will show you usage.
[2013-02-23 21:47:41,645] [ERROR] [Mastiff.Plugins.yara.get_sigs] : /opt/yara-1.6/yara is not a directory or does not exist.
[2013-02-23 21:47:41,645] [INFO] [Mastiff.Analysis] : Finished analysis for /tmp/malware/86658467c74b39210de96111ee6f66d5.
Navigate to the directory you have set as the work log in the mastiff.conf to see the results
tekmalinux@TekMALinux:/work/log/86658467c74b39210de96111ee6f66d5$ ls -l
total 424
-rw-r--r-- 1 root root 267312 Feb 23 21:47 86658467c74b39210de96111ee6f66d5.VIR
-rw-r--r-- 1 root root 137 Feb 23 21:47 fuzzy.txt
-rw-r--r-- 1 root root 3440 Feb 23 21:47 mastiff.log
-rw-r--r-- 1 root root 1024 Feb 23 21:47 mastiff-run.config
-rw-r--r-- 1 root root 42100 Feb 23 21:47 peinfo-full.txt
-rw-r--r-- 1 root root 13317 Feb 23 21:47 peinfo-quick.txt
drwxr-xr-x 2 root root 4096 Feb 23 21:47 resources
-rw-r--r-- 1 root root 1332 Feb 23 21:47 resources.txt
-rw-r--r-- 1 root root 7704 Feb 23 21:47 sig.der
-rw-r--r-- 1 root root 27152 Feb 23 21:47 sig.txt
-rw-r--r-- 1 root root 42606 Feb 23 21:47 strings.txt
Nice, it looks like we pulled certificate info based on the sig.txt being there. To give you an example of the type of data you get, here is a cat of the peinfo-quick.txt:
MASTIFF does not currently have a native method to scan multiple files at once. While that is on the horizon for the project that is not a problem for us as we can just script out a quick program to do this. Of course you can always use mine.
#!/usr/bin/python
importos
# MASTIFF Autorun
# @TekDefense
# www.TekDefense.com
# Quick script to autorun samples from maltrieve to MASTIFF
malwarePath='/tmp/malware/'
forr,d,finos.walk(malwarePath):
forfilesinf:
malware=malwarePath+files
printmalware
os.system('mas.py'+' '+malware)
Simply change the directory in the script to point to where you have the samples and run the python program. Also be sure to keep this script in the same directory as mas.py.
[2013-02-23 22:00:56,698] [ERROR] [Mastiff.Plugins.yara.get_sigs] : /opt/yara-1.6/yara is not a directory or does not exist.
[2013-02-23 22:00:56,698] [INFO] [Mastiff.Analysis] : Finished analysis for /tmp/malware/ba91f309a81c1f6f1d7dcc5cb5094328.
/tmp/malware/a544ffb08f6177f6382df6101f78bfdc
Now that you have performed analysis against a bunch of samples you can analyze the results, or open up the sqllite database to pull some statistics.
As you can probably tell by now, I am really enjoying MASTIFF, in fact I am looking for any excuse to run it daily. Last week I was given a perfect event to apply MASTIFF too and that was Mandiant's report on APT1. VirusShare@VXShare was able to quickly compile a bunch of samples which a lot of folks started playing around with. I decided to run 20 or so of the samples through MASTIFF. If you would like to download those results you can get them in the download section.
I mentioned in the video that I was getting an error when running MASTIFF. I am not sure what is generating the error exactly quite yet, as I have checked that all the appropriate imports are in place. Once I figure it out I'll let you guys know what is going on. The error is below:
[2013-02-23 21:47:40,904] [ERROR] [yapsy] : Unable to import plugin: /opt/mastiff/mastiff-0.5.0/plugins/EXE/EXE-singlestring
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/Yapsy-1.10.1_pythons2n3-py2.7.egg/yapsy/PluginManager.py", line 486, in loadPlugins
In this episode of Tektip we review a tool we created kippo2Wordlist.
Description: kippo2Wordlist is a python program that reads logs from kippo to create a wordlist that can be used for anything a standard wordlist is used for such as pipal analysis, cracking passwords, and the like.
Installation: You can download the script from github. You can also clone the git repository if you have git installed. Place in any directory you like. I put it at:
/opt/kipp2Wordlist/
If you are using honeydrive and haven't changed where the logs for kippo go you are all set. Just run the script and it will function as designed.
Description: mwcrawler is a simple python script that parses malicious url lists from well known websites (i.e. MDL, Malc0de) in order to automatically download the malicious code. It can be used to populate malware repositories or zoos.
Friday, March 1, 2013 at 4:01PM 
