Finally the new Automater release is out!

With the exception of my review of the Volatility Malware and Memory Forensics class yesterday, it has been a while since I have posted here. Time for me to get back into the swing of things. The best way to do so is with a new release to the tool that really launched code development projects on TekDefense.

Automater is a tool that I orginially created to automate the OSINT analysis of IP addresses. It quickly grew and became a tool to do analysis of IP Addresses, URLs, and Hashes. Unfortunately though, this was my first python project and I made a lot of mistakes, and as the project grew it bacame VERY hard for me to maintain. 

Luckily, a mentor and friend of mine (@jameshub3r) offered his time and expertise to do an enitre re-write of the code that would focus on a modular extensible framework. The new code hits the mark as far as that is concerned. The real power of Automater is how easy it is to modify what sources are checked and what data is taken from them without having to modify the python code. To modify sources simply open up the sites.xml file and modify away. I'll do another post later that goes into more detail there.

To view a bit more about installation and usage head over to the new Automater page.

You can download the code directly on Github. Remeber Automater is not a single file anymore, you need to download all of the files in the Automater repo to the same directory. To the first person that reports a valid bug to me, I'll send you a random game on Steam.

Here are a few screenshots to hold you over until you get it running.

Automater updates

So as many of you have may have noticed, I have updated Automater a few times over the last couple of months to address some specific issues and add some functionality. The changelog is as follows:

Changelog:

1.2.4

[+] Modifed Robtex data pull to match sites new formatting

[+] Added Virustotal search for the hash function

1.2.3

[+] Added HTTP Proxy support. Will pull OS default proxy settings.

[+] Modified some variables for consistency 

[+] Added comments

[-] Removed JoeBox from hash search

1.2.2

[+] Fixed FortiGuard rating https://github.com/1aN0rmus/TekDefense/issues/10

[+] Display help when no arguments are given https://github.com/1aN0rmus/TekDefense/issues/8

[+] Added Hash Search functionality https://github.com/1aN0rmus/TekDefense/issues/7

[+] Sources for Hash search are VxVault, ThreatExpert, JoeSandBox, and Minotaur

1.2.1

[+] Modified regex in Robtex function to pick up "A" records that were being missed.

[+] Alienvault reputation data added by guillermogrande.  Thank you!

1.2

[+] Changed output style to @ViolentPython style

[+] Fixed IPVoid and URLVoid result for new regexes

[+] Fixed form submit for IP's and URLs that were not previously scanned

So in short, it now has proxy support, pulls data from a few new places and will now take hashes as well. Don't worry we are not done with Automater though, I have a lot more planned.

Automater was the tool I wrote to learn basic python. As this was my first python project I made a lot of rookie mistakes. The code works and does what it is supposed to do, but it is sloppy and not optimized in the least. With that in mind, I plan to work on the next mjor release which will be a complete re-write of Automater from the ground up. Doing this should hopefully give us a more stable and extensible product.

See usage, installation, and download instructions at http://www.tekdefense.com/automater/

TekTip ep29 - Collect and track hashes with hashMonitor

In this episode of TekTip we take a look at a new tool I created called hashMonitor. hashMonitor will monitor specific twitter and web resources for database dumps that include MD5, SHA1, or SHA256 hashes. Once found, hashMonitor will store the hashes in a local database which can then be used for cracking purposes.

To learn more about the tool usage and installation go to http://www.tekdefense.com/hashmonitor/

ProTip: hashMonitor + cronjob = Profit!  *Set to run every 30 minutes or so*

Tektip ep27 - hashCollect.py

With the #OpIsreal stuff going on right now there has been many more password dumps put out than usual. For instance using Andrew Mohawks PasteLert web app I get alerted anytime there is a pastebin post that includes the hash e10adc3949ba59abbe56e057f20f883e. e10adc3949ba59abbe56e057f20f883e is the hash of the most common password: 123456. I set up the alert for this hash because it will catch password dumps regardless of the language. I admit there are some faults though, particularly if the site that the passwords are dumped from have password requirements that would not allow a password of 123456. The following is a graph that shows the typical number of dumps I see with these parameters:

As you can see, #OpIsreal has caused a significant uptick in the number of password dumps that include the hash for 123456.

My typical process once I get a hold of the dumps from these is I download the file, manually pull out the typical header data like the name of the operation and all the propaganda, then I use the cut command to pull out just the hashes. While this isn't too lengthy of a process, I am a lazy man. From this laziness, comes hashCollect.py.

HashCollect.py is a python tool I wrote that will scrape md5 hashes out of a specific file or url. While this script is pretty bare right now it gets the job done. I have many plans for it, that you will hopefully see soon.

You can dowload hashCollect along with my other scripts at GitHub.

The help command will show the options:

root@bt:~/workspace/Automater# ./hashCollect.py -h

usage: hashCollect.py [-h] [-u URL] [-f FILE] [-o OUTPUT]

Hash Collector

optional arguments:

  -h, --help            show this help message and exit

  -u URL, --url URL     This option is used to search for hashes on a website

  -f FILE, --file FILE  This option is used to import a file that contains

                        hashes

  -o OUTPUT, --output OUTPUT

                        This option will output the results to a file.

Run hashCollect against a file:

root@bt:~/workspace/Automater# ./hashCollect.py -f hashes 

09a85c0ef4169a24210d741838e8c8d9

43a9b1c680ea8f8df293e58b9ce77b9d

9f22025be8346e4d4d7db80ed890b511

f1887d3f9e6ee7a32fe5e76f4ab80d63

704992a0216ae39f1ebf3771fd5cd23c

a5915fe9b6ed8d251fd342b74106e34b

16ea0d4fadc502c247209194645e4f4a

9719536c0f2d1a578b323853998e03ba

93279e3308bdbbeed946fc965017f67a

de46896de0010ae616f9c6cb3f7e4cfc

bd9b23306ab802765a63870b29d1239b

b3d6288bfd707aee52db620839f3a381

25d55ad283aa400af464c76d713c07ad

17e6ec4b774be1bfbd12e26a68f9d9bf

7d8c3a265ad7aa2f4e20b1a93fde3c54

1c2f7107394f0d29999a1c23e1deaf44

53c86172178bc31dacba8b501f34b976

aa2b0de3de9b517b592059ca5d6cfa4c

caac935aae3e50060442ee55bc9e1a3f

9824bbc389f1c39f2b2cdfa839938d05

a474b36564cc2730d27f716f3c7c2fe1

236558a7ec33e3223db4471024833013

f229ea34d627074a1fd0a474f4a51c7b

e10adc3949ba59abbe56e057f20f883e

a474b36564cc2730d27f716f3c7c2fe1

fd5972161600fb43f057efd443d77589

aa881d0c78e0e60642e006ca88c9495f

e10adc3949ba59abbe56e057f20f883e

1a92fc27d687aefa619c24851cbb1213

49518adbec43b4264c0ea840c2e233d5

1f247e3f69c363f18dec2e343008d142

Run hashCollect against a URL:

root@bt:~/workspace/Automater# ./hashCollect.py -u http://pastebin.com/2ysAGFJY

6b586e2d4ca26d9438589a85585ca0b2

db17a0bf8505f7620291a8efc466ce86

5a815fd388b6027b949d58977277a006

dbab8786ed0eca3dbb82e401ce976d7e

d772ffda95ce3417456c80a8f85606d2

a9e9f1c9d9296f0c38467efc7dca1a24

96f003089b0ad3a71261ed5a1533c794

39dcaf7a053dc372fbc391d4e6b5d693

5079a6b1fb2015dbdb0c4b205f917307

1349437e3137826639b4f5165bc7e02b

3f94e8774be14358a45e2dda6a60216a

ce5225d01c39d2567bc229501d9e610d

6403675579f6114559c90de0014cd3d6

81dc9bdb52d04dc20036dbd8313ed055

81dc9bdb52d04dc20036dbd8313ed055

c67fc3a08cc21eaecb0fadf68129c314

4e270f490ab6943cbfbe95c9b936d7bb

81dc9bdb52d04dc20036dbd8313ed055

348a6a2356c3aebe392aaad3f646c30c

0e69229f5978ebc338f2cfb8cc8caad1

dc1caba8d678508cc3f6985ae35d7c9f

4f4bdca0d270dbbff6647a356fe2ba3f

e41a2cb50c3362f0015404effec8761e

1e4a9c23007eb10f758cbf3362c8ae41

b59c67bf196a4758191e42f76670ceba

15de21c670ae7c3f6f3f1f37029303c9

44d61d552280cf5e9c55dc11ff18cbf2

1794e1d48bbf9f73d53dec1951f053d2

bb2782795456847fb533d51eefd9d360

fe43196710222556c8bcc1c23c022a74

81dc9bdb52d04dc20036dbd8313ed055

0a03d5e4473c0629cfb20c5c31543b06

fe43196710222556c8bcc1c23c022a74

c944634550c698febdd9c868db908d9d

c944634550c698febdd9c868db908d9d

c944634550c698febdd9c868db908d9d

9e94b15ed312fa42232fd87a55db0d39

a01610228fe998f515a72dd730294d87

c0079e0d4e801bc94bce6fa2ed9e008c

ece5cdf7b946687f0077abb714054a65

a31e7b7f49ae4bb4dcc460d51b0bada0

e13b330c974ba77aef5bd4504eebe5e9

cd474f6341aeffd65f93084d0dae3453

4eceed354639b6ed2b236be2eb5a065e

7e7e69ea3384874304911625ac34321c

c889144fa7ab1a735872c290d8781899

2fbec6c8aa22b895bc40efa6e89b4bac

8621c6b58f7dde244ad2261610383fae

05a70454516ecd9194c293b0e415777f

192cc6356b292ce2a105c222ab6042ae

784ec60b05fc2eaa5c74e4775220fbb1

fdcfc3f14ebf698bbd76b1157ef709fa

e10adc3949ba59abbe56e057f20f883e

e0c64dbc8cb6abc98a0c696d168ebdb9

82ce647f22861f30627ee0ae50ce0adb

02cc9f0bf98299b63cdfd77fbfec7172

44dc880ec5f9237fa80be3177161fc6f

fdcfc3f14ebf698bbd76b1157ef709fa

fe01ce2a7fbac8fafaed7c982a04e229

29988429c481f219b8c5ba8c071440e1

42bb5b74add1fe6bb353cf5e14562fb6

41ad29597e715721522a30733b96a6f3

2a85ca2b0c07c342facc02ac61d57171

4ffc938c6c948859ce9c4ec827e1e40a

6967cabefd763ac1a1a88e11159957db

33026ce64a49d23be2d07d04b6ef4384

f9a13a115a69b22323e7ef9ef9fedcb6

f92e053a1fd2c673cb899db192ad0f2e

8cbfe3eb54787170a9ad6af435964828

d860b866e9023673fd802d97b97fc357

e391997bd526a092ac4d7f9b50da8904

ae0e4bdad7b5f67141743366026d2ea5

e10adc3949ba59abbe56e057f20f883e

c4de8ced6214345614d33fb0b16a8acd

4ddc0354b46b390a933bb6d2353fff26

fa1ee3a6f55b9b5cacc571a76c3842a1

de38aee67bacae29c2e8d868418293bd

a381c2c35c9157f6b67fd07d5a200ae1

5bc06f5800d415cc95e1349edbaca425

902fbdd2b1df0c4f70b4a5d23525e932

aa15b9243a9f99d122d5803606e3c4df

e10adc3949ba59abbe56e057f20f883e

3514603c3f975120a33354aeed9039a3

da897ba0fe30eff270424ac0e768840f

2be5b6590b60d5f4eca7e13c4083af7e

f502e183d729ab3ad224f1dfcc0708e2

c5fe25896e49ddfe996db7508cf00534

c9dab21c609875c00eaa19f04d19e2d0

b0253ef863f3a4a2e746c793fa71ae7d

46f0cac183682913b2d9e685cd7da3a7

4ef02ee44e55ca014df93b75eb956103

1618a9fe1c58f2bedd2fdccefaa6da21

1618a9fe1c58f2bedd2fdccefaa6da21

abc2e2f32e486fc2e1072003cc88149a

b269e1a566f861efa042e7ea7a08b062

8562ae5e286544710b2e7ebe9858833b

4251dd1cece37b7ee6ba2c2e40039bdd

204f8213a4cc1aaffa1fb123406d1ae9

8bb75b3015682d910daf88b6d728be2c

ff2cd3d917770fdcbbd541faf5423413

c91793b6ef51da231364176994d678a9

b76c3936d26110aad104844a0496e614

827ccb0eea8a706c4c34a16891f84e7b

4297f44b13955235245b2497399d7a93

775df0ec6881d9fcb545b5cd5a409873

ae950f6eecfe4d911b6b959ec3965231

d726335216d643e3c467eb0cdfc3d4e7

1dfa9fe971cd0711ce70e794063bea22

1311c5a589710f5030ae0fa36a20774c

e85984bd537ecc6d027b43bef22e4f12

dab456a52cb642e187cd307a5cfbef79

81dc9bdb52d04dc20036dbd8313ed055

Output the results to a file:

root@bt:~/workspace/Automater# ./hashCollect.py -u http://pastebin.com/2ysAGFJY -o /tmp/outputfileforhashes.txt

[+] Printing results to file: /tmp/outputfileforhashes.txt 

That's it for now, but I will grow this out soon. Some of the features I am thinking about adding are:

• Allow custom regex
• Allow for pulling other hashes like SHA256
• Check hashes against online hash crackers
• Output to a database
• Create a frontend
• What would you like to see?

Have any suggestions. Let me know 1aN0rmus@TekDefense.com.