TekDefense

Sunday

Nov182012

TekTip ep14 - Pipal Password Analysis of Yahoo password dump

Sunday, November 18, 2012 at 2:25PM

Last week are good friends over at Bruteforce Labs posted a quick tutorial for Pipal. I figured the TekDefense user base may also benefit from this tool.

Pipal @ DigiNinja

Description: A password analysis tool that gives relevant statistics of passwords given a password dump.

Uses: Analyze password trends, create better wordlists, educate users

Installation:

Pipal on Github

*Requires Ruby1.9.x

*BT5 comes with pipal 1.0. Update Pipal if on Backtrack to 2.0

Usage:

1. First you will need a password dump to play with. There are several out in the wild. You can find some here:

http://www.skullsecurity.org/wiki/index.php/Passwords

For my demo I will use the recent (kinda) Yahoo dump

2. Get the file ready for pipal:

You only want the passwords in a file for Pipal, cut out the rest.

cat yahoousersandpass.txt | cut -d: -f 3 > yahoopassesonly.txt

3. Run Pipal:

./pipal.rb ~/leakedpasswords/yahoopassesonly.txt -o yahoodemo

4. Analyze results

We analyzed 442837 passwords in this dump!

Total entries = 442837

Total unique entries = 342509

Here we see some pretty standard bad passwords:

Top 10 passwords

123456 = 1667 (0.38%)

password = 780 (0.18%)

welcome = 437 (0.1%)

ninja = 333 (0.08%)

abc123 = 250 (0.06%)

123456789 = 222 (0.05%)

12345678 = 208 (0.05%)

sunshine = 205 (0.05%)

princess = 202 (0.05%)

qwerty = 172 (0.04%)

Base passwords are password that contain a word but are not only that word:

Top 10 base words

password = 1374 (0.31%)

welcome = 535 (0.12%)

qwerty = 464 (0.1%)

monkey = 430 (0.1%)

jesus = 429 (0.1%)

love = 421 (0.1%)

money = 407 (0.09%)

freedom = 385 (0.09%)

ninja = 380 (0.09%)

sunshine = 367 (0.08%)

As we see in most password dumps, most people go with 8 character passwords. This is a common requirement, and has been drilled into people for a while now, so no surprise there. 116 people had a 1 character password though? I usually don't try passwords less than 4 characters when I password crack, guess I might need to bring them back in.

Password length (length ordered)

1 = 116 (0.03%)

2 = 70 (0.02%)

3 = 302 (0.07%)

4 = 2748 (0.62%)

5 = 5324 (1.2%)

6 = 79629 (17.98%)

7 = 65610 (14.82%)

8 = 119133 (26.9%)

9 = 65964 (14.9%)

10 = 54759 (12.37%)

11 = 21218 (4.79%)

12 = 21729 (4.91%)

13 = 2657 (0.6%)

14 = 1492 (0.34%)

15 = 837 (0.19%)

16 = 568 (0.13%)

17 = 262 (0.06%)

18 = 125 (0.03%)

19 = 88 (0.02%)

20 = 177 (0.04%)

21 = 10 (0.0%)

22 = 7 (0.0%)

23 = 2 (0.0%)

24 = 2 (0.0%)

27 = 1 (0.0%)

28 = 4 (0.0%)

29 = 2 (0.0%)

30 = 1 (0.0%)

Password length (count ordered)

8 = 119133 (26.9%)

6 = 79629 (17.98%)

9 = 65964 (14.9%)

7 = 65610 (14.82%)

10 = 54759 (12.37%)

12 = 21729 (4.91%)

11 = 21218 (4.79%)

5 = 5324 (1.2%)

4 = 2748 (0.62%)

13 = 2657 (0.6%)

14 = 1492 (0.34%)

15 = 837 (0.19%)

16 = 568 (0.13%)

3 = 302 (0.07%)

17 = 262 (0.06%)

20 = 177 (0.04%)

18 = 125 (0.03%)

1 = 116 (0.03%)

19 = 88 (0.02%)

2 = 70 (0.02%)

21 = 10 (0.0%)

22 = 7 (0.0%)

28 = 4 (0.0%)

23 = 2 (0.0%)

24 = 2 (0.0%)

29 = 2 (0.0%)

30 = 1 (0.0%)

27 = 1 (0.0%)

|

|

|

|

|

| |

| |

||||

|||||

|||||

|||||

|||||

|||||

|||||||

|||||||

||||||||||||||||||||||||||||||||

00000000001111111111222222222233

01234567890123456789012345678901

One to six characters = 88189 (19.91%)

One to eight characters = 272932 (61.63%)

More than eight characters = 169905 (38.37%)

66% only used lowercase alpha characters or only used numbers.

Only lowercase alpha = 146516 (33.09%)

Only uppercase alpha = 1778 (0.4%)

Only alpha = 148294 (33.49%)

Only numeric = 26081 (5.89%)

A common trend is for people to capitalize the first character, or add a number or special character to the end of a password.

First capital last symbol = 1259 (0.28%)

First capital last number = 17467 (3.94%)

While months were used in passwords a decent amount in this dump, it doesn't look like days made up many of them.

Months

january = 106 (0.02%)

february = 30 (0.01%)

march = 192 (0.04%)

april = 284 (0.06%)

may = 725 (0.16%)

june = 386 (0.09%)

july = 245 (0.06%)

august = 238 (0.05%)

september = 68 (0.02%)

october = 182 (0.04%)

november = 154 (0.03%)

december = 130 (0.03%)

Days

monday = 48 (0.01%)

tuesday = 15 (0.0%)

wednesday = 9 (0.0%)

thursday = 18 (0.0%)

friday = 47 (0.01%)

saturday = 6 (0.0%)

sunday = 30 (0.01%)

Months (Abreviated)

jan = 1007 (0.23%)

feb = 172 (0.04%)

mar = 4719 (1.07%)

apr = 472 (0.11%)

may = 725 (0.16%)

jun = 798 (0.18%)

jul = 656 (0.15%)

aug = 504 (0.11%)

sept = 184 (0.04%)

oct = 425 (0.1%)

nov = 519 (0.12%)

dec = 404 (0.09%)

Days (Abreviated)

mon = 4431 (1.0%)

tues = 16 (0.0%)

wed = 212 (0.05%)

thurs = 29 (0.01%)

fri = 479 (0.11%)

sat = 365 (0.08%)

sun = 1237 (0.28%)

Another common trend is for users to add the year of their birth, or wedding, or the current year to their password. While it may be surprising that 2010, 2011, and 2012 didn't have many hits if you take the source into account it makes sense. The Yahoo dump comes from an old database that was used as part of a migration for a company that Yahoo bought call Associated Content. This purchase occurred in 2010.

Includes years

1975 = 255 (0.06%)

1976 = 266 (0.06%)

1977 = 278 (0.06%)

1978 = 332 (0.07%)

1979 = 339 (0.08%)

1980 = 353 (0.08%)

1981 = 331 (0.07%)

1982 = 359 (0.08%)

1983 = 338 (0.08%)

1984 = 392 (0.09%)

1985 = 367 (0.08%)

1986 = 361 (0.08%)

1987 = 413 (0.09%)

1988 = 360 (0.08%)

1989 = 401 (0.09%)

1990 = 304 (0.07%)

1991 = 276 (0.06%)

1992 = 251 (0.06%)

1993 = 218 (0.05%)

1994 = 202 (0.05%)

1995 = 147 (0.03%)

1996 = 171 (0.04%)

1997 = 140 (0.03%)

1998 = 155 (0.04%)

1999 = 189 (0.04%)

2000 = 617 (0.14%)

2001 = 404 (0.09%)

2002 = 404 (0.09%)

2003 = 345 (0.08%)

2004 = 424 (0.1%)

2005 = 496 (0.11%)

2006 = 572 (0.13%)

2007 = 765 (0.17%)

2008 = 1145 (0.26%)

2009 = 1052 (0.24%)

2010 = 339 (0.08%)

2011 = 92 (0.02%)

2012 = 130 (0.03%)

2013 = 50 (0.01%)

2014 = 28 (0.01%)

2015 = 24 (0.01%)

2016 = 25 (0.01%)

2017 = 26 (0.01%)

2018 = 33 (0.01%)

2019 = 84 (0.02%)

2020 = 163 (0.04%)

Years (Top 10)

2008 = 1145 (0.26%)

2009 = 1052 (0.24%)

2007 = 765 (0.17%)

2000 = 617 (0.14%)

2006 = 572 (0.13%)

2005 = 496 (0.11%)

2004 = 424 (0.1%)

1987 = 413 (0.09%)

2001 = 404 (0.09%)

2002 = 404 (0.09%)

Red and Blue make up the majority of colors in the passwords.

Colours

black = 706 (0.16%)

blue = 1143 (0.26%)

brown = 221 (0.05%)

gray = 76 (0.02%)

green = 655 (0.15%)

orange = 250 (0.06%)

pink = 357 (0.08%)

purple = 346 (0.08%)

red = 2202 (0.5%)

white = 244 (0.06%)

yellow = 228 (0.05%)

violet = 66 (0.01%)

indigo = 35 (0.01%)

As stated previously, people tend to tack numbers and special characters at the end of passwords. These statistics support that theory.

Single digit on the end = 47391 (10.7%)

Two digits on the end = 73640 (16.63%)

Three digits on the end = 31095 (7.02%)

Last number

0 = 17553 (3.96%)

1 = 46694 (10.54%)

2 = 24623 (5.56%)

3 = 29232 (6.6%)

4 = 17692 (4.0%)

5 = 17405 (3.93%)

6 = 17885 (4.04%)

7 = 20402 (4.61%)

8 = 17847 (4.03%)

9 = 19919 (4.5%)

|

|

|

|

|

| |

| |

|||

|||

||||| ||||

||||||||||

||||||||||

||||||||||

||||||||||

||||||||||

||||||||||

0123456789

Last digit

1 = 46694 (10.54%)

3 = 29232 (6.6%)

2 = 24623 (5.56%)

7 = 20402 (4.61%)

9 = 19919 (4.5%)

6 = 17885 (4.04%)

8 = 17847 (4.03%)

4 = 17692 (4.0%)

0 = 17553 (3.96%)

5 = 17405 (3.93%)

Last 2 digits (Top 10)

23 = 12364 (2.79%)

12 = 6416 (1.45%)

11 = 5476 (1.24%)

01 = 5097 (1.15%)

00 = 4098 (0.93%)

21 = 3669 (0.83%)

08 = 3627 (0.82%)

07 = 3598 (0.81%)

22 = 3587 (0.81%)

13 = 3548 (0.8%)

Last 3 digits (Top 10)

123 = 9446 (2.13%)

456 = 2443 (0.55%)

234 = 2160 (0.49%)

007 = 1477 (0.33%)

000 = 1268 (0.29%)

008 = 1150 (0.26%)

009 = 1086 (0.25%)

111 = 1056 (0.24%)

777 = 980 (0.22%)

101 = 895 (0.2%)

Last 4 digits (Top 10)

3456 = 2151 (0.49%)

1234 = 1968 (0.44%)

2008 = 1033 (0.23%)

2009 = 927 (0.21%)

2345 = 750 (0.17%)

2007 = 674 (0.15%)

2000 = 535 (0.12%)

2006 = 502 (0.11%)

1111 = 436 (0.1%)

2005 = 436 (0.1%)

Last 5 digits (Top 10)

23456 = 2121 (0.48%)

12345 = 724 (0.16%)

56789 = 316 (0.07%)

45678 = 305 (0.07%)

11111 = 269 (0.06%)

34567 = 231 (0.05%)

54321 = 197 (0.04%)

00000 = 162 (0.04%)

99999 = 150 (0.03%)

23123 = 132 (0.03%)

Most popular area codes based ont the 3 character numbers found.

US Area Codes

456 = Inbound International (--)

234 = NE Ohio: Canton, Akron (OH)

Now here is some data that can be directly applied to password cracking.

Character sets

loweralphanum: 224095 (50.6%)

loweralpha: 146516 (33.09%)

numeric: 26081 (5.89%)

mixedalphanum: 23238 (5.25%)

loweralphaspecialnum: 6070 (1.37%)

mixedalpha: 5122 (1.16%)

upperalphanum: 3416 (0.77%)

mixedalphaspecialnum: 3340 (0.75%)

loweralphaspecial: 2079 (0.47%)

upperalpha: 1778 (0.4%)

mixedalphaspecial: 486 (0.11%)

upperalphaspecialnum: 222 (0.05%)

specialnum: 188 (0.04%)

upperalphaspecial: 46 (0.01%)

special: 16 (0.0%)

Character set ordering

stringdigit: 185323 (41.85%)

allstring: 153416 (34.64%)

alldigit: 26081 (5.89%)

othermask: 25117 (5.67%)

digitstring: 24962 (5.64%)

stringdigitstring: 18677 (4.22%)

digitstringdigit: 4648 (1.05%)

stringspecialdigit: 2359 (0.53%)

stringspecial: 1111 (0.25%)

stringspecialstring: 833 (0.19%)

specialstringspecial: 168 (0.04%)

specialstring: 126 (0.03%)

allspecial: 16 (0.0%)

Hashcat masks (Top 10)

?l?l?l?l?l?l: 40693 (9.19%)

?l?l?l?l?l?l?l?l: 32439 (7.33%)

?l?l?l?l?l?l?l: 29129 (6.58%)

?l?l?l?l?l?l?d?d: 20316 (4.59%)

?l?l?l?l?l?l?l?l?l: 16185 (3.65%)

?l?l?l?l?l?l?l?l?d?d: 12632 (2.85%)

?d?d?d?d?d?d: 12583 (2.84%)

?l?l?l?l?l?l?l?d: 10620 (2.4%)

?l?l?l?l?l?l?l?l?l?l: 10310 (2.33%)

?l?l?l?l?l?l?l?d?d: 10281 (2.32%)

1aN0rmus@tekdefense.com

http://www.securitytube.net/user/1aN0rmus

www.youtube.com/user/TekDefense

Admin |

Post a Comment |

tagged

1aN0rmus,

Pipal,

Tekdefese,

Yahoo,

analysis,

crack,

dump,

password,

tektip in

TekTip

Sunday

Nov042012

Automater - IP and URL analysis tool

Sunday, November 4, 2012 at 11:35AM

Update: Automater gets its own project page http://www.tekdefense.com/automater/

One challenge I have faced, as well as seen other analyst face as well is the amount of time it takes to investigate an IP Address or URL. If you are like most analyst you have probably used at least some if not all of the following web tools to investigate IPs or URLs involved in an incident:

As I mentioned previously to lookup an IP against all or even some of these takes more time than it should. Additionally, as they all are formatted differently and have different options, you can not export them in a format that can be attached to a trouble ticket, evidence file, or even an email. Noticing this issue, I attempted to fix it by creating a python program that will scrape the previously mentioned resources to pull out the information relevant to what the analyst needs.

Meet Automater:

The tool currently only queries IPVoid, Robtex, and Fortiguard currently, but I am working on adding modules from all the resources I mentioned earlier in the article. The help option will explain where I am heading with the project, all though I am not quite there yet.

root@bt:~/workspace/Automater# ./Automater.py -h

ONLY -t AND -h WORK CURRENTLY!!

-t: target ip or url. URL must include http://

-s: source engine (robtex, ipvoid, fortiguard)

-a: all engines

-h: help

-f: import a file of IPs and/or URLs

-o: output results to file

-i: Interactive Mode

Examples:

./Automater.py -t 123.123.123.123 -a -o result.txt

./Automater.py -f hosts.txt -s robtex -o results.txt

Automater right now only takes the -t and the -h options and only works for one target at a time. This again will change as I modify it. URL support has not been added yet either.

I am posting this now in its pre-release form because I would like to hear from the community what types of features they would like to see added. I would also like to know of any bugs you can find. Lastly, I am of course interested in anyone who would like to contribute to the project. If all goes as planned I would like to have the tool fully functional within a couple of weeks. Once complete I will attempt to pitch the tool to Doug Burks to add to his Security Onion Distro. I think this could be a really nice tool for analysts.

Contribute to, or download the tool on Github.

Report any Bugs or feature requests to 1aN0rmus@TekDefense.com

DEMO:

Admin |

Post a Comment |

tagged

ipvoid,

python,

robtex,

tekdefense,

tektip in

News,

TekTip

Sunday

Oct212012

TekTip ep12 - Regex Basics

Sunday, October 21, 2012 at 12:00AM

Description: Regular expressions are a way to match specific patterns in strings.

Demo Setup: For the demo, I am using BT5 with eclipse and pydev. I am demonstrating with python, but there are many other methods that could be used.

Uses: Singature creation pcre, scripting, programming

"\w": word character

"\W": Not word character

"\d" : digit

"\D" : Not digit

"\s" : space

"\S" : not space

"\" : escape

"." : Any character except \n (new line)

"|" : or

"{}" : Range

"+" : One or more

Examples:

String: Hi, my name is 1aN0rmus. My phone number is 555-555-5555 my address is 123 Internet Lane.

Regex: '\d\w\w\d\w\w\w\w'

Result: 1aN0rmus

Regex: '\d\w+\d\w+'

Result: 1aN0rmus

Regex: '\d{3}.\d{3}.\d{4}'

Result: 555.555.5555

Regex: '\d{1,5}\s\w+\s\w+'

Result: 123 Internet Lane

String: saDaSDASD/forum/Themes/core/images/sdfs.exe SAdasd

Regex: '\/forum\/Themes\/core\/images\/\w+\.exe'

Result: /forum/Themes/core/images/sdfs.exe

Scripts seen on this episode can be downloaded at https://github.com/1aN0rmus/TekDefense. Feel free to help out with either script. I'm a beginner when it comes to python.

Thank you,

1aN0rmus@tekdefense.com

http://www.securitytube.net/user/1aN0rmus

www.youtube.com/user/TekDefense

Admin |

Post a Comment |

tagged

BT5,

pcre,

pydev,

python,

regex,

tekdefense,

tektip,

ubuntu in

TekTip

Saturday

Oct132012

TekTip ep11 - Kippo SSH Honeypot

Saturday, October 13, 2012 at 11:30PM

http://code.google.com/p/kippo/

Description: Kippo is a medium interaction SSH honeypot designed to log brute force attacks and, most importantly, the entire shell interaction performed by the attacker

Uses: Alert to potiential threats, watch how hackers operate, gather exploits and malware

Installation:

Follow steps @ http://bruteforce.gr/installing-kippo-ssh-honeypot-on-ubuntu.html

http://bruteforce.gr/honeybox Honeybox is a distro that contains numerous honeypot software, all on a single box. Additionally, the distro preconfigures the honeypot to utilize some of the many enhancements Brutforce Labs have created for these honeypots.

*If at home, to make this accessible from the internet you will need to enable port forwarding at your modem, and potientially your Virtual Machine software.

Usage:

kippo/kippo.cfg : Main configuration file

kippo/honeyfs : This is the fake filesystem that wll be presented to the user.

kippo/data/userdb.txt : This file allows us to modify the username and password combinations that will work when attackers attempt to log into the honeypot.

kippo/log/tty/ : In this directory you will find the logs for each session established by attackers.

./start.sh

- will start kippo

/kippo/utils/playlog.py : Replay an attacker session from the kippo/log/tty directory.

Usage: playlog.py [-bfhi] [-m secs] [-w file] <tty-log-file>

-f keep trying to read the log until it's closed

-m <seconds> maximum delay in seconds, to avoid boredom or fast-forward to the end. (default is 3.0)

-i show the input stream instead of output

-b show both input and output streams

-c colorify the output stream based on what streams are being received

-h display this help

i.e.

~/kippo/utils/playlog.py 20121012-115031-8544.log

1aN0rmus@tekdefense.com

http://www.securitytube.net/user/1aN0rmus

www.youtube.com/user/TekDefense

Admin |

1 Comment |

tagged

Cyber,

Kippo,

shell,

ssh,

tekdefense in

TekTip

Monday

Oct082012

TekTip ep10 - Proxychains!

Monday, October 8, 2012 at 1:59PM

ProxyChains-3.1 (http://proxychains.sf.net)

Proxychains is a tool to force connections through multiple proxies. What makes this tool special is putting applications without native proxy capabilities through a proxy.