Sunday
Nov182012
TekTip ep14 - Pipal Password Analysis of Yahoo password dump
Sunday, November 18, 2012 at 2:25PM Last week are good friends over at Bruteforce Labs posted a quick tutorial for Pipal. I figured the TekDefense user base may also benefit from this tool.
Description: A password analysis tool that gives relevant statistics of passwords given a password dump.
Uses: Analyze password trends, create better wordlists, educate users
Installation:
*Requires Ruby1.9.x
*BT5 comes with pipal 1.0. Update Pipal if on Backtrack to 2.0
Usage:
1. First you will need a password dump to play with. There are several out in the wild. You can find some here:
http://www.skullsecurity.org/wiki/index.php/Passwords
For my demo I will use the recent (kinda) Yahoo dump
2. Get the file ready for pipal:
You only want the passwords in a file for Pipal, cut out the rest.
cat yahoousersandpass.txt | cut -d: -f 3 > yahoopassesonly.txt
3. Run Pipal:
./pipal.rb ~/leakedpasswords/yahoopassesonly.txt -o yahoodemo
4. Analyze results
We analyzed 442837 passwords in this dump!
Total entries = 442837Total unique entries = 342509
Here we see some pretty standard bad passwords:
Top 10 passwords123456 = 1667 (0.38%)password = 780 (0.18%)welcome = 437 (0.1%)ninja = 333 (0.08%)abc123 = 250 (0.06%)123456789 = 222 (0.05%)12345678 = 208 (0.05%)sunshine = 205 (0.05%)princess = 202 (0.05%)qwerty = 172 (0.04%)
Base passwords are password that contain a word but are not only that word:
Top 10 base wordspassword = 1374 (0.31%)welcome = 535 (0.12%)qwerty = 464 (0.1%)monkey = 430 (0.1%)jesus = 429 (0.1%)love = 421 (0.1%)money = 407 (0.09%)freedom = 385 (0.09%)ninja = 380 (0.09%)sunshine = 367 (0.08%)
As we see in most password dumps, most people go with 8 character passwords. This is a common requirement, and has been drilled into people for a while now, so no surprise there. 116 people had a 1 character password though? I usually don't try passwords less than 4 characters when I password crack, guess I might need to bring them back in.
Password length (length ordered)1 = 116 (0.03%)2 = 70 (0.02%)3 = 302 (0.07%)4 = 2748 (0.62%)5 = 5324 (1.2%)6 = 79629 (17.98%)7 = 65610 (14.82%)8 = 119133 (26.9%)9 = 65964 (14.9%)10 = 54759 (12.37%)11 = 21218 (4.79%)12 = 21729 (4.91%)13 = 2657 (0.6%)14 = 1492 (0.34%)15 = 837 (0.19%)16 = 568 (0.13%)17 = 262 (0.06%)18 = 125 (0.03%)19 = 88 (0.02%)20 = 177 (0.04%)21 = 10 (0.0%)22 = 7 (0.0%)23 = 2 (0.0%)24 = 2 (0.0%)27 = 1 (0.0%)28 = 4 (0.0%)29 = 2 (0.0%)30 = 1 (0.0%)Password length (count ordered)8 = 119133 (26.9%)6 = 79629 (17.98%)9 = 65964 (14.9%)7 = 65610 (14.82%)10 = 54759 (12.37%)12 = 21729 (4.91%)11 = 21218 (4.79%)5 = 5324 (1.2%)4 = 2748 (0.62%)13 = 2657 (0.6%)14 = 1492 (0.34%)15 = 837 (0.19%)16 = 568 (0.13%)3 = 302 (0.07%)17 = 262 (0.06%)20 = 177 (0.04%)18 = 125 (0.03%)1 = 116 (0.03%)19 = 88 (0.02%)2 = 70 (0.02%)21 = 10 (0.0%)22 = 7 (0.0%)28 = 4 (0.0%)23 = 2 (0.0%)24 = 2 (0.0%)29 = 2 (0.0%)30 = 1 (0.0%)27 = 1 (0.0%)|||||| || ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||0000000000111111111122222222223301234567890123456789012345678901One to six characters = 88189 (19.91%)One to eight characters = 272932 (61.63%)More than eight characters = 169905 (38.37%)
66% only used lowercase alpha characters or only used numbers.
Only lowercase alpha = 146516 (33.09%)Only uppercase alpha = 1778 (0.4%)Only alpha = 148294 (33.49%)Only numeric = 26081 (5.89%)
A common trend is for people to capitalize the first character, or add a number or special character to the end of a password.
First capital last symbol = 1259 (0.28%)First capital last number = 17467 (3.94%)
While months were used in passwords a decent amount in this dump, it doesn't look like days made up many of them.
Monthsjanuary = 106 (0.02%)february = 30 (0.01%)march = 192 (0.04%)april = 284 (0.06%)may = 725 (0.16%)june = 386 (0.09%)july = 245 (0.06%)august = 238 (0.05%)september = 68 (0.02%)october = 182 (0.04%)november = 154 (0.03%)december = 130 (0.03%)Daysmonday = 48 (0.01%)tuesday = 15 (0.0%)wednesday = 9 (0.0%)thursday = 18 (0.0%)friday = 47 (0.01%)saturday = 6 (0.0%)sunday = 30 (0.01%)Months (Abreviated)jan = 1007 (0.23%)feb = 172 (0.04%)mar = 4719 (1.07%)apr = 472 (0.11%)may = 725 (0.16%)jun = 798 (0.18%)jul = 656 (0.15%)aug = 504 (0.11%)sept = 184 (0.04%)oct = 425 (0.1%)nov = 519 (0.12%)dec = 404 (0.09%)Days (Abreviated)mon = 4431 (1.0%)tues = 16 (0.0%)wed = 212 (0.05%)thurs = 29 (0.01%)fri = 479 (0.11%)sat = 365 (0.08%)sun = 1237 (0.28%)
Another common trend is for users to add the year of their birth, or wedding, or the current year to their password. While it may be surprising that 2010, 2011, and 2012 didn't have many hits if you take the source into account it makes sense. The Yahoo dump comes from an old database that was used as part of a migration for a company that Yahoo bought call Associated Content. This purchase occurred in 2010.
Includes years1975 = 255 (0.06%)1976 = 266 (0.06%)1977 = 278 (0.06%)1978 = 332 (0.07%)1979 = 339 (0.08%)1980 = 353 (0.08%)1981 = 331 (0.07%)1982 = 359 (0.08%)1983 = 338 (0.08%)1984 = 392 (0.09%)1985 = 367 (0.08%)1986 = 361 (0.08%)1987 = 413 (0.09%)1988 = 360 (0.08%)1989 = 401 (0.09%)1990 = 304 (0.07%)1991 = 276 (0.06%)1992 = 251 (0.06%)1993 = 218 (0.05%)1994 = 202 (0.05%)1995 = 147 (0.03%)1996 = 171 (0.04%)1997 = 140 (0.03%)1998 = 155 (0.04%)1999 = 189 (0.04%)2000 = 617 (0.14%)2001 = 404 (0.09%)2002 = 404 (0.09%)2003 = 345 (0.08%)2004 = 424 (0.1%)2005 = 496 (0.11%)2006 = 572 (0.13%)2007 = 765 (0.17%)2008 = 1145 (0.26%)2009 = 1052 (0.24%)2010 = 339 (0.08%)2011 = 92 (0.02%)2012 = 130 (0.03%)2013 = 50 (0.01%)2014 = 28 (0.01%)2015 = 24 (0.01%)2016 = 25 (0.01%)2017 = 26 (0.01%)2018 = 33 (0.01%)2019 = 84 (0.02%)2020 = 163 (0.04%)Years (Top 10)2008 = 1145 (0.26%)2009 = 1052 (0.24%)2007 = 765 (0.17%)2000 = 617 (0.14%)2006 = 572 (0.13%)2005 = 496 (0.11%)2004 = 424 (0.1%)1987 = 413 (0.09%)2001 = 404 (0.09%)2002 = 404 (0.09%)
Red and Blue make up the majority of colors in the passwords.
Colours
black = 706 (0.16%)blue = 1143 (0.26%)brown = 221 (0.05%)gray = 76 (0.02%)green = 655 (0.15%)orange = 250 (0.06%)pink = 357 (0.08%)purple = 346 (0.08%)red = 2202 (0.5%)white = 244 (0.06%)yellow = 228 (0.05%)violet = 66 (0.01%)indigo = 35 (0.01%)
As stated previously, people tend to tack numbers and special characters at the end of passwords. These statistics support that theory.
Single digit on the end = 47391 (10.7%)Two digits on the end = 73640 (16.63%)Three digits on the end = 31095 (7.02%)Last number0 = 17553 (3.96%)1 = 46694 (10.54%)2 = 24623 (5.56%)3 = 29232 (6.6%)4 = 17692 (4.0%)5 = 17405 (3.93%)6 = 17885 (4.04%)7 = 20402 (4.61%)8 = 17847 (4.03%)9 = 19919 (4.5%)|||||| || |||||||||||| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||0123456789Last digit1 = 46694 (10.54%)3 = 29232 (6.6%)2 = 24623 (5.56%)7 = 20402 (4.61%)9 = 19919 (4.5%)6 = 17885 (4.04%)8 = 17847 (4.03%)4 = 17692 (4.0%)0 = 17553 (3.96%)5 = 17405 (3.93%)Last 2 digits (Top 10)23 = 12364 (2.79%)12 = 6416 (1.45%)11 = 5476 (1.24%)01 = 5097 (1.15%)00 = 4098 (0.93%)21 = 3669 (0.83%)08 = 3627 (0.82%)07 = 3598 (0.81%)22 = 3587 (0.81%)13 = 3548 (0.8%)Last 3 digits (Top 10)123 = 9446 (2.13%)456 = 2443 (0.55%)234 = 2160 (0.49%)007 = 1477 (0.33%)000 = 1268 (0.29%)008 = 1150 (0.26%)009 = 1086 (0.25%)111 = 1056 (0.24%)777 = 980 (0.22%)101 = 895 (0.2%)Last 4 digits (Top 10)3456 = 2151 (0.49%)1234 = 1968 (0.44%)2008 = 1033 (0.23%)2009 = 927 (0.21%)2345 = 750 (0.17%)2007 = 674 (0.15%)2000 = 535 (0.12%)2006 = 502 (0.11%)1111 = 436 (0.1%)2005 = 436 (0.1%)Last 5 digits (Top 10)23456 = 2121 (0.48%)12345 = 724 (0.16%)56789 = 316 (0.07%)45678 = 305 (0.07%)11111 = 269 (0.06%)34567 = 231 (0.05%)54321 = 197 (0.04%)00000 = 162 (0.04%)99999 = 150 (0.03%)23123 = 132 (0.03%)
Most popular area codes based ont the 3 character numbers found.
US Area Codes456 = Inbound International (--)234 = NE Ohio: Canton, Akron (OH)
Now here is some data that can be directly applied to password cracking.
Character setsloweralphanum: 224095 (50.6%)loweralpha: 146516 (33.09%)numeric: 26081 (5.89%)mixedalphanum: 23238 (5.25%)loweralphaspecialnum: 6070 (1.37%)mixedalpha: 5122 (1.16%)upperalphanum: 3416 (0.77%)mixedalphaspecialnum: 3340 (0.75%)loweralphaspecial: 2079 (0.47%)upperalpha: 1778 (0.4%)mixedalphaspecial: 486 (0.11%)upperalphaspecialnum: 222 (0.05%)specialnum: 188 (0.04%)upperalphaspecial: 46 (0.01%)special: 16 (0.0%)Character set orderingstringdigit: 185323 (41.85%)allstring: 153416 (34.64%)alldigit: 26081 (5.89%)othermask: 25117 (5.67%)digitstring: 24962 (5.64%)stringdigitstring: 18677 (4.22%)digitstringdigit: 4648 (1.05%)stringspecialdigit: 2359 (0.53%)stringspecial: 1111 (0.25%)stringspecialstring: 833 (0.19%)specialstringspecial: 168 (0.04%)specialstring: 126 (0.03%)allspecial: 16 (0.0%)Hashcat masks (Top 10)?l?l?l?l?l?l: 40693 (9.19%)?l?l?l?l?l?l?l?l: 32439 (7.33%)?l?l?l?l?l?l?l: 29129 (6.58%)?l?l?l?l?l?l?d?d: 20316 (4.59%)?l?l?l?l?l?l?l?l?l: 16185 (3.65%)?l?l?l?l?l?l?l?l?d?d: 12632 (2.85%)?d?d?d?d?d?d: 12583 (2.84%)?l?l?l?l?l?l?l?d: 10620 (2.4%)?l?l?l?l?l?l?l?l?l?l: 10310 (2.33%)?l?l?l?l?l?l?l?d?d: 10281 (2.32%)
1aN0rmus@tekdefense.com
http://www.securitytube.net/user/1aN0rmus
www.youtube.com/user/TekDefense