John the Ripper

The program john (or ‘John the Ripper’, abbreviated JtR) is a program by Solar Designer (Alexander Peslyak) that attempts to retrieve cleartext passwords, given hashes.

Documentation

Docs can be found in many places (including this page). On the home site there are pages entitled INSTALL OPTIONS MODES CONFIG RULES EXTERNAL EXAMPLES FAQ.

Getting it

The web site is www.openwall.com/john. Download some version. The ‘official’ versions are by Solar Designer, the jumbo-patches were contributed by lots of people.

Invocation

Basic invocation:
% john passwdfile
Here passwdfile is a file with one line per hashed password. The format is discussed below. It is possible to give several files (that use the same hash type):
% john pwfile1 pwfile2 ...

The invocation without arguments produces a help message.

% john
John the Ripper password cracker, ver: 1.7.9-jumbo-5 [linux-x86-64]
Copyright (c) 1996-2011 by Solar Designer and others
Homepage: http://www.openwall.com/john/

Usage: john [OPTIONS] [PASSWORD-FILES]
--config=FILE             use FILE instead of john.conf or john.ini
--single[=SECTION]        "single crack" mode
--wordlist=FILE --stdin   wordlist mode, read words from FILE or stdin
                --pipe    like --stdin, but bulk reads, and allows rules
...
There are many options. It does not matter whether one gives them as -option or as --option. Option values (if any) are preceded by : or =. Option names may be abbreviated. E.g. -w:FILE and --wordlist=FILE mean the same thing.

John's home

If you get the message
fopen: $JOHN/john.ini: No such file or directory
then either (e.g., if you plan a dictionary attack)
% touch john.ini
or change directory to the run directory that has the john executable and various other useful files.

(The program john wants to read john.conf, and when that is not found it tries the alternative name john.ini, and when that is not found it complains and exits. Each invocation of john has a ‘home’, usually the directory where its executable was found. It will write its output files john.log, john.rec and john.pot there. One might think that $JOHN refers to an environment variable one can set to tell john where its home is, but it is only an internal name; when run as somepath/john the program john first tries open("somepath/john.conf", O_RDONLY) and, if that fails, open("$JOHN/john.conf", O_RDONLY), where $JOHN/ is not expanded but is just this 6-byte string. When run as john, the current directory will be john's home. Compile with -DJOHN_SYSTEMWIDE=1 to make it use ~/.john. A useful side effect of systemwide installation is that tilde expansion works in options like -w:~/dir/words.)

One can specify alternative locations for the configuration file john.conf and the result file john.pot with the --config= and --pot= options.

Password file format

The hashed password file is a file with one line per hashed password, with format
root:7z2J3pT/HjmkM
user:MNPXFCWHbDSNM
or perhaps something like
root:vdNPtcyOZKFAU:0:1::/:/bin/sh
guest:QWOovvLrOrb.E:100:10::/usr/guest:/bin/rsh
with further junk following the hashed password. These 13-symbol hashes are from Unix crypt(), and use DES. One also has 32-hexdigit hashes from MD5, e.g.,
admin:51a42fa118e77f95f70d4efff4395f8d
and a host of further hash types. Often, john recognizes the type itself, but when it doesn't, as in this latter case, the type of hash must be specified:
% john --format=raw-md5 passwdfile
Now john will churn away, trying to brute force all hashes. Find the results in john.pot.

Earlier versions of john required at least two fields username:pwhash on each input line (and one had to insert a dummy name if only the password hash was available). Later versions also handle bare lists of hashes, without preceding username:

5d41402abc4b2a76b9719d911017c592
7d793037a0760186574b0282f2f435e7

Dictionary attack

The above invocation will find all passwords eventually, but some passwords will be found much more quickly if one tries a word list first.
% john -w:wordlist.txt passwdfile

Word mangling

Much better than a straight dictionary attack is one where word mangling is enabled. This will modify dictionary words in many ways, so that you also gives You, you1, 2you, 4you, youyou, you?, You!, YOU, uoY, etc. The attack now becomes
% john -w:wordlist.txt -rules passwdfile
One may view the mangling in action via
% john -w:wordlist.txt -rules -stdout | less

You can also define your own word-mangling rules. For example, edit john.conf and add a section

[List.Rules:LinkedIn]
-: ^[nN]^[iI1!]^[dD]^[eE3]^[kK]^[nN]^[iI1!]^[|lL]
-: $[|lL]$[iI1!]$[nN]$[kK]$[eE3]$[dD]$[iI1!]$[nN]
Now
% john -w:wordlist.txt -rules=LinkedIn passwdfile
will try passwords starting or ending with some version of LinkedIn. (Note that both lines indicate 2*4*2*3*2*2*4*3 = 2304 modifications for each word tried. This is slow but finds eds|1nk3d1n. One third of the time is saved by omitting | here.)

Incremental mode

In incremental mode john does not use a word list, but just tries all possible passwords. (To be precise: all of length at most 8 over the 95-char printable ASCII alphabet.) Examining all possibilities may take millions of years, so john picks an ordering of the passwords tried that attempts to maximize the number of passwords found early. Thus, john will try likely 8-char passwords before unlikely 6-char ones. A disadvantage is that it may be difficult to state which strings have been tried already.

One can restrict the passwords tried to a given character set (with given frequencies):

% john -i:digits passwdfile
will try all-digit passwords of length at most 8. Longer numbers do not work.

(If you want to try longer numbers, it does not suffice to edit john.conf and change MaxLen = 8 in the [Incremental:Digits] section:

% john -i:digits --format=raw-md5 todo
MaxLen = 9 exceeds the compile-time limit of 8
so a recompilation is required. But feeding john with these numbers is easier:
% seq 999999999 | john -stdin --format=raw-md5 todo
Since we are going to try them all anyway, no statistics is needed.)

More generally, the -i:charset option refers to the charset.chr file with symbol frequencies. One can generate a charset file, maybe from a file of already cracked passwords, using the --make-charset= option.

Simultaneous invocations

John can handle several simultaneous jobs, each writing to the same john.pot file. Use a --session parameter.
% john --session=s2 passwdfile
An interrupted nameless session can be resumed by
% john --restore
If the session was named, then
% john --restore=s2
The default recovery and log files are john.rec and john.log. For this named session s2.rec and s2.log.

Reading from stdin

Given a script that generates passwords to try, too many to store in a file, read them from stdin:
% generate | john -stdin passwdfile
This is much faster - disk I/O is slow.

Use -pipe instead of -stdin when also the -rules option is given.

External mode

One can generate or filter potential passwords using code (in john.conf) written in a tiny subset of C. For an example, see here. Usually an external utility is more convenient.

Speed

One is mostly interested in the number of passwords tried per second. Instead, john reports under c/s the number of comparisons per second, that is roughly the number of passwords tried multiplied by the number of hashes (in other words, roughly the number of encryptions multiplied by the number of hashes, divided by the number of salts). Ach. However, with the -test option c/s stands for encryptions per second.

Precise benchmark results depend strongly on the patch version of john and on the chosen compile target. I find that the make target linux-x86-64i produces a faster john than linux-x86-64 (for MD5 and SHA1) in all cases I tried.

On some machine (Intel Core 2 Q9550 @ 2.83GHz):

% john -test
Benchmarking: Traditional DES [128/128 BS SSE2-16]... DONE
Many salts:     3137K c/s real, 3137K c/s virtual
Only one salt:  3020K c/s real, 3020K c/s virtual
...
Benchmarking: FreeBSD MD5 [SSE2i 12x]... DONE
Raw:    32052 c/s real, 32052 c/s virtual
...
Benchmarking: dynamic_0: md5($p)  (raw-md5)  [SSE2i 10x4x3]... DONE
Raw:    21752K c/s real, 21752K c/s virtual
...
On this machine john will check 3M DES passwords each second, but only 30K FreeBSD passwords. A different machine (Intel i7 950 @ 3.07GHz) shows similar results. There is a list of benchmarks.

Using more cores

By default john uses only a single core on these quadcore machines. Edit Makefile and uncomment the line
OMPFLAGS = -fopenmp -msse2
Recompile:
% make clean; make linux-x86-64i
% cd ../run
% OMP_NUM_THREADS=4 ./john --format=des -test
Benchmarking: Traditional DES [128/128 BS SSE2-16]... (4xOMP) DONE
Many salts:     11403K c/s real, 2850K c/s virtual
Only one salt:  10027K c/s real, 2506K c/s virtual
% OMP_NUM_THREADS=4 ./john --format=raw-md5 -test
Benchmarking: Raw MD5 [SSE2i 10x4x3]... DONE
Raw:    21839K c/s real, 21839K c/s virtual
Now DES is a factor 3.5 faster, as expected, but nothing changes for raw-md5. Probably it is better to run several independent instances of john simultaneously.

LinkedIn hashes

After the release of 6.5 million LinkedIn hashes, a patch JtR-Jumbo-5-LinkedIn-SHA1.diff for john was published that handles the raw-sha1_li format of SHA1 with the first twenty bits zeroed. Be careful:
% echo -n sunshine09 | shasum
3b1787e7bd710592ee36264a72d6aa35c2d165f9  -
% grep `echo -n sunshine09 | shasum | cut -c6-40` combo_not.txt
a96807e7bd710592ee36264a72d6aa35c2d165f9
000007e7bd710592ee36264a72d6aa35c2d165f9
Here ‘sunshine09’ is a password that fits the second, but not the first hash. (If the first hash is an actual password hash, this is a very remarkable coincidence: a 140-bit SHA1 near-collision.) By mistake the formats raw-sha1 and raw-sha1_li are both labeled dynamic_26.

Feedback

Send comments, additions and corrections to aeb@cwi.nl .