You can trash password cracker performance on AWS all day, but it sure beats the pants off my $40 server.

What This Is All About

Pyrit is a GPU password cracking tool. I'm going to use it as a password recovery tool to see how long it takes to find my home network's password.

AWS is Amazon Web Services, the rent-a-supercomputer marketplace. You can rent all kinds of computers, but most interesting for our purposes, you can rent a GPU cluster.

A helpful guide here: http://j0k3rrr.blogspot.com/2011/07/wpa-cracking-on-amazons-ec2-instance.html

How To Do It

AWS Account Setup

First things first: set up an AWS account (requires credit card and phone number).

There is some useful AWS documentation here, that guides you through the steps of getting set up with AWS: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/get-set-up-for-amazon-ec2.html

Amazon IAM

Now you will need to create users and groups for your EC2 instances. This can be done with Amazon's IAM, Integrated Account Manger. This will allow you to have the proper credentials to access an Amazon EC2 instance.

Administrator Group

This will give you administrator access on any EC2 instances that you fire up. Create the administrator group as described in the instructions here: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/get-set-up-for-amazon-ec2.html

Make Users

Create your users: I made a root account and a normal account. Download their credentials file.

Now you'll use the IAM interface to add both of these users to the Administrators group, and set their passwords.

The instructions here are good: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/get-set-up-for-amazon-ec2.html

When you go to the IAM main page, which lists all the users, you should also see a special URL you can use to sign in as the user that you have created. (Users?)

The url will look something like this:

https://[some multi-digit number that is actually your Amazon Web Services ID].signin.aws.amazon.com/console

Log In As Your User

Once you log in as your root user, you'll see an AWS console. One more step before you fire up an instance, and that's adding an SSH key.

Create Key Pair

Now, let's return to the AWS Control Panel.

The Linux instances you rent through Amazon can't be accessed with a password - they use public and private keys to connect. So you'll need to give a public key to Amazon.

You first create a key. You will name it (since you need one key per region, you can just name it with your username plus whatever region it is for).

Then you'll download a .pem file. This is your private AWS key. You will use this to prove to Amazon that, yes, you are in fact you, each time that you connect to your compute notes.

Keep the private key secret:

$ chmod 400 charles-oregon-key-pair.pem

Now you can connect with SSH, and point SSH to the private key with the -i flag:

$ ssh -i charles-oregon-key-pair.pem [etc etc etc]

Create Virtual Private Cloud

Whew! Almost done. Now you have to determine if you have a Virtual Private Cloud (VPC) by default. When I logged into my AWS account, and opened the EC2 portal, I saw "VPC" in the upper right hand part of the page.

Create Security Group

Now you have to create a security group that can access the node you're renting. Add access to port 22 for SSH. This allows your user account to access the nodes through port 22.

It will also be useful to have HTTP and HTTPS traffic turned on, so add those too.

That'll open ports 22, 80, and 443 on our AWS node.

Finally DONE

We're finally done setting everything up in AWS, we're ready to grab our $2 GPU node.

Getting the GPU Instance

Okay, you're almost ready to oggle at all the things on offer at the AWS store. First you have to log out of AWS (I mean really log out) and then go to the log in page. When you enter your AWS account name, there should be a text link below that says something like "Log in as root". You'll need to do this to log in as root.

Note that even if you use IAM to create a user named "root," it is essentially no different from any other account.

Operating System

Your first choice is operating system. I went with Ubuntu - classic - nothing fancy. We'll install everything we need from source.

Hardware

Your second choice is hardware. You can use the drop down menu to find GPU instances. There were two available to me: a GPU cluster with 8 CPU cores, 60 GB memory, and 16 GB RAM, or a monster 32 CPU core machine with 60 GB, 240 GB SSD storage, and a 10 Gigabit network link. Woah. That's serious.

I picked the "low end" GPU machine and kept going.

Configuring Instance

Settings related to the instance.

Adding Storage

This is where I got confused. On the Hardware page, the specs list a 60 GB hard drive. But here, the hard drive is 8 GB. What gives?

Configure and Security

Now you have the option of creating a new security group or using an existing one. I picked teh security group I already created, which had SSH, HTTP, and HTTPS enabled.

Launch, and More Key Pairs

Once you click the Launch button, you have to pick how you're going to connect to the instance - what security keys you're going to use. I picked "Choose an existing pair" and picked the pair of keys that I had already set up.

Log In

Once you've got your instance, you can log in to the machine. You'll need your private key, and you'll need the public DNS of the instance from the AWS console.

AWS GPU Node Notes

Processors

$ cat /proc/cpuinfo 
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 45
model name	: Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
stepping	: 7
microcode	: 0x70d
cpu MHz		: 2600.040
cache size	: 20480 KB
physical id	: 0
siblings	: 8
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx rdtscp lm constant_tsc rep_good nopl xtopology eagerfpu pni pclmulqdq ssse3 cx16 pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx hypervisor lahf_lm xsaveopt
bogomips	: 5200.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 45
model name	: Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
stepping	: 7
microcode	: 0x70d
cpu MHz		: 2600.040
cache size	: 20480 KB
physical id	: 0
siblings	: 8
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx rdtscp lm constant_tsc rep_good nopl xtopology eagerfpu pni pclmulqdq ssse3 cx16 pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx hypervisor lahf_lm xsaveopt
bogomips	: 5200.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 45
model name	: Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
stepping	: 7
microcode	: 0x70d
cpu MHz		: 2600.040
cache size	: 20480 KB
physical id	: 0
siblings	: 8
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx rdtscp lm constant_tsc rep_good nopl xtopology eagerfpu pni pclmulqdq ssse3 cx16 pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx hypervisor lahf_lm xsaveopt
bogomips	: 5200.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 45
model name	: Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
stepping	: 7
microcode	: 0x70d
cpu MHz		: 2600.040
cache size	: 20480 KB
physical id	: 0
siblings	: 8
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx rdtscp lm constant_tsc rep_good nopl xtopology eagerfpu pni pclmulqdq ssse3 cx16 pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx hypervisor lahf_lm xsaveopt
bogomips	: 5200.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 45
model name	: Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
stepping	: 7
microcode	: 0x70d
cpu MHz		: 2600.040
cache size	: 20480 KB
physical id	: 0
siblings	: 8
core id		: 0
cpu cores	: 4
apicid		: 1
initial apicid	: 1
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx rdtscp lm constant_tsc rep_good nopl xtopology eagerfpu pni pclmulqdq ssse3 cx16 pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx hypervisor lahf_lm xsaveopt
bogomips	: 5200.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 45
model name	: Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
stepping	: 7
microcode	: 0x70d
cpu MHz		: 2600.040
cache size	: 20480 KB
physical id	: 0
siblings	: 8
core id		: 1
cpu cores	: 4
apicid		: 3
initial apicid	: 3
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx rdtscp lm constant_tsc rep_good nopl xtopology eagerfpu pni pclmulqdq ssse3 cx16 pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx hypervisor lahf_lm xsaveopt
bogomips	: 5200.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 45
model name	: Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
stepping	: 7
microcode	: 0x70d
cpu MHz		: 2600.040
cache size	: 20480 KB
physical id	: 0
siblings	: 8
core id		: 2
cpu cores	: 4
apicid		: 5
initial apicid	: 5
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx rdtscp lm constant_tsc rep_good nopl xtopology eagerfpu pni pclmulqdq ssse3 cx16 pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx hypervisor lahf_lm xsaveopt
bogomips	: 5200.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 45
model name	: Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
stepping	: 7
microcode	: 0x70d
cpu MHz		: 2600.040
cache size	: 20480 KB
physical id	: 0
siblings	: 8
core id		: 3
cpu cores	: 4
apicid		: 7
initial apicid	: 7
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx rdtscp lm constant_tsc rep_good nopl xtopology eagerfpu pni pclmulqdq ssse3 cx16 pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx hypervisor lahf_lm xsaveopt
bogomips	: 5200.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

GPU Info

$ sudo apt-get install nvclock
$ nvclock -i

It seems your card isn't officialy supported in NVClock yet.
The reason can be that your card is too new.
If you want to try it anyhow [DANGEROUS], use the option -f to force the setting(s).
NVClock will then assume your card is a 'normal', it might be dangerous on other cards.
Also please email the author the pci_id of the card for further investigation.
[Get that value using the -i option].

Oops. Okay, we'll look at PCI information:

$ lspci | grep NVIDIA
00:03.0 VGA compatible controller: NVIDIA Corporation GK104GL [GRID K520] (rev a1)

$ lspci -v

00:03.0 VGA compatible controller: NVIDIA Corporation GK104GL [GRID K520] (rev a1) (prog-if 00 [VGA controller])
	Subsystem: NVIDIA Corporation Device 1014
	Physical Slot: 3
	Flags: bus master, fast devsel, latency 0, IRQ 5
	Memory at ec000000 (32-bit, non-prefetchable) [size=16M]
	Memory at e0000000 (64-bit, prefetchable) [size=128M]
	Memory at ea000000 (64-bit, prefetchable) [size=32M]
	I/O ports at c100 [size=128]
	Expansion ROM at ee000000 [disabled] [size=512K]
	Capabilities: [60] Power Management version 3
	Capabilities: [68] MSI: Enable- Count=1/1 Maskable- 64bit+
	Capabilities: [78] Express Endpoint, MSI 00
	Capabilities: [b4] Vendor Specific Information: Len=14 <?>

Hard Disks

I was hoping that once I logged onto the node, it would clear up some of the confusion about what I was seeing with the hard drive selection, 8 GB vs the advertised 60 GB. Turns out it is both:

$ df -h

Filesystem      Size  Used Avail Use% Mounted on
/dev/xvda1      7.8G  782M  6.6G  11% /
none            4.0K     0  4.0K   0% /sys/fs/cgroup
udev            7.4G   12K  7.4G   1% /dev
tmpfs           1.5G  332K  1.5G   1% /run
none            5.0M     0  5.0M   0% /run/lock
none            7.4G     0  7.4G   0% /run/shm
none            100M     0  100M   0% /run/user
/dev/xvdb        64G   52M   61G   1% /mnt

8 GB SSD is the root directory, mount point /, so that's how much you have "on-board," but the 60 GB xvdb, well, I guess that's cloud storage.

Build Stuff

Build Pyrit

Instructions on the Pyrit page

Build Pyrit CUDA