WPA cracking with AMD Stream and a Radeon HD4870

You might know the pyrit project. After installing the AMD Stream SDK, I decided to add support for Stream to pyrit.

Here's my preliminary patch. I contacted the pyrit author, he wants to merge it in the next days.

A good starting point was the CUDA code of pyrit. Also, I had a look at some Stream samples from the AMD SDK.

The input streams would be the i- and opad values, along with the essid/password HMAC data.
The output stream would be the PMK data.
Each stream transfers one integer value, so it's 5 streams each for i- and opad, another 5 for the HMAC data and another 5 for each part of the PMK.

One has to read the data to the streams before invoking the GPU function call and write the output from the PMK streams to an array afterwards.

Streams can only be used in C++, so I had to stick with C++. Gluing it to cpyrit is done using an extern "C" function that's callable from C.

Like the CUDA code, I'm using a SHA1 implementation with all 80 rounds unrolled. This code is executed on the GPU 4095 times for each of the two PMK parts.

The brook brcc compiler doesn't like arrays. Luckily the SHA1 array accesses don't depend on the actual data being processed, but only on the SHA1 round number. Thus the indices are constant.
So I wrote a tiny perl script that resolves the array indices (round number - x masked with y) using eval() and substitutes each array element by a seperate variable. For example W[5] is substituted by W_5. With some help from lobo I finally replaced the perl script with a few lines of python in the pyrit setup.py script.

But the array part wasn't the ugly part. The ugly part is a compiler bug.
As one can see in the CUDA code, W[5] is initialized with 0x80000000. brcc doesn't like the most significant bit. brcc just ignores that bit!
So I removed round by round both on the host and the GPU and compared the results. Finally I found the problem with the 0x8...
My temporary 'fix' is passing that value to the GPU using a value of the GPU function call.
But I gotta write a mail to AMD the next days.

The last part was optimization. Like the CUDA version I tried doing both PMK parts on the GPU consecutively but performance dropped.
I also tried calculating 4 PMK parts (= 2 full PMKs) at the same time using the int4 SIMD data type. This also resulted in a performance drop, so I removed it. Later I also tried the int3 and int2 type and finally got the best performance with the int2 type. Maybe it's due to cache exhaustion with int3/4.

Using int2 both parts of the PMK are calculated at the same time now. I tried transferring the i/opad only once and expanding it on the GPU but this resulted in less performance as well. So the same values are transferred twice now. (for the 1st and 2nd part of the PMK) It's faster that way.

With my Radeon HD4870 I'm finally getting about 9650 PMKs/s at the normal clocks and 10000 PMKs/s with modest and stable atitool overclocking to 790/1050 MHz.

I don't really see room for further optimization. Thus I file this project under DONE & FINISHED :-)