Recently, I set out on a quest to build an 8-GPU cracking machine that would chew through most of the hash types I came across on engagements. It’s something that I’ve wanted to do for quite some time now, and finally got around to doing it.

You’re probably asking, “Isn’t this a bit overkill?” Absolutely. There’s really no reason to go this hardcore; especially since our company also has a rig to crack hashes. However, I do intend to use the GPUs for other projects though (no, not mining) that I’ll blog about another time.

I had a couple ways to go about doing this – the first was to go the easy (but very expensive) route and purchase a Sagitta Brutalis.

These looked to start at $21,169 but were configured – according to the site – with 8 x NVIDIA GTX 1080 Founder Editions (FEs). Not the Tis or RTX series. This was a bit expensive for my taste, especially for just the original 1080s. The second option was to research and build my own rig, which I was leaning towards doing anyway.

Hardware

For the chassis/mobo, I ended up going with the TYAN FT77C-B7079 barebones kit. Right off the bat, you can tell this kit was built for GPU workloads. The mobo contains 8 x PCI-E Gen3 x16 double-spaced slots, and a power distribution board that makes it a breeze to connect power to the cards, while also giving it a clean look.

Everything is set up in the chassis to maximize airflow from the front to the back. Six fans are placed near the front of the chassis for air intake, then air is pushed over the CPU heat sinks and video cards to exhaust out at the rear.

For added convenience, backplanes are pre-installed for the PSUs, system fans, and drives that make them plug-and-play. The kit (should) come with everything you need such as power cables, GPU cables, and rails if you plan on mounting it.

The official site lists several vendors that sold the kit. After checking each one out, I ended up going with Thinkmate since the prices were reasonable, and they provided plenty of configuration options (including blower style RTX cards). They will also build everything for you, so if you decide to add GPUs to your configuration, you could get everything shipped to you ready-to-go. But since they were charging about $200 more per RTX 2070 card from what I found at Newegg, I decided to get the GPUs separately.

My initial plan was to grab 7 x NVIDIA GTX 1080 Ti FEs then add a spare GTX 1080 I had to the mix. Stock has always been a problem going this route, and even more so now that NVIDIA stopped selling them on their official store. eBay prices were all over the place for used 1080 Ti FEs, and you never really know how much of a beating a card has taken due to the mining craze.

Rather than spending $650-$700 on each used card, I decided to buy seven blower-style RTX 2070s for $500-550 a piece from Newegg. These RTX 2070s sit between the 1080 and 1080 Ti in terms of performance. That is: slightly faster than a 1080, but slower than a 1080 Ti.

Once everything arrived, all I needed to do on the hardware side was to plug the cards in and make sure everything else was good and tight. Since Thinkmate ships it to you prebuilt and tested, there was nothing else to do except make sure it posts after powering it on (which it did).

This is probably to be expected with most servers in this form-factor, but this thing is LOUD. I actually ended up removing 3 of the 6 intake fans just to help lower the noise a bit. As long as you have this in a well-ventilated area, you shouldn’t run into any cooling issues with just three fans. Obviously, YMMV.

Software

For the OS, I decided to go with Ubuntu Server 18.04 since I planned for this to be a headless server. Installation was fairly straightforward. If you’re on Windows, you can follow the instructions here to create a bootable USB stick, but other installation tutorials exist if you’re on a different OS. After installing the OS, some additional configuration was needed such as blacklisting the Nouveau OpenGL drivers prior to installing the official NVIDIA drivers.

Once the drivers were installed, I generated an Xorg config so I could use nvidia-settings to control the fans on the GPUs. Since this tool requires an X service to be running on the server, I’ll direct you to this guide for steps on setting up your environment and generating the config. I highly recommend doing this since you could run into thermal throttling issues if the fans aren’t set to 100% while cracking.

Benchmarks

Finally, this post wouldn’t be complete without some benchmarks of the beast in action. Feel free to leave a comment if there’s other hashes you’d like to see:

Hashmode: 1000 - NTLM

Speed.#1.........: 47484.1 MH/s (12.38ms) @ Accel:16 Loops:1024 Thr:1024 Vec:1
Speed.#2.........: 47611.7 MH/s (12.33ms) @ Accel:16 Loops:1024 Thr:1024 Vec:1
Speed.#3.........: 48585.6 MH/s (12.12ms) @ Accel:16 Loops:1024 Thr:1024 Vec:1
Speed.#4.........: 48021.4 MH/s (12.29ms) @ Accel:16 Loops:1024 Thr:1024 Vec:1
Speed.#5.........: 47821.3 MH/s (12.20ms) @ Accel:16 Loops:1024 Thr:1024 Vec:1
Speed.#6.........: 47934.0 MH/s (12.22ms) @ Accel:16 Loops:1024 Thr:1024 Vec:1
Speed.#7.........: 47483.0 MH/s (12.38ms) @ Accel:16 Loops:1024 Thr:1024 Vec:1
Speed.#8.........: 43379.8 MH/s (15.09ms) @ Accel:32 Loops:1024 Thr:1024 Vec:2
Speed.#*.........:   378.3 GH/s


Hashmode: 5500 - NetNTLMv1 / NetNTLMv1+ESS

Speed.#1.........: 27490.0 MH/s (21.56ms) @ Accel:16 Loops:1024 Thr:1024 Vec:1
Speed.#2.........: 27735.2 MH/s (21.49ms) @ Accel:16 Loops:1024 Thr:1024 Vec:1
Speed.#3.........: 28118.7 MH/s (21.11ms) @ Accel:16 Loops:1024 Thr:1024 Vec:1
Speed.#4.........: 27654.0 MH/s (21.44ms) @ Accel:16 Loops:1024 Thr:1024 Vec:1
Speed.#5.........: 27882.7 MH/s (21.34ms) @ Accel:16 Loops:1024 Thr:1024 Vec:1
Speed.#6.........: 27883.1 MH/s (21.40ms) @ Accel:16 Loops:1024 Thr:1024 Vec:1
Speed.#7.........: 27399.7 MH/s (21.64ms) @ Accel:16 Loops:1024 Thr:1024 Vec:1
Speed.#8.........: 22634.9 MH/s (29.21ms) @ Accel:32 Loops:1024 Thr:1024 Vec:1
Speed.#*.........:   216.8 GH/s


Hashmode: 5600 - NetNTLMv2

Speed.#1.........:  2044.7 MH/s (73.52ms) @ Accel:16 Loops:256 Thr:1024 Vec:1
Speed.#2.........:  2047.4 MH/s (73.42ms) @ Accel:16 Loops:256 Thr:1024 Vec:1
Speed.#3.........:  2092.0 MH/s (71.86ms) @ Accel:16 Loops:256 Thr:1024 Vec:1
Speed.#4.........:  2054.1 MH/s (73.18ms) @ Accel:16 Loops:256 Thr:1024 Vec:1
Speed.#5.........:  2063.5 MH/s (72.82ms) @ Accel:16 Loops:256 Thr:1024 Vec:1
Speed.#6.........:  2064.7 MH/s (72.80ms) @ Accel:16 Loops:256 Thr:1024 Vec:1
Speed.#7.........:  2039.3 MH/s (73.71ms) @ Accel:16 Loops:256 Thr:1024 Vec:1
Speed.#8.........:  1858.8 MH/s (89.81ms) @ Accel:32 Loops:256 Thr:1024 Vec:1
Speed.#*.........: 16264.6 MH/s


Hashmode: 13100 - Kerberos 5 TGS-REP etype 23

Speed.#1.........:   346.6 MH/s (54.28ms) @ Accel:128 Loops:64 Thr:64 Vec:1
Speed.#2.........:   349.3 MH/s (53.86ms) @ Accel:128 Loops:64 Thr:64 Vec:1
Speed.#3.........:   347.1 MH/s (54.21ms) @ Accel:128 Loops:64 Thr:64 Vec:1
Speed.#4.........:   351.2 MH/s (53.58ms) @ Accel:128 Loops:64 Thr:64 Vec:1
Speed.#5.........:   352.4 MH/s (53.39ms) @ Accel:128 Loops:64 Thr:64 Vec:1
Speed.#6.........:   347.3 MH/s (54.17ms) @ Accel:128 Loops:64 Thr:64 Vec:1
Speed.#7.........:   352.1 MH/s (53.43ms) @ Accel:128 Loops:64 Thr:64 Vec:1
Speed.#8.........:   309.3 MH/s (67.57ms) @ Accel:256 Loops:64 Thr:64 Vec:1
Speed.#*.........:  2755.3 MH/s


Hashmode: 2100 - Domain Cached Credentials 2 (DCC2), MS Cache 2 (Iterations: 10240)

Speed.#1.........:   367.7 kH/s (79.53ms) @ Accel:16 Loops:512 Thr:1024 Vec:1
Speed.#2.........:   368.8 kH/s (79.26ms) @ Accel:16 Loops:512 Thr:1024 Vec:1
Speed.#3.........:   376.6 kH/s (77.61ms) @ Accel:16 Loops:512 Thr:1024 Vec:1
Speed.#4.........:   370.0 kH/s (79.00ms) @ Accel:16 Loops:512 Thr:1024 Vec:1
Speed.#5.........:   372.1 kH/s (78.53ms) @ Accel:16 Loops:512 Thr:1024 Vec:1
Speed.#6.........:   372.0 kH/s (78.58ms) @ Accel:16 Loops:512 Thr:1024 Vec:1
Speed.#7.........:   367.6 kH/s (79.53ms) @ Accel:16 Loops:512 Thr:1024 Vec:1
Speed.#8.........:   348.7 kH/s (92.88ms) @ Accel:32 Loops:512 Thr:1024 Vec:1
Speed.#*.........:  2943.5 kH/s


Hashmode: 2500 - WPA-EAPOL-PBKDF2 (Iterations: 4096)

Speed.#1.........:   447.0 kH/s (40.70ms) @ Accel:16 Loops:128 Thr:1024 Vec:1
Speed.#2.........:   447.4 kH/s (40.67ms) @ Accel:16 Loops:128 Thr:1024 Vec:1
Speed.#3.........:   456.3 kH/s (39.86ms) @ Accel:16 Loops:128 Thr:1024 Vec:1
Speed.#4.........:   448.8 kH/s (40.53ms) @ Accel:16 Loops:128 Thr:1024 Vec:1
Speed.#5.........:   449.9 kH/s (40.41ms) @ Accel:16 Loops:128 Thr:1024 Vec:1
Speed.#6.........:   452.0 kH/s (40.25ms) @ Accel:16 Loops:128 Thr:1024 Vec:1
Speed.#7.........:   445.3 kH/s (40.84ms) @ Accel:16 Loops:128 Thr:1024 Vec:1
Speed.#8.........:   427.4 kH/s (94.48ms) @ Accel:32 Loops:256 Thr:1024 Vec:1
Speed.#*.........:  3574.3 kH/s


Hashmode: 5400 - IKE-PSK SHA1

Speed.#1.........:   881.7 MH/s (63.86ms) @ Accel:32 Loops:64 Thr:768 Vec:1
Speed.#2.........:   885.8 MH/s (63.57ms) @ Accel:32 Loops:64 Thr:768 Vec:1
Speed.#3.........:   899.4 MH/s (62.61ms) @ Accel:32 Loops:64 Thr:768 Vec:1
Speed.#4.........:   886.1 MH/s (63.54ms) @ Accel:32 Loops:64 Thr:768 Vec:1
Speed.#5.........:   889.1 MH/s (63.31ms) @ Accel:32 Loops:64 Thr:768 Vec:1
Speed.#6.........:   887.5 MH/s (63.44ms) @ Accel:32 Loops:64 Thr:768 Vec:1
Speed.#7.........:   878.4 MH/s (64.10ms) @ Accel:32 Loops:64 Thr:768 Vec:1
Speed.#8.........:   773.9 MH/s (80.96ms) @ Accel:32 Loops:128 Thr:768 Vec:1
Speed.#*.........:  6981.8 MH/s