Developer’s Guide to Building a High Performance Desktop Machine
Laptops can be useful for day-to-day development, but a more powerful machine is required when it comes to working with Very Large Databases (VLDBs) and Big Data. Having some background in building PCs, I recently put together a powerful desktop for my own needs that could handle my current project and any future projects involving large scale databases. I learned quite a bit about the current generation of hardware available, mostly by sifting through forums and blog posts from gamers that build high performance gaming machines.
As with anything, understanding each subsystem and identifying the key performance issues will often cut through the confusion and help to construct the most optimum solution. Here is my simple introduction and guide to selecting the best components for your next development machine purchase or build, while saving you time and anguish trying to sort through the variety of information available.
Buying vs. Building a Machine
So, should I just buy a new machine? Often, this can solve your immediate problem. Apple offers a solution for powerful desktops, as do Dell, HP, and others. Simply buy a machine with a faster processor, more memory, and multiple drives and you can get far.The problem lies in the fact that many of these builders don’t offer the most powerful or current hardware available, as they tend to lag by 3-6 months. This typically provides an advantage, as the builders will determine what parts work the best and then combine them together. However, unless you go for a high-end machine from these builders, they will often substitute one part for another, based on their current inventory. The result is a machine that is not quite to spec, or lagging behind the most current technology available. If you prefer to find the best machine performance possible, then the best option is to build a custom machine.
If you are reading this article, my assumption is that you plan on working with large amounts of data using multiple processes or virtualized servers within a development environment, and will therefore benefit from the best performance you can buy. To do this, you must have a basic understanding of how to select the best parts for your machine that will work together properly. Either way, this guide will help you better understand your options before you buy or build your next development workstation.
Selecting the Processor and Memory
Processors are commonly thought of as throw-away parts. Find the fastest and install it. However, with the current selection of processor families, there are often several choices. On the Intel side, there are currently options for the i5 and i7 families. Within each family are options for the number of cores (often 2, 4, or 6 cores), the L2 and L3 cache size, and the QPI rating. The fastest i7 processor with the most cores, L2 and L3 cache, and the highest QPI will often provide the most processing power when it comes to working with Big Data solutions like Hadoop or distributing computing applications.
Memory can be a little more confusing. Since the i7 processor can talk to the memory using triple channels, it is best to select the proper memory to complement your processor selection. Memory is commonly rated with four numbers, such as 7-7-7-21, indicating the latency of the memory (deeper research about these numbers is interesting but left as an exercise for the reader). This means finding the fastest DDR3 triple channel memory with the lowest latency and buying enough to handle your development needs. Often, you have to inform the BIOS of your motherboard about the speed of your memory, otherwise it will default to a slower rating for stability. More on that later.
Motherboard
If you have every walked into a store like Fry’s, you will see a large selection of motherboards. They vary is sizes, features, and capability. Most motherboards are designed for the average price-conscious consumer that wants to see the most number of features for the dollar. In reality, development desktops need something different. They need to be reliable for long-term use, with 24×7 uptime and stability. I have found that Asus is often a good choice as a motherboard manufacturer, particularly their Sabretooth X58 motherboard. Do some research for the operating system you will be running to ensure the best compatibility, as even the best manufacturer may take a short-cut occasionally and install a controller chip that isn’t as friendly for some operating systems (especially Ubuntu/Linux).
Hard Drives and RAID
If you own a modern laptop, you may already have a solid state drive (SSD drive) installed. They are silent, having no moving parts, and are quite fast to boot your favorite operating system as some can achieve over 300MB/s during read access. These drives are sufficient for most uses and for boot drives for the OS, but often they are not as efficient when it comes to mass storage requirements. Big Data and VLDBs often require large amounts of throughput and storage during development, which is currently beyond the capacity of modern SSDs. This means that traditional spindle-based SATA drives are still the best option for our needs. This does not, however, prevent you from mixing your storage subsystem with a combination of SSD drive(s) for the OS and spindle-based drives for mass data storage. The primary concern when selecting your hard drives are speed, reliability, and capacity.
SATA drives are commonly available in different speeds: 5400, 7200, and 10k rpm. The faster and smaller the drive, the faster data can be accessed and written. SAS drives are a newer form of spindle-drive commonly found on higher end server machines and can achieve speeds of up to 15k rpm and even faster seek times. They are very expensive, however, and therefore often out of the reach of most developers. Another factor is the mean time between failure (MTBF) rating, which is an estimate of the number of hours before the drive is likely to fail. Lower-grade consumer drives often have lower MTBF and are more likely to fail while server-class drives have a much higher MTBF. They also run cooler, which is another important factor for long life and reliability. The best choice at this time is the Western Digital Velociraptor 10k rpm drives, currently found in 300 and 600GB sizes. They have a higher MTBF, run cool, and are generally a good choice for high performance desktop use.
RAID controllers allow multiple drives to be combined and viewed as a single device by the operating system. RAID 0 allows two or more drives to be striped (written to and read from in parallel), although if any one fails the entire system fails. RAID 1 allows for two or more drives to be mirrored, providing a simple form of redundancy and parallel reads (assuming the controller allows it). RAID 5 and 6 provide a more economical way to provide data redundancy but can sometimes hinder performance when it comes to the way some database servers read and write data to the filesystem. I’ve found that the most optimum RAID choice for development machines is RAID1+0 (sometimes called RAID 10). This means that there are two RAID 0 drive arrays that are then combined into a RAID 1 pair, allowing for very fast reads and redundant data storage when writing.
When selecting a controller for your RAID configuration, many utilize the onboard RAID controller from their motherboard. While many motherboards are now using more popular RAID controllers from known vendors, they still require the use of the processor to perform the RAID logic. Before you choose to use the onboard controller, do some research for issues during RAID recoveries and rebuilding after drives fail. Often, these controllers are great when they work but terribly slow or unreliable during a rebuild after a drive failure. My recommendation is to select a separate RAID controller card that has a dedicated processor, onboard memory (sometimes with a battery backup), and support for 4, 8, 16, or 32 drives. These cards are typically $300 or more in price, while lower priced cards are likely no different than the controllers available on today’s motherboards. I prefer the Adaptec 5805 series RAID controller. These cards can sometimes be obtained at a discount due to an open box or overstock. Caution: be sure to purchase the kit, not just the standalone card, or else you will be missing the cable sets that are often required to connect the card to the drives.
One final note about hard drive selection: the larger the hard drives selected, the longer it takes to rebuild a RAID device when a drive fails. If you have a drive failure and replace your drive, you may still be running in degraded mode (one or more drives are unavailable and being rebuilt) for some time. This can be a huge factor in the middle of a development cycle, so minimizing the rebuild time is a must.
Case and Power Supply
Selecting a case is mostly an aesthetic decision, although there are a few factors to consider: number of available hard drive bays, location of the power supply, and airflow. The number of available drive bays must meet or exceed the number of drives you intend on installing and may require that you locate a case that supports the smaller SSD drives if you plan on installing one without an adapter for the larger SATA-sized bays. Power supplies are traditionally located at the top of the case, though more are starting to locate them at the bottom for easier access, a lower likelihood of tipping over, and aesthetics. Finally, identifying a case that has good airflow from front-to-back, plenty of locations for fans of 120mm or larger, and even CPU funnels will help extract the heat produced from your processor, video card, and hard drives efficiently. I prefer the Antec Nine Hundred v3 (not the v2) or the Twelve Hundred, as they provide plenty of fans, great airflow, plenty of hard drive bays, and have filters on the front fans for easily removing dust.
Power supplies are often the most overlooked and underrated part of the desktop. One is just like the other, right? Wrong! The parts inside the power supply can vary in quality and power efficiency, and can easily take down your entire machine if they fail. Find a power supply from a well-rated manufacturer that offers the appropriate rating in watts for your machine. Most i7 processors with up to 4 hard drives and a moderate video card can utilize a gold series 800W or better power supply. Also, look for modular power supplies, as they allow you to select only the cables you need for your installation, reducing the number of cables you have to deal with inside your desktop case. Finally, ensure that the cables supplied with the power supply will reach all devices in your selected case, from the motherboard power sockets to the hard drives and DVD.
My Setup
To conclude this guide, I’ll outline the setup I built in the fall of 2010, to help get you started. I found that this was the best combination of power and low noise for a pleasant yet powerful work environment:
- Intel i7 980X Extreme Edition Processor (6 core) w/ stock CPU fan on quiet mode
- G.Skill 7-7-7-21 1600 DDR3 Memory
- ASUS Sabretooth X58
- Adaptec 5805 RAID controller (up to 8 drives)
- 4 x Velociraptor 600GB 10k rpm drives (6GB/s) in RAID 10
- Antec Nine Hundred v3 case
- ASUS AHA5450 Silent ATI video card
- Ubuntu Linux 10.10 w/ native ATI drivers installed (I prefer Gentoo but Ubuntu is much easier to maintain on a daily basis)
Memory Performance Results
Memtest86+ results before and after I adjusted the BIOS to use the 7-7-7-21 memory settings:
- Mem stable at 9-9-9-24 (BIOS default):
BLCK 133 Mhz // QPI 6.4 GT/s // RAM 668 Mhz // L1=65509 MB/s // L2=22574 MB/s // L3=6449 MB/s // Memory speed 4949 MB/s - Mem stable at 7-7-7-21 (spec):
BLCK 141 Mhz // QPI 6.7 GT/s // RAM 705 Mhz // L1=69176 MB/s // L2=22328 MB/s // L3=15679 MB/s // Memory speed 9069 MB/s
Hard Drive Performance Results
Finally, here are some quick performance tests with the 4 drives in RAID 10. While I could add 4 more drives to achieve higher results, these results are great and provide a nice balance between the performance vs. cost, heat, and power consumption.
ubuntu@ubuntu:~$ sudo hdparm -tT /dev/sda
/dev/sda:
Timing cached reads: 17128 MB in 2.00 seconds = 8570.32 MB/sec
Timing buffered disk reads: 1562 MB in 3.00 seconds = 520.29 MB/sec
Conclusion
I hope that this article has been beneficial and will assist you in the key decisions needed to build a powerful desktop machine for high performance development. If you have any questions, please drop me a comment or a note using the contact form and I’ll do my best to answer any questions you may have.