Server Guide Part 1: Introduction to the Server World
by Johan De Gelas on August 17, 2006 1:45 PM EST- Posted in
- IT Computing
TCO
Originally described by the Gartner group, TCO sounds like something that does not belong on a hardware enthusiast site. It has frequently been abused by managers and financial people who understand very little of IT to delay necessary IT investments, so many view it as a pejorative term.
However it is impossible to make a well thought-out server buying decision without understanding TCO, and many typical server hardware features are based on the idea of lowering TCO. Hardware enthusiasts mostly base their buying decision on TCA or Total Cost of Acquisition. The enthusiast motherboard and chipset business is a typical example of how to ignore TCO. As the products are refreshed every 6 months, many of the new features don't work properly, and you find yourself flashing the BIOS, installing new drivers and tweaking configurations before you hopefully get that RAID, Firewall or sound chip to work properly. Luckily you don't have to pay yourself for all the hours you spend....
TCO is a financial estimate of the total cost of buying and using a server. Think of it as the cost that it takes to buy, deploy, support and adapt a certain server during it's lifecycle. So when evaluating servers you should look at the following costs:
There are two big problems with the "hardware choice does not matter much" kind of reasoning. The first is that the TCA is still a big part of the total TCO. For example this study[1] estimates that the price of buying the server is still about 40-50% of the TCO, while maintenance comprises a bit more than 10% and operation costs take about 40% of TCO pie. Thus we can't help but be wary when a vendor claims that a high price is okay, because the maintenance on his product is so much lower than the competition.
Secondly, certain hardware choices have an enormous impact on the rest of the TCO picture. One example is hot-spare and hot-swappable RAID arrays which on average significantly reduce the time that a server is unreachable. This will also become clearer as we dig deeper into the different hardware features of modern servers and the choices you will have to make.
RAS features
Studies done by IBM say that about 50% of the hardware failures are related to hard disk problems and 25% are due to a power supply failure. Fans with 8% are a distant third, so it is clear you need power supplies and hard disks of high reliability, the R of RAS. You also want to increase availability, the A of RAS, by using some redundancy for the most vulnerable parts of your server. RAID, redundant power supplies and fans are a must for a critical server. The S in RAS stands for Serviceability, which relates to hot-swappable/pluggable drives and other areas. Do you need to shut down the server to perform maintenance; what items can be replaced/repaired while keeping the system running? All three items are intertwined, and higher-end (and more expensive) servers will have features designed to improve all three areas.
Originally described by the Gartner group, TCO sounds like something that does not belong on a hardware enthusiast site. It has frequently been abused by managers and financial people who understand very little of IT to delay necessary IT investments, so many view it as a pejorative term.
However it is impossible to make a well thought-out server buying decision without understanding TCO, and many typical server hardware features are based on the idea of lowering TCO. Hardware enthusiasts mostly base their buying decision on TCA or Total Cost of Acquisition. The enthusiast motherboard and chipset business is a typical example of how to ignore TCO. As the products are refreshed every 6 months, many of the new features don't work properly, and you find yourself flashing the BIOS, installing new drivers and tweaking configurations before you hopefully get that RAID, Firewall or sound chip to work properly. Luckily you don't have to pay yourself for all the hours you spend....
TCO is a financial estimate of the total cost of buying and using a server. Think of it as the cost that it takes to buy, deploy, support and adapt a certain server during it's lifecycle. So when evaluating servers you should look at the following costs:
- The total cost of buying the server
- The time you will spend installing it in your network
- The time you will spend on configuring the software and remote management
- Facility management: the space it takes in your datacenter and the electricity it consumes
- The hours you spend on troubleshooting, reconfiguring, securing and repairing the server
- The costs associated with users waiting for the system to respond
- The costs associated with outages and failures, with users not being able to reach your server
- The upgrade costs and the time you spend on upgrading your server to meet new demands
- Cost of security breaches, etc.
There are two big problems with the "hardware choice does not matter much" kind of reasoning. The first is that the TCA is still a big part of the total TCO. For example this study[1] estimates that the price of buying the server is still about 40-50% of the TCO, while maintenance comprises a bit more than 10% and operation costs take about 40% of TCO pie. Thus we can't help but be wary when a vendor claims that a high price is okay, because the maintenance on his product is so much lower than the competition.
Secondly, certain hardware choices have an enormous impact on the rest of the TCO picture. One example is hot-spare and hot-swappable RAID arrays which on average significantly reduce the time that a server is unreachable. This will also become clearer as we dig deeper into the different hardware features of modern servers and the choices you will have to make.
RAS features
Studies done by IBM say that about 50% of the hardware failures are related to hard disk problems and 25% are due to a power supply failure. Fans with 8% are a distant third, so it is clear you need power supplies and hard disks of high reliability, the R of RAS. You also want to increase availability, the A of RAS, by using some redundancy for the most vulnerable parts of your server. RAID, redundant power supplies and fans are a must for a critical server. The S in RAS stands for Serviceability, which relates to hot-swappable/pluggable drives and other areas. Do you need to shut down the server to perform maintenance; what items can be replaced/repaired while keeping the system running? All three items are intertwined, and higher-end (and more expensive) servers will have features designed to improve all three areas.
Facebook
Twitter
RSS
32 Comments
View All Comments
JarredWalton - Thursday, August 17, 2006 - link
Fixed.Whohangs - Thursday, August 17, 2006 - link
Great stuff, definitely looking forward to more in depth articles in this arena!saiku - Thursday, August 17, 2006 - link
This article kind of reminds me THG's recent series of articles on how computer graphics cards work.For us techies who don't get to peep into our server rooms much, this is a great intro. Especially for guys like me who work in small companies where all we have are some dusty Windows 2000 servers stuck in a small server "room".
Thanks for this cool info.
JohanAnandtech - Friday, August 18, 2006 - link
Thanks! Been in the same situation as you. Then I got a very small budget for upgrading our serverroom (about $20000) at the university I work for and I found out that there is quite a bit of information about servers but all fragmented, and mostly coming from non-independent sources.splines - Thursday, August 17, 2006 - link
Excellent work pointing out the benefits and drawbacks of Blades. They are mighty cool, but not this second coming of the server christ that IBM et al would have you believe.Good work all round. It looks to be a great primer for those new to the administration side of the business.
WackyDan - Thursday, August 17, 2006 - link
Having worked with blades quite a bit, I can tell you that they are quite a significant innovation.I'll disagree with the author of the article that there is no standard. Intel co-designed the IBM bladecenter and licensed it's manufacture to other OEMS. Together, IBM and Intel have/had over 50% share inthe blade space. THat share along with Intel's collaboration is by default considered the standard int he industry.
Blades, done properly, have huge advantages over their rack counterparts. ie; far less cables. In the IBM's the mid-plane replaces all the individual network and optical cables as the networking modules (copper and fibre) are internal and you can get several flavors... Plus I only need one cable drop to manage 14 servers....
And if you've never seen 14 blades in 7u of space fully redundant, your are missing out. As for VMware, I've seen it running on blades with the same advantages as it's rack mount peers... and FYI... Blades are still considered rack mount as well...No you are not going to have any 16/32 ways as of yet.... but still, Blades really could replace 80%+ of all traditional rack mount servers.
splines - Friday, August 18, 2006 - link
I don't disagree with you on any one point there. Our business is in the process of moving to multiple blade clusters and attached SANs for our excessively large fileservers.But I do think that virtualisation does provide a great stepping-stone for business not quite ready to clear out the racks and invest in a fairly expensive replacement. We can afford to make this change, but many cannot. Even though the likes of IBM are pushing for blades left right and centre I wouldn't discount the old racks quite yet.
And no, I've haven't had the opportunity to see such a 7U blade setup. Sounds like fun :)
yyrkoon - Friday, August 18, 2006 - link
Wouldnt you push a single system that can run into the tens of thousands, to possibly hundreds of thousands for a single blade ? I know i would ;)Mysoggy - Thursday, August 17, 2006 - link
I am pretty amazed that they did not mention the cost of power in the TCO section.The cost of powering a server in a datacenter can be even greater than the TCA over it's lifetime.
I love the people that say...oh a got a great deal on this dell server...it was $400 off of the list price. Then they eat through the savings in a few months with shoddy PSUs and hardware that consume more power.
JarredWalton - Thursday, August 17, 2006 - link
Page 3:"Facility management: the space it takes in your datacenter and the electricity it consumes"
Don't overhype power, though. There is no way even a $5,000 server is going to use more in power costs over its expected life. Let's just say that's 5 years for kicks. From http://www.anandtech.com/IT/showdoc.aspx?i=2772&am...">this page, the Dell Irwindale 3.6 GHz with 8GB of RAM maxed out at 374W. Let's say $0.10 per kWHr for electricity as a start:
24 * 374 = 8976 WHr/Day
8976 * 365.25 = 3278484 WHr/Year
3278484 * 5 = 16392420 WHr over 5 years
16392420 / 1000 = 16392.42 kWHr total
Cost for electricity (at full load, 24/7, for 5 years): $1639.24
Even if you double that (which is unreasonable in my experience, but maybe there are places that charge $0.20 per kWHr), you're still only at $3278.48. I'd actually guess that a lot of businesses pay less for energy, due to corporate discounts - can't say for sure, though.
Put another way, you need a $5000 server that uses 1140 Watts in order to potentially use $5000 of electricity in 5 years. (Or you need to pay $0.30 per kWHr.) There are servers that can use that much power, but they are far more likely to cost $100,000 or more than to cost anywhere near $5000. And of course, power demands with Woodcrest and other chips are lower than that Irwindale setup by a pretty significant amount. :)
Now if you're talking about a $400 discount to get an old Irwindale over a new Woodcrest or something, then the power costs can easily eat up thost savings. That's a bit different, though.