This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
cluster:123 [2013/10/10 22:54] hmeij |
cluster:123 [2013/10/18 13:52] hmeij |
||
---|---|---|---|
Line 1: | Line 1: | ||
\\ | \\ | ||
**[[cluster: | **[[cluster: | ||
- | |||
- | To be send to Dave Baird. | ||
==== Replace Dell Racks ==== | ==== Replace Dell Racks ==== | ||
Line 19: | Line 17: | ||
Next step was to collect vendor quotes for a target budget of $82K, 3 years of Dell energy consumption, | Next step was to collect vendor quotes for a target budget of $82K, 3 years of Dell energy consumption, | ||
- | Old hardware: | + | Old hardware: |
- | 30 nodes, 2.66 ghz, 4 mb L-cache (for cpu), 240 cores (job slots), | + | 30 nodes, 2.66 ghz, 4 mb L-cache (for cpu), 240 cores (job slots),\\ |
80 gb local drive, 340 gb total ram, 12,555 watts (power no cooling), 670 gigaflops (actual measure) | 80 gb local drive, 340 gb total ram, 12,555 watts (power no cooling), 670 gigaflops (actual measure) | ||
- | New hardware: | + | New hardware |
- | 14 nodes, 2.60 ghz, 20 mb L-cache (for cpu), 224 cores (job slots), | + | 14 nodes, 2.60 ghz, 20 mb L-cache (for cpu), 224 cores (job slots),\\ |
1TB local drive, 1,792 gb total ram, 5,400 watts (power no cooling), 4,659 gigaflops (theoretical) | 1TB local drive, 1,792 gb total ram, 5,400 watts (power no cooling), 4,659 gigaflops (theoretical) | ||
- | |||
- | In the representative example for new hardware the total energy consumption would be 10,800 watts. | ||
- | The total cost of running the new hardware would be $5,913 per year. That would imply savings of $21,576 per year. The job slot count would be 112 but with hyperthreading technology that can be doubled. We'd still want the 1,792 memory footprint (8 gb/core) and the gigaflops (2,329) still far exceeds Dell's performance. | + | New hardware v2 (half of v1): 23,652 KwH/year for cooling or 22% of Old hardware\\ |
+ | 7 nodes, 2.60 ghz, 20 mb L-cache (for cpu), 112 cores (job slots),\\ | ||
+ | 1TB local drive, 1,792 gb total ram, 2,700 watts (power no cooling), 2,329 gigaflops (theoretical) | ||
+ | |||
+ | If we reduced the node count to 7 (the minimum configuration to meet the job slot count of the Dell hardware), the total energy consumption (power plus cooling) would be 5,400 watts. | ||
In two years, the new hardware would have saved $43,152 on energy costs based on the low water mark (Dell' | In two years, the new hardware would have saved $43,152 on energy costs based on the low water mark (Dell' | ||
Line 35: | Line 35: | ||
* There are enough Infiniband ports available to put all new hardware nodes on such a switch (add cards and cables cost for each node) | * There are enough Infiniband ports available to put all new hardware nodes on such a switch (add cards and cables cost for each node) | ||
* The internal disks on each node need to be of a high speed (10K or better) and of a certain size (300 GB or larger) mimicking the Dell disk arrays (adds costs) | * The internal disks on each node need to be of a high speed (10K or better) and of a certain size (300 GB or larger) mimicking the Dell disk arrays (adds costs) | ||
- | * we maybe able to add two more nodes by switching to a more exapnsive | + | * we maybe able to add two more nodes by switching to a more exapansive |
+ | * accomplished by switching from 8 core 2650v2 (130 watt) 2.6 ghz CPU to 10 core 2660v2 (95 watt) 2.2 ghz CPU | ||
But it is all very doable within a budget of $45-$50K. And it can be the solution for: | But it is all very doable within a budget of $45-$50K. And it can be the solution for: | ||
Line 45: | Line 46: | ||
The Libert family rejoices. | The Libert family rejoices. | ||
+ | |||
+ | ==== Update ==== | ||
+ | |||
+ | The table below contains data for a cluster whose nodes are all on the Infiniband switch (and also ethernet switch for provision and data). | ||
+ | |||
+ | |||
+ | ^Tnodes^Tcores^THcores^Tmem gb^Watts^TEnergy^TEnergy $/ | ||
+ | |10|160|320|2, | ||
+ | |||
\\ | \\ | ||
**[[cluster: | **[[cluster: |