This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
cluster:112 [2013/04/17 14:12] hmeij [Dell Jan-2013] |
cluster:112 [2013/10/08 17:57] hmeij [Quotes Oct-2013] |
||
---|---|---|---|
Line 19: | Line 19: | ||
| Cooling (Btus((per hour, 1,000 Btus=293 Watts per hour)))| | | Cooling (Btus((per hour, 1,000 Btus=293 Watts per hour)))| | ||
| Annual Costs((power and cooling)) ($)| | 46, | | Annual Costs((power and cooling)) ($)| | 46, | ||
+ | | $1, | ||
+ | |||
+ | * 2009 Dell measurements put power consumption at 100,740 KwH per year (how?) | ||
+ | * 2013 Dell measurements (see bottom table) put power consumption at 109,956 KwH per year (for 30 servers) | ||
====== Details ====== | ====== Details ====== | ||
Line 89: | Line 93: | ||
** This is what the picture would look like using the 2009 spreadsheet data** | ** This is what the picture would look like using the 2009 spreadsheet data** | ||
- | * 240 cores, make them all ethernet based | + | What we should look at is teraflops |
- | * slight change but doable | + | |
- | * 1U yields 16 cores, 15U yields 240 cores with a total of 1,920 GB memory | + | |
- | * major change, Dell offers 388 GB | + | |
- | * acquisition costs | + | |
- | * 15 x $5,782 = $86,730 + PDUs, ethernet switch, cables | + | |
- | * power/AC costs: $23, | + | |
- | * add both costs of new hardware and divide by dell total annual costs power+cooling | + | |
- | * ROI: 3.4 years - but this is on a core by core basis | + | |
- | + | ||
- | What we should look at is teraflops... | + | |
- | * 0.66 teraflops, measured with Linpack | + | * 0.66 teraflops, measured with Linpack |
- | * E5-2660 is rated at 140 Gflops so one node provides 280 Gflops (theoretically) | + | * E5-2660 is rated at 140 Gflops so one node provides 280 Gflops (theoretical) |
* 2 and 1/3rd 1U nodes then replace the cluster | * 2 and 1/3rd 1U nodes then replace the cluster | ||
* acquisition costs | * acquisition costs | ||
Line 111: | Line 105: | ||
* ROI: 3/4 year - but this is on a teraflop by teraflop basis | * ROI: 3/4 year - but this is on a teraflop by teraflop basis | ||
- | And if we double the teraflops...1.6 Tflops (actually +1Tflops), 1.5 years ROI for 96 cores. | + | And if we double the teraflops...1.6 Tflops (actually + 1 Tflops), 1.5 years ROI for 96 cores. |
Line 129: | Line 123: | ||
- | ===== Dell Jan-2013 ===== | + | ====== Dell April-2013 ====== |
- | ** Actual data obtained | + | ** Actual data obtained April - May, 2013 ** |
- | Kill-A-Watt Meter Stats; pulled one power supply ... mean/1000 * 24 * 365 * 30! | + | Kill-A-Watt Meter Stats; pulled one power supply ... mean/1000 * 24 * 365 * 30! |
- | ^ Node^ | + | ^ Node^ |
| c04| 8| 2.29| 06: | | c04| 8| 2.29| 06: | ||
| c32| 8| 6.94| 19: | | c32| 8| 6.94| 19: | ||
| c27| 8| 7.42| 19: | | c27| 8| 7.42| 19: | ||
- | | c10| 8| 10.21| | ||
| c06| 8| 27.46| | | c06| 8| 27.46| | ||
| c00| 8| 8.26| 21: | | c00| 8| 8.26| 21: | ||
| c03| 8| 10.36| | | c03| 8| 10.36| | ||
+ | | c04| 8| 12.02| | ||
+ | | c09| 8| 10.36| | ||
+ | | c10| 8| 10.21| | ||
+ | | c17| 8| 27.07| | ||
+ | | c18| 8| 11.23| | ||
+ | | c20| 8| 50.23| | ||
+ | | c23| 8| 8.71| 24: | ||
+ | | c25| 8| 9.80| 25: | ||
+ | | c29| 8| 30.22| | ||
+ | | c31| 8| 10.30| | ||
+ | | c32| 8| 11.42| | ||
+ | | c33| 8| 46.23| | ||
+ | | c35| 8| 13.69| | ||
+ | |||
+ | * Average for "still running" | ||
+ | * (watts/1000 Kw per hour) * 24 hours * 365 days * 30 servers | ||
+ | * In 2009 dollars that is $25,000 in power (plus that in cooling) or $50,000 per year | ||
+ | * What is that in 2013 dollars (with cogen online in Pine Street)? | ||
+ | |||
+ | We have measured 19 nodes power consumption (pulling one unit out) with a Kill-A-Watt meter over 775+ hours to arrive at a mean consumption rate of 418.4 watts. That totals to 109,956 KwH/year in power consumption ((watts/ | ||
+ | |||
+ | |||
+ | |||
+ | Based on 12.5 cents per Kwh (this is an all inclusive cost including natural gas cost, heat recoup costs, distribution, | ||
+ | |||
+ | |||
+ | |||
+ | If we could replace, or approximate, | ||
+ | |||
+ | ====== Quotes Oct-2013 ====== | ||
+ | |||
+ | For $82K one could get CPU-HPC wise ... | ||
+ | |||
+ | *[[hhttp:// | ||
+ | * E5-2650v2 " | ||
+ | * E5-2680v2 172.8 gflops | ||
+ | |||
+ | ^vendor^Tnodes^chip^ghz^cache^cores^Tcores^disk^ram^ram/ | ||
+ | |dell|30|X5355|2.66|4mb|8|240|80gb|4-16gb|0.5-2gb|340|Y|Y|N|12, | ||
+ | |micro|14|2650v2|2.60|20mb|16|224|1tb|128gb|8gb|1, | ||
+ | |advcl|10|2680v2|2.80|25mb|20|200|60gb|256gb|12.8gb|2, | ||
+ | |exxct|12|2660v2|2.20|25mb|20|240|1tb|128gb|6.4gb|1, | ||
+ | |hp| | | | | | | | | | | | | | | | | ||