Differences

This shows you the differences between two versions of the page.

--- cluster:112 [2013/04/17 14:12]
hmeij [Dell Jan-2013]
+++ cluster:112 [2013/10/02 15:42]
hmeij [Dell April-2013]
@@ Line 19: / Line 19: @@
 |  Cooling (Btus((per hour, 1,000 Btus=293 Watts per hour)))|  |  ((2009 Wesleyan estimate))39,240|  ((vendor published))36,175|  ((vendor published))20,171|  ((vendor published))17,913|  |
 |  Annual Costs((power and cooling)) ($)|  |  46,091|  42,930|  25,864|  23,506|  ((for Wesleyan/vendors data))using 2009 data|
+|  $1,000/Tflop|  |  69.8|  28.6|  1.3|  6.5|  ((for Wesleyan/vendors data))using 2009 data|
+  * 2009 Dell measurements put power consumption at 100,740 KwH per year (how?)
+  * 2013 Dell measurements (see bottom table) put power consumption at 109,956 KwH per year (for 30 servers)
 ====== Details ======
@@ Line 89: / Line 93: @@
 ** This is what the picture would look like using the 2009 spreadsheet data**
-  * 240 cores, make them all ethernet based
+What we should look at is teraflops replacement ...
-    * slight change but doable
-  * 1U yields 16 cores, 15U yields 240 cores with a total of 1,920 GB memory
-    * major change, Dell offers 388 GB
-  * acquisition costs
-    * 15 x $5,782 = $86,730 + PDUs, ethernet switch, cables
-    * power/AC costs: $23,506/year x3 years = $70,518
-  * add both costs of new hardware and divide by dell total annual costs power+cooling
-    * ROI: 3.4 years - but this is on a core by core basis
-What we should look at is teraflops...
-  * 0.66 teraflops, measured with Linpack
+  * 0.66 teraflops, measured with Linpack (actual)
-  * E5-2660 is rated at 140 Gflops so one node provides 280 Gflops (theoretically)
+  * E5-2660 is rated at 140 Gflops so one node provides 280 Gflops (theoretical)
   * 2 and 1/3rd 1U nodes then replace the cluster
   * acquisition costs
@@ Line 111: / Line 105: @@
     * ROI: 3/4 year - but this is on a teraflop by teraflop basis
-And if we double the teraflops...1.6 Tflops (actually +1Tflops), 1.5 years ROI for 96 cores.
+And if we double the teraflops...1.6 Tflops (actually + 1 Tflops), 1.5 years ROI for 96 cores.
@@ Line 129: / Line 123: @@
-===== Dell Jan-2013 =====
+====== Dell April-2013 ======
-** Actual data obtained 11 April - 18 April, 2013 **
+** Actual data obtained April - May, 2013 **
 Kill-A-Watt Meter Stats; pulled one power supply ... mean/1000 * 24 * 365 * 30!
-^  Node^  threads^  kwh^  time hh:mm^  mean^  comment^  annual kwh^ power annual^ +cooling annual (x2)^^
+^  Node^  jobs^  kwh^  time hh:mm^  mean^  comment^  annual kwh^ power annual^ +cooling annual (x2)^  ^
 |  c04|  8|  2.29|  06:30|  352|  idle, no jobs|   92,509|  $21,393|  $42,786|  wow|
 |  c32|  8|  6.94|  19:35|  356|  jobs finished|   93,556|  $21,610|  $43,220|  |
 |  c27|  8|  7.42|  19:30|  381|  jobs finished|  100,127|  $22,971|  $45,942|  |
-|  c10|  8| 10.21|  23.55|  425|  still running|  111,690|  $25,367|  $50,734| *|
 |  c06|  8| 27.46|  65:11|  395|      4 running|  103,806|  $23,734|  $47,468|  |
 |  c00|  8|  8.26|  21:00|  393|  still running|  103,280|  $23,625|  $47,250| *|
 |  c03|  8| 10.36|  24:03|  432|  still running|  113,530|  $25,748|  $51,496| *|
+|  c04|  8| 12.02|  28:08|  429|  still running|  112,741|  $25,585|  $51,170| *|
+|  c09|  8| 10.36|  24:00|  432|  still running|  113,530|  $25,748|  $51,496| *|
+|  c10|  8| 10.21|  23:55|  425|  still running|  111,690|  $25,367|  $50,734| *|
+|  c17|  8| 27.07|  71:00|  381|  still running|  |  |  | *|
+|  c18|  8| 11.23|  24:41|  455|  still running|  |  |  | *|
+|  c20|  8| 50.23|   143:|  351|  still running|  |  |  | *|
+|  c23|  8|  8.71|  24:00|  363|  still running|  |  |  | *|
+|  c25|  8|  9.80|  25:02|  392|  still running|  |  |  | *|
+|  c29|  8| 30.22|  66:22|  455|  still running|  119,574|  $27,001|  $54,002| *|
+|  c31|  8| 10.30|  25:10|  409|  still running|  107,485|  $24,496|  $48,992| *|
+|  c32|  8| 11.42|  24:00|  476|  still running|  123,379|  $27,789|  $55,578| *|
+|  c33|  8| 46.23|   113:|  409|  still running|  |  |  | *|
+|  c35|  8| 13.69|  28:53|  474|  still running|  |  |  | *|
+  * Average for "still running" nodes is 418.4 watts or 109,956 KwH/year
+  * (watts/1000 Kw per hour) * 24 hours * 365 days * 30 servers
+  * In 2009 dollars that is $25,000 in power (plus that in cooling) or $50,000 per year
+  * What is that in 2013 dollars (with cogen online in Pine Street)?
+We have measured 19 nodes power consumption (pulling one unit out) with a Kill-A-Watt meter over 775+ hours to arrive at a mean consumption rate of 418.4 watts. That totals to 109,956 KwH/year in power consumption ((watts/1000 Kw per hour) * 24 hours * 365 days * 30 servers), which is a low water mark as the racks in question also contain switches, two UPS units, and a disk array. (Note: Peter did a side calculation using  the 4 watt measurement on panel directly and came up with 126,000 KwH/year which can be considered the high water mark estimate).
+Based on 12.5 cents per Kwh (this is an all inclusive cost including natural gas cost, heat recoup costs, distribution, maintenance etc) the hardware burns away $13,744.50 per year. Best guess is cooling costs are at least that (another possible low water mark) so the total cost for all power consumption is $27,489 per year. If we run the hardware for another 3 years that total cost is $82,467.
+If we could replace, or approximate, the 30 compute nodes' computational power (0.6 teraflops) and job slots (240 cores) with new hardware that consumes 50% less in power, our ROI is 6 years based on the low water mark numbers.

DokuWiki

User Tools

Site Tools

Differences

Page Tools