DokuWiki

User Tools

Site Tools

Trace:
cluster:112

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
Next revision Both sides next revision
cluster:112 [2013/04/16 14:07]
hmeij [Dell Jan-2013] 		cluster:112 [2013/05/14 14:39]
hmeij [Overview]
Line 19: Line 19:
 |  Cooling (Btus((per hour, 1,000 Btus=293 Watts per hour)))|  |  ((2009 Wesleyan estimate))39,240|  ((vendor published))36,175|  ((vendor published))20,171|  ((vendor published))17,913|  | |  Cooling (Btus((per hour, 1,000 Btus=293 Watts per hour)))|  |  ((2009 Wesleyan estimate))39,240|  ((vendor published))36,175|  ((vendor published))20,171|  ((vendor published))17,913|  |
 |  Annual Costs((power and cooling)) ($)|  |  46,091|  42,930|  25,864|  23,506|  ((for Wesleyan/vendors data))using 2009 data| |  Annual Costs((power and cooling)) ($)|  |  46,091|  42,930|  25,864|  23,506|  ((for Wesleyan/vendors data))using 2009 data|
 +|  $1,000/Tflop|  |  69.8|  28.6|  1.3|  6.5|  ((for Wesleyan/vendors data))using 2009 data|
  
 +
 +  * 2009 Dell measurements put power consumption at 100,740 KwH per year (how?)
 +  * 2013 Dell measurements (see bottom table) put power consumption at 109,956 KwH per year (for 30 servers)
  
 ====== Details ====== ====== Details ======
Line 89: Line 93:
 ** This is what the picture would look like using the 2009 spreadsheet data** ** This is what the picture would look like using the 2009 spreadsheet data**
  
-  * 240 cores, make them all ethernet based  +What we should look at is teraflops replacement ...
-    * slight change but doable +
-  * 1U yields 16 cores, 15U yields 240 cores with a total of 1,920 GB memory +
-    * major change, Dell offers 388 GB +
-  * acquisition costs +
-    * 15 x $5,782 = $86,730 + PDUs, ethernet switch, cables +
-    * power/AC costs: $23,506/year x3 years = $70,518 +
-  * add both costs of new hardware and divide by dell total annual costs power+cooling +
-    * ROI: 3.4 years - but this is on a core by core basis +
- +
-What we should look at is teraflops...+
  
-  * 0.66 teraflops, measured with Linpack +  * 0.66 teraflops, measured with Linpack (actual) 
-  * E5-2660 is rated at 140 Gflops so one node provides 280 Gflops (theoretically)+  * E5-2660 is rated at 140 Gflops so one node provides 280 Gflops (theoretical)
   * 2 and 1/3rd 1U nodes then replace the cluster   * 2 and 1/3rd 1U nodes then replace the cluster
   * acquisition costs   * acquisition costs
Line 111: Line 105:
     * ROI: 3/4 year - but this is on a teraflop by teraflop basis     * ROI: 3/4 year - but this is on a teraflop by teraflop basis
  
-And if we double the teraflops...1.6 Tflops (actually +1Tflops), 1.5 years ROI for 96 cores.+And if we double the teraflops...1.6 Tflops (actually + 1 Tflops), 1.5 years ROI for 96 cores.
  
  
Line 129: Line 123:
  
  
-===== Dell Jan-2013 =====+====== Dell April-2013 ======
  
-** Actual data obtained 11 April - 18 April, 2013 **+** Actual data obtained April - May, 2013 **
  
-Kill-A-Watt Meter Stats; pulled one power supply ... mean/1000 * 24 * 365 * 30!+Kill-A-Watt Meter Stats; pulled one power supply ... mean/1000 * 24 * 365 * 30! 
  
-^  Node^  threads^  kwh^  time hh:mm^  mean^  comment^  annual kwh^ power annual^ +cooling annual (x2)^^ +^  Node^  jobs^  kwh^  time hh:mm^  mean^  comment^  annual kwh^ power annual^ +cooling annual (x2)^  ^ 
-|  c04|  8|  2.29|  06:30|  352|  idle, no jobs|   92,509|  $21,393|  $42,786|  |+|  c04|  8|  2.29|  06:30|  352|  idle, no jobs|   92,509|  $21,393|  $42,786|  wow|
 |  c32|  8|  6.94|  19:35|  356|  jobs finished|   93,556|  $21,610|  $43,220|  | |  c32|  8|  6.94|  19:35|  356|  jobs finished|   93,556|  $21,610|  $43,220|  |
 |  c27|  8|  7.42|  19:30|  381|  jobs finished|  100,127|  $22,971|  $45,942|  | |  c27|  8|  7.42|  19:30|  381|  jobs finished|  100,127|  $22,971|  $45,942|  |
-|  c10|  8| 10.21|  23.55|  425|  still running|  111,690|  $25,367|  $50,734|  | +|  c06|  8| 27.46|  65:11|  395|      4 running|  103,806|  $23,734|  $47,468|  | 
-|  c06|  8| 27.46|  65:11|  395|      4 running|  103,806|  $23,734|  $47,468|best number((Very close to 2009 estimate of  $46,091))| +|  c00|  8|  8.26|  21:00|  393|  still running|  103,280|  $23,625|  $47,250| *| 
-|  c03|  8|  8.26|  21:00|  393|  still running|  103,280|  $23,625|  $47,250|+|  c03|  8| 10.36|  24:03|  432|  still running|  113,530|  $25,748|  $51,496| *| 
 +|  c04|  8| 12.02|  28:08|  429|  still running|  112,741|  $25,585|  $51,170| *| 
 +|  c09|  8| 10.36|  24:00|  432|  still running|  113,530|  $25,748|  $51,496| *| 
 +|  c10|  8| 10.21|  23:55|  425|  still running|  111,690|  $25,367|  $50,734| *| 
 +|  c17|  8| 27.07|  71:00|  381|  still running|  |  |  | *| 
 +|  c18|  8| 11.23|  24:41|  455|  still running|  |  |  | *|  
 +|  c20|  8| 50.23|   143:|  351|  still running|  |  |  | *| 
 +|  c23|  8|  8.71|  24:00|  363|  still running|  |  |  | *| 
 +|  c25|  8|  9.80|  25:02|  392|  still running|  |  |  | *| 
 +|  c29|  8| 30.22|  66:22|  455|  still running|  119,574|  $27,001|  $54,002| *| 
 +|  c31|  8| 10.30|  25:10|  409|  still running|  107,485|  $24,496|  $48,992| *| 
 +|  c32|  8| 11.42|  24:00|  476|  still running|  123,379|  $27,789|  $55,578| *| 
 +|  c33|  8| 46.23|   113:|  409|  still running|  |  |  | *| 
 +|  c35|  8| 13.69|  28:53|  474|  still running|  |  |  | *| 
 + 
 +  * Average for "still running" nodes is 418.4 watts or 109,956 KwH/year (watts/1000)*24*365*30 servers 
 +  * In 2009 dollars that is $25,000 in power (plus that in cooling) or $50,000 per year 
 +  * What is that in 2013 dollars (with cogen online in Pine Street)? 
 +  *  
 + 
 + 
  
  
cluster/112.txt · Last modified: 2013/10/08 19:04 by hmeij

Page Tools

Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
CC Attribution-Share Alike 4.0 International Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki