User Tools

Site Tools


cluster:176

Warning: Undefined array key -1 in /usr/share/dokuwiki/inc/html.php on line 1458

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
cluster:176 [2018/11/20 13:57]
hmeij07 [HPC Power]
cluster:176 [2019/03/06 14:29] (current)
hmeij07 [Metered]
Line 7: Line 7:
 As part of our reevaluation of our data center cooling capacity and overhaul, we need to get a handle on non-emergency power consumption in data center. This will be done by a third party consultant by clamping power cables in the penthouse of Excley. So I bought myself a metered PDU and have been busy plugging entire racks into it one at a time. I then measure "idle" and "peak" amperage consumption. The end picture should reveal how much Kw the HPC consumes of all non-emergency power (that is not on enterprise UPS). And we'll figure out how much Physical Plant spends supporting the HPC. Thanks PP! As part of our reevaluation of our data center cooling capacity and overhaul, we need to get a handle on non-emergency power consumption in data center. This will be done by a third party consultant by clamping power cables in the penthouse of Excley. So I bought myself a metered PDU and have been busy plugging entire racks into it one at a time. I then measure "idle" and "peak" amperage consumption. The end picture should reveal how much Kw the HPC consumes of all non-emergency power (that is not on enterprise UPS). And we'll figure out how much Physical Plant spends supporting the HPC. Thanks PP!
  
-  * "Idle" means not very busy, most of the time something is happening (ie backups)+  * "Idle" means not very busy, most of the time something is happening anyway (ie backups)
   * "Peak" means very busy, most queues entirely full.   * "Peak" means very busy, most queues entirely full.
   * Metered Amps * 208V = Watts   * Metered Amps * 208V = Watts
   * Watts * (24 * 365)/1000 = KwH/Y   * Watts * (24 * 365)/1000 = KwH/Y
-  * kwH/Y * $0.12 = $/Y (mix cogen and utility, 2013 assumption by Pete of PP)+  * kwH/Y * $0.125 = $/Y (mix cogen and utility, 2013 assumption by Pete of PP)
   * Double the $/Y to include cooling costs (Pete of PP assumption)   * Double the $/Y to include cooling costs (Pete of PP assumption)
  
-^  HPC  ^^  Idle ^  Peak  ^  Idle  ^  Peak  ^  Idle  ^  Peak  ^  Idle  ^  Peak  ^ +^  HPC  ^^  Idle ^  Peak  ^  Idle  ^  Peak  ^  Idle  ^  Peak  ^  Idle  ^  Peak  ^    
-^  Rack  ^  L6-30P  ^  Amps ^  Amps  ^  Watts  ^  Watts  ^  KwH/Y  ^  KwH/Y  ^  $/Y  ^  $/Y  ^ +^  Rack  ^  L6-30P  ^  Amps ^  Amps  ^  Watts  ^  Watts  ^  KwH/Y  ^  KwH/Y  ^  $/Y  ^  $/Y   Note  ^ 
-|  R4R04  |  3  |  22  |  37  |  4,576  |  7,696  |  40,085  |  67,416  |  5,010  |  8,427  | +|  R4R04  |  3  |  22  |  37  |  4,576  |  7,696  |  40,085  |  67,416  |  5,010  |  8,427   "sharptail rack"  | 
-|  R4R05  |  4  |  17  |  34  |  3,536  |  7,072  |   30,975  |  61,950  |  3,872  |  7,743  | +|  R4R05  |  4  |  17  |  34  |  3,536  |  7,072  |   30,975  |  61,950  |  3,872  |  7,743   "greentail rack"  | 
-|  R4R06  |  2  |  6  |    |  1,204  |    |  8,970  |    |  1,121  |    | +|  R4R06  |  2  |  6  |    |  1,204  |    |  8,970  |    |  1,121  |    |  empty  
-|  R4R07  |  2  |  14  |  34  |  2,912  |  7,072  |  25,509  |  61,950  |  3,189  |  7,744  | +|  R4R07  |  2  |  14  |  34  |  2,912  |  7,072  |  25,509  |  61,950  |  3,189  |  7,744   "mw128/tinymem"  | 
-|  R4R08  |  2  |  4  |    |  832  |    |  7,288  |    |  911  |    | +|  R4R08  |  2  |  4  |    |  832  |    |  7,288  |    |  911  |    |  mostly empty  
-|  Total  |  13  |  63  |  105  |  13,060  |  21,840  |  114,405  |  191,318  |  14,300  |  23,915  |+|  Total  |  13  |  63  |  105  |  13,060  |  21,840  |  114,405  |  191,318  |  14,300  |  23,915    | 
 + 
 +In 2013 we found for our Dell Racks purchased in 2006 (full of PE1950 and PE2950 servers) [[cluster:123|Replace Dell Racks]] 
 + 
 +  * 115,000 KwH/Y per rack for power 
 +  * 14,375 $/Y per rack for power 
 + 
 +So a great improvement in energy efficiency. We probably grew from near 5 teraflops to near 60 teraflops in this time period (double precision, floating point, theoretical).
  
-  * in 2013 we found for Dell Racks: 
  
  
Line 71: Line 77:
     * mw128 10 nodes, idle = 4 amps, for all nodes 8 amps     * mw128 10 nodes, idle = 4 amps, for all nodes 8 amps
     * mw128 10 nodes, peak = 8 amps, for all nodes 16 amps     * mw128 10 nodes, peak = 8 amps, for all nodes 16 amps
 +      * 11/28/2018 17 amps observed (with 6/18 nodes at load 36)
 +      * 03/05/2019 23 amps observed (...wengang jobs, mw128 nodes only! 23+16=39of60 possible)
     * two switches = 2 amps     * two switches = 2 amps
     * tinymem 7 nodes, idle = 2 amps, for all nodes 4 amps     * tinymem 7 nodes, idle = 2 amps, for all nodes 4 amps
cluster/176.1542740230.txt.gz · Last modified: 2018/11/20 13:57 by hmeij07