DokuWiki

User Tools

Site Tools

Trace:
cluster:91

Warning: Undefined array key -1 in /usr/share/dokuwiki/inc/html.php on line 1458

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
cluster:91 [2010/12/08 14:25]
hmeij 		cluster:91 [2011/01/07 15:49] (current)
hmeij
Line 6: Line 6:
 Grabbed the Linpack source and compiled against /opt/openmpi/1.4.2 ... using the Make.Linux_PII_CBLAS makefile.  Had to grab the ''atlas'' libraries from another host.  We changed $HOME and pointed to libmpi.so ($MPdir and $MPlib) and repointed $LAdir.  Then it compiled fine. Grabbed the Linpack source and compiled against /opt/openmpi/1.4.2 ... using the Make.Linux_PII_CBLAS makefile.  Had to grab the ''atlas'' libraries from another host.  We changed $HOME and pointed to libmpi.so ($MPdir and $MPlib) and repointed $LAdir.  Then it compiled fine.
  
-===== Runs =====+More about [[http://en.wikipedia.org/wiki/LINPACK|Linpack on wikipedia]]
  
-So based on what we did with the Dell burn in, follow this [[cluster:26|HPLinpack Runs]] link, some calculations:+===== HP ===== 
 + 
 +So based on what we did with the Dell burn in, follow this [[cluster:26|previous Linpack Runs]] link, some calculations:
  
   * N calculation: 32 nodes, 12 gb each is 384 gb total which yields 48 gb double precision (8 byte) elements … 48 gb is 48*1024*1024*1024 = 51,539,607,552 … take the square root of that and round 227,032 … 80% of that is 181,600   * N calculation: 32 nodes, 12 gb each is 384 gb total which yields 48 gb double precision (8 byte) elements … 48 gb is 48*1024*1024*1024 = 51,539,607,552 … take the square root of that and round 227,032 … 80% of that is 181,600
Line 44: Line 46:
 ===== Results ===== ===== Results =====
  
-And about the best results we found was with+And about the best results (1.5 teraflops) we found, was with
  
   * N = 191,600   * N = 191,600
Line 89: Line 91:
 **[[cluster:0|Back]]** **[[cluster:0|Back]]**
  
-===== Image =====+===== Hmm =====
  
 And that revealed a host with 10 gb memory instead of 12gb. And that revealed a host with 10 gb memory instead of 12gb.
Line 95: Line 97:
 <code> <code>
  
-[root@petaltail ~]# cat /opt/src/hmeij-tmp/foo  
 [root@greentail Linux_PII_CBLAS]# pdsh grep MemTotal /proc/meminfo  [root@greentail Linux_PII_CBLAS]# pdsh grep MemTotal /proc/meminfo 
 n10: MemTotal:     12290464 kB n10: MemTotal:     12290464 kB
Line 132: Line 133:
  
 </code> </code>
 +
 +===== Dell =====
 +
 +Since the cluster will be shut down December 28th we have an opportunity to run Linpack on the Dell cluster.
 +
 +  * ETHERNET
 +  * N calculation: 20 nodes, 4/8/16 gb mix for a total of 192 gb which yields 24 gb double precision (8 byte) elements … 24 gb is 24*1024*1024*1024 = 25,769,803,776 … take the square root of that and round 160529 … 80% of that is 128,423
 +  * NB: start with 64, then 128, try 192 ...
 +  * PxQ: perfect square of 10x16=160, the number of cores we have
 +
 +<code>
 +============================================================================
 +T/V                N    NB     P     Q               Time             Gflops
 +----------------------------------------------------------------------------
 +WR00L2L2       40800   128    10    16             184.42          2.455e+02
 +----------------------------------------------------------------------------
 +||Ax-b||_oo / ( eps * ||A||_1  * N        ) =        0.0069974 ...... PASSED
 +||Ax-b||_oo / ( eps * ||A||_1  * ||x||_1  ) =        0.0105682 ...... PASSED
 +||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo ) =        0.0020883 ...... PASSED
 +============================================================================
 +</code>
 +
 +
 +  * INFINIBAND
 +  * N calculation: 16 nodes, 8 gb per node for a total of 128 gb which yields 16 gb double precision (8 byte) elements … 16 gb is 16*1024*1024*1024 = 17,179,869,184 … take the square root of that and round 131,072 … 80% of that is 104,850
 +  * NB: start with 64, then 128, try 192 ...
 +  * PxQ: perfect square of 10x16=160, the number of cores we have
 +
 +<code>
 +============================================================================
 +T/V                N    NB     P     Q               Time             Gflops
 +----------------------------------------------------------------------------
 +WR00L2L2       52425    64    11    11             294.28          3.264e+02
 +----------------------------------------------------------------------------
 +||Ax-b||_oo / ( eps * ||A||_1  * N        ) =        0.0059082 ...... PASSED
 +||Ax-b||_oo / ( eps * ||A||_1  * ||x||_1  ) =        0.0133907 ...... PASSED
 +||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo ) =        0.0024480 ...... PASSED
 +============================================================================
 +</code>
 +
 +So a total of 2.455e+02 + 3.264e+0 or about 572 Gflops, 0.5 teraflops.
 +
 +
 +===== BSS =====
 +
 +
 +Since the cluster will be shut down December 28th we have an opportunity to run Linpack on the sharptail cluster.
 +
 +  * N calculation: 46 nodes, 24 gb per node for a total of 1,104 gb which yields 138 gb double precision (8 byte) elements … 138 gb is 138*1024*1024*1024 = 148,176,371,712 … take the square root of that and round 384936 … 80% of that is 307,950
 +  * NB: start with 64, then 128, try 192 ...
 +  * PxQ: perfect square of 9x10=92, close to the number of cores we have.
 +
 +
 +Hmm, unable to make this work across all the nodes at the same time.  Not sure why.  My estimates are that with 92 cores and 1,126 gb of memory this cluster should be able to do 500-700 Gflops.
 +
  
 \\ \\
 **[[cluster:0|Back]]** **[[cluster:0|Back]]**
  
cluster/91.1291836343.txt.gz · Last modified: 2010/12/08 14:25 by hmeij

Page Tools

Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
CC Attribution-Share Alike 4.0 International Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki