cluster:149
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
cluster:149 [2016/08/30 14:33] hmeij07 [Supermicro] |
cluster:149 [2016/12/06 20:13] (current) hmeij07 [Supermicro] |
||
|---|---|---|---|
| Line 6: | Line 6: | ||
| - | ==== The problem | + | ==== The Storage Problem |
| In a commodity HPC setup deploying plain NFS, bottle necks can develop. | In a commodity HPC setup deploying plain NFS, bottle necks can develop. | ||
| Line 98: | Line 98: | ||
| Elegant, simple, and you can start smaller and scale up. We have room for 2 on the QDR Mellanox switch (and 2 up to 5 on the DDR Voltaire switch). Buying another QDR Mellanox adds $7K for an 18 port switch. IPoIB would be desired if we stay with Supermicro. | Elegant, simple, and you can start smaller and scale up. We have room for 2 on the QDR Mellanox switch (and 2 up to 5 on the DDR Voltaire switch). Buying another QDR Mellanox adds $7K for an 18 port switch. IPoIB would be desired if we stay with Supermicro. | ||
| - | What's even more desired is to start our own parallel file system with BeeGFS | + | What's even more desired is to start our own parallel file system with [[http://www.beegfs.com/ |
| + | |||
| + | **Short term plan** | ||
| + | |||
| + | * Grab the 32x2T flexstorage hard drives and insert into cottontail' | ||
| + | * Makes for a 60T raw raid 6 storage place (2 hot spares) | ||
| + | * move the sharptail /snapshots to it (remove that traffic from file server) | ||
| + | * Dedicate greentail' | ||
| + | * Remove / | ||
| + | * Extend /sanscratch form 27T to 37T | ||
| + | * Dedicate sharptail' | ||
| + | * Keep old 5T /sanscratch as backup, idle | ||
| + | * Remove 15T / | ||
| + | * Extend /home for 10T to 25T | ||
| + | * Keep 7T /archives until those users graduate, move to Rstore | ||
| + | |||
| + | **Long term plan** | ||
| + | * Start a BeeGFS storage cluster | ||
| + | * cottontail as MS (management server) | ||
| + | * sharptail as AdMon (monitor server) and proof of concept storage OSS | ||
| + | * pilot storage on idle / | ||
| + | * also a folder on cottonttail:/ | ||
| + | * n38-n45 (8) as MDS (metadata servers, 15K local disk, no raid) | ||
| + | * Buy 2x 2U Supermicro for OSS (object storage servers for a total of 80T usable, raid 6, $12.5K) | ||
| + | * Serve up BeeGFS file system using IPoIB | ||
| + | * Move /home to it | ||
| + | * Backup to older disk arrays | ||
| + | * Expand as necessary | ||
| ===== Loose Ends ===== | ===== Loose Ends ===== | ||
| Line 106: | Line 133: | ||
| * backup Openlava scheduler for cottontail.wesleyan.edu | * backup Openlava scheduler for cottontail.wesleyan.edu | ||
| * backup replacement for greentail.wesleyan.edu (in case it fails) | * backup replacement for greentail.wesleyan.edu (in case it fails) | ||
| + | |||
| + | Bought. | ||
| + | --- // | ||
| Warewulf golden image it as if it is greentail. | Warewulf golden image it as if it is greentail. | ||
cluster/149.1472567629.txt.gz · Last modified: 2016/08/30 14:33 by hmeij07
Page Tools
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
