This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
cluster:151 [2016/10/27 19:20] hmeij07 [Tuning] |
cluster:151 [2016/10/28 17:41] hmeij07 [/mnt/beegfs/] |
||
---|---|---|---|
Line 10: | Line 10: | ||
This page is best read from the bottom up. | This page is best read from the bottom up. | ||
+ | ==== cluster idea ==== | ||
+ | |||
+ | idea: buy 2 now 4k+4k then 3rd in july 4k? | ||
+ | move test users over on 2 nodes, test, only change is $HOME | ||
+ | ctt (mngt+admingiu), | ||
+ | |||
+ | make ctt2 master meta node? how? | ||
+ | |||
+ | |||
+ | ==== / | ||
+ | |||
+ | * Source content 110G in XFS with ~100,000 files in ~2,000 dirs | ||
+ | * /home/hmeij (mix of files, nothing large) plus | ||
+ | * / | ||
+ | | ||
+ | * File content spread across 2 storage servers | ||
+ | * petaltail:/ | ||
+ | * swallowtail:/ | ||
+ | * 56G used in beegfs-storage per storage server | ||
+ | * ~92,400 files per storage server | ||
+ | * ~1,400 dirs per storage server | ||
+ | |||
+ | * Meta content spread across 2 meta servers (n37 and n38) | ||
+ | * 338MB per beegfs-meta server so 0.006% space wise for 2 servers | ||
+ | * ~105,000 files per metadata server | ||
+ | * ~35,000 dirs almost spread evenly across " | ||
+ | |||
+ | * Client (n7 and n8) see 110G in /mnt/beegfs | ||
+ | * 110G in /mnt/beegfs | ||
+ | * ~100,000 files | ||
+ | * ~2,000 dirs | ||
+ | |||
+ | Looks like: | ||
+ | |||
+ | < | ||
+ | |||
+ | # file content | ||
+ | |||
+ | [root@swallowtail ~]# ls -lR / | ||
+ | / | ||
+ | total 672 | ||
+ | -rw-rw-rw- 1 root root 289442 Jun 26 2015 D8-57E42E89-30 | ||
+ | -rw-rw-rw- 1 root root 3854 Jun 26 2015 D9-57E42E89-30 | ||
+ | -rw-rw-rw- 1 root root 16966 Jun 26 2015 DA-57E42E89-30 | ||
+ | -rw-rw-rw- 1 root root 65779 Jun 26 2015 DB-57E42E89-30 | ||
+ | -rw-rw-rw- 1 root root 20562 Jun 26 2015 DF-57E42E89-30 | ||
+ | -rw-rw-rw- 1 root root 259271 Jun 26 2015 E0-57E42E89-30 | ||
+ | -rw-rw-rw- 1 root root 372 Jun 26 2015 E1-57E42E89-30 | ||
+ | |||
+ | [root@petaltail ~]# ls -lR / | ||
+ | / | ||
+ | total 144 | ||
+ | -rw-rw-rw- 1 root root 40 Jun 26 2015 DC-57E42E89-30 | ||
+ | -rw-rw-rw- 1 root root 40948 Jun 26 2015 DD-57E42E89-30 | ||
+ | -rw-rw-rw- 1 root root 100077 Jun 26 2015 DE-57E42E89-30 | ||
+ | |||
+ | # meta content | ||
+ | |||
+ | [root@sharptail ~]# ssh n38 find / | ||
+ | / | ||
+ | / | ||
+ | |||
+ | [root@sharptail ~]# ssh n39 find / | ||
+ | (none, no mirror) | ||
+ | |||
+ | </ | ||
==== Tuning ==== | ==== Tuning ==== | ||
Line 32: | Line 98: | ||
* no such values for sdb1 | * no such values for sdb1 | ||
* can only find min_free_kbytes, | * can only find min_free_kbytes, | ||
+ | * stripe and chunk size | ||
+ | |||
+ | < | ||
+ | |||
+ | [root@n7 ~]# beegfs-ctl --getentryinfo / | ||
+ | Path: | ||
+ | Mount: /mnt/beegfs | ||
+ | EntryID: root | ||
+ | Metadata node: n38 [ID: 48] | ||
+ | Stripe pattern details: | ||
+ | + Type: RAID0 | ||
+ | + Chunksize: 512K | ||
+ | + Number of storage targets: desired: 4 | ||
+ | |||
+ | </ | ||
+ | * The cache type can be set in the client config file (/ | ||
+ | * buffered is default, few 100k per file | ||
* tuneNumWorkers in all / | * tuneNumWorkers in all / | ||
Line 38: | Line 121: | ||
* metadata server tuning | * metadata server tuning | ||
* read in more detail | * read in more detail | ||
+ | |||
+ | ==== Installation ==== | ||
+ | |||
+ | * made easy [[http:// | ||
\\ | \\ | ||
**[[cluster: | **[[cluster: |