This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
cluster:151 [2016/10/28 19:33] hmeij07 [cluster idea] |
cluster:151 [2016/10/31 19:25] hmeij07 [Mirror Data] |
||
---|---|---|---|
Line 20: | Line 20: | ||
* 2-3 new units storage (+snapshots/ | * 2-3 new units storage (+snapshots/ | ||
* cottontail2 meta + n38-n45 meta, all mirrored | * cottontail2 meta + n38-n45 meta, all mirrored | ||
+ | |||
+ | ==== beegfs-admin-gui ==== | ||
+ | |||
+ | * '' | ||
+ | |||
+ | ==== Mirror Data ==== | ||
+ | |||
+ | ** Before ** | ||
+ | |||
+ | < | ||
+ | [root@cottontail2 ~]# beegfs-df | ||
+ | METADATA SERVERS: | ||
+ | TargetID | ||
+ | ======== | ||
+ | 48 | ||
+ | 49 | ||
+ | | ||
+ | |||
+ | STORAGE TARGETS: | ||
+ | TargetID | ||
+ | ======== | ||
+ | | ||
+ | | ||
+ | </ | ||
+ | |||
+ | ** Before ** | ||
+ | |||
+ | < | ||
+ | |||
+ | # define buddygroup - these are storage target IDs | ||
+ | [root@n7 ~]# beegfs-ctl --addmirrorgroup --primary=13601 --secondary=21701 --groupid=101 | ||
+ | Mirror buddy group successfully set: groupID 101 -> target IDs 13601, 21701 | ||
+ | |||
+ | [root@n7 ~]# beegfs-ctl --listmirrorgroups | ||
+ | | ||
+ | | ||
+ | 101 | ||
+ | | ||
+ | # enable mirroring for data by directory -numTargets needs to be set to max nr of storage servers? | ||
+ | [root@n7 ~]# beegfs-ctl --setpattern --buddymirror / | ||
+ | New chunksize: 524288 | ||
+ | New number of storage targets: 2 | ||
+ | Path: / | ||
+ | Mount: /mnt/beegfs | ||
+ | |||
+ | # copy some contents in (~hmeij is 10G) | ||
+ | [root@n7 ~]# rsync -vac --bwlimit /home/hmeij / | ||
+ | |||
+ | </ | ||
+ | |||
+ | ** After ** | ||
+ | |||
+ | < | ||
+ | |||
+ | |||
+ | </ | ||
+ | ==== Quota ==== | ||
+ | |||
+ | * [[http:// | ||
+ | * setup XFS | ||
+ | * enable beegfs quota on all clients | ||
+ | * enforce quota | ||
+ | * set quotas using a text file | ||
+ | * seems straightforward | ||
+ | * do BEFORE populating XFS file systems | ||
==== Mirror Meta ==== | ==== Mirror Meta ==== | ||
- | Definitely | + | Definitely |
- | Content | + | Content |
- | - | + | |
+ | |||
< | < | ||
- | # enable | + | # enable |
[root@n7 ~]# beegfs-ctl --mirrormd / | [root@n7 ~]# beegfs-ctl --mirrormd / | ||
Mount: '/ | Mount: '/ | ||
Line 46: | Line 113: | ||
[root@sharptail ~]# ssh n38 find / | [root@sharptail ~]# ssh n38 find / | ||
/ | / | ||
+ | ^^^^^^ | ||
# and find | # and find | ||
[root@sharptail ~]# ssh n39 find / | [root@sharptail ~]# ssh n39 find / | ||
Line 54: | Line 121: | ||
</ | </ | ||
+ | |||
+ | Writing some initial content to both storage and meta servers; vanilla out of the box beegfs seems to balance the writes across both equally. Here are some stats. | ||
+ | |||
+ | |||
==== / | ==== / | ||
Line 78: | Line 149: | ||
Looks like: | Looks like: | ||
+ | |||
+ | * NOTE: failed to mount /mn/beegfs is the result of out of space storage servers. | ||
< | < | ||
Line 114: | Line 187: | ||
* global interfaces files ib0-> | * global interfaces files ib0-> | ||
- | * priority order, seems useful | + | * connInterfacesFile = / |
- | * set in a file somewhere | + | * set in / |
* backup beeGFS EA metadata, see faq | * backup beeGFS EA metadata, see faq | ||
Line 160: | Line 233: | ||
* made easy [[http:// | * made easy [[http:// | ||
+ | * rpms pulled from repository via petaltail in '' | ||
< | < | ||
Line 168: | Line 242: | ||
============= | ============= | ||
cottontail [ID: 1] | cottontail [ID: 1] | ||
- | | + | |
meta_nodes | meta_nodes | ||
============= | ============= | ||
n38 [ID: 48] | n38 [ID: 48] | ||
- | | + | |
n39 [ID: 49] | n39 [ID: 49] | ||
- | | + | |
storage_nodes | storage_nodes | ||
============= | ============= | ||
swallowtail [ID: 136] | swallowtail [ID: 136] | ||
- | | + | |
petaltail [ID: 217] | petaltail [ID: 217] | ||
- | | + | |
</ | </ | ||
\\ | \\ | ||
**[[cluster: | **[[cluster: |