This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
cluster:151 [2016/10/31 19:25] hmeij07 [Mirror Data] |
cluster:151 [2016/11/10 20:09] hmeij07 [Mirror Meta] |
||
---|---|---|---|
Line 24: | Line 24: | ||
* '' | * '' | ||
+ | |||
+ | ==== Resync Data ==== | ||
+ | |||
+ | [[http:// | ||
+ | |||
+ | //If the primary storage target of a buddy group is unreachable, | ||
+ | |||
+ | Testing out fail over and deletion of data on secondary then a full resync process: | ||
+ | |||
+ | |||
+ | * started a full --resyncstorage --mirrorgroupid=101 --timestamp=0 | ||
+ | * got --getentryinfo EntryID for a file in my / | ||
+ | * did a cat / | ||
+ | * brought primary storage down | ||
+ | * redid the cat above (it hangs for a couple of minutes, then displays the file content) | ||
+ | * while primary down, I ran rm -rf / | ||
+ | * a cat now generates the expected file not found error | ||
+ | * brought up primary and started a full --resyncstorage --mirrorgroupid=101 --timestamp=0 | ||
+ | * the nr of files and dirs discovered is as expected lower by the correct values | ||
+ | * when I now search for the EntryIDs obtained before they are gone from / | ||
+ | |||
+ | Nice that it works. | ||
+ | |||
+ | So you can full storage content mirror. You'll still need rsnapshots to recover lost data or point in time restores. | ||
==== Mirror Data ==== | ==== Mirror Data ==== | ||
+ | |||
+ | When not all storage servers are up, client mounts will fail. This is just an optional " | ||
+ | |||
+ | In order to able able to take a storage server off line without any impact, all content needs to mirrored. | ||
** Before ** | ** Before ** | ||
Line 59: | Line 87: | ||
| | ||
# enable mirroring for data by directory -numTargets needs to be set to max nr of storage servers? | # enable mirroring for data by directory -numTargets needs to be set to max nr of storage servers? | ||
+ | # changed to 11/02/2016: | ||
+ | [root@n7 ~]# beegfs-ctl --setpattern --buddymirror / | ||
[root@n7 ~]# beegfs-ctl --setpattern --buddymirror / | [root@n7 ~]# beegfs-ctl --setpattern --buddymirror / | ||
New chunksize: 524288 | New chunksize: 524288 | ||
Line 74: | Line 104: | ||
< | < | ||
+ | [root@n7 ~]# beegfs-df | ||
+ | |||
+ | METADATA SERVERS: (almost no changes...) | ||
+ | STORAGE TARGETS: (each target less circa 10G) | ||
+ | TargetID | ||
+ | ======== | ||
+ | | ||
+ | | ||
+ | |||
+ | # lets find an object | ||
+ | [root@n7 ~]# beegfs-ctl --getentryinfo / | ||
+ | Path: / | ||
+ | Mount: /mnt/beegfs | ||
+ | EntryID: 178-581797C8-30 | ||
+ | Metadata node: n38 [ID: 48] | ||
+ | Stripe pattern details: | ||
+ | + Type: Buddy Mirror | ||
+ | + Chunksize: 512K | ||
+ | + Number of storage targets: desired: 2; actual: 1 | ||
+ | + Storage mirror buddy groups: | ||
+ | + 101 | ||
+ | |||
+ | # original | ||
+ | [root@n7 ~]# ls -lh / | ||
+ | -rwxr-xr-x 1 hmeij its 4.9G 2014-04-07 13:39 / | ||
+ | |||
+ | # copy on primary | ||
+ | [root@petaltail chroots]# ls -lh / | ||
+ | -rw-rw-rw- 1 root root 4.9G Apr 7 2014 / | ||
+ | ^^^^^^^^ | ||
+ | |||
+ | # copy on secondary | ||
+ | [root@swallowtail ~]# find / | ||
+ | / | ||
+ | [root@swallowtail ~]# ls -lh / | ||
+ | -rw-rw-rw- 1 root root 4.9G Apr 7 2014 / | ||
+ | ^^^^^^^^ | ||
+ | |||
+ | # seems to work, notice the '' | ||
</ | </ | ||
+ | |||
+ | Here is an important note, from community list: | ||
+ | |||
+ | * " | ||
+ | * so the important line that tells you that this file is mirrored is "Type: Buddy Mirror" | ||
+ | * " | ||
+ | |||
+ | Another note: I changed paths for mirrormd and buddymirror to ''/ | ||
+ | |||
+ | < | ||
+ | |||
+ | [root@cottontail2 ~]# beegfs-net | ||
+ | meta_nodes | ||
+ | ============= | ||
+ | cottontail2 [ID: 250] | ||
+ | | ||
+ | |||
+ | [root@cottontail2 ~]# beegfs-ctl --listnodes --nodetype=meta --details | ||
+ | cottontail2 [ID: 250] | ||
+ | | ||
+ | | ||
+ | ^^^ | ||
+ | |||
+ | </ | ||
+ | |||
==== Quota ==== | ==== Quota ==== | ||
Line 85: | Line 179: | ||
* seems straightforward | * seems straightforward | ||
* do BEFORE populating XFS file systems | * do BEFORE populating XFS file systems | ||
+ | |||
+ | ==== Meta Bakcup/ | ||
+ | |||
+ | [[http:// | ||
+ | |||
+ | ==== Resync Meta ==== | ||
+ | |||
+ | [[http:// | ||
==== Mirror Meta ==== | ==== Mirror Meta ==== | ||
+ | |||
+ | //Metadata mirroring can currently not be disabled after it has been enabled for a certain directory// | ||
Definitely want Meta content mirrored, that way I can use the n38-n45 nodes with local 15K disk, plus maybe cottontail2 (raid 1 with hot and cold spare). | Definitely want Meta content mirrored, that way I can use the n38-n45 nodes with local 15K disk, plus maybe cottontail2 (raid 1 with hot and cold spare). | ||
Line 96: | Line 200: | ||
# enable meta mirroring, directory based | # enable meta mirroring, directory based | ||
+ | # change to 11/04/2016: used --createdir to make this home. | ||
+ | [root@n7 ~]# beegfs-ctl --mirrormd / | ||
[root@n7 ~]# beegfs-ctl --mirrormd / | [root@n7 ~]# beegfs-ctl --mirrormd / | ||
Mount: '/ | Mount: '/ | ||
Line 113: | Line 218: | ||
[root@sharptail ~]# ssh n38 find / | [root@sharptail ~]# ssh n38 find / | ||
/ | / | ||
- | ^^^^^^ | + | |
# and find | # and find | ||
[root@sharptail ~]# ssh n39 find / | [root@sharptail ~]# ssh n39 find / |