This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
cluster:151 [2016/11/10 19:58] hmeij07 [Mirror Meta] |
cluster:151 [2016/11/29 16:20] hmeij07 [Resync Data #2] |
||
---|---|---|---|
Line 25: | Line 25: | ||
* '' | * '' | ||
- | ==== Resync Data ==== | + | ==== Resync Data #2 ==== |
+ | |||
+ | If you have 2 buddymirrors and 2 storage servers each with 2 storage objects, beegfs will write to all primary storage targets even if numtargets is to 1 ... it will use all storage objects so best to numtargets' | ||
+ | |||
+ | How does one add a server? | ||
+ | |||
+ | < | ||
+ | |||
+ | # define storage objects, 2 per server | ||
+ | [root@petaltail ~]# / | ||
+ | [root@petaltail ~]# / | ||
+ | [root@swallowtail data]# / | ||
+ | [root@swallowtail data]# / | ||
+ | |||
+ | |||
+ | [root@cottontail2 ~]# beegfs-df | ||
+ | METADATA SERVERS: | ||
+ | TargetID | ||
+ | ======== | ||
+ | | ||
+ | |||
+ | STORAGE TARGETS: | ||
+ | TargetID | ||
+ | ======== | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | |||
+ | # define mirrrogroups | ||
+ | [root@cottontail2 ~]# beegfs-ctl --addmirrorgroup --primary=21701 --secondary=13601 --groupid=1 | ||
+ | [root@cottontail2 ~]# beegfs-ctl --addmirrorgroup --primary=13602 --secondary=21702 --groupid=2 | ||
+ | |||
+ | [root@cottontail2 ~]# beegfs-ctl --listmirrorgroups | ||
+ | | ||
+ | | ||
+ | 1 | ||
+ | 2 | ||
+ | |||
+ | # define buddygroups, | ||
+ | [root@cottontail2 ~]# beegfs-ctl --setpattern --buddymirror / | ||
+ | New chunksize: 524288 | ||
+ | New number of storage targets: 1 | ||
+ | Path: /home1 | ||
+ | Mount: / | ||
+ | |||
+ | [root@cottontail2 ~]# beegfs-ctl --setpattern --buddymirror / | ||
+ | New chunksize: 524288 | ||
+ | New number of storage targets: 1 | ||
+ | Path: /home2 | ||
+ | Mount: / | ||
+ | |||
+ | # drop /home/hmeij in / | ||
+ | [root@petaltail mysql_bak_ptt]# | ||
+ | 3623 | ||
+ | [root@petaltail mysql_bak_ptt]# | ||
+ | 3678 | ||
+ | [root@swallowtail data]# find / | ||
+ | 3623 | ||
+ | [root@swallowtail data]# find / | ||
+ | 3678 | ||
+ | |||
+ | # with numtargets=1 beegfs still writes to all primary targets found in all buddygroups | ||
+ | |||
+ | # rebuild test servers with from scratch with numparts=2 | ||
+ | # drop hmeij/ into home1/ and obtain slightly more files (couple of 100s), not double the amount | ||
+ | # /home/hmeij has 7808 files in it which gets split over primaries but numparts=2 would yield 15,616 files? | ||
+ | # drop another copy in home2/ and file counts double to circa 7808 | ||
+ | [root@cottontail2 ~]# beegfs-ctl --getentryinfo | ||
+ | Path: /home1 | ||
+ | Mount: / | ||
+ | EntryID: 0-583C50A1-FA | ||
+ | Metadata node: cottontail2 [ID: 250] | ||
+ | Stripe pattern details: | ||
+ | + Type: Buddy Mirror | ||
+ | + Chunksize: 512K | ||
+ | + Number of storage targets: desired: 2 | ||
+ | [root@cottontail2 ~]# beegfs-ctl --getentryinfo | ||
+ | Path: /home2 | ||
+ | Mount: / | ||
+ | EntryID: 1-583C50A1-FA | ||
+ | Metadata node: cottontail2 [ID: 250] | ||
+ | Stripe pattern details: | ||
+ | + Type: Buddy Mirror | ||
+ | + Chunksize: 512K | ||
+ | + Number of storage targets: desired: 2 | ||
+ | |||
+ | Source: /home/hmeij 7808 files in 10G | ||
+ | |||
+ | TargetID | ||
+ | ======== | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | [root@cottontail2 ~]# rsync -ac --bwlimit=2500 /home/hmeij / | ||
+ | [root@cottontail2 ~]# rsync -ac --bwlimit=2500 /home/hmeij / | ||
+ | TargetID | ||
+ | ======== | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | |||
+ | </ | ||
+ | |||
+ | ==== Resync Data #1 ==== | ||
[[http:// | [[http:// | ||
Line 36: | Line 142: | ||
* started a full --resyncstorage --mirrorgroupid=101 --timestamp=0 | * started a full --resyncstorage --mirrorgroupid=101 --timestamp=0 | ||
* got --getentryinfo EntryID for a file in my / | * got --getentryinfo EntryID for a file in my / | ||
- | * did a cat /mnt.beegfs/ | + | * did a cat /mnt/beegfs/ |
* brought primary storage down | * brought primary storage down | ||
* redid the cat above (it hangs for a couple of minutes, then displays the file content) | * redid the cat above (it hangs for a couple of minutes, then displays the file content) | ||
Line 133: | Line 239: | ||
[root@petaltail chroots]# ls -lh / | [root@petaltail chroots]# ls -lh / | ||
-rw-rw-rw- 1 root root 4.9G Apr 7 2014 / | -rw-rw-rw- 1 root root 4.9G Apr 7 2014 / | ||
+ | ^^^^^^^^ | ||
# copy on secondary | # copy on secondary | ||
Line 139: | Line 246: | ||
[root@swallowtail ~]# ls -lh / | [root@swallowtail ~]# ls -lh / | ||
-rw-rw-rw- 1 root root 4.9G Apr 7 2014 / | -rw-rw-rw- 1 root root 4.9G Apr 7 2014 / | ||
+ | ^^^^^^^^ | ||
# seems to work, notice the '' | # seems to work, notice the '' | ||
Line 178: | Line 286: | ||
* do BEFORE populating XFS file systems | * do BEFORE populating XFS file systems | ||
+ | ==== Meta Backup/ | ||
+ | |||
+ | [[http:// | ||
+ | |||
+ | < | ||
+ | |||
+ | # latest tar | ||
+ | rpm -Uvh / | ||
+ | |||
+ | # backup | ||
+ | cd /data; tar czvf / | ||
+ | |||
+ | # restore | ||
+ | cd / | ||
+ | |||
+ | # test | ||
+ | cd /data; diff -r beegfs_meta beegfs_meta.orig | ||
+ | # no results | ||
+ | |||
+ | </ | ||
+ | |||
+ | |||
+ | |||
+ | ==== Resync Meta ==== | ||
+ | |||
+ | [[http:// | ||
+ | |||
+ | * older versions | ||
+ | * new future version will work like storage mirror with HA and self-healing | ||
==== Mirror Meta ==== | ==== Mirror Meta ==== | ||
Line 208: | Line 345: | ||
[root@sharptail ~]# ssh n38 find / | [root@sharptail ~]# ssh n38 find / | ||
/ | / | ||
- | ^^^^^^ | + | |
# and find | # and find | ||
[root@sharptail ~]# ssh n39 find / | [root@sharptail ~]# ssh n39 find / | ||
Line 285: | Line 422: | ||
* set in / | * set in / | ||
- | * backup | + | * backup/restore/mirror |
- | * attempt a restore | + | * see more towards top this page |
- | * or just snapshot | + | |
* storage server tuning | * storage server tuning |