Differences

This shows you the differences between two versions of the page.

--- cluster:151 [2016/10/31 14:36]
hmeij07 [Quota Enable and Enforce]
+++ cluster:151 [2016/11/04 15:24]
hmeij07 [Mirror Data]
@@ Line 25: / Line 25: @@
   * ''cottontail:/usr/local/bin/beegfs-admin-gui''
-==== Quota Enable and Enforce ====
+==== Mirror Data ====
+When not all storage servers are up, client mounts will fail. This is just an optional "sanity check" which the client performs when it is  mounted. Disable this check by setting "sysMountSanityCheckMS=0" in beegfs-client.conf. When the sanity check is disabled, the client mount will succeed even if no servers are running.
+In order to able able to take a storage server off line without any impact, all content needs to mirrored.
+** Before **
+<code>
+[root@cottontail2 ~]# beegfs-df
+METADATA SERVERS:
+TargetID        Pool        Total         Free    %      ITotal       IFree    %
+========        ====        =====         ====    =      ======       =====    =
+         low      29.5GiB      23.3GiB  79%        1.9M        1.5M  82%
+         low      29.5GiB      23.1GiB  78%        1.9M        1.5M  82%
+         low     122.3GiB     116.7GiB  95%        7.8M        7.6M  98%
+STORAGE TARGETS:
+TargetID        Pool        Total         Free    %      ITotal       IFree    %
+========        ====        =====         ====    =      ======       =====    =
+         low     291.4GiB      50.6GiB  17%       18.5M       18.4M 100%
+         low     291.2GiB      61.8GiB  21%       18.5M       15.8M  85%
+</code>
+** Before **
+<code>
+# define buddygroup - these are storage target IDs
+[root@n7 ~]# beegfs-ctl --addmirrorgroup --primary=13601 --secondary=21701 --groupid=101
+Mirror buddy group successfully set: groupID 101 -> target IDs 13601, 21701
+[root@n7 ~]# beegfs-ctl --listmirrorgroups
+     BuddyGroupID   PrimaryTargetID SecondaryTargetID
+     ============   =============== =================
+             13601             21701
+# enable mirroring for data by directory -numTargets needs to be set to max nr of storage servers?
+# changed to 11/02/2016:
+[root@n7 ~]# beegfs-ctl --setpattern --buddymirror /mnt/beegfs/home --chunksize=512k
+[root@n7 ~]# beegfs-ctl --setpattern --buddymirror /mnt/beegfs/hmeij-mirror-data --chunksize=512k --numtargets=2
+New chunksize: 524288
+New number of storage targets: 2
+Path: /hmeij-mirror-data
+Mount: /mnt/beegfs
+# copy some contents in (~hmeij is 10G)
+[root@n7 ~]# rsync -vac --bwlimit /home/hmeij /mnt/beegfs/hmeij-mirror-data/
+</code>
+** After **
+<code>
+[root@n7 ~]# beegfs-df
+METADATA SERVERS: (almost no changes...)
+STORAGE TARGETS: (each target less circa 10G)
+TargetID        Pool        Total         Free    %      ITotal       IFree    %
+========        ====        =====         ====    =      ======       =====    =
+         low     291.4GiB      40.7GiB  14%       18.5M       18.4M  99%
+         low     291.2GiB      51.9GiB  18%       18.5M       15.8M  85%
+# lets find an object
+[root@n7 ~]# beegfs-ctl --getentryinfo /mnt/beegfs/hmeij-mirror-data/hmeij/xen/bvm1.img
+Path: /hmeij-mirror-data/hmeij/xen/bvm1.img
+Mount: /mnt/beegfs
+EntryID: 178-581797C8-30
+Metadata node: n38 [ID: 48]
+Stripe pattern details:
++ Type: Buddy Mirror
++ Chunksize: 512K
++ Number of storage targets: desired: 2; actual: 1
++ Storage mirror buddy groups:
+  + 101
+# original
+[root@n7 ~]# ls -lh /mnt/beegfs/hmeij-mirror-data/hmeij/xen/bvm1.img
+-rwxr-xr-x 1 hmeij its 4.9G 2014-04-07 13:39 /mnt/beegfs/hmeij-mirror-data/hmeij/xen/bvm1.img
+# copy on primary
+[root@petaltail chroots]# ls -lh /var/chroots/data/beegfs_storage/buddymir/u2018/5817/9/60-58179513-30/178-581797C8-30
+-rw-rw-rw- 1 root root 4.9G Apr  7  2014 /var/chroots/data/beegfs_storage/buddymir/u2018/5817/9/60-58179513-30/178-581797C8-30
+# copy on secondary
+[root@swallowtail ~]# find /data/beegfs_storage -name 178-581797C8-30
+/data/beegfs_storage/buddymir/u2018/5817/9/60-58179513-30/178-581797C8-30
+[root@swallowtail ~]# ls -lh /data/beegfs_storage/buddymir/u2018/5817/9/60-58179513-30/178-581797C8-30
+-rw-rw-rw- 1 root root 4.9G Apr  7  2014 /data/beegfs_storage/buddymir/u2018/5817/9/60-58179513-30/178-581797C8-30
+# seems to work, notice the ''buddymir'' directory on primary/secondary
+</code>
+Here is an important note, from community list:
+  * "actual: 1" means "1 buddy mirror group"
+    * so the important line that tells you that this file is mirrored is "Type: Buddy Mirror".
+  * "desired: 2" means you would like to stripe across 2 buddy groups. (targets are buddygroups here)
+Another note: I changed paths for mirrormd and buddymirror to ''/mnt/beegfs/home'' and now I see connectivity data for meta node cottontail2 which was previously missing because I working on sub directory level.
+<code>
+[root@cottontail2 ~]# beegfs-net
+meta_nodes
+=============
+cottontail2 [ID: 250]
+   Connections: RDMA: 1 (10.11.103.250:8005);
+[root@cottontail2 ~]# beegfs-ctl --listnodes --nodetype=meta --details
+cottontail2 [ID: 250]
+   Ports: UDP: 8005; TCP: 8005
+   Interfaces: ib1(RDMA) ib1(TCP) eth1(TCP) eth0(TCP)
+               ^^^
+</code>
+==== Quota ====
   * [[http://www.beegfs.com/wiki/EnableQuota|External Link]]
@@ Line 45: / Line 164: @@
 # enable meta mirroring, directory based
+# change to 11/04/2016: used --createdir to make this home.
+[root@n7 ~]# beegfs-ctl --mirrormd /mnt/beegfs/home
 [root@n7 ~]# beegfs-ctl --mirrormd /mnt/beegfs/hmeij-mirror
 Mount: '/mnt/beegfs'; Path: '/hmeij-mirror'
@@ Line 98: / Line 218: @@
 Looks like:
+  * NOTE: failed to mount /mn/beegfs is the result of out of space storage servers.
 <code>

DokuWiki

User Tools

Site Tools

Differences

Page Tools