• How do we build a sustainable desktop backup environment?

To find out we need some data about what is going on.

Bullets are action items to address.

After fixing the reclamation schedule and letting it run all day long with a reclamation threshold of 25%, blackeagle's scratch count went up to 100 or so. Since then, mid summer, it has continually shown a downward trend. Like so (it is accelerating a bit with everybody back):

Occasionally we see the disappearance of tapes/volumes. Like below. This happens when two or more tapes/volumes undergo a “relabel” event because of our setting RELABELSCRATCH=YES. Hong does not remember why we have this set, might have been an earlier problem with VTL. Not damaging as tapes/volumes will relabel at next interval. I initially thought it was related to session load.

• IBM suggest setting this to NO in October maintenance.

We need to relieve the early afternoon sched1216, it is simply too full. It puts the server under tremendous loads (loads above 15 have been observed, 8 is business as usual)

It appears most servers are simply stuffed into the first schedule listed. Since the first window is for laptops we need to move nodes out of that slot into others, surely not all of these are laptops. 1,048 nodes so the ideal number is to have about 200 per schedule window.

UPDATE 09/20/2011:

• SCHED1216 has been extended to run till 18:00 hours (did not change the schedule name …)
• SCHED0912 has been created
• Database backup and maintenance window is now 6am - 9am (reclamation might happen but migration is stopped)

AADZENYAHVTL AAHMAD-11 ABALDWINXPLP ABERGEN-10 ABERLIND-GX620 ABRUNO-11 ACABAL07 ACBURKE-2010 ACOOKE-11 ADUPUY-09 FIELDSTEEL-10PC AGILORDONEZ-2011 AGLAUZTODRAN-06 AGONZALEZ-LAPTOP AHADLER-10 AHAENSCH-08 AHORNSTEIN-10 AINFANTE-2008 AKELLY-11 AKNAUERHASE09 ALEVEY-07 AMACQUEEN-DESKTOP AMCLANE-10 AMILLER01-12 AMILLER01XP AMULLINS-06 AOSTOR-08 APATALANO-11 APEREZGIRONES APETERS-10 ARCHPGIS ARUBACHA-10 ARUSHDY-2011 ASANTANIELLO07 ASCHMIDTLAP-11 ASHARMA-09 ASHUSTERMAN-10 ASTUERZEL-11 ASZEGEDYMASZ ATAMHANKAR-11 ATANAKA-106 ATANAKA-2009 ATANG-10 AVANDERBURG07 AWANG01-11 AWANG01-2010 AWARREN-11 AWIGHTMAN-10 BALU-2007 BCHERNOFF-27IMAC BCHERNOFF-329ESC BCHERNOFF-LABGX620 BCHERNOFF-LABGX620B BCHERNOFF-PC BCRAIGHEAD-11 BHOOD-10 BJUHASZ-11 BJWILSON-DESKTOP BKATTEN-10 BMASTERS-11 BMESLIN-2007 BNANGLE-08 BNORTHROP-2009 BSALAFIA-10 BSTEWART-2008 BSTEWART-GRADG5 BSTEWART-MACBOOKPRO BSTRICKLAND-HP CANDREWSXP06 CASHTON09 CBARAW-IMAC CBRODOWSKI-06 CCRUZ01-08 CDANIELS-07 CDANIELS-08 CFHVISITOR-05 CFHVISITOR2-05 CHENOWETHLABMAC3 CHENOWETHLABMAC5 CJPOTTER09 CKEARNEYVTL CLANSERHP07 CLAZOR-VM CLECHOWICZ09 CLEIDY-07 CMAINES-10 CMCCANNHP-08 CMILLER-08 CMILLER-12 COSTROW CPARENTE CPINA-07 CPOTTER01-11 CRASMUSSEN-08 CROCKWELL-MAC CRODRIGUE-08 CSALAS-WIN7 CSANISLOW-HM24 CSANISLOWLAB4-11 CSCULLYHP07 CSMITH03-12 CSORKIN-2011 CTHORNTONHP07 CZINSER-2008 DABROWN-NETBK DACSLIDESCANNER DALBERT-2005 DALEXSON-2008 DALLEN-2010 DALLEN-LCQHA033 DALLEN-POSIM DALONG-11 DALONG08 DBEVERIDGE-37HA DBEVERIDGE-66PC4 DBEVERIDGE-DESKTOP DBEVERIDGE-LABIMAC DBODZNICK-2008 DBODZNICK-LABGX280 DBOULE-OSXPOSIM DEUDELL-12 DGANDOLFO-11 DHENRY-08 DHIEZMAN-VM DJBLOOM-11 DKATZ01-10 DLABONTE-NETBK DLAWRENCELOAN DMATTHEWS-09 DMEREDITH-GX280 DMEYERXP06 DMOON-10 DOLIVER-2008 DOLIVER-LABMINI DPFLEDERER-10 DPFLEDERERHP-11 DPHILLIPS-08 DPNELSON DRICH-VM DROYER-445 DSCHORRVTL DSHEEHAN-08 DSHOREVTL DSIERPINSKI-10 DSIERPINSKI-10A DSIMSHP07 DSTRICKLAND-HP DWHITE-2009 DWINAKOR-10 EATAYLOR-LABAKTA EATAYLOR-LABDELL EATAYLOR-LAPTOP EATAYLOR-OFFICE EBELOW08 ECHARRY-2010 ECHENOWETHLABMAC2 ECONSTUWORK-09 EFOWLER-10 EGERSHON-10 EKLEINBERG-09 ELIM-09 EMCALISTER-11 EMCALISTERIMAC-09 EMCCORMICK-08 EMEREDITH09 EMORAN2006 ENERENBERG EPARIS-2010 ESAVAGE-PHYS ESLUSSER-11 ESZADY-07 ETAKAHASHI ETHOMAS-G5 ETHOMAS-LAPTOP ETRAUBE-12 EVISVARDI-10 FCARNEGIENET FCOHAN-2008 FELLIS-2004 FGRAMUGLIAXP07 FHAUSER09 FKOERTING-09 FKUAN-09 FSTARR-SHAKTI FWILLIAMS-10 GAARON-2008 GBERNARD-07 GEBSTEINHP-08 GGALLAROTTI-10 GGIBBONS-VM GGOLDBERG-12 GOVVISITOR-07 GPETERSSON-2009 GPYKE-12 GSHAW-10 GSKILLMAN-10 GULYSSE-10 GVOTH-2008B GWHEELER-MAC GYOHE-10 HBARTHLAB-MBP HERIC HFCHAMBERS-PC HMCNEILVTL HNEEDLER-08 HPMIMIME-10 HPSTEVENS-2008 HWHEELER09 IALESHKOVSKY-2011 IHARJITOVTL IKARAMCHETI-11 IMUKERJI-2009 IMUKERJI-273HA-PR IRUSSU-2006 ISAAKA JADAMS-2009 JALDEN01-2011 JAMCKENNA-MAC JBAIRDBURNET-12 JBIDDISCOMBE-MAC JBONIN-09 JBRUNO-2010 JBSILVERHP07 JBURNETTHPLAP JCASTRO-10 JCROOKE2XP JCUTLER-11 JDANDREA-11 JDEBOER-IMAC JDONADY-2008 JDRISCOLL-08 JDRISCOLLHP08 JFINN-11 JFOUNTAIN-10 JGILARDE-2006 JGOODALE-VM JGREENWOOD-IMAC JHICKS-11 JJACOBSEN-10 JKAMM-10 JKAMMLOAN-01 JKAUANUI-08 JKAUANUI-12 JKAUANUI-LT JKNEE-2008 JLABELLA-G5 JLEONE-08 JMCGUIRE-10 JMCINTOSH-2005 JMEERTS-2008 JMILLER04XP JMORAWSKI-MINI JMORAWSKILAB-05 JMORAWSKILABG5-06 JNAEGELE-2008 JNAEGELE-MACBOOK JOSEPHREED-11 JPECK-4500 JPFISTER-10 JPOWZYK-2006 JQUIJADA-12 JRAITI-11 JRANDALLVTL JRIDER-2011 JRKIRN-2011 JROSE01DELL-09 JROUSE-11 JSCHIFF JSEAMON-09 JSHORTELL-11 JSLANE-MAC JSLANEMAC JTOPSHE09 JTUCKER-08 JVAREKAMP-MARSHGX280 JVODAK09 JWALTER-10 JWANG03-11 JWESTMACBOOK JWILLIS-11 JWOLFE-LAPTOP JZWELLING-07 KANDERSON-10 KATTENINTERN-10 KBIRNEY-12 KKOLCIO KNICHOLS-08 KNORRISELITE-10 KRIM-12 KROBERTS-11 KSAVINELLI07 KSMITH-VM KTENEYCKVTL KTHORPE-MBP KTOLOLYAN-10 KVANBLARCOM-10 KVOELKELMEIM-LAPTOP KWARREN-MAC KWEIL-LAPTOP KWEIL-SOPHIE KWHALEN-07 LAPPEL-2006 LCALDWELL-11 LCARRASQUILL-10 LCOHEN01-12 LCOHEN01-MB LDOMBROWSKI-2011 LGRABEL-LABIMAC LGUENOVA-11 LJSTARK-10 LLAPLANT-08 LLENSING LLUKENS-DESKTOP LMCQUEENEY-VM LMIMS-07 LMWRIGHT LNUSSDORFER-11 LSACKS-11 LSBROWN-08 LSCOTT-09 LSTARRMAC-10 LWEINBERG LWESTLING-2007 MAHADDAD-08BE MANJON-RES-VM MARMSTRONG-2011 MATH613B MATH630B-GX270 MATHTEMP-07 MBNGUYEN-12 MCALNEN-2009 MCALTER-2007 MCALTER-2012 MCGARRETT-09 MEISNER-11 MEMLABIMAC-05 MEMLABIMAC-09 MFITZGIBBONS-2007 MGILMORE-2008 MGILMORE-PLANETARY1 MGILMORE-PLANETARY2 MGKELLY09 MHERLIHY-6930 MHINGORANI-2010 MHINGORANI-LABG5 MHINGORANI-NEWKINTEK MIMAI-10 MIMAI-EAS MISLEIB-10 MKEANELAPTOP MKINGSLEY-09 MKLANESKI-MAC MKURTZ07 MLOOMER-03 MMCALEAR-2012 MMCALEAR-LABOS9 MMCALEAR-OFFICE MMELENDEZ-10 MMUROLO-LAPTOP MMYERSBROWN-08 MNGUYEN-12 MOLERUDVTL MOTEIZA-2011 MRESNICKHP-08 MROBERTS-10 MROTH-2010 MRUBENSTEIN-11 MSCHUG-09 MSCIOLA-HP MSCIOLAHP-11 MSIMONSONHP07 MSINGER-2008 MSINGER-LAPTOP MSNOW-2009 MTETER-10 MWEIR-2008 MWEIR-LABG5 MWHALEN-MACBOOK MWHALEY-2009 MWHITCOMB-10 MZOCCO-08 NAKSAMIJA-2010 NGARRETT-10 NGARRETT07 NGREENE-11 NKORDA-10 NMEISLAHN-2009 NSHAPIRO-2011 OBOOKAS08 OHOLMES-09 OLEWIS-10 PABRAHAMXP PBOLTON-LABIMAC21 PBOLTON-LABIMAC27 PCARNEYMACBOOK-2009 PDOWDEY-2010 PDSMITH-09 PGOTTSCHALK-11 PHALL09 PHILL-09 PHOLYCROSS-06 PKLECHAPORTE-09 PLANETARY3 PMACKLP PMARK PMATTHUSEN-2011 PMELLEY09 PPATTON-07 PRESOR-2008 PRESOR-GIS2 PRODRIGUEZMO-09 PRUTLAND-11 PSCHWABER-11 PSMATHERS-MAC PTATGE PTATGE-2011 PTIEN-09 PTURENNE2007 PWAGONER RADELSTEIN-09 RAUTRY-10 RBRAULT-CHEMMACBOOK RBRAULT-CHEMPCLAPTOP RBRAULT-DESKTOP RCAMERON-09 RCULLITONXP07 RELPHICK-10 RGAUDALUPE-11 RGORDON-2007 RGROSSMAN-10 RGUADALUPEXP06 RHOWARD-11 RHOWARD05 RIJOHNSON-2012 RJENKINS RJENKINS-IMAC RJOHNSON-MAC RMBERMAN-07 RMEHMET-09 RMURPHY-09 RNISSE-10 ROLSON-2009 ROLSON-LABIMAC RPRATT-2009 RPRATT-G5 RRASMUSSEN-MACBOOK RROLLEFSON-2007 RROSENTHAL-09 RROSENTHAL-WIN7 RSCHATZ-09 RSCHMIDT-10 RSCHUBEL-07 RSILLASENXP RSTEELE-10 RSTRIEGELTEST SBACHNER-11 SBACKER07 SBROUGHVTL SBUNAEVVTL SCAYETANO-2008 SCHAKRAVARTI-10 SCHYUNG08 SCOLLINSVTL SCROUCHER-08 SDAILEY-07 SDEVOTO-2008 SDEVOTO-HALIBUT SDEVOTO-JUDAS-PRIEST SESULTAN-2007 SESULTAN-LABIMAC SFAZZINO-2005 SGUHA-12 SHIGGINS SHILL-DESKTOP SHOLMES-2008 SHOLMES-G5 SHUSTLABIMAC2-09 SHUSTLABIMAC3-10 SJACARUSO09 SKHALIGHI_MO-PHYSGRAD SKIRSCHE-11 SLASTRINA-2007 SLAZAREXP07 SLOURIE SMANJON-2008 SMBROWN-NETBOOK SMCCANN-09 SMCCANNHP-09 SMCQUEEN-10 SNOVICK-2009 SOCONNELL-2008 SPARSAMOGHAD-PHYSGRAD239 SPASSMAN-2010 SPASSMANSERVER SPAVIS08 SPHELPS-10 SPRINGERHP-07 SRUDEN-06 SSADLERVTL SSOLOVEI-ANIMALCARE SSTEMLERLABMAC-09 SSTEMLERVSTA-09 SVANEK-2008 SVONSCHOELER-GX280 SWEINER-11 SWILIARTY-09 SWU03-XP SZHENG-2011 TCKU-2007 TCKU-LABIMAC TCKU-LABIMAC2 TIMADA-11 TIRANI-09 TKANNAM-2009 TKANNAM-XP06 TKAWASHIMA-2011 TLWILSON10 TMACRI04 TMORGAN-2010 TMORGAN-XP06 TOVERTON-12 TSABO-10 TTELFAIR-2011 VDIAZ-11 VGILLISPIE-10 VGRAHAM-10 VHARRIS-2008 VHARRIS-EESOFFICEWORKSTATION VMARINELLI-2008 VNYE-10 VSABINLAPTOP VSMOLKIN-11 WCOMFORT-10 WESTMORELAND-2007 WHOLDERMACBOOK-2009 WJOHNSTON-09 WJOHNSTON-MBP WKCHAN-09 WNELLIGAN-10 WPINCH-11 WPRINGLE-2007 WRAYACK-10 WRITWORKS-06 WSTOWE-08 WSTOWE-11 WSTOWE-12 WSTRACKINGUPPER XZHU-WIN7 YKORDONSKIY-XP YMALIN

ACOOKEGW07 AFRY-2010 AFRY-LABG5 AFRY-ROOM137 AMARZOUCA-WIN7 AMCFADZEN-11 AROJAS-06 ATANASI-VM AWALSH-09 AWENSINGER BDANIEL-06 BLOOMBERG-11 BNORTHROP-745 BSTEWART-LAB08 BTRAVERS-11 CCOLFER-VM CKAUFMAN-DELL CMATTZA-2011 COTHON-2010 CROGAN DBEVERIDGE-66PC2 DFOYLE-11 DHINTONXP DKLARE-10 DKOEPF-HP DLOW-10 DMEYER-11 DOLIVER-LABPC DRHENRY-OBHS-VM DSCHNAIDT EAS-MACMINI EATAYLOR-LABIMAC ELEE01-10 EMILROY EMMORAN ETINKER-11 ETINKER-VM FCOHAN-HOT-GREEN-SLIME FDEGIOVANNI-2011 GAARON-LAB1 GVOTH-LAB4 GVOTH-LABPC3 GVOTH-VIDEO3 HEPSTEIN-09 HEPSTEIN-10 HTRELOAR-2012 IMUKERJI-LAB2 JBURNETTXP JCONNARY07 JCORMACK-10 JCROOKE2KP JESTEVESXP-07 JFIELDS-MAC JGILARDE-LSMMETA JGYURITS-09 JHUEBLER07 JKEHOE09 JKNEE-LABPC JLOWRIE-GW JNAEGELE-LAB1 JNAEGELE-LAB990 JRAFFERTY-11 JRAFFERTY07 JRKIRN-PC2 JVAREKAMP-2010 KKUENZLI-2010 KLOGSDON-09 KMULLEN-10 KONEILL-11 KPUGLIESE-07 KPUGLIESE-08 KSAVINELLI11 KSCHEIBE-10 KSCHEIBEXP KSHINOHARA KSTEFANOWICZ-09 KVECCHITO-VM KWOLFE-09 LBORHMAN-09 LCARLSON LGRABEL-TEMPMAC LHUWEL-2010 LJSTARK-VM LKEISER03 LNEARY-2011 MDAVIS-11 MFRISCH-2010 MGILMORE-GX260 MGILMORE-LABBOOTCAMP MHINGORANI-LABDELL8100 MICAHWIN7 MNAKAMURA-2011 MNEALE-10 MOSPINA-2011 MWEIR-LCQ NBRUCE-2011 NICKDAVID-08 NSCHWARTZ-11 ODRAKE-MAC PABRAHAMXP1 PHOTOID-10 PRESOR-MACMAC PSTAYE-10 PTYNANMAC RBRAULT-STORM153HA RCHIAPETTA-MACMINI RKISSINGER-VM RLANE-165 RMCKENNA-VM ROBINZUP-WIN7 RPRATT-LABG5 SANTONIO2-09 SBACKER11 SCWIRKA09 SIGNSHOP-11 SJAVORSKI-10 SMCGOWAN-XP SNOVICK-LABGX620 SNOVICK-MACHINE THARLEY07 TLINDROS-11 TPARKER-IMAC VLANE-2010 WESTMORELAND-LABXW6600 WSUSANTHAWIJ-239ESC YPRAMUDYA-PHYSGRAD235

AAISSA-2010 ACHATFIELD-2011 AGERTZ-ASIAN ARUDENSKY-2011 ASTARON-VM BALLY-2011 BCHERNOFF-2009MAC BCHERNOFF-FISH2 BCHERNOFF-LABIMAC BFAY-11 BSTEWART-LABGX260 CCARR-VM CFA-MARKETING CFLANAGAN-10 CHAGNERXP07 CHENOWETHLABPC1 CNASCIMENTO-2010 COTHON-LABPC2 CPIKE01-08 DBEVERIDGE-66PC1 DCANALIA-04 DHUTSON-11 DISPATCH1-10 DOLIVER-LABHP DSKURA01-10 DTREISTER-11 DWRIGHT08A ECAREY-08 EDELVALLE-GW EHARRISXP07 EMAINELLA-10 FCOHAN-LABHP FMARSILLIXP07 GAARON-LAB2 GAARON-TEMP GGDOVIN-VM GGIBBONSWIN7-VM GGRUBBS-2011 GRODRIGUEZ07 GUN-11 GVOTH-LAB2 GVOTH-VIDEO7 HORAVEC-2011 IMUKERJI-HPLC INVESTINTERN-1 INVESTMENT1-11 JBEST-2011 JBUCKLEY-11 JCHIARI-2011 JENXUTOWIN7-VM JFIELDSGWE06 JGILARDE-2012 JGILARDE-SEM JGOETZ-2011 JGUERRERO-2010 JHAMILTON07 JKNEE-RM025 JMAGUDA-GX280 JNAEGELE-LAB2 JOUTLAW-10 JSEADEEK-10 JSTEELE2-09 JTHOM-10 KCOLLINS-2011 KHOOK-10 KHOOKXP06 KMURPHYHP08 KSRUBASWIN7-VM KTAILLONWIN7 KWINSTON LANTOSZWIN7-VM LCUMMINGS-VM LDIERKER08 LGAMBELL-09 LHUWEL-LAB745 LMASCARO-VM LNEARY LSPRINGER-11 LSTETHERS-10 MBLAKE-VM MCALTER-LABGX620 MDAMEWIN7-VM MGILMORE-PLANETARY MGILMORE-PLANETARYBC MGLYNN-09 MHUMPHRIES-2011 MPERALTA-11 MSINGER203 MSNOW-2012 PBOLTON-HA038A PCBEST-10 PCROWLEY-10 PGRANDE-07 PJACKSON-09 PLEONE-IMAC PRESOR-MACPRO PTULLY-10 QRODE-11 RBRAULT-153HADELL RCONNOLLY09 RCULLITON11 RLANE-2010 RLANE-LABIMAC RLANE-RTPCR RMCCALLUM-10 RPRATT-BEAST RPRATT-LABIMAC2 RPRATT-LABPC RSCHMIDTXP07 RWILSON-09 RZUPXP07 SANTONIO-09 SASMITHWIN7-VM SBENSONWIN7-VM SDEVOTO-ZZTOP SFERRIS-VM SHCOHEN-08 SHOLMES-LABWIN7 SJHIGGINS-10 SKULESZA-09 SMARTINXP06 SNORTHGRAVES08 SOCONNELL-LAB419 SOPHAT-VM SSARVER-10 SUMARSAM TMORGAN-LABPC2 TSALAFIA-VM USDAN-4WINDS WSAADMIN3-10 YSAMUELMAC

ABALDWIN09 ABLOOM-2010 ABRUNO-VM ACBURKE-LABG5 ACURRAN-IMAC AFRY-LAB620 AKIDDLEWIN7-VM AKLEIN01-10 AMARTIN-2010 AMORINWIN7-VM ARUSHDY BCHERNOFF-2009 BCHERNOFF-LABBC7 BSCHUKOSKE-VM CCOLFER04 CCPFELLOW-VM CFLORES-11 CKAUFMAN09 CLINICXP CPOISSON CRIZZA08 CSCHUKOSKE-10 CWOOD2005 D2WORKHORSE DBEVERIDGE-66LABPC3 DDYER-DELL DHINTONGW DISPATCH2-10 DISPATCH3-10 DMOORE07 DOLIVER-LABDIM EBROWN02-09 EKONERDING-10 ELECSHOPXP ESZADY-VM FCOHAN-LABVISTA GAARON-LAB3 GBROWNXP GVOTH-VIDEO8 HAIELLO-10 HMINETTI-MAC HSTEGMAIERWIN7 HWEIDNER-08 IADEBOYE-2012 IMUKERJI-FLUOROMAX4 IMUKERJI-LABPC1 IYORK-11 IYORK-411 JDANDREA-06 JESTEVESXP07 JGILARDE-3DRECON JGILARDE-AMT1 JGILARDE-CZV JHEIDORN-WIN7 JJURGEN-11 JKNEE-TEMP JMAGUDA-GX280 JMLOZANOWSKI-10 JMORAWSKI-10 JMORAWSKI-VM JPLATT-GX280 JRKIRN-G5 JSIRY-2011 JTILLEY-11 JVODAK-09 KBEAUDETTE-2011 KBORDNER-06 KCATALDI-09 KHOBBS-10 KKARPA-08 KMACKOWIN7-VM LBERMAN-2011 LCALHOUN08 LCARTA-VM LFLANNIGAN-VM LGRABEL-IMAGEPC LHUWEL-LAB05 LHUWEL-LABGX270 LMIMS-11 LSACKS07 MBIDDLE-10 MCLAWSON-12 MFAYWIN7-VM MGAWLAK MGILMORE-PLAN3TBC MGILMOREBOOT MLINK-VM MLINKWIN7-VM MMCALEAR-IMAC MMCALEAR-LABIMAC MSINGER-LABPC MWEIR-PPA NDABBSWIN7-VM NDANNER-2010 NMCLEES-411 OBARTLETTHP-08 PBOLTON-LAB038PC PHOLYCROSS-11 PMEYER-2007 RBLUMEL-ORIGIN RCONN RCROPPERXP RLANE-EMAC RLANGER7 RMCCALLUM-09 RPRATT-274TYPHOON RPRATT-PC RSHAW-2010 RSRUSSO SATWELL-VM SCONDONXP SDEVOTO-OZZY SHINER-TEST SHORST-11 SLAZARE11 SMARTIN-11 SNOVICK-LABHP SPRIVITERA-2011 SSTEMLER-09 STARABA-10 TBOSTICK-11 TCKU-2012 TKOTTOS-HAMIDWI TMAHEU-09 TPOLICKI-07 TSTANLEY-HP WBURKHART-MAC WESTMORELAND-HA42 WFIRSHEIN-09 WSAADMIN4-10 YSAMUELMACMINI-09

ACBURKE-LABHP ACOOKE-PC11 AFRY-128HA AMARTIN-WORKSTATION ASCHOLAR-GX260 ATURCO-06 BCAHILLXP07 BCHAPPELL-WIN7 BNANGLE11 BNORTHROP-LAB360 CDOLANSKY-10 CGONZALES-VM CHAGNER-12 CLALANDE CLWILKINS-12 COTHON-LAB1 CPINA-11 CRACEWIN7-VM DBEVERIDGE-LABPC1 DBLACKMINI-11 DGRASSO-VM DHINTON-10 DKELLY-09 DKOEPF-2009 DLAWRENCE07 DLEIPZIGER08 DLEISAWITZ-2011 DMEYER-10 DSCHORR EGLICK-09 EMALMSKOG-2012 EROOSBROWN-IMAC FCOHAN-EGGS FCOHAN-LABVISTA2 FELLIS-2012 GAARON-XW6600 GDRISCOLLWIN7 GFOUNTAINXP07 GVOTH-VIDEO4 IRUSSU-2012 JDRECHSLER-10 JGILARDE-EDAX JIMILLER-2011 JJYLKKAHPLAP07 JKAMMWIN7-VM JLAWTON-10 JMAGUDA-12 JRABAMAC JRKIRN-LABMICRO JRKIRN-LABSILVER JSCHENKER-09 JSTEELE-10 JTCALLAGHAN-08 JTHOM2-09 JVAREKAMP-MLABGX270 JWAREHAM-11 JWATSON-HP KMCQUEENEY-WIN7 KNICHOLSVTL KPORTER01-11 KWILIARTYPC-07 LANTOSZ-VM LAWESTLING-2010 LCALHOUNVTL LDERUSHA-09 LHENDRIX-10 LPINETTE-09 LVILLARD07 LWOODARD-IMAC MARMSTRONG-2006 MBAILEY-09 MDATREDELL-11 MGILMORE-BOOTCAMPPC MHANSON-09 MHINGORANI-MAX3 MHINGORANI-XW6600 MKHAMIS-12 MKOZIOL-DESKTOP MLOOMER-VM MSINGER-2010 MSLOBIN-2011MINI MSNOW-2009 NANGELETTI07 NMCLEES-10 OFLORES PPAIGE09 RDARNA-10 RLANE-108 RLOONEY-06 RPANDOLFO-09 RPRATT-LABPC1 RYORK SDEVOTO-ACDC SELLIOTT09 SESULTAN-LABPC2 SFUSSO SGRUCAN-10 SHETTLEWORTH-VM SLIN-HA286BIO SNOVICK-GRAD2 SNOVICK-GRAD3 SOCONNELL-LABNEW SPRIVITERA TCKU-LAB1 TKIEFER-10 TMORGAN-LABPC1 TWEST-09 VINYLCUTTER VOICERECORDING WESTMORELAND-12

• DSS person to generate node switch lists or
• arbitrarily do it server side

Total backup volume is currently 45 TB, up from 42 TB mid summer. Top 25 nodes account for 15.3 TB or 34%.

• find out why these nodes are so large and reduce them, period.

The number of files has a significant impact on client (metadata needs to be transferred) as well as on database (each file is a metadata record in multiple tables). What gets backed up is impacted by this, for example, some analyses reveals that perhaps 15% of the number of files reaching the BACKUPPOOL (incremental, incoming) has a filename that contains the path …/*CACHE*/…

There are a total of 118 million files backed up.

• Tackle top5 for sure, maybe even top10, identify content, set up exclude filters, period.

Lots of different filespaces are being backed up. Here is an example.

This node is currently 710 GB but could expand to slightly over 6 TB.

Mostly a Mac problem.

• Can we find these drives and exclude them? (ie, include /Volume/Macintosh HD, exclude rest)
• Exclude external drives /Volumes/MyPassport? (western digital)
• Exclude /Volumes/Iomega HDD?

ANSWER http://www.oucs.ox.ac.uk/hfs/help/excluding.xml.ID=excludedrives

explains nicely what Jim was talking about excluding drives via GUI and also points out you can do it via client dsm.opt settings. but it also shows me how we can do this server side wise, so i tried with bstewart-2008 after making sure /Volumes/WD1.5TB1 and /Volumes/WD1.5TB-2 were unique, added excludes for them and boom they were gone, node back to 350GB.

so if we adopt a convention to force / to mount at some unique mountpoint, which it already does, we could do, evaluate from bottom up

exclude /Volumes/…/*
include /Volumes/SomeApprovedVolume/…/*
include.dir /Volumes/Macintosh HD

In the meantime i can set up global excludes for some of our problem cases whose mountpoint are unique and if they do not need to be backed up.

report of 09/19/2011: roughly 45 drives with a total utilized capacity of 7.7 TB

• last column will help you find the hogs: GB util = (capacity MB/1024) * (pct util/100)
• RBRAULT-DESKTOP needs to be dealt with
• MWEIR-2008 has a, gasp, 5 TB array on it

It may be that some of these drives are not being backed up, as they may have been excluded before, but were not delete from TSM.

One measure of growth is the calculation between scratch tapes/volumes wanted and reclaimed.

Based on a moving average, in this case the last 7 days so weekends are included, total bytes migrated from the BACKUPPOOL to tapes/volumes can be summed for all those days. Then an average daily rate can be calculated to smooth over hills and valleys. Next we look at how full are tapes/volumes that can not receive any more data (tapes are of size 50 GB).

This leads to a calculation how many tapes/volumes TSM wants to store the “migrated” volume out of the BACKUPPOOL to to tapes/volume based on “full” tape usage efficiency. That can then be followed up by searching for the rate of tapes/volume reclamation.

The difference is the rate of growth.

We need to get the grow rate down by

• reducing volume of content backed up (large size nodes, large files nodes)
• reducing junk being backed up by applying exclude filters
• eliminate external drives form being backed up
• ???

Incremental backup volume, as well as first backups volumes of course, arrive at the BACKUPPOOL before being packaged up and migrated to tapes/volumes. So one interesting view is which nodes backup large amounts of volumes up repeatedly under incremental backup? And if so what?

For a running log comparing day to day behavior look at blackeagle:/tmp/tsm_backup_yesterday_report.log. For example:

• Analyze these top nodes, look at client logs for last few days, come up with filters.

So the question now is the same content repeatedly backed up? Probably easier to determine from client side logs. Looking at content arriving at the BACKUPPOOL the following will be added to the server side exclude filters … this does not tell us per node if same content arrives day after day but gives us a clue if there is junk we can avoid backing up.

tsm> q con /blackeagle/stgeql/stg?.dsm

exclude

• Filters have been applied. Wait for 60 days to take effect, sigh.

Since there is so much junk arriving at the BACKUPPOOL i have implemented new filters for windows and macintoshes (Sept 09)

The filter definitions can be found https://wiki.wesleyan.edu/display/itstss/TSM+recommended+excludes

IBM's Recommended section and the Optional Trash section have been both implemented.

• We should consider sticking the rest in, and wait 60 days, sigh. APPLIED 9/19/2011

It would be nice to be able to query the content of nodes of large size or with huge numbers of files. The first step is displaying their filespaces looking for external drives. The second step is to get an inventory of all files and their file sizes for a particular node.

However the table CONTENTS is so large that any SQL attempts via a script fail to return data (select file_name, file_size from contents where node_name='this_node_name')

• Needs to be client driven for now.

• Back up the file only if the file or its attributes changed since the previous backup
• If a file is being modified during backup, then do not back up file
• Days between backups: 0
• How many different versions of the file should be kept: 2
• Number of days to keep inactive versions: No limit.
• For backup files that a user has deleted from the client node: keep 1 version
• For backup files that a user has deleted from the client node: 60 days

Both approaches below are too taxing on TSM and the SQL generate no output in desired time window. We could consider:

• changing inactive version retention to 60 days
• lower deleted file retention to 30 days

What is the file space used by inactive files by node?

• no answer yet

What is the file space used by deleted files by node?

• select node_name,filespace_name,hl_name,ll_name, object_id from backups where deactivate_date=today
• run at hour X, rerun at hour X plus one day
• compare output, missing files have been delete in that day
• use object_id and get file_size from table contents