Steps to initiate the Interim Portfolio for the go-live weekend
Backup PSPROD 81 database on Fuji (~45 minutes)
Stop the listener on Fuji; rename the listener log to start fresh; restart the listener (while the backup is happening)
Alter the sqlnet.ora, tnsnames.ora on the interim portfolio machines, locally redirecting the tns entries PSPROD and SFIS to PS81TRNG on Bigsky.
interim portfolio machines are portfolio3,4 and 6 and are currently in the wesep round robin
change the oracle client configuration on the shared /opt/oracle9 so that all machines that use that client are directed to PS81TRNG, which includes clusterw1 and 2 and other machines that may use this client that we may have missed
overlay the PS81TRNG database on Bigsky with the fresh backup of PSPROD 81 from Fuji.
Stop the listener on Bigsky; rename the listener log to start fresh; restart the listener (while the backup is happening)
Alter the sqlnet.ora, tnsnames.ora on Bigsky, locally redirecting the tns entries PSPROD and SFIS to PS81TRNG on Bigsky.
remove the IP addresses of all portfolio machines from the PSPROD 81 database on Fuji in order to prevent any mistaken portfolio contact.
remove all IP addresses of all portfolio machines from the PS81TRNG database on Bigsky *except* for the interim portfolio machine IP addresses, in order to prevent any mistaken rogue portfolio contact.
login to the CURL.OPS$NOTESR5 database schema, and create database link PSPROD to redirect to PS81TRNG and *not* PSPROD database.
Techy Timeline
Thursday, July 12, 10:00 AM:
not needed, no drops. disable snapshot controller onecard & blackboards
DONE marked with '#PSUPG#' disable portfolio5 cron jobs, includes bulkemail
DONE marked with '#PSUPG#' disable htt process on: clusterw3 - nobody cron (1 htt_scanner, 1 htt_starter), goofy - www cron (2 htt_starter, 2 htt_scanner), portfolio5 - notesr5 cron (6 htt_converter)
DONE create a local oracle client for portfolio5
DONE configure local tnsnames.ora & portfolio5 (ports 80/443 also know as portfolio5b) to function as final test portfolio for saturday/sunday testing
Thursday, July 12, 12:00 PM:
DONE bring eportfolio down (3/4/6 running against PSPROD/fuji + CURL) with alternate landing page
DONE bring blackboard down with alternate landing page
DONE bring portfolio5 test vhosts down
??? bring IAS down (pub/int/ext)
DONE summerreg down
Thursday, July 12, 3:00 PM:
DONE start of final backup of databases by SW CURL/PSPROD pre-upgrade
DONE bring CURL up as soon as possible, notify Henk, BB becomes available
DONE followed by database overlay on PS81TRNG/bigsky
DONE bring PS81TRNG/bigsky up
DONE IAS repoint to PS81TRNG
avoid oranames … change tnsname.ora on _shared_ oracle client, point SFIS → PS81TRNG
DONE bring interim eportfolio up (3/4/6 running RO mode) … 07/11/07 links not available in original CURL
blackboard is up, wesmaps is up …
Friday, July 13, 5:00 PM:
DONE wessms: on portfolio5 has been hardwired to make connections to real CURL and PS81TRNG so that it can remain running when the test server is brought up and CURL refers to CURLTEST
DONE turn bulkemail back on and point to real CURL
Saturday, July 14, 1:00 PM:
Saturday, July 14, 5:00 PM:
Sunday, July 15, 11:00 AM:
portfolio5b and portfoliotest, no action items (links are already open in CURLTEST)
–> this is the campus wide testing period which is schedule to last till ~5:00 PM (can be extended)
NO ??? firewall to only allow access for wesleyan.edu (requires off-site VPN usage)
Sunday, July 15th, midnight:
Monday, July 16, 06:00 AM:
prep to go-live
eportfolio down, blackboard down, wesmaps down for final CURL backup before upgrade …
SW conversion of original CURL to new
CURL (done in place) for 08:00 AM release
SW last overlay of SFIS89PROD for 08:00 AM release
IAS repoint to SFIS89PROD
DONE wessms: undo connections to real CURL and PS81TRNG and point back to CURL/SFIS in wessms_utils.pl
DONE bulkemail: undo connections to real CURL and PS81TRNG
Monday, July 16, 8:00 AM:
blackboard up, wesdmaps up …
DONE switch back to original config on shared oracle client (use oranames)
DONE interim portfolio restarts and switches from oldCURL/PS81TRNG to newCURL/SFIS89PROD.
sometime monday PM, portfolio5b rejoins the eportfolio servers in normal operations
Monday, July 16 onwards:
DONE start cronjobs, includes bulkemail
no needed, no drops. enable snapshot controller onecard & blackboards
DONE start htt process on condor's goofy