We’re live!
This past weekend was the GoLive for the new systems at work. We updated the following:
- New Physical Hardware
- New RHEL Version
- All OS Instances are now VMs in VMWare
- Databases updated to the latest version of Oracle
- A few DBs went to RAC
- Latest version of Apache
- New “routing” system where all web requests hit an “edge node” before being proxied back to the application cluster where they belong
- Migrated J2EE containers (from an old one to a new modern one)
- New application deployment process
- Updated FTP servers
- New Load Balancers
- Updated systems monitoring software
- Our developers had to port over 200 applications from the old app server to the new one — a HUGE undertaking!
Sooo many changes. I was at work from 7:00AM till 11:00PM on Saturday, but that was it! We were scared we were going to have to work for days closing out all of the Sev1 issues, but we were done in less than 24 hours! (I was suppose to be the day shift — thankfully it never happened!)
Around 9:00PM we sorta realized that if we kicked it into gear we could finish up everything in one shot, so that’s what we did. The last few hours were a madhouse, but at the end of the night it was all done.
Today was the first day of having the full production load, and the day was very very boring. No issues on the systems side! (One big software issue came up today, which has already been patched.)
There are some lingering Sev2 and Sev3 issues that will go out in a week and a half, but for as many changes as we had… to have no Sev1 issues less than 48 hours later is amazing! (In the IT world a Sev1 is an issue that has so much impact to the business that it needs to be worked 24/7 — which we were prepared to do.)
This was by far the biggest project I’ve ever worked on. We had over 100 people on this team – I was one of the leads on the systems side.
Yaay! Alll that hard work paid off!
BunnyNet 