I work for a large public sector agency on a team responsible for supporting email services for close to 100,000 employees. I am responsible for the design, deployment and day-to-day operation of the computers hosting the email services. For comparison purposes, I would suspect that I am responsible for approximately 3 medium-pop realms worth of users on WoW. This is my attempt at explaining a technical environment and it's issue in as user-friendly of terms as possible.
Design: If I were to design a 'realm', I would want to mirror what we do with our email environment. It meets the needs of a high-demand environment, where the clients are using a fat client to connect and requires pretty much 24x7 access.
First off there would be resiliency. This means multiple computers that are supporting active clients
Second would be security. Imagine your current residence (home, apartment, condo, etc.). Would you rather have all of your stuff simply out in the open, laying out on the lawn? Or would you want some walls, a door, locks and maybe even a high-end security system protecting your valuables? Our accounts, our characters and the entire virtual world are our valuables. Protecting that realm would involve perimeter security surrounding the entire server cluster, possibly around all clusters (i.e. region of US, EU, etc.). There would be a single point of entrance into this area for all connections to the cluster. I would likely place a high-end security device that could monitor the traffic going in, and turn away bad traffic. It's easy to manage. I won't need a millions of Internet connections (or IP addresses)
The alternative is placing each and every server directly on the Internet (i.e. out on the lawn). Doing this would open up each and every server to all sorts of malware, hacks and malicious intent. We would see a LOT more issues if this configuration was used. Anyone driving by our property could simply come by our 'place' and pickup something of value.
Day-to-Day: Blizzard grew to over 11 million players, then shrunk back down to 7 million.
Diagnosis: There are two things I think went on. Based on the twitter feeds of several individuals, I don't think I am far off.
Now, realize that these "salesmen" are not the only people hammering on the front door. The attacks could also be you and me. (wait... keep reading) At my RL job, we've had legitimate users, too chatty with our environment and take down servers. For example, Apple regularly releases a new iPhone OS and many people immediately update their devices. These updates are not always well tuned for our email servers and may require us to patch them or Apple may have a bug that needs fixed. Every once in awhile, their phone finds a corrupted calendar entry and asks 100,000 (an hour) for an update to that entry. This chattiness prevents anyone else from pulling their calendar updates. Now, with each major expansion, Blizzard often changes access interfaces on the game. These changes break add-ons. Add-ons that people come to love and rely on. Add-ons that they'll run even if outdated warning comes up. To stop the traffic, we end up blocking this user's access to their email until they fix their phone. Blizzard cannot do this for 1 million customers (easily).
Secondly, over commitment. I think Blizzard was a bit surprised in the number of concurrent users.
Going forward: Blizzard has gotten over the hump. The erratic release-week traffic has died down; I am back at the office now. We are already starting to see max-level 100s on the realms; saw 2 last night. Blizzard plays their cards right, we'll continue to see old players come-back to the game for about a year. After that, I would anticipate drop off of subscriptions if another expansion doesn't drop. They just need to make sure realm-performance maintains.
Thanks for the time Blizzard,