Got Microsoft Exchange? Earlier, I started talking about how, at its Tech-Ed IT-fest, Microsoft last week offered some fascinating insights into its internal use of Exchange. Today, I'll talk about how Microsoft IT has ditched traditional high availability: using no RAID or clustering.
If you've not already done so, you'll probably want to check out part one of this series. Go ahead, I'll wait...
Alternatively, here's a recap of Microsoft IT's new strategy highlights:
- Not using disk arrays, but direct-attached storage
- Not using RAID or clustering, but continuous replication onto spare servers
- Not using tape backup, but cheap disk arrays
Today, let's look closer at the second of these radical differences from MSIT's previous server architecture...
No RAID or clustering: As if using direct-attached SAS drives wasn't surprising enough, the next Microsoft IT revelation raised even more eyebrows. For a mission-critical, storage-centric application such as Exchange, not to use RAID is, on the face of it, a bizarre decision.
What MSIT has done is to take advantage of the maturation of Exchange's continuous-replication functionality; specifically Standby Continuous Replication. Each message store server is continuously replicated to a hot-spare server.
I forget if MSIT said these hot-spares were in a separate data center, but this would make sense for resilience. The ability to replicate over longer distances is one of the main points of SCR. So having replicated hot-spares permits MSIT to not bother with RAID or traditional shared-storage clustering. There are also other benefits, as we'll see in part 3 of this series.
MSIT lays out the stores so that, per 8-core Xeon server, about 11,000 users' mailboxes are partitioned into 35 databases, each designed to fit onto a single 1TB SAS drive. The 35 drives are connected to an array controller -- but in JBOD mode (no RAID, remember?), which makes each drive appear as a separate mount point.
Users are restricted to a 5GB quota; if you're good at mental math, you'll see that MSIT's planning assumption is that the average user will keep about 3GB of data. In practice, this also leaves enough room for about 30 days of deleted items. MSIT's experience is that each of those servers will on average suffer one single drive failure per year. So over their expected three year lifetime, they'll need to replace about 10% of the drives. When a drive fails, that store database fails over to the replica server. MSIT claims that availability is better than four-nines (i.e., less than an hour of downtime, total, per year). It's amazing how Exchange has improved over the years. I remember the days when it seemed as though MSIT could barely keep the servers up, despite restricting users to just a few hundred MB of data.
In the final part of this series, I talk about how MSIT back up their servers without using tape.
How about your mailbox role storage architecture? Leave a comment below...
|Richi Jennings is an independent analyst/consultant, specializing in blogging, email, and security. A cross-functional IT geek since 1985, you can follow him as @richi on Twitter, pretend to be richij's friend on Facebook, or just use good old email: TLV@richij.com.|
You can also read Richi's full profile and disclosure of his industry affiliations.