Metrico Wireless, a mobile testing firm, just released a comprehensive measure of how the iPhone 4S performed on the three U.S. carriers: iPhone 4S data speeds, Web browsing fastest on AT&T, test shows AT&T did well on the data speeds, but Sprint and Verizon Wireless had plusses in other areas as well.
The results were based on thousands of Metrico lab tests and field tests in several cities, looking at voice quality as well as data speeds and Web page load times, among other factors.
Metrico must be commended for its thoroughness and its reserve in refusing to say one carrier lost or won. That's a lesson for the news media and instant testers who buy a new phone and walk around town to run a few speeds tests. I've done those kind of quick tests myself, so I'm certainly guilty.
But even with Metrico's thorough test, what have we really learned? If Sprint is indeed slower on data download and upload and Web page load times with its iPhone 4S, does that mean I should return my device and go to AT&T or Verizon Wireless instead? If i did do so, at the cost of a restocking fee, and all the other penalites and time wasted that I might incur, I might be sacrificing Sprint's superior voice quality discovered in the tests.
As Metrico pointed out, some users favor a smartphone for data over voice, and vice versa, so knowing how a carrier is strongest or weakest might be helpful to a buyer. But probably not that much.
The reason I say probably not is that wireless consumers don't switch carriers that often, using the old farmer's logic that the broken tractor you know is better than the tractor you are considering buying that you don't know. Consumers also base wireless service considerations on cost and other factors that matter as much as network performance.
Still, the carriers do fight fiercely over churn, and care greatly who leaves their service for another carrier and why. You would think that was their main concern if you ever got five minutes with one of their CEOs.
In that sense, the Metrico report on iPhone 4S performance on the three carriers is probably more useful to the carriers and their business partners than to average smartphone consumers. With a report like Metrico's in hand that denotes a carrier has a faster data speed or better voice quality, on average, it is possible for the carrier to make broad claims in marketing and TV ads. It is possible, therefore, to appear better than the competition.
Metrico's five unnamed cities for its testing are puportedly representative of the entire country. But even if Metrico would have gone to 100 cities, the results wouldn't be that useful to the individual wireless customer who lives in an underserved wireless area, or a crowded one.
There is always going to be a busy cell tower that will slow down speeds, and it will probably happen at the most urgent time that somebody needs a connection, like when a disaster occurs, or a babysitter has an urgent request. When cable TV first became popular, if a very short outage occurred during a SuperBowl game or other popular event, the consumer reaction was pronounced; all the other positives of the cable provider were wiped out.
Most of us are logical enough to realize that having higher averages in anything is better than not. Still, the best way to judge a smartphone's network services is to see how it works where we live or work.
If you ask the person in the next cubicle how smartphone X works on carrier Y or how they both work for the neighor next door, you are bound to have far more insight than any national test of averages done by the most sterilng research firm.
This insight should seem obvious, but maybe it's not.
The other rather obvious reality overlooked in network tests is that many buyers will want an iPhone 4S no matter which carrier they choose. The phone is also a computer that does a range of things on Wi-Fi or disconnected from the network entirely.