A new study from the Pew Research Internet Project appears to show that Internet users know very little about the Internet they use.
What the study (PDF) actually shows is that Pew researchers got confused about the difference between information and understanding.
The survey, which the Pew Center took in September, was designed to measure the "Web IQ" of Internet users based on the response of 1,066 Internet users to 17 questions about tech-related topics that may or may not have had anything to do with how well respondents understood the Internet or how competent they were at using it.
Some of the 17 questions (take the quiz) Pew asked were right on target. Asking whether the Internet and the Web are the same thing does test how well a user understands that one is a global tangle of datacenters, networks, telecom wires and troll dens. The other is a graphical interface designed to let users get to all those resources without having to learn how to navigate the hardware and networks in between first. Only 23 percent knew the difference.
Questions Pew asked about whether a PDF can be sent by email and what "URL" stands for also show something about how competently respondents use the Web. Questions about Moore's Law and net neutrality went a little farther afield, but touched on drivers of growth and points of conflict non-technical users might easily have ignored.
Pew Research Center How, though, does asking survey respondents to identify a picture of Facebook COO Sheryl Sandberg demonstrate their competence or understanding of the Internet? She's a major force in social networking, certainly, but not so influential or widely known that knowing her face would be a better indication of Web savvy than, say, the ability to find a research report online and know whether or not to trust its results.
Would it be more important for users to tell Pew that a wiki is an editable collaboration tool than it would be for them to know how to use firewalls and antivirus to avoid being hacked, scammed or (to some extent) surveilled while online? Would recognizing a wiki be a better recommendation than knowing how to find research online and use it for work projects without violating copyrights, patents or licenses when the result was posted?
Sure, being one of the 60 percent who knew Twitter messages are limited to 140 characters would be useful, but if you run over, Twitter will just turn the extra letters red and refuse to tweet until you trim your tome. It won't betray your trust and sell your deepest secrets to marketers bent on exploiting that knowledge at your expense, as might happen you were among the 66 percent who didn't know that a privacy policy doesn't mean a Web site will keep what you tell it confidential.
What was the first university on Facebook? What year was the first iPhone released? What was the first popular Web browser? Who cares? If your goal is to find out what users understand about how the Internet works and how effectively they use it right now, how would knowing which piece of software was first to lose the browser wars move you closer to that goal?
It's interesting that 83 percent of Internet users recognized a picture of Bill Gates. But is recognizing the former chairman of a monopoly unseated because it was slow to evolve as important as recognizing information skewed to serve an ulterior motive, a download likely to be a virus or a scam dressed up as a job offer?
Knowing that hashtags are widely used on Twitter is not the same as knowing how stupid it sounds to say hashtags out loud. #shutupalready
Pew's Web IQ – or "Web IQ" as Pew used it in most instances, which didn't reinforce its credibility – makes a decent headline and interesting looking infographic.
It does not do much to demonstrate how well Internet users understand or use the Internet, however. It fails for the same many arguments fueled by frantic Google searches fail: too much data, not enough relevance or understanding.
The fault is not entirely Pew's. Facts are so easy to come by online that it's easy to fall into the geek-superiority trap of assuming that the winner of any argument is the one who is first to vomit a critical mass of facts on an opponent regardless of how accurate or relevant any of them may be.
Pew, a non-partisan think-tank that isn't tied to a particular vendor or technology, which makes it more objective than most sources of tech-related survey data, is one of the most consistent sources of high-quality research and analysis of Internet usage and digital media. When I saw headlines about ignorant users I was disappointed, but didn't doubt the report's conclusions, especially given the piles and piles of research showing that end users know little and care less about a lot of issues geeks take very seriously.
The focus of this particular survey was so divided by its attention to pop-tech elements of the tech world that its authors forgot that knowing when the iPhone came out has nothing to do with knowing how to email a PDF and the ability to recognize tech-industry executives as if they were movie stars says bad things about one's tendency to idolize celebrities, but nothing good about one's progress toward the ninth circle of geekishness.
The result is a survey that asks so few significant questions and dilutes them by giving equal weight to so much trivia that it doesn't provide enough information to draw much of any conclusion, let alone the obvious one.
The existence of a fact does not guarantee that it is relevant; possession of a fact doesn't imply deep understanding of its context or impact. Understanding its impact does not automatically indicate the ability to mount a competent response.
Demonstrating you have a few bits of genuinely useful information scattered among shards of trivia (as in the Web IQ test) does not indicate mastery of the Internet. It indicates that you are a victim of the 'net's tendency to drown investigators in nearly significant almost-facts while hiding the good stuff behind paywalls, bandwidth limits, dead sites and impenetrable headlines.
If Pew wanted a test that would show how well people navigate tons of useless information to find the bits they actually need, it shouldn't have asked Internet users the questions behind "Web IQ."
They should have asked what users think the difference is between a question that shows someone really understands the Internet, mobile technology and geekdom in general, and which look more like a list of search terms than an effort to find a real answer.