Spies like us like blogs (and neural NOTwork)

Ssshhh! It's IT Blogwatch, in which spies start to adopt blogs, wikis, Web 2.0, and "open source intelligence". Not to mention how not to use a huge neural network...

Clive Thompson wrote this in yesterday's New York Times:

Throughout the intelligence community, spies are beginning to wonder why their technology has fallen so far behind — and talk among themselves about how to catch up. Some of the country’s most senior intelligence thinkers have joined the discussion, and surprisingly, many of them believe the answer may lie in the interactive tools the world’s teenagers are using to pass around YouTube videos and bicker online about their favorite bands. Billions of dollars’ worth of ultrasecret data networks couldn’t help spies piece together the clues to the worst terrorist plot ever. So perhaps, they argue, it’ s time to try something radically different. Could blogs and wikis prevent the next 9/11?


During the cold war, threats formed slowly. The Soviet Union was a ponderous bureaucracy that moved at the glacial speed of the five-year plan ... If an analyst requested information from another agency, that request traveled through elaborate formal channels ... Then on Sept. 12, 2001, analysts showed up at their desks and faced a radically altered job. Islamist terrorists, as 9/11 proved, behaved utterly unlike the Soviet Union. They were rapid-moving, transnational and cellular ... Al Qaeda operatives organized their plots in a hivelike fashion, with collaborators from Afghanistan to London using e-mail, instant messaging and Yahoo groups; rarely did a single mastermind run the show. To disrupt these new plots, some intelligence officials concluded, American agents and analysts would need to cooperate just as fluidly — trading tips quickly among agents and agencies. Following the usual chain of command could be fatal.

Ross Dawson's glad to see this topic getting attention from the mainstream media:

Spying ain’t what it used to be ... I have written repeatedly before about the rise of “open source intelligence” and how social network analysis tools are being used in the intelligence community. One of the most fundamental shifts over the last decades is the far greater availability of information (ranging across billions of websites, untold mobile camera photos, commentary and insights from millions of subject experts, through to the powerful purvey of Google Earth). For intelligence agencies, this dramatically shifts the central issue from gathering exclusive information, to making sense of an almost infinite amount of data, which is available to everyone.

The emergent properties of an effectively integrated community of blogs and wikis mean that the most relevant and important information floats to the top. These kinds of capabilities must be tapped by intelligence operations in order to filter and assess what is worth responding to and raising to the executive level. Linear report writing, editing, and escalation doesn’t have a hope of working effectively in this environment.

Chris Anderson jokes:

The average spam filter is apparently light years ahead of the state-of-the-art in military data analysis ... But the really sobering thing is Clive's reminder that this is not just due to incompetence and bureaucracy. US intelligence is also intentionally hobbled by laws designed to protect privacy, civil liberties and other excessive government intrusion into our lives. Total Information Awareness probably would have failed on technology grounds, but it never even got a chance to prove anything one way or another because it ran afoul of laws and politics first. Since we think those laws are Good Things, Clive asks whether a more open alternative -- the Web 2.0 tactics of peer-production and the wisdom of crowds -- within the intelligence services are a better approach.


I was surprised that Clive didn't take the question to the obvious next level. What if, rather than just encouraging blogs and wikis behind military firewalls, they encouraged them out in the open? ... In other words, what if spotting early-warning terrorism signals became an open-source project?

Will Richardson groks the education angle:

Forget the ways in which the tools enhance learning, communication skills, literacty skills and all that educational stuff. Our students need to learn blogs and wikis FOR THE SAKE OF NATIONAL SECURITY! What principal, what school board, I dare say what community could argue against that? ... We really could be at a tipping point of some type

Marc Hedlund argues semantics:

The use of "Open Source" in the title (adopting the phrase from earlier writing on the same topic) is annoying; a "source" to the CIA is much different than the "source" in open source. "Harnessing collective intelligence" is exactly what they're talking about, though. I was surprised to find no reference to tags in the article, since that seems like exactly the solution to the problem statements the article poses ... In the case of Flickr, tags allow people to describe an opaque object (a photograph) that we don't have good tools for searching textually otherwise. In the case of intelligence gathering, the object being searched is opaque for other reasons, usually clearance reasons -- to protect a source, for instance ... a tag could be used to describe the opaque object here, too, and that people could subscribe to particular tags of interest. Where permissions (clearances) are unclear or blocked, tag activity analysis (coincident tag usage, tags appearing across widely separated sources, and so on) might expose areas where information sharing should be allowed to occur.

Andrew McAfee calls it, "Very encouraging news":

It appears that many of the most important people at the top of the recently-established Office of the Director of National Intelligence (DNI) sincerely want [Enterprise 2.0] to take root. These people include the CIO, the head of analysis, and his CTO; I imagine these gentlemen have some clout, and know how to get things done within the establishment ... They're certainly willing to solicit and try new ideas.  the DNI has experimented with blogs and wikis, including an 'Intellipedia', and sponsored the Galileo Awards, where analysts could submit essays describing new approaches.


It also seems the DNI is making some smart choices with its E2.0 initiatives.  They're building a three-tier Intellipedia, for example, that will mirror the existing access levels of Top Secret, Secret, and Unclassified.  I hope they're ensuring that searches in the more tightly restricted environments can also return results from the lower levels, and that some form of tagging exists. 

But Mathew Ingram sounds a note of caution:

Most of the research that has been done on the attacks of September 11, 2001 has shown that many of the pieces were there to indicate that something serious was in the works, but no one put them all together ... What better way to get people sharing information than with wikis and blogs?

I can almost hear Web 2.0 skeptics like Nick “The Prophet of Doom” Carr and Andrew “Web 2.0 is Communism 2.0″ Keen snorting with derision at this idea. Blogs and wikis for spies? What will they think of next. But Clive’s story deals with the downside of social media as well — including the issue of getting people to actually use the tools when they are available. Spies are notoriously secretive, even when dealing with other spooks. How do you get them to share?

Buffer overflow:

Around the Net Around Computerworld Previously in IT Blogwatch

And finally... Neural NOTwork

Richi Jennings is an independent technology and marketing consultant, specializing in email, blogging, Linux, and computer security. A 20 year, cross-functional IT veteran, he is also an analyst at Ferris Research. Contact Richi at blogwatch@richij.com.

Copyright © 2006 IDG Communications, Inc.

Shop Tech Products at Amazon