CIA's technology arm taps open source for enterprise search

IDG News Service - The company in charge of providing technology to the U.S. intelligence community has invested in an open-source firm to provide enterprise-search technology to the CIA and other intelligence agencies.

In-Q-Tel is investing in Lucid Imagination, which provides support, maintenance, training and add-on software for the Apache Software Foundation's Lucene and Solr search projects. Lucene is an information retrieval library that can be used for full-text indexing and search. Solr is an enterprise-search server based on Lucene.

The companies did not disclose the nature of the investment but said that it is aimed at making Lucid's open-source enterprise-search software more prevalent in the U.S. intelligence community.

Lucid officially launched in February after securing initial funding in October of last year, said Anil Uberoi, chief marketing officer of the fledgling company.

"You can think of us as the Red Hat of Lucene," he said, providing support and maintenance to customers who want to use Lucene and Solr for enterprise search. In fact, many of the leading developers who commit code to those projects are the founding technical members of the company, he said.

In-Q-Tel handles technology requisitions for about 18 organizations, including the CIA and the U.S. intelligence community at large, Uberoi said. Many of these organizations have wanted to use the Apache Lucene and Solr projects to do enterprise search, but were hesitant to do so without official vendor backing, he said.

"They were very nervous about not having a commercial entity to support this," Uberoi said.

Indeed, search analyst Stephen Arnold in a blog post said that several intelligence organizations already have been using Lucene and Solr for enterprise search, and having Lucid to back their investments gives it the green light for more mission-critical applications.

"With Lucid imagination, a well-funded commercial entity offering certified distributions of Lucene and Solr, SLA-based support subscriptions, training, high-level consulting and value-added software, both new and existing users now have access to enterprise-grade support and services to optimize their enterprise search efforts," he wrote.

According to Lucid, the Lucene/Solr technology is downloaded more than 9,000 times per day, and more than 4,000 organizations are using the software for enterprise search.