QuickStudy: Web Harvesting

By Russell Kay

June 21, 2004 12:00 PM ET

Computerworld - It's hard to argue with the proposition that the World Wide Web is the largest repository of information that has ever existed. In just over a decade, the Web has moved from a university curiosity to a fundamental research, marketing and communications vehicle that impinges upon the everyday life of most people in the developed world. But there's a catch, of course. As the amount of information on the Web grows, that information becomes ever harder to keep track of and use.

More
Computerworld
QuickStudies

This vast amount of freely available information is spread over billions of Web pages, each with its own independent structure and format. So how do you find the information you're looking for in a useful format—and do it quickly and easily without breaking the bank?

Search Isn't Enough

Search engines are a big help, but they can do only part of the work, and they are hard-pressed to keep up with daily changes. For all the power of Google and its kin, all that search engines can do is locate information and point to it. They go only two or three levels deep into a Web site to find information and then return URLs. They also find and return meta descriptions and meta keywords embedded in Web pages, but these may well be inaccurate.

Consider that even when you use a search engine to locate data, you still have to do the following tasks to capture the information you need:

Scan the content until you find the information.

Mark the information (usually by highlighting with a mouse).

Switch to another application (such as a spreadsheet, database or word processor).

Paste the information into that application.

A better solution, especially for companies that are aiming to exploit a broad swath of data about markets or competitors, lies with Web harvesting tools.

Web harvesting software automatically extracts information from the Web and picks up where search engines leave off, doing the work the search engine can't. Extraction tools automate the reading, copying and pasting necessary to collect information for analysis, and they have proved useful for pulling together information on competitors, prices and financial data of all types.

Comment Recommend Digg Twitter ShareThis

Email Print Send Feedback Reprints

Additional Resources

Xerox

Win a color printer!

By using solid ink technology only from Xerox, you could save up to 65% by printing color for the cost of black and white. Enter for a chance to WIN a PhaserTM 8860 network color printer!

Microsoft

Upgrade to Internet Explorer 8 today.

Save time and mitigate security risk. Deploy it now.

Sybase

NEXT-GENERATION MOBILITY PLATFORMS

In this white paper, IDC analyzes the role of next-generation mobile enterprise platforms as organizations seek a more strategic deployment of mobile solutions.

Learn the important issues you must consider before starting your next mobility initiative. Get your mobility white paper from IDC now, compliments of Sybase.

What People Are Saying

1 comments | Add new

See all comments (1) | Add new

White Papers & Webcasts

Accelerate SSL Encrypted Applications
The amount of SSL traffic is growing in the enterprise. Because it is encrypted, it cannot be properly controlled and accelerated. Blue Coat...

Usability Is Everything
Learn what sets Workday's HR and Payroll solutions apart from the competition....

ESG Lab Field Audit
Many companies have successfully implemented Riverbed WAN optimization solutions within their Cisco networks. This ESG Lab Field Audit document explores the success that...

The Value of Real SaaS at Workday
Cost savings, speed to value, and innovation brought to the enterprise by Workday's software-as-a-service solutions for HR and Payroll....

Shape Your Apps Strategy to Reflect New SaaS Licensing and Pricing Trends
Why are smart companies choosing software-as-a-service? Find out in the complimentary Forrester Research report...

SaaS at Flextronics, Inc.
Dave Smoley, CIO of Flextronics, discusses the real value of software-as-a-service and why he chose Workday for his HR solution....

Natural User Interface for Enterprise Applications
Learn how a revolutionary user interface can make a complex enterprise application so intuitive even casual users can jump right in....

Why Compliance Pays
This OnDemand webcast explores the relationship that firms with best compliance records have higher revenue, greater customer retention, lower financial losses from data...

A Truly Global HCM System
Learn about a system built with advanced object-oriented technology that support multi-national requirements and costs less to implement, maintain and upgrade....

Agile Enterprise Content Management (ECM) for Rapid ROI
Find out how combining ECM and BPM will help adress issues about content rich business processes....

All White Papers | All Webcasts