The Web search arena just became a little more crowded as Amazon.com has announced that it is opening programmer access to its Web crawler, Alexa, for a small fee.
Heretofore, Alexa has been known more for traffic rankings than searches, but the newly unveiled Alexa Web Search Platform will give developers  access to online tools for searching, indexing, publishing, computing , and storing Internet data .
"Today, Alexa is releasing the Alexa Web Search Platform Beta, effectively opening up the Alexa Web Crawl and ushering in a new era where anybody can create new search services without having to invest millions of dollars in crawl, storage , processing, search, and server technology," read a post by Geoffrey Mack, product manager at Alexa, on the Alexa Web site.
The price for each feature offered by the service is $1. Developers can rent CPUs by the hour, store 1 gigabyte of data per year, process 50 gigabytes of internal data, upload or download 1 gigabyte of data, or publish up to 4,000 new searches using Alexa's self-service publication system , all for $1 each.
Plenty of Information
Alexa has been crawling all publicly available sites on the Internet since 1996, gathering information to create what it calls snapshots of the Web. The collected data is then used to create services, such as traffic rankings.
Alexa claims to gather approximately 1.6 terabytes, or 1,600 gigabytes, of online content every day. That works out to four to five billion pages per month. It takes the company roughly two months to complete a snapshot of the Web for which Alexa will have gathered some 4.5 billion pages from more than 16 million sites.
"Since 1996, Alexa has been crawling and storing the Web at millions of pages per day," Mack's blog post continued. "Alexa has also been building out the infrastructure to store and analyze the data and serve it to toolbars, browsers, and Web sites worldwide. Now, all of that infrastructure is yours to use via the Alexa Web Search Platform."
Niche Searches Take Off
(continued...)
|
