Amazon is selling access to its 5 billion document, 100-terabyte web-index. The index is complied by Alexa, an Amazon division that also powers the Internet Archive's Wayback Machine. Amazon is renting access to the whole, raw database, so that you can build your own search tools and data-mining projects with it. Pretty awesome — indexes are hard to build and maintain, requiring a lot of computer horsepower, storage and bandwidth, but once you've built a copy of the Web, there's plenty of imaginative ways you can tweak it to produce valuable new services. By treating the index as a saleable asset instead of a trade secret, Amazon is really ripping apart the traditional wisdom of search engines:
Anyone can also use Alexa's servers and processing power to mine its index to discover things – perhaps, to outsource the crawl needed to create a vertical search engine, for example. Or maybe to build new kinds of search engines entirely, or …well, whatever creative folks can dream up. And then, anyone can run that new service on Alexa's (er…Amazon's) platform, should they wish.
It's all done via web services. It's all integrated with Amazon's fabled web services platform. And there's no licensing fees. Just "consumption fees" which, at my first glance, seem pretty reasonable. ("Consumption" meaning consuming processor cycles, or storage, or bandwidth).
The fees? One dollar per CPU hour consumed. $1 per gig of storage used. $1 per 50 gigs of data processed. $1 per gig of data uploaded (if you are putting your new service up on their platform).
(Thanks, John!)
