Wikia Search is a new, wiki-inspired search-engine project that attempts to create a transparent set of ranking algorithms that fight spam and promote good stuff to the top. This is in contrast to Google, Yahoo, and other search engines, where the ranking algorithms are treated as trade secrets and high-risk tactics that have to be guarded from spammers.
The idea of a ranking algorithm is that it produces "good results" -- returns the best, most relevant results based on the user's search terms. We have a notion that the traditional search engine algorithm is "neutral" -- that it lacks an editorial bias and simply works to fulfill some mathematical destiny, embodying some Platonic ideal of "relevance." Compare this to an "inorganic" paid search result of the sort that Altavista used to sell.
But ranking algorithms are editorial: they embody the biases, hopes, beliefs and hypotheses of the programmers who write and design them. What's more, a tiny handful of search engines effectively control the prominence and viability of the majority of the information in the world.
And those search engines use secret ranking systems to systematically and secretly block enormous swaths of information on the grounds that it is spam, malware, or using deceptive "optimization" techniques. The list of block-ees is never published, nor are the criteria for blocking. This is done in the name of security, on the grounds that spammers and malware hackers are slowed down by the secrecy.
But "security through obscurity" is widely discredited in information security circles. Obscurity stops dumb attackers from getting through, but it lets the smart attackers clobber you because the smart defenders can't see how your system works and point out its flaws.
Seen in this light, it's positively bizarre: a few companies' secret editorial criteria are used to control what information we see, and those companies defend their secrecy in the name of security-through-obscurity? Yikes!
The Wikia Search project has assembled the basic technologies for a search engine, including a search application, search algorithm and Web crawler. The project will allow technology enthusiasts to help filter sites and rank search results, using a community model akin to that of Wikipedia.
The idea is to challenge the established players by offering a search service that is more transparent to end users, meaning they can see how search results are arrived at. Wales has described Yahoo and Google as opaque services that don't explain how results are arrived at.
(Disclosure: Jimmy Wales and I are writing a book together about a related subject)
I did an interview with the Changelog podcast (MP3) about my upcoming talk at the O’Reilly Open Source conference in London, explaining how it is that the free and open web became so closed and unfree, but free and open software stayed so very free, and came to dominate the software landscape.
I’ve been writing “design fiction” for years (see, for example, Knights of the Rainbow Table), and when people ask me to explain it, I say something like, “An engineer might make a prototype to give you a sense of how something works; an architect will do a fly-through to give you a sense of its […]
Sweden’s ruling coalition of Social Democrat and Green parties has a tax plan that will make it cheaper to fix broken things and more expensive to buy things that can’t be fixed after they break. From FastCo: The proposed legislation would cut regular tax on repairs of bikes, clothes, and shoes from 25% to 12%. […]
CloudPress is a responsive WordPress theme builder that allows you to create a whole site in less than 30 minutes. CloudPress comes with tools like pre-built headers, content blocks, and footers—all you have to do is pick what you like, and drag and drop. With your subscription, you get access to 13 professionally designed WordPress themes, over 80 […]
If you own a dog, you’ve most likely heard of BarkBox – the monthly subscription box for dogs. What started as a simple idea to try out the subscription model on pet owners has since developed a cult following of dog lovers. If you haven’t given it a try yet, this one month free deal is the […]
With the iPhone headphone jack having gone by the wayside, we’re excited about the addition of the FRANKLIN Bluetooth Headphones in our store. These headphones are foldable so they’re easy to carry around, but most importantly, they pack impressive sound. Our biggest struggle with Bluetooth headphones is the worry of them dying at the worst moment. This pair lasts an impressive 8-10 […]