In previous posts, I wrote about the epic battles that are brewing between spammers and content farms—which are turning the web into a massive garbage dump—and search providers, which have to choose between profit and customer satisfaction. This is a serious problem. The content farms are “dumbing down” the web by churning out thousands of mostly low-quality articles, every day, on topics that Google tells them they can make money from. All of these players are raking in billions of dollars at our expense.
I had the opportunity to moderate a panel discussion this week between Google, Microsoft, and Blekko. The event, which I emceed, was called Farsight 2011: Beyond the Search Box, and was organized by BigThink and Microsoft. As I joked, it seemed odd that Google was playing the role of “evil” monopolist; Microsoft, the “good” contender, whilst Blekko was a fly on the wall.
I had originally invited Google SEO development head, Amit Singhal, and Microsoft Research GM, Ashok Chandra. But Ashok dropped out in favor of Corporate VP of Bing, Harry Shum; and Amit dropped out in favor of Webspam team head, Matt Cutts. I was very disappointed, because Matt has the reputation of being a really nice guy—a “teddy bear”. Harry has the reputation of being feisty. I was afraid that Harry and Rich Skrenta (Blekko CEO) would devour Matt. And it would seem as if I had set up an ambush for Google.
Little did I know that Google had its own ambush cooking: that Matt was more like a tiger than like a teddy bear.
At the conference, I gave my spiel on my vision of search: how I want my computer to serve me and tell me what I want to know, rather than my having to cater to its whims by entering specific keywords in a text box and reading through text links—which are often baited by spammers. I challenged Matt to tell everyone what Google was doing about the spam. Matt, instead, went on the warpath and accused Bing of stealing Google’s information. He disclosed a sting operation that his team had run. He expressed outrage at Microsoft’s ethics. Harry Shum fired back, defended Bing, and accused Google of playing games.
There has been extensive media coverage of this. Harry Shum and Yusuf Mehdi of Microsoft both posted blogs to respond to Google’s allegations. So I don’t need to visit the same territory. You can watch the video of the event and form your own opinions. There was a lot more discussed in 40 minutes than was covered by the media, so this is worth watching.
Both sides have strong views and believe they are right. In opening the debate, I said that, as a professor, I can’t condone any kind of plagiarism or cheating—and that is what Microsoft’s usage of Google data seems to amount to. But in the tech world, such information exchange is the norm. Everyone cheats and this may be a good thing for innovation. So there is no black and white here. Both sides are right and they are wrong.
The one thing that is clear is that Google pulled off a huge PR coup. It changed the topic. Media coverage isn’t about spam and how Google profits from this any more; we are debating how valuable Google’s search results are.
Here are the real issues we should be discussing:
- Who really owns the data that Google and Bing are tussling over? Is it the search providers—which “cheat” and copy from all over the web? Or is it the content creators—us—who they “steal” from? Why do Google and Microsoft believe that they own our information? And why aren’t they paying us for using this?
- Facebook rivals Google in web traffic and will get way ahead. And Google can’t search within Facebook’s walls. Doesn’t this give a huge, long-term, advantage to Bing, which can (within limits)?
- Blekko announced a bold decision to block content farms—sites such as eHow and Answerbag. Will Google and Microsoft take similar steps? Will they be able to forsake the revenue? Can the volumes of spam we are dealing with even be screened algorithmically or do we need curated search solutions?
- We need a standard measure of web quality. Google says that it has not noticed any reduction in web quality. Yet most experts agree that this has declined significantly over the past two or three years. Why doesn’t Google, as the market leader, work with its competitors to create an open measure that can be used by everyone? Let Google prove to us that it is, indeed, better than the rest.
- Why not allow web users to designate what sites are spam and make this information publicly available? Google lets you filter your own results, but why not share these data with everyone? Sites that believe they are unfairly labelled can lodge an appeal. Why the secrecy?
So let’s get back on topic. Harry Shum and Matt Cutts can duke it out in a bar somewhere. What I want is for them to clean up the web and give me the best search results.
Editor’s note: Vivek Wadhwa is an entrepreneur turned academic. He is a Visiting Scholar at UC-Berkeley, Senior Research Associate at Harvard Law School , Director of Research at the Center for Entrepreneurship and Research Commercialization at Duke University, andDistinguished Visiting Scholar at The Halle Institute for Global Learning at Emory University. You can follow him on Twitter at @vwadhwa and find his research at www.wadhwa.com.