A Billion Queries A Day Later, Twitter Finally Working On Search Again

We just noted that Twitter quietly rolled out an entirely new search backend over the past few weeks, and nobody seemed to notice. That’s actually great news, because it means nothing went terribly wrong in the transition. But now comes the real challenge for Twitter: making search useful once again.

Twitter Search has long held the promise of something great. Original born as an independent company, Summize, when Twitter bought them in 2008, it seemed clear that they were about to get serious about search. But as scaling problems continued and escalated, and as the user base continued to grow, search took a backseat. I would bet that if you polled Twitter users now about one thing they’d like to see improved, it would be search.

As it stands, Twitter search is fine, but it’s way too basic. Yes, there is an advanced search area on the stand-alone search site, but few users probably know about it — and even less probably use it. Searches are done on twitter.com, and for those, you’re basically just getting a reverse chronological listing of tweets. Sure, this has been augmented over time by Top Tweets and Promoted Tweets, but it’s still the same basic idea.

But here’s what Twitter wrote today in announcing the new search architecture:

But you might wonder: Fine, it’s faster, and you guys can scale it longer, but will there be any benefits for the users? The answer is definitely yes! The first difference you might notice is the bigger index, which is now twice as long — without making searches any slower. And, maybe most importantly, the new system is extremely versatile and extensible, which will allow us to build cool new features faster and better. Stay tuned!

Key words: Cool. New. Features.

Twitter revealed some crazy stats about search today. They’re now seeing over 12,000 queries per second, which means more than a billion queries per day. That’s a little misleading because things like Trending Topic clicks are searches too — but still, it’s a massive search engine. And while Twitter has deals to send their data to the big boys — Google and Bing — the company has indicated in the past that they may start making moves that move them much closer to the traditional search engines. With each link shared on Twitter now passing through their own t.co shortener, Twitter knows a lot about what is being shared, and that could be very valuable in an improved search.

And, as we’re all very well aware, search is highly monetizable. Twitter clearly has the volume to go there if they want (they sort of are with some Promoted Products). That’s something the new CEO surely must be thinking about.

But most of all, people just want a way to access their entire history of tweets. Sadly, right now, we have to use a third-party tool to do that (incredibly, I’m still using the long-dormant FriendFeed as my Twitter search tool). With this new backend change, Twitter says the index is bigger and twice as long — but we want it all. Hopefully Twitter is finally gearing up to give us just that with search.