Google CSE – Why It May be the NBT

Google CFE is the acronym for Google’s new Custom Search Engine capability.  NBT, of course, means the Next Big Thing – the holy grail of entrepreneurs and VCs alike.

Bill Burnham does a great job of explaining why you wouldn’t want to get into the basic search engine business today (unless you’re Microsoft and Yahoo and can’t get out).  He points to a NYTimes article about Google’s huge new server farm on the Columbia River.  The obvious reason for it’s being there is lots of cheap (and green) electricity although parts of the river are also known for great windsurfing.  Anyway, you don’t want to have to build one of these things and then see if you can get a few folk to try your new search engine.

Bill blogs:

“This new reality is that search innovation will increasingly be about applications and not about the core infrastructure.  In fact, there’s a good chance that much of the core infrastructure will be available as a service to search-based applications.  Amazon is pioneering this “search as a service” with its opening up of the Alexa crawling and indexing APIs.  While in the blog world, Technorati already provides APIs for developers to integrate their blog search results directly into applications.   Even Google appears to be taking tentative steps in this direction with Google Co-Op which enables users to limit Google’s Search engine to a customized set of sites and keywords.”

My only disagreement with Bill Burnham on this is that I think Google Co-op is much more than a tentative step.  It may be the Windows 1.0 of search engine infrastructure (it IS pretty buggy and parts are brain dead).  Two things make Google CSE very significant: 1) it come from Google; 2) it has an interface which can be used quickly by non-technical people to build CSEs.  It also has an API for us nerds which I haven’t even played with yet because it’s so tempting to just goahead and build CSEs using the web-based forms and cutting and pasting a little HTML.

You get to be a platform by attracting developers.  Duh, everyone knows that.  But if Google can attract bloggers who don’t code and get zillions of CSEs built quickly on its platform, Google will have an even more substantial advantage in search than “merely” having the most hardware and the most users and being the current verb used for searching the web.

My guess is that the Google APIs will be used mainly by tool developers to make it possible for the vast mass of people who’d just as soon NOT hack to build better and better search engines on the Google platform.  You REALLY win as a platform when all the tool developers concentrate on making tools to make it easier to use your platform and leave your competitors to try to develop all their tools on their own.

A very smart thing that Google has done is to encourage collaborative creation of search engines.  You can invite up to a hundred of your closest friends to add sites and refinements to your CSE.  You can accept an unlimited number of volunteers to add the wisdom of the crowd to your CSE.  There are some primitive tools for managing a collaborative effort which’ll have to get better.

So what Google is doing is getting the creators of CSEs like me to recruit others to create applications for their platform.  Fiendishly clever.  I think it’ll work.

Michael Parekh, to whom  I owe a hat tip for pointing me to Burnham’s post, thinks that some day something’ll come along to dethrone Google as the great search platform.  “As insurmountable the obstacles seem to a new company being able to do search with the kind of pervasive adoption and economic success of a Google, I can't help but feel that someone can find a way,” he writes.  “…I mean who would invest the hundreds of millions or billions it would take to go up against Microsoft?” he continues tongue firmly in cheek.

I do agree with Michael.  Someday someone’ll overtop Google.  But I don’t think it’ll be by building a bigger server farm or any other kind of frontal assault.  Much more like Linux and Windows. Google could be vulnerable to a distributed search engine that runs on the machines of a vast number of willing users and does NOT demand ad space.

But that’s way in the future – maybe next year.  Meanwhile I’m working on my CSEs and you should be too.  Your website’s gonna need one.  Or you could start a business that leverages the coming success of CSEs.

The first post in this series is about an experiment with a closed CSE; the second post is about drafting bloggers in the My Way blog network to contribute to a collaborative CSE (already the majority of them have joined up and are actively improving the search).