Inside the Google Web API -- The Developer Appeal

It's been six months since the release of the Google Web API, and at least 10,000 developers (Java, .NET, Open Source and other enterprise disciplines) have downloaded the code and started to build applets and services. Integration Developer News spoke with Nelson Minar, one of the lead Google engineers on the project, to get an up close sense of how one of the biggest splashes in Internet-based web services to date is doing with the developer community.

Tags: Google Web, Google Web APIs, Developers, Java, SOAP, Perl, Programming,

with Nelson Minar
Engineer for Google's Web API

In the few months since the release of the Google Web Services API, the engineers behind the project seem to have proven one thing: There's a sizable pent up demand among developers for an interesting online web service to play with.

Integration Developer News recently spoke with Google software engineer Nelson Minar, one of the small handful of engineers and developers responsible for one of the first-launched, and best-known Internet-based web services -- The Google Web API. In our extensive interview, Minar told IDN that more than 10,000 Java and .NET developers downloaded the API, in its first week alone, last April. He also said that a wide range of developers are using the Google Web APIs for private, educational and commercial use.

IDN: Who came up with the Google API idea?

Minar: The idea was around in the company for a long time. For a couple of years, people have been asking for something like this.

IDN: So, you'd been thinking about web services and SOAP for a couple of years?

Minar: The fact that it's SOAP right now, is sort of because SOAP is the technology of the day. But the idea of an API to Google is an old idea. In fact, some people had found that we had one, but it was really only intended for partners. [The earlier API] was not appropriate for arbitrary use. So we thought we should should build something that anyone can use.

IDN: When did you start working on the API?

Minar: Only a few months [before its release]. Google is not very specific about engineering schedules.

IDN: Has the project evolved in scope since you conceived of it?

Minar: It's really focused on the search. And that is the core API - the search API is the one we put the most work into. We split the spellchecker API into a separate API because from a functional point of view, they're very different. Normally, when you come to Google, Search and Spellchecking are done in a single operation. From a user-interface point of view, it's good that they're combined the way that they are. But when you're writing a program, you probably want to write them differently. So that's why we split them.

IDN: April 9th was the date you released the Google API developer would you characterize the response since?

Minar: I think the response has been really great. I think that there are a lot of people who think it's cool. But, I haven't sat down and sifted out the people who think it's cool because it's a web service and they just want to play with it, versus the people who are like, "Oh I can build a business out of this." I just haven't looked closely enough to know the difference between them.

IDN: Are there more or less people using the Goggle APIs then you expected?

Minar: More -- actually, a lot more. This is mostly because I thought [the Google APIs] would be a service that only people who knew a whole lot about Web services would sign up for. So the one number that I did kind of get a kick out of, and is on the record, is that over 10,000 people in the first week downloaded the kit.

IDN: That's the number of people who downloaded the developer kit?

Minar: That's the number of people who signed up for the service. I thought that a few hundred Perl developers would try it out in the first week, right? But I think that there are enough people who understand web services programming and enough interest in what Google does, that the demand is pretty high and will remain so.

IDN: What is the most popular language?

Minar: I don't have any direct way to measure that. I would say that Java and .Net are probably leading, and that might very well be because those are the two strongest examples that we provided. We have a funny little Perl example, too. But we spent a lot of time on the Java and .Net examples, so it might just be that they're using what we encourage them to use.

IDN: What about commercial users? Are they coming forward?

Minar: One of the things that we asked people [that sign up] is, "If you have a commercial use for this stuff, let us know. Write us a note; tell us what you want to do." And we're now looking at how we can support those needs.

IDN: How would you characterize a typical commercial user at this early point?

Minar: I can't characterize it because it's all over the map. We have some people who are very polite and asking for permission to do something that is probably not an issue -- like a school that wants to do something. And then there are a couple of big companies that have interesting applications [that] fit into their business needs; and then there is everything in between.

IDN: Can you give us any better feel for how commercial developers are reacting to your APIs?

Minar: I will say I don't think there's anybody who's betting their company on this [Google API] service. And given that we're a beta service and we've said, "This is an experiment," I wouldn't expect anybody to do that. And I think it's going to take awhile before web services is mature enough that a company that's betting on using a web service is going to be a success.

IDN: Have any of the early Google API applications really grabbed your attention yet?

Minar: Well the "Professor-Verifier" application got my attention, mostly because it came out of left field. It was just not something I would have ever thought to do.

IDN: How does it work?

Minar: The idea is this: if you want to know if somebody who claims to be a professor at a school is actually a valid professor there - you have Google look up the professor's name and the university you think they're at. If you get a bunch of search results, then the professor is probably there, and if you get no results, then they're probably not there. It's not a 100% verifier, but it's a good first cut. So there's this funny use of Google as kind of a search engine that does more than just finding a web page.

IDN: How large an effort was it to create these tools?

Minar: I would say it was a small effort. It was not 20 people working around the clock. The nice thing about this SOAP stuff is that it is actually pretty easy, and there are examples for most of the common languages, mostly because there is enough developer interest [in Java, .NET, and Perl] that there are articles already written about the technology. So you can sort of start with an example and repurpose it. Where it took some time, is where you [need to] find a bug in your own file or your own service, or occasionally you find a bug in the client, and then you have to figure out what's wrong. But when it works right, it's pretty easy.

IDN: What are the chances of the other Google services becoming exposed through APIs?

Minar: I think it's really a matter of, "Is there enough demand for us to do the service." It costs us money to put these things out there, so we want to do them only if there are enough interesting uses of it to make it worth the time. And we're looking at that now. What we did is the thing a lot of companies ought to do, which is you pick your core value and you offer it as a service, because you know how to do that. And then you see what happens. If that's successful, you can branch out from there.

IDN: What do you think are the most important things for the development of web services?

Minar: In fact, there's not a whole lot that needs to be done for the basic stuff to work. The basics are working now. You can actually use this stuff. It's not experimental; it works. There's a little work to do on WSDL interop [which demonstrate Web Services Description Language interoperability between SOAP-based web services toolkits] I think this security thing is an interesting problem, because doing it well is difficult. There's a big commercial interest in that problem being solved, so I'm pretty confident that it will be.

IDN: What about Representational State Transfer (REST)?

Minar: I think REST is cool.

IDN: So when do you think web services will be real?

Minar: I think we're already seeing commercial web services out there, but a lot of them are sort of enterprise-to-enterprise integration and they happen to be using SOAP and HTTP. It's not very sexy; it's not consumer-oriented, but it's happening already. The core plumbing technology is pretty mature. I think beyond that, it's much more of business and application question than it is a technology issue.

Top 10 Tips to getting started with Google Web APIs

Interested in learning more about how the Google Web APIs might be useful to your projects, and how to get started? Here are IDN's "Top 10" from the Google FAQ list, which will give developers a quick tour of the Google API technologies, and tips for using the associated developers' kit with Java, .NET and Perl.

What are Google Web APIs used for?
The Google Web APIs service gives you query access to Google's web search, enabling you to develop software that accesses billions of web documents that are constantly refreshed. Possible applications include:
· Issuing regularly scheduled search requests to monitor the web for new information on a subject.
· Performing market research by analyzing differences in the amount of information available on different subjects over time.
· Searching via non-HTML interfaces, such as the command line, pagers, or visualization applications.
How does the Google Web APIs service work?
Developers write software programs that connect remotely to the Google Web APIs service. Communication is performed via the Simple Object Access Protocol (SOAP), an XML-based mechanism for exchanging typed information.

What functionality comes with Google Web APIs?
Developers can issue search requests to Google's 2 billion web pages and receive results as structured data, access information in the Google cache, and check the spelling of words. Google Web APIs support the same search syntax as the site.

What's in the Google Web API developer's kit?
The kit contains a WSDL file describing the Google Web APIs service, a custom Java client library, documentation on how to use the service with Microsoft .NET or Perl, and sample SOAP messages. Please take a look at README included in the kit for the most detailed description of the kit and the list of instructions.

What programming languages does the Google Web API support?
The service has been tested with Java (Apache SOAP and Apache Axis), Perl (SOAP::Lite version 0.52), Ruby (SOAP4R), and C# in MS Visual Studio .NET. The service has not been tested with clients in other languages but it should work with any language with web services support.
· For Java clients, developers have everything they need to get started in googleapi.jar. Advanced users may prefer to use their own Java SOAP implementations and use the WSDL file to generate client code.
· For Perl clients developers need SOAP::Lite version 0.52 (or newer).
· For C# clients developers need the standard .NET framework; either from Visual Studio .NET or the Framework alone. (The Google Web API service has been tested with a C# client auto-generated from WSDL by the wsdl.exe tool. It should work fine from any .NET language.)

How do I develop a "simple search" client in Microsoft Visual Studio.NET?
Follow these steps:
1. Create a new project, such as a Windows Application
2. Run wsdl.exe GoogleSearch.wsdl to generate GoogleSearchService.cs, the C# client class.
3. Add a reference to the System.Web.Services DLL to your project.
4. Write your code to call GoogleSearchService.

How do I get access to the Google Web APIs service?
You must first agree to the terms and conditions of the Google Web APIs service and create a Google Account. Google will then email you a license key to use with your Google Web APIs applications. Every time you make a call to the Google Web APIs service, you must include a license key, assigned by Google upon registration.

Can I use the source code distributed in the Google Web APIs developer's kit?
The developer's kit includes some example Google client Java code, a .NET programming example, and a WSDL file you can use to generate code if your environment supports it. The developer's kit contains some third-party libraries; your use of these libraries is subject to their terms and conditions. Google has included the licenses for these libraries in the developer's kit download package.

How are the Google Web APIs implemented?
Google Web APIs are implemented as a web service. The service supports several SOAP methods; these are described in an accompanying WSDL file that can be imported into your favorite web services programming environment. Alternately, you can use our custom Java library to call the service.

Does this service work through HTTP proxies?
Sometimes. The service should work through HTTP proxies, but in practice many HTTP proxies have trouble correctly forwarding SOAP.