Some Comments from the MIT Emerging Technology Conference

Jeff Bezos was one of the presenters at the conference I went to last week, and he managed to mention Amazon’s less glamorous “developer” products without sounding too much like he was marketing them. He was pointing out that Amazon hopes to find what fundamental technological problems they have solved for themselves and try to generate revenue from those things by offering them as services. It’s clear that an interesting new thing happening in that fundamental infrastructure software as a service is actually becoming practical.

Amazon S3: some of you may have heard about this, but it’s a remote file storage service for cheap. Highly available and probably limited by your own bandwidth you use Amazon’s public APIs to push your files to some unknown location and retrieve them on demand. Some comments:

• Only a Big Dog like Amazon could do this because of branding and comfort. ISVs like us could even use such a service for asset management and say that we use Amazon’s S3 service for reliable asset storage, or automate regular database dump and push to an S3 location transparent to the user. If it were Bob’s Remote Storage in Arkansas, people would not be likely to trust it, but Amazon has a good shot at getting this right.
• For companies with high storage requirements and possibly little or no capital or expertise available to build a proper machine room, this is amazing enabling technology. TCO for SANs and such is rough, never mind the initial cost. One can make a really compelling price/performance argument to use a service whose cost scales linearly with the growth. In fact, the cost probably flattens out somewhat over time.

Amazon EC2: This is the Amazon elastic computing cloud. At first, this just sounds like another grid technology for doing supercomputing. Actually, it’s sort of a virtual system-on-demand sort of thing done remotely. One example usage is that you would create some server image for your web application, then request from EC2 that you want 10 load balanced images just like that; Amazon then does some magic and these systems are available to you.

• This is again huge for companies who may not have a machine room, but need to handle spikes in traffic. Let’s say you put out a project release and you know you will get slashdotted. Just for the week surrounding your release, you can order up 50 extra machines of capacity *just for that week*. When the buzz wears off, you release those systems, and all of this without the ugly investment that is really hard to do well, and leaves a bunch of under-utilized systems taking up power and space in the garage. The cost is localized and incremental.
• Bezos observes that 70% of application development goes into “heavy lifting” (what I call “dumb, annoying, and necessary”) things like storage, system administration, and (for amazon) shipping and packaging. Things like S3 and EC2 can remove those things from the equation for small companies. This supports the very compelling notion of micro-ISVs.
• There was a time when only excellent mechanics were capable of owning automobiles. Now, even people like me can both own *and* drive a car, imagine that. The point is this: lowering the barrier on dumb stuff can really enable people with good ideas to get their ideas implemented.

Remote Web Search API: On 10/4, Amazon announced API-level integration with a public web crawler Alexa, and they may have been what she was alluding to, but suddenly searching the internet in vertical ways becomes more possible.

(I found the following non-sequitur in my notes I wrote as the MIT staff was fiddling with some wireless mics which were dropping in and out: “how geeky do you have to be to land the a-v tech job at MIT?”)

That’s just Amazon. This week, I have been doing a but of reading, and look what else is out there (just to name a few):

Yahoo BBAuth and Google Account Authentication: Yahoo and Google (probably others) have an “outsourced authentication” API available. Yahoo’s API is called BBAuth or Browser-based authentication. This is nice for mash-ups, in that I can write a novel application that leverages Yahoo-resident content (e.g., the contents of my Yahoo photos site) without having to build in any actual authentication code into my application. My app would (in a browser-based app) redirect to the yahoo login page, where an end user provides his yahoo username and password. The app gets a token back and can then access the user’s content, but the application is never responsible for the username, password, or profile data. Google’s API is shaped the same way so that an application which accesses data contained in a user’s content area (such as gmail) can be seen without the application really needing to know the user’s login or password. This holds good promise to allow my little application (whatever that is) to ignore the infrastructure required for solid authentication/authorization. I.e., no AD integration, no LDAP server to set up, no password security/aging support to write (yet again).

Google Checkout API: Implementing a shopping cart/secure checkout process was once a big pain. I was around in the internet bubble and implemented several of these, and it was always way too hard. Had low-cost technology like this been around in 1997 or 1998, many many more good site ideas would have survived the crash. Anyway, as expected, this is a generic shopping cart application where the risky and tricky part actually belongs to Google. Sure, there are look and feel issues, but again, strong branding helps people feel better about blurring the lines between sites.

All these are consistent with the pattern: infrastructure software services are mature enough to use in commercial applications, both as micro ISVs and Big Company ISVs (like us).

(Thanks to Phil for being an excellent sounding board in Cambridge while assimilating this stuff!)

TRETC (this tag is for the blog stuff-- They said to include this tag if you are blogging about the conference; interesting experiment while I blog this at http://srehorn.blogspot.com/ to see what happens. Now I can go search for other people’s impressions of this same content. What did they get out of the same talk?)