- The data available in the system is still, after a huge investment, pitifully small;
- You can't trace the data back to its source, so it's hard to know how reliable it is.
But ultimately, I think those limitations will keep it from being more than a niche player. A system like this is only generally useful if you can use it in an ad hoc manner -- if you can take *any* question, ask it of the system, and have a reasonable expectation of getting an appropriate answer. W|A fails both because its data is insufficient and because its ambition is too large. Frankly, many of the complaints I've seen have been people submitting questions that just aren't this engine's core competency, which is quantitative analysis and comparison. By trying to look like Google, open to any old query, W|A simply embarasses itself. If they'd tried a much more focused and limited launch, with an aim of gradual expansion, they'd be less exciting but wouldn't have people making fun of them.
Mind, I do think that there is a place for an engine like this -- this just isn't the best design. An online service that does these sorts of things would probably be more valuable if it was based on the open data on the Web. Such an open engine would make it easy to take any old XML RDF data and compare it to other data in the same schema; it might also promulgate some standards for how to associate related-but-different schemas to each other. It would have a search front end that allowed you to search the web for data sources on a topic, and choose the specific data sources you want, rather than being a black box around its pre-selected sources. It would keep track of which data sources users find most useful and reliable, and push those to the front. It would allow the adding of modules for various sorts of visualization. In short, it would be a *platform* for this sort of analysis, not trying to solve every problem by itself. And it would fund itself indirectly, probably through advertising (although there are real problems with that business model).
Of course, Google already has most of these pieces. Anyone want to place bets on how long it'll take before they steal all the best UI ideas from W|A, put it on top of their data and APIs, and produce something like this?