Justin du Coeur (jducoeur) wrote,
Justin du Coeur
jducoeur

Data, Big and Small

This week's notable link from LinkedIn is this delightful roundup of Five Trendy Open Source (Big-Data) Technologies. It goes through some of the newer hot products -- not stuff that's gotten mature like Hadoop, but newer concepts like Storm, Dremel, and Hana. Worth a read if you're doing any sort of big data at work, especially if you are in any way influencing architecture -- the enterprise world is driving advances in data processing at *remarkable* speed.

That said, it makes amusing reading for me right now. Everybody is talking about Big Data as the way to make money from enterprises. So I guess Querki might best be labeled the first truly serious Small Data project I've seen in a surprisingly long time. I'm explicitly not going after enterprise at all, at least not yet. (In a few years, if Querki is successful with consumers, we'll probably spin off a business-focused subsidiary. But first things first.) Indeed, for the time being I'm going to strictly limit the number of Things you can have in a Space, to somewhere in the tens-of-thousands range -- not even pocket change by Big Data standards.

Querki's underlying theory is that, while the Big Data problems are sexy to computer scientists and businesspeople, they have relatively little to do with the ordinary person on the information superhighway. Normal people are always trying to deal with *little* problems, involving only thousands, hundreds or even tens of things to keep track of. They don't care about lightning-speed processing of billions of records -- they care about being able to *easily* manage the small, everyday problems of the real world. And right now, they are looking sadly neglected.

I'm really quite enjoying this: there's nothing more exciting than finding a problem that nobody's dealing with well. Let's see if we can start a Small Data revolution, while the giants are all focused on the mountains in the distance...
Tags: querki, technology
Subscribe

  • the dogs

    We of course ate out throughout our stay in Edinburgh, mostly at local restaurants in the residential part of town. (Part of the motivation for…

  • Vacation: Motel 1

    [And now for the promised assorted notes from me] Our hotel in Edinburgh was something I haven't encountered in the US: inexpensive without feeling…

  • Oh, right -- we took this vacation

    Kate and I spent the beginning of the month in the UK, visiting first her northern relatives and then our friends pir and Miko. Here are Kate's…

  • Post a new comment

    Error

    Anonymous comments are disabled in this journal

    default userpic

    Your reply will be screened

    Your IP address will be recorded 

  • 11 comments

  • the dogs

    We of course ate out throughout our stay in Edinburgh, mostly at local restaurants in the residential part of town. (Part of the motivation for…

  • Vacation: Motel 1

    [And now for the promised assorted notes from me] Our hotel in Edinburgh was something I haven't encountered in the US: inexpensive without feeling…

  • Oh, right -- we took this vacation

    Kate and I spent the beginning of the month in the UK, visiting first her northern relatives and then our friends pir and Miko. Here are Kate's…