June 14th, 2012


Starting to feel like steam engine time for Querki

I'm drawing down to final days with Memento -- the plan is that I'll be leaving at the end of the month. The *theory* is that I'll be taking three months off for sabbatical, focusing mainly on Life -- doing necessary touchups on my poor neglected house, working on Jane's estate, actually finishing the Timeline, and things like that.

But I also want to do some serious programming: one of the frustrations of the past year is how little raw engineering I've been able to do. The first month or so is going to be focused on the high-priority project of The OP Converter. tpau is heading up a project to replace Jane's well-loved but impossible-for-mortals-to-maintain Order of Precedence with a modern DB (based on the Atlantian system, which is pretty well-designed); the bottleneck is the data. Even with all of Jane's miraculous discipline, the data has always been a little inconsistent and dirty; it's gotten a bit moreso since her passing. So my project for July is to write what I'm thinking of as the "OP Compiler": a tool written in Scala that will read in *all* of the OP files, normalize them into an in-memory database, identify inconsistencies, and write them out into SQL format for the new DB to ingest. It's not a gigantic project, but should be an entertaining diversion.

After that -- well, the question has always been whether my next step is to join a new startup or found one. And this past week, I've found myself shifting from the first to the second.

Mind, I've been pondering startup ideas for months. I even registered a couple of domains for one of them, focused on the interesting but fuzzy idea of sharing information about places in the real world -- this seems to be fascinating solution in search of a problem. But what seems to have finally fired me up is the idea of returning to the Querki project, which I've been neglecting for *many* years.

Once upon a time, I invented a system called ProWiki. This was an incredibly grungy piece of code, a big blob of Perl based on the very first wiki systems. On the surface it looks like a very simplistic wiki, but under the hood it contains a then-radical idea: the notion of fusing an object-oriented database with a wiki, to make it easier to keep track of *stuff*. I built ProWiki for my LARP writing, and still use it for that today, but the system has always been clunky, hard to use and painfully fragile. (It's a wonder the bear dances at all, as they say.)

So for about six years there's been a project lurking in the back of my mind to write what I dubbed Querki, the "queryable wiki". The idea would be to do this right: to build a system on modern technology, that would be fast, terribly easy to use, and in all ways state of the art. For all these years, I've been amassing architecture notes and other thoughts about the system, pondering both the requirements and how to build it.

And for no obvious reason, this past week, it all clicked into place. I've done a deep architecture revamp, and have a rapidly-growing ream of documentation about the features, the design, the goals and the use cases.

And the thing is, I've convinced myself that there is a business there. Which is where I may be calling on my friends, as outlined in the next post...

Querki: what I'm trying to accomplish

As described in my last post, I'm thinking about seriously diving into the Querki project, probably starting part-time after Pennsic, then maybe ramping up to full-time in October if it looks like it's a business idea worth pursuing. And as I do, I'm likely to be looking for interest and assistance of many kinds.

The project is, frankly, scary as hell. In part, that's because the idea isn't as unique as it was when I came up with ProWiki ten years ago -- both XWiki and Twiki have gone down somewhat similar paths, and have a serious foothold in the enterprise market.

But the thing is, I'm not *going* for the enterprise market. There's a huge market out there that is currently poorly-served: people who just want to keep track of *stuff*.

This shows up in a thousand places -- the infinite little websites that get built for special purposes, each its own little special snowflake. Hell, just within Carolingia in the past year we've built at least two of these: the new Carolingian Site DB, and the Cooks Guild Recipe DB. Both are functional, but both took more work to assemble than they should have, and both are kind of limited. And I find myself going, "We could do *so* much better than this".

So the notion is to focus on that market: the many people who just want an easy way to build little specialty sites for simple small databases. Whereas XWiki focuses on power, Querki focuses on ease of use. It's not about building huge enterprise databases, it's about making it Really Really Easy to build little databases of hundreds or thousands of Things. It's an online database for the consumer market, for the people who wouldn't normally even thinking about building a "database". (Indeed, I may deliberately avoid that word in public.)

And yes, I know that there are lots of cloud-based DB systems out there. Suffice it to say, I'm trying something quite different in some critical details: a prototype-styled OO DB instead of a conventional relational one. All my experience says that that is *way* easier for many real-world problems, so long as you don't care about scalability, and it fits nicely in a loosely-structured wiki environment.

So please forgive the burblage to come. This could prove to be a brief phase -- a week's enthusiasm that then burns out -- but it doesn't feel like it. I think I'm onto something here, and step one is going to be proving it to my friends.

Specifically: once it is at least basically up and running, I'm going to be looking to put projects onto it. I'm going to ask y'all to think about projects -- those things that you've built little sites for, or hacked in a third-party tool, and would like to do better. In some cases, I may ask if I can try replicating an existing site, and I won't kid you: my agenda is going to be to demonstrate that I can build something that is both *better* and *easier* than what you already have. I'm going to ask for honest criticism about any shortcomings you find, especially about anything existing systems can do that Querki can't. My hope is that I can prove that Querki is just plain better for 80% of the online-data problems you need to solve.

I'm also going to be looking for technical input. This time around, I'm going to try to avoid the go-it-myself of CommYou (one of the dumber mistakes I've ever made), and instead go for radical transparency, with a fully open-source project. That's a tad scary, but enough systems have demonstrated that you can build a good cloud-based solution that is completely open source that I'm inclined to give it a try. So if you're interested in participating in a really deep technical project (all the way down to language design), comment here and we can all talk about how we'd like to set it up. (In the long run, of course, I want to run the project via Querki itself, but for the first few months we're going to need some third-party project tools to communicate.) I would dearly love to get a couple dozen technically-inclined friends involved in the discussion. Those who want to actually get their hands dirty in the code would be more than welcome, but I'd also like folks who just want to muse on the architecture, the use cases, the usability and so on.

I think it's time to change the world, just a little bit. With some help, I think we just might be able to do so...