Justin du Coeur

Subject:	One door closes; a whole corridor of them open
Time:	04:33 pm

I haven't posted many diary entries lately -- that's mostly because I've been pretty cranky, and I never like posting in such a mood. That said, November was an illustration of a principle I've often found: that the universe tends to keep my life interesting, so long as I pay attention to the opportunities it gives me. So the next few entries will be catching up on what's been going on, as life not only became a bit of a roller-coaster, but one gaining speed pretty rapidly. Part one is about work and stuff.

When last we left our hero, he was trying to emulate a good British stiff upper lip, and halfway managing it, but let's get real: losing the CommYou project hurt like hell. While I'm really excited to see all of these ideas I've been talking about for years being put into practice, I had kind of been hoping to be the one to make the splash there. It's a blow to the ego to realize that, while I am very good at startups, I'm not very good at the entrepreneur-CEO role.

The adjustment took about three weeks, from the point at which I realized that Google Wave had washed CommYou out to sea (yes, yes -- get used to the bad metaphors), before really coming to terms with it. There have been two main things that have eased the transition:

Google aren't stupid: they know that, if Wave is really going to succeed, they can't just build a tool -- they need to incite a whole new infrastructure. Part of giving that any credibility is fostering not just open protocols, but reference implementations of those protocols that aren't under Google's thumb. So I've wound up diving into that end of things. In particular, there is a small group of folks who are pretty serious about seeing this not only happen, but happen right. Most are pretty clear about the weaknesses in Google's initial implementation, but understand the potential in the idea. I think most of us tacitly agree that the objective here is to foster competition, and that means making a real stab at beating Google at their own game.

It's pretty chaotic right now -- among other things, everyone prefers different server implementation languages -- but I believe we'll gradually pull it together. So CommYou or something like it may yet be back eventually, in somewhat different form.

I no longer have an excuse to spend a day a week on CommYou, though: my business plan is shattered. This means I'm now working fulltime again. The silver lining in *that*, of course, is an immediate 25% boost to my salary, which doesn't suck. Indeed, the effective boost is more than that -- I've been paying a huge amount of money to run the CommYou server, which I'm about to claw back. So the total rise in my take-home is probably more like a third.

So my ego is bruised, but it definitely makes my life easier.

There *was* a sense of being adrift, and not knowing what to do next, which isn't a sensation I enjoy: that was the source of much of the crankiness. But remember what I said above, about the universe keeping things lively? Well, we'll talk about that in the next couple of entries...

Tags: commyou, diary, work

comments: 3 comments or Leave a comment

October 18th, 2009

Subject:	Google Wave, and the death of CommYou As We Know It
Time:	06:39 pm

I've been putting off writing this entry for a week now, but I really should just wrestle with it.

I've spent much of the past two weeks playing with Google Wave, experimenting with it, and talking with people about it. Right now, I'm reading into the nascent project to build an open-source version of it. (Which Google claims to support 100% -- they're trying to develop a whole new Internet infrastructure here, and are fully aware that nobody's going to take it seriously if they have the only implementation.)

A lot of people have been asking, "What the heck *is* Wave?". The answer is complicated, and I'll get into it more in a later series of posts, but the short answer is, "CommYou".

It's really pretty startling, not least because I'm fairly sure that this isn't quite what Wave set out to focus on. When you look at what it is good at, Wave is principally a co-editing system. It's a generalized infrastructure for allowing people to work on Stuff together, live in realtime as well as more gradually. It is basically taking a lot of ideas that have been floating around for many years (not least, in the game industry), and applying them in a pretty rigorous and generalized way.

But of course, CommYou has always been largely about that, just specifically for conversation. I've talked a lot about "multi-modal" or "semi-realtime", and this is exactly what I've been trying to describe: conversations that speed up nicely when multiple people are present, slow down when zero or one are there, but generally keep going and stay relatively distinct and on-topic. The latter point is essential: like CommYou, Wave is about *accomplishing* stuff together, which is what distinguishes it from a random chat room. The ethic of Wave, like CommYou, is that off-topic threads should (ideally) get taken to a separate wave, so conversations are broken down by community *and* topic.

They wound up arriving at bloody near exactly the same answers I did, enough so that I've occasionally found myself wondering if they hacked into my server and stole my ToDoList. Heavyweight threads: check. Semi-realtime interactive conversation: check. Summary window that live-updates when a conversation changes, and shows when and how much changed: check. UI that efficiently pops open an new-message window below any random message: check. Tracking which messages you've already read, so you can catch up on new stuff easily: check. Metadata and plugins: check. Etc, etc -- while it's not identical, it is *very* close to what I've had in my mind when trying to describe CommYou's design.

Make no mistake, Wave is damned cool -- there are a lot of nay-sayers, but they're mostly missing the point: they're taking an early alpha (which is what we've got now), full of bugs and lacking in features and integration, and saying "there's no there there". But having spent the past two years thinking about this stuff, I see *exactly* where Google is going with this, and it's a game-changer. We're at the beginning of a new model of communication and collaboration, probably more important than the rise of IM, nearly as important as the Web itself. (And vastly more important than Twitter and its micro-blogging ilk, which will gradually be subsumed into the Wave-like systems over the next 2-4 years.) Many people have complained that there's nothing *new* in Wave, which particularly misses the point: as I've long said about CommYou, it's not about coming up with a single new-and-revolutionary idea, it's about taking all the existing communication models and combining them *correctly*.

But the sense of "squish" is palpable, and I'm still wrapping my brain around it, and processing the emotions. Mind, Wave doesn't yet do *everything* that was in the CommYou design, not even everything that is already in it. But there is nothing in CommYou that would be *hard* to do in Wave, so I expect them to catch up to me fairly fast, and they already do a lot that I hadn't even contemplated: Wave's purpose is similar to CommYou's, but it is even more grandiosely ambitious.

Most importantly, they've almost casually swatted down my business plan. The idea was always that CommYou would be a cheap-or-free consumer service, as a loss leader for selling integrated high-quality conversation to websites. But Google has already announced that they will be giving away site embedding, and have even begun to demo it. So I'm left without the possible future income stream that justified all that work and present forgone salary.

I may be ambitious, but I'm not dumb, and I don't like tilting at windmills. It's one thing to grab at the brass ring of a startup, knowing that the odds are weak. It's quite another to try to compete against Google giving something away for free, especially now that they're starting to realize what they've got. I'm not going to win this one, at least not in the way I've been thinking.

So the existing CommYou plans are looking to be toast. I'm crushed, but part of surviving in the startup world is being good at crying in your beer for a little while, then picking up, moving on and getting the hell over it.

All that said, I don't think it's all going to be entirely for naught. Those two years of work have taught me a huge amount, not least about how a system like this can and must work. The open-source project is still pretty early, and I have a huge amount I can contribute to it. I've still got the passion for this project (which, remember, started as me trying to build the conversation system I've always wanted), and a lot of relevant knowledge. So I'm diving in head-first, trying to catch up to all the developers who have been in the loop for five months already while I was in denial about it.

We may even see the return of CommYou. We'll see how it plays out, but as I said -- there is a lot that Wave doesn't yet do. They've made great progress on the conversation problem, but they're missing a lot yet and they have made some decisions I don't necessarily agree with. (In particular, that realtime conversation is character-by-character Talk style, not message-by-message IM style: that's cool, but I think it's a mixed blessing.) And they've scarcely touched the community side of things, which has always been as important to me as the conversation part.

We'll see. I probably can't justify doing this as a true part-time job any more, so it'll have to turn spare-time, and I don't have enough of that. But there's a strong temptation to help the open-source project get up to speed, and then revive CommYou as an experimental variation of Wave that ignores the co-editing part entirely and focuses on conversation and community. If so, I'm going to do it right this time, as a pure open-source project end-to-end, using seriously cutting-edge tech. Programming geeks out there, think about whether you might be interested in playing.

BTW, for those who asked for invitations: sorry, they're mostly gone. I got a lot more people asking than I had invites, so I've been doling them out gradually and carefully, but I'm down to a single one left. (They mostly went to people who have been particularly helpful on CommYou, unsurprisingly -- those are folks who I know have a clue about this stuff, and have been demonstrably passionate about using it.)

Tags: commyou, wave

comments: 7 comments or Leave a comment

October 2nd, 2009

Subject:	Okay -- so who wants to play with Google Wave?
Time:	09:13 am

Thanks to

fairdice, I've now got a Wave account, and am starting to read into it. My initial reaction is that it's not really trying to do *quite* the same things as CommYou, although it's possible that it is close enough that it will suck away all the energy in the space. That is, the focus is notably different, but the overlap in ideas is non-trivial -- they clearly came to some of the same conclusions I did about how an interactive system ought to work. We'll see; I'm trying not to get excessively stressed about it, although the competition inevitably is on my mind.

Really, though, I need to do some experimenting here, to get a better idea of how it works, and Wave is fundamentally about multi-user, particularly real-time. The high concept (which is much clearer now that I can get my hands on it) is that they've re-envisioned email as a real-time, interactive process. The UI is reminiscent of Gmail, but the "messages" are live, interactive objects that everybody involved can update and modify in real-time. It doesn't appear to be principally a conversation system, or a community system -- rather, it's a co-editing system that happens to have realtime chat embedded in it. (Seriously realtime: the chat is part of the realtime Wave, so you see everyone editing their chat messages as they do so.)

So: who wants to play? If you have Wave, and are interested in playing with it (especially tomorrow, when I'll have more time), please reply or drop me an email. If you are really interested, and want an invite, tell me, but please don't do so *too* casually: I only have a modest number of invite codes, so I want them to go to folks who are going to really kick the tires hard. It appears to be particularly relevant and useful if you have stuff you want to do collaboratively in real-time online -- probably doubly so if you are already used to GMail and Google Docs...

Tags: commyou, technology, wave

comments: 13 comments or Leave a comment

May 2nd, 2009

Subject:	Career != Hobby != Calling
Time:	02:25 pm

Thanks to

mindways for the link to this fine little essay, about the fact that modern society, too often, separates activities too neatly into the buckets of "career" and "hobby", missing the fact that there are vocations that don't necessarily match either well.

This point comes quite personally to me right at the moment. Indeed, it's been the main focus of the negotiations over my new job -- I think they've been a bit puzzled over my demand for a four-day week. The CEO even said, when I was interviewing with him, "Never confuse your career with your hobbies". And that's true, but kind of misses the point.

For me, the CommYou project has become something of a calling. I've been forced to recognize that it's not necessarily a career: while I might someday make money off of it, I can't count on that. But it's by no means a "hobby", either -- there's a connotation of casualness about "hobby" that doesn't fit here. While the damned project is moving far, far slower than I'd like, it is something I *have* to do, just as much as a writer must write or a painter must paint.

Hence the four-day work week. If I tried to put CommYou purely on the back burner, I'd never be able to do a real day job: it would consume and frustrate me. So instead, I'm employing my long-practiced skill at separating the parts of my life: I can keep CommYou in a box if and only if there is a decent-sized box to put it into. Four days a week belong to the job that pays real money; one goes to the calling. I suspect it's actually going to work better than the consulting gig did -- by having clearer lines of what time belongs to whom (as well as physical separation of *where* I do each, and on what computers), I should be able to *focus* on both better than I've been doing, which will help me really get going on CommYou 2.0...

Tags: commyou, diary, work

comments: 4 comments or Leave a comment

January 8th, 2009

January 2nd, 2009

November 19th, 2008

Subject:	Fear of the Unknown
Time:	02:47 pm

It is occasionally fascinating to step back, watch myself program, and critique it.

Today's observation is that I am still scared of database programming. I'm *doing* it, mind you, but I don't have that innate comfort with it that I do with nice in-memory OO programming. There's a lingering sense that I'm impersonating a real DB programmer.

This is particularly coming up today because I am realizing that I need to do a slightly nasty database refactoring for CommYou. (Currently, Communities are linked to Persons -- that is, a Person is a Member of a Community. I am realizing that this is just making my life harder: Communities should be linked to the Identity that they are imported from, which would make synchronization vastly easier and more efficient.)

In principle, this is straightforward, and it's the sort of thing I do in ordinary code without even blinking. After all, the IDE protects me from most of the simple errors, and the test harness from the rest. But refactoring the DB is just plain *scary* to me. I understand the process just fine, and can outline the steps I need to take in detail with no difficulty, but I find myself procrastinating about actually *doing* it.

I can give some objective reasons for that. For one, the IDE does *not* protect me from errors in this very well. And while it's very straightforward for me to test new data created after this change, testing the migrated data comprehensively is much harder. It would be easy for subtle bugs to sneak in.

But mostly, I suspect I'm just still a bit intimidated by the prospect, because I haven't done it much. Nothing for it but to press through, make the changes, and convince myself that the world didn't end...

Tags: commyou, programming

comments: 6 comments or Leave a comment

November 18th, 2008

Subject:	Another day, another product
Current Music:	No Doubt - Dark Blue
Time:	05:39 pm

It always surprises me nowadays when I think, "Surely this tool must exist on the Internet", and it doesn't seem to.

One of the topics that came up due to today's LJ outage is, "Is CommYou ever going to be a social network unto itself?" To which the answer is "no", I'm deliberately avoiding doing that. CommYou is about Conversation Dammit, and I don't want to mix it up with the 97 other features that people assume a "social network" should have. And I am bound and determined to open it up to *all* social networks gradually, so I don't want the temptations that come with it becoming one itself.

That said, I *do* occasionally contemplate the idea of building a sort of "micro-network", sitting over on the side as a separate system, that would serve as something for people who don't already have a compatible network. This would be a real and for true "social network" and nothing else: a system that lets you log in and manage friend lists. It would have *no* other native functionality, but it would probably serve as an OpenSocial container. (Because hey -- why not?) It might also allow outside manipulation of the network, so that external apps could, eg, provide UI to manage your friend list within their own context. Moreover, it would be heavily metadata-based, so that an external app could attach information to both objects and relationships. Basically, it would be the true social-network-as-platform.

To my astonishment, this doesn't seem to exist yet. It seems obviously useful to me, and has great potential for end users: it would be modular, so that you had exactly the functionality *you* care about. It would be built entirely on the open protocol stack (OpenID, OAuth, OpenSocial, PortableContacts) -- indeed, its entire raison d'etre would be that stack. Ideally, it would be constructed to be a network-of-networks, so that many instances of it could be run, and users could just cross-connect across them. (Or indeed, should be able to link to anybody via OpenID.)

Hmm. Does this exist? Does anything even close to it exist? It really should, but it looks like every social-network implementation gets the core idea mixed up with big bundles of extraneous ad-hoc features.

Damn -- I *really* don't need another project right now, but I abhor open market niches. And the core would be super-easy to write: all the complexity would be in opening it up completely. This is sorely tempting...

Tags: commyou, technology

comments: 5 comments or Leave a comment

November 5th, 2008

November 4th, 2008

November 3rd, 2008

Subject:	CommYou Alpha open for LiveJournal users
Current Music:	Ricky Martin - Dame Mas (Loaded) (Spanish Version)
Time:	04:29 pm
Current Mood:	nervously excited

Okay -- it's time for me to bite the bullet, stop being quite so much of a perfectionist, and invite y'all to come play.

As of today, CommYou is now officially up to its Alpha release. It's by no stretch of the imagination done, but I think it's up to the point where it is getting both useful and stable enough for more than my current half-dozen semi-regular users, and I'd like to get more eyes on it. It no longer requires Facebook -- indeed, the LiveJournal version is considerably more advanced than the Facebook version at the moment, although FB will be brought in line soon.

There's quite a bit of documentation on the CommYou website, which can tell you all the nuts and bolts. This includes a Quick Start Guide, which covers all the high points briefly, as well as a detailed guide that goes through everything in deep detail.

A lot of people have asked me what CommYou *is*, and it's a little tricky to answer, because I'm pulling a bunch of things together. But the high concept is that we're trying to make conversations that are interactive (like IM) *and* persistent (like LJ). That is, you can be part of a conversation in real time via IM, but it's not transient: you can easily catch up on what you've missed and join in later, restarting the conversation. Whereas a blogging system like LJ is mainly about the top post, CommYou is focused on the resulting conversation. It separates things into distinct conversations in distinct communities, unlike most of the micro-blogging platforms (which are generally just a huge stream of messages).

I expect it to be most useful for cases where you are looking for a conversation: trying to get feedback, make plans, or simply chat with your friends in semi-real-time. It's not as strong as LJ for deep detailed thought yet (because it doesn't yet have threading), but it should get there over time, and I believe it's already stronger than LJ for full-community interaction. I'm not especially trying to replace LJ in peoples' hearts; rather, I'm creating a new tool that is probably useful at different times.

So please give it a try -- get into it, start and participate in some conversations, and tell me what you think. Talk about the election as it's happening. Spread the word to those who you think would like it: I'll eventually have to lock it down to invitation-only, but I'm not bothering with that until we seem to be straining the system. Tell me what you think, and let's talk about how we can make it really hum.

(And my apologies in advance if something goes wrong: this should be a bit of a learning experience for me...)

Tags: commyou

comments: 7 comments or Leave a comment

October 24th, 2008

Subject:	Head. Desk. Bam. -- or, Any OpenID (or LJ) experts in the audience?
Current Music:	They Might Be Giants - Mickey Mouse Clubhouse Theme
Time:	01:28 pm

There are days that I really hate being anywhere near the cutting edge, because it mostly tends to cut my legs off.

It appears that, sometime in the very recent past (past couple of weeks), CommYou's LiveJournal integration broke. Far as I can tell, this isn't due to anything I did: the version on the live site is just as dead as on my test laptop. I'm not sure when it happened, because I don't re-login very often, but it's sometime since I put up release 0.3.0.

Best I can tell, LJ's OpenID implementation is now broken, but that's based on a very inexpert understanding of the protocol. I seem to be using Version 1.1 of the protocol, and according to this report, Version 1.1 isn't allowed to send the op_endpoint parameter. LJ is not only sending it, it's sending it *empty*, which is causing the OpenID libraries to crash. (If present, it's supposed to be a URL.) Don't know what changed; like I said, this was working a few weeks ago, and my code hasn't changed.

Grumph. Does anyone know OpenID well enough to confirm my understanding that this is LJ's fault? (It's certainly possible that the problem is in the SXIP library I'm using -- that it's doing something wrong and LJ stopped tolerating it.) Does anyone know LJ well enough to know who I should be reporting it to? For the moment, I'll probably just have to hack around it, but I assume I'm not the only person seeing this problem...

Tags: commyou, technology

comments: 10 comments or Leave a comment

October 21st, 2008

October 3rd, 2008

September 29th, 2008

September 18th, 2008

Subject:	And that is why you need multiple test harnesses
Current Music:	The Police - No Time This Time
Time:	01:32 pm

I'm a devotee of black-box testing. Yes, unit testing has been all the fashion for years now, but I confess that I think it's often carried much too far: testing every input of puny little classes is a fine way to waste time for very little benefit. By contrast, a true black-box test that surrounds the outside of the system, pretends to be a real user as honestly as possible, and puts the system through its paces, is invaluable. When time is tight, I'll always work on the black-box first, because the bang for the buck is very high. So CommYou's black-box harness is pretty elaborate, pulling up multiple browsers and pretending to be several users operating in the real UI, with no cheating.

That said, black-box testing has its limitations -- in particular, reality sometimes rudely intrudes. Take, for instance, the new LJ integration. One of the things I want to test is that the social network synchronizes properly: when you change your flist on LJ, the next time you synchronize the same changes should show up in CommYou. Problem is, LJ is Very Very Slow about propagating those changes, even internally: the sad fact of the matter is that it can take an hour before LJ *admits* that your flist has changed, and there's not much CommYou can do to speed that up. So it's essentially impossible to automate these tests for real, because that could take an hour per test.

This, however, is why CommYou (like almost everything I write) is based on the intensely-pluggable "Ecology" architecture. Basically, I'm a fanatic about making the whole system pluggable and indirectly-referenced, with no static references to anything. It's a mild pain upfront, but times like this are when it pays off.

So yesterday's project was writing a new test harness, at the mid-level. The core of the system is all real, so it's higher-level than an ordinary unit test, but it talks efficiently to the APIs instead of the slower web pages, and it replaces the LJ communications with a little module in the Ecology that simulates LiveJournal. So instead of it taking an hour to propagate each flist change, it's instantaneous, and I can hammer the real system with dozens of flist changes in a couple of seconds. And now I have an API-level test harness for these fiddlier tests that need to fake out the internals of the system. Quite useful: I'll do more with that.

(End result: 3 bugs found, out of 300 lines of new or rewritten code to import LJ communities as well as friends. Not too bad...)