January 13th, 2014


Not quite Zen Master of Refactoring yet, but getting there...

[Basically a programming diary entry.]

So about a week and a half ago, I mentioned that I was going to refactor Querki to use the Ecology Pattern. I confess, I was quite excited, and looking forward to it. I'm a fairly fundamentalist believer in refactoring as the way that you discover your code's correct design. (Having been through too many projects that spent months designing, and still never came terribly close to getting it right, I find that nowadays I mostly prefer to think about it until I begin to hit diminishing returns, and then just start coding -- in the long run, it's almost always more efficient.) I knew that I'd put it off too long, but I enjoy refactoring a lot.

Needless to say, it's been ten days of hell.

Yes, I knew that it had been too long to put off a serious refactor, but 100+ files and 17,000 SLOC is *much* too long. I really should have done this eight months ago.

That said, the process is drawing towards a close, and I'm happy about how it's come out. When I started out, an enormous amount of code was dumped into far too few huge files, mainly grouped by what they *were* instead of what they were *doing*. All of the interesting Types were in the file models.system.Types. More horribly, all of the Properties (many dozens of them) were in the file models.system.Properties. The various specialized kinds of QValues were defined in QValue.scala, because it had been a convenient place to dump them. models.system.SystemSpace depended on *everything* -- and everything depended on SystemSpace.

Initialization was horrifyingly ad-hoc -- the system was mostly composed of static objects, with deeply incestuous relationships. Properties required Types and Collections, which were themselves composed of Properties. Occasionally this would cause boot to crash due to recursive initializers, and I could do little but hack around the problems.

Now, that's been split into (quick check) 28 distinct subsystems, each implemented with its own Ecot and exposing no more interface than it needs to. The fraction of the files that cause the world to recompile has been reduced from about 90% to about 10%, and I'm continuing to improve that. Every file has been heavily rewritten, and most have been moved into more sensible locations; a rather scary fraction of the total lines of the system have been touched in one way or another. Each Ecot is now explicit about exactly which other interfaces it depends upon, and specifically which ones it needs in order to initialize. Lots of previously-public code, even some of the system Types, are now hidden away entirely inside of the Ecots that care about them.

Best of all, I just about have it to the point where there is *no* static state in the entire system. The Ecology is laced into damned near every object, and you can get to everything from the Ecology. This means that, when and if we have time, we should be able to run an arbitrary number of tests in parallel, each with its own distinctively stubbed version of the Ecology, without those tests stepping on or even knowing about each others' existence. Haven't quite gotten rid of the One Big Static Pointer to the Ecology yet, but I hope to be there by the end of today.

Indeed, the end result of all this is that Querki is impressively close to pure-functional, at least post-initialization -- I think there are on the order of a few dozen vars in the whole system, all of them tightly encapsulated. That probably doesn't *matter* much in the grand scheme of things ("almost pure functional" is one of those things like "almost pregnant"), but it makes it much easier for me to be proud of my work.

In retrospect, while my Engineer character could still stand to take another few levels of Refactoring, I can't really argue -- ten days to go from a monumental ball of spaghetti to a pretty well-decomposed Ecology, with clear internal interfaces and well-defined conventions for how to build each module, is pretty damned good. The code's ten times more maintainable, simply because it's now much clearer what belongs where. But still, I'll be glad to get done with this, and back to working on bugs and features, and feeling like I'm making progress...

[ETA: ahhh -- and now I'm getting to that refreshing downhill slide at the end of a big refactor, where I am now mostly *deleting* code. The feeling that each little tweak is making the system cleaner is delightful...]