Justin du Coeur (jducoeur) wrote,
Justin du Coeur

Right, this is why "mostly functional" is always a dangerous phrase

[For the programmers.]

As I have burbled many times before, Scala is a lovely language: powerful, concise, and flexible. Sometimes flexible like rope, and occasionally flexible like a noose. (Somewhere, I have a button that says, "There is not now, and never will be, a language in which it is the slightest bit difficult to write bad code." This is a reminder of that.) Scala is a functional language, but specifically and intentionally *not* a pure-functional one. You can have state and side-effects, and you are responsible for using them in a judicious way.

Gradually over the past two years, I've been moving more and more towards functional techniques, and have gotten to the point where "var" always causes me to twitch slightly, but I haven't entirely foresworn them. The problem is, vars can interact with functional programming techniques *really* badly.

Today's example came from my rewrite of the system to use Autowire. This is a nice little library for marshaling commands across the wire -- now that Querki's client is itself written in Scala, this means that I can simply call the server from the client, and *poof*, it mostly just works. No farting around with hand-coded JSON or intermediate languages to describe the API: I just give the API in ordinary Scala, provide an implementation on the server, and call it from the client. It's quite lovely.

The thing is, though, my server-side code generally requires *context* that the client doesn't know about -- the calling User, the current state of this Space, that sort of thing. The main structure is called the RequestContext, and it is laced all over Querki. But of course, the client knows nothing about that, so how do I work it into my calls?

That is, my server-side code looks somewhat like this:
... req is the request; local rc param is the RequestContext
route[ThingFunctions](this)(req).foreach { result =>
  senderSaved ! ClientResponse(result)                  
The "route" macro takes the given request, interprets it as a request to the ThingFunctions trait, unmarshalls it, calls that method on "this" (which implements ThingFunctions), remarshalls the result, and returns that as a Future that my code then sends back to the client once the call has resolved. Basically, it does most of the scutwork boilerplate. But there is no way to pass that "rc" into the called function. What should I do?

My old-fashioned reflexes led me right down the primrose path to hell here:
var _theRc:Option[RequestContext] = None
... req is the request; local rc param is the RequestContext
_theRc = Some(rc)
route[ThingFunctions](this)(req).foreach { result =>
  senderSaved ! ClientResponse(result)                  
_theRc = None
That is, stuff rc into a variable that the implementation can get to. (Note that this is all inside an Actor, so it's effectively single-threaded; I wasn't *that* dumb.) Yes, the implementing code would sometimes have to copy this into a closure if it needs to do something asynchronous, but that's *kind* of obvious, so I could roll with it. And I did have a twitch from the "var", but it works just fine, right?

Yeah, not so much. Remember the above paragraph, which says that route returns a *Future* of the result? That is, it returns asynchronously, so that the implementation can be async if necessary. Which means that the actual re-serialization can happen well after the above code resolves.

And the thing is, functional code does *not* always work the way you expect. For example, deep inside one of the implementing methods, I had this function:
      def filterProps(props:Seq[AnyProp]):Seq[PropInfo] = {
        val filtered = for {
          prop <- props
          if (!prop.ifSet(Core.InternalProp))
          if (!prop.ifSet(Basic.SystemOnlyProp))
          yield prop
        filtered.map(prop => ClientApi.propInfo(prop, _theRc.get))
Seems innocuous -- it filters out inappropriate Properties to return to the client, and formats them. And this function is itself synchronous, so it ought to be fine.

Only -- the input is the values from a Map. And Map.values() returns a *lazy* sequence -- that is, in good functional style, it doesn't evaluate until it needs to do so. If you describe standard maps and transforms, it just adds them to the to-do list, but doesn't actually evaluate the sequence yet.

Until it is time to re-serialize the results. Which happens in a Future, remember? And notice that reference to _theRc above? Yeah -- while that is properly filled while the above function is called, it is totally empty by the time we finally resolve that Future and finally get around to resolving the sequence -- and finally call _theRc. D'oh! Took me an hour to track that one down and figure out what was going on.

So today's project was all about fixing this mess:
val params = AutowireParams(user, state, rc + state, spaceRouter, this)
val handler = new ThingFunctionsImpl(params)
route[ThingFunctions](handler)(req).foreach { result =>
  senderSaved ! ClientResponse(result)                  
That is, rather than trying to be clever with var's, do something more correctly functional: encapsulate the state and the functionality into an instance of ThingFunctionsImpl, and create one of *those* to handle each request. Works just fine (indeed, required very little change to the actual handler code), and doesn't commit the evil of mutable state.

Moral of the story: functional programming is a Very Good Thing indeed. But the further you go down that road, the further you will find you *need* to go. Querki is increasingly getting to the point where "var" is by definition a bad smell. (Indeed, on the client I am starting to eliminate it in favor of the reactive-expression Scala.Rx library. I may chat about that more later...)
Tags: programming

  • How I Spent My Birthday

    (Warning: diary ramble ahead.) Intercon was scheduled a couple of weeks earlier than usual this year -- our experimental hotel last year wasn't…

  • Hamilton Sing-Along

    Almost done with a *very* long weekend at Arisia. Generally been a great time -- worked hard, got to spend lots of time with friends, and have had a…

  • Musical Comedy

    The annoying cough I've been dealing with for a week finally turned into a full-on, OMFG, now-I-see-why-everyone's-so-draggy Monster Headcold…

  • Post a new comment


    Anonymous comments are disabled in this journal

    default userpic

    Your reply will be screened

    Your IP address will be recorded