Monday, April 09, 2007

Calling Java vararg methods from dynamic languages

I'm adding code to support calling variable-argument methods (introduced in Java 5) from FreeMarker Template Language. What's the big deal, you might ask. A lot, I might answer.

As a matter of fact, it'd be a rather straightforward task if it weren't for a minor detail: overloaded methods.

When a method isn't overloaded, and you need to invoke it from a dynamic language such is FreeMarker, all is dandy and Bob is your uncle - you have to adhere to a single target formal argument list, easy as a breeze.

When a method is overloaded?

Ouch.

The thing is, with dynamic languages it's not as simple as finding the method that satisfies the mathematically precisely defined technical criteria known as "the most specific applicable method for given actual argument types". That's what Java compiler has to do, and believe me, Java compiler has it the easy way. With dynamic languages, we start from one step behind. We operate in another language, y'see, and we first must figure out how to marshal the actual arguments to Java types. Of course, the optimal way to marshal them might depend on the actual method we chose to invoke among all overloads. Which in turn depends on the Java types of arguments we use for invocation.

Chicken and egg. Catch 22. Strange loop. I wrote logic in FreeMarker to handle this years before, and it was quite involved even back then, and we didn't have variable arguments back then.

I had quite a few false starts on it, I'll admit that. Lot of code written and then rolled back. Lots of hours spent thinking about the issue. I have something that works now, written mostly while sitting on the porch of my parents' house where I retreated with wife and kids for Easter. But I'm not committing it to SVN repo just yet - need to test it a bit further before I unleash it on the crowd of lab animals, er, early adopters of FreeMarker 2.4.

There are some downright sinister corner cases, i.e. invoking

foo(a, b)

when you have:

public void foo(SomeClass a, SomeOtherClass b);
public void foo(YetAnotherClass a, SomeBozo b...);

Do you prepare the array of arguments for reflective invocation as [a,b], or [a,[b]] if your algorithm can not know at preparation time whether the fixed arg or the vararg method is going to be invoked? (Under the hood, varargs must always be passed as arrays). My decision was to prepare [a,b], and on the fly convert the last argument to one-element array if it happens that the vararg method was chosen (the ambiguity only arises if the vararg method would receive exactly one argument in its variable part, and would not arise if it would receive zero or more than one - you see what sort of arcane corner cases crop up in dynamic setting).

My ultimate hope is that I can create a sufficiently generic implementation for this that I can share it with other projects as well - Rhino and JRuby, for starters. All dynamic languages on JVM could benefit from a soundly implemented library for this functionality, at least until the bright future arrives where the whole world has already transitioned to JVMs that implement the hypothetical invokedynamic bytecode instruction. Although even if we had it, I actually doubt that the "strange loop" coordination required between marshaling arguments from another language to Java and selection of the most specific method would be handled by it - such marshaling is language specific, and while it can be abstracted behind an interface, extending with interfaces is a mechanism used by libraries in Java, not by the JVM itself, so it looks to me this'll remain a job for the library.