Constantly Changing

Running YourKit UI remotely

2010-09-09T17:17:00.004+02:00

I got into a somewhat unique situation few weeks ago: I had a JVM heap dump 16GB in size that I had to analyze using a memory profiler. It was obviously hopeless to analyze it on my laptop which only has 4GB of RAM. Fortunately, I was given access to a computer half the world away that had 64GB of RAM. What I did was I moved my YourKit profiler instance onto that machine and tried to run it using X.11 tunneled through SSH.

The good news is: it works.

The bad news is: it needs tweaking to work.

First thing I run into was that the GUI was hideously slow. I would click on a button in the profiler, and feedback would take several minutes. I'm not joking. Several minutes. Googling revealed a Sun Bug Database item "Antialiasing/Compositing is slow on remote display". It suggests that to run GUI Java apps over X.11, one should add

-Dsun.java2d.pmoffscreen=false

as a JVM startup flag. I added it to profiler's yjp.sh launch script, and Holy Moly, it became much better. Granted, it was now rendering screens progressively from top to bottom, but it did it in matter of seconds, not minutes. It now felt like something out of 1999 instead of 1993. Further Googling for various combinations of "X.11" and "slow" and "ssh" turned out sites that suggested it's possible to make x.11 over a bit faster, by telling SSH to use Arcfour or Blowfish ciphers instead of AES, as well as to compress the traffic:

ssh -c arcfour,blowfish-cbc -C -X host

This led to further improvements; the progressive rendering got faster by another factor of two. That's as good as it gets. Now it feels like something out of circa 2001. I can work with that.

I work for Twitter now

2010-06-28T17:41:00.002+02:00

I work for Twitter now.

"Why?"

Twitter is an amazing company that provides an amazing service to the humankind. I see it as a fundamental new modus of communication that's quickly becoming part of the global human culture. Whether the earthquake in Haiti or the post-election uprising in Iran, Twitter has proven to be a reliable and prompt medium for getting the latest news on what's happening in the world, directly from the people most involved in the events. (While simultaneously still allowing you to to tell your friends what you had for breakfast, of course.) I appreciate its singular focus on doing one thing right, and that it managed to remain simple to use. Twitter is experiencing a 16% month-over-month (that's exponential!) traffic increase, and I'm very excited at the prospect of throwing my weight behind making it able to cope with this welcome challenge. The company is chock full of extremely bright people that I will no doubt be a delight to work with, and maybe some of their awesome will rub on to me too...

I want to change the world for the better and make a positive influence on people's lives. Often, the most significant changes are the ones people don't even notice, because they make Things Work As They Are Supposed To. How big a change? I have no idea yet, but it could be vital. If it makes the difference between a smooth operation and a Fail Whale for someone caught in the next civil unrest who's using a momentarily available internet connection to let his family know he's okay, it can mean the world to them.

I appreciate Twitter's open-source fostering culture. Today, the barrier to entry for influencing lots of people's lives is actually pretty low. It's called Open Source Software, and it's easy to participate and create serious value. I count that for every hour I spent on my OSS projects, I saved in thousands of man hours to my users, a number that will keep growing until the last of my code goes out of use (will be a while...). The value leverage is just insane. I just never figured out how to make it pay the bills, so I can't devote myself 100% to it. Twitter plays great in the OSS ecosystem, so I hope that I'll be able to create value in two streams: improving Twitter through the OSS it relies on, and as a consequence, improving the said OSS.

"So are you moving to Bay Area?"

Yes. We're working on my visa, and once that's approved (hopefully around October), I'll move with my wife and children to San Francisco to join the company in person. Until then, I'm working remotely from home in Hungary. I'm absolutely thrilled at the prospect of living there, and I'm very much looking forward to this new adventure.

Windows Live nightmare

2010-02-22T15:00:00.000+01:00

Last week I purchased a copy of Gears of War - it cost me some 12 bucks in local currency, one of the benefits of only buying 2-3 year old games. Yup, I'm the guy from this comic. So it happened that this weekend I decided to play a bit with it, so went to install it on a Sunday afternoon.

As installs go, it was uneventful, and I got to launching the game.

First snag, it tells me that it needs to install an "AMD Processor Update" otherwise it will crash during gameplay. Okay, let it install it. Reboot machine. Restart the game. I can play now, right?

Wrong.

It again tells me it needs to install the same update. Uh-oh. Okay, give it another try. Reboot machine. Restart the game.

I get to the main screen. Fine. I can play the game now, right?

Wrong.

I attempt to start the campaign, and it tells me that unless I have a Windows Live profile, I won't be able to save games!

Excuse me?!

*Pause*

Oh well, let's create a Windows Live account, I can live with that. It opens a browser, I fill out few forms, and off we go. I can play the game now, right?

Wrong.

Next it tells me that now it needs to update the local Windows Live client binaries. It tells me that I can choose to not update now, but I won't be able to play the game until I do. Yeah, that's great choice, indeed, how very thoughtful of them to give me options.

We're now at about one hour mark from the moment I inserted the install DVD. But once it got updated, I can finally log in and play the game, right?

Wrong.

It now tells me that Windows Live is "not supported in my region." How splendid! If I live in Hungary, I'm not allowed to save my bloody games, is that what you are telling me? Apparently, yes.

Another hour of googling and forum reading ensues. I end up on some dead links at sites named xbox.com and gamesforwindows.com and the likes where I can supposedly change my region but in reality I can't. I end up on a site where finally I can do it (I think it's some long domain name with words "windows" and "account" and "services" in it in some order). I lie that I'm in UK. It seems to accept it. So, I can now play the game, right?

Wrong.

They still claim I'm in "unsupported region". Turns out, they're protecting against such smartypants as I am, naive people who believed that Internet is global and makes it irrelevant where do you sit physically. Nope. No matter what you change your location to, the initial location when you created your account sticks with you. At one forum, a user tells me that if I were to create a different account now (forget the one I already have, it's tainted forever he tells me), and initially log in through some obscure UK site, and supply plausibly looking UK address and phone number then I could... sheesh.

I give up, play Bioshock for 15 minutes (all I had left of my evening), go to sleep, cursing my actions that led me to being exposed to a Microsoft technology again.

Next morning, more forum trawling. Turns out that you can create a "local" or "offline" profile, and that it lets you save the game locally. Hey, that's all I really ever wanted! So, how do I do it? Because, sure enough, I didn't see this anywhere on the UI when they were gently pushing me in the direction of creating a Live account.

Turns out, you need to click some kind of a "Learn More" button during the account creation process, and the help document will contain a link (only if you scroll to the bottom of the page) that will open a dialog to create a local account. At least it didn't say "Beware of the tiger" on the door...

So, let me get this straight: some Microsoft genius, in an attempt to drive as many users as possible into signing up for their Live service thoroughly hid the offline account creation, even if this will cause genuine grief for a subset of those users that are in an "unsupported region".

To use an analogy (it'll be a car analogy, of course), it's as if you'd have a four-lane highway, and all traffic signs point you to drive there to reach your destination, and of course you do, but after a bit of a drive, there's suddenly a blockade on the road, and since the license plates on your car are from an "unsupported region", you aren't allowed to drive further, and the grumpy officers won't tell you how you're supposed to get to your destination, only that they have specific instructions to not let pass the lower castes such as you are. So you drive back, and eventually find some secret vagabond scratchings on a wall (that'd be the forum posts) that point you to an otherwise unmarked dirt path through a forest, that will let you eventually reach your destination.

In the end, I have my local profile, so I can play the game now, right?

Right. Except I feel like I've been kicked in teeth by the whole experience so far, so it'll be some time before I'll have any desire to go back to this. Honestly, yesterday evening I was an inch from ritually burning the DVD. And I know that next time I hear that a game requires Windows Live in any capacity, that'll be a good sign for me to steer clear of it.

November moment

2009-11-16T08:12:00.001+01:00

Oh, the sudden urgency in the people’s steps in the street as the raindrops start hitting the pavement. I don’t have an umbrella, so I’m drawing close to the buildings on the way home for some little shelter their walls and balconies can provide. I thought these buildings familiar, but either because I’m passing so close to them, or because of the rain, they smell different. It’s surprising, but pleasant and triggers some previously deeply buried childhood memories.

Review of the Commodore 64 emulator for the iPhone

2009-09-08T09:19:00.004+02:00

UPDATE: Apple yanked the app because of the BASIC interpreter hole, and the developers have plugged it and resubmitted the app. Way to go, Apple. I mean, what harm could that BASIC interpreter do? It has no means of loading external code - no access to local filesystem of the underlying OS, no network connectivity, nothing. Are they afraid I'll manually type-in a program from a listing published in a magazine or something? I got tons more sense of childlike wonder from toying with the interpreter than I could ever from playing Jack Attack and Dragon's Den combined! Sheesh... On to the original review:

Against better judgment, I ended up taking my iPhone with me to the bathtub yesterday evening. I needed to check something really urgent, like the IMDB rating of "Watchmen" or similar (just finished the book). You know, stuff that you risk water damage to your high-tech gadget. I ended up browsing my RSS feeds instead though and spotted news about Commodore 64 emulator being approved by Apple and available on the App Store.
Now, you need to know that the first home computer I've ever been exposed to was a Commodore 64, at around age of eleven. I've got an enormous amount of emotional attachment to it, being enthusiast hacker for it as a kid. So, I was sitting there in my bathtub thinking, "this would be nice to get" and then realized "wait a minute, I can get it; right here, right now."
Let's pause for a moment to think about it folks, just how big enabler Internet is in our lives. I mean, there I was, purchasing a Commodore 64 simulator from my bathtub, and a minute later, I launched it and started toying around with the UI.

That's where disappointment started to set in.

Oh, don't get me wrong. Developers of this application (and the excellent open source Frodo C64 simulator they based it on) deserve every praise possible. I've used Frodo several years earlier, and it is painstakingly precise reproduction of a Commodore 64, up to the point of hardware side effects. The iPhone application is tasteful, snappy, responsive, and the UI is okay.
I do actually have one gripe with the UI, though. See, Commodore 64 horizontal resolution is 320 pixels, exactly the same as that of the iPhone. Since the developers added a graphic around it resembling the venerable Commodore 1701 monitor, the mask obscures some of the pixels at the edge. This is very annoying actually, since lots of games display important information around the top and bottom edges. These machines back then emitted an image that had two distinct areas named "border" and "paper". Border was the outermost part of the image and had a single attribute: color. Paper was a smaller rectangle within the border where the image was actually displayed. The purpose of the "border" was specifically to act as a disposable margin that could be clipped to size by your monitor (in many cases, a TV screen) so that you don't lose paper pixels. The developers of the iPhone emulator thus improved aesthetics at expense of functionality.

But that's not what I'm disappointed with.

Honestly, I could be disappointed by the initial selection of games; they're all ancient simplistic titles even by Commodore 64 standards, old 1983 titles where developers still didn't know how to utilize the platform to its potential. There will be more games available later, so I'm not too worried about that. (I hope they will release Elite...)

That's not what I'm disappointed with.

What I'm disappointed with is the lack of the Commodore 64 essence, as I experienced it. That essence, ladies and gentlemen, is hacking. Even as a kid, I disliked it when people were using the machine exclusively as a game console. As a bit of personal history, let me tell you that my family was not exactly wealthy when I was a kid, and that too is an understatement. After I became interested in computers (following a series of articles in a monthly magazine for school kids), I would write my little programs (initially BASIC, later 6510 machine code) in a notebook, then annoying one of the three people in our village who had a machine to give me a bit of a time to try them out. I saw the potential for creating my own stuff in these machines, and couldn't imagine why would someone, who is fortunate enough to actually be able to afford one, use it only to play games on it. I disliked people who didn't see beyond gaming (I'm not saying it was justified and think the same today, I was an eleven year old kid then, remember?).

Commodore 64 was an utterly open platform. Even from BASIC, you could POKE around with it (for those not in the know, "POKE" is a basic command for directly setting contents of a memory location). Even before I would have a Commodore 64 of my own (which I had to wait all the way until 1989 to be able to afford one; cool kids were playing with Amigas by then) I would have a book that contained a description of the BASIC, of the contents of both BASIC and System ROM (complete with all system calls and memory locations it used for settings, buffers, and so on), of all peripheral chips (VIC, SID, CIA1, CIA2 etc.), their mapped memory locations and how to program them, a complete reference of 6510 machine code, and finally, complete hardware schematics for the machine. If you had this book, you could do anything with the machine. Anything. And believe me, I did a lot. Won't go into details now as I'm well into get-off-my-lawn territory here.

That's why I'm disappointed that I won't be able to do a lot with this one, except play old games on my phone. I don't imagine that under Apple's regime I would ever be able to fire up Turbo Assembler and write some rasters, if you know what I mean. And that's what makes me sad.

Apple even only approved the emulator once the developer has disabled the BASIC, so it's not even the familiar C64 screen that welcomes you on startup. It looks like this instead:

However, turns out there's a workaround; the BASIC ain't disabled, it's just hidden. If you go to "Advanced" and turn on "Always show full keyboard", then launch a game, go to "EXTRA" keys, and tap "RESET", you end up with the familiar screen:

(Mind you, the problem with monitor mask clipping the edge pixels is particularly articulated in this picture - you don't really see the first character in the line!) I had a bit more fun with it though now that I had a command line. For starters, let's see if there's something on the "floppy disk":

Sure there is! Looks like a floppy for "Dragon's Den" (one of the games included with the app). Interestingly, it does not matter what game I started before resetting the emulator: the "floppy" in device 8 was always that of "Dragon's Den". I didn't find other numbered devices containing other games "floppies" either. For now. However, here's a kicker. (Or an easter egg if it's intentional, although I don't think it is.) If you load the program from the floppy and run it manually:

OMG, YOU GET AN INTRO!:

That's correct ladies and gentlemen - the developers seem to had trouble locating a genuine copy of the game, so they bundled one that was distributed by a cracker group. Back in the day it was customary for people who cracked games for, erm, "unofficial distribution" to prepend a small program to it called "intro" which was a colorful, blinking screen with greetings to fellow members of the trade and good chiptunes. They also usually led to menus for enabling cheats and sometimes on-screen instructions for the game in case the player didn't have the original game manual at hand. Like, if you're playing on an iPhone, right? (Honestly, if that wasn't there I would have never figured out this particular game on my own. So the intro actually added value for me. Also, the fact someone had this game in cracked form but not in uncracked form strongly suggests that if it weren't for crackers, we wouldn't have this game today. Just sayin'.)

Needless to say, the intro isn't there if you launch the game from the iPhone app menu.

Verdict? Nice easter egg with the intro, folks. Beautiful, polished iPhone application. Potential for more game titles to come. But if you were more than a game player on Commodore 64, don't hold your breath. You'll likely never be able to upload any of your your own old time C64 creations on the device to show it off. For that, you'll still need an emulator on your desktop OS.

I hope at least they'll release Elite, though. That'd be awesome.

Relativity of simultaneity

2009-03-17T14:16:00.004+01:00

A problem with staying in a profession for long years is that there's less and less things that can truly excite or surprise you as the time passes. That's why I was particularly delighted to experience a moment of sudden enlightenment while listening to Rich Hickey talk about "Persistent Data Structures and Managed References" last thursday at QCon in London. The title of the talk might not sound exciting, but I have to tell you, if this were the only talk I attended, it alone would have been worth attending. I don't say this often. Slides are here, although you're missing out Rich's incredibly good verbal presenting style if you just read them, obviously.

Rich's view (with which I have to agree; in the time I knew Rich, I have yet to hear the first thing from him I'd disagree with, whether it be concurrency, memory models, or ideas for what to do in San Francisco if you only have a single day to explore it); so, Rich's view is that the idea of variables in traditional programming languages is broken, because they are based on a single thread of control at a time. This used to be true in single-threaded programs, and even to a degree for multi-threaded programs running on single-CPU systems.

He argues that we need well defined time models for state changes in our concurrent computational systems. A lack of a time model fails to capture the situation where two observers (threads, CPUs) can observe two events (changes of state) in different order.

Then it suddenly hit me! I learned this stuff in school! Relative timelines, observers... Rich is talking about relativity of simultaneity!

(Which is one of the simpler conseqeunces of Einstein's special theory of relativity.)

Wait a moment. Finding such a surprising parallel analogy between physical world surrounding us and our digital computational systems seems at first unlikely, but thinking more of it, it makes perfect sense. For event timelines to appear differently to different observers, we only need two prerequisites: truly parallel processing, and finite speed of change propagation. Both are true for the real world (quite massively parallel, and the changes propagate at at most the speed of light), and for digital computational systems (parallel already at two CPU cores, with changes occurring in CPU registers and needing time to propagate downstream the hardware memory model to be observable by other CPUs).

One of the wonderful things about Rich is that he's able to express these notions very clearly. It is all really obvious when you think about it, and I couldn't even say I wasn't aware of it for at least three years now, it's just that before hearing this talk, I always had a very slight hope that someday, someone (infinitely wiser than me) will somehow be able to, you know, solve this. Eliminate this problem. The thing I realize now is that it's inherent. You can no sooner eliminate relativity of simultaneity and the rest of consequences it brings to the table from our computational systems than you could cancel its effects in the physical world.

Rich does the only sane thing to do with a problem you can't eliminate - he embraces it. His programming language, Clojure, is an attempt at being the easiest way to write correct concurrent programs. Emphasis on all of "easy to write", "concurrent", and "correct". I learned about Clojure sometime last year when it came up at the jvm-languages mailing list. I read up the documentation on the website, and came away thoroughly impressed. Then I met Rich and saw a 30-minute Clojure presentation last year at the JVM language summit, saw the 60-minute version of the same talk now at QCon, and I'm still impressed. Clojure embraces the problem of concurrent state modifications by encapsulating the state into "references" - the only mutable data types in Clojure, which all point to immutable values. The references can change in time though to point to new values, and the big idea here is that each reference has defined concurrency semantics for how and when change occurs.

Now, there's likely a huge number of temporal semantics of how can some entity's state change. Clojure identifies several usual semantic categories though, and provides them for ease of use. We have synchronous coordinated (Clojure has a STM for that), synchronous uncoordinated, asynchronous uncoordinated, as well as isolated (thread local). I think you can build whatever you need using those.

Anyhow, during Rich's talk I had a moment of enlightenment. Not because I heard something entirely new, but because the pieces of the jigsaw puzzle finally fell into place and revealed a picture. The picture does away with a false hope I had, which is good as it is a clear and definite answer to a question, and having accepted it it is at least clear to me what's the direction forward. During the rest of the day, I was telling pretty much everyone I met and their dog about how I was shown the inherent manifestation of relativity in finite-speed concurrent systems. (Joe Armstrong was just smirking at it, probably thinking "you wet-behind-the-ears kids, this should've been obvious to you for decades", and then we proceeded making fun of all n-phase commit protocols, for any finite value of n).

On multicore systems, there's no longer absolute time. Every core runs on its own relative time and there's no definite sequencing of events. It's time we build this assumption into our code, and not use tools and methodologies that are ignorant of this fact. And as Rich would point it out, writing your code in Clojure is a good way towards the goal.

Speaking at QCon

2009-01-30T16:25:00.003+01:00

Just a heads up that I'll be speaking at QCon in London this March, about using JavaScript in the enterprise. Looking forward to meet you all there.

IP telephony on iPhone

2008-12-03T11:48:00.005+01:00

I spend quite a lot of time on calling my colleagues in US or UK (I am in Hungary), which can result in quite exorbitant phone bills. I tried to fix this lately by looking for IP telephony solutions for iPhone - I often have conference calls lasting upward of half an hour, and up to even two hours, so I don't want to be confined to my desk or any other physical location. I will often walk up and down in the apartment, subconsciously tracing patterns of the carpets with my feet (it's funny when I consciously realize I'm doing it), staring out of various windows at the sky, sometimes sprawling on a sofa and so forth. (Hey, working from home has its perks!). So, I need mobility within the apartment, hence I still want to use the phone.

Now, Skype would be the obvious choice - for a flat fee of 10.29 EUR/month I can get unlimited calls to both US and UK numbers. Major hurdle is that Skype ships no client for the iPhone. Fring actually does the job, it is a multiprotocol chat/voice client for iPhone (and some other phones) that supports Skype accounts too. Voice quality is not particularly good, but it's within tolerable. Fring however lacks an essential (for me) feature - it does not provide a DTMF keypad. I always dial into the company conference center, and must be able to punch in the conference number I wish to join. Fring can't do it. I can still use it to call direct extensions, but the bulk of my calls is through the conference system, and it's a no-go. I asked Fring support, and all they told me was:

Currently, it is not available to dial DTMF sounds on fring, this should be implemented in future versions.

For the record, Skype desktop client does have a DTMF keypad.

Next choice is TruPhone. They offer a client for iPhone (and also a bunch of other makers' devices). TruPhone operates their own call service, much like Skype does, and while there's no flat-fee subscription model, their 0.06 USD/minute plan is sure quite attractive (heck, even calling numbers within my cell provider's network, T-Mobile Hungary, is more expensive than that!). The voice quality is very good, even better than cellphone (one coworker remarked after I first used it to dial in to a conference that I sounded much clearer than I usually do). The client app also has a muting button and a DTMF keypad, the first useful, the second essential for conferencing. It would all be rosy, if only the client weren't extremely buggy. Sometime, with iPhone OS 2.1, the UI would slow down (you could see it animate scrolls pixel by pixel) and it'll then claim there's not enough memory. iPhone OS 2.2 seem to have fixed that, but now it instead often complains that it can't find "Wi-Fi coverage" (even though the iPhone itself clearly sees the network). The aforementioned DTMF keypad has the tendency to not react to touch, or conversely, to get jammed as if I touched the key but didn't lift my finger, sending a prolonged shrieking "beeeeep" of that particular number key tone through the earphones into my skull, full volume. (The iPhone built-in phone app nicely tones down the DTMF beeps on my end of the line, so I still have an audio feedback that I'm typing them, but don't hear them full-volume). Regardless, the "low memory" and "unable to find wi-fi" are the most critical problems - even turning the phone off and on again doesn't always help it with its wi-fi disorientation syndrome (and mind you, I should never be forced to power cycle my phone just to use an app!).

So, it's a mixed bag. I'd use Fring if it had a DTMF keypad because of a flat-rate unlimited calls provided by Skype. The second best right now is TruPhone, which provides very good voice quality and otherwise pleasant service with low pay-as-you-go rates, but their iPhone client is bordering on unusable because of all the bugs. I'll be eagerly awaiting app updates to both.

World of Goo

2008-11-18T23:26:00.004+01:00

I found a fantastic game. It's named World of Goo. Here's a trailer. And here's the Wikipedia entry. Check those ratings - 9/10, 9.5/10, 10/10, 93%, 96%.

It's hard to summarize what's right about this game. Most everything is.

Let me start with few obscure details that might appeal specifically to a computer geek like me.

First, this game runs on Windows, on Nintendo Wii, and on Mac OS X. Linux version comes soon. And guess what, the Mac OS X version is not a lame "Win32 API linked with Wine" kind of port. Proof that it isn't is that Wine-linked Win32 API using Mac software only runs on Intel CPUs. Not so this game. It runs on Macs with PowerPC CPUs (either a G4 or a G5). It's pretty much a native version on Mac OS X. It runs crisply on my iMac G5. I asked the developers in e-mail about their development methodology, and the developer Ron Carmel was kind to reply. (The whole company seems to be just two guys, working from wi-fi equipped coffee houses in San Francisco. They have years of experience working for big name game companies, though.) Anyway, Ron's response was:

at some point, when we realized we're going to go multi-platform (wii/pc at the time) we simply abstracted away anything and everything platform dependent from the code. this meant graphics, sound, input handling, window creation, threading, that sort of thing. the game only deals with a set of abstract classs that it gets from a singleton "environment" class. those classes provide a set of services (like graphics, sound, etc) and we have different implementations of those classes for windows, mac, wii, and soon linux.

I can't say just how much I respect people who take this approach. It's one of hallmarks of professionalism in my book. More often than not, even much more complex software systems can (and should) be developed to be really multiplatform. At least on unices, it is fairly common to maintain software as CPU-agnostic (and often OS-specifics-agnostic) source code with small CPU&OS adapters. These guys however didn't only do it across different unices, which are all closely related OSes. They did it across Windows, Mac OS X, and whatever is the OS of the Wii. That's 3 operating systems, and likely 3 different CPUs too (Intel, PPC, and whatever the Wii CPU is, although AFAIK, it's also some PPC variant). Consider how most large software houses can't be bothered to do this - they just target Windows primarily, even though they'd have lots of internal benefits if they cleaned up their source code to be multiplatform.

Second obscure detail, the game has no DRM. None. These guys were smart enough to realize it'd be a waste of resources to bother with it. Another big tip o'the hat to them.

What this game does have though is: captive, immersive, innovative gameplay based on building structures out of gooey balls that bind to each other, and leveraging mechanics in a very realistic physics simulation (on various levels, you'll need to use tension, elastics, gravity, wind, flotation, and so on). It's tremendous fun. Immersion is further helped by the fact that there's no UI aside from the mouse pointer - all interaction with the game world is by directly dragging and dropping the otherwise aimlessly wandering goo balls with the mouse.

There's lot of humor and cuteness baked in. You can get emotionally attached to little goo balls as you could to those lemmings back in the day.

And then there's the beautiful visual art (that at times reminds me of Tim Burton movies, particulary The Corpse Bride), and music that perfectly matches it to create the immersive atmosphere. There are overarching motives and hints of a backstory that are intriguing on their own.

It's ideal for a casual gamer who needs to blow of steam for an hour after a hard day's work. My son is playing it every evening. My daughter is playing it every evening. I am playing it every evening. You can play it too in minutes - there's a downloadable demo with the full first chapter of the game (the full game has four chapters and an "epilogue").

FreeCC, the modern JavaCC

2008-10-22T22:41:00.003+02:00

If you ever needed to write a parser, you hopefully didn't do it by hand but rather used a parser generator. The two predominant parser generators in Java are ANTLR and JavaCC. JavaCC project however suffers from paralysis for too many years now. The original developers aren't present for too many years now, and the current set of active committers didn't make any significant improvements for years. My FreeMarker colleague Jon Revusky has recently taken interest in it (FreeMarker having used JavaCC in the last six years, when Jon ditched the hand-written parser of the 1.x versions). Since his improvements to the JavaCC codebase weren't accepted by mainstream (largely dormant) JavaCC project, he did the only correct move: he forked it.

Enter FreeCC.

FreeCC is backwards compatible on source level with JavaCC - it can generate parsers from existing JavaCC sources, and the parsers behave identically (since it's a continuation of the original codebase with different name, it's not too surprising, but doing the work under fresh name, it doesn't hurt to emphasize), FreeCC, however has some modern new amenities, such as:

typesafe collections in its own code, instead of Java 1.1 Vector and Hashtable

instead of hardcoded prints in the source code, the parsers are generated using editable templates (can you say "multiple target languages support?")

other long overdue source code sanitization

generated parsers don't rely on static singletons anymore

These are all nice and dandy; code reworks in particular improve the project's long term maintainability by making its comprehension more easy for newly joining developers (in this regard, FreeCC is much more welcoming than JavaCC for a newbie hacker). These however don't really give you much of an advantage when you're writing your grammars. The following ones do, and they're the real kickers:

code injection feature eliminates (or at least strongly reduces) the need for manual post-editing of files

grammar include feature, which allow smaller grammar files to be reused in larger ones (with JavaCC, you had to copy-paste), as well as allow you to organize larger grammars into separate smaller files.

These features are also the sign of what kind of features are still to come: features that provide you with modern conveniences a programmer in a need of a parser generator would wish to have. These two new features above help you create maintainable grammar sources, and they are just the start. If you adopt these features however, there's no going back. Since JavaCC doesn't have these features, your grammar files will no longer be compatible with JavaCC, only with FreeCC. But you'll hardly want to go back. FreeCC has taken this particular parser generator codebase much further in its few months of existence than the JavaCC project did in years, and is gaining traction. Given the fact that FreeCC is a continuation of the current JavaCC codebase (which didn't really progress further since the fork), it is really risk-free to try it out for your next project (or even in current project!) instead of JavaCC. You can also expect that the developer will be more open to your feature requests, as Jon has a good track record of listening to community wishes in FreeMarker.

FreeCC is the JavaCC you'd want to use in 21st century.

At FreeMarker project, we've already switched to FreeCC; FreeMarker 2.4.0 and 2.3.15 will both have a parser built using FreeCC. Since Jon also works on FreeMarker, he's truly eating his own dog food. (Actually, there's even more to that. Since FreeCC in turn uses FreeMarker as the templating system for its output, he's eating his own dog food doubly! This seemingly creates a circular dependency between FreeMarker and FreeCC, except that luckily all that was needed was a FreeMarker JAR with parser built still using JavaCC to bootstrap the process initially; the projects are self-sufficient since and don't need JavaCC anymore.)

HTML encapsulated JSON

2008-10-08T10:31:00.003+02:00

Continuing the early morning post on having an XSL-like solution for JSON, where your webapp only outputs JSON files, and has attached stylesheet(s) that the browsers can use to display it as a nicely formatted HTML it intended for human audience. All solutions I outlined there needed an active change to existing technologies: custom HTTP header, or extension to JSON, but in any case they would need explicit support from browsers.

In other words, they would never get widely adopted.

But then, just as I went to sleep, it hit me that there's a solution that operates completely within the currently existing technologies. I'll call this solution "HTML encapsulated JSON". The premise is that you create a simple HTML page that has the JSON payload in its body, and has script elements that pull in the JSON-to-DOM transforming script. Something like this:


<html xmlns="http://www.w3.org/1999/xhtml">
  <head>
    <script src="/carlist-json-to-html.js" type="text/javascript" />
    <link rel="stylesheet" type="text/css" href="carlist.css" />
  </head>
  <body>
    <pre> 
        {   "cars": [
                {   "color": "red",
                    "make": "Ford",
                    "model": "Mustang",
                    "year": 1986,
                    "price": { 
                        "amount": 250,
                        "currency": "USD"
                    }
                },
                {   "color": "yellow",
                    "make": "Mazda",
                    "model": "6",
                    "type": "GT",
                    "year": 2002,
                    "price": { 
                        "amount": 14000,
                        "currency": "USD"
                    }
                }
            ],
            "dealer": {
                "name": "Honest Joe",
                "address": {
                    "street": "123 7th Avenue",
                    "city": "Dustfield"
                },
                "phone": "1-456-789789"
            }
        }
      </pre>
    </body>
</html>

As you can see, the JSON is put into html/body/pre (using "pre" to be fully DTD conformant). A machine client should be able to easily parse it out from there. Granted, it'll need to use both an XML parser and then a JSON parser, but that shouldn't be too big deal.

But the best part is definitely that you can very easily transform this into a nicely formatted HTML, by including a script (emphasized in red) to build a DOM from the JSON extracted from the body. Actually, you can just generate a fairly naked HTML, and then use a separate CSS stylesheet (as shown above) to format the HTML for presentation! And it's all standards compliant, and works with any modern browser!

Next, I'll try to come up with typical code for a JavaScript code template that would be used for a JSON-to-DOM conversion.

CSS for JSON

2008-10-08T04:49:00.008+02:00

The back-in-the-day promise of XSL stylesheets was that you could output an XML document from a HTTP request, and have it processed as-is by a machine recipient, or transformed into a pretty HTML for human viewing automatically.

Seeing how I (and probably many others) do prefer JSON to XML nowadays for my machine-to-machine communication, I'm thinking of an equivalent for JSON. In similar vein, XSL is almost entirely replaced by CSS today, and I keep thinking that we'd need an XSL or CSS equivalent for JSON.

Actually, we probably don't even need a new language for describing the transform. JavaScript would probably do just fine - read the JSON, build the DOM.

The other problem, assuming we already have the language for transforming JSON to pretty formatted HTML: how would you declare a transform in a JSON document?

In XML, you can use a processing instruction. In HTML, you can use the <link> element. Now, unlike XML's processing instructions, or HTML's <link> element, JSON notoriously lacks any support for any out-of-band information that isn't the actual content. Heck, it doesn't even support comments!

I can see three possible ways around it:

Declare the transform in the transport header, i.e. just slap on an "X-JSON-Stylesheet" or similar header into the HTTP response. The obvious problem is that this ties the solution to the transport.
Add it to a conventional location into the JSON data itself. I.e. have the top-level object literal have a "__metadata__" object, and put this and similar other information into it. i.e.:
```
{ "__metadata__": { "links": [{"url": "myStyleSheet.cssjs", "type": "stylesheet"}]} ... }
```
The obvious problems are: a) the top-level value in a JSON document ain't necessarily an object (can be an array, or even a primitive value) b) some JSON based formats won't be happy about having extra elements with magic names inside of them.
Extend JSON to add a syntactic equivalent of XML processing instructions. Honestly, I prefer this approach. Just adding single-line comment support, and then having a special prefix for the "processing instruction" comment (i.e. using a question mark, to keep it similar with XML PE) would do, I think:
```
//?stylesheet myStyleSheet.cssjs
{ real payload goes here ... }
```

In any case, it is up to browser vendors to get the ball rolling on this - both the declaration mechanism, and the transform language.

I think this would be a great technology to have, where I could just output JSON from my web applications, and have it both consumed by software, and presented in nicely human readable form in browsers.

Package private access in Open Source code

2008-09-02T17:40:00.004+02:00

I recently got in a same situation three times: someone wanted to use code I wrote in an Open Source project written in Java, and they couldn't, because the class/method in question had package private ("default") access, rendering it inaccessible outside of its package.

First, Charlie Nutter needs access to package-private class BeanMetaobjectProtocol in Dynalang-MOP.
Next, John Arkley needs access to package-private class AllHttpScopesHashModel in FreeMarker to help push it into Spring.
Finally, Steve Yegge needs access to the package-private constructor of Context class in Rhino.

Now, you know, when the same thing hits me three times in a row in short timeframe (incidentally, all three coming from people with high geek cred), that gets me thinking: What good is package-private access in an Open Source project anyway?!

It's a very valid question, really. Think about it: people can see the code - it's open. People want to use the code - it's normal. And when they want to, you frustrate them by declaratively preventing them from linking their code to your open source code. They can look, but they can't touch.

That seems wrong.

Of course, you could argue for package private access' validity. Here are some arguments you could come up with:

Package private helps implementation hiding on a package level! Well, duh, it does. However, if a class/method is useful to another class you wrote and that lives in the same package, it might also be useful to some poor outsider schmuck too! It must already conform to quite rigorous coding standards, as it will be used by some other classes, ones you wrote in the same package, so it'd better maintain state invariants and so on. Does it really make a difference if it's another class of yours, or another class of another developer, living in another package? I say: it shouldn't. If you think the class is just an auxiliary, and it's just cluttering the JavaDoc, just move it to a *.support subpackage instead (Spring does that extensively).

Package private helps you hide bits you don't want to be tied down by backwards compatibility! This is really a corollary of the first one. I used to be big on this one (that's probably why I got asked to loosen up access restrictions in the first place - because I used to place them there in the first place). Common wisdom is that once people start using your publicly available API, you'd better not break it on the next release. By making lots of stuff public, you increase the surface area that needs to be kept backwards compatible, right? Right, sort of.

Now, how about this instead: make it public anyway, just note in a JavaDoc entry (or even better, in an annotation, say @SubjectToChange) that there's no backwards compatibility guarantee on this method. If it's an annotation, people can even have an automated tool for checking for its use before they upgrade. Hell, you can even have a @BreaksCompatibility annotation! What it boils down to is: don't treat users of your library as children and decide what's good for them. Inform them that the API is volatile, but open it up, don't close it. It's not really closed anyway, as they can see the source. You're just erecting a glass wall in front of them; they can look but they can't touch.

And they'll come to bug you about it anyway.

My point of view right now is that it's better to provide a suggestive hint that an API is volatile in the documentation or annotation and let anyone use them at their own risk rather than build a non-negotiable restriction into the code (as in, can't negotiate it with a compiler; you can still definitely negotiate it with me).

Also, I'm not saying either that package private access is completely inadequate for Open Source libraries. I'll admit there might be valid use cases for it, but if there are, they're very few.

Now, before you'd accuse me of being a "make everything public" proponent, let me say that this reasoning doesn't apply at all to private access, and only partially applies to protected access. Let's see why:

Private access is completely justified. Let me point out one clear distinction between packages and classes: classes and instances of classes can have state. Packages can not. That's pretty much what makes the difference. Through refactorings, you can end up with methods in a class that violate its state invariants. Other methods in the class can call these methods as steps to transform the object from one valid state to another valid state, but it might be invalid in the interim. You would never expose such a method publicly. Well, you shouldn't expose it even package-privately either; that'd be sloppy programming (see my above remark about package-privately accessible methods having to be just as rigorously coded as public ones). A class (and class instance) is a unit of state encapsulation, a package is not. Hence, classes need private access.

Protected methods remain usable by 3rd party code, as long as it has classes that extend your base class. It mightn't be too ideal a constraint (forcing subclassing in order to access functionality), and you might rather want to design your libraries to favor composition (has-a) over subclassing (is-a) architecture. But if you must have classes intended to be subclassed, and there's functionality that's only ever used/extended/implemented by a subclass, make it protected. Especially when you have abstract protected methods that act as poor Java programmer's mixins (are intended to be called by code in the base class). Sometimes you'll allow (or downright expect) such abstract methods to violate an invariant in the object's state temporarily, as it'll be called as a step in a more complex state transition implemented by an algorithm in the base class method. In these cases, protected access is justified. But in other cases though, you might still consider making some protected methods public.

In conclusion: I'm currently fairly convinced that what package private access is good for is preventing your users from linking their code to useful bits of your code for purposes you didn't anticipate up front; it's often nothing more than an unnecessary garden wall.

Undeclared, Undefined, Null in JavaScript

2008-07-24T14:52:00.004+02:00

So, a coworker of mine run into a situation recently where they were testing for an expression in JavaScript as:

if(obj.prop != null && obj.prop.subprop > 0) ...

This basically guards you against dereferencing "subprop" if "obj.prop" is itself undefined or null. They were running into a situation where the first predicate would pass, and the second would fail. However, "obj.prop" always printed "undefined". Turns out, it actually had the string "undefined" as its value.

Ouch.

Anyway, said coworker pointed me to a blog entry titled Three common mistakes in JavaScript / EcmaScript. Said blog entry says:

if (SomeObject != null) {

Well, in JavaScript, which is a dynamic language, something that has not been assigned to is not null, it's undefined. Undefined is different from null. Why? Don't ask me. Well, anyways, you can use typeof to explicitly check for undefined, or use other more or less clean tricks, but the best way to deal with that is probably to just rely on the type-sloppiness of JavaScript and count on it to evaluate null and undefined as false in a boolean expression, like this:

if (SomeObject) {

It looks uglier, but it's more robust.

I have to disagree. It's not more robust. It will also catch the cases of SomeObject having numeric value of 0, or string value of empty string. Because their boolean coercion is also false. To make matters worse, the original example, using SomeObject != null actually works and is in most cases actually the most appropriate!

See, in JavaScript, null and undefined are actually equal according to the == and != operator! (ECMA-262 standard, section 11.9.3 tells us so.) In vast majority of cases, you don't care about the difference at all, so using someExpr != null is good enough.

If you really-truly must distinguish between undefined and null, you have some options. Curiously, while there is an actual built-in language literal for the null value (namely, null), the sole value of the Null type, there is no built-in language literal for undefined, the sole value of the Undefined type. The identifier "undefined" can be assigned to, so you can't write something simple as if(x === undefined):

var undefined = "I'm defined now";
var x; // he's really undefined
print(x === undefined); // prints false

Inconvenient, huh? So, how to test for undefined? Well, the common practice found in most books and tutorials on JavaScript seems to be using the JS built-in typeof() function, but I really don't like it, because this is implemented by way of a string comparison:

var x;
print(typeof(x) == "undefined");

will actually print true. But as I said, I think it's ugly.

My solution instead relies on the fact that undefined is equal to null, but is not strictly equal to null, therefore this expression also works:

var x;
print(x == null && x !== null);

will also print true, and it involes only two simple comparisons.

Which brings us to the question of what is the actual difference in JavaScript between an undeclared variable, a variable with undefined value, and a variable with null value. Let's see:

var x = {}; // empty object
var u; // declared, but undefined
var n =  null; // declared, defined to be null

function isUndefined(x) { return x == null && x !== null; }

print(isUndefined(x.x)); // prints true - access to undefined property on an object yields undefined
print(isUndefined(u)); // prints true - declared, but undefined value
print(isUndefined(n)); // prints false - the value is null, not undefined
print(isUndefined(z)); // runtime error -- z is undeclared

So, it's an error to dereference an undeclared variable, "z" in above example (it's okay to assign to it, which creates a new global variable). It is not an error to dereference a declared variable with undefined value, "u" in above example. Its value is the undefined value. Further, access to any undefined properties on objects also result in undefined value, duh. As for null, well, null is just a value like true, or false, or 4.66920166091; it's the single value of the type Null.

Hope this clears up the whole topic of undefined/null values (and undeclared variables) somewhat.

Objective-J

2008-06-07T14:07:00.004+02:00

So, here's some news: people at 280 Slides created a web application that allows you to build presentations in your web browser. It does look very nice, people are comparing it to Apple's Keynote application. All in all, yet another webapp out there; what's the big deal, right?

Well, the people who created 280 Slides were previously Apple employees. 280 Slides wasn't just written in JavaScript. No. These people created something called Objective-J, which is to JavaScript as Objective-C is to C. And then they implemented part of Apple's Cocoa application framework atop of it (named it Cappuccino), and finally implemented the application atop of it.

Now that's quite amazing.

Dion Almaer writes that

Objective-J is the language that takes JavaScript and makes it Objective (as Obj-C did to C). Lots of square brackets. When the browser gets served .j files, it preprocesses them on the fly. This means that you can do things like, use standard JavaScript in places.

Interesting. Objective-J will eventually be open sourced at objective-j.org, and I'll be quite curious to see what did they do. I suspect they have a transformer from Objective-J source code to plain JavaScript (presumably itself written in JS), and then the browser's JS runtime converts the source code to JS when it downloads it. But I might be wrong.

Then there's the interesting issue that Objective-C improved C with OO features. But what did Objective-J improve? JavaScript is extremely object-oriented to begin with, so this sounds more as if they wanted to bring the actual Objective-C flavor of OO to JavaScript instead, because that's what they're comfortable doing. They need to drive nails into a different wall now, and they'd still prefer to do it with their old hammer!

Don't get me wrong, I'm not making fun of them. Shaping one's tools in a new environment after ones you knew and loved in a previous environment is a valid activity if you percieve it as the path that allows you to be most productive. To build a new language atop of JS and then build an application framework atop of it, and then build a very usable and visually appealing application on top of it (very cross-browser compatible too) gets you a metric shitload of geek cred in my circles. It might turn out to be a catalyst for getting a lot of similarly nice future webapps out there. It might turn out to be the next big thing for JavaScript in browser.

I'm eagerly waiting for content to start popping up at objective-j.org, although of course the Objective-J.js can be readily inspected.

So, WebKit JS used an AST walker before?

2008-06-05T11:06:00.004+02:00

Color me flabbergasted.

WebKit recently announced a new JavaScript interpreter, named SquirellFish, claiming it is much faster than the previous interpreter in WebKit.

That's good and all, but in the "Why it's fast" section of the linked article, they say:

Like the interpreters for many scripting languages, WebKit’s previous JavaScript interpreter was a simple syntax tree walker.

It was? Oh my... They also say that:

SquirrelFish’s bytecode engine elegantly eliminates almost all of the overhead of a tree-walking interpreter. First, a bytecode stream exactly describes the operations needed to execute a program. Compiling to bytecode implicitly strips away irrelevant grammatical structure. Second, a bytecode dispatch is a single direct memory read, followed by a single indirect branch. Therefore, executing a bytecode instruction is much faster than visiting a syntax tree node. Third, with the syntax tree gone, the interpreter no longer needs to propagate execution state between syntax tree nodes.

Why, yes, indeed!
For the record, Rhino has been doing this for ages - AST is compiled to internal "JS bytecode" format that strips away grammar, and then interprets it. This works like this since, well, around the turn of the millenium. (Actually, Rhino can even kick it another notch and can also optionally compile the JS bytecode to Java bytecode, eliminating the interpreter altogether. Which bytecode then the JVM JIT compiler can further compile to machine code at its own discretion.)

Anyway, I digress. All I wanted to say is that I'm honestly amazed that a supposedly professional implementation of JavaScript (the one shipped in WebKit before SquirellFish came along, and by consequence, shipped in Apple's Safari browser) would use an AST walking interpreter. Yeah, I know, you'll say "most scripts run once on the page, so optimization is overkill", but with AJAX this is no longer true, and apparently, the WebKit team also thinks so.

On the positive side, they're finally moving on to a better solution, and I congratulate them on the undoubtedly hard decision to finally take the leap toward a more modern execution architecture.

"Mixed Language Environments" interview

2008-05-29T19:19:00.001+02:00

in case you're interested, the video interview where Kresten Krab Thorup interviewed Charlie Nutter, Erik Meijer, and myself about mixed language environments on virtual machines is now online. This was shot at the JAOO conference last September.

Interview about Rhino

2008-04-16T17:36:00.004+02:00

Floyd Marinescu interviewed me about Rhino at the JAOO conference in Aarhus, Denmark last September. You can watch the interview here. Yes, I know I said "scratch to itch" :-)

I actually stepped down from the Rhino maintainer role since this interview was shot, although I do remain with the project as a committer.

Airport Extreme disappoints

2008-04-09T15:03:00.005+02:00

UPDATE: Apparently, firmware upgrade 7.3.2 fixed the disk access problems. I'm now able to use a USB-attached disk to AEBS just fine. Skype also started working fine some time ago (even before the firmware upgrade). The speed issue (not being able to exceed 8 Mbit/sec on 802.11g with all-Apple equipment) unfortunately still remains.

I got a brand new Airport Extreme Base Station (AEBS) this Friday. (Kirk hauled it for me from Las Vegas to Hungary - Apple gear costs roughly twice as much here than it does in US.)

I already own a 250GB external HDD, so I wanted to attach it to AEBS for either file sharing or, hopefully, wireless Time Machine backups.

(Too bad Apple officially said only three days later that they don't support Time Machine backups with AEBS. Meaning, if it works for you, you're lucky, but if it doesn't, you're on your own.)

Anyway...

My experience is that even when Time Machine is not involved, AEBS can't handle a USB drive at all. Just attaching the drive to the base station will cause it to come online much more slowly (needs several minutes after plugged in). Also, whenever I wanted to mount the drive through Finder on any of the Macs in the house and transfer a file to/from it, AEBS would become inaccessible in a minute or two, then spontaneously reboot. With no drive attached, it's stable as a rock.

Of course, you could claim that the drive is at fault. Now, the same drive works perfectly when attached to either my MacBook Pro, or to my iMac G5, both running Mac OS X 10.5.2. It was formatted and partitioned (single partition, GUID scheme, HFS+ journalled filesystem) on the iMac G5. As in, freshly reformatted and repartitioned once again in a desperate attempt to make it work with AEBS. Run Disk Utility's "Repair Disk" on it too, just in case.

Now, when the drive is attached to either Mac and shared from it, the other Mac can mount it wirelessly and even do Time Machine backups to it, without any problems. So, you know, the drive itself seems in order. But if I connect it to the AEBS, it kills the AEBS. AEBS firmware is also upgraded to latest, 7.3.1. I wanted to reduce the clutter on my desk by moving the HDD and its wires to the AEBS and am fairly frustrated with the inability to do so.

I repeat: the symptoms have nothing to do with Time Machine. With Time Machine off, and trying to access the drive over AEBS as a vanilla networked volume, AEBS dies in a minute or so. I suspect a firmware bug.

And then there's the speed.

My MacBook Pro is a first-gen Core Duo, meaning it can only use 802.11g, and can't be upgraded to use 802.11n. Still, 802.11g is 54 MBit/sec, right? Well, the fastest data transfer rate I could achieve moving data to the AEBS-attached HDD (before it crapped out) was a bit over 8 MBit/sec. On a distance of few feet between the devices. Not happy. I can understand various protocol overheads would chip off the bandwidth but I can't understand getting 14% of advertised bandwidth. Not even if it were half-duplex.

Okay, what about Ethernet access? I tried connecting over Ethernet to see whether the AEBS crashing problem is inherent to Wi-Fi, or is also present over Ethernet (result: it is present over Ethernet too). The MacBook Pro said the connection is 1000MBit/sec - fair enough, as both it and the AEBS are supposedly Gigabit Ethernet. Both devices use speed and duplex autoselect based on the cable actually plugged in. So, with 1000MBit/sec theoretical maximum speed, what effective throughput could I get? 72MBit/sec, tops. It ain't the HDD that's slow - it's a 7200RPM drive with USB 2.0, and when connected directly to the computer it is pretty much screaming fast.

And then there are application compatibility problems.

My wife Kriszti speaks quite a lot with her aunt over Skype, and with AEBS she experiences much more stalled audio/video over Skype. I actually subscribe to two broadband connections: a 4MBit cable and also a 1MBit ADSL as a backup. I moved the old router to the 1MBit ADSL (which previously wasn't available over wireless, being only a backup). When Kriszti had repeated Skype problems, I told her to switch to using the old wireless router, and lo and behold Skype works okay for her again, so her Mac tends to remain connected to the old router's wireless network (so our machines aren't on the same network, which is also a source of inconveniences). So: cable ISP with old router = Skype ok; cable ISP with AEBS = Skype not okay; ADSL ISP with old router = Skype okay. There's a pattern here, although in order to be able to claim with 100% certainity that it's a problem with AEBS, I'd also need to test Skype with AEBS connected to the ADSL line which I won't do now as it'd require too much uncabling/hauling/recabling.

So, we have a base station that crashes when you try to use an USB HDD with it, with disappointing speed figures, and apparently also causes problems for Skype traffic. The only really good point is that contrary to the old taiwanese no-name router I replaced, it has a better range and implements WPA/WPA2 in a manner that Windows laptops can also understand; both of these don't really mean anything to me, but it certainly makes the nice girl next door (who we give free Internet access in exchange for occasionally babysitting our kids) lot happier.

All in all, a disappointment.

Xstream with Hibernate

2008-03-21T11:33:00.004+01:00

People have been asking in the comments to my post on XStream about how to properly integrate Hibernate with XStream, as there are few pieces of puzzle that don't necessarily fit together perfectly. I answered in the comments, but Blogger annoyingly doesn't allow a lot of formatting in comments, so I decided to write a separate follow-up post; here it goes.

First of all, I use Hibernate 2. I know, it's old, but it does the work and I won't fix it unless it is broken. I expect most of the advice will also apply to Hibernate 3.

I'm aliasing all classes using XStream.alias(). Reason being that on the server side, some data model classes are actually subclassed for additional (although non-Hibernate) related functionality. You might or might not need this.

This is however essential: We need to ensure that XStream will treat Hibernate lists, sets, and maps as Java lists, sets, and maps. I believe this is what helps avoid the infamous "com.thoughtworks.xstream.converters.ConversionException: Cannot handle CGLIB enhanced proxies with multiple callbacks..." problem. There are three things that need to be done:

use Xstream.addDefaultImplementation() to tell XStream to treat all Hibernate-enhanced collection classes as plain Java collections:

xstream.addDefaultImplementation(
        net.sf.hibernate.collection.List.class, java.util.List.class);
xstream.addDefaultImplementation(
        net.sf.hibernate.collection.Map.class, java.util.Map.class);
xstream.addDefaultImplementation(
        net.sf.hibernate.collection.Set.class, java.util.Set.class);

Finally, in order for XStream to actually handle these collections I needed to define and register some custom converters that are able to handle Hibernate collections as Java collections:

Mapper mapper = xstream.getMapper();
xstream.registerConverter(new HibernateCollectionConverter(mapper));
xstream.registerConverter(new HibernateMapConverter(mapper));

These custom converter classes are rather trivial, here are their definitions. All they really do is extend the XStream built-in collection and map converters, and declare their ability to handle Hibernate lists, sets, and maps:

import net.sf.hibernate.collection.List;
import net.sf.hibernate.collection.Set;
import com.thoughtworks.xstream.converters.collections.CollectionConverter;
import com.thoughtworks.xstream.mapper.Mapper;

class HibernateCollectionConverter extends CollectionConverter {
    HibernateCollectionConverter(Mapper mapper) {
        super(mapper);
    }

    public boolean canConvert(Class type) {
        return super.canConvert(type) || type == List.class || type == Set.class; 
    }
}

and

import net.sf.hibernate.collection.Map;
import com.thoughtworks.xstream.converters.collections.MapConverter;
import com.thoughtworks.xstream.mapper.Mapper;

class HibernateMapConverter extends MapConverter {

    HibernateMapConverter(Mapper mapper) {
        super(mapper);
    }

    public boolean canConvert(Class type) {
        return super.canConvert(type) || type == Map.class; 
    }
}

That's all I did and it eliminated all of my Hibernate+XStream problems - hope it will also help you.

Attributes and items

2008-03-20T13:50:00.003+01:00

So, here I am trying to further my metaobject protocol library. I'm now in the eat-my-own-dog-food phase of sorts, as - after having written a MOP for POJOs, I'm trying to write actual MOPs for some dynamic language implementations, most notably, Jython and JRuby. And pretty soon, I hit an obvious design problem (which is okay, really - this is still an exercise in exploring the right approach). Namely, lots of languages actually have two namespaces when it comes to accessing data belonging to an object. I'll refer to one of them as "attributes" (as that's what they're called in both Ruby and Python) and the other are "items", which are elements of some container object. All objects have attributes, but only some objects (containers) have items. Some languages don't make a distinction, most notably, JavaScript. In JavaScript, all objects are containers and they only have items (and an item can be a function, in which case it functions as a method on the object). Other languages (Ruby, Python) will distinguish between the two; the containers are arrays/lists and hashes/dictionaries/maps. As a matter of fact, it helps thinking of Java as having the distinction - JavaBeans properties are the attributes, and arrays, Maps, and Lists will have items.

I'd like to think that most people's mental model of objects actually distinguishes the two.

Now, to make matters a bit more complicated, in lots of languages the container API is actually just a syntactic sugar. Give an object a [] and a []= method, and it's a container in Ruby! Give it __getitem__, __setitem__, and few others, and it's a container in Python! Honestly, this is okay - as a byproduct of duck typing, one shouldn't expect to there be any sort of an explicit declaration, right?

For ordered containers, most languages will also make it possible to manipulate subranges as well.

Bottom line is, I feel this is a big deal to solve in interoperable manner, as the raison d'être of this library is to allow interoperability between programs written in different languages within a single JVM; I imagine in most cases the programs will pass complex data structures built out of lists and dictionaries to one another, so it feels... essential to get this right. It also feels like something that can rightfully belong in a generic MOP as most languages do have the concept of ordered sequences and associative arrays. Of course, I might also be wrong here; it is also an essential goal to not end up with a baroque specification that contains everything plus the kitchen sink.

So, here am I wondering whether this is something that can be made sufficiently unified across the languages to the point that if a Ruby program is given a Python dictionary, and it calls []= on it, it actually ends up being translated into a __getitem__ call. The goal seems worthwhile, and is certainly possible but I'm not entirely sure how much of an effort will it take. There's only one way to find out though :-)

Running Half-Life 2 natively on Mac OS X

2008-03-06T11:05:00.002+01:00

I run Half-Life 2 natively on Mac OS X yesterday evening, in the same configuration I run it on Mac: 1280x1024 resolution, 8x anisotropic filtering, High Dynamic Range bloom rendering, all graphic settings set to "High". It run like charm. I'm impressed. Half-Life 2 was the only reason until today to keep a Windows XP partition on my Mac, and use Boot Camp. But it was inconvenient, the rebooting, even if I did it about once a week on a sunday afternoon for few hours of gameplay.

The magic behind this is called CrossOver Gaming.

Yesterday, I got an e-mail from Codeweavers where they announced CrossOver Gaming Beta. For those unfamiliar, CrossOver is a Wine-based Win32 API compatibility layer for Mac and Linux, allowing Windows application binaries to run natively under these operating systems on machines with x86 CPU architecture. I did test drive CrossOver earlier (that's why I was on their e-mail list), but it left me unexcited, not by its own fault, but really because I had no need for any Windows applications at the time.

I was intrigued by the CrossOver Gaming product though, as it does answer a need I have, namely playing a game I own without needing a reboot (VMWare fusion doesn't run Half-Life 2). Or needing a Windows license. The difference between "regular" CrossOver and CrossOver Gaming is that Gaming will see more frequent releases, will be more bleeding edge, basically less conservative as to what goes into it and frequently updated to accommodate game compatibility problems. You won't necessarily want that from a software that you use to run, say, your Windows-only accounting package, but for games, this model makes perfect sense.

Well, it works pretty much as advertised on the box. It knows of a bunch of "supported" applications, one of them being Steam (and all games available through it). It will install a surrogate HTML library (lacking Internet Explorer, right, there's no Microsoft-shipped mshtml.dll in the system) that allows Steam's built-in store browser to work, then download and install MSXML redistributables and Steam itself. Steam then launched, I logged into my account, downloaded Half-Life 2: Episode One, crossed fingers, and launched it.

It runs just as it did under Windows, and I think I couldn't say anything more praiseworthy about this CrossOver product, even in its beta. I'm totally buying this when it comes out.

I'll briefly mention that there is another player in this space, Transgaming, but unlike Codeweavers, they forked off Wine and aren't donating code back upstream to Wine, so if you need to choose, it seems as if supporting Codeweavers seems like a better option from the moral point of view, as Codeweavers do donate back to Wine. Also, Transgaming's Cedega product doesn't allow users running unaltered Windows games on Mac, only on Linux. They don't offer Cedega for Mac; they have Cider, but that's not a runtime but rather a library that developers need to link against to produce Mac-runnable versions of their games written against the Win32 API. So in reality, CrossOver is the only solution for running a Windows game natively under Mac OS X. Fortunately, it seems to be a good one.

While waiting for the fourth season...

2008-03-04T17:20:00.003+01:00

I avoid most forms of filmed science fiction, reason being that most attempts underachieve badly compared to written works. The quality of written sci-fi is rarely matched or even approximated by movies and TV series. Notable exceptions are few, and the finest filmed sci-fi for the last few years is without a doubt the reimagined Battlestar Galactica (Firefly would come in close second). The final, fourth season starts in April, and I can hardly wait for it to start. In the meantime, there's a great two part interview with the show's creators about legal system, torture, and morality (part one), as well as economy and politics (part two) in the series.

If that weren't enough, here's a gorgeous Battlestar Galactica reinterpretation of Last Supper on Flickr.

Microsoft Open Source

2008-02-22T11:13:00.004+01:00

I was surprised to discover today that Microsoft has a hosting site for Open Source projects (analogous to, say, SourceForge.net or GNU Savannah): witness Codeplex. To my further surprise, there's also an Open Source community site at Microsoft, named Port 25. I haven't got time to investigate either of them more deeply yet, but plan to do so in future. This is intriguing.

(Found them both following Microsoft's Open Source Interoperability Initiative FAQ.)

Mind you, I was aware earlier that Microsoft has software released under OSI-approved licenses; there are more than few Microsoft-backed projects hosted on SourceForge.net (or did they move to Codeplex since? Hm...). What I was not aware of, and what I believe is a big deal is that Microsoft is now providing its own hosted infrastructure for Open Source project hosting and community discussion.

Reap What You Sow

2008-02-21T22:05:00.003+01:00

You won't catch me writing about politics too often, but I need to express my view on the declaration of independence in Kosovo. Actually, I'll mostly bother you with some of my family history, but the two are, for better or worse, somewhat intertwined.

As you might or might not know, I grew up on the territory of former Yugoslavia, in a small village in northeast Croatia, bordering on Serbia. My family lived in Croatia, but we had relatives in Serbia as well. My family's roots are from the Serbian Vojvodina province, which belonged to Hungary under name Vajdaság until it was annexed into the Serbian-Croatian-Slovenian Kingdom (forerunner of Yugoslavia) as part of the breakup operation of the Austro-Hungarian empire after World War I. (In a way, Serbia then gained a province in north not unlike to how these days it lost one in south.) As such, Vajdaság has a high (alas, dwindling) Hungarian population, and I come from this ethnicity.

I never felt any drawback growing up a non-croatian in Croatia. Nobody in Croatia ever as much as made a remark about me being ethnic minority. Not so in Serbia. Whenever I visited my grandmother in Novi Sad (Serbia) during summer vacation, I experienced strange things. She'd hush me to not speak Hungarian on the street or on the bus. She did neither. The name plate on her door had her name spelt in Serbian (serbian "Jelisaveta" instead of hungarian "Erzsébet" for "Elizabeth"). It was clear you can get into trouble for being different. My whole experience of Serbia was - as far as I can remember - that people there are highly xenophobic and intolerant of their ethnic minorities.

Then came 1991 and the breakup of Yugoslavia. The Croatian region I lived in was overrun by serbian paramilitary troops with full backing by Milosevic's serbian state army. They ruthlessly drove away or slaughtered nonserbian population from the territories they occupied. My family fled with one car trunk worth of belongings when these thugs were approaching. We lived next to an improvised Croatian police station, and we later learned we were targetted as "Croatian collaborators" by paramilitaries because we were on cordial terms with the police officers. They broke into our home on a night after they occupied the region. I have no doubts as to our fate if they found us there.

Mind you, at the time police officers with handguns were the only armed force the just-born Croatian Republic could stack against the Serbian-controlled "People's Army of Yugoslavia", the biggest and most heavily armed force in Balkans in 1991. They had all the chance of a snowflake in hell to defend our homes against the occupators.

While Serbia was significant territorial influence in the breakup of Yugoslavia, it clamped down even harder on its own ethnic minorities, trying to prevent further loss of grip on its remaining territories with the oppression in both Vajdaság and Kosovo growing year after year under Milosevic regime. It culminated when Serbia attempted to eradicate the Albanian minority (minority when viewed against overall population of Serbia, but a 95% majority in Kosovo) in 1999 using its military. This led to the well known NATO intervention when Serbia was bombed by US and its allies until its warlords lost the backing of the population and were overthrown in a revolution.

But the damage has been done. The Serbian state consistently over several decades mistreated and oppressed its ethnic minorities. After what they experienced under Serbian regime for decades, ethnic Albanians of Kosovo wouldn't trust'em as far as they can throw'em. The Serbian state is reaping what they sow now.

It's ironic, but I do actually believe that the recently elected Serbian government might actually be a modern european democratic government that would treat its minorities as a modern european democracy should. (Provided they don't assassinate their prime minister again for being too European...)

But it's simply too late.

There was a huge demonstration this evening in Belgrade. There were atrocities. Embassies and banks were burned. The prime minister spoke to the crowd, fueling it, and the police didn't stop the hooligans. It's sad how they still lay the blame everyone for the situation except themselves, and their decades of hostile politics. I have no illusions this will change soon. I have no doubts that the long oppressed Albanian people of Kosovo are better off in an independent state. They finally will have the chance to bring prosperity to the long neglected region. The region will finally have a government that feels it belongs to the land. As far as I remember, Kosovo was always extremely poor. Serbians have strong emotional ties to the region because Kosovo is the historical site of birth of Serbian state and church, but aside from that, Serbia was a very lousy custodian of the region, not bothering developing it, or helping it develop, or even just not actively hindering any economic progress in it in recent history.

My father packed his two children and wife into his car on 20th August of 1991, and pressed the pedal to the metal until we crossed the border to escape certain death from Serbian paramilitary thugs. Dad spent the rest of his life in exile. Even after our former Croatian homeland was liberated, the six years of Serbian rule set it back economically, infrastructurally, and most importantly socially for decades - it still didn't recover as most young people, including me, departed the region and didn't go back, decimating the society's renewal potential. There was simply no place to go back to, as that land was no more the same land we left. So Dad didn't return either although I know his heart ached for an alternate reality where all of this didn't happen, the peaceful continuation of days of old, something that is not ours to experience in our lifetimes, taken away from us by force by aggressive neighbors' selfish geopolitical interests. I wish Dad was still alive to witness how those same aggressive neighbors are now in pain too; while it doesn't cure our wounds, he would certainly find some poetic justice in it.

Schadenfreude? Damn well yes, we're entitled to it.

Oh, the irony

2008-02-20T23:20:00.006+01:00

I'm reading Vernon Vinge's "Rainbow's End". At one point, he describes a day of an old man who's been cured from Alzheimer's in near future. Everyone is using wearable ubiquitous, always connected devices to access any data anywhere, and he's given a foldable electronic paper like device (rudimentary compared to what young kids are using) to access the web:

He wandered around the house, found some of his old books in cardboard boxes in the basement. Those were the only books in the entire house. This family was effectively illiterate. Sure, Miri bragged that many books were visible any time you wanted to see them, but that was a half truth. The browser paper that Reed had given him could be used to find books online, but reading them on that single piece of foolscap was a tedious desecration.

The irony? Rainbow's End is available for free here legally, and I'm tediously desecrating it in my web browser :-)

As a matter of fact, I don't like ithe medium. It's not the first novel I read on my computer, and probably not the last (I read few Cory Doctorow novels this way, bought them all in book form since), but I much prefer holding a deadtree book in my hand for my night reading. Especially when I spent the entire day anyway in front of the said computer. (But have been few time in a situation when I wished for Command+F to quickly go back to something while reading a deadtree...)

OTOH, Tor books started a free ebook program "Watch the skies" recently (non-DRMed PDFs); Jon Scalzi's Old Man's War is coming out soon on it. Karl Schroeder's Ventus is also available for free. Neil Gaiman's American Gods will also be e-published for free availability soon. I sense a trend here.

Ignorance is bliss

2008-02-20T19:08:00.002+01:00

"Hi. My name's Attila, and I write shitty code."

The latest Really Bad Practice I managed to implement was making some business-level code aware of its underlying infrastructure. In particular, made them aware of Spring's ApplicationContext and such.

Ignorance is bliss, and this goes for code as well. A protocol handler unaware of transport peculiarities can be reused with any transport. Code that is unaware of memory management will automatically benefit from improvements in garbage collection.

The less your code knows about the context it is used for, the easier to reuse it in different context, but even more importantly, the easier for the context to manage it as it is supposed to do.

With dependency injection (DI) stuff like Spring, making components aware of the existence of it is bad, but it won't necessarily become apparent immediately. But when you want to implement something more involved; say, a graceful shutdown for your service, you'll suddenly no longer be able to have the infrastructure do the work for you. In my particular case, I could no longer rely on the dependency graph maintained by Spring after some of my components directly pulled some other components from the application context.

Of course it was a stupid thing to do. I usually know better.

As an excuse, let me say I only resorted to this in rather ugly situations. There are asynchronous callbacks from external systems, through mechanisms that make binding to the infrastructural context "normally" hard. And there's Java deserialization, the notoriously DI-unfriendly mechanism where you either resort to thread locals or statics (which reminds me of Gilad's new intriguing blog post "Cutting out the static", by the way). (Dependency injection in deserialized objects is something Guice user's guide will also admit being a problem for which the best and still quite unsatisfying solution is static injection.)

So yeah, I have the excuse of committing the faux pas when faced with a tough situation, but still. (Mind you, eradicating all Spring-awareness alone won't solve my problem of graceful service shutdown while it might still be waiting for asynchronous responses from an external system, but would certainly go long way toward it.)

The lesson is however clear; it is often the path of least resistance to reach from your code down to a supporting layer, but it can easily come back to bite you when the said layer was meant to be invisible. You think you might need to expose only a bit of a plumbing, but as time goes on, you realize that if you continue, you'll end up either uncovering the whole goddamned kitchen sink, or having to reimplement some of it. Then it's time to finally notice what you're doing (better late than never, I guess), backtrack, and refactor; bury the infrastructure back to where it belongs, not visible at all from the higher level. It sure does make some fringe cases harder to implement, but the important thing is that it keeps the easy bits easy.

Tom Lantos died today

2008-02-11T17:49:00.000+01:00

Tom Lantos died today. One less great Hungarian and one less great American in this world; I was a serious admirer of him and his work. Even if not always agreeing with all his views, I do believe he made the world a better place through most of the things he did. I remember being amazed by quite a lot of things he did, but I won't rehash them here - Wikipedia is as good source as any for this. I distinctly remember him from two years ago when I saw in the news that the (then) 78-year old member of the Congress (elected 14 times, no less) was arrested for civil disobedience while protesting in front of the sudanese embassy in Washington against violence in Darfur. I was proud. I'm sorry though that even his influence and chairing of House Foreign Affairs Committee was not enough to move US into ending the Darfur conflict. Maybe if he was given a bit more time...

Isten nyugasztalja békében, Tamás!

Laptops at risk at US ports of entry

2008-02-07T21:55:00.000+01:00

This keeps resurfacing in media every few months. This time, it's a Washington Post article about US Customs and Border Protection officers confiscating travelers' laptops (for indefinite time - some people didn't get theirs back for more than a year, despite being promised they'll get them in 10 to 15 days), or making copies of data on them, and/or forcing the people in possession of them to reveal their logon passwords. Also, people objecting to the procedures are denied entry to US.

Well, one more reason not to travel to US. At least, not with a laptop. Although, regardless of whether you have a laptop, they'll take all your ten fingers' prints when you enter, and that's also a rather strong cause not to. Over here, they take your fingerprints when you're taken in custody as a crime suspect. So depending on your cultural conditioning, having your fingerprints taken can be quite a humiliating experience. (I had my two thumbprints taken already on my previous US visits, and I detest the practice very much.)

Back to laptops and data.

As you might have seen from a previous weblog entry, I use FileVault on my laptop. Back when I used Windows, I used E4M for a similar purpose (although today I'd probably use TrueCrypt instead). FileVault is a 128 or 256-bit AES encrypted disk image for your home directory on Mac OS X. I even use encryption on my swap files.

I have very good reasons to keep all of this encrypted, reasons of both private and professional nature, that I do not wish to elaborate on further. If I were faced with the choice of handing over that data or being denied entry to US, I'd choose to not enter. Owners of some of the data that I keep on my laptop would certainly agree. (Yes, I keep data that doesn't belong to me but I'm trusted with it. If you work for a company in any significant position, chances are, you keep such data too).

Alternatively, in the near future, I can burn a BluRay disc with the contents of my home folder (encrypted), send it to my temporary US address in mail, and travel with laptop erased (or quickly erasable). Which still doesn't save me from the prospect of having my laptop confiscated at the border just because they can.

I'm lucky, 'cause I can mostly avoid going to USA if I don't want to. Some people on the other hand return home there; they don't have much choice aside from not leaving the country. Ugh.

Time Machine + FileVault experiences

2008-02-05T10:52:00.000+01:00

I was reluctant to use Leopard's Time Machine "set it and forget it" backup because I also use FileVault (which basically mounts an AES-encrypted disk image in place of your home folder). The web was full of warnings how Time Machine does not work with FileVault, or does but it only backs up your home folder only when you log out, and you lose the ability to restore individual files through the GUI and need to fiddle with manually mounting the backed up images if you want to fish out something from them. Seeing however how I was getting undisciplined with my manual backup routine, I decided it can't be worse than having no backups, and went ahead and gave it a try.

At first, I was surprised to see that contrary to advertised, it did actually back up the encrypted disk image that hosts my home folder. It did it every hour. Every hour, it'd push the 30GB disk image over to the other drive. That filled it up, well, rather quickly.

Digging around, it turns out that the reason for this is that I kept using the Tiger-created FileVault, that uses a single file for the disk image. And Time Machine will happily back it up. 30 GB/hour.

So, next step was trying to upgrade to Leopard FileVault format, which uses a new "sparsebundle" disk image format, which is basically a folder with 8-MB files called "stripes" that hold the contents of the disk, plus some other files for tracking what's where. The ugly part of it is that in order to "upgrade" FileVault, you have to actually turn it off first (so it unpacks your disk image contents on the main filesystem), then re-enable it. I left it to decrypt over night (it probably only took an hour, but I left it there and went to sleep), then re-encypted in the morning (which took 40 minutes for 15GB of content). And then a secure wipe of the free disk space.

An immediate enormous benefit is that my disk image shrunk from 30GB to 15GB. That's right: my old disk image took 30GB even when it was hosting only 15GB of content, and no amount of compacting would've taken it lower. And it wasn't because of filesystem slack - inspecting the image with Disk Utility showed that there's indeed 15GB in there reserved but not used.

Now it's only 15GB, as I would expect it to be, with another 15GB reclaimed on my HDD. Hooray.

Another enormous benefit is that I no longer have Time Machine push 30GB over the FireWire every hour. Whenever I log out though, FileVault will compact the disk image (as it did in Tiger), and then Time Machine will back up - only those 8-MB stripes that actually changed, so the process is rather quick.

It is easy to understand why doesn't Time Machine back up the FileVault home directory while it's mounted - it would be too easy to back it up in inconsistent state as data is shuffled across stripes. Of course, I wish Apple engineers had more time to think about this, and solved it in a smarter way. I myself could tell them two better ways to handle this:

One: add a shadow file to the disk image while backing up to hold concurrent changes, merge changes into the image file upon backup finish. The underlying BSD foundation of the OS supports this. It would, however, probably create a perceptible temporary freeze of the system while the changes in the shadow file are merged with the disk image.

Two: create a similar encrypted disk image on the backup drive, mount it when the backup reaches the home folder, and just perform the whole Time Machine backup procedure between two disk images. I actually had a homegrown solution that did precisely this using rdiff-backup back on Tiger. Actually, when I first heard of Time Machine, I sort of hoped Apple will base it on rdiff-backup, and use this method to handle FileVault accounts.

rdiff-backup has the advantage that it can incrementally back up small changes in large files using the rsync algorithm (Time Machine copies whole modified file each time), and my method also preserved this incremental backup property in FileVault accounts, on a per-file basis, preserving both filesystem and backup semantics. I guess they lacked one smart guy in the engineering division for Time Machine, who was probably busy helping the iPhone division make their deadline... Oh well.

Anyway, now with tolerably fine-grained FileVault backups, I'm happy. Yes, I need to log out in order for my home folder to get backed up, but I was doing this for a while anyway, using CCC or Disk Utility to copy the whole internal disk to external. I used to do backups once a week; now I get automatic backup of the system every hour (which could come in handy if ever, say, a software install goes awry; never happened on Mac with me before though), and automatic backup of my home directory whenever I log out (which is not less frequent than once per week). Of course, the majority of my machine's state change happens in my home folder, so having its backup be more frequent than the system backup would of course be preferred, but such are compromises - I can't afford to run without FileVault.

(You might ask what happened with my homegrown rdiff-backup solution? It fell victim to my switch from a PowerPC to Intel Mac, as it would've required me to recompile a bunch of GNU stuff from source (rdiff-backup and its paraphernalia) which I didn't have time to do at the time of the switch, so it fell into oblivion...)

Kurt Gödel: hacking the U.S. constitution

2008-01-14T08:43:00.000+01:00

I've been wandering through Wikipedia yesterday, and at one point ended up reading the page on Kurt Gödel. Gödel's achievements in the field of logic are indispensable to modern mathematics, and his incompleteness theorem has very far reaching implications in disciplines other than mathematics¹.

It is safe to say he was your typical deep thinker, and inward facing, not-too-closely in touch with reality type of person. This snippet from the page made me laugh real hard, because it so perfectly illustrates certain aspects of a math nerd. Listen:

Einstein and Morgenstern coached Gödel for his U.S. citizenship exam, concerned that their friend's unpredictable behavior might jeopardize his chances. When the Nazi regime was briefly mentioned, Gödel informed the presiding judge that he had discovered a way in which a dictatorship could be legally installed in the United States, through a logical contradiction in the U.S. Constitution. Neither judge, nor Einstein or Morgenstern allowed Gödel to finish his line of thought and he was awarded citizenship.

(emphasis mine)

Does anyone know whether he was allowed to finish his line of thought at some other time? (Not that I'd be personally interested in executing the idea, mind you.)

¹ Including the fact that free will of any mind is just an illusion it has because it can't contain a complete model of itself it could use for correctly predicting its own behaviour in advance. Enjoyable presentations of incompleteness theorem for layman include Douglas Hofstadter's "Gödel, Escher, Bach" (if you're 16 or older), or Raymond Smullyan's "The Lady or the Tiger?" (if you're under 16).

Sim City goes open source

2008-01-13T16:52:00.000+01:00

The source code for the original Sim City (the game responsible for countless hours I spent sitting in front of a computer when I was 16), have been released under GPL v3. If you ever played it (you did, right?), you had to admire all the cross-interaction of your planning decisions and certainly wondered about the underlying mechanics. Well now, you can read it first hand!
One of insightful quotes from the announcement:

The modern challenge for game programming is to deconstruct games like SimCity into reusable components for making other games!

That is a very important point, and illustrates well the unique aspect of exponential utilization opportunities of open source software. (Even if I personally believe more liberal licenses than GPL contribute towards this aspect more).

Java 6 on Leopard. Well, almost

2008-01-08T17:46:00.000+01:00

So, it looks like there's finally a developer preview of Java 6 for Mac OS X Leopard. Am I happy? Nope. Why? Here's why:

		CPU Architecture
		PowerPC	Intel
CPU bits	32-bit		Machines I have
CPU bits	64-bit	Machines I have	Machines running Java 6

It only runs on Macs with 64-bit Intel CPU. Now, I have a 64-bit PowerPC Mac, and I have a 32-bit Intel Mac, but no 64-bit Intel Mac, so no cake for me. Darn. Hopefully the final release will run on all hardware that Leopard itself can run on.

Boot Camp experiences

2008-01-08T11:02:00.000+01:00

So, I used Boot Camp (2.0, the one that comes with Leopard) to install Windows XP on a small (16 GB) partition on my MacBook Pro. Reason being that Half-Life 2 won't run under VMWare Fusion...

I'll summarize the experience here, with emphasis on problems I encountered and how to get around them.

First, the good things. In contrast with older Boot Camp versions (1.0 and 1.1 for Tiger), you no longer need to burn a CD with Windows device drivers for Apple hardware. They are now included on the Leopard install DVD, so after Windows is installed, you just need to pop in the Leopard DVD into the drive, and let Windows autorun feature start up the driver installer from it. It will also install Apple Software Update in Windows, and it will presumably keep the drivers up to date (there are no updates right now, so I can't verify this. It will offer to install QuickTime and iTunes though...). Mac keyboard extras work perfectly (volume control buttons, brightness control buttons, eject button). All hardware - graphics, sound, wireless seems to be perfectly supported. The trackpad gestures also work as expected. I only maybe wish they could have made the Windows recognize Cmd+Tab instead of Alt+Tab for application switching...

Now for the first obstacle: Windows would not display image on the external display attached to my MacBook Pro no matter what I did. Others reported the problem on various message boards, with different proposed solutions. After several dead ends, I went exploring on my own. (Note: this solution works for MacBook Pro machines with an ATI chipset.) I ended up downloading ATI Catalyst Software Suite. It unpacks into "C:\ATI" folder by default. Within that is "C:\ATI\CCC" (for "Catalyst Control Center") folder, with a "setup.exe". It will tell you that it needs .Net 2.0 runtime to run. You can get it here.

So, install .Net 2.0 runtime, then the Catalyst Control Center (CCC). Don't try to install the full ATI suite, as it will not succeed, only install CCC. Launch CCC after installed. There is a tab where you can enable/disable various outputs and specify their order. Turns out the ATI chip in my MacBook Pro has 3 outputs (built-in display and two externals), of which two are enabled (built-in and one external), and one external is disabled. The trick is to disable the default enabled external output, and enable the one that was disabled! Voila, the external display lits up! (Regardless of whether you're connecting directly via DVI, or, as I do, through a DVI-to-VGA adapter -- there were reports on the net claiming only DVI displays will work. Fortunately, this is wrong.) Finally, you can use CCC to swap the order of the displays; if you want the external display to be the primary display (one carrying the start button and the tray (as I do)), you'll also do this.

That's all there is to it.

Second obstacle: with a newly created Windows partition, Spotlight had problems after booting back to Leopard. It started indexing it, and never finished. It was pegged at "3 minutes remaining" for about a day, and couldn't be used for searching during this time. This was particularly painful for me, as Spotlight in Leopard is so much improved that I use it all the time, especially as a lightning-quick application launcher. Solution was to disable Spotlight for the Windows partition. I guess you could do it using Spotlight preferences, but I did it by typing

sudo mdutil -i off /Volumes/Windows \HD

from Terminal. (Of course, your Windows partition might be named differently, i.e. "NO NAME" instead of "Windows HD".) This immediately stopped indexing, and Spotlight was usable again. This might be a problem for you if you keep some content you wish to search on your Windows partition, but since I only keep few games on it, it's not a big deal for me.

Other than these two issues, I experienced no problems and am a happy camper (pun intended).

lolcatshost

2007-12-30T15:14:00.000+01:00

Some people clearly have too much time on their hands, witness lolcatshost!
(Via Little Gamers)

UPDATE: actually, it seems like the site was inspired by that Little Gamers webcomic.

Terminus

2007-12-30T14:03:00.000+01:00

Via Boing Boing:

A silent concrete monster follows a nervous businessman through Montreal in Terminus, an eerie and darkly funny short film directed by Trevor Cawood

Go here to watch, it's only eight minutes, well worth your time.

Bedtime

2007-12-28T22:56:00.000+01:00

So, just as Christmas passes on December 26, I suddenly fall sick. Perfect. I haven't logged a single sick day for more than over a year, maybe even two years, and now that I'd have few spare days to spend on things I'd like to do, I'm more-or-less tied to bed. Don't worry, nothing too serious, just stuffed nose, terrible coughs, and enough fever and chills to keep me not wanting (or, in fact, being too able) to move anywhere outside of the bed. The usual "season's greetings", you might say.

I'm reading though. I read some wicked good books in the past months, including Charlie Stross' Glasshouse, as well as an überweird Cory Doctorow piece (Somebody Comes To Town, Somebody Leaves Town) as well as few Karl Schroeder novels. I'm through most of them by now and am tasting some Vernon Vinge momentarily, which isn't as good as the more contemporary guys (Stross and Schroeder) are, but is still much better than Iain M. Banks is.

Or even better, I listen to wife reading Mr. Pratchett's The Bromeliad Trilogy to kids (and me!). I picked up Truckers in a book store about a month ago without knowing it is targeted for children, but after reading it I realized it'd be great fun to read it to kids in the evenings. Usually it is me doing the reading, but with sore throat... So, we finished Truckers today (second reading for me), and started on the Diggers (which, along with the closing volume of the trilogy, Wings, I picked up today at the same bookstore. I have a habit of buying only the first novel in a series and making subsequent purchases based on whether I'm still interested in it after the first book ends. For me, Iain M. Bank's first Culture novel didn't hit the bar, so I'm avoiding it for now).

Anyway, my Mom was over for holidays, and listened in to our reading of Truckers. As a result, I gave her one of Discworld novels (Mort) to read on her way home. I'm pleased to say that Mr. Pratchett has obtained yet another (in this case, 58 year old) fan :-)

Anyway, days are a bit bleak. I'm usually not well enough to hack or even to play much with kids, or anything of the sort. Getting to a store to shop for groceries to get us through New Year's Eve was quite an effort and I felt much worse afterwards.

Oh, I also picked up Half Life 2 retail box three weeks ago on an impulse while shopping for Christmas presents. Tried to install it this morning (lacking better pastime since I couldn't sleep from my cough rush), and it claims the CD key is invalid. Swell. Valve support didn't answer within a workday. Knowing me, if I don't get to try playing it during the holidays, I probably won't find time for it anytime soon, so it's as good as money thrown out of the window if they don't resolve this quickly. Way to go, selling people unusable DVDs.

I also managed to drop my photo camera from a waist height on a hard stone floor during Christmas - I was trying to take pictures of kids while they were having fun with Sparklers. In response to the drop, the camera's objective developed a very ugly skew. Not entirely surprisingly, it doesn't work anymore... Closest Canon service (and the only one in the country) is of course, where else, in Budapest, 200km away. Buying a new one might be a more viable option.

Oh well. I guess I'm going back to bed. Sorry everyone for a bad-mood rant...

www.opensource.org hacked?

2007-12-27T17:29:00.000+01:00

Does anyone know what happened to http://www.opensource.org? It's supposed to be the website of Open Source Initiative, but it looks like right now it serves the same content as http://hyperreal.org, whatever that might be... It is either hacked, or maybe the two sites are collocated on the same physical HTTP server and there was a config glitch. Anyway, quite annoying...

Rhino in Spring 1.2

2007-12-27T17:10:00.000+01:00

So, I've used some free time around the holidays to put the finishing touches on a new release of Rhino in Spring. The new release is 1.2, and it has one major new feature (as well as few featurettes). The major big thing: it is now fully clusterable with Terracotta!

Actually, that's pretty much it. There's few other featurettes as well, but the real big leap forward was restructuring of the internals so that it plays nicely in Terracotta distributed clusters. Configuration for clustering is dead easy: it comes with a Terracotta "configuration module", a special JAR file hosting an XML file with clustering declarations for the code. You just reference it in your tc-config.xml and that's it. The rest is Terracotta magic :-)

FreeMarker 2.3.11 released

2007-12-07T12:17:00.000+01:00

We released FreeMarker 2.3.11 on Wednesday. There are few new feature goodies as well as some significant performance improvements (both speed and memory) in there, not to mention bugfixes, so you might consider upgrading from earlier 2.3.x version. Head over to the project website for more info and download.

13949712720901ForOSX

2007-11-28T09:03:00.000+01:00

13949712720901ForOSX

Working Java 6 on Mac OS X - but not from Apple

2007-11-27T14:04:00.000+01:00

Turns out there's a working Java 6 for both Mac OS X Tiger and Leopard. It is a port of the Java Research License licensed codebase for BSD, so to use it legally, you need to agree to JRL first. It's developed by a person named Landon Fuller, and he says on the page:

The Mac OS X work is based heavily on the BSD Java port, which is licensed under the JRL. The BSDs develop Java under the JRL; FreeBSD has negotiated a license with Sun to distribute FreeBSD Java binaries based on the JRL sources.

As the Mac port stabilizes, I am merging my work upstream into the BSD port, and in turn, it is a goal of the FreeBSD Java project to merge their work into OpenJDK. I've signed a Sun Contributor Agreement in preparation for this...

There are some minor features lacking (minor if you're like me, using Java for server side): sound doesn't work yet, nor is there a native Aqua Swing UI.

Via Charlie Nutter (who also perftested it with JRuby and sounds quite impressed).

New MacBooks finally have 3D graphics in hardware

2007-11-18T20:52:00.000+01:00

I just noticed that new MacBook models ship with Intel GMA X3100 graphics chipset instead of GMA 950 in previous models. Notable difference is that X3100 has 3D rendering hardware - a shader processor with 8 execution units fully supporting transform & lighting, with support for DirectX 10.

That's quite a big news for people who previously dismissed MacBook as an option because of weak graphics chipset (i.e. people who'd dual boot into Windows for gaming).

On the downside, GMA X3100 still doesn't have dedicated RAM, and it'll actually take away 144MB of system RAM (160 if you use an external display), compared to 80MB that GMA 950 used, so YMMV.

"Some thoughts on security after ten years of qmail 1.0"

2007-11-17T14:45:00.000+01:00

There's a paper "Some thoughts on security after ten years of qmail 1.0" written by qmail's author, Daniel J. Bernstein. (I found it though Bruce Schneier's weblog). The paper is well worth reading because Daniel is extremely security-conscious. As he says:

In March 1997, I took the unusual step of publicly oﬀering $500 to the ﬁrst person to publish a veriﬁable security hole in the latest version of qmail: for example, a way for a user to exploit qmail to take over another account. My oﬀer still stands. Nobody has found any security holes in qmail. I hereby increase the oﬀer to $1000.

He actually gives lots of practical advice on how to make your code more secure: security holes are bugs, so you should strive to eliminate bugs in general. Also, you should strive to eliminate code (as number of bugs is usually proportional to number of bugs). Finally, you should minimize the amount of trusted code and have all untrusted code run in a sandbox (using his words, a "prison").

Now, to reflect a bit on sandboxing myself from a Java point of view, Java is fortunately quite well suited for minimizing trusted code. When you run a Java program under a security manager, code gains privileges based on where it came from, and the privileges of the current stack frame are the intersection of privileges of its code and all of its callers down the call stack.
There's a facility for privileged execution, SecurityController.doPrivileged(), where the code within the privileged block will run with full set of its own privileges, and not get narrowed down further with security restrictions placed on its callers. It can be a dangerous feature, and must be used judiciously lest you create an exploit vector through it. I.e. if your code accepts a file path from its callers, reads the file in a privileged block, and returns the file content to the caller, then a caller can potentially exploit this to gain access to contents of files it would otherwise not be able to see.
I'm actually quite conscious to enable easy integration with Java security manager in software I write - you will find a PolicySecurityController class in Rhino, as well as a SecureTemplateLoader in FreeMarker, both allowing running JavaScript code or FreeMarker templates, respectively, with their own set of privileges defined in the effective JVM-wide security policy based on where they're loaded from or whether they're digitally signed (same aspects are used to assign privileges to Java code). FreeMarker also uses SecurityController.doPrivileged() in few places judiciously.

Anyway, back to Daneil's paper: there's a wealth of food-for-thought for any practicing software developer: how we often sacrifice security for speed, how hidden data flow (global variables and singletons, people!) leads to security issues, and how this can be fixed by further decoupling of components with explicit narrow communication (i.e. message passing across processes), and so on.
A section entitled "Avoiding parsing" also speaks very closely to my heart.
So, recommended reading.

Wired profiles Randall Munroe

2007-11-16T18:48:00.000+01:00

Wired has a profile article on Randall Munroe. Munroe is the Xkcd guy. If you didn't know Xkcd before, you can thank me later for pointing it out for you.

"JVM Dynamic Languages Metaobject Protocol" now released

2007-11-14T20:22:00.000+01:00

I finally got around to packaging up the current state-of-art of my metaobject protocol library as a downloadable release (source + binaries + documentation), plus putting up a very basic website (hosting an information page + JavaDoc) for it.

I haven't got around to setting up dynalang.org yet, so the website is for now hosted at http://dynalang.sourceforge.net.

The fact there is now a release does not intend to confer either a sense of completeness or rigidity. It is versioned at humble 0.3. It is pretty much open to modifications and is also probably not complete yet (i.e. I fully expect people to need further features for integrating with their particular language runtime). The release just strives to make it easier for people to get started with it, as it's now available as a HTTP download instead of only through SVN. Also, having a release means there's now a baseline for purposes of tracking changes in a changelog file etc. Unit tests cover about 75% of the code right now, so it's fairly safe to say it does what it is intended to do, but of course, bugs are always to be expected.

In completely unrelated news, today's my birthday too :-)

Leopard firewall breaks Skype

2007-11-12T20:44:00.000+01:00

Apparently, Skype.app on Mac self-modifies its application package. Leopard OTOH digitally signs application packages after you download them from the net and allow them to run, and also when you allow them to accept connections through firewall. Skype's self-modification causes the signatures to get invalidated. After this, Skype will completely fail to launch. It'll bounce twice in the dock, and that's it.

Options: (a) don't use Skype (b) disable firewall. Damn. I'm going with (a) until Skype fixes this. Actually, I tried disabling the firewall, and for me even that didn't help. Double damn.

New Language Features for JDK7 - from the Java Community

2007-11-12T10:13:00.000+01:00

A newsletter from Belgian Java User Group (BeJUG) landed in my Inbox yesterday, and boy, am I glad to see it. You can read it here. Basically, BeJUG is submitting the JSR for closures in Java, based on Neal Gafter's proposal. This is huge news for several reasons. Reason one, we need closures in Java NOW. Reason two, this is the first JSR filed by a Java Users Group, and also the first Java language JSR led outside of Sun Microsystems.

The second jolly good news in the same newsletter is that Google also submitted a JSR for a "collection of smaller language features". Smaller? They're proposing type inference, extension methods, catch for multiple exception types (Brian Goetz will be glad. Hell, me too!), and improved syntax for easing the catch-rethrow pattern. These ain't small in my book.

We need to realize that C# has had type inference, extension methods, object initializers, anonymous types, and lambda expressions for two years now. It'd be about time for Java to catch up with the times, and I'm really glad to see community finally taking initiative to make it happen and not waiting for Sun.

Network traffic prioritization by specific cost

2007-11-08T21:16:00.001+01:00

I've been following with some interest the recent Comcast scandal where the ISP started inserting TCP reset packets into BitTorrent traffic of its clients, thus disrupting it.

Comcast claims they do it to protect the ability of "normal" users to use the bandwidth from "excessively using" users that hog it.

(I personally believe Comcast tries to minimize the overall amount of traffic that flows out of its network to other ISPs as it costs them money in the fiscal clearing of cross-ISP traffic.)

But let's entertain the idea for a moment that Comcast would really like to protect their "normal" (read: low-traffic) users from their "excessively using" users. How would I go about implementing a fair system that achieves this goal?

Well, certainly not by disrupting the traffic.

Below is an idea that I believe would result in fair allocation of bandwidth within the network of a single ISP with paying customers. I must forewarn you that while I design software systems for living and often need to deal with management of limited resources, I'm just a layman when it comes to the narrow field of packet-switched network traffic management, so what's written below might not make sense at all because of some arcane aspect or the other that I know nothing of :-)

Anyway, here it goes:

The ISPs already measure the amount of traffic generated by every user within a billing period, that's how they are able to detect "excessively using" users today. However, instead of handicapping a user that "excessively" uses the bandwidth, the metrics would be used only to select which user's packet to drop when there is insufficient bandwidth on the routers' outbound lines, but only then.

The strategy would be to always drop the packet coming from the IP address of the user who had the highest traffic in the billing period so far. Or, in case of differently priced packages, or even different billing periods, the user with highest traffic/fee ratio for his current billing period, expressed nominally in byte/dollar.

This way, the proverbial "only checking his mail" user would always experience a fast connection, followed by a user who listens to online radios and watches some amount of online videos, while the "excessive" p2p users would be only competing for leftover bandwidth among themselves.

It'd be easy to suitably extend the idea to a nondeterministic scheme where a packet to be dropped is picked randomly, but the probabilities are weighted by their user's dollar/byte ratio. That'd give a user who heavily used p2p at the start of the month a standing chance to still be able to check his e-mail even when another user comes along who floods the network with p2p traffic near the end of the month.

Feel free to point out why this wouldn't work, doesn't make sense, or is already invented in this or similar form and used somewhere :-) I've tried uncovering this technique using Google, but have found only port/service/application based QoS prioritizing, and didn't find anything specifically describing this -- namely, prioritizing based on user's specific monetary cost of traffic, amortized over a billing period.

As far as I can tell, this method preserves network neutrality, as it does not discriminate based on either the type of traffic or the destination of the data. It also doesn't degrade the throughput for anyone in any way as long as there is sufficient free bandwidth.

Leopard so far

2007-11-05T18:43:00.000+01:00

This Thursday, I upgraded my MacBook Pro to Mac OS X 10.5 Leopard. It was a holiday over here, so I had four days (Thursday to Sunday) to work out the wrinkles in case something didn't work. I did the usual precautions: mirror the boot drive to an external drive, boot from the mirror to make sure it works, then shut down the computer, physically disconnect the external drive, and only after all this start the OS install.

I decided to give myself a chance to avoid reinstalling everything, and thus chose the "Upgrade" option instead of a clean install. After an hour of DVD consistency check and an hour of copying files, the machine rebooted and I was greeted by a working Leopard. It went incredibly smoothly.

I proceeded checking whether everything works. First on the list was Cisco VPN Client which I need for work. I was sceptical, as the Cisco VPN client integrates tightly into the system, installing kernel extensions for networking. To my very pleasant surprise - it worked! Next was ssh, which also worked. VMWare also works, no problems there.

Then there's some more arcane stuff:

An unsupported printer driver for my Xerox Windows-GDI printer, cross compiled to a PowerPC binary from a Linux CUPS driver - works!

My heavily customized, start-on-boot MySQL setup - works!

There's of course no Java 6 yet, as discussed earlier, but the good news is Java 5 is at least upgraded from 1.5.0_07 (found in Tiger) to currently latest 1.5.0_13 in Leopard.

Eclipse 3.2.2 works overall. There are keyboard issues though. I.e. if I press Command+Shift+U ("Occurrences in File"), it brings up a context menu (to choose among "Identifier", "Implementing Methods", and "Throwing Exceptions"). However, both the menu and editor window receive the keyboard events for arrows, and when I hit Return, the context menu disappears altogether (without performing the selected operation), and the key is interpreted by the editor (i.e. inserts a new line...). That's one minor annoyance.

One of my pet peeves is also fixed finally: I can finally use the Return key to activate the "Allow" button in Keychain Access' "Copy to Clipboard" dialog. Oh, progress! Previously, if I didn't want mousing, I had to do Tab-Tab-Tab-Return. Still no keyboard equivalent to invoke the said dialog though... Also, the "new password assistant" palette now syncs with the password field, so I needn't copy/paste a generated password back to the main editor window of Keychain Access anymore. Yeah, small things, but they do matter.

Also, there was something amiss in my user account in Tiger - lots of programs forgot their license number when I logged off (Delicious Library and Disco most prominently), and it also lost my keyboard shortcuts whenever I logged off, so after a while I lost the habit of using custom shortcuts, as you can imagine... Upgrading to Leopard fixed both issues.

The system is noticeably snappier. Tiger had the tendency to stall under heavy load. "Heavy" is when I run a Terracotta server and two clustered Tomcat instance on the MacBook Pro, talking to a special Linux appliance running in VMWare, occasionally recompiling the code or interactively debugging from Eclipse. Oh, and running a BitTorrent client in the background to boot it. Now, Tiger did have issues with this even with 2GB of RAM, to the point of UI freezing up for 20-30 seconds whenever I switched applications with Command-Tab. Leopard tolerates such loads much better. I guess virtual memory management got smarter with regard to handling applications' working sets, and there are probably lots of minor kernel optimizations and reentrancy improvements. Overall, noticeably better. Lot less beachball sightings.

As for games: Diablo II works nicely even on an Intel mac (it's a PowerPC binary). Of course, only when you install it using the "carbonized" installer, the version on the CD requires Classic subsystem, which is gone from Leopard.

Age of Empires III has issues though. In single player, all is dandy, but playing in network is completely borked by network sync issues. Few seconds into a multiplayer session it'll say "Out of sync. Exit and try again". Truth be told, the other machine still had Tiger on it when my son and me tried it; I installed Leopard on it since.

After two days when I was sufficiently convinced that it works okay, I installed it on my wife's iMac G5 as well, and faced an interesting limitation. You know those custom iChat backgrounds Apple showcases as new iChat features? The ones where you can have the room behind your back replaced with a moving Niagara falls scenery? Well, you won't get them on a PowerPC Mac. I wanted to try it out for fun between our two machines, but the iMac's 2.1GHz G5 CPU is apparently not strong enough for them. Intel Macs only. Now, come on, Apple, I don't buy it. You probably just had a nice hand-optimized assembly with MMX and SSE instructions in there somewhere, and couldn't be bothered to port it over to PowerPC AltiVec. You know, the one you touted as being the superior vector unit. I think we're experiencing the first signs of PowerPC users falling out of grace. I don't want to sound dramatic, but I mean, I can maybe, just maybe understand a G4 not being strong enough for this task, but a 2.1 GHz G5? What about people with dual or quad G5s? This is a minor, but sad letdown.

Also, my MacBook Pro took on the habit of waking up at precisely 1:00:00 AM every night after the Leopard upgrade, without anything being set in the com.apple.AutoWake.plist file. Since 1 AM in my timezone (GMT+1) is actually midnight GMT, I suspected this to be some stray zero making its way into EFI or something. Pretty much, after I overwrote it by setting "Start or wake up every day" in Energy Saver to 9 AM, the phenomenon went away.

Well, that's it so far. I'm also fooling around with XCode 3. It's a very polished development environment, and I'm toying with the idea of creating some native Mac OS app. Y'know, just to unbox myself a bit from my "server side Java" box and try the "desktop app in Objective-C" world for change.

Scapegoating for Java crisis on Mac OS X

2007-10-30T22:20:00.000+01:00

So, pretty much all Java developers working on Mac OS X are filling up the blogosphere (sorry for using the cheesy term) screaming how Apple left them behind not releasing Java 6 for Leopard. This article (via DF) summarizes the overall sentiment (and somewhat hysterical atmosphere). Apple's own java-dev list is another place where people are raising concerned voices.

It's understandable where this outcry comes from. Go to any major Java conference. The Mac laptops are totally over-represented in the audience.

I could well be among them. I'm guilty of amplifying the "Macs are everywhere" effect at conferences myself with my MacBook Pro. I too am a Java developer working on Mac OS X.

I just see a different perspective to things. First off, it always struck me as actually very unnatural that Sun ships Java for Windows, Linux, and Solaris, but does not ship it for Mac OS X. I don't feel let down by Apple here.

I feel let down by Sun.

Sun already publishes a JDK and JRE for three different OS platforms. How hard would it be for them to publish it for Mac OS X as well?

It appeared to me that Sun just recently started natively supporting OpenOffice on Mac OS X. The concept is not completely alien to them, then.

Looking realistically, what is the supposed rationale behind having Apple ship Java for Mac OS X? My educated guess is that the reasons are twofold:

One, OS X was originally so small marketshare-wise that Sun simply couldn't be bothered. They probably had they doubts about the success of the OS. So Apple undertook porting Java to increase the potential software base of the OS and its overall utility. Remember, Mac OS X at the time was still quite young. Hopefully this has changed by now, especially seeing how lots of Sun employees also use Mac OS X. So there's me hoping that in 2007, Sun could actually be bothered.

Two, Apple provided some Apple-specific functionality, like Swing that looks almost like native Mac UI, plus a Cocoa bridge (which is not updated to keep up with Cocoa improvements for quite some time, and is declared by Apple to not be updated ever again).

Don't get me wrong, but the vast majority of Java developers on Mac doesn't give a shit about either Swing or Java-Cocoa bridge, or anything else there might be. Java isn't doing particularly well on desktops; strong Java market is on the server side. As a consequence, vast majority of Java developers don't deploy their software on Mac OS X. They deploy it on enterprise irons that typically run Linux or Solaris. I'm in this crowd.

Let me repeat: I don't give a flying shit about Mac OS X specific Java functionality. If I wanted to write a Mac desktop app, I'd use Objective-C. If I wanted to write a cross-platform desktop app (ha!), I'd probably first talk myself out of it, and if I really couldn't, I'd go for SWT. I'd be more than happy with Java on Mac that allows me to do my command line and server side stuff. I wouldn't care that it has us an ugly looking UI with Swing (as it does on all other platforms, anyway).

And I firmly believe it is Sun's job to give us one, not Apple's.

If you're following what Apple is doing with its developer strategy, it'd be clear to you Apple doesn't have any interest in Java on Mac OS X, and I can't actually blame them for that. Apple is slowly abandoning lots of technologies. It is slowly abandoning Carbon (Carbon will have no 64 bit support, ever). Apple is slowly abandoning non-Cocoa QuickTime bindings (they too will have no 64 bit support). Apple's developer strategy is unification: everything is Cocoa and Objective-C.

In an interesting twist that seemingly (but only seemingly) contradicts what I wrote above about unification, Apple ships Ruby and Python bundled with the OS. However, please observe that both of these are (a) open source and (b) support compiling on Mac OS X in their mainline source code. That is to say, the original developers of the code provide OS X support by sprinkling required #ifdef blocks around the code. Apple just needs to take it and bundle it, without further tailoring. That's what they do. To Apple, Python and Ruby are just Darwin-level tools. And even if they didn't bundle it, it'd be trivial for anyone to compile them from the source and install. Fink and DarwinPorts can give you newer versions of either if you need them. Neither (a) nor (b) can be said of Java. OpenJDK is not Java. Not yet. Sun is definitely heading in the right direction with open sourcing Java, but they started too late to avoid this particular crisis.

Relying on Apple to deliver the Mac part of the WORA promise was a mistake, one that could be easily seen by anyone for years, as Mac users were getting their JDK updates with six months or longer delays compared to platforms serviced by Sun. Sun would've had plenty of time to act upon this mistake.

Again, is it Apple's fault that we don't have an industry-grade open-source Java implementation that supports Mac OS X in its mainline source code? I don't think so. If you want to be angry at someone, or better yet, you want to petition someone about the situation, please direct your energies at Sun.

UPDATES:

OS X Java Definitive Timeline shows the problem of letting Apple - a party for who Java is low-priority business - take care of delivering it on its platform: worst lag in release schedule was 21 months for Java 1.4 to show up in Mac OS X.

Also there, Comic: Apple’s Pathetic Java Support? Oh Really? saying
Dear Steve,

Writing your own JDK is really hard. Maybe you should follow Microsoft’s lead and let Sun do the dirty work?
Well, yeah, if only Sun wanted to do it.

In Shipping means prioritizing Gruber also reinforces my point:
Perl, Python, and Ruby pretty much compile out of the box on Mac OS X. Apple doesn’t have to do much at all — at least relative to Java — to include them on Mac OS X. Why? Because that’s how these tools are designed and engineered — they’re made to “just build” on any Unix-like OS. It’s not Apple’s responsibility that Java isn’t like that — it’s Sun’s.
He's also quite confident we'll see Java 6 in Mac OS shipped soon, if the timeline above is any indication.

Fastest Vista laptop is MacBook Pro

2007-10-30T10:12:00.000+01:00

According to PC World's "The Most Notable Notebooks of 2007" article, of all laptops they tested this year, the one that runs Windows Vista fastest is - MacBook Pro! They conclude with:

... MacBook's score is far more impressive simply because Apple couldn't care less whether you run Windows.

(Via Daring Fireball)

Linux preinstalled on Acer laptops - a scam?

2007-10-18T08:50:00.000+02:00

Here's a bit of a firsthand bitter experience with laptops and Linux from last week. (It's also another hard reminder why I run Apple computer gear exclusively for myself for two years now.)

My wife's aunt wanted to buy herself a computer. She wanted the "usual" functionality, y'know, browse the web, e-mail, listen to music, watch movies, online chat. It also must have a hungarian UI. I suggested she gets a desktop computer, but she insisted on a laptop. Oh well, it's her money.

Her budget was half of what you'd need for a MacBook. Not wanting to enrich Microsoft's Windows division, I suggested we get a laptop that comes with Linux preinstalled. I had some successes installing Ubuntu Linux on some machines lately (one of them an oldie IBM ThinkPad where Ubuntu even recognized the PCMCIA wireless card without a hitch). Also, seeing how she lives 100 km from here, I didn't feel like doing tech support on it often, which is something I believe I would end up doing with Windows (if my wife's machine is any indication).

My argument was that for the same money, we'll get a stronger machine, since the manufacturer didn't have to scale back the hardware budget to accomodate a Windows license.

We ended up buying an Acer Aspire 5315 that's distributed locally in a Linux configuration for 120000 HUF (660 USD).

Now, the first thing is that it comes with a Linux distribution named "Linpus Linux". There's no install CD so I can't reinstall it if anything goes wrong. Also, I can't install any graphical interface. That's right. In 2007, a machine equipped with a 1280x800 screen, gigabit ethernet, wifi, DVD burner, and bluetooth comes with a preinstalled OS that dumps you to a shell prompt upon boot. No graphical environment whatsoever is preinstalled. I've tinkered with it a bit, and noticed it doesn't actually bring up the wireless network interface. We'll see later why.

Ok, so let's install a proper Linux on it. I started with Freespire - it's an Ubuntu derivative, but comes with non-free media codecs. Just what I need, so auntie doesn't nag me later that she can't play back MP3 and WMA files. It went up without a hitch, but I hit exactly three problems with it:

Sound drivers claim to work, but no sound comes out.

The Atheros wifi chipset is too new, so it's not yet supported by pci_ath

An attempt to install hungarian language ends up with error message saying qt_language_selector is not found

Trust me, I've spent fair ammount of time browsing all sorts of support forums to solve these, but had to throw in the towel in the end.

Next, I tried a mainline Ubuntu distro (Feisty Fawn). This ended rather quickly - I got dumped into a BusyBox command prompt by the installer. Turns out FF doesn't recognize the chipset, and thus can't detect the hard disk drive properly, and as a last ditch measure gives me a command prompt. Geez...

At this point, I decided to give Windows XP a try. I had a copy of WXP SP 2 at hand, so I tried installing that if only to see whether the sound might be defective in hardware (if WXP drives the sound chip, then it's apparently not). To my utter surprise and disbelief, WXP installer also failed to recognize there is a HDD in the machine. Ouch. Friends are telling me that I'd need to provide the installer with SATA drivers externally. Yeah. Provided I can find them. And then there's the minor issue that Windows XP installer only accepts external drivers through a - you guessed it - floppy drive! Yes, there are USB FDDs. I've also heard some newer BIOSes can present a USB pen drive as a floppy. Still, what would've it taken to write the damn thing so that it can accept external drivers from a CD?

Back to Linux - there's another Ubuntu derivative named Kiwi. I chose it because, similar to Freespire, it comes with nonfree codecs preinstalled, and can be installed in Hungarian language by default, so Freespire's language selector problems at least won't trouble me. It was also based on next Ubuntu release - Gutsy Gibbon.

Well, what can I say. It did boot and install, but the sound still didn't work, and I still couldn't get wifi to work. Yes, I did try ndiswrapper with Windows drivers, but even after I blacklisted pci_ath and rebooted, some part of the OS still pulled pci_ath, which took precedence, but was unable to drive the new chipset.

So, I had to admit defeat. I wasted sunday evening, monday evening, and part of tuesday evening on this. My time is worth more to me than this.

So I phoned an IT shop in the city and asked them for a quotation on Windows Vista Home Basic. (Seeing how XP doesn't install, how Acer itself suggests Vista, and how the driver CD only contains Vista drivers.) With Vista, sound works (not a hardware problem then), wireless works. All hardware works.

But here's my question: why does a company sell a computer with Linux preinstalled if there's no Linux distribution currently on planet that can, installed out of the box, correctly drive all of its hardware? What can be said of such a marketing practice? In case of Acer Aspire 5315, at least the wireless and audio didn't work (and I haven't tested either the Bluetooth or the DVD burner, so I can't say they either work or don't).

If they aren't aware of the hardware support limitations, shame on them for being unprofessional. If they are aware of it, shame on them for enticing us on purchasing the machine by representing that it has an OS installed appropriate for it. Duh.

Vista install is running as I'm writing this. I'm cheering myself up thinking of how next time I'll be installing an OS it'll be Leopard on my two Macs.

Language trendspotting at JAOO

2007-10-04T11:12:00.000+02:00

Surveying the talks this year at JAOO, it is hard not to notice that nobody is getting too excited about either Java or C# anymore.

Ruby of course had a full track to itself on monday, and a track named "Enterprise Application Frameworks" on Wednesday was also pretty much Ruby/Rails dominated.

On the other hand, other languages were in the limelight too. We had a great introductory presentation on Erlang by Joe Armstrong, one of the principal inventors of the language. Erlang is the old-new contestant for the title of "right answer to concurrency challenge", and it really looks like it solves the challenge correctly for a fair ammount of use cases I can think of. Basically, whenever you can afford a share-nothing message passing architecture.

We also had a presentation of Scala. Scala must be my favorite language for about two years now even if I didn't write anything in it yet :-). You just got to love all the modern amenities in it that are missing from Java: type inference, very good syntax for map and list literals, closures, a really polished generics implementation (it allows contravariance, by golly!), plus assorted functional programming goodies, not the least of them being Haskell-style datatypes. Oh, and it is static and compiles to a plain Java .class, creating a very low (virtually nonexistent) entry barrier for gradual introduction into your existing Java code.

Oh, and people are talking about JavaScript as well :-)

Glenn Vanderburg gave a talk titled "The Overlooked Power of JavaScript", which was exactly what it said. The language had quite an injustice done to it because of its unfortunate name choosing -- as we know and as Glenn pointed it out correctly, it is not related to Java at all, its ancestry comes from Scheme and NewtonScript (useless trivia: I actually wrote a commercial application for Newton in NewtonScript back in 1996).

JavaScript is an incredibly powerful language if you can get rid of the preconception that you need to handle it as you'd handle Java. Once you get past that and actually understand how things work in it, you can develop some mightily beautiful code in JavaScript, as the extreme dynamism of the language allows you refactorings simply not possible in more constrained languages. You can end up with some very tight internal DSLs. And I'm not talking only about code to run in a web browser -- I'm talking about applications running on desktop or on a web server.

To demonstrate the power of the language even further, there was a presentation of Flapjax on JAOO this year. Flapjax feels like a language, but in reality it is "just" a JavaScript library, but one that allows you to program your browser-run client applications in an event driven manner, plus supporting reactive evaluation (think self-recalculating cells in Excel) when events change values. "Event" in Flapjax can be almost anything -- a tick of a timer, a click of a mouse, asynchronous completion of a HTTP request, and so on. You can concisely express your program in terms of reactions to such events, and also always preserve consistent internal state of all variables thanks to its self-recalculations, all with fairly intuitive syntax. That's quite a something from "just a library" in JavaScript :-) Flapjax was covered on LtU as well about a year ago.

So, all-in-all, language diversity is again in fashion. Developers seem to finally recognize that a language is just a part of their toolbox, and that you need to choose the right tool for the right job. Of course, I know that the same developers usually need to also convince their managers about this point of view, but fortunately this seems like it'll become easier with language diversity now being an accepted trend at respected conferences.

JAOO

2007-09-21T23:16:00.001+02:00

Just a heads up that, following what seems to become an established previous tradition, I'll be attending the JAOO conference this year as well. See ya in Aarhus next week!

Laugh-Out-Loud Cats

2007-09-20T12:27:00.001+02:00

I'll admit that I was mildly annoyed by the whole LOLCats meme. I don't say I didn't get it; I did, I just felt it overdone after a while and couldn't be bothered to pay attention to yet another photo with funnilz missspelt all-caps would-be-jokes superimposed on them.

Until Ape Lad created something wonderful out of it.

Ape Lad, or Adam Koford is an illustrator maybe best known for his graphical interpretation of various hobos from John Hodgman's book "Areas of My Expertise", but I think that soon he'll be referred to primarily as "the guy who created Laugh-Out-Loud Cats".

So, what is it he did with the LOLCats phenomenon? He created a 1920-style cartoon featuring two vagabond cats, as he himself says "one Meowlin Q. Kitteh (a sort of cat hobo-raconteur) and his young hapless kitten friend, Pip". These cartoons are produced traditionally, pen-and-paper, and have an incredibly authentically-feeling atmosphere of the '20s, plus a fantastic chemistry resulting from mixture of innocence, mischief, and cuteness overload in the characters. Just click on the image to the left for the gallery -- it's all available for free viewing on his Flickr account, and I'm looking at its RSS feed with most anticipation every morning for updates (and I'm seldom disappointed, because the guy is really prolific)!

Of course, you might as well check out his other fine works, like his graphical interpretations of various HTTP error codes (to the left, you can see "415 Unsupported Media Type").

Liquids on planes in Europe will be allowed again!

2007-09-18T20:55:00.001+02:00

Frequent air travelers rejoice!

Apparently, Norwegian Ministry of Transportation asked the European Parliament to move to lift the liquid ban on airplanes. They say it is "annoying for the travellers", and a "large cost for society".

And apparently, the European Parliament agrees!

Sorry if I'm sounding euphoric about this, but this mindless fake security measure did cause me lot of headache in the past. Sometimes literally - there are those situations when I need to catch a transfer and the itinerary is so tight I barely can run from one gate to the other and I don't have time to grab an (overpriced) bottle of water at the airport, so I get a headache of dehydration by the end of the trip. small glasses of water and orange juice served at the airplane are not the same thing. Actually, lately I switched to carrying apples with me for liquid replenishment.

In the meantime, I need to find an (a) aerosol deodorant not exceeding 100ml and (b) shaving foam container not exceeding 100ml for my next week's trip to JAOO in Aarhus (I'm flying with hand baggage only). I figure I can just use the hotel bathroom soap in dire need instead of the shaving foam though. Oh man, can't wait for this madness to end.

Closures in Java NOW!

2007-09-16T18:24:00.001+02:00

I remember Neal Gafter's talk about his idea of how to implement closures in Java that he gave at TSSJS in Las Vegas this March, and I have to say, if we had this feature today, it'd be none too soon.

Lack of closures does give Java a reeeally bad feel in 2007. It makes it very backwards looking in terms of modern programming language amenities, compared to languages that already have them.

Here, let me show you an example.

As we all know, in Java, objects can't control their own locking - they can't customize their behavior when used in a synchronized block. Now, that's one entertaining problem in its own right, and would deserve its very own discussion, as basically, it makes correctness of synchronization dependent on implementation. But that's not the point of this post. The point of this post is how one might try to work around this and similar lackings, and how one will inevitably fail...

Okay, the example: let's suppose I have a class A. I sometimes need to lock instances of it. This is a no-brainer in Java and usually has the form of

synchronized(a) {
    ... do something...
}

Let's suppose I also have a subclass B of A. Instances of B have a reference to another object of some class S, and various instances of B share some of their state through shared instances of S for some reason:

public class B extends A {
    private final S shared;
    ...
}

Since the S instance is actually shared among several B instances, the correct locking semantics for B would be to always synchronize on the "shared" field of a B, when client code synchronizes on an instance of B. Unfortunately, there's no way in Java to declare to do a synchronized(b.shared) within every synchronized(b).

What's the next best thing I can do - provided I want to encapsulate the behavior and want to avoid some manual monster code¹ at every synchronization site? A smart aleck like me would add a method to A:

public void runLocked(Runnable r) {
    synchronized(this) {
        r.run();
    }
}

Then, I can override it in B:

@Override
public void runLocked(Runnable r) {
    synchronized(this) {
        synchronized(shared) {
            r.run();
        }
    }
}

Finally, I can replace all occurrences of a synchronized(a) block with:

a.runLocked(new Runnable() {
    public void run() {    
        ... do something...
    }
});

Sounds okay? Well, it isn't. Aside from the very obvious "too much visual clutter" problem, there are further problems with this:

What if "do something" throws a checked exception? (Answer: more monster code²)

What if "do something" modifies a local variable? (Answer: yet more monster code³)

What if "do something" contains a return, continue, or break statement? (Answer: you are seriously out of luck, but some really horrid monster code might help you out. It is so ugly though that it does question the ROI of the whole approach. No, I won't give an example.)

So, what happens is that you eliminate ugliness in one place, only to have it resurface because of these other problems in another place. It is not really possible to arrive at a win-win situation with current language constructs. (Alternatively, I'm not smart enough to figure it out. Either way, pity). Needless to say, if we had real closures in Java instead of the very weak imitation attempt in form of java.lang.Runnable, none of these would be a problem.

¹ the monster code in question would look something like:

synchronized(a) {
    if(a instanceof B) {
        synchronized(((B)a).getShared()) {
            ... do something ...
        }
    } else {
        ... repeat do something ...
    }
}

² the monster code for working around checked exceptions would be:

try {
    a.runLocked(new Runnable() {
        public void run() {
            try {
                ... do something ...
            } catch(RuntimeException e) {
                throw e;
            } catch(Exception e) {
                throw new UndeclareThrowableException(e);
            }
         }
    });
} catch(UndeclaredThrowableException e) {
    Throwable t = e.getCause();
    if(t instanceof Exception) throw (Exception)e;
    if(t instanceof Error) throw (Error)e;
    throw e;
}

And it can get even more fun if you're trying to only get a specific type of a checked exception (say, IOException) and not a generic Exception across the scope.

³ the monster code for working out local variable modifications would be:

final Object[] embeddedLocalVar = new Object[1];
embeddedLocalVar[0] = realLocalVar;
a.runLocked(new Runnable() {
    public void run() {
        ... do something ...
        embeddedLocalVar[0] = newValue;
    }
});
realLocalVar = embeddedLocalVar[0];

Pure beauty, innit?

"How can a spinozan cast a dualism spell?"

2007-09-07T16:23:00.001+02:00

If you enjoy reading works of Jostein Gaarder (come on, you must've read at least "Sophie's World", haven't you?), and if you likewise enjoy reading any or all of Greg Egan, Charles Stross, or Neil Gaiman, then you'd adore a hilarious webcomic about (in no particular order): philosophy (if you can actually get the joke in the post title), quantum physics, psychology (with regular appearances of "Tiny Carl Jung"!), all with a big dose of surrealism.

That's exactly what Dresden Codak is.

If I may recommend, start reading from this one, as it and the next few ones are extraordinarily brilliant (contrasted with the rest, which are just, well, ordinarily brilliant). Then revisit the older ones after you reached the end.

My only woe is that there aren't too many of them yet, and it's not updated too frequently. But hey, quality work needs time!

The World's Toughest Programmer: "How I became a programmer"

2007-08-19T20:39:00.000+02:00

I recently discovered the blog of The World's Toughest Programmer a.k.a Mike Lee, and I can't help already loving him. He hasn't written a lot in his blog for now, but what he did was very thought provoking so far. His newest writing "How I became a programmer" is a must read. Tagline:

I was born a programmer. The rest is just implementation detail.

:-)

Mac Mini colocation

2007-08-01T13:21:00.000+02:00

www.macminicolo.net (Seen it referred from here, being mentioned for their superb consumer care).

As tongue-in-cheek as this sounds at first, yep, these guys do actually provide managed hosting of Mac Minis (you can send one to them, buy one from them at Apple's prices, or even rent it). The idea is a bit weird, but makes sense if you need a dedicated server on the net, you want Mac OS X on it because you are familiar with it, and you don't want to shell out for an XServe. Oh, and yes, you can run Mac OS X Server on Minis if you really need it.

Fun and woes with VNC

2007-07-30T23:13:00.000+02:00

So, we are at a holiday in Palma de Mallorca momentarily. Unfortunately, we could only bring one of our Macs here - my MacBook Pro ' and had to left wifes 20" iMac at home for obvious reasons (it is not too portable, although I did lug it to UK and back once when it was my primary machine). Our hosts however have an oldie G4 eMac. Since both of us need a machine frequently, I tried to remedy the problem using VNC - Vine VNC server on the MacBook, Chicken of the VNC client on the eMac, so one of us can work logged on in their account from eMac while the other person sits at the MacBook proper.

It mostly works fine. But then, there are annoyances.

First annoyance - the system works on two desktops, except when the user of the remote desktop wants to use VMWare. In that case, using fast user switching hangs the remote VNC sesion. No big deal - we will just have the one of use not using it work remotely. (Surprisingly, that is me - Kriszti actually has some Windows-only software she runs).

Second, much bigger annoyance - keyboard incompatibility. I tried both an Apple keyboard plugged into the eMac as well as a Windows keyboard only to realize there's a deeper reason why I can't punch in many characters (curly brackets included - kills any attempt at coding). The reason is that the MacBook still maps keycodes accordingly to its builtin keyboard. I learned this after many futile attempts to press certain keys and evoke any kind of response, I finally brougt up the Keyboard Viewer, and it pretty much displayed the MacBook Pro uiltin keyboard layout... If I try to press keys on a full size keyboard that aren't present on the laptop's built in keyboard, theyre simply ignored. Quite a lot other keys aren't in the expected place. I guess it's partially to blame that all external keyboards in the house are Spanish and the laptop's is Hungarian... But even switching the keyboard layout in the remote machine to "Spanish ISO" doesn't help anything... I tried to plug in one of the USB keyboards in the laptop, to solve the keyboard mapping mismatch issue, as the OS would then try to map according to that. Didn't work.

So, it's far from being a perfect remote GUI solution, unfortunately. It's okay for casual checking of e-mail (mailbox being on the other machine) but really not much else. Sigh.

First cut at the JVM dynamic languages metaobject protocol

2007-07-26T01:01:00.000+02:00

You might remember I was musing about coming up with a framework library for extensible metaobject protocol to be used for dynamic language runtimes on JVM a while ago. The main thing it would give all those language runtimes is interoperability: you could write your program in mixture of JavaScript, Ruby, Python, etc. - using the best language for a particular subtask (or better yet, cherry-picking existing components regardless of their language), and still pass objects created from code in one language to code in another language, and have them all see those objects as being no different than their native objects.

Well, since talk is cheap compared to working code (and also because if I can express a concept as clearly as to have a computer execute it then I can probably make people understand it better too), I have a first version of the implementation, not yet fully complete, and first and foremost for the purposes of soliciting feedback from the community, available now.

(For the record, I blew several days of my vacation in Palma de Mallorca on getting this code into publishable state and suffered numerous scorns from my wife so far for doing so. Goes with the territory, both ways... ;-) )

The initial announcement was made on the JVM Languages group, go read the details (including where to find the code and documentation) there if you're interested.

Brief report from a Neil Gaiman signing event

2007-07-05T19:05:00.000+02:00

I'm officially crazy in lots of ways. A particular kind of crazy I am is the one I demonstrated yesterday, when I went to a Neil Gaiman signing event in Budapest.

What's crazy about it? Well, probably nothing if you actually live in Budapest. I don't, so for one thing, it's crazy driving 2 hours to Budapest in pouring rain with near zero visibility. Then it's mildly crazy standing in a queue for two hours (part of it in rain outside the bookshop that was the signing venue. No umbrella). The final crazy (although strictly necessary after the first one) is driving home after the event in pouring rain with near zero visibility after dark. Of course, I'm really crazy because I still feel that spending six hours and about forty litres of fuel is a good investment in return for getting seven of my Gaiman books dedicated, and am totally happy about it.

Y'know, in my environment, barely anyone knows who this guy is (now that I think of it, this says something about people surrounding me. I also cannot lend my copies of his books to many folks around here to spread the culture 'cause they're all in English - I don't need no Hungarian translations, most of my friends however can't read the English originals). So I was fairly surprised when I arrived at the bookstore to find a queue extending into the street. The event itself wasn't even too formal - Neil announced it on his blog few days earlier citing as the reason lots of e-mails from local fans (yours truly included) who asked if he could please arrange a signing during his two-week stay in Hungary, seeing how he's otherwise not really often present in this corner of the world.

As for the venue itself, it was the Sárkánytűz ("Dragon Fire"), a bookstore specializing in fantasy books, card games, etc. I really couldn't help feeling that the signing was organized in a wrong location. I look with disdain at most of the contemporary fantasy literature. Most of it is just bad pulp fiction, its authors unable to free themselves from the cliché dominated by elves, trolls, halflings, dragons and knights. Ironically, a genre called "fantasy" really lacks authors that would have any real imagination. Neil Gaiman, on the other hand creates completely original stories, with completely original creatures, and the worlds he paints in his books intersect with our reality. Nothing could be further from the Tolkien-imitating cliché of elves and dragons permeating most of what the bookstore had to offer.

There's apparently a funny phenomenon associated with people queueing in the street in Budapest. Namely, it raises curiosity in strangers. While I was still standing in the part of the queue that extended out in the street, I was approached twice - once by a nicely dressed lady in her fifties, once by a guy resembling a hobo who tries to disguise it. They both inquired about the nature of the event, the wording of their questions implying they were hoping for some sort of a sale. When I told them it was a signing, the guy who looked like a hobo in disguise quickly lost interest. The lady asked who is signing, and when she didn't recognize the name, she asked (seeing how it's a fantasy bookstore) if it is "some sort of fantasy" to which I replied that it definitely isn't. "Then it is sci-fi, right?". This instantly reminded me of "we have both kinds of music" line. I replied patiently that no, it isn't sci-fi either. Then what it is? Well, how do you summarize Neil Gaiman in a single sentence? You don't. I tried to explain to her that she could think of him as maybe a modern day equivalent to Edgar Alan Poe (which I know you might disagree with, as it's both oversimplification and also plain incorrect, but that was the best I could come up with quickly at the spot :-) ) Anyway, she did leave after this...

So, after two hours of queueing I finally got in front of the man himself, got to shake the hand that wrote all those stories and look into the eyes behind them all those same stories were born. Yes, why, I was impressed, even though Neil is really a very friendly and approachable guy (well, approachable through standing in a long queue, but that's barely his fault. I mean, it's really my fault for not reading more obscure authors). Driving up on the highway I had plenty of time to imagine all kinds of witty dialogue I might get into with him, but the cruel reality was that there were still lots of fans waiting for their turn, so it would've been rude to hold them up for inappropriately long time, so we only did a bit of a politeness chat, and I handed over all the seven books I brought for dedication. It'd really be foolish to expect anything more from a signing event, really. I got a drawing of a cracked heart in my copy of Fragile Things - very appropriate. Also, the original printed dedication for Anansi Boys is actually made for easy personalization, which he totally exploited. I can almost see how Neil came up with it after signing his books for years, and felt very smart about it when he wrote it. (If you don't know what I'm talking about, don't worry, just buy a copy of Anansi Boys and read the dedication. Of course, you can also read the rest of the book if you're so inclined, but that won't help you in better understanding what I just said. But it might be worth your time for unrelated reasons, though.)

I also asked whether he has (or at least has a promise to get) a book of Hungarian folklore tales, to which he replied that he actually got one last week and is halfway through it already. If he didn't, I'd have offered to send him one, which was an idea I got while reading The Monarch of the Glen novella from Fragile Things, seeing how it incorporated a motive from Norwegian folklore (namely, a huldra), and Hungarian folk tales definitely have their share of marvels waiting to be incorporated into contemporary literature (I certainly know, I read lots of those to my kids). Anyway, getting Neil to read some of those tales is already been taken care of, hooray. We'll be waiting for the results :-)

A very nice touch at the signing was Neil's daughter Maddy, who was handing out cookies to people standing in the line and later also offering sandwiches at the signing desk. I really needed a bit of refreshments before hitting the road home, so I was really grateful for those as well. I asked Maddy when will she give back her Dad's blog to him, and she said it was her last day as his guest blogger, but she lamented that she can not write the entry because she's here at the signing. I remarked that she could use an exotic technique known as "pen and paper" and later type it up in front of a computer; I have no idea if she took the suggestion, however her goodbye entry was up at 10:01 PM, which I don't actually understand as I can't imagine the signing was over by then and even less that they could reach their hotel room by then - I left at about 9:15, and the queue of waiting fans still extended through the bookstore to the door (nobody waiting on the street anymore though).

Barcelona

2007-07-05T16:22:00.000+02:00

So, Kriszti and me have been to Barcelona last week. Nominally, I went to speak at the TSSJS Europe, realistically I attended the conference just long enough to deliver my talk. That's what you get for taking your wife with you, who insists you spend the time going around the city and seeing places, instead of attending, say, Gregor Hohpe's talk about event driven programming (that one I really would've wanted to hear, but alas, I was stuck sightseeing Sagrada Familia instead; woe is me).

Barcelona must be one of the most joyful places I've ever been. The atmosphere of the whole city is amazing, the buildings, the people, everything is vibrant and alive. Architecturally, they have wide main streets with a very wide pedestrian area (usually with greens, benches, and fountains) in the middle, and lanes for cars surrounding it. Drivers are very mindful of the pedestrians and will wait for them to cross the street even when they (the cars) have the green light. Patiently.

Then there's Sagrada Familia. I don't think I've ever been as impressed by a cathedral as I was by it, especially since this is the first one that I can't view purely as a historical monument, main reason being it's still being built. You step into it, and it's full of construction workers. Cathedrals are always monuments to ages in which they were built, and this one is a monument to our age instead of some long gone one, making it even more easier to feel like it you have a personal connection to it. Y'know, like when you visit a cathedral and they tell you "... the XY cathedral has been built for 300 years..."; well, this one is still in its first 100 or so years :-)

Architecturally, it is also quite amazing, and departs from "traditional" cathedrals in quite a lot of places. Pillar structure is such that pillars branch at the top, giving the illusion of trees, further emphasized by the ceiling that also tries (quite successfully) to look like foliage.

Then there are various other Gaudi projects in the city: Casa_Milà, Parc Güell, and so on, all definitely worth several hours of one's time.

We also went to see an evening flamenco show with a dinner (courtesy Klaasjan) which turned out really great, both the flamenco performers and the dinner were terrific. And on the previous evening, we hung out at the hotel's "bierstube" (which is to say, a spanish attempt at emulating a german beer place). A funny moment was when the staff kicked out Kirk and Cliff because they attempted to come in with two boxes of externally acquired pizza. The attitude of the staff was especially rude considering we were otherwise on a table with few hundreds of euros on the tab anyway, so what does it matter if two pizzas are eaten without generating a profit for the venue? Anyway, the guys ended up eating their pizzas from their boxes on the street. Kriszti and myself joined in for a bite (no photos of this overly casual event, regretfully :-) ).

On saturday, wife and me also had the pleasure of meeting with Jon Revusky and his wife Nuria (they're long time Barcelona residents) for lunch. (We were treated to really copious amounts of really great tapas in Cervezeria Catalana.) Jon and me are collaborating on FreeMarker for about five, six, or even more years now, and this was the first time we actually met personally.

The only dark side of the trip was airline luggage handling. Needless to say, they (Lufthansa + Newco, their ground services provider in Barcelona) lost our luggage between our connecting flights. That in itself isn't that bad, but they didn't deliver it until the day of our flight back! So we went out to the airport on sunday, got our boarding passes (from a really helpful lady at the Lufthansa ticketing desk), then went through security into the transit area, found the baggage room, stood in the line at the Lost+Found office, finally got to the front of the queue, been taken to a back room with hundreds of suitcases, found our own, went out from baggage room, back to departures to drop off the suitcase for the trip home, then again through security into the transit area. Simple, huh?

When we arrived home to Budapest, we went to the Lufthansa office at the airport to file a claim for luggage delay. The ladies working there said they don't deal with it as they don't have sufficient capacity to handle all such claims (I can believe that. They certainly gave the impression they have no capacity for anything whatsoever) and they told us to contact their office in the city, for which they can only give us e-mail address and fax number. The lady at the desk then proceeded to jot down the e-mail address on a piece of paper: fly@lufthansa.de. I respectfully noted that it seems unlikely that this'd be the e-mail address of the city office in Budapest, but she was insistent that it indeed is. I was too tired to argue. I spoke to Kirk on the phone today, and he said Lufthansa office at the Budapest airport is the "most useless place on the planet". I tend to agree.

All in all, the Barcelona trip was great. Got to see a marvelous city, met some new folks and met again some already known ones, eaten lots of good local food, drank some sangria and local wine, did one talk, watched flamenco dancers, seen a great sunset by the sea. What more to expect? (Well, on-time baggage delivery, maybe.)

ROTFL

2007-05-31T23:20:00.000+02:00

OMG!!11 ITZ AWSUM!!

(No, the blog hasn't been hacked. Yes, click the link. See for yourself.)

IM OUTTA YR LOOP

No online Ant API?!

2007-05-18T11:26:00.000+02:00

(Warning: disgruntled rant below.)

Ok, so I'm writing an Ant task. I'd need the Ant API. Google "Ant API". First link is http://ant.apache.org/manual/api/index.html. Yeah, sounds about right; click. To my surprise, instead of the familiar JavaDoc page, I get this:

Apache Ant API has not been generated

If you see this page online at ant.apache.org, it is not a bug, but on purpose. We do not provide an online version of the API docs, they are included with all our distributions.

Now, just how arrogant is that? "On purpose"? Just exactly what purpose is that? Are you guys really this much bandwidth starved?

Just for the record, the JavaDoc is not included in all distributions. Y'know dear Ant team, since the Apache license pretty much takes away the power from you to dictate what's in a distribution, some distributions won't come with JavaDoc.

Like that obscure one that ships with XCode for Mac OS X, installed in /Developer/Java/Ant directory. I guess no one uses that anyway, right? I mean, what Java developer would be as insane to use Mac OS X, let alone the development tools that ship with it, right? I'm probably the only such loser on the planet, I can't imagine anything else can explain why noone has noticed this yet and told you about it, so you removed that silly claim from the page where the API docs should have been, and replaced it with, well, with the API docs. Give me a break.

You could say how that's Apple's fault, and I'd agree with you. But that you aren't helping at all is ridiculous, with as little as to keep an online accessible API docs for one of your highest-profile Java projects. In 2007, what Java open source project aspiring to claims of professionalism would purposefully refrains from hosting its JavaDocs on its site? If for nothing else, then all other factors being equal, people want to use their computers with least effort. With a computer connected to the Internet, the easiest way to get to an API is punch "$projectName API" into the browser's built-in Google search box and then hit the first link. Compare this to trying to find the relevant index.html on one's hard drive. Yeah, we're that spoiled.

Anyway, the first site on Google's search results list that can afford the horrid bandwidth and storage burden of hosting the Ant API is http://www.jajakarta.org/ant/ant-1.6.1/docs/ja/manual/api/help-doc.html. Kudos to them, and shame on whoever is responsible for this lameness in the Apache Ant team.

XStream

2007-05-16T10:09:00.000+02:00

Whenever I'm faced with a programming problem to solve, the logical first thing I do is to rummage around the 'Net, trying to find an open-source library (preferrably non-GPL, i.e CPL or BSD licensed) that already solves it. Because reinventing the wheel is costly, and most of the time the problems are such that I can't imagine to be the first to have came across them.

Few months ago, I had a requirements for passing objects between JVMs. "Doh, serialization", you'd say. Right. Except that the other people on the project felt it'd be very helpful from the diagnostics perspective if we could inspect the objects. So the logical idea - serialize using XML instead of Sun's binary serialization format. Turns out that the JavaBeans API actually provides support for something similar in the java.beans.XMLEncoder and XMLDecoder classes. It will serialize/deserialize JavaBeans using their public property getters and setters. But we really wanted something that operates on the ObjectInputStream/ObjectOutputStream idiom and serializes fields.

Eventually, I found the XStream project at CodeHaus.

The logical second thing I do when I found an open-source library is examine it. Unfortunately, lots of stuff floating out there isn't of particularly high quality, and branding also doesn't guarantee quality, be it "Apache" or "Codehaus" or anything else.

Now, after using XStream for quite a while now, I must say: well done! It delivers on its promises, the code is architecturally sound, and is insanely customizable. The generated XML is rather compact - i.e. if a value of the serialized field is exactly the same type as the declared type of the field, the type is not written out in the XML. Only if a subclass is used as the actual value. What's more, types can be aliased both on serialization and deserialization. This allowed us to ship objects from a GUI frontend JVM to a DB backend JVM, and have them being serialized/deserialized as different subclasses of a common abstract superclass hierarchy. In the GUI frontend, they were deserialized as classes with GUI operations, on the DB backend, they were deserialized as Hibernate enabled classes. Just brilliant.

I know you can do it with Sun's default serialization as well, but you must override resolveClass or replaceObject in the object input stream. Whereas with XStream, you could just configure the aliases in a factory once, and be done with it. Very elegant.

It is also possible to plug in custom serializers for certain types - i.e. we had to make sure Hibernate-specific collection classes were serialized as plain Java collection classes. It probably took me all of a 20 minutes to implement it, relying on documentation and the XStream (quite clear) source code.

Then there's a rather ingenious default mechanism for backreferencing objects. You can configure XStream to slap an "id" attribute on each element representing a serialized object, and when you need to reference it from a later written object, refer it by the ID. Anyone could come up with that, it's rather trivial. But XStream doesn't do that by default. Instead, by default it doesn't write any ID, but if a later written object needs to backreference an earlier written one, it'll use an XPath expression that selects the earlier object's element! (You even have a choice between absolute XPath or XPath relative to the referencing element.) Again, very elegant.

And to top it all, XStream is not for XML only any longer. Since the serializers are pluggable, they currently also have a JSON serializer, which sure can come in handy if you're AJAXing. (I'm not, so can't give an account of experience in this regard.)

All in all, I'm currently very happy with XStream; if you need to serialize/deserialize your Java objects to/from XML or JSON, I recommend you give it a try.

My current open-source TODOs

2007-05-15T21:10:00.000+02:00

I'll admit that I feel a bit thin-spread lately regarding my open source activities. There's just too much stuff going on that I'd need to attend to one way or the other. Of course, this is all competing for whatever leftover bits of my time after paid work ("urgent/important"), family, an attempt to reinstantiate a regular workout habit (both "not urgent/important") and the also inevitable everyday interruptions ("urgent/not important"). Just to make it clear, OSS work is to me in the same "not urgent/important" quadrant where family and workout take place.

In no particular order my current OSS activity goals are:

~~Implement support for invoking Java vararg methods from FreeMarker~~ (this actually goes off the list, as I completed it on the quiet sunday afternoon while kids were at a birthday party)
Implement support for JSP 2.0 SimpleTag interface in FreeMarker's JSP taglib runtime
Fix the JSP taglib lookup mechanism in FreeMarker's JSP taglib runtime
Make the overloaded method invocation + varargs into a reusable library so I can support vararg methods in Rhino and also have other open-source dynalang communities leverage it
Support vararg methods in Rhino
Cleanup the LiveConnect implementation in Rhino (primitive booleans not coerced automatically, and in general overloaded method resolution could be better, and it doesn't use JavaBeans bean info, and...)
Just generally try to attend to bug reports in Rhino
Attempt to refactor Rhino-in-Spring to reuse Spring WebFlow as much as possible, 'cause having two webflow implementations on the same base code would really help it emerge into a flexible foundation worth submitting as a "Java Web Flow" JSR

There you go - I actually feel a bit better having written down all of this. If you wonder why I haven't answered a Rhino question for a week or so, it might be because I was busy working on a FreeMarker issue, or vice versa. Or just there were no birthdays lately, so I didn't have a quiet sunday afternoon at my disposal for some immersive hacking. But rest assured, I'm going forward with all this stuff, just not exactly on "Internet time" speed :-)

Fish Tank

2007-05-11T21:04:00.000+02:00

I'm an avid reader of about a dozen different webcomics. Today, I just discovered another one, and it's just incredible in its originality. It's the Fish Tank, about adventures of three fish living in an aquarium (and quite often venturing outside of it). What's original about it? Well, these fishes occasionally get rid of piranhas (on an occasional visit to Brazil) by deorbiting a communication satelite on top of them using a satelite telephone, or get rid of a cat using explosives. Or are (in the current storyline) kidnapped by, of all things, telepath moths. Rooms, houses, or city blocks sometimes get demolished, or Alaskan wild forests set to fire. In general, there's lots of "Mission impossible" action going on, with fish improvising all sorts of devices from stuff found in their tanks and household items (not necessarily plausibly, but that's really not the point of it; they actually have laptops too!). The drawing is loveably sketchy (but consistent), and the characters are incredibly well built, complete with their love/hate relationships and plenty of ingenious throwaway jokes and inspired dialogue. Totally love it.

Dynamic language interop plans in .Net world too

2007-05-10T21:36:00.000+02:00

I've recently explored the topic of in-process cross-language interoperability (think Python program using Ruby created objects etc.) in my "Adapters or Navigators" article.

Interestingly, I stumbled across posts in Jim Hugunin's (Jython and IronPython founder) blog recently about the exact same topic in the context of .Net - they seem to already have some implementation ready as part of their Silverlight initiative. Here are the posts:

A Dynamic Language Runtime

The new Dynamic Language Runtime (DLR) adds a small set of key features to the CLR to make it dramatically better. It adds to the platform a set of services designed explicitly for the needs of dynamic languages. These include a shared dynamic type system, standard hosting model and support to make it easy to generate fast dynamic code. With these additional features it becomes dramatically easier to build high-quality dynamic language implementations on .NET. More importantly, these features enable all of the dynamic languages which use the DLR to freely share code with other dynamic languages as well as with the existing powerful static languages on the platform such as VB.NET and C#.

Next two posts deal with a sort of "universal adapter", a common object model required for cross-language interop:

One True Object (Part 1)
One True Object (Part 2)

In the part 2, he illustrates how a dynamic language runtime queries an object "Do you have member X?", and the object thinks "I'm a Python object, so I'll look it up by Python rules". He envisions the IDynamicObject interface that all object implementations of all languages must implement for interoperability.

This is exactly what I claim to not necessarily be the best approach in my "Adapters or navigators" article, but that having pluggable metaobject protocols instead of an expected interface might be a more flexible solution.

It's nice to see though that other people also have this problem in their crosshairs :-)

Parents of kids with Down syndrome campaign against prenatal testing for it

2007-05-09T11:35:00.000+02:00

The New York Times carries an article today titled "Prenatal Test Puts Down Syndrome in Hard Focus".

It reports how in US, families rising kids with Down syndrome are campaigning against a new, widely usable prenatal test that can be used to screen for it. 90% percent of pregnancies where the fetus is diagnosed with Down get aborted. Some parents are worried that as population of people affected with Down syndrome dwindles, the care programs for them will also disappear.

But they are also worried about a much more serious problem than that. The practice is well within the area of eugenics, trying to draw a line between eliminating a genetic condition undesired by the would-be parents versus lessening the diversity of the human race. The genetic condition can be viewed as undesired since Down syndrome results in having somewhat weaker physical features, being slightly mentally challenged, and having shorter life expectancy (49 years average), but most of these individuals are still able to live fully enjoyable lives in a loving family, even if they're placing a bigger burden on their parents, and parents who actually raise such kids and therefore have first-hand experience believe how Down is not a sufficient reason to deny those kids existence. I tend to sympathise with this point of view. From the article:

Sarah Itoh, a self-described “almost-eleven-and-a-half,” betrayed no trace of nervousness as she told a roomful of genetic counselors and obstetricians about herself one recent afternoon. She likes to read, she said. Math used to be hard, but it is getting easier. She plays clarinet in her school band. She is a junior girl scout and an aunt, and she likes to organize, so her room is very clean. Last year, she won three medals in the Special Olympics.

“I am so lucky I get to do so many things,” she concluded. “I just want you to know, even though I have Down syndrome, it is O.K.”

In-process cross-language object interaction: adapters or navigators?

2007-05-04T17:14:00.000+02:00

There's a new article on my website titled "In-process cross-language object interaction: adapters or navigators?".

I'm discussing two typical approaches for object interaction between a "higher level runtime" (HLR - i.e. a scripting language runtime) and "lower level runtime" (LLR - runtime for whatever language the high-level runtime is implemented in, i.e. JVM or CLR, or even straight C).

The two approaches discussed are the somewhat more common "adapter" pattern, and the somewhat less common "navigator" pattern.

HLR implementations usually go for the adapter approach, but I argue that the navigator approach is superior to it, especially when you want to extend the object interaction from the HLR-LLR dimension (i.e. representing native Java objects in a JavaScript runtime written in Java) to an orthogonal dimension, namely have interoperability between two distinct HLRs running on the same LLR in a decently intuitive manner, without knowing of one another in advance (i.e. use objects created in JRuby from within Jython, without Jython knowing about JRuby at all).

This level of interoperability opens up new possibilities for people implementing software systems, as they become free to implement different subsystems in languages that fit the job most.

Discussion welcome in comments.

Calling Java vararg methods from dynamic languages

2007-04-09T23:39:00.000+02:00

I'm adding code to support calling variable-argument methods (introduced in Java 5) from FreeMarker Template Language. What's the big deal, you might ask. A lot, I might answer.

As a matter of fact, it'd be a rather straightforward task if it weren't for a minor detail: overloaded methods.

When a method isn't overloaded, and you need to invoke it from a dynamic language such is FreeMarker, all is dandy and Bob is your uncle - you have to adhere to a single target formal argument list, easy as a breeze.

When a method is overloaded?

Ouch.

The thing is, with dynamic languages it's not as simple as finding the method that satisfies the mathematically precisely defined technical criteria known as "the most specific applicable method for given actual argument types". That's what Java compiler has to do, and believe me, Java compiler has it the easy way. With dynamic languages, we start from one step behind. We operate in another language, y'see, and we first must figure out how to marshal the actual arguments to Java types. Of course, the optimal way to marshal them might depend on the actual method we chose to invoke among all overloads. Which in turn depends on the Java types of arguments we use for invocation.

Chicken and egg. Catch 22. Strange loop. I wrote logic in FreeMarker to handle this years before, and it was quite involved even back then, and we didn't have variable arguments back then.

I had quite a few false starts on it, I'll admit that. Lot of code written and then rolled back. Lots of hours spent thinking about the issue. I have something that works now, written mostly while sitting on the porch of my parents' house where I retreated with wife and kids for Easter. But I'm not committing it to SVN repo just yet - need to test it a bit further before I unleash it on the crowd of lab animals, er, early adopters of FreeMarker 2.4.

There are some downright sinister corner cases, i.e. invoking

foo(a, b)

when you have:

public void foo(SomeClass a, SomeOtherClass b);
public void foo(YetAnotherClass a, SomeBozo b...);

Do you prepare the array of arguments for reflective invocation as [a,b], or [a,[b]] if your algorithm can not know at preparation time whether the fixed arg or the vararg method is going to be invoked? (Under the hood, varargs must always be passed as arrays). My decision was to prepare [a,b], and on the fly convert the last argument to one-element array if it happens that the vararg method was chosen (the ambiguity only arises if the vararg method would receive exactly one argument in its variable part, and would not arise if it would receive zero or more than one - you see what sort of arcane corner cases crop up in dynamic setting).

My ultimate hope is that I can create a sufficiently generic implementation for this that I can share it with other projects as well - Rhino and JRuby, for starters. All dynamic languages on JVM could benefit from a soundly implemented library for this functionality, at least until the bright future arrives where the whole world has already transitioned to JVMs that implement the hypothetical invokedynamic bytecode instruction. Although even if we had it, I actually doubt that the "strange loop" coordination required between marshaling arguments from another language to Java and selection of the most specific method would be handled by it - such marshaling is language specific, and while it can be abstracted behind an interface, extending with interfaces is a mechanism used by libraries in Java, not by the JVM itself, so it looks to me this'll remain a job for the library.

Month of travel

2007-03-05T07:43:00.000+01:00

So it starts my month of travel. I'm sitting at the Budapest Airport as I write this (kudos to Pannon for providing free wireless in the airport), passing time until boarding for the London flight. Then flying back home on friday, then a vacation with family from wednesday to sunday next week - lots of driving in Slovenia, but should really be fun. Then flying to Vegas the tuesday after that, then over to New York on saturday when Vegas is over, and finally coming home and putting the suitcase away for hopefully few months on friday, March 29.

They're calling for boarding.

New Horizons probe Jupiter flyby

2007-02-28T21:24:00.000+01:00

The NASA's New Horizons's probe made its closest approach to Jupiter today morning (well, morning in my time zone, that is) on its way to its ultimate destination (Pluto, to be reached in 2015). It sent back some of the best to-date pictures of Jupiter and some of its moons.

The probe also carries a compact disc that stores a file listing the names of some 430000 people who chose to send their name in the space. Were you nerd enough to sign up? Or even to sign up your kids? (By the time the news of probe reaching Pluto hits the media in 2015, my kids will be 16 and 14 years old respectively, and I hope they'll grow to be geeks as their dad is, thus appreciate this).

Snowflake magnified 36000 times

2007-02-28T21:05:00.000+01:00

Yes, I'm sort-of obsessed with snowflakes - this is not the first time I blogged about them. Here's a fascinating series of images of a single snowflake, imaged with an electron microscope, ranging in magnification factors from 93x to 36000x.

I often try to explain to my son just how small scale is the molecular structure of the material world that surrounds us, but it's hard to word it so that a 8-year old understands it. As they say, a picture is worth a thousand words, and these breathtaking pictures certainly prove the point.

You still can't write a good novel

2007-02-23T16:29:00.000+01:00

My brother-in-law was given the task of writing some integrated-circuitry emulator in C# on his internship job. He's a soon-to-graduate electrical engineer and rather an alien to software development. He was given a book, "Visual C# Step by Step" to supposedly help him along and complained to me how it doesn't really help him in figuring out how to actually write a good program, it just teaches him a programming language.

I told him that if he attended a course in French language, he still wouldn't learn how to write a good novel in French.

He understood the analogy. The problem is - at least if I remember my university years - an awful lot of educators around here don't get it.

Speaking in Vegas

2007-02-23T15:41:00.000+01:00

I'll be attending the TheServerSide Java Symposium (TSSJS) in Las Vegas between March 21 and 23. On friday, I'll be speaking about the Java 6 Scripting API. I'll be delivering the same talk on the European TSSJS event organized at the end of the June in Barcelona as well.

"Iterators are not intended to be serializable." Huh?

2007-02-05T09:09:00.000+01:00

In a recent attempt to reduce my "Unread" mailbox, I went back in it marking as "Read" everything that has been, well, read. So it happened that I stumbled across a bug report I filed with Sun around 7 months ago (yes, that mailbox was deep), and went to see what became of it. You can see it here.
The problem I reported is that various iterator objects shipped with Sun's JRE aren't serializable. Looking at the bug's page, I noticed that the "Evaluation" field has been filled out with:

Iterators are not intended to be serializable. Probably.... Not a Defect.

Wow. Whoever evaluated this wasn't particularly, to put it mildly, in short supply of short-sightedness. In accordance to this, the bug was marked as "Request for Enhancement".
Now please tell me if I see it wrong, and if there never-ever can arise a valid class design where a stateful object maintains certain aspect of its state by keeping an iterator into a collection, and said object should be serializable. At least for lists, the concept can even be emulated by holding a reference to the list object and an index variable, so it definitely doesn't sound like a very unsane idea.
I will admit though that I didn't come across an explicit case of it myself. How did I stumble upon this bug then? Continuations. I had a JavaScript program that had a loop going through a collection with an iterator, and captured a continuation within the loop and subsequently attempted to serialize the continuation. Naturally, the continuation being within a loop, the serialization graph of objects reachable through it contained the iterator. Blam.
Now, this admittedly might sound as a more esoteric case of serializing state that contains an iterator, but really, were I to use a "for" loop with explicit indexing instead of an iterator, it wouldn't have been a problem. Therefore, I can't see serializing an iterator to be such an off-the-wall idea to justify the anonymous Sun mouthpiece saying "Iterators are not intended to be serializable"?

Jobs: "Nobody uses Java anymore"

2007-01-30T08:00:00.000+01:00

I've been going through few weeks worth of the Apple's java-dev mailing list posts, and noticed one that referenced a David Pogue column in NYT from two and a half weeks ago that itself quotes an interview with Steve Jobs centered on iPhone. When Jobs is asked about Java support on iPhone, he says:

"Java’s not worth building in. Nobody uses Java anymore. It’s this big heavyweight ball and chain."

This is worrisome. Jobs didn't explicitly reduce the scope of his statement to iPhone and/or mobile phones in general, although since the interview itself was about iPhone, one can give him the benefit of the doubt and assume the "Nobody uses Java anymore" doesn't apply to the computing world outside the mobile devices and won't affect availability of JDK on Mac OS X in the future. (Which, by the way I do find a bit lacking - JDK/JRE on Mac OS X is still stuck at 1.4.2_09 and 1.5.0_06, while the Linux/Solaris/Windows crowd has long since moved on to much higher service releases.)

Actually, as far as mobile devices are considered, I can agree with Jobs that on them Java is indeed "heavyweight ball and chain". John Carmack writes on his blog about difficulties writing DoomRPG and Orcs&Elves for mobile devices using Java here and here, preferring to use BREW as its alternate development platform, with most memorable sentence being:

... it isn't that BREW is inherently brilliant, it just avoids the deep sucking nature of java for resource constrained platforms...

and:

The biggest problem is that Java is really slow. On a pure cpu / memory / display / communications level, most modern cell phones should be considerably better gaming platforms than a Game Boy Advanced. With Java, on most phones you are left with about the CPU power of an original 4.77 mhz IBM PC, and lousy control over everything.

So yes, Java on mobile devices is a frustration. I still wished Jobs didn't make an overly generic statement...

Intel announces 45nm transistor technology

2007-01-27T16:56:00.000+01:00

The New York Times reports that Intel has announced new technology for building microprocessors that replaces the "silicon gate electrode on a silicon dioxide insulator" design. The new technology uses a metal gate electrode on a hafnium insulator, eliminating much of the current leak that was the major obstacle in going below the 65nm technology. The new technology promises faster, more energy efficient processors, keeping up the Moore's Law. Intel plans to begin making microprocessors with the new technology as soon as the second half of this year.

To quote the NYT article:

Company researchers said the advance represented the most significant change in the materials used to manufacture silicon chips since Intel pioneered the modern integrated-circuit transistor more than four decades ago.

Also:

“This is evolutionary as opposed to revolutionary, but it will generate a big sigh of relief,” said Vivek Subramanian, associate professor of electrical engineering and computer sciences at the University of California, Berkeley.

I wonder whether this'll break the industrial trend of introducing ever more cores in a single CPU chip. My bet is that it won't, but it might allow software developers to get away with writing concurrency-unaware applications for a bit longer (not that I welcome that) if Intel can now indeed again make advances in CPU speeds.

"Clean coal" technologies being researched, says EU Energy Commisioner

2007-01-26T12:45:00.000+01:00

Andris Piebalgs, EU's Energy Commisioner said in a speech at the eBio General Assembly yesterday:

From the point of view of security of supply, coal is a wonderful source of energy. Europe has plenty of coal; outside Europe, reserves are spread widely. But the greenhouse gas performance of coal is worse than that of any other fuel.
"Clean coal" technologies are being developed to solve this problem. We will accelerate research into these technologies, and take other steps to create a market for them. But there is no chance of their making a significant contribution by 2020.

That's interesting. EU is under a rather heavy pressure to lessen its energy dependency on oil and gas from Russia, and living in a country that got some collateral damage from the last muscle flexing of russian energy monopoly I can completely understand that. But going back to coal burning, hoping that its ecologically disastrous side effects can be eliminated by some not-yet-existing, but presumably researched "clean coal" technologies is a bit worrisome. I mean, as far as I know, you just can't burn coil without getting CO2, since "burning" by definition is "oxydating". At best, those "clean coal" technologies might try to somehow further process that CO2, instead of emitting it into the environment. I'm admittedly a bit skeptical about it, but I definitely welcome any sufficiently rigorous scientific research into the issue.

Java puzzler: field hide-and-seek

2007-01-26T11:53:00.001+01:00

Someone recently asked me to explain the rules for field access in Java, and it sparked the little test program you see below. Can you guess the number sequence it prints?

class Class1 {
   String data;
    
    Class1(String data) {
        this.data = data;
    }

   String getData() {
        return this.data;
    }
}

class Class2 extends Class1 {
    String data;

    Class2(String data, String superData) {
        super(superData);
        this.data = data;
    }
}

public class Test {
    public static void main(String[] args) {
        Class1 obj1 = new Class1("1");
        Class2 obj2 = new Class2("2", "3");

        System.out.println(obj1.data);
        System.out.println(obj2.data);
        System.out.println((false ? obj1 : obj2).data);

        System.out.println(obj1.getData());
        System.out.println(obj2.getData());
        System.out.println((false ? obj1 : obj2).getData());
    }
}

It goes to show three interesting things (which are perfectly logical, we just don't think about them often):

compile-time type of the ?: expression is the most specific common supertype of the compile-time types of its two target subexpressions,
selection of target type for field access happens at compile time (no "virtual fields" in Java),
compile-time type of "this" expression is the class that declares the method,

Remember folks, avoid both field hiding and direct access to fields in the first place, so your code doesn't become a puzzler :-)

What did D'Anna see?

2007-01-26T11:52:00.001+01:00

(Listening to the BSG Season 2 soundtrack CD as I write this, of course. Package from Amazon arrived yesterday, woot).

Might as well dedicate another post to BSG... I'll admit it feels a bit strange to discuss a plot of a television series here (actually, it feels strange for me to be this enthusiastic about any TV show) but the thing is that this particular damn show actually provokes thoughts, touching with very good sense upon questions ranging from politics to religion to ability of nonhuman beings (I'm avoiding the term "artificial intelligence") to have what unsettingly resembles human minds - sentient, free willed, emotional, even religious.

Anyway, if you didn't see the last BSG episode, "Rapture", you might as well stop reading now, because it won't make any sense to you then.

In the last episode, D'Anna supposedly saw the faces of the last five cylon models in the temple of the Jupiter during the supernova breakout before ~~she died~~ her current body terminally malfunctioned. But did she really see them?

I have a suspicion that as much as it seems so on the surface of it, she actually didn't. If you remember, the Leoben instance that Adama encountered at the weapon storage station in the series pilot mentioned that his nervous system is negatively affected by the electromagnetic activity in the cloud that surrounds the station. I can only imagine how can an ongoing supernova breakout affect the nervous system of a humanoid cylon exposed to it standing on the surface of a nearby planet. So, yes, it might as well be that D'Anna hallucinated as a mental malfunction (and later physical too) caused by the supernova burst and just saw what her subconscious (assuming they have an analogous layer in their psyche) wanted her to see.

Further supporting this claim is that the other cylons don't think much about D'Anna's messiastic fixation and pragmatically only see it as a danger to their common goals. Also, thinking outside of the story, I see that the series writers don't stack unsolved mystery upon unsolved mystery as a means of driving the plot forward (fortunately, unlike Lost's writers, who pretty much seem to be lost in the plot, and definitely lost me as a viewer - pun fully intended), and in accordance to this, nothing else even hints that her mystical beliefs could be true. Well, except that the location where she saw the last five in her vision was the same location Baltar saw in his vision on Kobol at the end of season one. This could be explained if the location is something buried in minds of all cylons, and either Baltar is a cylon himself or the Six in his head is more than just a hallucination that suppresses his sense of guilt. Neither of these assumptions are provable based on currently available information though, but I believe latter is vastly much more probable than the former.

Oh, and there's some wonderful yin-yang contrast in how it's Cavil, the only atheist model among cylons who informs the religious, messisatically predisposed D'Anna that the rest of the community has sentenced her to being boxed in cold storage. I also don't have a moment's doubt that in the future of the series, Baltar will somehow get in a situation where he can restore her from cold storage into a body, and won't hesitate for a moment, as he believes she has the answer to his burning identity-crisis question.

Microsoft to the rescue of the mankind

2007-01-23T09:55:00.000+01:00

(click on the link to see the original flash animated ad). It's not a fake - this ad was indeed published by Microsoft and for a while appeared on Battlestar Galactica's official homepage on Sci-Fi channel.

Related fun fact: Did you know no TV channel carries Battlestar Galactica in Hungary? One of the commercial chanells bought the rights for the first season few years ago, but they stopped broadcasting it after only (I think) six episodes, citing poor viewer ratings. Morons. (Someone was a moron. Either the TV channel producers, or the company measuring the viewership ratings, or, err..., the local TV audience.) If it weren't for BitTorrent, I wouldn't be able to watch it at all. (And I also wouldn't have any motivation to buy the mini series + the first two seasons on DVD + all the soundtrack CDs from Amazon either, which I did recently. So much for studios' fairytale of "lost revenue due to copyright infringement by file-sharing".)

Graz for geeks

2007-01-06T14:20:00.000+01:00

Wife and me spent the few days around New Year in Graz, Austria. I've taken 245 pictures in total, and now published 13 of them on my Flickr page. I published only those though that have some geek appeal. Among the stuff is Graz's new palace of arts, a building resembling an alien spaceship complete with a "Resistance is futile" neon sign and pulsating lights, a snowman sculpture similar to Calvin's snowman art, half a millenium old clockwork, sculpture carrying the first Austrian cosmonaut's message encoded in waltz steps (no kidding) and more.

Java garbage collection analyzer

2007-01-06T13:36:00.000+01:00

IBM Pattern Modeling and Analysis Tool for Java Garbage Collector is "a tool that parses verbose GC trace, analyzes Java heap usage, and recommends key configurations based on pattern modeling of Java heap usage".

I remember I heard about a similar tool for HP JVM on HP-UX (I think Kirk Pepperdine told me about it on the train from Aarhus to Copenhagen this October when we were travelling home from JAOO) - it'd collect GC statistics and then provide you with a GUI where you could play with sliders for various memory settings, and it'd calculate how would have the GC behaved if the program was run with those settings.

On a somewhat related note, as someone who had to do some ammount of GC tuning in the past, I'll be curious about experiences people have with Java 6's GC that is said to behave optimally in most situations without any manual tuning... It sounds enthusiastic, but I'm too often reminded that in reality there is no silver bullet in automatic memory management.

No H.264 playback on Sony Ericsson W850?

2006-12-28T12:31:00.000+01:00

My new gadget, the Sony Ericsson W850, is able to play back video (in addition to being able to record video, take photos, play music, and only incidentally, to make phone calls). It has a 320x240 screen (same as gen-5 iPod), and according to its freely available tech specs, supports "MPEG-4 Simple Visual Profile Level 0, H.263, H.264 Baseline Profile Level 1". My experience though is that it can't play back H.264, at least it didn't play back anything I could produce. I tried both encoding a few minutes clip from a DVD using Handbrake, as well as converting another few minutes DivX clip using iSquint, and the results are the same with both: I tried every combination of settings in both programs, and the combinations where vanilla MPEG-4 is used result in a video file that plays okay on the phone, and the combinations where H.264 is used result in a video file that results in a black screen on the phone (although the audio plays okay).

I have to stress that I'm not saying the device doesn't play H.264. I'm just saying that several video files converted by the above two programs with H.264 option selected didn't play. I could've run into some profile capability violation or something similar - I'm not that much knowledgeable in video encoding. Some people say I should try experimenting with command line ffmpeg. Honestly, I won't. It's not that important, really. While I can find time to listen to music on the go, I really can't imagine I'll want to watch any video content squinting at the 2" screen. The ammount of time I decided to invest in this went only so far as to try comfortable GUI options, with motivation being "because it was there." If anyone has more success than me though, please let me know.

Another kludge in Java - varargs

2006-12-27T22:36:00.000+01:00

Varargs in Java. Seriously, who needed this?

You're probably aware of my position on Java Generics, how I view their implementation to be a horrific kludge, a compromise made in name of backward compatibility where the backward compatibility issues are in fact nonexistent.

Now, here's a new discovery - vararg methods. Someone asked on the Rhino list whether we can support them, and I answered that as long as he passes the vararg argument explicitly as an array, then yes. I decided to investigate the problem further within a codebase I'm truly familiar with - FreeMarker's BeansWrapper. I quite quickly came up with a scheme for supporting non-overloaded vararg methods. However, for overloaded ones, the exercise becomes, shall we say, a bit more involved. As I had to discover, the overloaded method resolution algorithm from Java Language Specification (JLS) 2nd edition was seriously overwritten into a three-stage algorithm in JLS 3rd edition, of which the "old" (mathematically quite elegant) algorithm from JLS 2 became the second stage. I probably need some additional time to grasp it, but it looks quite horrible now. If you wish, compare:

"Method Invocation Expressions" section in JLS 2

with

"Method Invocation Expressions" section in JLS 3.

Of course, the chapters regarding type conversions (required, as overloaded method resolution relies on the definition of "method invocation conversions", a subset of conversions that is allowed between actual and declared parameter types when invoking a method) are also substantially rewritten to cope with boxing, unboxing, and generic types.

And the worst part? Varargs are yet another Java the Language feature implemented without a Java the Virtual Machine support. They're purely syntactic sugar, with vararg argument being an array on bytecode level. Several annoying things:

reflection is unaware of it. You can't just pass more arguments to the method, you have to pass the vararags arguments in an explicitly allocated array.
No automatic object-to-primitive conversion when using reflection. If the vararg method's type is "int...", you must pass int[], not an Integer[]. Compare this with automatic
conversion for regular argument types.
Unintuitive behaviour. Quick, should the following example print 1 or 0:

public class Test {
  static void x(Object... i) {
    System.out.println(i.length);
  }
  public static void main(String[] args) {
    x(new Object[0]);
  }
}

Transparent Desktop Trick

2006-12-27T18:28:00.000+01:00

Take a photo of what's behind your computer screen, and set it as your desktop wallpaper. With little care, the results are amazing. See it here.

Labels

2006-12-27T18:05:00.000+01:00

So, Blogger got out of beta, and finally it added a labelling feature, which is something I wished for for quite a long time. I decided to label my post to sort them into categories. The categories are (in decreasing order of number of posts at the moment):

software for anything related to software development in general. Java, Open Source, concurrency, programming languages theory. Stuff you'd want to read if you're a fellow software developer.
side interests for serious stuff that I'm interested in as an amateur. Quantum mechanics, astronomy, math, society, politics etc.
mac is for stuff you don't care to read if you aren't interested in Apple computers.
personal is for writings of personal nature.
fun is for art/entertainment/humour things I came across and want to share them with you.
blog a very low volume, blog-related announcements (like this post).

I tried to keep the number of categories to a reasonable minimum, trying to partition the content into as few of them as possible, taking into account possible reasons people read the blog (i.e. it's different for those who track what I do in the software industry, and again different for friends and family).

The links above have associated RSS feeds, so you can choose to syndicate in your reader only those categories you care about. This will probably cause less frustration for me in the future as well, as sometimes I was reluctant to post a more personal writing because I didn't want to decrease the signal-to-noise ratio for (I presume) majority of people who only care for my software-related posts. Heck, I might even post some personal writings in Hungarian now :-)

MacHeist

2006-12-15T11:35:00.000+01:00

Via Lifehacker:

MacHeist Bundle is a bundle of 10^* highly popular Mac shareware applications. This week only (which means for the next 2 days and 18 hours now), you can purchase them in a bundle for $49 instead of their combined list price of 308$.

Additionaly, 25% of your purchase goes to charity.

What are you waiting for?

^*To be completely honest, it's 9 apps at the moment. 10th app is TextMate, and it'll get unlocked (meaning a license code will be sent to everyone who purchased the bundle) if the total raised for charity reaches $100k. It's currently at $67k, so there's realistic chance to get TextMate too as part of the bundle.

AAC is better than MP3, Walkman instead of iPod, and what Think Different means to me

2006-12-12T23:19:00.000+01:00

Fun fact I learned today: AAC encoding sounds better than MP3 at the same bitrate. Seen here. Re-ripping my CDs right now.

I also bought myself a new cellphone, a Sony Ericsson W850i, and despite some concerns I had initially, the gadget is just brilliant. It has a very decent digital music player built-in (Sony markets them under its Walkman brand) and comes with a 1GB memory stick (I can plug in a bigger one, up to 4GB, and am actively considering buying one), and I can just throw music files from my iTunes library to it using Mac OS X built-in Bluetooth File Exchange app, and it'll nicely index them and add them to its own library. Plays both MP3 and AAC, and recognizes Apple's proprietary iTunes metadata. Just sweet. Guess I'll defer buying myself an iPod for now. Kriszti and me are going on a longish trip to Graz for New Year, so we'll see how it holds up against prolonged use, battery life and overall experience-wise. The phone's contact list and calendar are also syncable with Mac's Address Book and iCal using iSync (after buying a plugin online for 1.43 GBP), a critical feature for me.

As a matter of fact, I initially bought a Samsung Z560, since it's the only phone on the continent at the moment offering HSDPA (think GPRS, only at 1.5MB/s!), but had to return it to T-Mobile after an hour of use and a short frantic search on discussions.apple.com after I realized that it can't sync with Mac, and is not likely it ever will. (Why? Because apparently Samsung has several divisions producing cellphones,
and each of them uses a different sync protocol, and all of them are proprietary...)

And this brings me to the point of that I'm slowly learning what the Apple slogan "Think Different" might mean to me. It means "shake off your Windows-user assumption that any periferial device will work with your computer". I bought a laser printer earlier this year that only works with Mac after installing some cross-compiled Linux drivers. I bought a cellphone that doesn't sync with Mac (fortunately could exchange it for another). I've learned the lesson. I'm shopping for a joystick now, but am not even looking at anything that doesn't feature "Mac compatible" on the box. There's simply no such thing as "maybe it'll work". It's either "it is supported and then Just Works(tm)", or it is "don't even try". In very rare circumstances you can find stuff that is not supported, but works with more or less effort, but it is really a question of dumb luck. Like Sony Ericsson W850i (with a 3rd party plugin - although to be perfectly honest, Apple supports all Sony Ericsson phones, W850i is just too new, and next release of iSync will probably support it out of box), or my Xerox laser printer (works if you dare install a cross-compiled inux CUPS driver). Think Different = Don't Assume It'll Work. Get Used To Doing More Research Than When You Used Windows.

Dungeon Master miscellanea

2006-12-06T17:55:00.000+01:00

My 16-17 year old self has spent more time playing Chaos Strikes Back on Atari ST than my 32-year old self dares to admit.

Turns out, there's a whole fan subculture for Dungeon Master and Chaos Strikes Back out there. There's The Dungeon Master & Chaos Strikes Back Encyclopaedia with information about games as well as a download section with several reincarnations of the games.

Most notable among them is one by one Paul R. Stevens who created an exact replication of the game. Here's how he did it, in his own words found here:

Finally, after many years, I got my hands on the binary executable for the game. I wrote a disassembler to turn it into human-readable op-codes and proceeded to translate it to C++ using the Microsoft Version 6.0 C++ compiler. Eight hours a day for six months. About 120,000 lines of pseudo-assembly language. Crazy thing to do. But it works.

Just insane. Clearly, one of the highest examples of computer nerdiness I've ever came across. Hat off. The biggest achievement is that the resulting code is fairly cross-platform; there are distributions for Windows, Linux, and Mac OS X. And I can certify, the gameplay feeling is identical to that you get from playing it on an original Atari ST. Well, except it loads much faster. And the pixels look blockier on these damned modern high-res displays than they did on a TV screen back in the day. And the Mac OS X port for some reason doesn't deal well with keymaps, so you lose the ability to strafe from the keyboard, which can make the game tedious at times. The Windows version fares better in this regard, and you can even assign mouse clicks on a particular screen coordinate to keys on the keyboard. As anyone who played these games in original will attest, this is an invaluable feature to have as it is much faster to hit 6,4,4,spacebar than to click on four icons on the screen to cast a fireball :-)

Then there's Dungeon Master Java by Alan Berfield, an also cross-platform version of Dungeon Master engine written in Java, with custom raytraced art, giving the game a bit more polished feel.

And then if you look a bit more around on the net with a search engine, you can also find an incredible ammount of custom-designed dungeons fans are creating.

Jeez, now if only I again had as much time as I had 16 years ago...

Poor Man's Message Prioritization

2006-12-02T14:15:00.000+01:00

Imagine that you have a distributed enterprise system held together with messaging middleware (JMS, to be specific). Imagine that it's processing a steady load of work in bulk, but not too often there is some time sensitive operation and your system needs to expedite a message or two to meet its time constraints.

If you're lucky, your chosen JMS provider supports message prioritization. Turns out that unfortunately many do not - they simply ignore the priority parameter in MessageProducer.send(). The one I happen to be using also does not support prioritization. Switching to a different provider turned out not to be an option unfortunately - after spending a nontrivial ammount of time (about a month) testing few other, both open source and commercial offerings, none had both the performance and high-availability of the one we originally used.

What I did to overcome the lack of prioritization was "inventing" (quote marks as quite possibly other people had the same idea already) what I'm referring to as "poor man's message prioritization" - instead of using a queue FOO, I now use two queues FOO and FOO_H (for "H"igh priority) and split the JMS sessions used to process the FOO queue into two separate sets of dedicated processors for FOO and FOO_H, typically with only few sessions servicing the "_H" queue, i.e. a 95%/5% split.

What's important to realize is that what's implemented here ain't a message prioritization. It's just separation of traffic. Logically, the queues FOO and FOO_H are treated from the system's point of view as a single queue, and I refer to the individual "physical" queues as "lanes". Sometimes I refer to this technique as "laned JMS".

It does however create the illusion of prioritization as long as the ratio of prioritized traffic to nonprioritized traffic is lower than the ratio of threads processing prioritized traffic to threads processing nonprioritized traffic. It's a workaround for an annoying lack of a feature in the JMS provider. I won't pretend I'm happy with this - it works, and the implementation I came up turned out to be quite clean, but it's still more work to do for me and more complexity in my code; it should've been taken off my back by the JMS provider vendor.

As for technical implementation, it might sound tedious to implement this across the whole codebase, but is actually not that awful:

On the message producing side, it's sufficient to write a wrapper implementation of JMS Connection interface that is mostly pass-through, except it creates wrapper Sessions, that are themselves mostly pass-through, except they create wrapper MessageProducers, that are again mostly pass-through, but implement the send method with a priority parameter so that if priority is less than 6, the message goes to the base queue, otherwise to the "_H" queue. This way, client code that sends messages just keeps using the Connection.createSession(), Session.createProducer(), and MessageProducer.send() API with priority parameters and is just blissfully unaware of laning in the background.

On the message consuming side, it'd be tricky to implement this across the board, but luckily all my JMS based message consumer services build on a common codebase that I modelled after the Servlet container/Servlet API model - basically, I have a "JmsServiceContainer" (analogous to Servlet container) that deals with session management and other JMS plumbing, and a "JmsService" interface (analogous to the Servlet interface) whose implementations are more-or-less factories for session-specific MessageListeners that do the actual work on a single message. So, I only had to write the enabling code for laned processing in the "JmsServiceContainer" - basically assign the same JmsService to both lanes of a queue. This way, the "JmsService" implementations also remain blissfully unaware - they're just creating message listeners, and don't care that the messages for those listeners now come from two different queues.

Business trip DTD

2006-12-02T13:57:00.000+01:00


<!ELEMENT business-trip (
    airport,
    (office,drink?,dinner,drink+,shower,sleep,breakfast?)+,
    airport
)>

There were efforts to somehow fit a new element, sightseeing into a revised specification, and specify it in the business-trip's content with an optional quantifier, however usability research has shown that real-world documents would almost never contain an instance of it, mainly due to external forces that work to keep the document size at minimum.

Broken LEGO (myth)

2006-11-29T17:51:00.000+01:00

My wife found a broken LEGO brick in our son's room this morning. My kids apparently managed to break a LEGO brick in half, and with it the associated myth (well, at least a belief I held) that the damned things are in fact unbreakable in a domestic environment.

Absolutely coincidentally, several hours later today, Slashdot's cover page is graced with an entry "How They Make LEGO Bricks" that actually links to a BusinesWeek article "The making of a... LEGO".

(Very Useless Fact of the Day: I keep my laptop (when it sits on my desk connected to an external display, keyboard, and mouse) sitting on 4 LEGO Duplo bricks I stole from kids to provide better airflow to it.)

Rhino relicensed under MPL/GPL

2006-11-14T17:46:00.000+01:00

Mozilla Foundation decided to relicense the Rhino JavaScript interpreter to use the MPL/GPL dual-licensing instead of NPL/GPL dual licensing. The reason is primarily that Apache Software Foundation published a 3rd-party licensing policy recently, and NPL was listed as "Excluded Licenses" in it, therefore prohibiting several Apache projects (Coccon, Batik) to redistribute Rhino in the future. MPL, on the other hand falls into Apache's "Binary Licenses Only" group, which'll allow those projects to bundle Rhino binaries with their distributions. Also, Mozilla's lawyers concluded that Mozilla has the legal power to change the licensing from NPL/GPL to MPL/GPL easily, so it looks like the best solution for the time being.

I plan to release the latest stable branch as Rhino 1.6R5 soon. The binaries will be completely identical to 1.6R4, the only difference being that they'll be recompiled from relicensed source, thus the line number tables in classfiles will reflect the difference in length of the boilerplate license code on top of each source file. Apache folks can then include Rhino 1.6R5 instead of Rhino 1.6R4 binaries in their project distributions, or if they're impatient they can compile it themselves from the "Rhino1_6R3_PATCH" branch in the CVS. (Of course, CVS HEAD is now MPL as well.)

1 << 5

2006-11-14T09:23:00.000+01:00

Today's the day when my life duration has doubled for the fifth time, relative to the last such event, (counting my first birthday as the 0th). (With bit of a luck, I'm in for yet another such event down the road.)

For those less math nerd folks out there: I turned 32 today.

I don't usually care too much about arbitrary milestones in the flow of time, my own birthday not really being an exception, yet last night I couldn't help thinking about it (not being able to fall asleep for about two hours after going to bed - I turned from good sleeper to quite erratic sleeper lately). When trying to assess what happened since the last birthday, it always seems life only brings gradual changes (the previous year carries a rather big exception, hopefully I'll be able to write about it sometime soon). However, on such a round number occasion (both %100000 and 0x20), it made me wonder about everything that happened or changed since I was half this age.

And I have to say, it's a hell of a lots of things. My 16-year old self was a quite blissfully unaware gymnasium student in Croatia whose biggest problem was how to skip history classes to go hack on the Apple-II machines in the school computer lab, as well as to write games and fractal generators for Commodore 64 or later for Atari ST. And girls. Were a problem too, that is.

Shortly thereafter, I endured one war and one exodus that completely uprooted me and my family, had to settle in a new country. A bit later found true love, then attended university, and over the passing years grew into several different roles, including that of a husband (for more than eight years now), of a father (for more than seven years now when my son Ákos was born, redoubled the role two years later with Zsuzsi), and of a (well, what at least feels like) reasonably respected IT industry professional. I'd really like to elaborate a bit on all of this here and now, but there are two big driving forces against it: (a) not wanting to bore you to death, and (b) the aforementioned three roles unfortunately don't leave me with much time at present to write sentimental weblog entries about myself.

All in all, the previous 16 years were probably the most dynamic period of my life, past and future. Regardless, I'm looking forward to the next 32 years; stay tuned for the "1 << 6"