Microsoft Open Source

I was surprised to discover today that Microsoft has a hosting site for Open Source projects (analogous to, say, SourceForge.net or GNU Savannah): witness Codeplex. To my further surprise, there's also an Open Source community site at Microsoft, named Port 25. I haven't got time to investigate either of them more deeply yet, but plan to do so in future. This is intriguing.

(Found them both following Microsoft's Open Source Interoperability Initiative FAQ.)

Mind you, I was aware earlier that Microsoft has software released under OSI-approved licenses; there are more than few Microsoft-backed projects hosted on SourceForge.net (or did they move to Codeplex since? Hm...). What I was not aware of, and what I believe is a big deal is that Microsoft is now providing its own hosted infrastructure for Open Source project hosting and community discussion.

Reap What You Sow

You won't catch me writing about politics too often, but I need to express my view on the declaration of independence in Kosovo. Actually, I'll mostly bother you with some of my family history, but the two are, for better or worse, somewhat intertwined.

As you might or might not know, I grew up on the territory of former Yugoslavia, in a small village in northeast Croatia, bordering on Serbia. My family lived in Croatia, but we had relatives in Serbia as well. My family's roots are from the Serbian Vojvodina province, which belonged to Hungary under name Vajdaság until it was annexed into the Serbian-Croatian-Slovenian Kingdom (forerunner of Yugoslavia) as part of the breakup operation of the Austro-Hungarian empire after World War I. (In a way, Serbia then gained a province in north not unlike to how these days it lost one in south.) As such, Vajdaság has a high (alas, dwindling) Hungarian population, and I come from this ethnicity.

I never felt any drawback growing up a non-croatian in Croatia. Nobody in Croatia ever as much as made a remark about me being ethnic minority. Not so in Serbia. Whenever I visited my grandmother in Novi Sad (Serbia) during summer vacation, I experienced strange things. She'd hush me to not speak Hungarian on the street or on the bus. She did neither. The name plate on her door had her name spelt in Serbian (serbian "Jelisaveta" instead of hungarian "Erzsébet" for "Elizabeth"). It was clear you can get into trouble for being different. My whole experience of Serbia was - as far as I can remember - that people there are highly xenophobic and intolerant of their ethnic minorities.

Then came 1991 and the breakup of Yugoslavia. The Croatian region I lived in was overrun by serbian paramilitary troops with full backing by Milosevic's serbian state army. They ruthlessly drove away or slaughtered nonserbian population from the territories they occupied. My family fled with one car trunk worth of belongings when these thugs were approaching. We lived next to an improvised Croatian police station, and we later learned we were targetted as "Croatian collaborators" by paramilitaries because we were on cordial terms with the police officers. They broke into our home on a night after they occupied the region. I have no doubts as to our fate if they found us there.

Mind you, at the time police officers with handguns were the only armed force the just-born Croatian Republic could stack against the Serbian-controlled "People's Army of Yugoslavia", the biggest and most heavily armed force in Balkans in 1991. They had all the chance of a snowflake in hell to defend our homes against the occupators.

While Serbia was significant territorial influence in the breakup of Yugoslavia, it clamped down even harder on its own ethnic minorities, trying to prevent further loss of grip on its remaining territories with the oppression in both Vajdaság and Kosovo growing year after year under Milosevic regime. It culminated when Serbia attempted to eradicate the Albanian minority (minority when viewed against overall population of Serbia, but a 95% majority in Kosovo) in 1999 using its military. This led to the well known NATO intervention when Serbia was bombed by US and its allies until its warlords lost the backing of the population and were overthrown in a revolution.

But the damage has been done. The Serbian state consistently over several decades mistreated and oppressed its ethnic minorities. After what they experienced under Serbian regime for decades, ethnic Albanians of Kosovo wouldn't trust'em as far as they can throw'em. The Serbian state is reaping what they sow now.

It's ironic, but I do actually believe that the recently elected Serbian government might actually be a modern european democratic government that would treat its minorities as a modern european democracy should. (Provided they don't assassinate their prime minister again for being too European...)

But it's simply too late.

There was a huge demonstration this evening in Belgrade. There were atrocities. Embassies and banks were burned. The prime minister spoke to the crowd, fueling it, and the police didn't stop the hooligans. It's sad how they still lay the blame everyone for the situation except themselves, and their decades of hostile politics. I have no illusions this will change soon. I have no doubts that the long oppressed Albanian people of Kosovo are better off in an independent state. They finally will have the chance to bring prosperity to the long neglected region. The region will finally have a government that feels it belongs to the land. As far as I remember, Kosovo was always extremely poor. Serbians have strong emotional ties to the region because Kosovo is the historical site of birth of Serbian state and church, but aside from that, Serbia was a very lousy custodian of the region, not bothering developing it, or helping it develop, or even just not actively hindering any economic progress in it in recent history.

My father packed his two children and wife into his car on 20th August of 1991, and pressed the pedal to the metal until we crossed the border to escape certain death from Serbian paramilitary thugs. Dad spent the rest of his life in exile. Even after our former Croatian homeland was liberated, the six years of Serbian rule set it back economically, infrastructurally, and most importantly socially for decades - it still didn't recover as most young people, including me, departed the region and didn't go back, decimating the society's renewal potential. There was simply no place to go back to, as that land was no more the same land we left. So Dad didn't return either although I know his heart ached for an alternate reality where all of this didn't happen, the peaceful continuation of days of old, something that is not ours to experience in our lifetimes, taken away from us by force by aggressive neighbors' selfish geopolitical interests. I wish Dad was still alive to witness how those same aggressive neighbors are now in pain too; while it doesn't cure our wounds, he would certainly find some poetic justice in it.

Schadenfreude? Damn well yes, we're entitled to it.

Oh, the irony

I'm reading Vernon Vinge's "Rainbow's End". At one point, he describes a day of an old man who's been cured from Alzheimer's in near future. Everyone is using wearable ubiquitous, always connected devices to access any data anywhere, and he's given a foldable electronic paper like device (rudimentary compared to what young kids are using) to access the web:

He wandered around the house, found some of his old books in cardboard boxes in the basement. Those were the only books in the entire house. This family was effectively illiterate. Sure, Miri bragged that many books were visible any time you wanted to see them, but that was a half truth. The browser paper that Reed had given him could be used to find books online, but reading them on that single piece of foolscap was a tedious desecration.

The irony? Rainbow's End is available for free here legally, and I'm tediously desecrating it in my web browser :-)

As a matter of fact, I don't like ithe medium. It's not the first novel I read on my computer, and probably not the last (I read few Cory Doctorow novels this way, bought them all in book form since), but I much prefer holding a deadtree book in my hand for my night reading. Especially when I spent the entire day anyway in front of the said computer. (But have been few time in a situation when I wished for Command+F to quickly go back to something while reading a deadtree...)

OTOH, Tor books started a free ebook program "Watch the skies" recently (non-DRMed PDFs); Jon Scalzi's Old Man's War is coming out soon on it. Karl Schroeder's Ventus is also available for free. Neil Gaiman's American Gods will also be e-published for free availability soon. I sense a trend here.

Ignorance is bliss

"Hi. My name's Attila, and I write shitty code."

The latest Really Bad Practice I managed to implement was making some business-level code aware of its underlying infrastructure. In particular, made them aware of Spring's ApplicationContext and such.

Ignorance is bliss, and this goes for code as well. A protocol handler unaware of transport peculiarities can be reused with any transport. Code that is unaware of memory management will automatically benefit from improvements in garbage collection.

The less your code knows about the context it is used for, the easier to reuse it in different context, but even more importantly, the easier for the context to manage it as it is supposed to do.

With dependency injection (DI) stuff like Spring, making components aware of the existence of it is bad, but it won't necessarily become apparent immediately. But when you want to implement something more involved; say, a graceful shutdown for your service, you'll suddenly no longer be able to have the infrastructure do the work for you. In my particular case, I could no longer rely on the dependency graph maintained by Spring after some of my components directly pulled some other components from the application context.

Of course it was a stupid thing to do. I usually know better.

As an excuse, let me say I only resorted to this in rather ugly situations. There are asynchronous callbacks from external systems, through mechanisms that make binding to the infrastructural context "normally" hard. And there's Java deserialization, the notoriously DI-unfriendly mechanism where you either resort to thread locals or statics (which reminds me of Gilad's new intriguing blog post "Cutting out the static", by the way). (Dependency injection in deserialized objects is something Guice user's guide will also admit being a problem for which the best and still quite unsatisfying solution is static injection.)

So yeah, I have the excuse of committing the faux pas when faced with a tough situation, but still. (Mind you, eradicating all Spring-awareness alone won't solve my problem of graceful service shutdown while it might still be waiting for asynchronous responses from an external system, but would certainly go long way toward it.)

The lesson is however clear; it is often the path of least resistance to reach from your code down to a supporting layer, but it can easily come back to bite you when the said layer was meant to be invisible. You think you might need to expose only a bit of a plumbing, but as time goes on, you realize that if you continue, you'll end up either uncovering the whole goddamned kitchen sink, or having to reimplement some of it. Then it's time to finally notice what you're doing (better late than never, I guess), backtrack, and refactor; bury the infrastructure back to where it belongs, not visible at all from the higher level. It sure does make some fringe cases harder to implement, but the important thing is that it keeps the easy bits easy.

Tom Lantos died today

Tom Lantos died today. One less great Hungarian and one less great American in this world; I was a serious admirer of him and his work. Even if not always agreeing with all his views, I do believe he made the world a better place through most of the things he did. I remember being amazed by quite a lot of things he did, but I won't rehash them here - Wikipedia is as good source as any for this. I distinctly remember him from two years ago when I saw in the news that the (then) 78-year old member of the Congress (elected 14 times, no less) was arrested for civil disobedience while protesting in front of the sudanese embassy in Washington against violence in Darfur. I was proud. I'm sorry though that even his influence and chairing of House Foreign Affairs Committee was not enough to move US into ending the Darfur conflict. Maybe if he was given a bit more time...

Isten nyugasztalja békében, Tamás!

Laptops at risk at US ports of entry

This keeps resurfacing in media every few months. This time, it's a Washington Post article about US Customs and Border Protection officers confiscating travelers' laptops (for indefinite time - some people didn't get theirs back for more than a year, despite being promised they'll get them in 10 to 15 days), or making copies of data on them, and/or forcing the people in possession of them to reveal their logon passwords. Also, people objecting to the procedures are denied entry to US.

Well, one more reason not to travel to US. At least, not with a laptop. Although, regardless of whether you have a laptop, they'll take all your ten fingers' prints when you enter, and that's also a rather strong cause not to. Over here, they take your fingerprints when you're taken in custody as a crime suspect. So depending on your cultural conditioning, having your fingerprints taken can be quite a humiliating experience. (I had my two thumbprints taken already on my previous US visits, and I detest the practice very much.)

Back to laptops and data.

As you might have seen from a previous weblog entry, I use FileVault on my laptop. Back when I used Windows, I used E4M for a similar purpose (although today I'd probably use TrueCrypt instead). FileVault is a 128 or 256-bit AES encrypted disk image for your home directory on Mac OS X. I even use encryption on my swap files.

I have very good reasons to keep all of this encrypted, reasons of both private and professional nature, that I do not wish to elaborate on further. If I were faced with the choice of handing over that data or being denied entry to US, I'd choose to not enter. Owners of some of the data that I keep on my laptop would certainly agree. (Yes, I keep data that doesn't belong to me but I'm trusted with it. If you work for a company in any significant position, chances are, you keep such data too).

Alternatively, in the near future, I can burn a BluRay disc with the contents of my home folder (encrypted), send it to my temporary US address in mail, and travel with laptop erased (or quickly erasable). Which still doesn't save me from the prospect of having my laptop confiscated at the border just because they can.

I'm lucky, 'cause I can mostly avoid going to USA if I don't want to. Some people on the other hand return home there; they don't have much choice aside from not leaving the country. Ugh.

Time Machine + FileVault experiences

I was reluctant to use Leopard's Time Machine "set it and forget it" backup because I also use FileVault (which basically mounts an AES-encrypted disk image in place of your home folder). The web was full of warnings how Time Machine does not work with FileVault, or does but it only backs up your home folder only when you log out, and you lose the ability to restore individual files through the GUI and need to fiddle with manually mounting the backed up images if you want to fish out something from them. Seeing however how I was getting undisciplined with my manual backup routine, I decided it can't be worse than having no backups, and went ahead and gave it a try.

At first, I was surprised to see that contrary to advertised, it did actually back up the encrypted disk image that hosts my home folder. It did it every hour. Every hour, it'd push the 30GB disk image over to the other drive. That filled it up, well, rather quickly.

Digging around, it turns out that the reason for this is that I kept using the Tiger-created FileVault, that uses a single file for the disk image. And Time Machine will happily back it up. 30 GB/hour.

So, next step was trying to upgrade to Leopard FileVault format, which uses a new "sparsebundle" disk image format, which is basically a folder with 8-MB files called "stripes" that hold the contents of the disk, plus some other files for tracking what's where. The ugly part of it is that in order to "upgrade" FileVault, you have to actually turn it off first (so it unpacks your disk image contents on the main filesystem), then re-enable it. I left it to decrypt over night (it probably only took an hour, but I left it there and went to sleep), then re-encypted in the morning (which took 40 minutes for 15GB of content). And then a secure wipe of the free disk space.

An immediate enormous benefit is that my disk image shrunk from 30GB to 15GB. That's right: my old disk image took 30GB even when it was hosting only 15GB of content, and no amount of compacting would've taken it lower. And it wasn't because of filesystem slack - inspecting the image with Disk Utility showed that there's indeed 15GB in there reserved but not used.

Now it's only 15GB, as I would expect it to be, with another 15GB reclaimed on my HDD. Hooray.

Another enormous benefit is that I no longer have Time Machine push 30GB over the FireWire every hour. Whenever I log out though, FileVault will compact the disk image (as it did in Tiger), and then Time Machine will back up - only those 8-MB stripes that actually changed, so the process is rather quick.

It is easy to understand why doesn't Time Machine back up the FileVault home directory while it's mounted - it would be too easy to back it up in inconsistent state as data is shuffled across stripes. Of course, I wish Apple engineers had more time to think about this, and solved it in a smarter way. I myself could tell them two better ways to handle this:

One: add a shadow file to the disk image while backing up to hold concurrent changes, merge changes into the image file upon backup finish. The underlying BSD foundation of the OS supports this. It would, however, probably create a perceptible temporary freeze of the system while the changes in the shadow file are merged with the disk image.

Two: create a similar encrypted disk image on the backup drive, mount it when the backup reaches the home folder, and just perform the whole Time Machine backup procedure between two disk images. I actually had a homegrown solution that did precisely this using rdiff-backup back on Tiger. Actually, when I first heard of Time Machine, I sort of hoped Apple will base it on rdiff-backup, and use this method to handle FileVault accounts.

rdiff-backup has the advantage that it can incrementally back up small changes in large files using the rsync algorithm (Time Machine copies whole modified file each time), and my method also preserved this incremental backup property in FileVault accounts, on a per-file basis, preserving both filesystem and backup semantics. I guess they lacked one smart guy in the engineering division for Time Machine, who was probably busy helping the iPhone division make their deadline... Oh well.

Anyway, now with tolerably fine-grained FileVault backups, I'm happy. Yes, I need to log out in order for my home folder to get backed up, but I was doing this for a while anyway, using CCC or Disk Utility to copy the whole internal disk to external. I used to do backups once a week; now I get automatic backup of the system every hour (which could come in handy if ever, say, a software install goes awry; never happened on Mac with me before though), and automatic backup of my home directory whenever I log out (which is not less frequent than once per week). Of course, the majority of my machine's state change happens in my home folder, so having its backup be more frequent than the system backup would of course be preferred, but such are compromises - I can't afford to run without FileVault.

(You might ask what happened with my homegrown rdiff-backup solution? It fell victim to my switch from a PowerPC to Intel Mac, as it would've required me to recompile a bunch of GNU stuff from source (rdiff-backup and its paraphernalia) which I didn't have time to do at the time of the switch, so it fell into oblivion...)