13949712720901ForOSX

13949712720901ForOSX

Working Java 6 on Mac OS X - but not from Apple

Turns out there's a working Java 6 for both Mac OS X Tiger and Leopard. It is a port of the Java Research License licensed codebase for BSD, so to use it legally, you need to agree to JRL first. It's developed by a person named Landon Fuller, and he says on the page:

The Mac OS X work is based heavily on the BSD Java port, which is licensed under the JRL. The BSDs develop Java under the JRL; FreeBSD has negotiated a license with Sun to distribute FreeBSD Java binaries based on the JRL sources.

As the Mac port stabilizes, I am merging my work upstream into the BSD port, and in turn, it is a goal of the FreeBSD Java project to merge their work into OpenJDK. I've signed a Sun Contributor Agreement in preparation for this...

There are some minor features lacking (minor if you're like me, using Java for server side): sound doesn't work yet, nor is there a native Aqua Swing UI.

Via Charlie Nutter (who also perftested it with JRuby and sounds quite impressed).

New MacBooks finally have 3D graphics in hardware

I just noticed that new MacBook models ship with Intel GMA X3100 graphics chipset instead of GMA 950 in previous models. Notable difference is that X3100 has 3D rendering hardware - a shader processor with 8 execution units fully supporting transform & lighting, with support for DirectX 10.

That's quite a big news for people who previously dismissed MacBook as an option because of weak graphics chipset (i.e. people who'd dual boot into Windows for gaming).

On the downside, GMA X3100 still doesn't have dedicated RAM, and it'll actually take away 144MB of system RAM (160 if you use an external display), compared to 80MB that GMA 950 used, so YMMV.

"Some thoughts on security after ten years of qmail 1.0"

There's a paper "Some thoughts on security after ten years of qmail 1.0" written by qmail's author, Daniel J. Bernstein. (I found it though Bruce Schneier's weblog). The paper is well worth reading because Daniel is extremely security-conscious. As he says:

In March 1997, I took the unusual step of publicly offering $500 to the first person to publish a verifiable security hole in the latest version of qmail: for example, a way for a user to exploit qmail to take over another account. My offer still stands. Nobody has found any security holes in qmail. I hereby increase the offer to $1000.

He actually gives lots of practical advice on how to make your code more secure: security holes are bugs, so you should strive to eliminate bugs in general. Also, you should strive to eliminate code (as number of bugs is usually proportional to number of bugs). Finally, you should minimize the amount of trusted code and have all untrusted code run in a sandbox (using his words, a "prison").

Now, to reflect a bit on sandboxing myself from a Java point of view, Java is fortunately quite well suited for minimizing trusted code. When you run a Java program under a security manager, code gains privileges based on where it came from, and the privileges of the current stack frame are the intersection of privileges of its code and all of its callers down the call stack.
There's a facility for privileged execution, SecurityController.doPrivileged(), where the code within the privileged block will run with full set of its own privileges, and not get narrowed down further with security restrictions placed on its callers. It can be a dangerous feature, and must be used judiciously lest you create an exploit vector through it. I.e. if your code accepts a file path from its callers, reads the file in a privileged block, and returns the file content to the caller, then a caller can potentially exploit this to gain access to contents of files it would otherwise not be able to see.
I'm actually quite conscious to enable easy integration with Java security manager in software I write - you will find a PolicySecurityController class in Rhino, as well as a SecureTemplateLoader in FreeMarker, both allowing running JavaScript code or FreeMarker templates, respectively, with their own set of privileges defined in the effective JVM-wide security policy based on where they're loaded from or whether they're digitally signed (same aspects are used to assign privileges to Java code). FreeMarker also uses SecurityController.doPrivileged() in few places judiciously.

Anyway, back to Daneil's paper: there's a wealth of food-for-thought for any practicing software developer: how we often sacrifice security for speed, how hidden data flow (global variables and singletons, people!) leads to security issues, and how this can be fixed by further decoupling of components with explicit narrow communication (i.e. message passing across processes), and so on.
A section entitled "Avoiding parsing" also speaks very closely to my heart.
So, recommended reading.

Wired profiles Randall Munroe

Wired has a profile article on Randall Munroe. Munroe is the Xkcd guy. If you didn't know Xkcd before, you can thank me later for pointing it out for you.

"JVM Dynamic Languages Metaobject Protocol" now released

I finally got around to packaging up the current state-of-art of my metaobject protocol library as a downloadable release (source + binaries + documentation), plus putting up a very basic website (hosting an information page + JavaDoc) for it.

I haven't got around to setting up dynalang.org yet, so the website is for now hosted at http://dynalang.sourceforge.net.

The fact there is now a release does not intend to confer either a sense of completeness or rigidity. It is versioned at humble 0.3. It is pretty much open to modifications and is also probably not complete yet (i.e. I fully expect people to need further features for integrating with their particular language runtime). The release just strives to make it easier for people to get started with it, as it's now available as a HTTP download instead of only through SVN. Also, having a release means there's now a baseline for purposes of tracking changes in a changelog file etc. Unit tests cover about 75% of the code right now, so it's fairly safe to say it does what it is intended to do, but of course, bugs are always to be expected.

In completely unrelated news, today's my birthday too :-)

Leopard firewall breaks Skype

Apparently, Skype.app on Mac self-modifies its application package. Leopard OTOH digitally signs application packages after you download them from the net and allow them to run, and also when you allow them to accept connections through firewall. Skype's self-modification causes the signatures to get invalidated. After this, Skype will completely fail to launch. It'll bounce twice in the dock, and that's it.

Options: (a) don't use Skype (b) disable firewall. Damn. I'm going with (a) until Skype fixes this. Actually, I tried disabling the firewall, and for me even that didn't help. Double damn.

New Language Features for JDK7 - from the Java Community

A newsletter from Belgian Java User Group (BeJUG) landed in my Inbox yesterday, and boy, am I glad to see it. You can read it here. Basically, BeJUG is submitting the JSR for closures in Java, based on Neal Gafter's proposal. This is huge news for several reasons. Reason one, we need closures in Java NOW. Reason two, this is the first JSR filed by a Java Users Group, and also the first Java language JSR led outside of Sun Microsystems.

The second jolly good news in the same newsletter is that Google also submitted a JSR for a "collection of smaller language features". Smaller? They're proposing type inference, extension methods, catch for multiple exception types (Brian Goetz will be glad. Hell, me too!), and improved syntax for easing the catch-rethrow pattern. These ain't small in my book.

We need to realize that C# has had type inference, extension methods, object initializers, anonymous types, and lambda expressions for two years now. It'd be about time for Java to catch up with the times, and I'm really glad to see community finally taking initiative to make it happen and not waiting for Sun.

Network traffic prioritization by specific cost

I've been following with some interest the recent Comcast scandal where the ISP started inserting TCP reset packets into BitTorrent traffic of its clients, thus disrupting it.

Comcast claims they do it to protect the ability of "normal" users to use the bandwidth from "excessively using" users that hog it.

(I personally believe Comcast tries to minimize the overall amount of traffic that flows out of its network to other ISPs as it costs them money in the fiscal clearing of cross-ISP traffic.)

But let's entertain the idea for a moment that Comcast would really like to protect their "normal" (read: low-traffic) users from their "excessively using" users. How would I go about implementing a fair system that achieves this goal?

Well, certainly not by disrupting the traffic.

Below is an idea that I believe would result in fair allocation of bandwidth within the network of a single ISP with paying customers. I must forewarn you that while I design software systems for living and often need to deal with management of limited resources, I'm just a layman when it comes to the narrow field of packet-switched network traffic management, so what's written below might not make sense at all because of some arcane aspect or the other that I know nothing of :-)

Anyway, here it goes:

The ISPs already measure the amount of traffic generated by every user within a billing period, that's how they are able to detect "excessively using" users today. However, instead of handicapping a user that "excessively" uses the bandwidth, the metrics would be used only to select which user's packet to drop when there is insufficient bandwidth on the routers' outbound lines, but only then.

The strategy would be to always drop the packet coming from the IP address of the user who had the highest traffic in the billing period so far. Or, in case of differently priced packages, or even different billing periods, the user with highest traffic/fee ratio for his current billing period, expressed nominally in byte/dollar.

This way, the proverbial "only checking his mail" user would always experience a fast connection, followed by a user who listens to online radios and watches some amount of online videos, while the "excessive" p2p users would be only competing for leftover bandwidth among themselves.

It'd be easy to suitably extend the idea to a nondeterministic scheme where a packet to be dropped is picked randomly, but the probabilities are weighted by their user's dollar/byte ratio. That'd give a user who heavily used p2p at the start of the month a standing chance to still be able to check his e-mail even when another user comes along who floods the network with p2p traffic near the end of the month.

Feel free to point out why this wouldn't work, doesn't make sense, or is already invented in this or similar form and used somewhere :-) I've tried uncovering this technique using Google, but have found only port/service/application based QoS prioritizing, and didn't find anything specifically describing this -- namely, prioritizing based on user's specific monetary cost of traffic, amortized over a billing period.

As far as I can tell, this method preserves network neutrality, as it does not discriminate based on either the type of traffic or the destination of the data. It also doesn't degrade the throughput for anyone in any way as long as there is sufficient free bandwidth.

Leopard so far

This Thursday, I upgraded my MacBook Pro to Mac OS X 10.5 Leopard. It was a holiday over here, so I had four days (Thursday to Sunday) to work out the wrinkles in case something didn't work. I did the usual precautions: mirror the boot drive to an external drive, boot from the mirror to make sure it works, then shut down the computer, physically disconnect the external drive, and only after all this start the OS install.

I decided to give myself a chance to avoid reinstalling everything, and thus chose the "Upgrade" option instead of a clean install. After an hour of DVD consistency check and an hour of copying files, the machine rebooted and I was greeted by a working Leopard. It went incredibly smoothly.

I proceeded checking whether everything works. First on the list was Cisco VPN Client which I need for work. I was sceptical, as the Cisco VPN client integrates tightly into the system, installing kernel extensions for networking. To my very pleasant surprise - it worked! Next was ssh, which also worked. VMWare also works, no problems there.

Then there's some more arcane stuff:

An unsupported printer driver for my Xerox Windows-GDI printer, cross compiled to a PowerPC binary from a Linux CUPS driver - works!

My heavily customized, start-on-boot MySQL setup - works!

There's of course no Java 6 yet, as discussed earlier, but the good news is Java 5 is at least upgraded from 1.5.0_07 (found in Tiger) to currently latest 1.5.0_13 in Leopard.

Eclipse 3.2.2 works overall. There are keyboard issues though. I.e. if I press Command+Shift+U ("Occurrences in File"), it brings up a context menu (to choose among "Identifier", "Implementing Methods", and "Throwing Exceptions"). However, both the menu and editor window receive the keyboard events for arrows, and when I hit Return, the context menu disappears altogether (without performing the selected operation), and the key is interpreted by the editor (i.e. inserts a new line...). That's one minor annoyance.

One of my pet peeves is also fixed finally: I can finally use the Return key to activate the "Allow" button in Keychain Access' "Copy to Clipboard" dialog. Oh, progress! Previously, if I didn't want mousing, I had to do Tab-Tab-Tab-Return. Still no keyboard equivalent to invoke the said dialog though... Also, the "new password assistant" palette now syncs with the password field, so I needn't copy/paste a generated password back to the main editor window of Keychain Access anymore. Yeah, small things, but they do matter.

Also, there was something amiss in my user account in Tiger - lots of programs forgot their license number when I logged off (Delicious Library and Disco most prominently), and it also lost my keyboard shortcuts whenever I logged off, so after a while I lost the habit of using custom shortcuts, as you can imagine... Upgrading to Leopard fixed both issues.

The system is noticeably snappier. Tiger had the tendency to stall under heavy load. "Heavy" is when I run a Terracotta server and two clustered Tomcat instance on the MacBook Pro, talking to a special Linux appliance running in VMWare, occasionally recompiling the code or interactively debugging from Eclipse. Oh, and running a BitTorrent client in the background to boot it. Now, Tiger did have issues with this even with 2GB of RAM, to the point of UI freezing up for 20-30 seconds whenever I switched applications with Command-Tab. Leopard tolerates such loads much better. I guess virtual memory management got smarter with regard to handling applications' working sets, and there are probably lots of minor kernel optimizations and reentrancy improvements. Overall, noticeably better. Lot less beachball sightings.

As for games: Diablo II works nicely even on an Intel mac (it's a PowerPC binary). Of course, only when you install it using the "carbonized" installer, the version on the CD requires Classic subsystem, which is gone from Leopard.

Age of Empires III has issues though. In single player, all is dandy, but playing in network is completely borked by network sync issues. Few seconds into a multiplayer session it'll say "Out of sync. Exit and try again". Truth be told, the other machine still had Tiger on it when my son and me tried it; I installed Leopard on it since.

After two days when I was sufficiently convinced that it works okay, I installed it on my wife's iMac G5 as well, and faced an interesting limitation. You know those custom iChat backgrounds Apple showcases as new iChat features? The ones where you can have the room behind your back replaced with a moving Niagara falls scenery? Well, you won't get them on a PowerPC Mac. I wanted to try it out for fun between our two machines, but the iMac's 2.1GHz G5 CPU is apparently not strong enough for them. Intel Macs only. Now, come on, Apple, I don't buy it. You probably just had a nice hand-optimized assembly with MMX and SSE instructions in there somewhere, and couldn't be bothered to port it over to PowerPC AltiVec. You know, the one you touted as being the superior vector unit. I think we're experiencing the first signs of PowerPC users falling out of grace. I don't want to sound dramatic, but I mean, I can maybe, just maybe understand a G4 not being strong enough for this task, but a 2.1 GHz G5? What about people with dual or quad G5s? This is a minor, but sad letdown.

Also, my MacBook Pro took on the habit of waking up at precisely 1:00:00 AM every night after the Leopard upgrade, without anything being set in the com.apple.AutoWake.plist file. Since 1 AM in my timezone (GMT+1) is actually midnight GMT, I suspected this to be some stray zero making its way into EFI or something. Pretty much, after I overwrote it by setting "Start or wake up every day" in Energy Saver to 9 AM, the phenomenon went away.

Well, that's it so far. I'm also fooling around with XCode 3. It's a very polished development environment, and I'm toying with the idea of creating some native Mac OS app. Y'know, just to unbox myself a bit from my "server side Java" box and try the "desktop app in Objective-C" world for change.