In UK next week

I'll be in Reading, UK between April 3 and 7. If anyone feels like meeting me over a beer in Reading or London area, drop me a note so we may be able to arrange something.

Bleep is about Copehagen interpretation

So, two days ago Kriszti and me watched "What the Bleep Do We Know!?™". Based on reviews, I had some good expectations about it, and while I must say my opinion is still quite fuzzy, the prevailing feeling is that of utter disappointment.

The movie does convey positive messages, promotes the benefits of positive thinking, saying that what you think affects who you are and what your destiny will be etc. In this regard, I can completely agree with it.

Then there's the big but.

First, the movie is shot as a half-documentary, half-fiction, densely interspersed. It aims to gain scientific credibility to its message (I'll talk about the actual message in a bit) by having lots of experts (as well as few "experts") telling their opinion in the documentary half, supposedly reflecting on the happenings in the fiction part. However, the things these people say often feel out of context and they create more confusion than they explain. For a movie supposedly wanting to promote a message and back it up with scientific credentials, the editing is done very poorly (assuming the raw interview material was not as bad in the first place). The Wikipedia page for the movie covers much of the controversy, including one of the interviewed scientists objecting that they edited his interview so that it looks like he supports the movie's claims where he really does not, as well as factual errors, displaying scientifically unproved experiments as facts, and lots of jumping to conclusions.

So, what's the movie about? Well, the movie bases its message on the Copenhagen interpretation of quantum mechanics. As you might know (and if you don't, go and read the link) in that interpretation, observing a quantum phenomenon causes the nondeterministic and irreversible collapse of the wavefunction. This interpretation's problem is that it requires an "observer", thereby introducing consciousness into the theory. The movie goes on to argue that this way, our consciousness actively affects the reality that surrounds us, by observing it, hence it jumps to the conclusion that we create the reality. A big negative point for the movie, in my opinion, is that unless you already studied quantum mechanics, you probably won't understand it. The interviewed persons say "quantum mechanics this" and "quantum mechanics that" all the time, but the explanation of the uncertainity principle is constrained to the scene in the basketball court, and while I was watching it I thought that if I didn't knew all of this already, I'd probably be no less left in the dark after this movie.

Anyway, I must disagree with the movie's conclusion about our consciousness creating the world around us, and us being indistinguishable from God. These views are very old, by the way. You can go back at least to Baruch Spinoza for the philosophical theory of unity of the nature (humans included) and God. You can refer to either George Berkeley or David Hume for the philosophy of subjective idealism. Nothing new here. Supporting these ideas with the Copenhagen interpretation seems to me a bit stretched.

Moreover, and this is my basic cause for disagreeing with the movie, is that there is a different quantum mechanics interpretation, the Many-worlds interpretation (MWI) that completely eliminates the need for any sort of observer for collapsing the wavefunction, as in this interpretation, the wavefunction never collapses. I won't go into explaining MWI here, I'll again direct you to the link above.

Rather, I'll tell you what does MWI mean to me. MWI, if you subscribe to it, does have one very interesting implication. Namely, that all possibilities realize themselves at the same time. On high level, whenever you are in a decision situation, reagardless of how you decide, all outcomes will realize themselves in the probabilistic space. As consciousness is widely regarded (not proven, though) as being a completely classical (in the "classical physics" sense, that is, not quantum mechanics level) phenomenon, the linear stream of events you experience as your consciousness is one path through the global wavefunction of the universe. Whenever there is a decision, the path forks, and you experience one of the paths, while multiple "you"s that share your identity up to that point will experience the other paths.

What does it mean in practice? It means that when you're maybe hesitating on something, like talking to that attractive girl sitting alone in the bar, or telling your coworker that he's being obnoxious about something, or dare to learn parachuting, etc., you need to realize that you will. And also that you won't. Both. At the same time. With different probabilities though. You only get to experience one of these paths, and there's no going back and retracing your steps once you did. You can consciously choose the outcome that'd otherwise be lower probability, leaving the higher probability but duller options to another you. (Although balancing bravery and foolishness is a good idea generally :-))

Sounds wild, and some will argue that assuming such constant forking is in violation with the Occam's Razor principle as it creates a continuum of parallel universes. Proponents will argue that there is no such thing, there is only one universe, represented with a single probabilistic wavefunction, particles exploring all paths through it, and the consciousness you're experiencing being one particular path of particles making up your physical self at the moment. There's also no information flow sideways or backwards that's a fond plot device of fiction works involving time travel and/or parallel universes. Proponents will say that the simpler formal expression of this interpretation actually makes it much more in line with Occam's Razor than the Copenhagen interpretation. (Indeed, MWI operates with less assumptions, is expressible with more elegance on mathematical level and doesn't need the concept of observer).

Also, it doesn't clear you of any personal responsibility, as free will is still completely realizable within this framework - remember, consciousness is a classic physics phenomenon, and regardless of the low-level mechanics and the fact that what you experience as yourself might be taking one path through the wavefunction, while others forking selves are experiencing all the other paths, it still makes you responsible for the acts on your path.

Whether I personally subscribe to MWI? Well, you see, it's hard to decide. I do. I don't. Both, at the same time :-) It's just a theory, and many regard it as unfalsifiable, which shuns it into the domain of belief rather than science. Sometimes, when things go bad, I can find comfort in thinking that at the same time, if this theory holds, then things also didn't go bad, and that some of my probabilistic parallel selves are having it better at the moment.

Check out Restlets

For all the fans of the REST approach out there (who also happen to code in Java), seems like someone is working to create a replacement for Servlet API that is explicitly designed for writing HTTP systems the REST way: check out Restlets. Haven't had a chance to look into it deeply, but it's definitely on my list of things to inspect more closely. The folks wisely created (similarly to Servlet spec) a separate spec and a separate reference implementation - this is quite important for widespread adoption, as it should allow things like alternate implementations, i.e. one built on top of servlet API (for leveraging already tried infrastructures of servlet containers out there).

That said, the reference implementation they ship is however a completely standalone server - real men handle their port 80 directly :-). Oh, the RI also uses FreeMarker as a default view technology, being a replacement for JSP :-). Well, I guess that makes me ever slighlty biased.

Wasting time on debugging memory errors again

I'm again losing days of work on debugging an OutOfMemoryError in a production system. The tricky part is that the code implements a very thin wrapper over a database, bulk processes messages, and is totally stateless. The software stack is JVM, then JDBC driver, then Hibernate, then Spring, then my code. There's no memory leak, I could confirm this much with a profiler - whatever was causing the trouble was allocating temp objects held by references on stack, so when the OutOfMemoryError unwound the stack, the smoking gun was gone...

Finally, I turned to JDK 6.0. It's in beta at the moment, but it has a very useful feature: a command line switch "-XX:+HeapDumpOnOutOfMemoryError" that'll cause a full heap dump (in HPROF heap dump format) whenever an OutOfMemoryError is thrown. After having the ops guys install the JDK 6.0 on the machine, I restarted the software under it, with the abovementioned switch, sit back, and waited for a memory error with a grin. And waited. And waited some more. Finally, waited for more than two hours while the system was running on full load. Nothing.

To my fullest and utter surprise, the memory error doesn't manifest itself when running under JDK 6.0, even after few hours of fully stressed operation. Damn. Isn't it typical? Maybe we have again hit a JDK-specific memory bug that got fixed in this later JDK? Unfortunately, I really cannot seriously propose to colleagues to run our production systems on a beta JDK...

Anyway, "-XX:+HeapDumpOnOutOfMemoryError" sounds like something that should have been part of the Java long, long ago. Big enterprise systems run into memory problems. That's a fact. There's few tasks as frustrating as trying to isolate them as the problem inherently manifests itself nonlocally. To have the JVM dump a heap snapshot at that point is invaluable. Don't having this feature caused me one sleepless night too many by now. I heard YourKit will have (or already has?) the ability to analyse HPROF snapshots, which would be really dandy for excavating in the results. Failing that, I still can use the HAT profiler, hopefully they have incorporated my patches to it in the past one year :-)