MWC: Augmented Reality

Following the announcements from Barcelona there are only a few interesting devices coming up which could make the difference for Augmented Reality on mobile phones. The Toshiba TG01 seems to be heavily optimized for media delivery, 3D and therefore brings hopefully some good number crunching capabilities. However, Toshiba was never a major player in the mobile phone market - it would be interesting to see how the market develops for them but I would suspect we are not seeing many of their devices. On the other hand the line-up from HTC, which apparently produces 80% of all Windows Mobile devices, is good but not really a leap forward in terms of media capabilities. There are some good improvements on the HTC touch diamond - it seems they got rid of that horrible plastic cover on the back and thus the camera - good for AR applications: we can see properly now - thanks HTC. The other interesting player is Acer which is making a big comeback with their line of OpenGL ES enabled phones.

For now, all the Android phones are not really suitable for realtime AR (please don't mix up location based services with AR - no, Wikitude is not an AR application). The speed delivered through the JavaVM with all the layers of abstraction, which is the advantage of Android, is counterproductive for AR - we need close to iron speed as the devices don't have power enough to make the overhead of Java neglible.

The iPhone wasn't really present - but it is not interesting for a wide spread AR application as Apple is not allowing us to use the camera. The point is, we can jailbreak our way in, use the private frameworks with self generated headers and do things with the camera but by doing so we break about everything in the EULA, DMCA and whatever could be convenient to throw in the path of progress.

Linux phones like the Palm Pre are the one I am more exited about. Palm released the first chapter of the Developer Guide. I appreciate the effort they put into their 2D/2.5D GUI interface but I am more intrigued that they also thought of "native" applications. There is GStreamer and if I am not completely wrong they have some proper graphics chipset to do the GUI - so it shouldn't be bad either. Well, lets hope they are not messing this one up.

Nokia is as boring as ever - they got rid of the PowerVR chipset in the N96. Nobody knows whats going on with the N97. For most of the newer phones the OpenGL ES support is dismal. The processors are good for some marker tracking but way underpowered for serious NFT or model based tracking. And I personally find the Symbian way of developing applications offputting, thrown back in the mid 90's. Well, I might need to reconsider as the successor to my current most preferred phone, the Samsung Omnia HD is using Symbian S60 now - grml.

As for me, my tracking in SSTT code performs well on the Samsung Omnia now, I could easily do 20fps but the camera only captures at 15fps. If light is low this even goes down to 7fps. Why do cameras always need to be stuffed with their automatic image-rubbishification methods. The other problem I face is the fillrate on the GDI driver. I can get 40fps with doing a 320x240 image in the middle of the screen (which looks awful). If I go up to the full size the framerate drops to 17fps. Conclusion, I need to decouple capture, tracking and rendering completely and do some lightweight syncing - otherwise either of the passes needed for AR will disturb the overall user experience.