I was cruising this forum and came across this post. I've been thinking of something along the same lines (sans the holographic display), though Id like the CAD approach, at this moment I am just experimenting on 3D modeling software...
http://www.blender.org/ in particular (python scriptable, that's a big thing for me).
What I've done is very basic (though not fully working right now and it's only available for OSX at the moment).
I've written a simple program in Objective-C that reads the irData from the wiimote, package it up, and send it as UDP packets over a local socket.
A python script reads from the socket and does the necessary mapping to the display coordinates.
At the moment, I'm only concerned with getting 2D to work well. I may add a second wiimote for accurate 3D support, and then work on using the python script inside Blender.
The python script + blender should work with no tweaks in any of the major three platforms. The wiimote stuff, however, has to be ported using the libraries and drivers of choice for each platform.
Anyhow, nice to see there's others with an interest in using this in 3D/CAD apps...
If you're still working on it, I'd like to hear how things are going...