Wiimote Project

Wiimote Projects => Wiimote Desktop VR/Head Tracking => Topic started by: yashardel on January 20, 2009, 11:06:09 AM

Title: How to estimate z coordinate ?
Post by: yashardel on January 20, 2009, 11:06:09 AM
I am doing a headtracking project in c++ using wiiuse code. I was wondering if the IR-Camera is only capable of detecting x and y coordinates , how one can find the other coordinate z based on the available data. For sure there should be method (maybe a geometrical approach) to do so otherwise the headtracking solution is not possible! If by any means , one can find relations that leads to the derivation of z , he/she can then use the (x,y,z) for tracking.
I would greatly appreciate ideas !

Thanks ...
Title: Re: How to estimate z coordinate ?
Post by: Wiweeyum on January 21, 2009, 04:13:07 AM
It seems that the headtracking code already simulates a z coordinate by allowing you to get closer to the screen and the image gets bigger. That's done by calculating how far apart the two dots are from eachother.

If you're talking about a bit more virtual reality, then this video here would be pretty helpful.

He builds his own sensor bar with four dots in a 3D geometric setup. Depending on where the dots are, based on their perspective, it calculates a full 3D location.

Does that answer your question?
Title: Re: How to estimate z coordinate ?
Post by: yashardel on January 21, 2009, 04:45:54 AM
Hi Wiweeyum!
Thanks for your response. It seems to be pretty closed to what I am looking for.
Now my problem is exatcly how to use perspective to calculate a full 3D location.
I need some details (code or references) that exaplains the theory behind.

Well, honestly I should say it seems alittle strange for me how this is possible to get the information from
One signle 2D wiimote and create a 3D scene. Before I was thiniking, probably the infrared intesntity recieved by wiimote is used to give an approximate esitmate of the depth infmoration (i.e distance/z coordinate) ...

What do YOU think...?!
Title: Re: How to estimate z coordinate ?
Post by: Wiweeyum on January 21, 2009, 05:13:15 AM
Well what he does in the video above (I'm no programmer, only a theorizer) is he has the left, top, and right dots on the same plane (the XY axis), and then he places the fourth dot further forward (on the Z axis). When the camera is looking at the sensor bar from the front, you get a diamond looking shape. When the camera is looking at the sensor bar from the side, then the Z axis dot (front middle) will move to the left and right quicker than the ones behind it. Make sense? Watch the video above at around 4:00. This is when he's just seeing the four dots on a black screen.

As for using intensity to calculate the Z axis, I think it would be possible (as can be seen if you play around with the sensitivity screen on the wii console) but it would be very very inaccurate. Blob detection (like used in the wiimote) will automatically adjust its contrast to make the brightest dots visible. You can see this in Johnny's finger tracking video HERE (http://www.youtube.com/watch?v=0awjPUkBXOU). When he reflects light off of his fingers without any reflective tape, the tracking picks up all the dots of light reflected and it's not accurate. When he puts the tape on the tip of his fingers though, the same amount of light is reflecting off of his hands, but the reflective tape is so much brighter than his skin that it only tracks the reflective tape. So therefore, the wiimote will automatically track the brightest object. Make sense?

Also, if you were to use only intensity to track the Z axis, then the LED lights would have to be exactly the same brightness every time. Since there are a number of different LEDS that people use, it just doesn't seem plausible.
Good idea though.

Another idea is to triangulate the location of a dot by using two wiimotes to watch the same dots from two different angles. This concept can be seen here in this video.
Title: Re: How to estimate z coordinate ?
Post by: yashardel on January 21, 2009, 06:28:05 AM
Well thanks alot for all infomration.

As for the case of two Wiimotes, it should be a good solution with a nice performance and if I think correctly one camera wil be responsible for x/y imaging and the other for y/z (example). However, due to limiations on my project I will not be able to use more than 1 WWIMOTE DEVICE.

Also for the case of intensities you pointed out important problems while implementing it that way. In addition to that, I guess tilting might be another problem because sensors might confuse it with depth movement. Since titlting also can cause the differnce in amount of intensities recieved by the camera.

But now back the main part, I guess I will use the idea of utilizng several LEDs . I tried to skecch the geometrey and see how it could be solved. I started with a trinagular pyramid proejction on x-y plane. (the vertices of the pyramid stand for each LED) and then tried to solve them lineally after writing all the equestion I could come up with but seems it's not so easy to find all the relations to solve the equations. I guess there should be some nonlinear equations that will be used for calculating the solution as the peron ion the clip aslo talks about "overdetermined nonlinear equation". If we can know the equations , then the problem is almost solved !!!
So much in need of ideas...!!!?

I am not also a programmer and my major is in sigal processing. i have a more theortical/mathemtical view toward the problem!!

Thanks for your time!

Title: Re: How to estimate z coordinate ?
Post by: yashardel on January 21, 2009, 08:08:40 AM
By the way, I have another question in this respect; it seems that John Lee's IR tracking system works pretty fine with 2 LEDs (rather than 4) and he is also using the perspective transmformation method. Given the fact, I am wondering if it's not easier (when it comes to mathmatical calculation) to do that with less noumber of LEDs ? how do you compare his method with the one in the video clip you just sent !

Thanks again!
Title: Re: How to estimate z coordinate ?
Post by: Wiweeyum on January 21, 2009, 06:07:47 PM
I wouldn't have the faintest idea on how to create equations for solving location of the camera with the four LED setup. I understand the concept, but I have very poor mathematics skills. My suggestion would be to email him and ask. I'm sure that he would be more than willing to help you out. Here's a link I found on the YouTube page where I got that video. It says:
More information, instructions how to build a custom IR beacon, and source code at

That would be a good place to start I think.

Johnny's setup does simulate depth with the two LEDs, but what it doesn't do is simulate different angles. You are able to move closer and side to side, but looking at an object via rotating around it isn't possible with Johnny's code.
Title: Re: How to estimate z coordinate ?
Post by: yashardel on January 22, 2009, 03:17:52 AM
Hi Wiweeyum!
I have seen the webpage and more detailed info is foud there, not yet so complete! I will contact him and try to get more precise information about the mathematics! Thanks for the tips!

I had really know idea about the flaw in John Lees project and I didn't know that it's not possible to look at an object via rotaing. Well, maybe it's beacuse that is not demonstared clearly in the video clip ! So basically what are you saying is that it works fine for Scrolling Left/Right and moving Far/Closed but NOT for Tilting, right!?

Wiweeyum, in John Lees C# code there are some variables related to IR-Data structure namely Size 1,Size 2,Size 3, Size 4. Could it be semehow related to IR-Size tracking that we discussed before?


Title: Re: How to estimate z coordinate ?
Post by: Wiweeyum on January 22, 2009, 03:35:53 AM
I think you have it down pretty good, and I think you've stretched me to the end of my knowledge. As soon as you start getting into the code, I'm lost. I really have no idea what the IR Size variables could be related to. I hope someone with more knowledge on the subject can help you out.

Is there a way to test those variables? Like somehow print those variables onto the screen visually in a way that constantly updates so you get an idea of how and when they change? I'd like to know myself.
Title: Re: How to estimate z coordinate ?
Post by: yashardel on January 23, 2009, 02:46:07 AM
I think I am now clear about the Geometrical mathematics that John uses to get the depth information. Infact, I think he is taking benefit of the knowledge that camera's Fie d of view if 45 dergrees or (pi/4): Given the fact, one can assume camera's field of view FOV=(pi/4) rad/pixel; And therefore for each dot postion, the corrsponding angle to the camera is easily found and having the angle and dot postion the z position is correspondingly derived! It's a very simple concept ; some useful information is given here :

But I am not still sure about the other method that the guy in your video clip talks about. His method seems to be more complex with alot of nonlinear equations to be solved, but how the two methods differ is an interesting question for me and I would appreciate anyone with knowledge in this area to share it with us!

Finally regarding the code, I am trying to see how the Size variables are used ! The link I sent you says that the intensities are checked before the calcualtions are started which seems logical! But so far not so much similar is found tracks in the code !

What do you think!?