topbanner_forum
  *

avatar image

Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length
  • Thursday March 28, 2024, 6:47 am
  • Proudly celebrating 15+ years online.
  • Donate now to become a lifetime supporting member of the site and get a non-expiring license key for all of our programs.
  • donate

Author Topic: An Analysis of CamSpace as a Webcam Mouse and Gaming Controller  (Read 24110 times)

VideoInPicture

  • Honorary Member
  • Joined in 2008
  • **
  • Posts: 467
    • View Profile
    • Circle Dock
    • Donate to Member
Well, it seems that CamSpace, http://www.camspace.com, is in Beta3 mode and they sent out invites to the people that signed up (I never got mine, though. Anyone got theirs?).  Here's three videos of someone trying to use CamSpace to emulate a mouse: http://www.youtube.c.../watch?v=0AYnNzr_uO4, http://www.youtube.c...&feature=related, http://www.youtube.c...&feature=related and here is a video of someone using it on Google Earth type application: http://www.youtube.c.../watch?v=9xoSS5QiyX4. Note that these videos appear to be from people who have actually gotten copies of the program and are not promotional videos.

Some of you may remember the WebCam Signature program I submitted for the DC Programming Contest that contained an experimental webcam mouse mode (http://webcamsignature.wikidot.com). From coding that program and watching these videos, I feel that I can provide some useful information for people that haven't gotten an invite yet.

From reading CamSpace's wiki and watching CamSpace demo videos, I can conclusively say that CamSpace primarily tracks colors (like WebCam Signature) to determine where to move your controls.  One piece of supporting proof is their colored wheel printouts at http://www.camspace.com/featured/.

The advantage of this method is that it is fairly simple to program and does not require any complex mathematics. You tell the CamSpace program what colors you want to track by holding up your colored object in certain positions that the program specifies. This is what allows you to use “any object” as your control to your computer. More on the “any object” part later in this posting.

The program tracks the areas of the blobs of colors to determine the z-depth.  However, the problem with this method is that it is difficult to impossible to distinguish if you are actually pulling your controller back/forwards or whether you are just tilting forwards  are an angle so that your controller has a smaller projected frontal area. Also, I’m not sure how useful the z-depth tracking will be in a real world setting because in order for it to work properly, the whole of your controlling object must be visible to your camera at all times or you will get jerky tracking. This arises from the fact that a camera has a limited field of view and as you move an object in the z-direction, you may inadvertently move out of the field of view. I will have to get the beta to test before I can judge how well the z-depth tracking works in their implementation.

They track the rotation of the blobs of colors to determine the angles. They do not appear to track the exact shape of the object because you can use objects of various shapes. Instead, they appear to track the longest axis of the colored blobs to determine the angle of orientation of your colored objects. In some videos, you can see a shorter axis that is perpendicular to the longest axis of the colored blobs. This is used to find the center of the blob. This center point appears to be what is used to calculate the position of your controller in the X-Y plane.

The program is most likely written in Microsoft Visual C++ from the screenshots I've seen, so don't hold your breath for a Linux or Mac version any time soon.  I’m deducing this from the program icons seen on http://www.camspace..../Emulation_Authoring. If you are a Microsoft Visual Studio user, you will recognize the familiar program icons in the top left of the windows. They most likely used Visual C++ to gain as much of a performance advantage as possible due to the inherently high cost of image processing in real time.

In the first three videos listed at the beginning of this post, you can see someone trying to use CamSpace to emulate a mouse. The first video shows that CamSpace can be made to work smoothly and allow you to move windows around by using a pinching motion of your fingers with colored plastic on them. However, there is a major hurdle that must be overcome before you can use a webcam as a general mouse. This major hurdle is that you must design a control scheme to signal to the computer when you want to select an object, move it, click, double-click, right-click, middle-click, scroll, and for those of you with extended mouse buttons, how to control the 4-th and 5th buttons. The implementation must allow you to move the mouse accurately while allowing you to execute the above commands. You will most likely have to use two hands to successfully emulate a standard mouse. One hand would be used for moving the mouse and the other would be used for signalling the other commands. It is difficult to do this with just one hand because as you move and shift your hand to create a different gesture, you will likely shift whatever it is on your hand that the program is tracking to position the mouse. This makes clicking on a button a frustrating affair let alone a text link.

The second video listed above shows some inherent problems of color based tracking methods. It shows that sometimes, the performance of this method can be quite slow as indicated by the less than 10 fps frame rate seen while trying to track three objects. I’m not sure whether this is due to the emulation used, lack of processing power, or a problem with the CamSpace program. Either way, if you are playing a game that already taxes your computer’s resources, you will have to upgrade to a better computer if you want decent tracking and a game frame rate. From coding WebCam Signature and playing around with other webcam tracking programs out on the web, I estimate that CamSpace will likely use around 20-30% of the processing time of a middle of the pack duo-core CPU, such as an AMD Athlon 64 X2. Ram usage should be fairly low, probably under 50 megs when running.

During the second video, you will notice that you can see the colored hand (being tracked) of the person but not the body of the person himself. At the end of the video, you can see the person come in front of the camera and his shirt (normally black) becomes white and is pixelated around the edges. This highlights the problem that the ability to track “any object” is in fact the ability to track “any object with a color that is not similar to colors seen in the background.” Similar colors will fool the program as well as shiny objects that are used for controllers. Changing light conditions or sunlight shining on your tracking object will also have adverse effects. Sunlight is bad in particular because it contains a good deal of near-infrared light that is not visible to the human eye but is picked up by many webcams. This creates noise in the colored blobs that affect the accuracy of tracking and may introduce jerky motions. Evidence of this can be seen by looking at the constantly changing pixelated edges of the colored blobs on the person’s hand in the second video.

From comments I have seen on CamSpace’s site and others, many people are calling CamSpace as a Wiimote killer and a gaming revolution. I believe these people are wrong. I do not believe you will be able to obtain tracking as accurate as you can with a Wiimote nor will it be anywhere as responsive. The Wiimote uses a specialized infrared camera coupled with dedicated hardware that allows it to outperform the vast majority of webcam/computer combinations out there. It is also much more resistant to lighting changes and is not affected by similar colors. The Wiimote just has to track two infrared LEDs that provide a much more predictable and reliable method of control than trying to emulate human vision. I also do not believe this will revolutionize gaming because there are already other virtual reality systems out there that allow you to track objects with greater accuracy than is possible with a color based system. Some of them are commercial systems with dedicated hardware, but some are open source like http://www.free-track.net/english/.

You are inherently unable to provide the accurate and fine tracking that a standard mouse can with a webcam system and current PC hardware because the vast majority of webcams have a maximum resolution of 640x480 (0.3 megapixels) whereas most monitors have a resolution that that is much greater, such as 1440x900 (1.3 megapixels) and up.  This means that the position of your mouse on the screen must be interpolated from a 0.3 megapixel image, not even considering the issue of processing power. Most likely, you will be using a resolution of 320x240 to enable your system to track at a high enough frame rate to prevent lag.

 Even if you used a camera with a resolution equal to that on your monitor, you will still not have a usable system because the processing requirements will be enormous. Think of trying to process a DVD movie frame by frame in real time and you will have an idea of the power needed to equal the resolution that can be provided by a standard mouse.

This is not to say the CamSpace is bad. In fact, it appears to be an excellent piece of software and it is about time that someone has made use of the webcam and make computer vision a useful thing. However, we must be aware of the inherent limitations of computer vision systems and not get carried away thinking that we are ready to throw away our mouse and Wiimote.

One thing that I hope CamSpace will do is release their software under an open source licence so that others can analyze the code and use it. While I was writing Webcam Signature, I did some extensive searching for example code that demonstrated computer vision in a useful manner. Something more useful than turning on a security camera. Well, the field is very sparse and most of the programs people are writing are proprietary and there is very LITTLE sharing going on. If computer vision is to advance to a useful state, we must share code so that new ideas can be readily implemented and old ones improved upon. This was the primary reason why I made Webcam Signature open source and I hope CamSpace will be open source as well.

Well, that is a long enough article for now. I’m sure there are some spelling mistakes and I’ll fix those in after a bit. I’ll write more if I have something to add.

Stay tuned for my second program that I’m submitting for the DC Programming Contest. It’s a good one.
Author of Circle Dock: http://circledock.wikidot.com
Author of Video In Picture: http://videoinpicture.wikidot.com
Author of Webcam Signature: http://webcamsignature.wikidot.com
Author of Easy Unicode Paster: http://easyunicodepaster.wikidot.com
« Last Edit: July 08, 2008, 07:40 AM by VideoInPicture »

vitalyb

  • Supporting Member
  • Joined in 2007
  • **
  • Posts: 143
    • View Profile
    • Donate to Member
Re: An Analysis of CamSpace as a Webcam Mouse and Gaming Controller
« Reply #1 on: July 08, 2008, 08:32 AM »
Good article. Some comments:

I've got my copy too and it is pretty responsive. A lot more than I expected. It has several issues:

1) You must be in a well-lit enviorment.
2) The overall GUI feel is a bit buggy.

However. With that said, the Soccer game (the one that is seen in the promotional video) plays great. You can actually bounce the virtual ball and it feels very responsive and AWESOME. They need to develop some better (physics based) games and it CAN be WII killer after all.

VideoInPicture

  • Honorary Member
  • Joined in 2008
  • **
  • Posts: 467
    • View Profile
    • Circle Dock
    • Donate to Member
Re: An Analysis of CamSpace as a Webcam Mouse and Gaming Controller
« Reply #2 on: July 08, 2008, 08:38 AM »
Good article. Some comments:

I've got my copy too and it is pretty responsive. A lot more than I expected. It has several issues:

1) You must be in a well-lit enviorment.
2) The overall GUI feel is a bit buggy.

However. With that said, the Soccer game (the one that is seen in the promotional video) plays great. You can actually bounce the virtual ball and it feels very responsive and AWESOME. They need to develop some better (physics based) games and it CAN be WII killer after all.

I still waiting for my beta invite even though I signed up a long time ago. Have you tried using it as a mouse? Also, what about the similar color issue?
Author of Circle Dock: http://circledock.wikidot.com
Author of Video In Picture: http://videoinpicture.wikidot.com
Author of Webcam Signature: http://webcamsignature.wikidot.com
Author of Easy Unicode Paster: http://easyunicodepaster.wikidot.com

vitalyb

  • Supporting Member
  • Joined in 2007
  • **
  • Posts: 143
    • View Profile
    • Donate to Member
Re: An Analysis of CamSpace as a Webcam Mouse and Gaming Controller
« Reply #3 on: July 08, 2008, 09:01 AM »
The betas are being sent slowly on purpose to handle issues that the current beta users report, so you'd have to be patient.

I tried to use the mouse pointer with it. It works, but it is hardly comfortable.

Similar colors do pose a problem however, in proper lighting it is a very minor one. CamTrax works by locking on a NEW object that it sees in a specified location. And the lock is a lot more complex than described in the first post - It locks on the form, on the color AND it keeps learning as you play with it. If you intentionally take the object out of the camera view it will try to lock on a different object with the same color, however, it hardly ever happens normally.

mouser

  • First Author
  • Administrator
  • Joined in 2005
  • *****
  • Posts: 40,896
    • View Profile
    • Mouser's Software Zone on DonationCoder.com
    • Read more about this member.
    • Donate to Member
Re: An Analysis of CamSpace as a Webcam Mouse and Gaming Controller
« Reply #4 on: July 08, 2008, 09:18 AM »
Thank you very much for posting your impressions of CamSpace -- really very interesting reading.  :up: :up: :up:

Nod5

  • Supporting Member
  • Joined in 2006
  • **
  • Posts: 1,169
    • View Profile
    • Donate to Member
Re: An Analysis of CamSpace as a Webcam Mouse and Gaming Controller
« Reply #5 on: July 08, 2008, 03:30 PM »
This analysis was a great read VideoInPicture!  :Thmbsup:

In all the linked youtube videos people have their webcam in the standard position above the screen. But maybe different placement would give better results? For example, for mouse simulation a webcam could be placed in front of the regular mouse position and the hand (with some marker on the fingers) could then occupy the regular mouse position. That way one hand could have mouse control while the other is at the keyboard, like with a real mouse.

A (quirky) thought I got when reading this: will it work with multiple webcams? In a driving game one cam could register a wheel object while another camera, on the floor, registers foot movement (acceleration/brake pedals)  :D


VideoInPicture

  • Honorary Member
  • Joined in 2008
  • **
  • Posts: 467
    • View Profile
    • Circle Dock
    • Donate to Member
Re: An Analysis of CamSpace as a Webcam Mouse and Gaming Controller
« Reply #6 on: July 08, 2008, 05:25 PM »
This analysis was a great read VideoInPicture!  :Thmbsup:

In all the linked youtube videos people have their webcam in the standard position above the screen. But maybe different placement would give better results? For example, for mouse simulation a webcam could be placed in front of the regular mouse position and the hand (with some marker on the fingers) could then occupy the regular mouse position. That way one hand could have mouse control while the other is at the keyboard, like with a real mouse.

A (quirky) thought I got when reading this: will it work with multiple webcams? In a driving game one cam could register a wheel object while another camera, on the floor, registers foot movement (acceleration/brake pedals)  :D


If you reposition the webcam to where the mouse usually is, I think you will lose the big benefit of the program, which is to provide a more natural input method and move away from traditional input mechanisms. You also have to be able to select objects and click with the mouse so you will likely need both hands to actuate the mouse. I haven't seen a viable input scheme from all the computer vision articles I have read that uses just one hand. To successfully implement a webcam mouse system with one hand (all the movements and clicks), you must resort to hand gestures which CamSpace is not able to analyze.

As for multiple webcams, that is not possible with the current version of CamSpace from all the wiki pages I have read and videos I have seem. From each webcam, you are only able to track a maximum of 4 objects so there are limitations. If you use multiple webcams, I think you would need a quad-core processor to handle the high processing load.
Author of Circle Dock: http://circledock.wikidot.com
Author of Video In Picture: http://videoinpicture.wikidot.com
Author of Webcam Signature: http://webcamsignature.wikidot.com
Author of Easy Unicode Paster: http://easyunicodepaster.wikidot.com
« Last Edit: July 08, 2008, 05:31 PM by VideoInPicture »

Nod5

  • Supporting Member
  • Joined in 2006
  • **
  • Posts: 1,169
    • View Profile
    • Donate to Member
Re: An Analysis of CamSpace as a Webcam Mouse and Gaming Controller
« Reply #7 on: July 09, 2008, 06:11 PM »
Yes, I can see the obstacles. The problem with CamSpace that you describe seem like a really tough design challenge for them to try to solve! I hope the devs come up with some smart way to at least lessen the problem.

If they used two or more webcams to record the hand/"mouse" from somewhat different angles, then that might solve much of the problem. But then they'd get the other problem with heavy system resource use.

This is off topic but the above problems got me thinking: do you know of some other promising new input tech that would be better at solving these problems? Are there some attempts at cheap 3D mice around? I've seen some devices using gyros. But how about a wireless mouse that instead uses multiple USB receivers positioned at different places in the room to somehow calculate the 3D position of the mouse? To my uninitiated ears it doesn't sound very expensive or impossible. But maybe there are large obstacles there too.

Nod5

  • Supporting Member
  • Joined in 2006
  • **
  • Posts: 1,169
    • View Profile
    • Donate to Member
Re: An Analysis of CamSpace as a Webcam Mouse and Gaming Controller
« Reply #8 on: October 10, 2008, 04:54 PM »
CamSpace Beta 7 (released September 17) is open for everyone. Has anyone tried it out? Impressions?
http://blog.camspace.com/

VideoInPicture

  • Honorary Member
  • Joined in 2008
  • **
  • Posts: 467
    • View Profile
    • Circle Dock
    • Donate to Member
Re: An Analysis of CamSpace as a Webcam Mouse and Gaming Controller
« Reply #9 on: October 10, 2008, 05:23 PM »
I was one of those in the closed betas and got to try Camspace out. It does work like what you see in the videos and is well implemented but you really need to have lots of lighting in the room and the object you choose should not be similar to the colors in your background or the program can track off course. I still don't think it's a threat for conventional gaming or the Wii because it's pretty hard to use it to replace hardware controllers, especially if you want to push a button or you have lots of intricate commands for the game. I've seen people try to use it to replace a mouse but it doesn't work very well because you can't emulate all the buttons and mouse movements easily and naturally. Perhaps if you build an operating system around Camspace it would work out better.
Author of Circle Dock: http://circledock.wikidot.com
Author of Video In Picture: http://videoinpicture.wikidot.com
Author of Webcam Signature: http://webcamsignature.wikidot.com
Author of Easy Unicode Paster: http://easyunicodepaster.wikidot.com

Nod5

  • Supporting Member
  • Joined in 2006
  • **
  • Posts: 1,169
    • View Profile
    • Donate to Member
Re: An Analysis of CamSpace as a Webcam Mouse and Gaming Controller
« Reply #10 on: October 13, 2008, 05:35 PM »
Ok, so no major changes compared to the previous versions then... I presently don't have a working webcam to test the CamSpace beta on but I'll try to get one soon.