title: track eye pupil in a video
body: <p>I am working on a project aimed to <strong>track eye pupil</strong>. For this I have made a head-mounted system that captures the images of the eye. Completed with the hardware portion I am <strong>struck in software</strong> part. I am using <strong>opencv</strong>. Please let me know what would be the most efficient way to track the pupil. <strong>Houghcircles</strong> didn't performing well. </p>
  
  <p>After that I have also tried with <strong>HSV</strong> filter and here is the code and 
  link to screenshot of the raw-image and processed one. Please help me to resolve this issue. The link also contains video of eye pupil that I am using in this code.</p>
  
  <p><a href="https://picasaweb.google.com/118169326982637604860/16November2011?authuser=0&amp;authkey=Gv1sRgCPKwwrGTyvX1Aw&amp;feat=directlink">https://picasaweb.google.com/118169326982637604860/16November2011?authuser=0&amp;authkey=Gv1sRgCPKwwrGTyvX1Aw&amp;feat=directlink</a></p>
  
  <p><strong>Code:</strong></p>
  
  <pre><code>include "cv.h"
  
  include"highgui.h"
  
  IplImage* GetThresholdedImage(IplImage* img)
  {
  
      IplImage *imgHSV=cvCreateImage(cvGetSize(img),8,3);
      cvCvtColor(img,imgHSV,CV_BGR2HSV);
      IplImage *imgThresh=cvCreateImage(cvGetSize(img),8,1);
      cvInRangeS(imgHSV,cvScalar(0, 84, 0, 0),cvScalar(179, 256, 11, 0),imgThresh);
      cvReleaseImage(&amp;imgHSV);
      return imgThresh;
  }
  
  void main(int *argv,char **argc)
  {
  
      IplImage *imgScribble= NULL;
      char c=0;
      CvCapture *capture;
      capture=cvCreateFileCapture("main.avi");
  
      if(!capture)
      {
          printf("Camera could not be initialized");
          exit(0);
      }
      cvNamedWindow("Simple");
      cvNamedWindow("Thresholded");
  
      while(c!=32)
      {
          IplImage *img=0;
          img=cvQueryFrame(capture);
          if(!img)
              break;
          if(imgScribble==NULL)
              imgScribble=cvCreateImage(cvGetSize(img),8,3);
  
          IplImage *timg=GetThresholdedImage(img);
          CvMoments *moments=(CvMoments*)malloc(sizeof(CvMoments));
          cvMoments(timg,moments,1);
  
          double moment10 = cvGetSpatialMoment(moments, 1, 0);
          double moment01 = cvGetSpatialMoment(moments, 0, 1);
          double area = cvGetCentralMoment(moments, 0, 0);
  
          static int posX = 0;
          static int posY = 0;
  
          int lastX = posX;
          int lastY = posY;
  
          posX = moment10/area;
          posY = moment01/area;
           // Print it out for debugging purposes
          printf("position (%d,%d)\n", posX, posY);
          // We want to draw a line only if its a valid position
          if(lastX&gt;0 &amp;&amp; lastY&gt;0 &amp;&amp; posX&gt;0 &amp;&amp; posY&gt;0)
          {
              // Draw a yellow line from the previous point to the current point
              cvLine(imgScribble, cvPoint(posX, posY), cvPoint(lastX, lastY), cvScalar(0,255,255), 5);
          }
          // Add the scribbling image and the frame...
  
          cvAdd(img, imgScribble, img);
  
          cvShowImage("Simple",img);
          cvShowImage("Thresholded",timg);
          c=cvWaitKey(3);
          cvReleaseImage(&amp;timg);
          delete moments;
  
      }
      //cvReleaseImage(&amp;img);
      cvDestroyWindow("Simple");
      cvDestroyWindow("Thresholded");
  
  }
  </code></pre>
  
  <hr>
  
  <p>I am able to track the eye and find the center coordinates of pupil precisely. </p>
  
  <p>First I thresholded the image taken by the head mounted camera. After that I have used contour finding algorithm then I find the centroid of all the contours. This gives me the center coordinates of eye pupil, this method is working fine in real time and also detecting eye blinking with very good accuracy. </p>
  
  <p>Now, my aim is to embed this feature into a game(a racing game). In which If I look to left/right then the car moves left/right and If I blink the car slows down. How could I proceed now??? Would I need a game engine to do that? </p>
  
  <p>I heard of some open source game engines compatible with visual studio 2010(unity etc.). Is it feasible??? If yes, how should I proceed ? </p>