Starburst 1.0.1 – requirements - you can download it here
*Note that this version is not the real-time tracking version, that is currently linux only.

- A computer that is capable of running matlab (note that this was run on a P4 3.2Ghz with 2gigs of ram and processing a 3min video took over 50min to process)
- Windows XP 32bit (untested with 64bit or vista)
- Matlab for windows (this version was built with version 7.6.0.324)
- ffmpeg for windows (this version was built with rev12665 which is included in this release)
- a video file of the eye which is to be tracked (from a camera fixed to the head via glasses or something)
- a video file of what the head is pointing at (also fixed via glasses) (see hardware section for an example)
- a program that tells you the exact time position of the videos (I used virtual dub)

Video file requirements and tips

- Any format that ffmpeg can read will be compatible with this project. I used mpeg
- I used a camera flash to start a synchronization point that would be visible in both video streams.
- The two streams must be the same frame rate. By default, matlab will re-compose the final video at 29.97 frames per second, so if you have the option of recording at this rate, great.
- A minimum resolution of 640x480 is recommended.
- The eye video will be converted to B/W so it is ok to use a B/W camera if available (generally cheaper and higher quality)
- The camera that is tracking the eye should produce a clear, in-focus, zoomed in picture of the eye. If it is too dark, the algorithm will run very slow and inaccurate.
- Ideally, the camera viewing what the head is pointing at (scene) should have a wide angle, as wide as possible. If you end up with a narrow view (like the one I made), you may want to mount laser pointers to allow the user to know what the camera limitations are. (when the eye leaves the boundaries of the scene camera, eye tracking stops working)

Instructions

- While recording your eye tracking videos, be sure to flash the user with a camera flash or equivalent strobe to allow a spot where you can see where to sync the videos. (this is because it is not reasonable to assume that we can start recording from the 2 cameras at exactly the same time)
- You will also need to calibrate the eye tracker every time you record a video (re-using calibration data has not been tested). You can do this by setting up a grid of 3x3 points, and look at them from left to right, top to bottom successively. An example calibration image has been provided as calib.bmp and you can see this calibration in my example video which can be downloaded. It is a good idea to visibly indicate when you are starting the calibration, (such as pointing with your hand or blinking with your eye, or both) this will be useful later.

- After you have recorded the 2 video files, place them in the same directory as ffmpeg and the matlab (.m) script files. (with some more code modification, you may be able to use any directory, but for now, only the root directory works) I have recombined the jpg files that were extracted from the original prototype videos, you can download them here and use them to practice with.

- Use a program like virtual dub to find the exact time (in seconds) of the sync flash in each of the videos. (the example videos have already been synced, but you should understand this synchronization concept as it is important)

- Open any of the .m files in matlab (this will let matlab know where the working directory is)
- In the command entry area of matlab, enter:
extract_images_from_video
you will be prompted asking for the 2 videos you just created.
- you will then be asked for the time (in seconds) of the point of sync of each video
- matlab will then execute ffmpeg to extract the frames from each of the videos.

- if you have had no errors, then you are ready for the next part
extract_calibration_from_video
- you will be prompted for a directory, choose the working directory where all the files are. this is also where the calibration file will be saved as calibration.mat (.mat file is matlab variable file)

- matlab will then ask for user input regarding the frame where calibration starts, to keep things simple, this number should be a factor of 5. (you can see each frame of the scene video in the scene directory (this was created while extracting the images from the video)

- You will then be prompted for a .jpg file, this is to tell matlab when to stop processing the images. After checking in both the eye and scene subdirectories, you should know which one has the lesser number of frames. Select the last frame of either scene or eye jpg files so matlab knows where to stop. (if you wish to stop at an earlier frame, that will be ok too)

- a window will popup with instructions with both eye and scene images. “button” refers to mouse clicks; meaning, left button = left mouse click, right button = right mouse click.
(if your mouse does not have a middle button, you might have to edit the .m file for some other form of user input)
- **be sure to read and follow the instructions carefully as they are counter-intuitive.
- First find which frame has the eye looking at the first calibration point, then proceed from there.
- after you have completed 9 calibration points, the cross hairs should disappear, and if there was no errors (red text in the matlab main window) then this step is complete.

- Finally, enter this into the matlab main window
calculate_limbus_and_gaze

- Again, you will be prompted for the working directory (which contains the calibration.mat file generated in the previous step)

- You will also be asked for a frame number to indication when to start the process, during calibration is a good time, but if you want to omit this part of the video out then enter a later frame number.

- Again, you will be asked for the .jpg file which is the last frame that will be processed. You can enter an earlier frame, but do not exceed the lesser number of either eye or scene frames.

- A window will popup asking to verify the calibration data is valid, follow the instructions carefully (again they are counter-intuitive).

- The final step is another popup window where you need to click the center of the eye.
If matlab starts processing and you see incrementing (frames) numbers on the main window, then it is working. This part may take a while depending on your computer system and the length of the video being processed.
You can preview the output by looking in Result_Small_Eye directory where the processed images are being outputted as .jpg files (they will automatically be formatted to a video upon completion)

- the final output will be small_eye_result.mpg located in the working directory. ellipse_result.dat will also be generated which contains the tracking data in a text format. You can open this file in wordpad or any spreadsheet. (a .mat file is also generated which is a matlab variable file containing the same data as the .dat file)