Arduino from the head which allows slight motion.

Based Eye-Writer Using PS3 Eye Camera

                      Ms. Mrinmayi
Malaikar,  Ms. Madhumita Mankar, Mr.
Vikrant Mehendale, Mr. Parth Mitkari

R.S. Patil,

of Electronics and Telecommunication Engineering

Vidyapeeth College of Engineering, Belpada, Navi Mumbai



Abstract: Eye-Writer is an Eye Tracking device and a custom
software. It is an
application which will help them to draw or write and it is an
application of eye tracking. Eye Tracking is a sensor technology to know where
your eyes are focused.  Individuals suffering from paralysis cannot
move their body but they can still move their eyes. So Eye-Writer is without
using the rest of the body. The original Eye-Writer was meant for a motionless
head. The Eye-Writer which we have designed has a LED and camera mounted away
from the head which allows slight motion.


Eye-Writer is an ongoing, open source, collaborative research project that is
an attempt to provide people with a neuromuscular disorder and injuries with
creative technology. The Eye-Writer is an advantageous eye-tracking glasses and
custom software that allows artists and graffiti writers to draw with their
eyes alone, suffering from paralysis by Amyotrophic Lateral Sclerosis, as well
as other neuromuscular disorders and injuries

     The original design contained glasses, but we have a new design, which we have called “eyewriter 2.0”, which improves the accuracy of the device and allows people with a slight head movement to use an eye successor. The original eye machine, designed for a paralyzed Graffiti artist TEMPT1, is designed to be worn on a completely immobile head. The 2.0 design, where a camera and a LED system are mounted remotely from the head, can be used by people whose head moves slightly, such as MS patients and people wearing glasses, etc.


Eye tracking is the process of measuring either the gaze direction (what one looks at) or the movement of an eye with respect to the head. An eye-tracker is a device for measuring eye positions and eye movements. Most modern eye-trackers use near-infrared technology together with a high-resolution camera (or other optical sensor) to follow the direction of the gaze. The underlying concept, usually referred to as PCCR, means that the camera follows the pupil center and that it reflects light from the cornea.Keywords: PCCR pupil center corneal reflection



     Edmund Huey built an early eye
tracker, using a sort of contact lens with a hole for the pupil. The lens was connected to an
aluminium pointer that moved in response to the movement of the eye. Huey
studied and quantified regressions (only a small proportion of saccades are
regressions), and show that some words in a sentence are not fixated. 1

     The first non-intrusive eye trackers were
built by Guy Thomas Buswell in Chicago, using beams of light that were
reflected on the eye and then recording them on film. Buswell made systematic
studies into reading and picture
viewing. 3

     In the 1950s, Alfred L. Yarbus did important eye tracking research
and his 1967 book is very highly quoted. He showed the task given to a subject
has a very large influence on the subject’s eye movement. He also wrote about
the relation between fixations and interest. 5

     In the 1970s, eye tracking research
expanded rapidly, particularly reading research. A good overview of the
research in this period is given by Rayner. 1

     In 1980, Just and Carpenter formulated the
influential Strong eye-mind Hypothesis,
the hypothesis that “there is no appreciable lag between what is fixated
and what is processed”. If this hypothesis is correct, then when a subject
looks at a word or object, he or she also thinks about (process cognitively),
and for exactly as long as the recorded fixation. The hypothesis is often taken
for granted by beginning eye tracker researchers. 2

     During the 1980s, the eye-mind hypothesis
was often questioned in light of covert
attention, the attention to
something that one is not looking at, which people often do. If covert
attention is common during eye tracking recordings, the resulting scan path and
fixation patterns would often show not where our attention has been, but only
where the eye has been looking, and so eye tracking would not indicate
cognitive processing.

     According to Hoffman, current
consensus is that visual attention is always slightly (100 to 250 ms) ahead of
the eye. But as soon as attention moves to a new position, the eyes will want
to follow. 3

     We still cannot infer specific cognitive
processes directly from a fixation on a particular object in a scene.  For instance, a fixation on a face in
a picture may indicate recognition, liking, dislike, puzzlement etc. Therefore
eye tracking is often coupled with other methodologies, such as introspective
verbal protocols. 5





The term ‘eye tracking’ as used here means an estimation of the direction of the user. In most cases, estimating the direction of the can means identifying the object on which the can falls.     The accuracy of eye movement measurement is strongly dependent on a clear differentiation of the pupil and detection of corneal reflection. The visible spectrum probably generates uncontrolled mirror reflection, while illuminating the eye with infrared light that is not perceptible by the human eye, the delineation of the pupil and the iris makes an easy task while the light enters the pupil directly, it simply reflects from the iris. This means that a clear reflection is generated (with little noise) and can therefore be monitored with ease.     Near infrared light is directed towards the middle of the eyes (the pupils) causing visible reflections in the cornea (the most optical element of the eye), which are followed by a camera.


i) Size and dilation of the student     An increase in pupil size is called pupil dilation and a decrease in size is called pupillary constriction. The tablet size primarily responds to changes in light (ambient light) or stimulus material (for example, videostimulus).ii) Distance to the screenTogether with the pupil size, eye-trackers also measure the distance to the screen and the relative position of the respondent. Leaning forward or backward for a remote device is tracked immediately and avoids any approach.iii) Ocular deformationMost eye-trackers measure the positions of the left and right eyes independently of each other. This allows for the extraction of vergence, i.e. whether left and right eyes move together or apart. This phenomenon is only a natural consequence of focusing close and far away. Divergence often occurs when our mind drifts, if we lose our focus or concentration.iv) FlashesEye-tracking also provides essential information about cognitive workload by monitoring flashing. Cognitive demanding tasks can be associated with delays in flashing, the so-called attention flashing. However, many other insights can be derived from blinking. For example, a very low frequency of flashes is usually associated with higher levels of concentration. A fairly high frequency is indicative of drowsiness and a lower focus and concentration.



The working of the model is divided into three basic steps. First of all, create LED lighting devices for the sides of the screen and the center. From a technical point of view, the system works by having 3 IR illuminators blink every frame. On even frames it uses the central illuminator (placed around the camera lens) and on odd frames it uses the 2 side illuminators. On even frames, the pupil appears clear, because the IR light actually bounces against the back of your eye, such as the red-eye effect. Our pupil looks dark on odd frames. The difference between the two allows us to isolate and follow the student in real time. In addition, the glare (reflections of the IR illuminators) of the dark frame is maintained and this, plus the information about the pupil, is calibrated to screen the position using a least-squares fit process for a comparison that shows an image of shimmering / pupil offers a position to screen position.     In particular, an infrared light source illuminating the eye of a wearer can generate reflections of the wearer’s cornea with a relatively high intensity, also called ‘glints’. The glints can be displayed with an infrared camera. When the infrared light sources are illuminated at a relatively lower intensity, it is possible to determine the pupil location. Glints, in combination with the pupillary location, can be used to accurately determine the viewing direction and eye rotation. The particular gaze direction could be used in different eye tracking applications. By controlling the light sources to change the intensity levels and by combining multiple images of the eye to record multiple glistening locations with the pupillary location, eye-tracking can be performed with better accuracy and with fewer light sources.        Second, hacking the PS3 eye camera to get the vertical synchronization (when the video frame is taken) and to make it sensitive to IR.       Third, programming and building the arduino / circuit to control the blinking. Finally, setting the basis for the system and going through the basic principles of the softwar





B Software Results


i)    Using EyeWriter software – Catch Me    Look at the Catch box. As you tail, the color of the box turns green.When the box is completely green, it is caught and appears somewhere else. Continue to capture the boxes to test your eye tracking calibration



ii) Using Eye-writer software – TypingLook at which key you want to press. As you watch, the color of the keys turns green and blue. When the button flashes blue, it is pressed. You can see what you have typed at the top of the screen. 




i) The project
will inspire the formation of social networks, broadening the scope of research
into its potency not only as a tool for expression by drawing, but also in the
fields of medicine and welfare.

 ii) Anyone can build and use the Eye-Writer by
utilizing a combination of computers and other familiar instruments, and
installing software that is available for free. It is probably the most versatile
device for drawing with the eyes.



Thus Eye-Writer is a boon to the disabled people
suffering from neuromuscular diseases. It’s cost is low which is affordable
even to the common man. It
improves the accuracy of the device and allows people whose heads are moving
even slightly to use an Eye-Writer, unlike the previous version where eye
tracking glasses were used. It is not a bulky setup as the
person using Eye-Writer will not have to wear the glasses.



T. Baltrusaitis, P. Robinson, and L.-P. Morency. Constrained local neural
fields for robust facial landmark detection in the wild. In Com-puter Vision
Workshops (ICCVW),2013 IEEE InternationalConfer- ence on, pages 354–361. IEEE,

A.Vetrivel et al, International Journal of Computer Science and
Mobile   Computing, Vol.5 Issue.3, March-
2016, pg.     423-430

3  D. 
Beymer, D.  Russell and  P. 
Orton An eye tracking  study of
how font size and type influence online reading, Proceedings of the 22nd British
HCI Group Annual Conference on People and Computers: Culture, Creativity,
Interaction – Volume 2, Pages 15-18, 2008.

Z Hao, and Q Lei, “Vision-based Interface: Using Face and Eye Blinking Tracking
with  Camara”,  Second 
International Symposium on 
Intelligent Information Technology Applications 2008, pp 306-310.

N H Cuong, and H T Hoang, “Eye Gaze Detection with a Single WebCAM Based   on  
Geometry   Features   Extraction”,   2010  
11th     International Conference
on Control, Automation, Robotics and vision, Singapore, 7-10th December, 2010,
pp 2507-2512.

6 L  Lu,  Y  Yang,  L Wang,  and  B  Tang,
 on  Gray Projection”, 2009 Third International Symposium on Intelligent Information
Application, 2009, pp 58-60.

7   Z Hao, and Q Lei, “Vision-based Interface: Using Face and Eye Blinking
Tracking with  Camara”,  Second 
International Symposium on  Intelligent Information Technology Applications 2008, pp 306-310