REAL-TIME AFFECTIVE STATES IDENTIFICATION FOR HUMAN-ROBOT
INTERACTION
BY
ELLIANA ISMAIL
A dissertation submitted in fulfilment of the requirement for the degree of Master of Science in
Mechatronics Engineering
Kulliyyah of Engineering International Islamic University
Malaysia
APRIL 2013
ii
ABSTRACT
Under the context of using robot to perform rehabilitation on stroke patient, the robot must be able to understand the patient so to be able to interact with the patient more effectively. Current practise to rehabilitate stroke patient depends on the resource available that mostly involve trained human therapists. Progresses basically are monitored continuously in qualitative manner and the therapy session needs to be done in regular basis repetitively. However, the practice is costly and does not provide a quantitative way to measure the progress of the affected person.
A robot on the other hand can work precisely and continuously and able to record the progresses of a patient quantitatively. The therapy using a robotic system can be made more efficient when the physiological state of the affected muscle of the patient complemented with the affective state (psychological state) of the patient is known. For this research, the focus of the affective state is the engagement level of the patient when subjected to rehabilitation procedure of his upper limb (i.e.
moving his arm to follow specific trajectory). For evaluating engagement level, the electrooculogram (EOG) signal is captured when the patient is doing the therapy.
The signals are fed into fuzzy classifier to deduce the engagement level of the patient. In developing the fuzzy classifier, the related data is required to deduce the engagement level. A series of experiments are designed where the patients are asked to track a set of prescribed paths on the computer screen which has different level of difficulties within the allocated times and have to obey different speed constraints. The position error from the trajectory tracking is measured together with the electrooculogram (EOG) signal which is recorded by using a G-tec data acquisition system simultaneously. The information on the endogenous type of eye blinking is extracted from the electrooculogram (EOG) and it plays an important role to study the engagement level. Following the experiment, a series of questionnaires that has been carefully designed are given to the subjects to verify the engagement level deduced from the experiment done earlier by the subjects. A robotic platform is then used to verify the engagement level in real-time. The engagement model in the form of fuzzy classifier is used to adapt the speed of the robotic platform which is useful for the human-robot interaction. In particular, if the level of engagement is high, the subject is subjected to more challenging trajectory to be tracked. This is useful especially for the robot assisted type of stroke rehabilitation. The analysis on the questionnaire and the deduction of the level of engagement from the experimental results shows an accuracy of 95%. The robotic system is also able to adapt its speed whenever the level of engagement level changes. The research is only limited to one physiological signal namely the electrooculogram (EOG). Besides that, this research only consider onto one affective state which is the engagement.
iii
ثحبلا ةصلاخ
بجيو ،ةيغامدلا ةتكسلا نع ضيرملا ليھأت ةداعإ ذيفنتل توبورلا مادختساب قايس نمض عم لعافتلا ىلع ةرداق نوكت ىتح ضيرملا مھف ىلع ارداق نوكي نأ توبورلا ىلع ةيلاعف رثكأ وحن ىلع ضيرملا ةيغامدلا ةتكسلا ضيرملا ليھأت ةداعلإ ةيلاحلا ةسرامملا .
ا ةحاتملا دراوملا ىلع دمتعي ةبردملا ةيرشبلا نيجلاعملا بلاغلا يف لمشت يتل
اساسأ مدقت .
مظتنم ساسأ يف هب مايقلا بجي جلاعلا ةرودلا ةقيرطو ةيعون يف رمتسم دصر متيو رركتم مدقتلا سايقل يمك ةليسو رفوت لا و ةفلكم ةيلمعلا ةسرامملا نإف ،كلذ عمو .
باصملا صخشلل قدب لمعت ىرخأ ةيحان نم توبورلل نكميو .
رداقو رمتسم لكشبو ة
ةيمكلا ةيحانلا نم ضيرمل ةيلمع مدقت ليجست ىلع ماظن مادختساب جلاعلا ءارجإ نكميو .
ضيرملا نم ةباصملا تلاضعلل ةيجولويسفلا ةلاحلا فرعي امدنع ةيلاعف رثكأ ةيتوبورلا ةينادجولا ةلودلا عم لمكتست ةيسفنلا ةلاحلا )
ضيرملل ( لاح يف زيكرتلاو ،ثحبلا اذھل .
ة
ليھأت ةداعإ تاءارجلإ نوضرعتي امدنع ضيرملل ةكراشملا ىوتسم وھ ةينادجو هنم يولعلا فرطلا نيعم راسم ةعباتمل هعارذ كيرحت يأ )
،ةكراشملا ىوتسم مييقتل .(
طاقتلا متيو
electrooculogram . جلاعلا هب موقي ضيرملا نوكي امدنع ةراشإ ( EOG ) ماغ فنصملا ىلإ تاراشلإا ةيذغت متيو لبق نم ةكراشملا ىوتسم ىلع للادتسلال ض
ضيرملا للادتسلال ةلصلا تاذ تانايبلا نم بلطي ،ضماغ فنصملا ةيمانلا نادلبلا يف .
ةكراشملا ىوتسم ىلع بقعتل ىضرملا نم بلطي ثيح براجتلا نم ةلسلس ميمصت مت .
م فلتخم ىوتسم اھيدل يتلا رتويبمكلا ةشاش ىلع ةددحملا تاراسملا نم ةعومجم ن
ةفلتخم ةعرس دويقل عايصنلااو اھل صصخملا تقولا دودح يف تابوعصلا أطخ ساقي .
ةراشإ عم بنج ىلإ ابنج راسم عبتت نم فقوم )
EOG ( electrooculogram متي يتلا
تانايبلا ىلع لوصحلا مادختساب اھليجست G
- دحاو تقو يف ماظن TEC
تامولعملا .
تسي ضماو نيعلل ةيتاذلا عون ىلع ةدوجوملا نم جرخ
electrooculogram )
EOG ، (
ةكراشملا ىوتسم ةساردل اماھ ارود بعلي هنأو نم ةلسلسل ارظن ،ةبرجتلا هذھ دعب .
ةكراشملا ىوتسم نم ققحت يتلا عيضاوملا ىلع ةيانعب اھميمصت مت يتلا تانايبتسلاا تاعوضوملا نم قباس تقو يف هلمع ةبرجت نم هصلاختسا ةصنم مادختسا متي مث .
يللآا يقيقحلا تقولا يف ةكراشملا ىوتسم نم ققحتلل يف ةكراشملا جذومن مادختسا متي .
ةديفم يھ يتلا ةيتوبورلا ةصنم ةعرس عم فيكتلا ىلع فنصملا ضماغ لكش توبورلاو ناسنلإا نيب لعافتللال ةكراشملا ىوتسم ناك اذإ ،صوصخلا هجو ىلع .
عتل ايدحت رثكأ راسم ىلا عوضوملا اذھ عضخي ،ةيلاع اھبق
ةبسنلاب ةصاخو ديفم اذھو .
ةيغامدلا ةتكسلا ليھأت ةداعإ نم ةدعاسمب توبورلا عونلل مسحو نايبتسلاا ىلع ليلحتلا .
ةقد رھظي ةيبيرجتلا جئاتنلا نم ةكراشملا ىوتسم
٪ 95 ةرداق اضيأ يھ يللآا ماظنلا .
ةكراشملا ىوتسم تارييغتلا ىوتسم املك هتعرس عم فيكتلا ىلع طقف رصتقيو .
ىلع
يھو ةيجولويسفلا ةدحاو ةراشلإ ثوحبلا electrooculogram
) EOG . ( ىلا ةفاضلاابو
بلا اذھو ، كلذ . رظنلا لا وھ يذلا يفطاعلا ةدحاو ة لود ىلع طقف بتشا
iv
APPROVAL PAGE
I certified that I have supervised and read this study and that in my opinion, it conforms to acceptable standards of scholarly presentation and is fully adequate, in scope and quality, as a dissertation for the degree of Master of Science in Mechatronics Engineering.
………
Shahrul Naim Sidek Supervisor
………
Md. Raisuddin Khan Co-Supervisor
I certified that I have read this study and that in my opinion, it conforms to acceptable standards of scholarly presentation and is fully adequate, in scope and quality, as a dissertation for the degree of Master of Science in Mechatronics Engineering.
………..
Amir Akramin Shafie Internal Examiner
………
Norlaili Mat Safri External Examiner
This dissertation was submitted to the Department of Mechatronics Engineering and is accepted as a fulfilment of the requirement for the degree of Master of Science in Mechatronics Engineering.
...
Md. Raisuddin Khan Head, Department of Mechatronics Engineering This dissertation was submitted to the Kulliyyah of Engineering and is accepted as a fulfilment of the requirement for the degree of Master of Science in Mechatronics Engineering.
………
Md Noor Hj Salleh
Dean, Kulliyyah of Engineering
v
DECLARATION
I hereby declare that this dissertation is the result of my own investigations, except where otherwise stated. I also declare that it has not been previously or concurrently submitted as a whole for any other degrees at IIUM or other institutions.
Elliana Ismail
Signature:……… Date:………
vi
INTERNATIONAL ISLAMIC UNIVERSITY MALAYSIA
DECLARATION OF COPYRIGHT AND AFFIRMATION OF FAIR USE OF UNPUBLISHED RESEARCH
Copyright © 2013 by Elliana Ismail. All rights reserved
REAL-TIME AFFECTIVE STATES IDENTIFICATION FOR HUMAN- ROBOT INTERACTION
I hereby affirm that The International Islamic University Malaysia (IIUM) hold all rights in the copyright of this Work and henceforth any reproduction or use in any form or by means whatsoever is prohibited without the written consent of IIUM. No part of this unpublished researched may be reproduced, stored in a retrieval system, or transmitted, in any form or by means, electronics, mechanical, photocopying, recording or otherwise without prior written permission of the copyright holder.
Affirmed by Elliana Ismail
..……….. …...
Signature Date
vii To:
My beloved parents and my brother
viii
ACKNOWLEDGEMENTS
In the name of ALLAH the Almighty, most benevolent and most merciful.
In particular I wish to express my profound gratitude and appreciation to my supervisor, Assoc. Dr. Shahrul Na`im Sidek for his supervision, guidance and encouragement throughout the duration of study. Appreciation is also extended to my co supervisor, Assoc. Dr. Raisuddin Khan for sharing his knowledge and guidance.
A token of appreciation to my family for their prayer for my success, boundless love, understanding, patience, support and encouragement throughout pursuing my study.
Finally special thanks to all my postgraduate friends for useful advice and tips.
Thank you.
ix
TABLE OF CONTENTS
Abstract………... ii
Abstract in Arabic………... iii
Approval Page………... iv
Declaration Page………... v
Copyright Page……….... vi
Dedication………... vii
Acknowledgements………... viii
List of Tables………... xi
List of Figures………... xii
List of Symbols………... xiv
List of Abbreviations………... xv
CHAPTER 1: INTRODUCTION……….... 1
1.1 Background………... 1
1.2 Problem Statement……….... 2
1.3 Research Objective………... 3
1.4 Research Scope………... 4
1.5 Research Significance………... 5
1.6 Research Methodology………... 6
1.7 Thesis Organization………... 10
CHAPTER 2: LITERATURE REVIEW……… 11
2.1 Introduction………... 11
2.2 Electrooculogram (EOG) Signal………... 11
2.3 Psycho-Physiological Signal in Human-Robot Interaction…………... 14
2.4 Stroke Patients Rehabilitation………... 18
2.5 Summary………... 24
CHAPTER 3: SYSTEM DESCRIPTIONS……….... 26
3.1 Introduction………... 26
3.2 System Description: System Block Diagram……….... 27
3.3 Electrooculogram (EOG) Signal………... 27
3.4 G-tec Acquisition System………... 30
3.5 Trajectories and Position Data………... 33
3.6 Procedures for the Experiment………... 45
3.7 Fuzzy Classifier………... 47
3.8 Questionnaires………... 47
3.9 Discrete-Event System (DES)………... 48
x
3.10 Mechanical System of the Prototype Robot……….... 50
3.11 Summary………... 52
CHAPTER 4: AFFECTIVE STATES ANALYSIS………... 53
4.1 Introduction………... 53
4.2 Endogenous Eye Blinks Analysis………... 53
4.3 Position Data-Error Analysis……….... 56
4.4 Fuzzy Classifier………... 71
4.4.1 Membership Functions……….... 72
4.4.2 Fuzzification of Inputs……….... 73
4.4.3 Identifying Rules………... 76
4.4.4 Fuzzy Inference………... 79
4.5 Questionnaire Analysis………... 80
4.5.1 Frequency……….... 81
4.5.2 Level of Engagement………... 82
4.5.3 T-Test Analysis………... 85
4.6 Summary………... 86
CHAPTER 5: HUMAN-ROBOT INTERACTIONS………... 87
5.1 Introduction………... 87
5.2 Mechanical System of the Prototype Robot………... 87
5.3 Robot-assisted Rehabilitation………... 88
5.4 Discrete-Event System (DES) and Robotic Platform………... 89
5.5 Result and Analysis………... 92
5.6 Summary………... 101
CHAPTER 6: CONCLUSIONS AND RECOMMENDATIONS………. 103
6.1 Conclusions………... 103
6.2 Recommendations………... 105
BIBLIOGRAPHY………. 106
PUBLICATIONS……… 110
APPENDIX I: Questionnaires……… 111
APPENDIX II: Mechanical System of the Prototype Robot……….. 114
APPENDIX III: Matlab Programming………... 119
xi
LIST OF TABLES
Table No. Page No.
2.1 Simplified Literature review 21
3.1 Function of g-tec acquisition system 32
4.1 Average endogenous eye type blinks 56
4.2 Average error in computer screen pixels 58
4.3 Fuzzy Associate Memory 76
4.4 Age Frequency 82
4.5 Gender Frequency 82
4.6 Engagement level of the subjects 83
4.7 Factors that affect the level of engagement 85
4.8 Non-parametric correlations 86
5.1 Control state and definition 91
5.2 Control symbol and definition 91
5.3 Plant symbol and definition 92
xii
LIST OF FIGURES
Figure No. Page No.
1.1 Flowchart of Methodology 9
3.1 System block diagram 27
3.2 Position of electrodes 28
3.3 LIFETAB electrodes 29
3.4 Endogenous eye blinks 30
3.5 g. USBamp acquisition system 31
3.6 g. USBamp’s side view 31
3.7 G-tec acquisition system’s rear view 32 3.8 (a) - (w) Various Trajectories profile 33 3.9 Complete controlled experimental setup 46
3.10 Placement of the electrodes 46
3.11 Fuzzy classifier 47
3.12 General structure of Discrete-Event system (DES) controller 49
3.13 3D model of robotic platform 50
3.14 Assist-robot rehabilitation 51
4.1 Electrooculogram (EOG) signals 54
4.2 Endogenous eye blinks 55
4.3 (a)-(w) Trajectories profile 70
4.4 Average error, average eye blinks for all the subjects 71
xiii
4.5 Input-output mapping 71
4.6 Fuzzy logic model 72
4.7
(a) Average blinks, (b) Average Error, (c) Level of engagement
75
4.8 Three dimensional output surface of the modelled system 80
4.9 Average error, average eye blinks and level of engagement for
all the subjects 84
5.1 CAD model of robot-assisted rehabilitation system 88 5.2 Constraint-Induced Movement Therapy (CIMT) procedure 89 5.3 General structure of Discrete-Event system (DES) controller 90
5.4
(a) State profile, (b) Velocity profile, (c) Position profile for
experimental results 94
5.5
(a) State profile, (b) Velocity profile, (c) Position profile for
simulation results 97
5.6
Error generated from the robotic platform for simulation results
97
5.7
(a) State profile, (b) Velocity profile, (c) Position profile for
experimental results 99
5.8 (a) State profile, (b) Velocity profile, (c) Position profile for simulation results
101
5.9
Error generated from the robotic platform for simulation results
101
xiv
LIST OF SYMBOLS
Plant symbols
Control state
̃ Control symbol
MF(x) Membership function EX Average error in pixels
BX Average endogenous eye blinks LX Level of engagement
xv
LIST OF ABBREVIATIONS
EOG Electrooculogram
mRMR Minimum redundancy maximum relevance
SVM Support vector machine
ESD Eyelid’s State Detecting
SCGBP Scalded conjugate gradient back-propagation
SAR Socially assistive robot
EKG Electrocardiogram
EMG Electromyogram
DES Discrete-event system
AgCl Argentum Chloride
CAPI C language Application Programming Interface
SPSS Statistical Package for the Social Sciences
1
CHAPTER 1 INTRODUCTION
1.1 BACKGROUND
Affective computing is a mathematical tool to enable computing machine to recognize, express and respond intelligently to human emotions. Emotions are defined as disorganized responses, largely visceral, resulting from the lack of an effective adjustment (Picard, 2007). According to Picard (2007), emotions are regarded as non- scientific principles. Scientific principles are derived from rational thought, logical arguments, testable hypothesis, and repeatable experiments.
Measures used in affective state assessment could be categorized into different groups according to different criteria such as verbal, non-verbal intrusive and non- intrusive as reported by Xiangyang (2002). Affective state could be extracted from psycho physiological measures such as surface electromyography (SEMG) signal, skin conductance, body temperature and other types of biofeedback namely the heart rate and respiration.
The word 'Affect' is derived from the Latin word, affectus which could be defined as a grouping of physical phenomena in the form of emotions, feelings, or passions followed by impressions of pleasure, satisfaction, liking, joy or even sorrow.
Since affect is a natural and social part of human communication, people naturally express it when they interact with the machines.
Affect recognitions are important for communication, and can be considered as one of the greatest physiological needs of people (Buck, 1984). Therefore, machine
2
that is capable to recognize affective states and synthesize a proper response would be very useful for human-robot interaction.
Human-robot interaction is a field of study to understand, to design, and to evaluate the robotic systems for human use (Micheal and Alan, 2007). The interaction between human -robot can be categorized as follows:
i. Remote interaction- the human and robots are separated spatially.
ii. Proximate interaction-the human and robot are collocated.
Robots are sent for space exploration, underground mining and other hazardous work that human might not be able to do. The latter could be found in the field of rehabilitation, where robot is used to complement therapist to conduct therapy sessions.
Therefore, this research focused on collecting the psycho-physiological signal namely the electrooculogram (EOG) that is useful in constructing fuzzy classifier in order to deduce the level of engagement. The output of fuzzy classifier which is the level of engagement is applied to a robotic system that has linear motion featured control.
1.2 PROBLEM STATEMENT
Under the context of using robot to perform rehabilitation on stroke patient, the robot must be able to understand the patient so that the robot is able to interact with the patient intelligently. Rehabilitation for stroke patient involves three major means namely physical therapy, occupational therapy, and vision therapy. Physical therapy aids the patients to recover physical function such as walking, running, and standing.
Occupational therapy involves managing and practising the daily skills needed for the patients includes eating, cleaning up, dressing, and going to toilet. Stroke patients
3
normally will have visual problems due brain damage. Optometrists are tasked to recommend proper therapy depending on patients’ visual damages. In order to rehabilitate stroke patients, it involves a team of physicians, nurses, speech therapists, mental-health professionals and the optometrists.
The current practice to rehabilitate the stroke patients mostly depends on the human trained therapists. Their progresses basically are monitored continuously in qualitative manner and therapy session needs to be done in regular basis repetitively.
However, it is costly while not providing a quantitative way to measure the progress of the affected person. On top of that, therapists might provide different opinion on the assessment being done for each of the therapy sessions and prone to fatigue.
Robot on the other hand can work precisely, continuously and able to measure the progress of patients quantitatively. Thus using robot in the therapy session could complement the therapist. The implicit communication using a robotic system can be made more efficient when the affective states (psychological) such as engagement, stress level is known besides knowing the physiological states.
Affective states need to be measured in real-time so that the robot could interpret the human affective states and respond intelligently accordingly.
1.3 RESEARCH OBJECTIVES
The objective of this research is to develop a system that can detect in real-time from an array of psycho-physiological signals, the affective states such as engagement and anxiety of a person so as to embed the information in human-robot applications. In particular the objectives of this research are:
(1) To extract and analyze the psycho physiological signals using statistical methods through data acquisition system (i.e.: G-tec, MATLAB).
4
(2) To classify the affective states using appropriate learning methods and validate the states by questionnaires.
(3) To embed the affect-recognizer in a robot’s existing functionality so that it responds to the affective states accordingly and to study the system performance.
1.4 RESEARCH SCOPE
In this research, psycho-physiological signal namely the electrooculogram (EOG) is collected using the G-tec acquisition system through a controlled designed experimental setup. The endogenous eye blinks is extracted from the electrooculogram (EOG) signal which is collected through the experiment. Besides the endogenous eye blinks, the position data collected would provide error in terms of computer screen pixels which is useful in order to deduce the level of subject’s engagement. The data from the experiment; the average endogenous eye blinks and the average error are used to build the model of engagement level under fuzzy classifier framework. The inputs of the fuzzy classifier are the average endogenous eye blinks and the average error while the output of the fuzzy classifier is the level of subject’s engagement. The output of the fuzzy classifier which is applied to a robotic platform model with linear motion control featured is useful for rehabilitation applications. In this experiment, the data is collected from eighteen healthy subjects aged from 20 to 40 years old. The deduction of affective state is done in real-time in this research refers to the level of engagement being deduced after each rehabilitation process’s cycle. The definition of a cycle is when the subject has completed the desired trajectory from the linear motion featured control robotic platform.
5 1.5 RESEARCH SIGNIFICANCE
Malaysia reported 10000 new stroke cases every year (The Star, 2005). The common treatment used for stroke patients are by undergoing neurological physiotherapy.
Neurological physiotherapy is a treatment for patients with neurological disorder which is related to brain, spiral, cord, or the nerves. The neurological physiotherapy is a one to one treatment between the stroke patients and the occupational physiotherapists. The occupational physiotherapists tuned the stroke patients affected physical function into a specific movements repetitively that will eventually activate the brain and the central nervous system and this will improve the function of the affected parts.
The digital technology is growing rapidly along with the signal processing techniques. These enhance the creation of new advance devices such as g-tec acquisition system and the biopac that is able to aid the rehabilitation processes. In this research, the data is collected using the g-tec acquisition system in order to deduce the affective state which is the level of engagement for the patients. It is believed that when the affective state of the patient is known, the recovery rate for the patient is faster. Robotic rehabilitation on the other hand will reduce the functions of the occupational physiotherapists. The robotic rehabilitation is able to monitor the progresses continuously, precisely and quantitatively. Therefore, the neurological physiotherapy can be made more efficient when the psycho physiological state of a person is known along with the aid of the robotic rehabilitation.
In the suggested system proposed, during the rehabilitation procedures using the robotic platform, the patient is connected to the g-tec acquisition system in order to collect the data for the endogenous eye blinks. The robotic platform is visualized as the path tracking where the patient will need to track it accordingly. The failure to
6
track the path generates the error in pixels. These important data; average endogenous eye blinks and average error in pixels are then fed to a fuzzy logic classifier that will deduce the level of engagement of the patients. The level of engagement will activate the speed acquired accordingly depending on the needs of the patients. If the patients are not engaged in the rehabilitation procedures, the speed will reduce in order to gain back the patients’ engagement level. On the other hand, the speed will increase along with the increment of the level of engagement of the patients.
1.6 RESEARCH METHODOLOGY
First Stage: Preliminary Understanding of the Research Topic
The benchmark for this research is set according to the previous researches done by other researchers. In Malaysia, the organization that is in charge of that rehabilitation for stroke patients is the National stroke Association of Malaysia (nasam). The practise to treat stroke patient is basically undergoing the neurological physiotherapy.
The occupational therapist is in charge to re train the stroke patients’ brain in order to allow the affected physical to function as normal. The neurological physiotherapy needed to be done repetitively until the patients gain the functional affected physical.
The previous researches done only concentrate on collecting the signals that taken from human body such as the electromyogram(EMG), Electroencephalography (EEG), and electrocardiography(ECG). In this research, we are concentrating on collecting different human body signal which is the electrooculogram (EOG) that generated from the frontal area of human eyes. Since the EOG signal is related to the movement of the eyes, therefore the affective state that will be extracted from the EOG signal is the engagement level of the patients. The engagement level is used to
7
monitor the progress of the patients upon completing the robotic rehabilitation process.
Second Stage: Theoretical Framework
In the second stage, an inclusive review of the topic research will be carried out.
General information on the affective computing, affective states, and physiological signals will be reviewed in hope to achieve the goal of the research. At this stage, the process will be directed towards the theoretical aspects of the study to acquire the precise understanding of the issues and problems involved. Books, articles, newspaper cuttings, magazines and the internet are among important sources of materials.
Published data or written reports are very useful references for better understanding of the problems. These studies will be referring to various written works by people in their respective fields which relate to the topic of the research in order to assist deeper understanding of the subject.
Third Stage: Design of Experimental Setup
The data collected are mainly secondary data. The data are collected via experiments where the physiological signal is collected and recorded. Besides that, a series of questionnaires are given to the patients that are rated based on a 5 Likert scales.
Fourth Stage: Data Extraction Analysis
Data that were collected are analyzed using the appropriate learning methods available along with the analysis of the questionnaires.
Fifth Stage: Recommendation Design
This stage entails the recommended design for the system which is the design architecture for the fuzzy classifier and the discrete-event system
8
Sixth Stage: Implementation of Engagement Model to a Robotic Platform
In this stage, the affective-recognizer is embedded in a robot’s existing functionality in order to study the performance of the recommended system.
9
Figure 1.1: Flowchart of Methodology
Third Stage YES
NO YES
Fourth Stage
NO
YES YES
YES
Simulation Preliminary Stage
NO
NO Data Collection
Endogenous Eye blinks
Tracking Error
Data Analysis
Questionnaires Analysis Second Stage
Literature Review Identifying issues and
problems
Formulation goals and objectives
Human-robot interaction
Design Architecture
Discrete-even system Fuzzy logic
classifier
Sixth Stage Fifth Stage
END DES controller
