• Tiada Hasil Ditemukan

Evaluation model for usability practitioners: a mobile application for low vision users

N/A
N/A
Protected

Academic year: 2023

Share "Evaluation model for usability practitioners: a mobile application for low vision users"

Copied!
74
0
0

Tekspenuh

(1)

The copyright © of this thesis belongs to its rightful author and/or other copyright owner. Copies can be accessed and downloaded for non-commercial or learning purposes without any charge and permission. The thesis cannot be reproduced or quoted as a whole without the permission from its rightful owner. No alteration or changes in format is allowed without permission from its rightful owner.

(2)

EVALUATION MODEL FOR USABILITY PRACTITIONERS: A MOBILE APPLICATION FOR LOW VISION USERS

AHLAM MOHAMED OMAR ELMGHIRBI

DOCTOR OF PHILOSOPHY UNIVERSITI UTARA MALAYSIA

2022

(3)
(4)

Permission to Use

In presenting this thesis in fulfilment of the requirements for a postgraduate degree from Universiti Utara Malaysia, I agree that the Universiti Library may make it freely available for inspection. I further agree that permission for the copying of this thesis in any manner, in whole or in part, for scholarly purpose may be granted by my supervisor(s) or, in their absence, by the Dean of Awang Had Salleh Graduate School of Arts and Sciences. It is understood that any copying or publication or use of this thesis or parts thereof for financial gain shall not be allowed without my written permission. It is also understood that due recognition shall be given to me and to Universiti Utara Malaysia for any scholarly use which may be made of any material from my thesis.

Requests for permission to copy or to make other use of materials in this thesis, in whole or in part, should be addressed to:

Dean of Awang Had Salleh Graduate School of Arts and Sciences UUM College of Arts and Sciences

Universiti Utara Malaysia 06010 UUM Sintok

(5)

Abstrak

Ramai orang bergantung pada aplikasi mudah alih untuk melakukan aktiviti harian.

Walau bagaimanapun, orang yang mempunyai kurang daya penglihatan yang tidak dapat membetulkan penglihatan mereka sepenuhnya dengan cermin mata memerlukan aplikasi yang boleh digunakan yang sepadan dengan kebolehan mereka. Malangnya, model penilaian kebolehgunaan sedia ada menggunakan ukuran umum yang tidak berkesan untuk menilai aplikasi mudah alih bagi memastikan kesesuaiannya untuk pengguna kurang daya penglihatan. Oleh itu, objektif utama kajian ini adalah untuk membangunkan model penilaian kebolehgunaan aplikasi mudah alih untuk pengguna kurang daya penglihatan. Fasa pertama dan kedua kajian adalah untuk mengenal pasti keperluan kebolehgunaan dan ukuran aplikasi mudah alih bagi pengguna kurang daya penglihatan. Kaedah yang digunakan ialah menganalisis kandungan literatur menggunakan kaedah Kajian Literatur Sistematik, dan menemu bual sepuluh pengguna aplikasi mudah alih penglihatan rendah. Perisian analisis kualitatif telah digunakan untuk menganalisis data temu bual separa berstruktur. Dalam fasa ketiga, model baru telah dibangunkan menggunakan kaedah Quality in Use Integrated Measurement. Model yang dicadangkan kemudiannya disemak menggunakan pendekatan kajian pakar oleh enam pakar pengetahuan dan pengamal untuk penambahbaikan dalam model akhir. Akhirnya, model yang dibangunkan telah dinilai dalam sesi kumpulan fokus dengan enam pengamal, dan juga ujian kebolehgunaan dengan sembilan pengguna penglihatan rendah yang dianalisis menggunakan statistik deskriptif. Kajian ini menghasilkan suatu model yang mengandungi enam dimensi kebolehgunaan, lima belas kriteria kebolehgunaan dan lima puluh enam metrik kebolehgunaan. Model ini boleh digunakan untuk mengenal pasti kesukaran pengguna kurang daya penglihatan, dan mencadangkan penyelesaian tambah baik kepada rekabentuk aplikasi mudah alih. Model ini memperkayakan badan pengetahuan dalam bidang Interaksi Manusia-Komputer, terutamanya dalam bidang penilaian kebolehgunaan. Ia membantu pengamal kebolehgunaan menemui masalah penggunaan aplikasi mudah alih yang dihadapi oleh pengguna kurang daya penglihatan, yang sukar ditemui oleh model terdahulu, dan mencadangkan penyelesaian tambah baik ke atas rekabentuknya. Ini memastikan kemudahan dan privasi untuk pengguna kurang daya penglihatan apabila menggunakan aplikasi mudah alih.

Kata kunci: Model penilaian kebolehgunaan, Aplikasi mudah alih, Orang kurang daya penglihatan.

(6)

Abstract

Many people depend on mobile applications to perform daily activities. However, people with low vision that unable to fully correct their vision with glasses require usable applications that match their abilities. Unfortunately, existing usability evaluation models use generalized measurements that are ineffective to evaluate mobile applications to ensure their suitability for low vision people. Therefore, the main objective of this study is to develop a mobile application usability evaluation model for low vision users. The first and second phases of this study were to identify the usability requirements and measures of mobile application for low vision users.

The methods used were analysing literature content using Systematic Literature Review method and interviewing ten low vision mobile application users. A qualitative analysis software was used to analyse semi-structured interview data. In the third phase, a new model was developed using the Quality in Use Integrated Measurement method. The proposed model was then reviewed using the expert review approach by six knowledge and practitioners for improvements in the final model.

Finally, the developed model was evaluated in a focus group session with six practitioners, and also usability testing with nine low vision users which was analysed by using descriptive statistical. This study has developed a model that includes six usability dimensions, fifteen usability criteria and fifty-six usability metrics. The model is able to identify low vision users’ difficulties and suggest a solution to improvise the mobile application design. The model may enrich the body of knowledge in the Human-Computer Interaction area, especially in the field of usability evaluation. It helps usability practitioners to discover mobile application usage problems faced by low vision users, which are difficult to discover by previous models and suggest a solution to improvise the design. This ensures convenience and privacy for low vision users when using a mobile application.

Keywords: Usability evaluation model, Mobile application, Low vision people.

(7)

Acknowledgment

First of all, I am grateful to Allah Almighty for the blessings bestowed upon me and for giving me the strength and health to complete this study. Without the blessings of Allah Almighty, it would not have been possible to continue my life powerfully and successfully.

I would like to thank my thesis supervisor Prof. Ts. Dr. Azham Hussain and Dr. Nur Hani Zulkifli Abai for their excellent guidance, care, patience, encouragement and sharing of all research experiences during my study. I also appreciate the valuable contributions of the chairperson and members of my viva voce panel, namely Prof. Dr.

Osman Ghazali (chairperson); Assoc. Prof. Ts. Dr. Ahmad Naim Che Pee (external examiner); Assoc. Prof. Dr. Mazida Ahmad (internal examiner). My thanks to Dr.

Nasiruddin Haron (Viva Unit), all the lecturers and staff of the School of Computing and Awang Had Salleh College of Arts & Sciences at UUM University.

My special thanks go to my parents who raised me with love, courage, attention and support. I am also very grateful to my husband Ashraf Amar Treish and our lovely children, late son Ahmad, Ainour, Raghad, younger son Ahmad and Omysah, the youngest. They have supported me despite my husband's health condition. I am very grateful to my husband for his strength and patience in bearing all the pressure.

I would also like to thank all members of my family and my husband's family, relatives, friends and colleagues in Libya and Malaysia, and many others too numerous to mention.

(8)

Table of Contents

Permission to Use ... i

Abstrak ………...………. ii

Abstract ……….…. iii

Acknowledgment ... iv

Table of Contents ... v

List of Tables... x

List of Figures ... xii

List of Appendices ... xiv

CHAPTER ONE INTRODACTION ... 1

1.1. Overview... 1

1.2. Background ... 1

1.3. Problem Statement ... 4

1.4. Research Questions ... 7

1.5. Research Objectives... 8

1.6. Research Scope ... 8

1.7. Significance of the Research ... 9

1.8. Relationship of Research ... 10

1.9. Thesis Organization ... 11

1.10. Summary ... 13

CHAPTER TWO LITERATURE REVIEW ... 14

2.1. Introduction... 14

2.2. Human Computer Interaction ... 14

2.3. Usability and Human Computer Interaction ... 15

2.4. Usability Evaluation ... 16

2.5. General Usability Evaluation Models and ISO Standards ... 16

2.5.1. Nielsen Model ... 17

2.5.2. MUSiC Model ... 18

2.5.3. Enhanced Usability Model ... 19

2.5.4. Integrated Model for Software Usability ... 20

2.5.5. CPUM Model ... 20

2.5.6. Usability Dimensions in ISO 9241-11 (1998) ... 22

(9)

2.6. Usability Evaluation for Mobile Applications ... 26

2.6.1. PACMAD Model (2013) ... 26

2.6.2. Mobile Goal Question Metric (mGQM) Model ... 28

2.6.3. Usability Evaluation Model for Mobile Applications ... 30

2.6.4. A Framework for Software Usability and User Experience Measurement in Mobile Industry ... 32

2.6.5. Extended PACMAD Usability Model ... 32

2.6.6. (USB-CAT) Catalog ... 34

2.6.7. Usability Evaluation Model for Mobile Banking Applications Interface … ... 35

2.6.8. Framework for Evaluating Mobile Applications for Health Education 36 2.6.9. W3C/WAI Guidelines Apply to Mobile ... 36

2.7. Usability Evaluation for Disabled ... 39

2.8. Visually impaired... 46

2.8.1. Mobile Applications Services for Visually Impaired ... 47

2.8.2. Challenges and Usability Requirements of Low Vision ... 50

2.8.3. Mobile Application Usability Evaluation for Visually Impaired ... 56

2.9. Current Usability Measures ... 60

2.9.1. Identification of Usability Dimensions ... 60

2.9.2. Identification of Usability Criteria ... 63

2.9.3. Identification of Usability Metrics ... 66

2.10. Supporting Theories to Usability Evaluation Researches ... 67

2.11. Technique in Constructing Metrics ... 74

2.12. Applying QUIM Method to Generate Proposed Model ... 76

2.13. Summary ... 77

CHAPTER THREE RESEARCH METHODOLOGY ... 78

3.1. Introduction... 78

3.2. Research Design ... 78

3.3. Phase 1: Gathering Requirements ... 80

3.3.1. Analysis the Literature Contents ... 80

3.3.2. Conducting Interviews ... 81

3.4. Phase 2: Theoretical Study ... 83

3.4.1. Planning the Review ... 85

(10)

3.4.3. Reporting the Review ... 88

3.5. Phase 3: Model Development ... 89

3.5.1. Developing Model ... 89

3.5.2. Verification Model ... 90

3.5.3. Model Improvement ... 92

3.6. Phase 4: Model Evaluation ... 92

3.6.1. Domain Practitioner Evaluation ... 94

3.6.2. Usability Testing ... 96

3.7. Summary ... 100

CHAPTER FOUR ANALYSIS OF MOBILE APPLICATION USAGE AMONG LOW VISION PEOPLE ... 102

4.1. Introduction... 102

4.2. Empirical Study Objective ... 102

4.3. Respondents ... 103

4.4. Qualitative Data Analysis ... 104

i. Identifying Codes ... 104

ii. Codes Classification ... 105

4.5. Empirical Study Findings ... 106

4.5.1. Interview Findings ... 107

4.5.2. Matching Interview Findings and Analysis of the Literature Contents124 4.5.3. Summarizing Findings ... 126

4.6. Mapping Interview Findings with SLR Findings ... 127

4.7. Summary ... 132

CHAPTER FIVE MODEL DEVELOPMENT ... 133

5.1. Introduction... 133

5.2. Effective Factors on Usability Measures Formation ... 133

5.3. Usability Dimensions for the Proposed Model ... 136

5.4. Usability Criteria ... 144

5.5. Extracting Usability Metrics ... 150

5.5.1. Efficiency ... 150

5.5.2. Effectiveness ... 154

5.5.3. Satisfaction ... 158

5.5.4. Low Vision Readability ... 164

(11)

5.5.6. Accessibility ... 174

5.6. Model Verification through Expert Review ... 182

5.7. The Proposed Model Improvement ... 192

5.7.1. Descriptions of Added and Modified Metrics ... 197

5.8. Summary ... 205

CHAPTER SIX MODEL EVALUATION ... 206

6.1. Introduction... 206

6.2. Evaluation through Focus Group ... 206

6.2.1. Focus Group Objective ... 207

6.2.2. Participants Identification ... 207

6.2.3. Meeting Schedule for Focus Group ... 208

6.2.4. Material Preparation for Focus Group ... 209

6.2.5. Conducting Focus Group ... 209

6.2.6. Focus Group Evaluation Findings ... 210

6.3. Evaluation through Usability Testing ... 215

6.3.1. Usability Testing Objective ... 216

6.3.2. Identification of Participants for Usability Testing ... 216

6.3.3. Meeting Scheduling for Usability Testing ... 217

6.3.4. Material and Tasks Preparation for Usability Testing ... 217

6.3.5. Conducting Usability Testing ... 217

6.3.6. Usability Testing Evaluation Findings ... 218

6.4. Discussion on Validated Metrics ... 238

6.4.1. Proposed Objective Metrics ... 238

6.4.2. Proposed Subjective Metrics ... 239

6.5. Summary ... 242

CHAPTER SEVEN DISCUSSION AND CONCLUSION ... 244

7.1. Introduction... 244

7.2. Discussion of the Findings... 244

7.2.1. Discussion on Identifying the Low Vision Users’ Usability Requirements ... 245

7.2.2. Discussion on Identifying Current Mobile Application Usability Measures for Low Vision ... 245

7.2.3. Discussion on Developing the New Model for Low Vision Users... 246

(12)

7.2.4. Discussion on Evaluating the Developed Model for Low Vision

People… ... 247

7.3. Contributions ... 249

7.3.1. A Mobile Application Usability Evaluation Model for Low Vision Users. ... 249

7.3.2. A Mobile Application Usability Measures for Low Vision Users ... 251

7.3.3. A Mobile Application Usability Metrics for Low Vision Users ... 252

7.3.4. Guidelines for Developing a Specific Usability Evaluation Model ... 252

7.3.5. Recommendations for Using the Developed Model ... 254

7.4. Research Summary ... 255

7.5. Limitations and Recommendations ... 256

7.6. Conclusions ... 258

REFERENCES ... 259

(13)

List of Tables

Table 1.1 Relationship of Research ... 10

Table 2.1 Usability Attributes in Nielsen Model ... 17

Table 2.2 A comparison between General Usability Evaluation Models with ISO Standards (ISO 9241-11 and ISO 25023) ... 24

Table 2.3 Inadequate General Models for Low Vision Users ... 25

Table 2.4 Usability Dimensions in Several Usability Evaluation Models for Mobile Applications ... 37

Table 2.5 The Current Situation of Usability for Disabled ... 45

Table 2.6 Distance Visual Acuity ... 46

Table 2.7 The Types of Low Vision ... 47

Table 2.8 Visually Impaired Mobile Applications ... 49

Table 2.9 Reading Problems ... 51

Table 2.10 Accessibility Problems ... 52

Table 2.11 Screen Reader Problems ... 53

Table 2.12 Visually Impaired Usability Requirements ... 54

Table 2.13 Frequency of Usability Dimension Being used in the Review Studies ... 61

Table 2.14 Frequency of Usability Criteria Being Used in the Review Studies ... 63

Table 3.1 Interview Question Guide ... 82

Table 3.2 Terms Related to the Keywords ... 85

Table 3.3 Central Points of the Study ... 86

Table 3.4 Journal Articles and Conference Proceedings Downloaded ... 87

Table 3.5 Final Papers for Review ... 88

Table 3.6 Validation Measures for the Proposed Model Evaluation ... 96

Table 3.7 Data Categories ... 98

Table 3.8 Four Point Scaling Method ... 99

Table 4.1 Participants Demographic ... 103

Table 4.2 Codes Classification ... 105

Table 4.3Usability Requirements by Participants ... 116

Table 4.4 Matched Usability Requirement ... 125

Table 4.5 Main Usability Requirements for Low Vision ... 127

Table 4.6 Usability Metrics and Related Usability Requirements ... 129

Table 5.1 Selected Dimensions for Proposed Model ... 143

(14)

Table 5.2 Categorization of the Criteria into Selected Dimensions ... 149

Table 5.3 Subjective and Objective Metrics of the Proposed Model ... 179

Table 5.4 Experts’ Background... 183

Table 5.5 Verification of the Proposed Usability Measures by Experts ... 184

Table 5.6 Experts Judgement the Proposed Model ... 190

Table 5.7 Experts’ Comments/Suggestions for the Proposed Model ... 191

Table 5.8 List of Dropped Metrics ... 193

Table 5.9 List of Added and Modified Metrics... 196

Table 5.10 Improved Version of Objective and Subjective Metrics for the Proposed Model ... 201

Table 5.11 Data Calculation Method of Objective Metrics ... 204

Table 6.1 The Domain Experts' Background ... 208

Table 6.2 Result for Gain Satisfaction ... 210

Table 6.3 Result for Model Satisfaction ... 212

Table 6.4 Result for Task Support Satisfaction ... 213

Table 6.5 Result for Objective Metrics ... 221

Table 6.6 Geometric Mean for Task Completion Time ... 222

Table 6.7 Average Error Rate per Task ... 228

Table 6.8 Average Number of Action per Task ... 229

Table 6.9 Statistic Description and Cronbach’s Alpha ... 231

Table 6.10 Participant Satisfaction Level ... 235

Table 6.11 Subjective Measures Evaluation per Dimension ... 236

Table 6.12 Capability of Objective Metrics to Collect Data ... 239

Table 6.13 Capability of Subjective Metrics to Collect Data ... 240

(15)

List of Figures

Figure 2.1. Enhanced usability model ………..………….…… 19

Figure 2.2. Integrated model for software usability ……….. 20

Figure 2.3. Cross-platform usability factors ………. 21

Figure 2.4. Comparison of usability models ………. 27

Figure 2.5. Mobile Goal Questions Metrics ……….. 29

Figure 2.6. Dimensions and contextual factors of the model ……… 31

Figure 2.7. GQM for Extended PACMAD usability model ……….. 33

Figure 2.8. Usability evaluation model for mobile banking applications interface ... 35

Figure 2.9. Usability evaluation (UE) factors……… 36

Figure 2.10. MAEHI model ………... 39

Figure 2.11. Portion of CLUE checkpoints related to acoustics ……… 41

Figure 2.12. A GQM model hierarchical structure ……… 74

Figure 3.1. Research methodology ……… 79

Figure 3.2. Process and activities of phase 1 ……… 80

Figure 3.3. Process and activities phase 2 ……… 84

Figure 3.4. Stages of Systematic Literature Review SLR ………. 84

Figure 3.5. Process and activities of phase 3 ……… 89

Figure 3.6. Process and activities phase 4 ……… 93

Figure 4.1 Mapping interview findings to usability dimensions derived from SLR 128 Figure 5.1. Proposed a mobile application usability evaluation model for low vision users……….. 144

Figure 5.2. Chosen criteria for the proposed model ……… 146

Figure 5.3. A mobile application usability evaluation model for low vision users.. 178

Figure 5.4. Experts’ impression on the proposed model ………. 191

Figure 5.5. The improved a mobile application usability evaluation model for low vision ………... 200

Figure 6.1. Total percentage of agreement among expert ………... 215

Figure 6.2. Usability testing ……… 218

Figure 6.3. Demographic information on gender ……… 219

Figure 6.4. Demographic information on age ………. 219

Figure 6.5. Demographic information on low vision reason ……….. 220

Figure 6.6. Successful completion rate per task ……….. 224

(16)

Figure 6.7. Task success level ………. 225

(17)

List of Appendices

Appendix A Selected Papers for Review and Analysis ……….…. 272

Appendix B Dimensions from SLR ……….…….……….…. 274

Appendix C Permission Letter ……….………... 278

Appendix D Expert Request for Nomination Letter ..………. 281

Appendix E VERIFICATION FORM ……….…….……….. 282

Appendix F Expert Request for Focus Group Meeting ………...………290

Appendix G VALIDATION FORM ……….………...………...….... 291

Appendix H Focus Group Session Slides ……….…………..…………...…….… 300

Appendix I Usability Testing Tasks ……….………..………...…………... 303

Appendix J Usability Testing Slides ……….………...………..………... 304

Appendix K Usability Testing Questionnaire ………..……….……….. 306

Appendix L Objective Data Individual Score ………...…...………... 310

Appendix M Subjective Data Individual Feedback ……..………..… 312

(18)

CHAPTER ONE INTRODUCTION

1.1.Overview

This chapter provides a general overview of mobile applications, usability evaluation models, and low vision users for mobile applications. Additionally, this chapter presents the problem statement, research questions and research objectives to be achieved by the end of this study. Finally, it presents the study significance and scope.

1.2.Background

The global usage of mobile applications has been rising in recent years. A total of 6.95 billion mobile phone users were recorded in 2020 (S. O’Dea, 2020). In the same year, a total of 218 billion mobile applications were downloaded (Statista Research Department, 2021a). At the beginning of 2021, Google Play Store boasted 3.48 million applications while Apple's App Store had 2.22 million applications (Statista Research Department, 2021b). As of January 2021, the number of active Internet users worldwide was estimated at 4.66 billion (Johnson, 2021). These numbers indicate the significance of mobile technology in most people’s life, including those with disabilities.

There is an increase in mobile applications for people with disabilities – such as physical, psychiatric or intellectual – which can help this group of people to expand their communication and social interaction, engage in e-commerce, and access information easily (Bryen et al., 2017). In particular, there are about 2.2 billion people with various degrees of vision impairment worldwide. According to the

(19)

REFERENCES

Abran, A., Khelifi, A., Suryn, W., & Seffah, A. (2003). Usability meanings and interpretations in ISO standards. Software Quality Journal, 11(4), 325–338.

Abubakar, H. I., Hashim, N., & Hussain, A. (2015). Verification process of usability evaluation model for m-banking application. In Proc. Recent Adv. Comput. Sci.

14th Int. Conf. Appl. Comput. Appl. Comput. Sci.(ACACOS), 182-189.

Abubakar, H. I. (2016). Usability Evaluation Model for Mobile Banking Applications Intrface. [Unpublished Doctoral Dissertation]. Universiti Utara Malaysia, Kedah Malaysia.

Abubakar, Hamisu Ibrahim, Hashim, N. L., & Hussain, A. (2015). A methodology for verification of a new model for evaluating the usability of an m-banking application. International Journal of Mathematical Models and Methods in Applied Sciences, 9, 592–599.

Abubakar, Hamisu Ibrahim, Hashim, N. L., & Hussain, A. (2016). Usability Evaluation Model for Mobile Banking Applications Interface: Model Evaluation Process using Experts’ Panel. Journal of Telecommunication, Electronic and Computer Engineering The, 8(10), 53–57.

Al-Khalifa, H. S. (2014). A framework for evaluating university mobile websites.

Online Information Review, 38(2), 166–185.

Al-Maani, D. I., & Bani-Salameh, H. (2017). A generic model for evaluating the usability of learning management systems. In ICC.

Al-Tarawneh, F. H. (2014). A framework for COTS software evaluation and selection for COTS mismatches handling and non-functional requirements.

(Doctoral Dissertation, Universiti Utara Malaysia).

http://etd.uum.edu.my/4488/

Albert, W., & Tullis, T. (2013). Measuring the user experience: collecting, analyzing, and presenting usability metrics. In Newnes.

Aljaber, T., & Gordon, N. (2017). A guidance and evaluation approach for mHealth education applications. In International Conference on Learning and Collaboration Technologies. Springer, Cham.

Alroobaea, R., & Mayhew, P. J. (2014). How Many Participants are Really Enough for Usability Studies ?. In 2014 Science and Information Conference. IEEE. 48- 56.

Álvarez, T., Rusu, C., Álvarez, F., Benítez-Guerrero, E., & Esparza, A. (2017).

Applying usability evaluation methods with blind users : A systematic mapping study. In Proceedings of the XVIII International Conference on Human Computer Interaction. ACM, 1-4.

An, S. (2013). Schema Theory in Reading. Theory & Practice in Language Studies, 3(1), 130-134.

Andone, I., Błaszkiewicz, K., Eibes, M., Trendafilov, B., Montag, C., & Markowetz, A. (2016). How age and gender affect smartphone usage. In Proceedings of the

(20)

2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct. ACM, 9-12.

Antwi, S. K., & Hamza, K. (2015). Qualitative and quantitative research paradigms in business research: A philosophical reflection. European Journal of Business and Management, 7(3), 217–225.

Arifin, Y., Sastria, T. G., & Barlian, E. (2018). User Experience Metric for Augmented Reality Application: A Review. Procedia Computer Science, 135, 648–656.

Aziz, N. S., Kamaludin, A., & Sulaiman, N. (2013). Assessing web site usability measurement. IJRET: International Journal of Research in Engineering and Technology, 2(9), 386–392.

Baharuddin, R., Singh, D., & Razali, R. (2013). Usability dimensions for mobile applications-a review. Res. J. Appl. Sci. Eng. Technol, 5(6), 2225–2231.

Beecham, S., Hall, T., Britton, C., Cottee, M., & Rainer, A. (2005). Using an expert panel to validate a requirements process improvement model. 76(3), 251–275.

Beer, A., & Felderer, M. (2018). Measuring and improving testability of system requirements in an industrial context by applying the goal question metric approach. In Proceedings of the 5th International Workshop on Requirements Engineering and Testing. ACM., 25-32.

Bevan, N., Carter, J., Earthy, J., Geis, T., & Harker, S. (2016). New ISO Standards for Usability, Usability Reports and Usability Measures. In International Conference on Human-Computer Interaction. Springer, Cham., 268-278.

Bevan, N. (1995). Measuring usability as quality of use. Software Quality Journal, 4(2), 115–130.

Bevan, N., Carter, J., & Harker, S. (2015). Iso 9241-11 revised: What have we learnt about usability since 1998? Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 9169, 143–151.

Bevan, N., & Macleod, M. (1994). Usability measurement in context. Behaviour and Information Technology, 13(1–2), 132–145.

Bhuiyan, M., Zaman, A., & Miraz, M. H. (2017). Usability Evaluation of a Mobile Application in Extraordinary Environment for Extraordinary People. ArXiv Preprint ArXiv:1708.04653.

Boland, A., Cherry, M., & Dickson, R. (2013). Doing a Systematic Review. Sage, London, 1–16.

Brito, D., Viana, T., Sousa, D., Lourenço, A., & Paiva, S. (2018). A mobile solution to help visually impaired people in public transports and in pedestrian walks.

International Journal of Sustainable Development and Planning, 13(2), 281–

293.

Bryen, D. N., Bornman, J., Morris, J., Moolman, E., & Sweatman, F. M. (2017). Use of Mobile Technology by Adults Who Use Augmentative and Alternative Communication: Voices from Two Countries. Assistive Technology Outcomes

(21)

& Benefits (ATOB), (11), 66-81.

Cabero-Almenara, J., Romero-Tena, R., & Palacios-Rodríguez, A. (2020).

Evaluation of Teacher Digital Competence Frameworks Through Expert Judgement: the Use of the Expert Competence Coefficient. Journal of New Approaches in Educational Research (NAER Journal), 9(2), 275–293.

Carey, M. A., & Asbury, J. E. (2016). Focus group research. Routledge.

Carroll, J. M., & Campbell, R. L. (1989). Artifacts as psychological theories: The case of human-computer interaction. Behaviour & Information Technology, 8(4), 247–256.

Carvalho, M. C. N., Dias, F. S., Reis, A. G. S., & Freire, A. P. (2018). Accessibility and Usability Problems Encountered on Websites and Applications in Mobile Devices by Blind and Normal-Vision Users. In Proceedings of the 33rd Annual ACM Symposium on Applied Computing. ACM, 2022-2029.

Christen, M., & Abegg, M. (2017). The effect of magnification and contrast on reading performance in different types of simulated low vision. Journal of Eye Movement Research JEMR, 10(2).

Clarke, K. R., & Warwick, R. M. (2001). Change in marine communities. An approach to statistical analysis and interpretation, 2.

Crossman, A. (2019). Deductive Versus Inductive Reasoning. Retrieved from https://www.thoughtco.com/deductive-vs-inductive-reasoning-3026549.

Cruz Zapata, B., Fernández-Alemán, J. L., Toval, A., & Idri, A. (2018). Reusable Software Usability Specifications for mHealth Applications. Journal of Medical Systems, 42(3), 1–9.

Csapó, Á., Wersényi, G., Nagy, H., & Stockman, T. (2015). A survey of assistive technologies and applications for blind users on mobile platforms: a review and foundation for research. Journal on Multimodal User Interfaces, 9(4), 275–286.

Damaceno, R. J. P., Braga, J. C., & Mena-Chalco, J. P. (2018). Mobile device accessibility for the visually impaired: problems mapping and recommendations. Universal Access in the Information Society, 17(2), 421–435.

Darin, T., Andrade, R., & Sánchez, J. (2018). CLUE : A Usability Evaluation Checklist for Multimodal Video Game Field Studies with Children Who Are Blind. In Proceedings of the 51st Hawaii International Conference on System Sciences.

Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1989). User acceptance of computer technology: a comparison of two theoretical models. Management Science, 35(8), 982–1003.

de Borba Campos, M., Sánchez, J., Damasio, J., & Inácio, T. (2015). Usability evaluation of a mobile navigation application for blind users. In International Conference on Universal Access in Human-Computer Interaction. Springer, Cham, 117-128.

de Oliveira, K. M., Lepreux, S., Kolski, C., & Seffah, A. (2014). Predictive usability evaluation: aligning HCI and software engineering practices. In Proceedings of

(22)

the 26th Conference on l’Interaction Homme-Machine. ACM, 177-182.

Deraman, A. B., & Salman, F. A. (2019). Managing usability evaluation practices in agile development environments. International Journal of Electrical &

Computer Engineering, 9(2), 1288–1297.

Dillon, A. (2001). The evaluation of software usability. London: Taylor and Francis.

Doiphode, R., Ganore, M., Garud, A., Ghuge, T., & Guide, P. K. (2017). Be My Eyes : Android Voice Application for Visually Impaired People.

Dubey, S., Gulati, A., & Rana, A. (2012). Integrated Model for Software Usability.

International Journal on Computer Science and Engineering, 4(03), 429–437.

Dumas, J. S. & Redish, J. (1999). A practical guide to usability testing. In Intellect books.

Erdem, R. (2017). Students with Special Educational Needs and Assistive Technologies: A Literature Review. Turkish Online Journal of Educational Technology-TOJET, 16(1), 128–146.

Etikan, Musa, & A. (2016). A comparison of convenience sampling and purposive sampling. American Journal of Theoretical and Applied Statistics, 5(1), 1–4.

Faulkner, L. (2003). Beyond the five-user assumption : Benefits of increased sample sizes in usability testing. Behavior Research Methods, Instruments, &

Computers, 35(3), 379–383.

Fekete, J. D., Van Wijk, J. J., Stasko, J. T., & North, C. (2008). The value of information visualization. In Information visualization. In Information Visualization. Springer, Berlin, Heidelberg, 1-18.

Ferreira, J. M., & Acuña, S. T. (2017). A Software Application for Collecting Usability Empirical Data about User Efficiency, Effectiveness and Satisfaction.

XII Jornadas Iberoamericanas de Ingeniería de Software e Ingeniería Del Conocimiento JIISIC’2017, 11.

Fiebai, B., Awoyesuku, E. A., & Chikezie, C. O. (2018). Functional Low Vision Services in a Tertiary Hospital: Our Initial Experience. Journal of Advances in Medicine and Medical Research, 1–6.

Flick, U. (2018). An introduction to qualitative research. In Sage Publications Limited.

Galvis, M. P. (2013). Metacognition and Reading Comprehension: Alireza Karbalaei Gangwar, H., Date, H., & Ramaswamy, R. (2015). Understanding determinants of cloud computing adoption using an integrated TAM-TOE model. Journal of Enterprise Information Management.

Gaur, A., & Kumar, M. (2018). A systematic approach to conducting review studies : An assessment of content analysis in 25 years of IB research. Journal of World Business, 53(2), 280–289.

Georgsson, M., & Staggers, N. (2016). Quantifying usability: An evaluation of a diabetes mHealth system on effectiveness, efficiency, and satisfaction metrics with associated user characteristics. Journal of the American Medical

(23)

Informatics Association, 23(1), 5–11.

Goodhue, D. L., & Thompson, R. L. (1995). Task-technology fit and individual performance. MIS Quarterly, 19(2), 213–236.

Goumopoulos, C., Papa, I., & Stavrianos, A. (2017). Development and Evaluation of a Mobile Application Suite for Enhancing the Social Inclusion and Well-Being of Seniors. In Informatics. Multidisciplinary Digital Publishing Institute, 4(3).

Griffin-Shirley, N., Banda, D. R., Ajuwon, P. M., Cheon, J., Lee, J., Park, H. R., &

Lyngdoh, S. N. (2017). A Survey on the Use of Mobile Applications for People Who Are Visually Impaired. Journal of Visual Impairment & Blindness, 111(4), 307–323.

Groth, A., & Haslwanter, D. (2016). Efficiency, effectiveness, and satisfaction of responsive mobile tourism websites: a mobile usability study. Information Technology & Tourism, 16(2), 201–228.

Grussenmeyer, W., & Folmer, E. (2017). Accessible touchscreen technology for people with visual impairments: a survey. ACM Transactions on Accessible Computing (TACCESS), 9(2), 1-31.

H. Zahra, F., Hussain, A., bt M. (2019). Verification process of metric based usability evaluation model for chronic disease management mobile applications.

International Journal of Innovative Technology and Exploring Engineering, 8, 475–482.

Hallowell, M. R., & Gambatese, J. A. (2010). Qualitative Research: Application of the Delphi Method to CEM Research. Journal of Construction Engineering and Management, 136(1), 99–107.

Hammarberg, K., Kirkman, M., & De Lacey, S. (2016). Qualitative research methods: When to use them and how to judge them. Human Reproduction, 31(3), 498–501.

Harrison, R., Flood, D., & Duce, D. (2013). Usability of mobile applications:

literature review and rationale for a new usability model. Journal of Interaction Science, 1(1), 1-16.

Hashim, N. L., & Isse, A. J. (2019). Usability Evaluation Metrics of Tourism Mobile Applications. Journal of Software Engineering and Applications, 12(7), 267–

277.

Hewett, T. T., Baecker, R., Card, S., Carey, T., Gasen, J., Mantei, M., ... & Verplank, W. (1992). ACM SIGCHI curricula for human-computer interaction. ACM.

Hoekstra, R., Vugteveen, J., Warrens, M. J., & Kruyen, P. M. (2019). An empirical analysis of alleged misunderstandings of coefficient alpha. International Journal of Social Research Methodology, 22(4), 351–364.

Holzinger, A., Searle, G., Kleinberger, T., Seffah, A., & Javahery, H. (2008).

Investigating usability metrics for the design and development of applications for the elderly. In International Conference on Computers for Handicapped Persons. Springer, Berlin, Heidelberg, 98-105.

Holzinger, A. (2005). Usability engineering methods for software developers.

(24)

Communications of the ACM, 48(1), 71–74.

Hussain, A. B., Abbas, S. A. A., Abdulwaheed, M. S., Mohammed, R. G., &

abdullah Abdulhussein, A. (2015). Usability evaluation of mobile game applications: A systematic review. International Journal of Computer and Information Technology, 04(03), 547–551.

Hussain, A. (2012). Metric Based Evaluation of Mobile Devices : Mobile Goal Question Metric ( mGQM ). Salford Business School University of Salford , Salford , UK Submitted in Partial Fulfilment of the Requirements of the Degree of Doctor of Philosophy , January, January.

Hussain, A., Razak, H. A., & Mkpojiogu, E. O. C. (2017). The perceived usability of automated testing tools for mobile applications. Journal of Engineering, Science and Technology (JESTEC), 12(4), 89–97.

Hussain, A. Bin, Aldeen, S., Abbas, A., Abdulwaheed, M. S., Mohammed, R. G., &

Abdullah Abdulhussein, A. (2015). Usability Evaluation of Mobile Game Applications: A Systematic Review. International Journal of Computer and Information Technology, 4(3), 547–551.

International Organization for Standardization. (1998). ISO 9241-11: Ergonomic requirements for office work with visual display terminals (VDTs) - part 11:

guidance on usability. International Organization for Standardization, 5.

International Organization for Standardization. (2008). 9241-151. Ergonomics of human-system interaction--Part 151: Guidance on World Wide Web user interfaces and industry standards for user centered design. International Organization for Standardization. http://www.nigelbevan.com/papers/web usability standards.pdf

International Organization for Standardization. (2016). Iso/Iec 25023 : Systems and software engineering — Systems and software Quality Requirements and Evaluation (SQuaRE) — Measurement of system and software product quality.

International Organization for Standardization.

Irick, M. L. (2008). Task-technology fit and information systems effectiveness.

Journal of Knowledge Management Practice, 9(3), 1–5.

Issa, T., & Isaias, P. (2015). Usability and Human Computer Interaction (HCI). In Sustainable Design. Springer, London, 19-36.

Jaramillo-Alcázar, A., & Luján-Mora, S. (2017). Mobile serious games: An accessibility assessment for people with visual impairments. ACM Reference, 6, 1–6.

Jiang, J. J., & Klein, G. (2009). Expectation-confirmation theory: Capitalizing on descriptive power. In Handbook of research on contemporary theoretical models in information systems. IGI Global, (pp. 384-401).

Johnson, J. (2021). Worldwide digital population as of January 2021. Statista - The Statistics Portal. https://www.statista.com/statistics/617136/digital-population- worldwide/

Joyce, G., Lilley, M., Barker, T., & Jefferies, A. (2017). Mobile application usability heuristics: Decoupling context-of-use. In International Conference of Design,

(25)

User Experience, and Usability. Springer, Cham, 410-423.

Kabir, M. A., & Han, B. (2016). An Improved Usability Evaluation Model for Point- of-Sale Systems. International Journal of Smart Home, 10(7), 269–282.

Kaiser, G., & Presmeg, N. (2019). Compendium for early career researchers in mathematics education. Springer Nature.

Kaptelinin, V. (1996). Activity theory: Implications for human-computer interaction.

In Context and consciousness: Activity theory and human-computer interaction (pp. 103–116).

Katy, H. M. (2016). Measuring usability for application software using the quality in use integration measurement model. (Doctoral Dissertation, Universiti Tun Hussein Onn Malaysia).

Khallaf, R., Naderpajouh, N., & Hastak, M. (2017). Systematic Literature Review as a Methodology for Identifying Risks. The 09th International Conference on Construction in the 21st Century (CITC-9) ‘Revolutionizing the Architecture, Engineering and Construction Industry through Leadership, Collaboration and Technology.’

Khan, A., Khusro, S., & Alam, I. (2018). BlindSense : An Accessibility-inclusive Universal User Interface for Blind People. Engineering, Technology & Applied Science Research, 8(2), 2775–2784.

Khan, M., Sulaiman, S., Said, A. M., & Tahir, M. (2011). Development of usability evaluation framework for haptic systems. Development of Usability Evaluation Framework for Haptic Systems. In VR Innovation (ISVRI), 2011 IEEE International Symposium. IEEE, 343-344.

Khan, M., Sulaiman, S., Said, A. M., & Tahir, M. (2012). Empirical validation of usability evaluation framework for haptic systems. In Computer & Information Science (ICCIS), 2012 International Conference. IEEE, 2, 1058-1061.

Kim, D., Coursaris, C. (2011). A Meta-Analytical Review of Empirical Mobile Usability Studies. Journal of Usability Studies, 6(3), 117–171.

Kim, G. J. (2015). Human – Computer Interaction Fundamentals and Practice.

Auerbach Publications.

Kitchenham, B. (2004). Procedures for performing systematic reviews. Keele, UK, Keele University, 33(2004), 1-26.

Kitchenham, B., & Charters, S. (2007). Guidelines for performing Systematic Literature Reviews in Software Engineering.

Komuro, M., & Komoda, N. (2008). An Explanation Model for Quality Improvement Effect of Peer Reviews. In Computational Intelligence for Modelling Control & Automation, 2008 International Conference. IEEE, 1159- 1164.

Krippendorff, K. (2018). Content analysis: An introduction to its methodology. Sage publications.

Kunda, D. (2003). STACE: social technical approach to COTS software evaluation.

In Component-Based Software Quality. Springer, Berlin, Heidelberg, (pp. 64-

(26)

84).

Lai, P. (2017). The Literature Review of Technology Adoption Models and Theories for the Novelty Technology. JISTEM-Journal of Information Systems and Technology Management, 14(1), 21–38.

Lazar, J., Feng, J. H., & Hochheiser, H. (2017). Research methods in human- computer interaction. In Morgan Kaufmann.

Lee, Y., Jung, M. Y., Shin, G. W., Bahn, S., Park, T., Cho, I., & Lee, J. H. (2018).

Safety and usability guidelines of clinical information systems integrating clinical workflow: A systematic review. Healthcare Informatics Research, 24(3), 157–169.

Little, R. J., & Rubin, D. B. (2019). Statistical analysis with missing data. John Wiley & Sons.

Lobo, J. F. C., Mousalli-Kayat, G., & Rivas, F. (2010). Methodologies development and software quality metrics in educational applications. In Proceedings of the 9th WSEAS International Conference on Teleccommunications and Informatics, 195–200.

Loiacono, E. T., Djamasbi, S., & Kiryazov, T. (2013). Factors that affect visually impaired users’ acceptance of audio and music websites. International Journal of Human Computer Studies, 71(3), 321–334.

Luke, D. A., Rappaport, J., & Seidman, E. (2002). Setting phenotypes in a mutual help organization: Expanding behavior setting theory. In Ecological Research to Promote Social Change. Springer, Boston, MA, 217-238.

Lune, H., & Berg, B. L. (2016). Qualitative research methods for the social sciences.

Pearson

MacKenzie, I. S. (2003). Motor behavior models for human-computer interaction.

HCI Models, Theories, and Frameworks: Toward a Multidisciplinary Science, 27–54.

Majrashi, K., & Hamilton, M. (2015). A cross-platform usability measurement model. Lecture Notes on Software Engineering, 3(2), 132–144.

Martakis, A., & Daneva, M. (2013a). Handling requirements dependencies in agile projects: A focus group with agile software development practitioners. In IEEE 7th International Conference on Research Challenges in Information Science (RCIS), 1–11.

Martakis, A., & Daneva, M. (2013b). Handling requirements dependencies in agile projects: A focus group with agile software development practitioners. In IEEE 7th International Conference on Research Challenges in Information Science (RCIS). IEEE, 1-11.

Matraf, M. S. B., & Hussain, A. (2018). Modeling Measurement Metrics for E-Book App on Mobile. Journal of Telecommunication, Electronic and Computer Engineering (JTEC), 10(11), 63–67.

Mi, N., Cavuoto, L. A., Benson, K., Smith-Jackson, T., & Nussbaum, M. A. (2014).

A heuristic checklist for an accessible smartphone interface design. Universal

(27)

Access in the Information Society, 13(14), 351–365.

Mohamed, S. F. P., Baharom, F., & Deraman, A. (2015). ESPAC model: Extended software process assessment and certification model for agile and secure software processes. ARPN Journal of Engineering and Applied Sciences, 10(3), 1364–1373.

Morris, M. R., Johnson, J., Bennett, C. L., & Cutrell, E. (2018). Rich Representations of Visual Content for Screen Reader Users. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 1-11.

Mortensen, D. (2020). How to do a thematic analysis of user interviews. Interaction Design Foundation. https://www.interaction-design.org/literature/article/how-to- do-a-thematic-analysis-of-user-interviews

Mostafa, A. F. (2015). Moblogging Type and Its Relation with Usability and Development of Knowledge Management Skills for Blind Students. Journal of Educational Technology, 12(2), 7–18.

Mustafa, S. H., & Al-Zoua’bi, L. F. (2008). Usability of the academic websites of Jordans’ Universities An evaluation study. In Proceedings of the 9th International Arab Conference for Information Technology, 31–40.

http://faculty.ksu.edu.sa/lalzouabi/Documents/Usability of the Academic Websites of Jordan’s Universities An Evaluation Study.pdf

Nathan, S. S., Hussain, A., & Hashim, N. L. (2018). Usability Evaluation of DEAF Mobile Application Interface: A Systematic Review. Journal of Engineering and Applied Sciences, 13(2), 291–297.

Nathan, S. A. / P. S. (2017). A Usability Evaluation Model for Hearing Impaired Mobile Applications Intrfaces. [Unpublished Doctoral Dissertation]. Universiti Utara Malaysia, Kedah Malaysia.

Nathan, S. S., Hussain, A., & Hashim, N. L. (2017a). Objective Measurements Analysis for Usability Evaluation of Mobile Applications for Deaf People.

Journal of Telecommunication, Electronic and Computer Engineering (JTEC), 9(2).

Nathan, S. S., Hussain, A., & Hashim, N. L. (2017b). Subjective Measurements Analysis of Usability Evaluation for Deaf People Mobile Applications. 9(2), 137–140.

Navarro, C. X., Molina, A. I., Redondo, M. A., & Juárez-Ramírez, R. (2016).

Framework to Evaluate M-Learning Systems: A Technological and Pedagogical Approach. IEEE Revista Iberoamericana de Tecnologias Del Aprendizaje, 11(1), 33–40.

Nielsen, J. (1994). Usability engineering. In Elsevier.

Nielsen, J. (2012). Usability 101: Introduction to usability. Nielsen Norman Group.

O. Nyumba, T., Wilson, K., Derrick, C. J., & Mukherjee, N. (2018). The use of focus group discussion methodology: Insights from two decades of application in conservation. Methods in Ecology and Evolution, 9(1), 20–32.

Palinkas, Horwitz, Green, Wisdom, Duan, & H. (2015). Purposeful sampling for

(28)

qualitative data collection and analysis in mixed method implementation research. Administration and Policy in Mental Health and Mental Health Services Research, 42(5), 533–544.

Pardede, P. (2008). A Review on Reading Theories and its Implication to the Teaching of Reading. UKI: ELT’n Edu~ A Cyber ELT & Edu Service from

Universitas Kristen Indonesia.

https://parlindunganpardede.wordpress.com/articles/language-teaching/a- review-on-reading-theories-and-its-implication-to-the-teaching-of-reading/

Park, K., Goh, T., & So, H.-J. (2014). Toward accessible mobile application design:

developing mobile application accessibility guidelines for people with visual impairment. In Proceedings of HCI Korea, 31-38.

Pereira, R., Baranauskas, M. C. C., & Liu, K. (2018). An Essay on Human Values in HCI. SBC Journal on Interactive Systems, 9(1), 4–16.

Peters, K., & Halcomb, E. (2015). Interviews in qualitative research. Nurse researcher, 22(4), 6-7.

Popov, L., & Chompalov, I. (2012). Crossing Over: The Interdisciplinary Meaning of Behavior Setting Theory. International Journal of Humanities and Social Science, 2(19), 18–27.

Richey, R. C. (2005). Validating instructional design and development models.

Innovations in Instructional Technology: Essays in Honor of M. David Merrill, 171–185.

Ryan, C., & Gonsalves, A. (2005). The effect of context and application type on mobile usability: an empirical study. In Proceedings of the Twenty-Eighth Australasian Conference on Computer Science, 38, 115-124.

S. O’Dea. (2020). Forecast number of mobile users worldwide 2020-2024 | Statista.

In Statista - The Statistics Portal.

https://www.statista.com/statistics/218984/number-of-global-mobile-users- since-2010/

Saleh, A. M., & Ismail, R. B. (2015). Usability evaluation frameworks of mobile application: a mini-systematic literature review. Global Summit on Education GSE.

Saleh, A. M., Ismail, R., Fabil, N., Norwawi, N. M., & Wahid, F. A. (2017).

Measuring Usability: Importance Attributes for Mobile Applications. Advanced Science Letters, 23(5), 4738–4741.

Saleh, A., Isamil, R. B., & Fabil, N. B. (2015). Extension of PACMAD Model for Usability Evaluation Metrics Using Goal Question Metrics (GQM) Approach.

Journal of Theoretical and Applied Information Technology, 79(1), 90–100.

Salman, F. A., Deraman, A. B., Jalil, M. B. A., & Fahmy, S. (2017). Usability Measurement Model for Mobile Web Applications. In The 10th Edition of Postgraduate Research Workshop (PRW) at SOFTEC Asia 2017 Conference Usability,10-13.

Samaradiwakara, G. D. M. N., & Gunawardena, C. G. (2014). Comparison of Existing Technology Acceptance Theories and Models to Suggest a Well

(29)

Improved Theory / Model. International Technical Sciences Journal, 1(1), 21–

36.

Sarlin, P. (2014). Mapping financial stability. Springer.

Sauro, J., & Lewis, J. R. (2016). Quantifying the user experience: Practical statistics for user research. Morgan Kaufmann. Morgan Kaufmann

Seffah, A., Donyaee, M., Kline, R. B., & Padda, H. K. (2006). Usability measurement and metrics: A consolidated model. Software Quality Journal, 14(2), 159–178.

Shawgi, E., & Noureldien, N. A. (2015). Usability measurement model (UMM): a new model for measuring websites usability. International Journal of Information Science, 5(1), 5–13.

Shinde, V., & Rathi, V. (2018). Different Types of CAPTCHA: A Literature Survey.

Open Access International Journal of Science& Engineering (OAIJSE), ISSN, 3(1), 2456–3293.

Siebra, C., Gouveia, T., Macedo, J., Correia, W., Penha, M., Anjos, M., ... & Santos, A. L. (2016). Observation based analysis on the use of mobile applications for visually impaired users. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct. ACM, 807-814.

Sierra, J. S., & Togores, J. (2012). Designing mobile apps for visually impaired and blind users. In The Fifth International Conference on Advances in Computer- Human Interactions, 47–52.

Statista Research Department. (2021a). Mobile app usage - Statistics & Facts.

Statista - The Statistics Portal. https://www.statista.com/topics/1002/mobile- app-usage/#dossierSummary

Statista Research Department. (2021b). Number of apps available in leading app stores 2021. Statista - The Statistics Portal.

https://www.statista.com/statistics/276623/number-of-apps-available-in- leading-app-stores/

Tan, J., Ronkko, K., & Gencel, C. (2013). A Framework for Software Usability and User Experience Measurement in Mobile Industry. 2013 Joint Conference of the 23rd International Workshop on Software Measurement and the 8th International Conference on Software Process and Product Measurement, 156–

164.

Tory, M., & Möller, T. (2005). Evaluating visualizations: Do expert reviews work?

IEEE Computer Graphics and Applications, 25(5), 8–11.

Turner, C. W., Lewis, J. R., & Nielsen, J. (2006). Determining Usability Test Sample Size. International Encyclopedia of Ergonomics and Human Factors, 3(2), 3084–3088.

Van Solingen, R., Basili, V., Caldiera, G., & Rombach, H. D. (2002). Goal Question Metric (GQM) Approach. Encyclopedia of Software Engineering, 2, 1–10.

Venkatesh, V., & Bala, H. (2008). Technology acceptance model 3 and a research

(30)

agenda on interventions. Decision Sciences, 39(2), 273–315.

Venkatesh, V., & Davis, F. D. (2000). A theoretical extension of the technology acceptance model: Four longitudinal field studies. Management Science, 46(2), 186–204.

Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. D. (2003). User acceptance of information technology: Toward a unified view. MIS Quarterly, 425–478.

Villanueva de Debat, E. (2006). Applying Current Approaches to the Teaching of Reading. In English Teaching Forum, 44(1), 8–15.

Vuolle, M., Aula, A., Kulju, M., Vainio, T., & Wigelius, H. (2008). Identifying Usability and Productivity Dimensions for Measuring the Success of Mobile Business Services. Advances in Human-Computer Interaction, 2008, 1–9.

Waddell, T. F., Zhang, B., & Sundar, S. S. (2016). Human – Computer Interaction.

1–9.

Walliman, N. (2017). Research methods: The basics. Routledge.

Web World Wide Consortium. (2015a). Media Accessibility User Requirements.

https://www.w3.org/TR/2015/NOTE-media-accessibility-reqs- 20151203/#visual-low-vision

Web World Wide Consortium. (2015b). Mobile accessibility: How WCAG 2.0 and other W3C/WAI guidelines apply to mobile. https://www.w3.org/TR/mobile- accessibility-mapping/

Web World Wide Consortium. (2017a). Accessibility Principles.

https://www.w3.org/WAI/fundamentals/accessibility-principles/#standards Web World Wide Consortium. (2017b). Mobile Accessibility at W3C.

https://www.w3.org/WAI/standards-guidelines/mobile/

Web World Wide Consortium. (2018a). Requirements for Web Content Accessibility Guidelines 2.1. https://w3c.github.io/wcag21/requirements/

Web World Wide Consortium. (2018b). Web Content Accessibility Guidelines (WCAG) 2.1. https://www.w3.org/TR/WCAG21/

WebAIM Organization. (2013). Visual Disabilities - Low Vision.

https://webaim.org/articles/visual/lowvision

WebAIM Organization. (2017). WebAIM: Screen Reader User Survey #7 Results.

https://webaim.org/projects/screenreadersurvey7/#

Wessels, L., & Drennan, J. (2010). An investigation of consumer acceptance of M‐banking. International Journal of Bank Marketing.

WHO. (2018). Blindness and Vision Impairment. http://www.who.int/en/news- room/fact-sheets/detail/blindness-and-visual-impairment

Wich, M., & Kramer, T. (2015). Enhanced Human-Computer Interaction for Business Applications on Mobile Devices: A Design-Oriented Development of a Usability Evaluation Questionnaire. 2015 48th Hawaii International Conference on System Sciences, January, 472–481.

(31)

Wiegers, K. E. (2002). Seven truths about peer reviews. Cutter IT Journal, 15(7), 31–37.

World Health Organization. (2021). Blindness and vision impairment.

https://www.who.int/news-room/fact-sheets/detail/blindness-and-visual- impairment

Yahya, F., Walters, R. J., & Wills, G. B. (2017). Using Goal-Question-Metric (GQM) Approach to Assess Security in Cloud Storage. In Enterprise Security.

Springer, Cham, 223-240.

Yeratziotis, A., & Zaphiris, P. (2018). A Heuristic Evaluation for Deaf Web User Experience (HE4DWUX). International Journal of Human-Computer Interaction, 34(3), 195–217.

Zahra, F., Hussain, A., & Mohd, H. (2017). Usability evaluation of mobile applications; Where do we stand?. In AIP Conference Proceedings, 20056(2017).

Zarour, M. (2018). The use of evaluation theory and square standards to develop user needs experience evaluation method. Journal of Engineering Technology, 6(1), 46–65.

(32)

Appendix A

Selected Papers for Review and Analysis

Descriptions Journal/Proceeding/Conference Authors Usability in

general and mobile application

Journal of Usability Studies Coursaris and Kim (2011).

Software Quality Journal Abran et al. (2003) Bevan (1995) Seffah et al. (2006) International Conference on Learning and

Collaboration Technologies (Springer, Cham) Aljaber et al. (2017) International Conference on Human-

Computer Interaction Bevan et al. (2016)

Kurosu (2015) International Journal of Research in

Engineering and Technology (IJRET) Aziz et al. (2013) Procedia Computer Science Arifin et al. (2018) Res. J. Appl. Sci. Eng. Technol Baharuddin et al. (2013) International Workshop on Requirements

Engineering and Testing (ACM). Beer & Felderer (2018) Journal of Medical Systems Cruz Zapata et al.

(2018) Conference on l’Interaction Homme-Machine

(ACM) de Oliveira et al. (2014)

International Journal on Computer Science

and Engineering Dubey, Gulati & Rana

(2012)

Journal of Interaction Science Harrison et al. (2013) PHD Dissertation (University of Salford) Hussain (2012) Doctoral Dissertation, Universiti Tun Hussein

Onn Malaysia Katy (2016)

International Conference on

Teleccommunications and Informatics Lobo, Mousalli-Kayat

& Rivas (2010)

Nielsen Norman Group Nielsen (2012)

Journal of Theoretical and Applied

Information Technology Saleh et al. (2015)

Researchgate Salman et al. (2017)

International Journal of Information Shawgi & Noureldien (2015)

International Workshop on Software Measurement and the 8th International Conference on Software Process and Product Measurement

Tan, Ronkko & Gencel (2013)

In User Centric E-Government Springer,

Cham. Taşkın, Coşkun &

Tüzün (2018) Journal of Engineering Technology Zarour (2018).

International Journal of Computer

Applications Dubey, Rana & Sharma

(2012) Journal of Telecommunication, Electronic

and Computer Engineering (JTEC) Matraf & Hussain (2018)

Hussain, Hashim &

Nathan (2018) International Conference on IEEE Zali (2016) IEEE International Symposium on VR Khan et al. (2011)

(33)

Innovation (ISVRI)

International Workshop On Meaningful

Measures Bevan (2008)

HCI and visually impaired

Assistive Technology Outcomes & Benefits

(ATOB) Bryen et al (2017)

Annual ACM Symposium on Applied

Computing Carvalho et al. (2018)

Journal on Multimodal User Interfaces Csapó et al. (2015) Universal Access in the Information Society Damaceno et al. (2018)

Hawaii ICSS Darin et al. (2018)

International Conference on Universal Access

in Human-Computer Interaction de Borba et al. (2015)

ACM Reference Jaramillo-Alcázar &

Luján-Mora (2017) Engineering, Technology & Applied Science

Research Khan, Khusro & Alam

(2018) International Journal of Human Computer

Studies Loiacono, Djamasbi &

Kiryazov (2013) Conference on Human Factors in Computing

Systems (ACM) Morris et al. (2018)

Journal of Educational Technolog Mostafa (2015)

HCI Korea Park, Goh & So (2014)

Informatics Multidisciplinary Digital

Publishing Institute Goumopoulos, Papa &

Stavrianos (2017) Universal access in the information society Cavuoto et al (2014) International Conference on Human-

Computer Interaction with Mobile Devices and Services Adjunct (ACM)

Siebra et al (2016)

International Conference on Advances in

Computer-Human Interactions Sierra & Togores (2012)

International Conference on Human-

Computer Interaction Álvarez et al. (2017)

Other related ACM Transactions on Accessible Computing

(TACCESS) Grussenmeyer &

Folmer, (2017) Turkish Online Journal of Educational

Technology (TOJET) Erdem (2017)

JIISIC Ferreira & Acuña

(2017)

(34)

Appendix B Dimensions from SLR

No. Dimension Total Count

1 Efficiency 29

2 Satisfaction 27

3 Effectiveness 27

4 Learnability 26

5 Accessibility 19

6 Safety 12

7 Understandability 11

8 Operability 10

9 Usefulness 9

10 Errors 9

11 Memorability/ Remember ability 9

12 Attractiveness 7

13 Ease of use 6

14 Simplicity 5

15 Productivity 5

16 Cognitive load 4

17 Flexibility 4

18 Attitude 4

19 Accuracy 4

20 Navigation 4

21 Trustfulness 3

22 Universality 3

23 Acceptability 3

24 Consistency 2

25 Readability 2

26 Features 2

27 Reliability 2

28 Generalizability 2

29 Security 1

30 Likeability 1

31 Comprehensibility 1

32 Portability 1

33 Intention to use 1

34 Utility 1

35 Functionality 1

Rujukan

DOKUMEN BERKAITAN

The motivations behind development of mobile learning in a flexible manufacturing environment are mobility, self-paced learning, user friendly interface, autonomous

Hence, this project is developed with the objectives: To design a mobile application that allows users to check their well-being level, to provide a mobile

The end product of this project is a mobile application for public transportation which could improve the convenience of using public transport. This project is developed based on the

usability models in general for desktop and mobile applications, In the future, this study will be extended to study on guidelines for deaf mobile application and

For this study, the objective of the heuristic evaluation is to improve the user interface design of the mobile numerical application, evaluate the educational design and

The study also aims to determine the usability, and factors that might influence museum visitors' usage of 'Mobile Apps. Learning' application, that was

DAAO Form 47-2A Evaluation Document for New Registered Aircraft, including DAAO Form 47-11 Application for Certificate of Registration and DAAO Form 47-13 Application for Renewal

This mobile application will able to generate an electronic-claiming form which include total distance travelled and route travelled recorded data for user to send to