An evaluation framework for information systems from the users’ perspective: a scoping review
DOI:
https://doi.org/10.59681/2175-4411.v18.2026.1539Palavras-chave:
Evaluation models, Information System, Information TechnologyResumo
This scoping review aims to synthesize scientific literature on information systems evaluation methods from the users’ perspective, focusing on their characteristics, construction stages, and measurement properties. The search was conducted in 2024 across the following databases: ACM Digital Library, Embase, Engineering Village, ERIC, IEEE Xplore, LILACS, MEDLINE (via PubMed), Scopus, and Web of Science. The study selection process was independently carried out by two reviewers. A total of 18,837 articles were identified in the searches, and 11 were ultimately included in the review. The studies presented various evaluation frameworks specifically for health information systems, with one study also evaluating information systems from other fields. Only five studies reported the frameworks’ validation procedures, and very few provided comprehensive results of the validations or sufficiently thorough analyses. This scoping review underscores the methodological diversity in information systems evaluation models and highlights the limitations of current use. It emphasizes the need for standardized approaches to enhance comparability, transparency, and long-term system development.
Downloads
Referências
Pleskach V, Pleskach M, Zelikovska O. Information Security Management System in Distributed Information Systems. In: 2019 IEEE International Conference on Advanced Trends in Information Theory, ATIT 2019 - Proceedings. 2019. DOI: https://doi.org/10.1109/ATIT49449.2019.9030484
Gregersen E. 5 Components of Information Systems | Britannica. Britannica2021;
Stoykova S, Shakev N. Artificial Intelligence for Management Information Systems: Opportunities, Challenges, and Future Directions. Algorithms2023;16(8). DOI: https://doi.org/10.3390/a16080357
Maia MAQ, Barbosa RR, Williams P. Usabilidade e experiência do usuário de sistemas de informação: em busca de limites e relações. Ciência da Informação em Revista 2020;6(3). DOI: https://doi.org/10.28998/cirev.2019v6n3c
BSI. ISO 9241-210:2019, Ergonomics of human-system interaction - Part 210: Human-centred design for interactive systems. 2019.
Mie L, Cintho M, Rodrigues Machado R, Cabral Moro CM, Correspondente A. Métodos para Avaliação de Sistema de Informação em Saúde. 2016.
Osborne SE. Evaluating Health Care Information Systems: Methods and Applications. J Adv Nurs 1994;20(1). DOI: https://doi.org/10.1046/j.1365-2648.1994.20010202-6.x
de Morais RM, Costa AL. Um modelo para avaliação de sistemas de informação do SUS de abrangência nacional: O processo de seleção e estruturação de indicadores. Revista de Administracao Publica 2014;48(3). DOI: https://doi.org/10.1590/0034-76121512
Garcia-Smith D, Effken JA. Development and initial evaluation of the Clinical Information Systems Success Model (CISSM). Int J Med Inform 2013;82(6). DOI: https://doi.org/10.1016/j.ijmedinf.2013.01.011
Gürsel G, Zayim N, Gülkesen KH, Arifoǧlu A, Saka O. A new approach in the evaluation of hospital information systems. Turkish Journal of Electrical Engineering and Computer Sciences 2014;22(1). DOI: https://doi.org/10.3906/elk-1110-15
Kanungo S, Duda S, Srinivas Y. A structured model for evaluating information systems effectiveness. Syst Res Behav Sci 1999;16(6). DOI: https://doi.org/10.1002/(SICI)1099-1743(199911/12)16:6<495::AID-SRES238>3.0.CO;2-R
Kukhareva P V., Weir C, Del Fiol G, Aarons GA, Taft TY, Schlechter CR, et al. Evaluation in Life Cycle of Information Technology (ELICIT) framework: Supporting the innovation life cycle from business case assessment to summative evaluation. J Biomed Inform 2022;127. DOI: https://doi.org/10.1016/j.jbi.2022.104014
Lau F, Hagens S, Muttitt S. A proposed benefits evaluation framework for health information systems in Canada. Healthc Q 2007;10(1).
Norouzi H, Mehrolhassani MH, Hajesmaeel-Gohari S, Ahmadian L, Ghaemi MM, Mohammadi M, et al. Identification and classification of indicators for evaluating health information systems. Health Policy Technol 2024;13(2). DOI: https://doi.org/10.1016/j.hlpt.2023.100833
Sockolow PS, Weiner JP, Bowles KH, Lehmann HP. A new instrument for measuring clinician satisfaction with electronic health records. CIN - Computers Informatics Nursing 2011;29(10). DOI: https://doi.org/10.1097/NCN.0b013e31821a1568
Stockdale R, Standing C. An interpretive approach to evaluating information systems: A content, context, process framework. Eur J Oper Res 2006;173(3). DOI: https://doi.org/10.1016/j.ejor.2005.07.006
Stylianides A, Mantas J, Roupa Z, Yamasaki EN. Development of an evaluation framework for health information systems (DIPSA). Acta Informatica Medica 2018;26(4). DOI: https://doi.org/10.5455/aim.2018.26.230-234
Yusof MM, Kuljis J, Papazafeiropoulou A, Stergioulas LK. An evaluation framework for Health Information Systems: human, organization and technology-fit factors (HOT-fit). Int J Med Inform 2008;77(6). DOI: https://doi.org/10.1016/j.ijmedinf.2007.08.011
Reichenheim M, Bastos JL. What, what for and how? Developing measurement instruments in epidemiology. Rev Saude Publica 2021;55.
Barzekar H, Ebrahimzadeh F, Luo J, Karami M, Robati Z, Goodarzi P. Adoption of hospital information system among nurses: A technology acceptance model approach. Acta Informatica Medica 2019;27(5). DOI: https://doi.org/10.5455/aim.2019.27.305-310
Ammenwerth E. Technology Acceptance Models in ealth nformatics: TAM and UTAUT. Stud Health Technol Inform 2019;263. DOI: https://doi.org/10.3233/SHTI190111
Rana RH, Alam K, Gow J. Health outcome and expenditure in low-income countries: does increasing diffusion of information and communication technology matter? Inf Technol Dev 2020;26(3). DOI: https://doi.org/10.1080/02681102.2019.1678455
Downloads
Publicado
Como Citar
Edição
Secção
Licença
Direitos de Autor (c) 2026 Vanessa Pereira Corrêa Rampinelli, Catiele Raquel Schmidt, Júlia Meller Dias de Oliveira, Ianka Cristina Celuppi, Arthur Thives Mello, Jades Fernando Hammes, Célio Luiz Cunha, Raul Sidnei Wazlawick, Danielle Soares Rocha Vieira, Eduardo Monguilhott Dalmarco
Este trabalho encontra-se publicado com a Licença Internacional Creative Commons Atribuição-NãoComercial-CompartilhaIgual 4.0.
