Ability to understand and apply current concepts about clinical reasoning and informatics support for clinical decisions. We used a blended-learning model to take advantage of the online and face-to-face modalities, and developed a virtual learning environment in the Moodle platform. We also measured attitudes and opinion regarding the programs at the end of each course. Our study assessment instrument and the summative exams developed by the BMI Department covered the same themes in equivalent content proportions. In questions were selected from the item bank, developing a item MCQ exam that covered the BMI-1 course content, and a second exam with the same number of items for the BMI-2 course.
Many items were targeted to higher cognitive levels like application and problem solving.
Evaluation Methods in Medical Informatics | Charles P. Friedman | Springer
The same test was administered the first and last days of each course. The tests were applied through the Moodle online platform used in the BMI courses, and was voluntary.
- Low-power processors and systems on chips.
- Great Golf: Essential Tips from Historys Top Golfers!
- Security and Trust Management: 7th International Workshop, STM 2011, Copenhagen, Denmark, June 27-28, 2011, Revised Selected Papers!
- Evaluation methods in biomedical informatics.
- Browse more videos.
- Bibliographic Information.
- Evaluation Methods in Medical Informatics?
A hospital hired personnel to manage the database of their patient population. One employee sold the database to a healthcare products company. Which of the following informatics ethics principles were violated? The main difference between Bioinformatics and Medical Informatics is that the first focuses on the following area:.
An intern sees a patient in the clinic with pregnancy-induced hypertensive disorder, and has not had time to review the medical literature where some important evidence about a therapeutic modality has recent been published. What source of uncertainty is most likely to occur in this situation? We also applied a program evaluation survey to the students at the end of each course, a item questionnaire that explored several aspects of the programs.
After the information was collected, it was summarized without identifiers by personnel not directly involved in the study. Furthermore, there were two partial summative exams in each course, as components of the formal curriculum assessment activities, with two components: a item multiple-choice question test, and a practical hands-on test in the computer lab that explored competencies such as use of Medline, appraisal of the basic elements of a telemedicine consultation case, among others.
At the end of each course the students had a final summative exam.
Only the data from students that responded the pre and post-tests were included. P-values less than 0. Paul, MN www. The study was in compliance with the Declaration of Helsinki of ethical principles for research involving human subjects. There was no individual written informed consent, since the study was done as part of the program evaluation and quality improvement process, the tests were voluntary and the data are described in aggregate and anonymous fashion.
The gender distribution of the sampled population was The psychometric analysis for the diagnostic pre-post test and the BMI Department summative examinations was performed with the software Iteman, which uses the Classical Measurement Theory CMT conceptual framework. The lower reliability of the pre-post test is acceptable and reasonable for a non-summative diagnostic test.
The p-value difficulty index in psychometric parlance, which represents the percentage of items answered correctly in the test; the higher the p-value the easier the test of the diagnostic post-test was about 0. Since we included the totality of our available student population, and due to the large size of our student cohorts, we deemed not necessary to do an a priori sample size calculation.
This is reasonable since our simple sizes are large and the pre-post differences are considerable. At the end of all the BMI courses There are few published reports of BMI curricula in medical schools, most are program descriptions, their development, contents and teaching methods. Our study measured knowledge acquisition of BMI in undergraduate medical students, and demonstrated a substantial increase in knowledge after the educational experiences and a positive opinion about the courses.
Their curricular model is more integrated than ours, which allows for better longitudinal coordination and integration at a higher level. Their medical school has been implementing BMI education for a longer period, since , and has evolved to a coherent educational intervention through several modifications. One of the more challenging tasks in our medical school during the implementation and evaluation of the BMI curriculum, is the size of our student and faculty body.
UNAM Faculty of Medicine has more than undergraduate students and more than faculty, so our organizational structure increases the complexity of curricular innovations implementation. The instruments they used were opinion questionnaires and self-assessed estimates of knowledge, which are not as indicative of learning as external objective instruments, similar to the ones used in our study. Self-assessment is a complex and difficult topic in medical education, but the majority of evidence suggests that physicians and clinicians in training are poor at self-assessment [ 34 , 35 ].
Their self-reported assessment of knowledge used a Likert scale of agreement with several statements related to BMI abilities, which is not directly comparable with our estimates of knowledge that used an objective test with percent correct scores. In our setting we had a higher response rate in our student evaluations, probably due to the mandatory nature of course and teacher evaluation in our institution. In summary, the direction of change in BMI knowledge was positive in both studies, although the magnitude of this change cannot be directly compared due to the different scales of the measurement instruments.
Overall, the nature of the obstacles and barriers to their BMI curricular innovation appear to be similar to ours, and require comprehensive longitudinal evaluation and intense integration efforts. After using several search strategies in many databases we were not able to find other research papers that measured BMI knowledge acquisition with instruments that had evidence of validity.
Our BMI-1 and BMI-2 courses are associated with a large and statistically significant increase in knowledge, and overall had a positive evaluation by the students. The consistency of the knowledge increase in the consecutive courses adds validity to our findings, the pre and immediate post measures are almost identical in the two consecutive student cohorts for each BMI course, suggesting a similar baseline level of knowledge and an equivalent amount of knowledge acquisition.
In social sciences usually the larger the effect size, the greater the impact of an intervention. Cohen suggested that an effect size of 0. In our study all effect sizes are above 1. It is important to note that the study instruments were applied in a voluntary non-summative fashion, so the students may not have applied the same effort to answer them correctly as for summative tests. Students tend to have higher scores in summative exams than in formative diagnostic voluntary tests [ 37 ].
Nonetheless, our finding of a significant increase in knowledge with the same instrument in several consecutive cohorts of students lends reproducibility and validity to our results. Cook and Bordage recently proposed a classification of medical education research studies based on their purpose: description studies what was done? Research performed to document if educational interventions work is justified, because many teaching strategies and courses are implemented in medical schools and hospitals without any evidence of their effectiveness, and all require resources to be applied and implemented.
We recognize these potential threats but made our best efforts to control them to the extent possible within the confines of the study design; our main outcomes were knowledge and satisfaction, and knowledge was measured only with MCQ tests, which are limited for assessing competence and performance, however the study participants are novice medical students with no formal clinical responsibilities, so knowledge improvement was a more realistic goal; the study reflects only one school and our particular curriculum, so the external validity of the findings to other institutions with different programs could be questionable, nonetheless our curriculum has a solid design with sound educational strategies, and the consistency in our findings with different cohorts of students argues for the validity to our conclusions.
We plan to design a study that evaluates different medical schools, to address some of the issues discussed in our study. The pre and post-test instruments were the same, as mentioned in Methods. This risk of bias cannot be completely excluded in quasiexperimental design.
We argue, however, that our sample sizes, the magnitude of the differences, the diagnostic nature of our tests, and the time interval between pre and post-test one semester can contribute to attenuate this potential bias. We acknowledge that the high reliability in our study satisfaction questionnaire does not exclude response bias and threats to validity.
How early should BMI be taught in the medical curriculum? Shortliffe maintains that BMI training should start at the undergraduate level in medical school and the healthcare professions [ 4 , 5 ], when the professional identity of physicians, nurses and other practitioners is being developed. Ideally BMI should be incorporated in an integrated and longitudinal fashion throughout the curriculum to achieve optimal educational outcomes, this is nonetheless not always feasible. There are advantages and disadvantages to this approach, and each medical school should approach the curricular design task carefully, taking into account their educational model and available resources, making explicit efforts to integrate BMI competencies during clinical training.
Evaluation Methods Biomedical Informatics by Charles Friedman, Softcover
There is controversy in the medical education literature about the different levels of outcomes that should be sought in educational interventions and research studies. The lower levels of this framework are satisfaction and knowledge acquisition, and the higher levels more important and relevant to society are behavioral changes of physicians, improvement in clinical outcomes in patients and in the community. These arguments are relevant, but the model has been criticized recently. Educational interventions that occur early in medical school training cannot be held to the same standards as interventions that occur when a doctor is already practicing or is involved in continuing medical education activities, the time gap is too large and there are too many confounding factors and intervening variables [ 42 ].
Since our students in the first two years still do not have direct responsibility with patients, we think that knowledge acquisition and satisfaction are more realistic and reasonable endpoints. On the other hand, there is a substantial amount of literature that suggests that increases in knowledge and test scores in MCQ exams have a positive and statistically significant correlation with competence and performance. Typical correlations between measures of knowledge and performance are in the range of 0.
The increasing sophistication of technology and educational strategies predict a fascinating scenario where they can interact and make the learning of BMI much more integral to clinical practice, as Otero and Hersh suggest in a recent Web 3. Finally, the current emphasis on evidence-based medicine, healthcare learning organizations, patient safety and quality of care, provide an appropriate scenario for advancing the importance of teaching and learning BMI in health professions schools and academic health centers [ 1 , 45 ].
Our study shows a significant increase in BMI knowledge after an educational intervention in four medical student cohorts, and an overall positive evaluation by the students. Computer Applications in Health Care and Biomedicine. London, UK: Springer-Verlag; Anderson B. Contemporary issues in medicine—medical informatics and population health: report II of the Medical School Objectives Project.
Acad Med. Greiner A, Knebel E. Health professions education. In: Greiner A, Knebel E, editors. A bridge to quality. Washington, D. C: National Academies Press; Biomedical informatics: changing what physicians need to know and how they learn. Shortliffe E. Biomedical informatics in the education of physicians. Ohno-Machado L. Careers in informatics: a diversity of options with an abundance of jobs.
J Am Med Inform Assoc. Defining the medical subspecialty of clinical informatics. Clinical informatics: prospects for a new medical subspecialty. Development and implementation of a biomedical informatics course for medical students: challenges of a large-scale blended-learning program. Otto A, Kushniruk A. Incorporation of medical informatics and information technology as core components of undergraduate medical education - time for change! Stud Health Technol Inform. Educating medical students as competent users of health information technologies: the MSOP data.
- Featured channels.
- See a Problem?.
- Essentials of Radiology.
- Evaluation Methods in Biomedical Informatics (Health Informatics).
- Atlas of Airborne Fungal Spores in Europe!
The evolution of a novel biomedical informatics curriculum for medical students. Gac Med Mex. The use and interpretation of quasi-experimental studies in medical informatics. How to design and evaluate research in education.
Recommendations of the international medical informatics association IMIA on education in biomedical and health informatics. Methods Inf Med. Downing SM. Twelve steps for effective test development. Handbook of test development. Mahwah, N. It seems that you're in Germany. We have a dedicated site for Germany. Authors: Friedman , Charles P. The BMJ said of the first edition of Evaluation Methods in Biomedical Informatics: "A landmark book, which should have a place of honour in the library of anyone seeking a deeper understanding of clinical informatics in general and evaluation in particular.
As information technology takes on an increasingly central role in health care and biomedical research, successful and reliable methods for evaluation become more and more imperative. Evaluation Methods in Biomedical Informatics, Second Edition, is the best resource available to support discussion, development, and implementation of evaluation procedures in these settings. In addition to revision of all material carried over from its predecessor, this Second Edition includes:.
A completely revised and expanded chapter on qualitative evaluation at the organizational and cultural levels. An extensively re-written introduction to the types of studies on biomedical information systems and technologies. New scenarios and examples drawing on a wider range of application areas including bioinformatics. This updated second edition provides valuable introductory material exploring the differences and similarities, and usefully covers many of the statistical and other techniques common to both.