Volume 16, Issue 10 (January 2023)
Qom Univ Med Sci J 2023, 16(10): 786-803 |
Back to browse issues page
Ethics code: IR.KAUMS.NUHEPM.REC.1401.047
Download citation:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
Zare M, Tagharrobi Z. The Framework of Designing, Reporting, and Evaluating the Methodological Quality for Psychometric Studies: The Revised COSMIN Checklists. Qom Univ Med Sci J 2023; 16 (10) :786-803
URL: http://journal.muq.ac.ir/article-1-3594-en.html
URL: http://journal.muq.ac.ir/article-1-3594-en.html
1- Trauma Nursing Research Center, Faculty of Nursing and Midwifery, Kashan University of Medical Sciences, Kashan, Iran.
2- Trauma Nursing Research Center, Faculty of Nursing and Midwifery, Kashan University of Medical Sciences, Kashan, Iran. ,tagharrobi.za@kaums.ac.ir
2- Trauma Nursing Research Center, Faculty of Nursing and Midwifery, Kashan University of Medical Sciences, Kashan, Iran. ,
Full-Text [PDF 932 kb]
(610 Downloads)
| Abstract (HTML) (1246 Views)
Full-Text: (516 Views)
Introduction
One of the challenges in studies related to measuring patient-reported outcomes (PRO) is the selection of appropriate measurement tools. Psychometric studies introduce reliable tools. Considering the increasing number of instruments for PROs, it is necessary to systematically study the psychometric properties of the instruments to select and use the best tools. Existence of a comprehensive and valid scale or checklist to evaluate the quality of psychometric studies is necessary. One of these tools is COSMIN (Consensus-based Standards for the selection of health Measurement instruments) checklist. Considering the presentation of the revised forms for this checklist and the importance of evaluating and reporting psychometric instruments, this study aims to review the revised forms of COSMIN checklist.
Methods
In this narrative review study, a search was conducted in international databases (PubMed, Scopus, Proquest, Web of Science, and CINAHL), and national databases (Magiran, IranDoc, ISC, and SID) for all articles that used COSMIN checklist to measure the quality of psychometric studies using related keywords in Persian and English without considering time limitations
Results
In the first study using the Delphi method (2006), a working group consisting of 57 experts in epidemiology, measurement, statistics, and biostatistics, introduced a classification of characteristics for instruments in three areas of validity (including content and face validity, structural validity, hypothesis testing, cross-cultural validity, and criterion validity), reliability (including internal consistency, reliability, and measurement error) and responsiveness (including responsiveness and interpretability). Considering the limitations of the initial version, the COSMIN committee changed it to strengthen its reliability and ease of use. These changes led to creating four versions including (1) the COSMIN Risk of Bias Checklist, (2) the COSMIN Risk of Bias Tool to assess the quality of studies by focusing on reliability and measurement error, (3) the COSMIN Study Design Checklist, and (4) the COSMIN Reporting Guideline. With the introduction of new checklists, the COSMIN committee advised the researchers not to use the original version and only use the newly introduced checklists according to the study objectives.
Discussion
Considering the need to use valid and reliable PRO measures in research and clinic settings, as well as the increasing number of instruments for PRO constructs, there was a need for a scale to provide criteria for measuring the psychometric properties of the instruments. In this regard, the COSMIN checklists were introduced. However, they have some limitations such as being specialized and the need for teaching how to complete it, being opinion-based (not evidence-based), lack of sufficient evidence to confirm the reliability of the checklist, failure to determine the importance of some psychometric properties compared to others, and lack of assessing essential details in evaluation of some psychometric properties. In overall, COSMIN checklists can provide logical indicators and standards for measuring the psychometric properties of instruments; therefore, they can be used for measuring the quality of psychometric studies, designing studies, or reporting psychometric studies by researchers, reviewers, and journal editors.
Ethical Considerations
Compliance with ethical guidelines
This study was approved by the Ethics Committee of Kashan University of Medical Sciences (Code: IR.KAUMS.NUHEPM.REC.1401.047).
Funding
This research did not receive any grant from funding agencies in the public, commercial, or non-profit sectors.
Authors contributions
All authors equally contributed to preparing this article.
Conflicts of interest
The authors declared no conflict of interest.
Acknowledgements
In this way, all the researchers, especially COSMIN researchers, whose articles were reviewed in this study, as well as the Ethics Committee of Kashan University of Medical Sciences and Healthcare Services, are thanked and appreciated.
One of the challenges in studies related to measuring patient-reported outcomes (PRO) is the selection of appropriate measurement tools. Psychometric studies introduce reliable tools. Considering the increasing number of instruments for PROs, it is necessary to systematically study the psychometric properties of the instruments to select and use the best tools. Existence of a comprehensive and valid scale or checklist to evaluate the quality of psychometric studies is necessary. One of these tools is COSMIN (Consensus-based Standards for the selection of health Measurement instruments) checklist. Considering the presentation of the revised forms for this checklist and the importance of evaluating and reporting psychometric instruments, this study aims to review the revised forms of COSMIN checklist.
Methods
In this narrative review study, a search was conducted in international databases (PubMed, Scopus, Proquest, Web of Science, and CINAHL), and national databases (Magiran, IranDoc, ISC, and SID) for all articles that used COSMIN checklist to measure the quality of psychometric studies using related keywords in Persian and English without considering time limitations
Results
In the first study using the Delphi method (2006), a working group consisting of 57 experts in epidemiology, measurement, statistics, and biostatistics, introduced a classification of characteristics for instruments in three areas of validity (including content and face validity, structural validity, hypothesis testing, cross-cultural validity, and criterion validity), reliability (including internal consistency, reliability, and measurement error) and responsiveness (including responsiveness and interpretability). Considering the limitations of the initial version, the COSMIN committee changed it to strengthen its reliability and ease of use. These changes led to creating four versions including (1) the COSMIN Risk of Bias Checklist, (2) the COSMIN Risk of Bias Tool to assess the quality of studies by focusing on reliability and measurement error, (3) the COSMIN Study Design Checklist, and (4) the COSMIN Reporting Guideline. With the introduction of new checklists, the COSMIN committee advised the researchers not to use the original version and only use the newly introduced checklists according to the study objectives.
Discussion
Considering the need to use valid and reliable PRO measures in research and clinic settings, as well as the increasing number of instruments for PRO constructs, there was a need for a scale to provide criteria for measuring the psychometric properties of the instruments. In this regard, the COSMIN checklists were introduced. However, they have some limitations such as being specialized and the need for teaching how to complete it, being opinion-based (not evidence-based), lack of sufficient evidence to confirm the reliability of the checklist, failure to determine the importance of some psychometric properties compared to others, and lack of assessing essential details in evaluation of some psychometric properties. In overall, COSMIN checklists can provide logical indicators and standards for measuring the psychometric properties of instruments; therefore, they can be used for measuring the quality of psychometric studies, designing studies, or reporting psychometric studies by researchers, reviewers, and journal editors.
Ethical Considerations
Compliance with ethical guidelines
This study was approved by the Ethics Committee of Kashan University of Medical Sciences (Code: IR.KAUMS.NUHEPM.REC.1401.047).
Funding
This research did not receive any grant from funding agencies in the public, commercial, or non-profit sectors.
Authors contributions
All authors equally contributed to preparing this article.
Conflicts of interest
The authors declared no conflict of interest.
Acknowledgements
In this way, all the researchers, especially COSMIN researchers, whose articles were reviewed in this study, as well as the Ethics Committee of Kashan University of Medical Sciences and Healthcare Services, are thanked and appreciated.
Type of Study: Review Article |
Subject:
اپیدمیولوژی
Received: 2022/10/16 | Accepted: 2022/11/14 | Published: 2023/01/1
Received: 2022/10/16 | Accepted: 2022/11/14 | Published: 2023/01/1
References
1. Prinsen CAC, Mokkink LB, Bouter LM, Alonso J, Patrick DL, de Vet HCW, et al. COSMIN guideline for systematic reviews of patient-reported outcome measures. Qual Life Res. 2018; 27(5):1147-57. [DOI:10.1007/s11136-018-1798-3] [PMID] [PMCID] [DOI:10.1007/s11136-018-1798-3]
2. Griffiths C, Armstrong-James L, White P, Rumsey N, Pleat J, Harcourt D. A systematic review of patient reported outcome measures (PROMs) used in child and adolescent burn research. Burns. 2015; 41(2):212-24. [DOI:10.1016/j.burns.2014.07.018] [PMID] [DOI:10.1016/j.burns.2014.07.018]
3. Ritmala-Castren M, Lakanmaa RL, Virtanen I, Leino-Kilpi H. Evaluating adult patients' sleep: An integrative literature review in critical care. Scand J Caring Sci. 2014; 28(3):435-48.[DOI:10.1111/scs.12072] [PMID] [DOI:10.1111/scs.12072]
4. Mokkink LB, Terwee CB, Knol DL, Stratford PW, Alonso J, Patrick DL, et al. Protocol of the COSMIN study: COnsensus-based Standards for the selection of health Measurement INstruments. BMC Med Res Methodol. 2006; 6:2.[DOI:10.1186/1471-2288-6-2] [PMID] [PMCID] [DOI:10.1186/1471-2288-6-2]
5. Mokkink LB, Terwee CB, Stratford PW, Alonso J, Patrick DL, Riphagen I, et al. Evaluation of the methodological quality of systematic reviews of health status measurement instruments. Qual Life Res. 2009; 18(3):313-33. [DOI:10.1007/s11136-009-9451-9] [PMID] [DOI:10.1007/s11136-009-9451-9]
6. Terwee CB, Mokkink LB, Knol DL, Ostelo RW, Bouter LM, de Vet HC. Rating the methodological quality in systematic reviews of studies on measurement properties: A scoring system for the COSMIN checklist. Qual Life Res. 2012; 21(4):651-7. [DOI:10.1007/s11136-011-9960-1] [PMID] [PMCID] [DOI:10.1007/s11136-011-9960-1]
7. Mokkink LB, Terwee CB, Knol DL, Stratford PW, Alonso J, Patrick DL, et al. The COSMIN checklist for evaluating the methodological quality of studies on measurement properties: A clarification of its content. BMC Med Res Methodol. 2010; 10:22. [DOI:10.1186/1471-2288-10-22] [PMID] [PMCID] [DOI:10.1186/1471-2288-10-22]
8. Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, et al. The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: An international Delphi study. Qual Life Res. 2010; 19(4):539-49. [DOI:10.1007/s11136-010-9606-8] [PMID] [PMCID] [DOI:10.1007/s11136-010-9606-8]
9. Haywood KL, Garratt AM, Fitzpatrick R. Quality of life in older people: a structured review of generic self-assessed health instruments. Qual Life Res. 2005; 14(7):1651-68. [DOI:10.1007/s11136-005-1743-0] [PMID] [DOI:10.1007/s11136-005-1743-0]
10. Wind H, Gouttebarge V, Kuijer PP, Frings-Dresen MH. Assessment of functional capacity of the musculoskeletal system in the context of work, daily living, and sport: A systematic review. J Occup Rehabil. 2005; 15(2):253-72. [DOI:10.1007/s10926-005-1223-y] [PMID] [DOI:10.1007/s10926-005-1223-y]
11. Marinus J, Ramaker C, van Hilten JJ, Stiggelbout AM. Health related quality of life in Parkinson's disease: A systematic review of disease specific instruments. J Neurol Neurosurg Psychiatry. 2002; 72(2):241-8. [DOI:10.1136/jnnp.72.2.241] [PMID] [PMCID] [DOI:10.1136/jnnp.72.2.241]
12. Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, et al. The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes. J Clin Epidemiol. 2010; 63(7):737-45. [DOI:10.1016/j.jclinepi.2010.02.006] [PMID] [DOI:10.1016/j.jclinepi.2010.02.006]
13. Tate RL. Measuring outcomes and monitoring progress in the era of evidence-based clinical practice. Brain Impair. 2019; 20(3):276-88. [DOI:10.1017/BrImp.2019.28] [DOI:10.1017/BrImp.2019.28]
14. Terwee CB, Jansma EP, Riphagen II, de Vet HC. Development of a methodological PubMed search filter for finding studies on measurement properties of measurement instruments. Qual Life Res. 2009; 18(8):1115-23. [DOI:10.1007/s11136-009-9528-5] [PMID] [PMCID] [DOI:10.1007/s11136-009-9528-5]
15. Beattie M, Lauder W, Atherton I, Murphy DJ. Instruments to measure patient experience of health care quality in hospitals: A systematic review protocol. Syst Rev. 2014; 3:4. [DOI:10.1186/2046-4053-3-4] [PMID] [PMCID] [DOI:10.1186/2046-4053-3-4]
16. Lenz ER. Measurement in nursing and health research. New York: Springer Publishing Company; 2010. [Link]
17. Terwee CB, Bot SD, de Boer MR, van der Windt DA, Knol DL, Dekker J, et al. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol. 2007; 60(1):34-42. [DOI:10.1016/j.jclinepi.2006.03.012] [PMID] [DOI:10.1016/j.jclinepi.2006.03.012]
18. Mokkink LB, de Vet HCW, Prinsen CAC, Patrick DL, Alonso J, Bouter LM, et al. COSMIN Risk of Bias checklist for systematic reviews of Patient-Reported Outcome Measures. Qual Life Res. 2018; 27(5):1171-9. [DOI:10.1007/s11136-017-1765-4] [PMID] [PMCID] [DOI:10.1007/s11136-017-1765-4]
19. COSMIN. Checklists for Assessing Study Qualities [Internet] 2022 [Updated 2023 April 18]. [Available from: [Link]
20. Mokkink LB, Boers M, van der Vleuten CPM, Bouter LM, Alonso J, Patrick DL, et al. COSMIN Risk of Bias tool to assess the quality of studies on reliability or measurement error of outcome measurement instruments: A Delphi study. BMC Med Res Methodol. 2020; 20(1):293. [DOI:10.1186/s12874-020-01179-5] [PMID] [PMCID] [DOI:10.1186/s12874-020-01179-5]
21. Mokkink LB, Prinsen CA, Patrick DL, Alonso J, Bouter LM, de Vet HC, et al. COSMIN study design checklist for patient-reported outcome measurement instruments. Gut. 2020; 70:1-32. [DOI:10.1136/gutjnl-2020-320729] [PMID] [DOI:10.1136/gutjnl-2020-320729]
22. Higgins JPT, Green S. Cochrane handbook for systematic reviews of interventions. 2011 [Updated 2011 March 11]. Available from: [Link]
23. Terwee CB, Prinsen CAC, Chiarotto A, Westerman MJ, Patrick DL, Alonso J, et al. COSMIN methodology for evaluating the content validity of patient-reported outcome measures: A Delphi study. Qual Life Res. 2018; 27(5):1159-70. [DOI:10.1007/s11136-018-1829-0] [PMID] [PMCID] [DOI:10.1007/s11136-018-1829-0]
24. Qin S, Nelson L, McLeod L, Eremenco S, Coons SJ. Assessing test-retest reliability of patient-reported outcome measures using intraclass correlation coefficients: Recommendations for selecting and documenting the analytical formula. Qual Life Res. 2019; 28(4):1029-33. [DOI:10.1007/s11136-018-2076-0] [PMID] [PMCID] [DOI:10.1007/s11136-018-2076-0]
25. Gagnier JJ, Lai J, Mokkink LB, Terwee CB. COSMIN reporting guideline for studies on measurement properties of patient-reported outcome measures. Qual Life Res. 2021; 30(8):2197-218. [DOI:10.1007/s11136-021-02822-4] [PMID] [DOI:10.1007/s11136-021-02822-4]
26. Bull C, Byrnes J, Hettiarachchi R, Downes M. A systematic review of the validity and reliability of patient-reported experience measures. Health Serv Res. 2019; 54(5):1023-35.[DOI:10.1111/1475-6773.13187] [PMID] [PMCID] [DOI:10.1111/1475-6773.13187]
27. McKenna SP, Heaney A. Setting and maintaining standards for patient-reported outcome measures: Can we rely on the COSMIN checklists? J Med Econ. 2021; 24(1):502-11. [DOI:10.1080/13696998.2021.1907092] [PMID] [DOI:10.1080/13696998.2021.1907092]
28. Szekeres M. Clinical relevance commentary in response to: The validity and clinical utility of the Disabilities of the Arm Shoulder and Hand questionnaire for hand injuries in developing country contexts: A systematic review. J Hand Ther. 2018; 31(1):91-2. [DOI:10.1016/j.jht.2017.12.002] [PMID] [DOI:10.1016/j.jht.2017.12.002]
29. Kwok EYL, Rosenbaum P, Thomas-Stonell N, Cunningham BJ. Strengths and challenges of the COSMIN tools in outcome measures appraisal: A case example for speech-language therapy. Int J Lang Commun Disord. 2021; 56(2):313-29.[DOI:10.1111/1460-6984.12603] [PMID] [DOI:10.1111/1460-6984.12603]
30. de Vet HC, Adèr HJ, Terwee CB, Pouwer F. Are factor analytical techniques used appropriately in the validation of health status questionnaires? A systematic review on the quality of factor analysis of the SF-36. Qual Life Res. 2005; 14(5):1203-18. [DOI:10.1007/s11136-004-5742-3] [PMID] [DOI:10.1007/s11136-004-5742-3]
Send email to the article author
| Rights and permissions | |
 |
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License. |
Owned by Qom University of Medical Sciences
Published by Qom University of Medical Sciences Press
eISSN: 2008-1375
pISSN: 1735-7799
Email: journal[at]muq.ac.ir
j.mua.ac[at]gmail.com
