The psychometric assessment of internalized stigma instruments: a systematic review

Sharon Anne Maria Stevelink, Ingeborg C Wu, Carlijn G.N. Voorend, Wim H. van Brakel

Abstract


Objective: Many instruments have been developed to assess stigma and related constructs across health conditions. The objective of this study was to provide an overview of the best validated instruments to measure internalized (or self-) stigma in health conditions where stigma plays a major role.

Method: A systematic literature review was conducted by searching relevant databases and by reviewing the bibliographies of relevant papers. Quantitative studies were included if the items used, or a sample of the instrument, were presented in the paper and if the studies focussed on the initial development of the instrument or a further validation. Health conditions included were HIV/AIDS, mental health, leprosy, asthma, epilepsy, cancer, obesity and tuberculosis. Psychometric properties of the included studies were assessed using several quality criteria: content validity, internal consistency, construct validity, criterion validity, reproducibility, responsiveness, floor and ceiling effects and interpretability. We rated the quality of the psychometric testing of the instruments using criteria proposed by Terwee et al. and the COSMIN consortium.

Results: Thirty-three papers were included of which 21 were identified as actual instrument development studies. Of the 21 instruments included only two instruments received three positive ratings on the quality criteria. Twelve  instruments were rated at least three times as indeterminate, especially for the criteria internal consistency and construct validity. This was due to insufficient samples sizes for factor analyses and hypotheses that were not formulated properly. At least one negative rating was given for five instruments. Content and construct validity as well as internal consistency were most often assessed, whereas agreement and responsiveness received least attention.

Conclusions: This review attempted to rate the available instruments to measure internalized stigma using current standard quality criteria for testing psychometric properties. Only the Child Attitude Towards Illness Scale and the Internalized Stigma of Mental Illness received three positive ratings on the quality criteria. Based on the results, we conclude that the majority of the instruments needs further testing of their psychometric properties.

Implications: The need was identified for a simplified testing protocol to design an instrument development study, to assess certain psychometric properties and to specify the preferred statistical methods for testing these. In addition, researchers should be aware that re-validation of instruments is necessary once application will take place in cultures and study populations other than the one(s) the instrument was initially developed for.


References


Adewuya, A. O., Owoeye, A. O., Erinfolami, A. O., & Ola, B. A. (2010). Correlates of self-stigma among outpatients with mental illness in Lagos, Nigeria. Int.J.Soc.Psychiatry.

Arole, S., Premkumar, R., Arole, R., Maury, M., & Saunderson, P. (2002). Social stigma: a comparative qualitative study of integrated and vertical care approaches to leprosy. Lepr.Rev., 73, 186-196.

Austin, J. K. & Huberty, T. J. (1993). Development of the Child Attitude Toward Illness Scale. J Pediatr.Psychol., 18, 467-480.

Austin, J. K., MacLeod, J., Dunn, D. W., Shen, J., & Perkins, S. M. (2004). Measuring stigma in children with epilepsy and their parents: instrument development and testing. Epilepsy Behav., 5, 472-482.

Barney, L. J., Griffiths, K. M., Christensen, H., & Jorm, A. F. (2010). The Self-Stigma of Depression Scale (SSDS): development and psychometric evaluation of a new instrument. Int.J Methods Psychiatr.Res., 19, 243-254.

Berger, B. E., Ferrans, C. E., & Lashley, F. R. (2001). Measuring stigma in people with HIV: Psychometric assessment of the HIV stigma scale. Research in Nursing & Health, 24, 518-529.

Brohan, E., Elgie, R., Sartorius, N., & Thornicroft, G. (2010a). Self-stigma, empowerment and perceived discrimination among people with schizophrenia in 14 European countries: The GAMIAN-Europe study. Schizophrenia Research, 122, 232-238.

Brohan, E., Gauci, D., Sartorius, N., & Thornicroft, G. (2011). Self-stigma, empowerment and perceived discrimination among people with bipolar disorder or depression in 13 European countries: The GAMIAN-Europe study. Journal of Affective Disorders, 129, 56-63.

Brohan, E., Slade, M., Clement, S., & Thornicroft, G. (2010b). Experiences of mental illness stigma, prejudice and discrimination: a review of measures. Bmc Health Services Research, 10.

Bunn, J. Y., Solomon, S. E., Miller, C., & Forehand, R. (2007). Measurement of stigma in people with HIV: A reexamination of the HIV stigma scale. Aids Education and Prevention, 19, 198-208.

Burgener, S. C. & Berger, B. (2008). Measuring perceived stigma in persons with progressive neurological disease: Alzheimer's dementia and Parkinson's disease. Dementia, 7, 31-53.

Corrigan, P. W. (1998). The impact of stigma on severe mental illness. Cognitive and Behavioral Practice, 5, 201-222.

Corrigan, P. W., Watson, A. C., & Barr, L. (2006). The self-stigma of mental illness: Implications for self-esteem and self-efficacy. Journal of Social and Clinical Psychology, 25, 875-884.

Dinos, S., Stevens, S., Serfaty, M., Weich, S., & King, M. (2004). Stigma: the feelings and experiences of 46 people with mental illness. Qualitative study. Br.J.Psychiatry, 184, 176-181.

Durso, L. E. & Latner, J. D. (2008). Understanding self-directed stigma: development of the weight bias internalization scale. Obesity (Silver.Spring), 16 Suppl 2, S80-S86.

Emlet, C. (2005). Measuring stigma in older and younger adults with HIV/AIDS: An analysis of an HIV stigma scale and initial exploration of subscales. Research on Social Work Practice, 15, 291-300.

Ersoy, M. A. & Varan, A. (2007). Reliability and validity of the Turkish version of the Internalized Stigma of Mental Illness Scale. Turk Psikiyatri Dergisi, 18, 163-171.

Esplen, M. J., Stuckless, N., Berk, T., Butler, K., & Gallinger, S. (2009a). The FAP self-concept scale (adult form). Fam.Cancer, 8, 39-50.

Esplen, M. J., Stuckless, N., Hunter, J., Liede, A., Metcalfe, K., Glendon, G. et al. (2009b). The BRCA Self-Concept Scale: a new instrument to measure self-concept in BRCA1/2 mutation carriers. Psychooncology., 18, 1216-1229.

Fife, B. L. & Wright, E. R. (2000). The dimensionality of stigma: A comparison of its impact on the self of persons with HIV/AIDS and cancer. Journal of Health and Social Behavior, 41, 50-67.

Franke, M. F., Munoz, M., Finnegan, K., Zeladita, J., Sebastian, J. L., Bayona, J. N. et al. (2010). Validation and Abbreviation of an HIV Stigma Scale in an Adult Spanish-Speaking Population in Urban Peru. Aids and Behavior, 14, 189-199.

Fung, K. M. T., Tsang, H. W. H., Corrigan, P. W., Lam, C. S., & Cheng, W. M. (2007). Measuring self-stigma of mental illness in China and its implications for recovery. International Journal of Social Psychiatry, 53, 408-418.

Heimlich, T. E., Westbrook, L. E., Austin, J. K., Cramer, J. A., & Devinsky, O. (2000). Brief report: Adolescents' attitudes toward epilepsy: further validation of the Child Attitude Toward Illness Scale (CATIS). J Pediatr.Psychol., 25, 339-345.

Herdman, M., Fox-Rushby, J., & Badia, X. (1998). A model of equivalence in the cultural adaptation of HRQoL instruments: the universalist approach. Qual.Life Res., 7, 323-335.

Holzemer, W. L., Uys, L. R., Chirwa, M. L., Greeff, M., Makoae, L. N., Kohi, T. W. et al. (2007). Validation of the HIV/AIDS stigma instrument-PLWA (HASI-p). Aids Care-Psychological and Socio-Medical Aspects of Aids/Hiv, 19, 1002-1012.

Jimenez, J. C., Puig, M., Ramos, J. C., Morales, M., Asencio, G., Sala, A. C. et al. (2010). Measuring HIV felt stigma: a culturally adapted scale targeting PLWHA in Puerto Rico. Aids Care-Psychological and Socio-Medical Aspects of Aids/Hiv, 22, 1314-1322.

Kalichman, S. C., Simbayi, L. C., Cloete, A., Mthembu, P. P., Mkhonta, R. N., & Ginindza, T. (2009). Measuring AIDS stigmas in people living with HIV/AIDS: the Internalized AIDS-Related Stigma Scale. AIDS Care, 21, 87-93.

Kanter, J. W., Rusch, L. C., & Brondino, M. J. (2008). Depression self-stigma: a new measure and preliminary findings. J Nerv.Ment.Dis., 196, 663-670.

Kellison, I., Bussing, R., Bell, L., & Garvan, C. (2010). Assessment of stigma associated with attention-deficit hyperactivity disorder: Psychometric evaluation of the ADHD Stigma Questionnaire. Psychiatry Research, 178, 363-369.

Lillis, J., Luoma, J. B., Levin, M. E., & Hayes, S. C. (2010). Measuring weight self-stigma: the weight self-stigma questionnaire. Obesity (Silver.Spring), 18, 971-976.

Link, B. G., Mirotznik, J., & Cullen, F. T. (1991). The effectiveness of stigma coping orientations: can negative consequences of mental illness labeling be avoided? J Health Soc.Behav., 32, 302-320.

Link, B. G. & Phelan, J. C. (2001). Conceptualizing stigma. Annual Review of Sociology, 27, 363-385.

Link, B. G., Struening, E. L., Rahav, M., Phelan, J. C., & Nuttbrock, L. (1997). On stigma and its consequences: evidence from a longitudinal study of men with dual diagnoses of mental illness and substance abuse. J Health Soc.Behav., 38, 177-190.

Livingston, J. D. & Boyd, J. E. (2010). Correlates and consequences of internalized stigma for people living with mental illness: a systematic review and meta-analysis. Soc.Sci.Med., 71, 2150-2161.

Magasi, S. & Post, M. W. (2010). A comparative review of contemporary participation measures' psychometric properties and content coverage. Arch.Phys.Med.Rehabil., 91, S17-S28.

Mak, W. W. & Cheung, R. Y. (2010). Self-stigma among concealable minorities in Hong Kong: conceptualization and unified measurement. Am.J Orthopsychiatry, 80, 267-281.

Mokkink, L. B., Terwee, C. B., Knol, D. L., Stratford, P. W., Alonso, J., Patrick, D. L. et al. (2006). Protocol of the COSMIN study: COnsensus-based Standards for the selection of health Measurement INstruments. BMC.Med.Res.Methodol., 6, 2.

Mokkink, L. B., Terwee, C. B., Knol, D. L., Stratford, P. W., Alonso, J., Patrick, D. L. et al. (2010). The COSMIN checklist for evaluating the methodological quality of studies on measurement properties: a clarification of its content. BMC.Med.Res.Methodol., 10, 22.

Moses, T. (2009). Stigma and self-concept among adolescents receiving mental health treatment. Am.J Orthopsychiatry, 79, 261-274.

Pan, A. W., Chung, L., Fife, B. L., & Hsiung, P. C. (2007). Evaluation of the psychometrics of the Social Impact Scale: a measure of stigmatization. Int.J Rehabil.Res., 30, 235-238.

Petersen, H. V., Domanska, K., Bendahl, P. O., Wong, J., Carlsson, C., Bernstein, I. et al. (2011). Validation of a Self-Concept Scale for Lynch Syndrome in Different Nationalities. J Genet.Couns..

Rao, D., Choi, S. W., Victorson, D., Bode, R., Peterman, A., Heinemann, A. et al. (2009). Measuring stigma across neurological conditions: the development of the stigma scale for chronic illness (SSCI). Qual.Life Res., 18, 585-595.

Rensen, C., Bandyopadhyay, S., Gopal, P. K., & Van Brakel, W. H. (2010). Measuring leprosy-related stigma - a pilot study to validate a toolkit of instruments. Disabil.Rehabil, 33, 711-719.

Ritsher, J. B., Otilingam, P. G., & Grajales, M. (2003). Internalized stigma of mental illness: psychometric properties of a new measure. Psychiatry Research, 121, 31-49.

Sayles, J. N., Hays, R. D., Sarkisian, C. A., Mahajan, A. P.,

Spritzer, K. L., & Cunningham, W. E. (2008). Development and psychometric assessment of a multidimensional measure of internalized HIV stigma in a sample of HIV-positive adults. Aids and Behavior, 12, 748-758.

Scambler, G. (1998). Stigma and disease: changing paradigms. Lancet, 352, 1054-1055.

Sorsdahl, K. R., Mall, S., Stein, D. J., & Joska, J. A. (2011). The prevalence and predictors of stigma amongst people living with HIV/AIDS in the Western Province. AIDS Care, 1-6.

Sowell, R. L., Lowenstein, A., Moneyham, L., Demi, A., Mizuno, Y., & Seals, B. F. (1997). Resources, stigma, and patterns of disclosure in rural women with HIV infection. Public Health Nurs., 14, 302-312.

Stevelink, S. A., Van Brakel, W. H., & Augustine, V. (2011). Stigma and social participation in Southern India: Differences and commonalities among persons affected by leprosy and persons living with HIV/AIDS. Psychol.Health Med., 1-13.

Terwee, C. B., Bot, S. D. M., De Boer, M. R., van der Windt, D. A. W. M., Knol, D. L., Dekker, J. et al. (2007). Quality criteria were proposed for measurement properties of health status questionnaires. Journal of Clinical Epidemiology, 60, 34-42.

Van Brakel, W. H. (2006). Measuring health-related stigma--a literature review. Psychol.Health Med., 11, 307-334.

Van Rie, A., Sengupta, S., Pungrassami, P., Balthip, Q.,

Choonuan, S., Kasetjaroen, Y. et al. (2008). Measuring stigma associated with tuberculosis and HIV/AIDS in southern Thailand: exploratory and confirmatory factor analyses of two new scales. Trop.Med.Int.Health, 13, 21-30.

Visser, M. J., Kershaw, T., Makin, J. D., & Forsyth, B. W. C. (2008). Development of parallel scales to measure HIV-Related stigma. Aids and Behavior, 12, 759-771.

Watson, A. C., Corrigan, P., Larson, J. E., & Sells, M. (2007). Self-stigma in people with mental illness. Schizophrenia Bulletin, 33, 1312-1318.

Weiss, M. G. (2008). Stigma and the social burden of neglected tropical diseases. PLoS Negl.Trop.Dis., 2, e237.

Weiss, M. G., Doongaji, D. R., Siddhartha, S., Wypij, D., Pathare, S., Bhatawdekar, M. et al. (1992). The Explanatory Model Interview Catalog (Emic) - Contribution to Cross-Cultural Research Methods from A Study of Leprosy and Mental-Health. British Journal of Psychiatry, 160, 819-830.

Weiss, M. G., Ramakrishna, J., & Somma, D. (2006). Health-related stigma: rethinking concepts and interventions. Psychol.Health Med., 11, 277-287.

World Health Organisation (2001). International Classification of Functioning, Disability and Health (ICF) Geneva: WHO.

Wright, K., Naar-King, S., Lam, P., Templin, T., & Frey, M. (2007). Stigma scale revised: Reliability and validity of a brief measure of stigma for HIV plus youth. Journal of Adolescent Health, 40, 96-98.


Full Text: PDF


Creative Commons License
This work is licensed under a Creative Commons Attribution-NoDerivs 3.0 License.