Posts

Shout-It-Now: Utilizing innovative technology to enhance mobile HIV counseling, testing and linkage-to-care in South Africa.

Authors: Joseph Daniels, PhD1, Arnošt Komárek, PhD2, Bruce Forgrieve, MBA3, Kathryn Pahl, PhD3, Stephen Stafford, BA3, Laurie Campbell Bruns, MPH1, Thomas Coates, PhD1

 

  1. Center for World Health, David Geffen School of Medicine, UCLA, USA
  2. Faculty of Mathematics and Physics, Charles University in Prague, Czech Republic
  3. Shout-It-Now, Cape Town, South Africa

Abstract:

Background: Mobile HIV counseling and testing (HCT) increases uptake by hard-to-reach and infrequent testers like men and adolescents in South Africa. There is limited understanding of effective mobile HCT programs utilizing tools like technology and telephonic linkage to care counseling to increase HIV testing and linkage-to-care rates. We examine data from the Shout-it-Now (S-N) program that uses such tools in South Africa.

Methods: S-N utilizes various forms of technology and ongoing telephonic counseling within a six-step program of HIV testing and linkage-to-care support, and program data was analyzed over an 18-month period. Summative statistics was conducted on participant registration, HIV risk assessment, HIV testing and linkage-to-care profiles. HIV prevalence and linkage-to-care estimates were calculated using the Clopper Pearson method.

Results: S-N tripled the number of participants who completed HIV testing compared to similar mobile HCT programs. There were 72,220 participants who completed the program with 40% of participants being men at each site. There were 3343 participants who tested HIV-positive, and there were higher proportions of women who tested HIV-positive. Among the HIV-positive participants, 1335 opted into the telephonic linkage-to-care support program, and there was a 72.95% linkage-to-care mean rate.

Interpretation: Integrating technology, edutainment and ongoing telephonic support increased HIV testing rates, HIV case detection, and linkage-to-care rates compared to similar HCT models while improving the referral and support system for linkage-to-care. S-N is highly efficient and offers a cost-effective model for a large scale up of community-based HCT services.

Author contributions:

All authors conceived and designed the study. BF, KP, and SS acquired the data. JD, AK, TC, SS, KP and BF analyzed and interpreted the data. JD drafted the manuscript. JD, TC, SS, KP, AK critically revised and approved the final manuscript. BF, KP and SS obtained the funding.

Role of Funding Source:

Shout-It-Now was funded via CDC COAG: 1U2GGH000285, “Strengthening Prevention in South Africa through Innovative Mobile HCT”.  The funding source had no role in: study design to include the collection, analysis and interpretation of data; writing the manuscript; and submitting it for publication.

Ethics Committee Approval:

The Shout-It-Now program received ethical approval from CDC/PEPFAR Office of the Associate Director for Science. Participants provided consent for HCT electronically via the S-N system.

 

Research in Context:

Evidence before this study

We searched PubMed for studies and meta-analysis conducted to understand the effectiveness of community-based HIV counseling and testing (HCT) interventions in sub-Saharan African contexts, with a particular focus on mobile HCT interventions with linkage-to-care data. We selected studies conducted in a single country context and multi-country contexts; sample sizes varied depending on the scale of intervention. Search criteria included HCT and sub-Saharan Africa, men, women, adolescents, youth, South Africa; Project Accept; mobile HCT; community-based HCT; and mobile HCT and linkage-to-care.

Added value of this study

This study of one mobile HCT model updates a meta-analysis of community-based HCT. As a technology-driven mobile HCT model, Shout-It-Now demonstrates significantly higher number of testers and linkage-to-care rates than comparable models.

Implications of all the available evidence

Mobile HCT can expand testing beyond the number of testers measured in current programs, if technology and culturally appropriate entertainment are utilized. Mobile HCT can provide linkage-to-care support with the adaptation of technology and then demonstrate outcomes in this area.

Introduction

The HIV epidemic remains generalized in South Africa with a sustained national HIV prevalence rate of 12.6% across all age ranges and 25% HIV prevalence among those 25-49 year old. More than 6.4 million South Africans are living with HIV/AIDS and many engage in behaviors that put them at risk of infecting others. While South Africa has demonstrated increased uptake of HIV counseling and testing (HCT) through both clinic and community centered initiatives, some estimate that as many as 50% of HIV positive people remain undiagnosed with continued needs to increase and diversify testing options.[1-6] To complement clinic-based testing initiatives, South Africa needs scalable and sustainable community-based HCT (CBHCT) models that foster routine testing habits and overcome the linkage-to-care challenges common among many CBHCT models.[7] The World Health Organization has recommended increasing the diversity of CBHCT models that work in conjunction with clinic-based HCT.[8] Common CBHCT models proven to increase HCT access include stand-alone centers and mobile units that serve venues such as shopping centers, transportation hubs, workplaces and schools.[9] Studies examining mobile HCT have shown increased uptake among first-time testers, men and youth who are reluctant to test in a clinic.[10-15] Mobile testing has also been proven to access larger numbers of individuals in both urban and rural settings, and can be cost-effective and acceptable with high quality service and accessibility.[1, 16, 17]

Current CBHCT models provide health education, counseling and testing services in tents or vans at community locations on a rotating basis.[18] People testing HIV-positive are subsequently referred to local clinics for HIV care. These models demonstrate high participation but have limited methods for tracking individuals from testing to linkage-to-care as few mobile HCT studies and programs that utilize technology to deliver participant education and track testing habits and linkage-to-care status. The challenge of linking HIV-positive clients diagnosed via CBHCT services to clinical care is well documented with some studies reporting 14% linkage-to-care rates by mobile CBHCT units.[3] Even referrals from hospitals to health centers have demonstrated a high attrition rate.[19] Among the key factors that have been found to impede linkage-to-care are participant economic barriers for clinic travel and poor administrative links between CBHCT sites and clinics.[20-22] Some CBHCT providers have sought to improve linkage-to-care by building referral relationships between themselves and clinics and by using approaches like referral letters and home visits to confirm linkage-to-care.[23, 24] However, these approaches have not effectively in addressed the mobile HIV testing and linkage-to-care gap.

We present the Shout-it-Now (S-N) program of mobile CBHCT, which utilizes a mix of client-centered services and technology to increase CBHCT scalability, acceptability and accountability. S-N differs from most mobile HCT providers in that it uses process-driven procedures supported by technology. We analyzed data collected over 18 months of S-N implementation (January 2012-June 2013) to assess: 1) HIV testing rates and 2) linkage-to-care rates. We discuss how S-N improves on the current mobile CBHCT model for deployment at a larger scale to successfully engage clients who don’t access HCT in clinical settings.

 

Methods

The S-N program combines technology (i.e. biometrics, video edutainment, telephonic linkage-to-care counseling) with culturally competent HCT services and a software system that seamlessly supports staff and clients through every step of the S-N process from registration to linkage to care. Technology is utilized to streamline organizational processes, educate participants, and support participants through each step of the program. The cost per participant is $17, and this is comparable to similar mobile HCT models and lower than stand-alone HCT.[5, 25]

Sites and Participant Recruitment. S-N operated two 14-person mobile teams in two South African districts: Tshwane district (an urban area including the City of Pretoria, population 2.9 million) in Gauteng province and Capricorn district (a rural area, population 1.3 million) in Limpopo province. The teams set up in shopping centers, markets, and central business districts, utilizing a prescribed operations protocol. Eight to ten counselor tents (3X3 meters) were pitched in a horseshoe formation. At the entrance were two additional tents, one for registration and one for a computer lab where participants watched S-N’s educational/risk assessment video on laptops. S-N sites were made highly visible, with colorful banners, music playing through a public announcement system, and S-N staff engaged passersby about the program and invited them to participate. Participants were 15-49 years of age and not asked about their previous testing history or HIV status.

Shout-it-Now Model. Six sequential steps form the S-N HCT program and are explained below. The first five steps were conducted during the HCT encounter at the mobile site and the final sixth step was conducted telephonically for up to seven weeks after a HIV-positive diagnosis via a centralized call center. For every client, each step was tracked using technology and the result of each step was automatically collected for program monitoring and evaluation via S-N’s integrated software program.

  1. Biometric Registration. The registration process begins with fingerprint registration that links clients anonymously and confidentially to a profile they build as they progress through the six-step S-N process. Once their fingerprint is registered, they are issued a unique identifier printed on a wristband that they then wear to track progression through the S-N process. Participants then complete a brief online demographic profile including socio-economic measures, cultural and racial identity, date of birth, and language. S-N’s use of biometric technology is supported by studies demonstrating that such technology is feasible and acceptable in low-resource settings and has been shown to be a significant factor in the engagement of participants in HIV testing and counseling and linkage-to-care.[24-26]

 

  1. HIV Edutainment. Online social networking, gaming, edutainment and telenovelas have been successfully used to deliver health knowledge, education and messaging.[27-32] S-N’s HIV education content is delivered by a 13-minute edutainment video viewed by each client on a personal laptop. The video features young South African celebrities (sports stars, musicians, designers and TV personalities), animation, dance and docudrama to teach clients about HIV risk, testing, disease development and living healthy with HIV. In the video, messages addressing both men’s and women’s HIV risk factors were delivered by men and women in order to create an inclusive educational experience.[7]

 

  1. Online Risk Assessment and Culturally Competent Pre-test Counseling. Embedded in the educational video was a set of HIV risk assessment questions. Participants were informed that questions about their health behaviors and opinions would appear during the video and that by answering these questions, they could improve the relevance and quality of their counseling session. The HIV risk assessment questions addressed sexual risk, TB and STI history, medical history, lifestyle, physical health, and medical male circumcision. The video ran automatically and was available in all local languages with counselors assisting participants who had low literacy levels. At the end of the video, clients were asked if they would like to consent for an HIV test.

 

Once a client consented to HIV testing, participants met privately with a counselor trained in key population cultural competence in a tent for pre-test counseling and testing. Counselors reviewed risk assessment profiles with their clients and then assessed their level of comprehension of educational content. The online risk assessments helped counselors tailor their HIV risk reduction messaging to each client’s needs. When pre-test counseling was completed, counselors asked the participant again if they consented to HIV testing. If participants declined testing, the counselor tried to motivate the client to accept testing by addressing the reasons for refusal but if the client could not be convinced to test, they exited the program at this point. Participants consenting to HIV testing were then tested.

 

  1. HIV testing. Counselors administered rapid HIV testing using kits certified by the South African National Department of Health and supplied by the respective provincial Departments of Health. There were two steps in the S-N testing process. First, a counselor collected a specimen using the test kit, which was linked to the client by a sticker bearing their unique identifier already stored in the S-N database. Participants were then escorted to a waiting area with activities and music videos while the test was processed. Second, the test kit and specimen were delivered to a dedicated staff member who read the HIV test result (the S-N database time stamped all test kits to protect against HIV test results being read too early or too late) and recorded the result in the S-N database by scanning the barcode on the test kit and inputting the result.

 

  1. Post-HIV test results counseling. The S-N database notified counselors on their laptops when their clients’ results were accessible via the client’s electronic record. At that time, counselors called clients back to their tent to deliver results and post-test counseling using a script specific to the client’s HIV result.

 

HIV-Negative Results: Counselors followed a 3-step procedure: 1) deliver the result and ascertain accurate comprehension of the result; 2) encourage routine HIV testing; 3) summarize the participant’s risk profile and help them devise a prevention plan to minimize their HIV risk.

HIV-Positive Results: Counselors followed a multi-step procedure: 1) deliver the preliminary test result and ascertain accurate comprehension of the result; 2) administer a different rapid HIV test to confirm results; 3) address individual concerns and questions; and 4) confirm results of second test. If the second test was negative, participants were referred to a clinic for a laboratory confirmation test. If the second test was positive, the counselor proceeded with a second set of steps:  1) deliver the HIV-positive diagnosis in a clear unambiguous way; 2) check for understanding of the result; 3) allow for emotional response; 4) assess participant’s support system; 5) discuss disclosure; 6) emphasize importance of accessing HIV care (including the importance of CD4 testing); and 7) discuss prevention options. After these steps were completed, individuals testing HIV-positive were provided a referral letter to a local clinic for follow-up care and were encouraged to participate in S-N’s ongoing linkage-to-care Call Center service.

 

  1. Telephonic Linkage-to-care Support Program. Participants opting to receive ongoing linkage-to-care services were provided free telephonic support from care coordinators at the S-N Call Center. Care coordinators had training in linkage-to-care counseling and were fluent in all South African languages. Within 48 hours of an HIV-positive diagnosis, a care coordinator called the client and commenced the process of motivating them to link to HIV care. Care coordinators used the participant’s e-profile stored in the S-N database to inform the telephonic support they provided to the participant until they linked to HIV care. Generally this support was provided up to seven weeks. Documentation of linkage to care was from the clients’ self report and was verified by descriptions of clinic-delivered CD4 results or enrollment in ART or in wellness programs.

 

Data analysis. Quantitative data from participant registration, HIV risk assessment, HIV testing and linkage-to-care profiles was analyzed to generate summative statistics using R software.[26] For HIV prevalence and linkage-to-care estimates, binomial confidence intervals were calculated using the Clopper Pearson method.

 

Results

HIV Testing Engagement. A total of 72,220 individuals participated in Shout-It-Now across the two districts from June 2012-June 2013 (Table I) with an average completion time of 30 minutes per participant. There were higher numbers of women than men who completed the S-N program though the representation of men was over 40% in each site. The highest representation of participants by age was the 20-29 age group at 56.47% in Gauteng and 59.08% in Limpopo. Participant representation decreased sequentially from the 30-39, 15-19 and 40-49 age groups.

HIV Case Findings. In Table II, HIV prevalence rates by sex and age is shown along with confidence intervals. In this analysis of data collected over one year, the S-N program identified 3343 HIV-positive participants at 4.6% HIV prevalence. The HIV prevalence rates among testers in S-N were higher among women who were 30 years of age or older compared to men. In this age range, the HIV prevalence rate for women was 8.5-13.0% across the sites whereas it was 5.6-6.9% for men in the same age categories. The HIV prevalence rate range for both men and women from 15-29 years of age was 0.6-5.3% with 2.38% as the average rate. Most S-N participants tested HIV-negative across the sites, and the confidence intervals remained small for HIV testing based on gender alone with no distinct HIV prevalence rate differences between the two sites.

Linkage-to-Care. The distribution of those who opted into the linkage-to-care (LTC) support program by sex and age are represented in Table III. Out of the 3343 participants who tested HIV-positive through S-N, 1335 (39.93%) opted into the LTC program with the highest representation by women between the ages of 20-39 years. Participants who declined LTC support were either already in HIV care, lost to follow-up due to change in contact information, or not interested in being referred for care. Of those participating in the LTC program, 974 participants linked to care within seven weeks of completing the program. The range of linkage-to-care rates for adult men and women in both Gauteng and Limpopo are over 25.0-95.56% with a 72.95% mean rate for linkage-to-care, Table IV. Men and women in Limpopo had the lower linkage-to-care rates compared to Gauteng participants with the lowest rates among adolescent men (25.00%) and men over 40 years of age (35.59%). Otherwise, there was no distinctive difference in linkage-to-care rates between men and women across the sites in each age range over 20 years old.

 

Discussion

The findings from the Shout-It-Now (S-N) program demonstrate exceptionally high HIV testing rates and above-average linkage-to-care rates among underserved communities including men and youth who are less likely to test.[1, 2, 4, 24] This HCT model nearly tripled the average number of participants compared to similar models over an 18-month period and maintained a high representation of both men and women including adolescents 15-17 years of age.[18] Further, the linkage-to-care rates for S-N maintain a mean of 72%, which is 58% higher of any reported model using a referral mechanism.[23]

High numbers of HIV-testers and HIV-cases detected. There was a 4.6% mean HIV prevalence across age, sex and site among testers in the S-N model with rates for women comparable to their national prevalence in South Africa.[2] Although the mean HIV prevalence is lower than the national prevalence, the number of HIV testers in S-N per site was significantly higher than comparable clinic-based testing methods and other mobile models.[17, 27] Further, HIV testing rates for men was less than women in S-N but high compared to national rates. This finding supports studies that show mobile testing increases the numbers of men in HCT and knowing their results within a context where 17% of men have ever tested for HIV.[1] With high numbers of HIV testers, the number of HIV-positive cases detected was high in S-N as well, such as 20-39 year old women with nearly double the number of cases detected compared to clinics.[17]

Improved linkage-to-care rates with referral support. Single method referral mechanisms like referral letters to HIV clinics used by many community-based HCT providers have demonstrated an average 14% linkage-to-care rate.[18] Utilizing a multi-method approach that included client-focused post-test counseling, referral letters to local clinics and ongoing telephonic support, there was a 72% mean linkage-to-care rate for men and women who participated in the LTC program through S-N. The S-N e-profile that participants developed when they completed the mobile HCT increased participant tracking efficiency, and this allowed care coordinators to engage participants in tailored telephonic linkage-to-care counseling. Further, there is increased need to develop novel and engaging approaches to address late presentation for HIV treatment in clinics.[1, 10] S-N demonstrated that multi-method approach that included telephonic support leads to higher rates of linkage-to-care, which has not been shown to our knowledge especially for those hard to reach like men and adolescents.[28-30]

S-N program addresses mobile HCT gaps. Technology allows mobile HIV testing programs to build voluntary participant tracking and support mechanisms from HCT to HIV care, which has been a challenge to implement and demonstrate outcomes.[4, 23] We show how technology allows participants to build an S-N e-profile as they proceed through the program steps. This e-profile gives the program data to provide tailored HIV testing and support for linkage-to-care, and participants may choose to retain their profiles if they plan to test with the program at a later date. These findings suggest that S-N engaged participants by using technology and educational media, which resulted in nearly triple the number of testers compared to any reported mobile model.[31-34] Technology applications used by S-N demonstrated high numbers along the continuum of care, and this technology was easily adaptable to urban and rural areas with high HIV prevalence and low numbers of testers.[2, 4]

 

Conclusion

Scaling up affordable HCT models that effectively engage large numbers of people to learn their HIV status and then efficiently link them to clinical care is a key priority in the global AIDS response. A highly organized service delivery process, use of multiple technologies to engage clients in education and track their HIV testing progression, risk profiles and test results can improve the effectiveness and scalability of HCT programs. We show how the Shout-It-Now (S-N) program is cost-effective compared to similar mobile HCT programs and addresses the test-and-linkage-to-care gap in many mobile HCT models. S-N is highly efficient and offers a cost-effective model for a large scale up of community-based HCT services.

 

Acknowledgements

The Shout-It-Now program received ethical approval from CDC/PEPFAR Office of the Associate Director for Science. Participants provided consent for HCT electronically via the S-N system.

 

Conflict of Interest:

Authors state that there are no conflicts of interest.

 

References

  1. van Rooyen H, McGrath N, Chirowodza A, Joseph P, Fiamma A, Gray G, et al. Mobile VCT: reaching men and young people in urban and rural South African pilot studies (NIMH Project Accept, HPTN 043). AIDS Behav 2013,17:2946-2953.
  2. Govindasamy D, Kranzer K, Ford N. Strengthening the HIV cascade to ensure an effective future ART response in sub-Saharan Africa. Trans R Soc Trop Med Hyg 2014,108:1-3.
  3. Govindasamy D, Meghij J, Kebede Negussi E, Clare Baggaley R, Ford N, Kranzer K. Interventions to improve or facilitate linkage to or retention in pre-ART (HIV) care and initiation of ART in low- and middle-income settings–a systematic review. J Int AIDS Soc 2014,17:19032.
  4. Govindasamy D, van Schaik N, Kranzer K, Wood R, Mathews C, Bekker LG. Linkage to HIV care from a mobile testing unit in South Africa by different CD4 count strata. J Acquir Immune Defic Syndr 2011,58:344-352.
  5. Grabbe KL, Menzies N, Taegtmeyer M, Emukule G, Angala P, Mwega I, et al. Increasing access to HIV counseling and testing through mobile services in Kenya: strategies, utilization, and cost-effectiveness. J Acquir Immune Defic Syndr 2010,54:317-323.
  6. Kranzer K, van Schaik N, Karmue U, Middelkoop K, Sebastian E, Lawn SD, et al. High prevalence of self-reported undiagnosed HIV despite high coverage of HIV testing: a cross-sectional population based sero-survey in South Africa. PLoS One 2011,6:e25244.
  7. Nachega JB, Uthman OA, del Rio C, Mugavero MJ, Rees H, Mills EJ. Addressing the Achilles’ heel in the HIV care continuum for the success of a test-and-treat strategy to achieve an AIDS-free generation. Clin Infect Dis 2014,59 Suppl 1:S21-27.
  8. Organization WH. Service delivery approaches to HIV testing and couseling (HTC): A strategic HTC programme framework. In. Geneva, Switzerland; 2012.
  9. van Schaik N, Kranzer K, Wood R, Bekker LG. Earlier HIV diagnosis–are mobile services the answer? S Afr Med J 2010,100:671-674.
  10. Hensen B, Taoka S, Lewis JJ, Weiss HA, Hargreaves J. Systematic review of strategies to increase men’s HIV-testing in sub-Saharan Africa. AIDS 2014.
  11. Hutchinson PL, Mahlalela X. Utilization of voluntary counseling and testing services in the Eastern Cape, South Africa. AIDS Care 2006,18:446-455.
  12. Maughan-Brown B, Nyblade L. Different dimensions of HIV-related stigma may have opposite effects on hiv testing: evidence among young men and women in South Africa. AIDS Behav 2014,18:958-965.
  13. Peltzer K, Matseke G, Mzolo T, Majaja M. Determinants of knowledge of HIV status in South Africa: results from a population-based HIV survey. BMC Public Health 2009,9:174.
  14. Tabana H, Doherty T, Swanevelder S, Lombard C, Jackson D, Zembe W, et al. Knowledge of HIV status prior to a community HIV counseling and testing intervention in a rural district of south Africa: results of a community based survey. BMC Infect Dis 2012,12:73.
  15. Venkatesh KK, Madiba P, De Bruyn G, Lurie MN, Coates TJ, Gray GE. Who gets tested for HIV in a South African urban township? Implications for test and treat and gender-based prevention interventions. J Acquir Immune Defic Syndr 2011,56:151-165.
  16. Menzies N, Abang B, Wanyenze R, Nuwaha F, Mugisha B, Coutinho A, et al. The costs and effectiveness of four HIV counseling and testing strategies in Uganda. AIDS 2009,23:395-401.
  17. Sweat M, Morin S, Celentano D, Mulawa M, Singh B, Mbwambo J, et al. Community-based intervention to increase HIV testing and case detection in people aged 16-32 years in Tanzania, Zimbabwe, and Thailand (NIMH Project Accept, HPTN 043): a randomised study. Lancet Infect Dis 2011,11:525-532.
  18. Mabuto T, Latka MH, Kuwane B, Churchyard GJ, Charalambous S, Hoffmann CJ. Four models of HIV counseling and testing: utilization and test results in South Africa. PLoS One 2014,9:e102267.
  19. Nordberg E, Holmberg S, Kiugu S. Exploring the interface between first and second level of care: referrals in rural Africa. Trop Med Int Health 1996,1:107-111.
  20. Atkinson S, Ngwengwe A, Macwan’gi M, Ngulube TJ, Harpham T, O’Connell A. The referral process and urban health care in sub-Saharan Africa: the case of Lusaka, Zambia. Soc Sci Med 1999,49:27-38.
  21. Hardon AP, Akurut D, Comoro C, Ekezie C, Irunde HF, Gerrits T, et al. Hunger, waiting time and transport costs: time to confront challenges to ART adherence in Africa. AIDS Care 2007,19:658-665.
  22. Macintyre K, Hotchkiss DR. Referral revisited: community financing schemes and emergency transport in rural Africa. Soc Sci Med 1999,49:1473-1487.
  23. Nsigaye R, Wringe A, Roura M, Kalluvya S, Urassa M, Busza J, et al. From HIV diagnosis to treatment: evaluation of a referral system to promote and monitor access to antiretroviral therapy in rural Tanzania. J Int AIDS Soc 2009,12:31.
  24. van Rooyen H, Barnabas RV, Baeten JM, Phakathi Z, Joseph P, Krows M, et al. High HIV testing uptake and linkage to care in a novel program of home-based HIV counseling and testing with facilitated referral in KwaZulu-Natal, South Africa. J Acquir Immune Defic Syndr 2013,64:e1-8.
  25. Bassett IV, Govindasamy D, Erlwanger AS, Hyle EP, Kranzer K, van Schaik N, et al. Mobile HIV screening in Cape Town, South Africa: clinical impact, cost and cost-effectiveness. PLoS One 2014,9:e85197.
  26. Team RC. R: A language and environment for statistical computing. In. Vienna, Austria: R Foundation for Statistical Computing; 2014.
  27. Laeyendecker O, Piwowar-Manning E, Fiamma A, Kulich M, Donnell D, Bassuk D, et al. Estimation of HIV incidence in a large, community-based, randomized clinical trial: NIMH project accept (HIV Prevention Trials Network 043). PLoS One 2013,8:e68349.
  28. Crist JD. Theory derivation and the telenovela: facilitating Mexican-American family use of home healthcare. Nurs Sci Q 2008,21:39-40.
  29. Vaughn S. Stroke and heart disease prevention education via telenovela: a focus group’s evaluation. Rehabil Nurs 2012,37:215-219.
  30. Myburgh H. The clinic as a gender space: masculinities, health seeking behavior and HIV & AIDS. Consult African Intel 2011:1-5.
  31. Vaughan PW, Rogers EM, Singhal A, Swalehe RM. Entertainment-education and HIV/AIDS prevention: a field experiment in Tanzania. J Health Commun 2000,5 Suppl:81-100.
  32. Wilkin HA, Valente TW, Murphy S, Cody MJ, Huang G, Beck V. Does entertainment-education work with Latinos in the United States? Identification and the effects of a telenovela breast cancer storyline. J Health Commun 2007,12:455-469.
  33. Young SD. Social media technologies for HIV prevention study retention among minority men who have sex with men (MSM). AIDS Behav 2014,18:1625-1629.
  34. Coates TJ, Kulich M, Celentano DD, Zelaya CE, Chariyalertsak S, Chingono A, et al. Effect of community-based voluntary counselling and testing on HIV incidence and social and behavioural outcomes (NIMH Project Accept; HPTN 043): a cluster-randomised trial. Lancet Glob Health 2014,2:e267-277.

 

Table I: Characteristics of testers by site  
GAUTENG

N=40716

n(% of N)

LIMPOPO

N=31504

n(% of N)

 

TOTAL

N=72220

n(% of N)

 

GENDER  
Female 23009 (56.51) 17707 (56.22) 40721 (56.38)
Male 17712 (43.49) 13792 (43.78) 31499 (43.62)
AGE  
15-19 5658 (13.90) 6733 (21.37) 12391 (17.16)
20-29 22993 (56.47) 15777 (59.08) 38770 (53.68)
30-39 8169 (20.06) 5584 (17.72) 13753 (1904)
40-49 3896 (9.57) 3410 (10.82) 7306 (10.12)

 

 

 

Table II: HIV prevalence rates by sex, age
GUATENG LIMPOPO TOTAL
HIV-Positive

N = 2009

n(% of N)

HIV-Negative

N = 36840

n(% of N)

95% CI HIV-Positive

N = 1334

n(% of N)

HIV-Negative

N = 29296

n(% of N)

95% CI HIV-Positive

N = 3343

n(% of N)

HIV-Negative

N = 66136

n(% of N)

95% CI
FEMALE
15-19 79 (2.0) 3803 (98.0) 1.6, 2.5 69 (1.6) 4146 (98.4) 1.3, 2.1 148 (1.8) 7949 (98.2) 1.6, 2.1
20-29 679 (5.3) 12088 (94.7) 4.9, 5.7 381 (4.4) 8351 (95.6) 3.9, 4.8 1060 (4.9) 20439 (95.1) 4.6, 5.2
30-39 450 (13.0) 3022 (87.0) 11.9, 14.1 323 (12.3) 2310 (87.7) 11.1, 13.6 773 (12.7) 5332 (87.3) 11.8, 13.5
40-49 196 (10.6) 1654 (89.4) 9.2, 12.1 142 (8.5) 1533 (91.5) 7.2, 9.9 338 (9.6) 3187 (90.4) 8.6, 10.6
MALE
15-19 12 (0.8) 1531 (99.2) 0.4, 1.4 14 (0.6) 2344 (99.4) 0.3, 1.0 26 (0.7) 3875 (99.3) 0.4, 1.0
20-29 205 (2.2) 8985 (97.8) 1.9, 2.6 146 (2.2) 6456 (97.8) 1.9, 2.6 351 (2.2) 15441 (97.8) 2.0, 2.5
30-39 260 (6.1) 4033 (93.9) 5.4, 6.8 167 (6.0) 2610 (94.0) 5.2, 7.0 427 (6.0) 6643 (94.0) 5.5, 6.6
40-49 128 (6.9) 1724 (93.1) 5.8, 8.2 92 (5.6) 1546 (94.4) 4.6, 6.9 220 (6.3) 33270 (93.7) 5.5, 7.2
*Both men and women

 

 

Table III: HIV-positive cases by sex, age referred from site to LTC support program

Gauteng Limpopo Total
Female Male Female Male Female Male
15-19 37 3 28 4 65 7
20-29 257 90 159 57 416 147
30-39 135 85 130 72 265 157
40-49 71 51 97 59 168 110
Total 500 229 414 192 914 421

 

 

Table IV: Linkage to care rates by sex, age
  GUATENG LIMPOPO TOTAL FOR BOTH SITES
  Female Male Female Male Female Male
N = 451/500* N = 191/229 N = 239/414 N = 93/192 N = 690/914 N = 284/421
n(% of N) n(% of N) n(% of N) n(% of N) n(% of N) n(% of N)
15-19 32/37  (86.49) 2/3  (66.67) 17/28  (60.71) 1/4  (25.00) 49/65  (75.38) 3/7  (42.86)
20-29 232/257 (90.27) 77/90 (85.56) 104/159 (65.41) 32/57 (56.14) 336/416  (80.77) 109/147  (74.15)
30-39 129/135 (95.56) 78/85 (91.76) 81/130 (62.31) 39/72 (54.17) 210/265 (79.25) 117/157 (74.52)
40-49 58/71 (81.69) 34/51 (66.67) 37/97 (38.14) 21/59 (35.59) 95/168 (56.55) 55/110 (50.00)
* Numerator is reported linked to care. Denominator is total testing HIV-positive in S-N who opted in to the LTC program.

 

 

Using a Call Center to Expand Mobile HCT and Link HIV-positive Individuals to Care

by Michiel Adriaan van Zyl, Ph.D1, Leslie Lauren Brown, LCSW1,  Kathryn Pahl, Ph.D2,

1Kent School of Social Work, University of Louisville, 502-852-2430 | 2Shout-It-Now, Cape Town, South Africa

Engaging newly diagnosed HIV individuals in treatment is a significant global challenge. As South Africa expands HIV counseling and testing (HCT) services, the growing numbers of people diagnosed with HIV will need innovative links to care approaches in order for treatment to be most effective. While definitions vary, we have defined  “linkage to care” as connecting an HIV+ individual to medical care so that CD4 cell test results are obtained and ART eligibility assessed. The current study reveals findings from a non-governmental organization’s “Links to Care” program. A two-pronged expanded HCT service was used, which included a community outreach approach to address HIV testing and a call centre to track each patient’s linkage to care post HIV diagnosis. In the evaluation sample (n=1096), all participants were diagnosed as HIV positive, from either Limpopo or Gauteng provinces, with 95.5% of individuals reportedly being newly diagnosed. The majority of individuals (51%) were linked to care with a mean time to linkage of 31 days (with most individuals linked in less than 14 days). More females (54%) were linked to care than males (47%), and had higher CD4 cell counts than males; females had a mean CD4 cell count of 440, while males took longer to link to care and had a lower mean CD4 cell count of 331. Females 23 or younger had the lowest linkage rate of all females. Success rate differed by region, with 46.5% of individuals being linked to care in the rural area compared to 28% in the urban area. Reasons for the failure to link individuals to care are analyzed. Findings suggest that expanding HCT services to include innovative links to care approaches can improve linkage to care and subsequently impact HIV prevention.

Despite successful preventative efforts, South Africa continues to have the largest HIV/AIDS epidemic in the world (Mall, Middlekoop, Mark, Wood, & Bekker, 2013; van Rooyan, Barnabas, Baeten, Phakathi, Joseph, Krows, Hong, Murnane, Hughes, & Celum 2013). In order to maximize preventative efforts and curb transmission rates, more innovative and empirically sound interventions are needed. Research has shown HIV counseling and testing (HCT) to be instrumental in HIV prevention, yet very little literature has focused on ways to improve HCT and to link individuals to services after their HIV diagnosis. Mobile HCT enhances the traditional HIV testing model (typically only offered in health facilities) by providing community outreach and personalized HIV testing services; this approach aims to serve difficult-to-reach populations. Although antiretroviral therapy (ART) availability is integral to HIV treatment and prevention, the effectiveness of ART can be contingent upon the timeliness of linkage to care (Losina, Bassett, Giddy, Chetty, Regan, Walensky, Ross, Scott, Uhler, Kats, Holst, Freedberg, 2010), underscoring the need to improve services that impact linkage to care.

Preventative efforts are further confounded by the large number of individuals who are aware of their status but not engaged in care (De Koker, Lefevre, Matthys, van der Stuyft, & Delva, 2010; Van Zyl, Barney & Pahl, in Press). A recent study in a township in Durban, South Africa, found that 10% of participants in a community-based mobile testing linked to care after receiving an HIV diagnosis (Bassett et al, 2013). Because treatments such as ART rely on linkage to care, the effectiveness of HCT cannot be measured only by the numbers of those receiving HIV tests but rates of linkage to care should also be considered.

ART reduces morbidity by improving patient survival (Crum, Riffenburgh, Wegner, Agan, Tasker, Spooner, Armstrong, Fraser, & Wallace, 2006; Giordano, Gifford, White, Suarez, Almazor, Rabeneck, Hartman, Backus, Mole, & Morgan, 2007; Mayer, 2011), lowering infectiousness (Andrews, Wood, Bekker, Middlekoop, Walensky, 2012), and curbing secondary transmissions due to suppressed HIV loads (Jenness, Myers, Neaigus, Lulek, Navejas, & Raj-Singh, 2012 according to Quinn, Wawer, Sewankambo, Serwadda, Wabwire-Mangen, Mehan, Lutalo & Gray (2000). The scale up of ART has revolutionized AIDS responses by producing large-scale results, having already saved 9 million life-years is sub-Saharan Africa (UNAIDS, 2012), it is projected to save many more, since 70 percent of individuals maintaining ART treatment are expected to be alive after five years (Verguet, Lim, Murray, Gakidou, & Salomon, 2013). Conversely, delayed ART initiation is associated with hastened mortality and continued risky behavior, thus weakening preventative efforts. With the majority of individuals beginning ART too late, many researchers report an increased chance of lower CD4 counts. Fairall et al. (2008), Keiser et al. (2008), & Kigozi et al. (2009), Bassett et al., 2010 (as cited in Mugglin, Estill, Wandeler, Bender, Egger, Gsponer, & Keiser, 2012). Lawn et al. (2005) and Losina et al. (2010) (as cited in Kayigamba, Bakker, Fikse, Mugisha, & Asiimwe, 2012) report an increased risk of opportunistic infections and ultimately higher mortality rates. Even delaying treatment a few months can be deleterious for individuals with low CD4 cell counts, considering that more than half of the individuals who seek ART for the first time have a CD4 count <100 (Larson, Brennan, McNamara, Long, Rosen, Sanne, & Fox, 2010). A Center for Disease Control and Prevention (CDC) study in the United States found that almost 30% of HIV+ individuals delayed entry to medical care more than three months after being diagnosed (Reed, Hanson, McNaghten, Bertolli, Teshale, Gardner, & Sullivan, 2009).

The goal of the current study was to determine trends associated with linkage to care when using mobile HCT combined with a call-centre approach. Specific questions addressed were: (1) What percentage of HIV positive individuals reached in the mobile HCT program linked to care, and how does the demographic data vary for those linked to care and those not linked to care? (2) How long does it take to link clients to care and are rates in establishing linkage to care similar or different for various demographic groups? 

The Call-Centre Approach

Each day, the call centre receives the names and contact information of HIV+ individuals tested during mobile HCT outreach. Staff at the call centre make follow-up telephone calls and provide information on clinic enrolment in a supportive and understanding manner, while emphasizing the importance to link to care. Call centre staff continue to be in contact with patients until they are successfully linked to a clinic. This call centre approach is different from the way other call centres operate in the sense that most of the calls are outbound, though not exclusviely so: Individuals can also call into the call centre or send a free “please call me” text message, whereupon a call centre staff member will contact them.

Methods                                                                                                                                 

Research Setting and Procedures                                                                                                    

 This secondary data analysis reviews data collected during seven months of operation of an HIV Linkage to Care call centre. The sample was 1096 individuals who tested HIV+ during mobile HCT outreach in Limpopo and Gauteng provinces in South Africa between April 1, 2012 and October 31, 2012; data was gathered for an additional five weeks after the end of October in order to track success in linking individuals to care. “Linkage to care” was defined as connecting an HIV+ individual to medical care so that CD4 cell test results are obtained and ART eligibility assessed. Individuals whose CD4 cell count remained high were encouraged to continue counseling and follow-up services, while those meeting eligibility requirements were encouraged to begin ART. De-identified administrative data of the call centre was provided for analysis. The institutional review board at the University of Louisville in Louisville, Kentucky, U.S.A., reviewed this secondary study for human subject protection.

Analytical Procedures

Descriptive statistics were used to describe the sample, and differences between various cohorts were detected by means of t-tests and CHAID (Chi-squared Automatic Interaction Detector) analysis (e.g. age, gender, and language). CHAID is in essence a tree classification method that relies on the Chi-square test to determine the best next split at each step of the classification tree. CHAID is similar to regression analysis in that it selects the predictors that best account for the variance in a dependent variable, in this case linkage status (e.g. linkage status “linked” or “not linked”). However, CHAID goes a step further by identifying those variables (e.g. age, gender, and language) that most differentiate clients who are linked to care from those who are not linked to care.

Description of sample demographics                                                                                             

The majority of individuals (68.7%) referred to the call centre during the study period were female and 31.3% were male. The mean age was 32.82 years (SD=10.29) with males having a higher mean age (M=35.82; SD=10.9) than that of females (M=31.46;SD=9.71)(t(1092)=6.626; p<.001). English was the language of choice with 41.9%, followed by Sepedi (36.5%), Sesotho (4.9%), Tsonga (3.5%), Venda (2%), Setswana (1.6%) and isiZulu (.2%).The language was not known for 7% and the category “other language” accounted for less than 1%. The team from the Gauteng province call centre was responsible for 621 (56.7%) of individuals and the team from the Limpopo province dealt with 475 (43.3%) of individuals.

Results                                                                                                                       

Percentage of individuals linked to care and trends in rates and demographics differences  

Linkage to care was established for 563 (51.4%) of HIV+ individuals referred to the call centre during the seven-month sample. The large majority (95.9%) of those for whom linkages were established were newly diagnosed as HIV+. The HIV status of those not linked to care prior to current testing was unknown. More females (403 or 53.5% out of 753) than males (160 or 46.61% out of 343) (χ2=.037;df=1) were linked to care. The mean age (M=31.1; SD=9.96) of individuals linked to care was similar (M=32.52; SD=10.63) to those not linked to care (t(1092)=.936, p=.339). This was true for both males (linked M=36.5; SD=11.43; not linked M=35.24; SD=10.41; t(340)=1.059, p=.29) and females (linked M=31.77, SD=8.98; not linked M=31.1,SD=10.48; t(750)=.938,p=.349). The mean CD4 count for males was 331.06 (SD=205.53) and for females 439.72 (SD=231.75) (t(548)=5.107 , p<.001), indicating that males require on average more immediate ART intervention than females.

Using  the linkage status (Figure 1) (linked and not linked) as the dependent variable and age, gender and team” (urban and rural) as the independent variables, a CHAID analysis was conducted to identify significant differences in cohorts defined by categories of the independent variables. “Rural” referred to an area in Limpopo Province with intermediate population density, and “urban” to a metropolitan area in Gauteng Province. Gender was the most predictive of the three independent variables and more females were linked to service (53.5%) than males (46.6%). For females, age contributed to differences in linkage status. A relatively high percentage of people in the age group 33-43 was linked to care (61.8%), which was also true for people for whom age was unknown. Females under 23 was the female age group with the lowest percentage linked to care (41.8%). For this group, the rural team linked more (52.3%) young females to care than the urban team (32.9%).

Reasons for not being linked to care within 30 days of referral to the call centre were recorded for 442 people (See Figure 2). These individuals could have been linked to care within the seven-month period of the study, but the 30-day cut-off was used to identify a group at risk for not being linked to care. Of the 442 individuals, most (56.3%) were contacted many times but they still did not follow through with a visit to a clinic. The second reason most often recorded (17.9%) for not being linked to care in this group was incorrect contact information. Just over fourteen percent (14.5%) asked not to be called and 11.3% had no telephone, the only means available for the call centre to contact individuals. Reasons for not being able to link to care differed significantly for the two call centres. The urban team made a higher percentage of repeated attempts to contact individuals without success (64.7%) than the rural team (44.9%). The rural team had a higher percentage of individuals referred to them without a telephone (20.3%) as opposed to the urban team (4.7%).                                   

Time to link different cohorts of individuals to care

The mean time to link individuals was 31.1 days (SD=28.6).  A CHAID analysis was done by using “days to link” (with two categories) “0-14 Days” and “>14 Days” as the dependent variable. Independent variables were “month linked”, “CD4 count” and “team” (urban and rural) (See Figure 3). It took longer than 14 days for most people to be linked during April, May, and June (77.5% were linked after 14 days), but this changed for later months. During August, September, and October, 50.9% were linked to services within 14 days. The success achieved in later months, however, was not equal across all cohorts. Those with CD4 cell counts of <350 were not linked within the same time frame as those with higher CD4 cell counts during the period August to October. About 40% of individuals with low counts were linked within 14 days in August, September, and October, in contrast to about 60% of those with higher CD4 counts linked in the same timeframe. There were no differences in CD4 cell-count groups during other months. July appears to be a transitional month in the sense that the percentage of individuals linked within 14 days was higher than the percentage during the months of April, May, and June, but lower than the percentage of individuals linked during August, September, and October. July was also the only month with a significant difference in the percentage of individuals linked to care by the urban and rural teams, with the rural team linking 46.5% of individuals and the urban team 28%.

Summary and Discussion:              

Results (just over half of HIV+ individuals linked to care) compared favorably with previous studies on mobile HCT, which reported lower linkage to care despite services being free and in close proximity (Krazner, Zeinecker, Ginsbert, Orrell, Kalawe, Lawn, Bekker, & Wood, 2012; MacPherson, Corbett, Makombe, van Oosterhout, Manda, Choko, Thindwa, Swuire, Mann, & Lalloo, 2012). Bassett (2013) reported only 10% linkage to care for community-based mobile testers, although from a larger sample size and with a shorter collection period. Krazner et al. (2010), using a measurement period of six months, found higher rate of linkage among individuals testing through sexually transmitted infection services (84.1%) but found linkage rates for individuals testing on their own initiative were similar to what we found (53.4%).  Losina et al. (2010) found that although about 54% engaged in CD4 cell testing, only about 47% of those returned for their CD4 cell results. In our study, 4.1% of individuals reported having previously been diagnosed as HIV positive, while other similar studies found 6.5% (Larson et al., 2010) and 36% (van Rooyen et al., 2013) of their sample had been previously diagnosed.

Most individuals referred to the call centre during the data collection period were female, who were more likely to link to care than males; yet females under 23 had a lower percentage linked to care than all other female age groups (with the rural team linking more young females to care than the urban team). While the average amount of time for a patient to link to care was 31.1 days, linkage to care varied by month, with more individuals linking in less than 14 days during August, September, and October. July was also the only month with a significant difference in the percentage of individuals linked to care by the urban and rural teams, and more individuals linked to care in the warmer climate during this month. The average CD4 count for males was lower than females, indicating that more males in the group that were linked to care required ART interventions sooner than females.

There are several limitations to this study. First, the study does not include a comparison of linkage rates before and after the call centre approach was implemented. The study design does not include a comparison with other approaches to linking HIV+ individuals to care. Second, the study included only two provinces of South Africa. Findings cannot be generalized to other regions with different demographic compositions and infection incidence. Third, the study involved individuals in a seven-month timeframe. One possible seasonal factor was identified: individuals in a colder climate during the middle of winter linked at a lower rate. A study over 12 months may detect other possible seasonal patterns. Fourth, limited information was obtained for the group not linked to care. 

This study also had strengths. The sample size (n=1,096), although not large, is larger than many other HIV linkage to care studies that tracked individuals for a comparable length of time: Losina et al. (2010) with 454 HIV-positive patients in the Durban area; Rooyen et al. (2013) with 201 HIV-positive patients (also in KwaZulu-Natal). The sample we studied had the added advantage of diversity, as participants were from both urban and rural areas and included participants with a wide range in age and languages.

Study findings offer useful insights that can inform future studies about trends associated with linkage to care as well as delayed entry to care. Future studies should focus also on factors associated with delayed entry to care (Reed et al., 2009) and HIV+ individuals with CD4 counts of <350, who seem to take longer to link than individuals with a higher count. Also, as definitions of successful linkage to care presently vary, future studies should use a standardized definition of delayed care (Finnie et al., 2009); without such a benchmark, an important piece of HIV prevention may remain in the shadows.

Secondary data analysis to determine the reliability and validity of an adolescent HIV risk screening questionnaire.

By M.A. van Zyl, C. Studts, K Pahl

 

Abstract

Identification of adolescents at high risk for HIV infection in South Africa is a key component of current and future prevention efforts. In 2011, a non-governmental organization (NGO) administered a 12-item self-report risk measure to adolescents (N = 3,872) in South Africa as part of an innovative voluntary testing and counseling (VCT) intervention. Secondary data analyses employing item response theory (IRT) methods assessed the original 12-item measure, a reduced 7-item measure used by the NGO staff, and a 5-item measure developed in the current study. The 5-item measure demonstrated acceptable levels of reliability and validity, with all items discriminating sufficiently between adolescents at different levels of risk.

However, both uniform and non-uniform differential item functioning (DIF) were revealed as problems: items performed differently with groups differing by age, ethnicity, and gender.

Consequently age-, ethnicity-, and gender-specific percentile-based norms were developed. The IRT analyses also highlighted the extremely high levels of risk required for adolescents to select the highest response option (on four-point Likert-type scales) for each item. This was related to 14.5% of adolescents in the sample—primarily comprised of lower-socio economic high school students in in three regions of South Africa—indicating that they engaged in behaviors much more risky than the behaviors of their peers. Implications for policy and practice are discussed.

The identification of adolescents with high risk of contracting HIV offers the potential of providing preventative interventions targeted at the high risk HIV-negative adolescent population. A 12-item measure to identify adolescents at high risk for HIV infection was administered in 2011 by a non-governmental organization (NGO) as part of innovative voluntary testing and counseling (VCT) intervention to a group of adolescents (N = 3,872) in South Africa. NGO staff subsequently shortened the measure from 12 items to 7 items. This study retrospectively investigates the validity and reliability of the risk screening measures and proposes a new, briefer 5-item measure.

 

Background

Promising results of a pilot study (Van Zyl, Barney & Pahl, under review) of an innovative HIV prevention program targeted at adolescents, Shout-It-Now, led to wide scale implementation of the program in South Africa. Computers and internet access were made available to schools and community settings by the Shout-It-Now program and its sponsors, allowing adolescents to individually access online program content related to HIV prevention. Adolescents participated by viewing an online video of South African celebrities talking about issues related to HIV/AIDS prevention, including (a) condom usage, (b) Voluntary Counseling and Testing (VCT), (c) safer sex, and (d) responsible decision-making. Celebrities representing a variety of South African ethnic groups provided information in English, interspersed with local vernacular. The styles of the videos were similar to MTV television programs, using popular music and attention-retaining visual material. During the video a number of ‘pop-up’ questions appeared to reinforce messages being conveyed in the video program. The video took about 12 minutes to view, in line with anticipated attention spans and efforts to reduce participant burden.

Following the video presentation, adolescents were invited to participate in VCT. No coercion was placed on those who declined. Participants who had questions or concerns regarding the HIV test were invited to speak with a trained counselor. All who agreed to be tested for HIV were given a confidential one-on-one counseling session with a trained VCT counselor, followed by an HIV test with a registered nurse. In cases of conflicting or undefined results, a confirmatory test was conducted. Testing was conducted in accordance with UN guidelines and regulations of the South African government. On the same day as testing, participants were given their results within the framework of a confidential post-test counseling session. During this session, risk reduction strategies were discussed. Participants who tested positive for HIV were referred to appropriate treatment centers, care and support services, and were invited to access the program’s 24-hour hotline. Following the post-test counseling session, participants were given compensation such as music and cell phone minutes.

In implementing the Shout-It-Now intervention, program staff perceived the need to identify those at higher levels of risk and to offer additional services to them. To meet this need, a risk questionnaire was developed by program staff. Questions focused on three types of risk behaviors: risky sexual behaviors (condom use, number of sexual partners and being forced to have intercourse), alcohol and drug use, and absenteeism. These risk behaviors are related: Du Rant et al. (1999) described strong correlations among adolescent health risk behaviors, specifically early onset smoking and use of other substances (alcohol and drugs) with absenteeism and poor academic performance. These associations were evident across socio- demographic groups. Similarly, Guttmacher et al. (2002) identified correlations between school absenteeism and other adolescent risk taking behaviors. Also, the surveillance system of the Center for Disease Control for sexual behaviors that contribute to unintended pregnancy and sexually transmitted diseases, including HIV infection, monitor two of the three types of risk as indicators of high risk behavior: (i) drug and alcohol use before sexual intercourse, and (ii) risky sexual behaviors regarding condom use and number of sexual partners (CDC, 2010).

An initial set of 20 items intended to measure the three types of risk behaviors was compiled by the NGO staff and reviewed in focus group discussions with adolescents aged 14 to 18 years old. Twelve of the twenty items emerged with consistently shared meaning in discussions and included items measuring all three types of risk behaviors. As indicated in Table 1, the 12 items had response options on semantic differential scales of varying lengths (i.e., several questions had 4 response options, others had 3 response options, etc.). The 12 items were included as a screening measure in the online program and administered to 3,872 adolescents in two cities (Johannesburg and Cape Town) and one rural area (Burgersfort, Limpopo) in South Africa. The NGO program staff subsequently decided to use only 7 of the 12 items that they regarded, based on face validity, as the most important to determine risk. Different weights were allocated to response options according to the clinical team’s perceptions of relative risk associated with each response option. The summed score on the questionnaire was used to identify those with high risk for contracting HIV. No formal investigations of the psychometric properties of the 12-item or 7-item measures were conducted.

Table 1: Risk Assessment Questions and corresponding 7-item scale and 5-item scale items

 

 

#

Original 12-item questionnaire 7-Item Scale

Items

5-item Scale

Items*

Item Response options and Weights Cronbach

alpha = .74

Cronbach

alpha = .79

1 If you have a boyfriend or a girlfriend, please tell us: 0 I don’t have a boyfriend or girlfriend

1 She or he is not more than 5 years older than me

3 She or he is more than 5 years older than me

   
2 How often do you have three or more drinks at one time? 0 Never

1 Less than monthly 2 Monthly

3 Weekly

4 Daily or almost daily

Item 1  
3 Have you ever been forced to have sex when you did not

want to?

0 No, never.

2 Yes, but only once or twice 4 Yes, it happens often

   
4 How many days did you bunk school in the last year? 0 Never

1 Once or twice

2  Between 3 and 5 days

3  Between 5 and 10 days

4  More than 10 days

Item 3  
5 How do your parents feel about you bunking school? 0 They don’t allow it at all 1 They allow it occasionally

3 They allow it whenever I want to bunk

4  They don’t care if I bunk school

   
6 How many times in the last year did you have sex without a condom? 0 Never

2 Once

3 A few times 4 Many times

Item 4 Item 1
7 How many times in the last year did you have sex while drunk? 0 Never

2 Once

3 A few times 4 Many times

Item 2 Item 2
8 How many times in the last year did you have sex while high on drugs? 0 Never

2 Once

3 A few times 4 Many times

  Item 3
9 How many times in the last year did you have sex for gifts or favours? 0 Never

2 Once

3 A few times 4 Many times

Item 5 Item 4
10 How many people have you had sex with in the past 6 months? 0 None

2 One

3 Two or three

4 More than 3 people

Item 6 Item 5
11 How often do you smoke dagga? 0 never

2 occasionally

3 daily

   
12 How often do you use any

other drugs (e.g. tik, crack, cocaine, sniff glue, etc)

0 never

2 occasionally

3 daily

Item 7  

Study Aims

The purposes of this study were two-fold: first, to determine the reliability and construct validity of the 7-item HIV Risk measure used by the NGO; and second, to determine if a brief risk measure with better psychometric properties could be developed from the initial 12-item questionnaire. The research was conducted to inform the NGO about the reliability and validity of the measure being used to determine which adolescents were at high risk for HIV infection.

Development of the original risk measure had two primary limitations. First, determination of items to be included in the measure was guided primarily by the face validity of questions. Face validity alone is insufficient to justify wide scale implementation of an instrument measuring a high stakes construct such as risk for HIV infection.

Second, the 7-item measure developed by the clinical team used a weighted total score to identify high risk adolescents. There are several problems associated with assigning weights to different response options. Cognitive bias in this scoring approach may be a problem. For example, one may believe or even have evidence that a certain drug (e.g., methamphetamine) is more detrimental to health than another substance (e.g., alcohol), and therefore regard users of methamphetamine as having higher risk for HIV infection compared to alcohol users. However, there is a cognitive bias in this perception, related to generalizing one type of health risk to another. Giving a higher weight on a risk scale to a question that asks about the use of drugs as opposed to alcohol may be intuitively appropriate, but not empirically sound. Another problem with the weighted scoring approach used for the 7-item risk questionnaire stems from possible range compression, related to the clinical team’s subjective assessments of the amount of risk associated with each question’s response options. Range compression occurs when response options are limited to a small number of outcomes or possibilities, when in fact a much wider range of options are possible or likely. Consequently, there is a significant loss of precision in the assessment.

To address these limitations, a formal psychometric assessment of the risk measure was conducted. Traditional psychometric analyses were complemented with item response theory (IRT) analyses to (1) obtain detailed item- and test-level information about the performance of the risk measures, and (2) investigate the possibility of developing a briefer, psychometrically sound risk measure from the original pool of 12 items.

Method

This study was conducted with existing data provided by the NGO. The data were from a sample of 3,872 adolescents in two cities (Johannesburg and Cape Town) and one rural area (Burgersfort, Limpopo) in South Africa. Students in grades 8 to 12 from six public secondary schools in lower socio-economic communities were invited to participate in the intervention.

These schools voluntarily participated in Shout-It-Now during a four month time period in 2011. The NGO reported a 93% participation rate of all students present on the day the Shout-It-Now program was delivered. In addition to the six schools, data were also obtained from program outreach at a shopping mall, at which adolescents in grades 8 through 12 were eligible to  participate and were recruited by…?. This strategy added to the diversity of the sample, ideal for validation studies where the aim is not to describe the characteristics of a specific population, but rather to determine the psychometric qualities of an instrument. Approval and oversight to conduct research using these de-identified data was obtained from the University of Louisville’s Institutional Review Board.

The original 7-item risk measure was scored using a summative model, adding the weighted scores for each item response to obtain a total score. This scoring approach assumes that the measure is unidimensional. The unidimensionality assumption was tested using exploratory factor analysis (EFA) on the 7-item scale. Reliability and validity of the 7-item scale were also assessed, including each item’s mean corrected item-total correlation and the measure’s overall internal consistency reliability (i.e., Cronbach’s alpha).

Next, the full set of 12 items was analyzed to determine whether a brief risk measure with good psychometric properties could be developed. First, the factor structure of the 12 questions was determined using EFA. An iterative process of determining factor structure and interpreting factors was employed to identify a unidimensional construct, which was then assessed for reliability and validity.

This process facilitated the identification of a 5-item risk measure. Item response theory (IRT) analyses were employed to determine the psychometric properties of the newly developed brief risk measure. Item response theory provides more detailed psychometric information than classical test theory and is suitable to address the hypotheses generated in response to the second research question. First, unidimensionality of the 5-item measure was investigated using EFA. Next, the IRT assumption of local independence (i.e., the requirement that items should be statistically independent from one another after controlling for the level of risk (Steinberg & Thissen, 1996; Wainer & Thissen, 1996; Yen, 1993)) was assessed by inspecting the absolute values of residual correlations for each pair of items. A criterion of |r| ≥ .20 (Reeve et al., 2007) was used to determine violation of local independence. Following the testing of IRT assumptions, maximum-likelihood estimation procedures were used to fit a two-parameter logistic model to the data: MULTILOG 7.03 (Thissen et al., 2003) software was used to fit Samejima’s (1968) Graded Response Model and obtain item parameter estimates and estimates of participants’ levels of HIV risk. The amount and precision of measurement information provided by the newly developed 5-point scale were assessed using test information, item information, and item parameter estimates.

Once these psychometric properties were established, differential item functioning (DIF) analysis was conducted to determine if each item and the scale performed consistently across groups categorized by ethnicity, gender, and age. The R package lordif (Choi et al, 2011) was used in these analyses. The lordif package relies upon ordinal logistic regression for uniform and non-uniform DIF detection, employing Monte Carlo procedures to identify thresholds indicating whether items exhibit DIF, minimizing Type I error. In this approach, the impact of DIF on IRT parameter estimates is assessed by comparing model fit between nested ordinal logistic regression models with and without group terms (i.e., for ethnicity, gender, and age). Main effects of group are included to test for uniform DIF, while interactions between group and risk level are included to test for non-uniform DIF. Significant DIF is identified using likelihood-ratio (LR) tests between models. In the ordinal logistic regression approach to DIF-detection, an iterative approach is used in which group-specific parameter and trait estimates are updated and re-estimated until consistent identification of items with DIF over subsequent iterations is achieved (Choi et al., 2011). Monte Carlo simulations were used to determine if empirical threshold values systematically deviated from the nominal level. Two additional DIF analyses were also employed, assessing the magnitude of (a) changes in pseudo-R2 values, and (b) differences in parameter estimates between groups of interest.

Results

The majority of the 3,872 participants were female (54.6%). The largest ethnic group in the sample was Black (87.5%), followed by Coloured (10.8%), White (1%), Indian (0.3%), and Other (0.3%). The mean age was 17.1 years (SD = 4.3), and most participants were in grade 10, with grade 8 (22.5%) and grade 11 (20.2%) also well represented in the sample. Nearly all (97.0%) of participants were seen in a school setting, with the remainder (3.0%) seen in a shopping mall. Most participants (86.4%) were from the wider Cape Town area, with 10.7% from Johannesburg and 3.0% from Burgersfort in the Limpopo Province. The sample was from mainly lower socio-economic areas and was equally split between those whose families did versus did not own a car.

The 7-Item Measure

An EFA of the 7-item risk measure yielded one factor that explained 40.6% of the variance. The 7 items loaded on one factor with an Eigenvalue of 2.84. The Kaiser-Meyer-Olkin Measure of Sampling Adequacy was .83, indicating an adequate sample size for the analysis.

Bartlett’s test of sphericity was highly significant (p<.001). The factor loadings were between .33 and .78. Internal consistency reliability as measured by Cronbach’s alpha was .74. Only three items had relatively high correlations with the total scale score (> .50). The corrected mean item- total correlation was .46 (SD = .14). Applying a 90th percentile cut-off score to determine high risk using the total weighted score approach, 496 (12.8%) were identified as falling into the high risk category.

The Original 12-Item Measure

The original set of 12 items, when subjected to a principle component analysis and varimax rotation, yielded three factors that explained 52% of the variance. The first factor consisted of 6 items, with loadings between .44 and .73. The second factor had 4 items with loadings between .54 and .67, and the third factor had only 2 items with loadings of .78 and .84. The third factor appeared to focus on drug use (“How often do you smoke dagga?” and “How often do you use any other drugs (e.g. tik, crack, cocaine, sniff glue, etc)?”). The first two factors included 2 cross-loading items, and distinguishing between these factors was difficult, as each had items related to having sex, alcohol use, and missing school. Given the difficulty in interpreting the first two factors, it was decided to follow a different approach in deriving a unidimensional measure from the 12 items.

A 5-Item Measure

In reviewing the 12 items for content, 6 items were identified as focusing on the conditions associated with having sex. No central theme could be determined for the remaining 6 items. The 6 items associated with conditions when having sex were subjected to factor analysis. In a principal component analysis of these 6 items, one factor was extracted that explained 48% of the variance. The internal consistency reliability (Cronbach’s alpha) of the 6-item instrument was .77, but one of the items (“Have you ever been forced to have sex when you did not want to?”) correlated poorly with the total scale score (.30). After this item was removed, Cronbach’s alpha of the 5-item risk measure increased to .79, all items correlated at .50 or higher with the total scale score, and the mean corrected item-total correlation was .57.

IRT Analysis of the 5-Item Measure 

Unidimensionality and local  independence of the 5-item risk measure  were supported by EFA results: A single factor was extracted with an eigenvalue of 2.71 that accounted for 54% of the variance, and absolute values of residual correlations  for  each  pair  of  items ranged from .00 to .08. Fitting the Graded Response Model yielded four parameter estimates for each item: a (discrimination), b1 (difficulty threshold between option 1 and option 2), b2 (difficulty threshold between option 2 and option 3), and  b3 (difficulty threshold  between option 3 and option 4). High values of a suggest that items are able to distinguish between participants at similar levels of risk. Item parameter estimates and standard errors for each item are presented in the first column of Table 2.

Table 2: Graded Response Model Item Parameter Estimates for Total Sample and Subgroups

Parameter Estimate (SE)
Item Parameter Total Sample Male Female Black “Other” Age ≤ 19 Age ≥ 20
    (N=3,872) (N=1,757) (N=2,115) (N=3,387) (N =487) (N=3,483) (N=388)
  a 1.53 1.53 1.77 1.5 2.29 1.51 1.56
    (-0.1) (-0.14) (0.14) (0.09) (0.62) (0.1) (0.21)
  b1 -0.21 -0.11 -0.27 -0.19 -0.5 -0.12 -0.73
    (-0.05) (-0.07) (0.07) (0.05) (0.23) (0.06) (0.19)
  b2 1.32 1.32 1.23 1.33 1.15 1.38 1.03
    (-0.08) (0.14) (0.1) (0.09) (0.28) (0.1) (0.23)
1 b3 10.08* 11.91 8.92 10.78 7.98 10.76 8.92
  a 2.56 2.3 2.79 2.51 3.9 2.44 3.16
    -0.18 0.21 0.29 0.18 1.3 0.19 0.55
  b1 0.13 0.07 0.19 0.13 -0.02 0.2 -0.25
    -0.04 0.06 0.07 0.05 0.16 0.05 0.11
  b2 1.42 1.36 1.51 1.41 1.53 1.44 1.31
    -0.07 0.1 0.11 0.07 0.26 0.08 0.15
2 b3 8.18 8.05 9.45 8.34 9.85 8.48 8.99
  a 1.53 1.48 1.59 1.53 1.98 1.44 1.97
    -0.23 0.36 0.31 0.17 1.28 0.27 0.55
  b1 0.07 0.02 0.11 0.09 -0.43 0.13 -0.22
    -0.19 0.12 0.2 0.11 0.53 0.18 0.31
  b2 1.5 1.49 1.48 1.49 1.76 1.59 1.14
    -0.28 0.18 0.26 0.16 0.89 0.4 0.29
3 b3 8.88 9.44 7.87 8.72 14.43 9.68 5.99
  a 1.43 1.46 1.43 1.44 1.5 1.4 1.58
    -0.14 0.17 0.23 0.14 1.06 0.15 0.36
  b1 0.31 0.18 0.5 0.31 0.31 0.34 0.14
    -0.07 0.09 0.13 0.08 0.47 0.08 0.17
  b2 1.76 1.6 2 1.76 1.66 1.82 1.52
    -0.16 0.19 0.31 0.16 0.99 0.19 0.31
4 b3 15.77 14.81 15.44 15.53 20.02 11.89 19.16
  a 0.77 0.93 0.61 0.76 1.05 0.76 0.82
    -0.07 0.1 0.12 0.08 0.46 0.08 0.2
  b1 1.12 0.24 2.56 1.11 1.27 1.18 0.78
    -0.17 0.11 0.59 0.16 0.53 0.18 0.31
  b2 3.31 2.32 5.21 3.37 2.25 3.37 2.97
    -0.54 0.36 1.67 0.51 1.09 0.55 1.03
5 b3 15.62 17.17 17.66 14.81 18.29 14.91 20.07

Three of the five items (items 1, 3, and 4) demonstrated high discrimination (a = 1.43 to 1.53)(Baker, 1985), and one item (item 2) very high discrimination (a = 2.56). Only item 5 had moderate discrimination (a = .77). The lowest difficulty parameter, b1, clustered around one-third of a standard deviation above the mean risk level (M = .28). The lowest b1 parameter estimate was for item 1 (b1 = -.21, se =.05), while item 5 exhibited the highest b1 parameter estimate (b1 = 1.12, se = .07). This range of values suggests that risk levels near the mean were associated with selecting option 1 rather than option 0 on all 5 items.

Notably, the b3 difficulty parameter estimates were extremely high (M = 11.07 standard deviations above the mean). A total of 563 (14.5%) respondents endorsed option 3 on at least one item. Item 4 (b3 = 15.77) was the most difficult of the set, requiring extremely high levels of risk for participants to select the highest response option on this item. However, the two items with the lowest difficulty levels for their upper thresholds were item 2 (b3 = 8.18) and item 3 (b3 = 8.88), also requiring very high levels of risk for endorsement of option 3 versus 2. The extreme standardized scores have large standard errors, and the values should be interpreted relative to the mean of 0 and other large parameter estimates, and not in terms of absolute values.

Information for measuring risk with the total number of 5 items was higher than the standard error (i.e., most precise) from approximately 1.0 standard deviation below the mean to about 2.6 standard deviations above the mean. The test information curve peaks from 0.2 to 1.6 standard deviations above the mean, a range appropriate for precise measurement in a screening instrument, as screening should accurately assess those with low levels of risk (such as at one standard deviation below the mean) and those with high risk (such as those with more than 1 standard deviations above the mean).

As some items offered more information than others at similar levels of risk, some items could be omitted. Other aspects of item performance need to be considered before such a decision is made, including the degree to which an item exhibits measurement bias, or DIF.

DIF Analysis of the 5 -Item Measure Ethnic differences.

Differences in item parameter estimates by ethnic groups were investigated using Black versus Other, in which Other included all groups other than Black (i.e., Coloured, White, Indian, and Other). With lordif’s default settings, the program terminated in two iterations. All five items were identified as potentially having DIF. Sparseness due to all items being flagged limited the range of diagnostics possible for detailed DIF analysis. However, it was apparent that the mean slope of the true score functions for all 5 items was substantially lower for Blacks than for Others (1.55 vs. 2.15), indicating non-uniform DIF. The LR χ2 test for uniform DIF, comparing Model 1 and Model 2, was significant for items 1, 2, 4, and 5 (p < .001). This was also true for the 2-df test of non-uniform DIF (comparing Models 1 and 3, p < .001) for the same items. The overall 1-df test was significant for items 1 and 5. The non-uniform component of DIF revealed by the LR χ2 test can also be observed in the substantial group .001) for the same items. The overall 1-df test was significant for items 1 and 5. The non-uniform component of DIF revealed by the LR χ2 test can also be observed in the substantial group differences of the slope parameter estimates for item 1 (2.29 vs. 1.50), item 2 (3.90 vs. 2.51), and item 5 (1.07 vs. .76). When weighted by the focal group trait distribution, the expected impact of DIF as reflected by McFadden’s pseudo-R2 measures varied across items from .02 to .08 with a mean of .05 for R2 . The impact apparent in the R2 was less and varied from .00 to .03 with a mean of .01. The percent change in β1 for all 5 items tended to be small (M = 4%), with a maximum difference of 11% noted for item 2, which is higher than the frequently used criterion of 10% change in β1 to conclude that DIF exists (Crane et al., 2004).

The mean Monte Carlo probability threshold values associated with the χ2 statistic across items were .008, .01 and .01 for χ212 (testing for uniform DIF), χ213 (testing for non-uniform DIF), and χ223 (testing for DIF overall while controlling Type I error), respectively. On average, the empirical threshold values for the probability associated with the χ2 statistic were close to the nominal α level. The Monte Carlo simulation results confirmed that the LR χ2 test maintains the Type I error adequately in this dataset.

Gender and age. Similar analyses were conducted for two other comparison groups, categorized by gender and age. For these analyses, age was categorized as younger (≤ 19 years) or older (≥ 20 years). The program terminated for gender in two iterations, flagging all items. Similarly, the program terminated for age in five iterations, flagging all items.

The mean slope for the true score functions was lower for males in comparison to females (1.54 vs. 1.64) and substantially lower for younger compared to older subjects (1.51 vs. 1.82), indicating non-uniform DIF. The LR χ2 test for uniform DIF, comparing Model 1 and Model 2 (χ2 ), was significant for all items for gender and items 1, 2, 4, and 5 for age. The 2-df test (χ2 ) for non-uniform DIF (comparing Models 1 and 3, p<.001) was significant for all items for both gender and age. The overall 1-df test (χ2 ) was significant for items 1, 3, 4 and 5 in the case of gender, and items 1, 2 and 5 for age. The non-uniform component of DIF revealed by the LR χ2 test can also be observed in the difference of the slope parameter estimates; for example, in the case of age for the younger vs. older groups, respectively, for item 2 (2.44 vs. 3.16), item 3 (1.44 vs.1.97), and item 5 (1.51 vs.1.82). The McFadden’s pseudo-R2 measures varied across items from .0025 to .0231, with a mean of .01 for R213 and .0037 to .0195 with a mean of .013 for R213 respectively for gender and age. The impact apparent in the R223 was also small, with a mean of .003 for gender and age. When aggregated over all the items in the test, differences in item characteristic curves may become small due to canceling of differences in opposite directions.

However, this does not mean that the impact on trait estimates is not of concern.

The theta values (i.e., levels of risk) with the highest information for the total sample and the various groups are reported in Table 3. Maximum item information estimates reflected by theta values of .2, 1.6 were most common in all groups for items 1, 2 3 and 4. Exceptions included lower values (-1.4, .0) for item 1 in the case of younger participants, and for item 2 for the total sample, as well as for older participants in the case of item 3. Information for item 3 for the ethnic “other” group was highest in the 1.8, 3.0 range.

Table 3: Maximum Item Information Estimates and Locations for Total Sample and Groups

Item Total Sample Ethnic: Black Ethnic: Other Young Older Males Females
1 0.65 0.627 1.174 0.634* 0.653 0.657 0.846
2 1.965* 1.646 3.798 1.551 2.458 1.398 1.988
3 0.663 0.666 0.986** 0.59 1.036* 0.618 0.712
4 0.581 0.59 0.644 0.558 0.703 0.607 0.579
5

a Theta

0.175

values incr

0.17

ease in steps

0.171

4,0.0 **

0.196

1.8,3.0 *

0.248***

**0.2,1.6

0.108

Total mean and percentile scores (90, 93, 95 and 99) on the 5-point risk scale for eight different groups (age by ethnicity by gender) are presented in Table 4. Scores varied widely between the groups. For example, the mean score for older female subjects of Other ethnicity is 5.10, but for younger females in this ethnic group, the mean score is only 0.41. The 90th percentile scores for Other ethnicity ranges from 1.00 for younger females to 11.00 for older males and females. For older Blacks, the impact of gender on scale total scores appears small, but for younger Blacks, gender has a significant impact (M = 3.12 for males vs. 2.08 for females; 90th percentile = 8.00 vs. 5.00, respectively).

Table 4: Mean and percentile scores for 5-Item Risk Scale by age, ethnicity and gender.

5-Item Risk Scale
             

N

 

Mean

Standard Deviation Percentile 90 Percentile 93 Percentile 95 Percentile 99
Age ≤ 19 years Ethnicity Black Gender Male 1472 3.12 3.28 8.00 9.00 10.00 13.00
          Female 1745 2.08 2.54 5.00 6.00 7.00 11.00
      Other Gender Male 204 .89 2.27 4.00 5.00 6.00 9.00
          Female 264 .41 1.24 1.00 3.00 3.00 7.00
  ≥ 20 years Ethnicity Black Gender Male 74 4.58 3.60 9.00 10.00 12.00 15.00
        Female 96 3.48 3.59 9.00 10.00 11.00 15.00
    Other* Gender Male 7 3.86 3.98 11.00 11.00 11.00 11.00
        Female 10 5.10 4.23 11.00 14.00 14.00 14.00
Total

Sample

         

3872

 

2.4

 

3.0

 

7.00

 

8.00

 

9.00

 

12.00

*Due to skewness in distribution and small cell n, percentile scores equivalent to the “Black Older Group” are recommended for use to differentiate high risk individuals.

Using the age-, ethnicity-, and gender-specific 90th percentile as a criterion to determine high risk with the 5-item measure, 499 (12.9%) adolescents were identified as falling into the high risk group. This proportion is equivalent to the 496 (12.8%) identified as high risk using the summative weighted scoring approach for the 7-item measure. However, only 313 (8.1%) cases were identified by both scales as high risk, and the inter-rater agreement between the two scales was significant, but not high (Cohen’s kappa = 0.57; p < 0.001).

Discussion

The 7-item risk measure used at present is unidimensional with acceptable reliability for group administration, but its internal consistency is inadequate for differentiating between individuals (≥.65 is required at the group comparison level and ≥.80 at the individual level; Nunnally, 1994). In addition, the validity of the 7-item scale as measured by corrected mean item-total correlation (.45) was relatively low; a corrected mean item-total correlation of at least .50 is desired (Hudson, 1982).

In contrast, the 5-item risk measure was unidimensional, valid (mean corrected item-total correlation ≥.50), and reliable enough for use in differentiating individual levels of risk (Cronbach’s alpha = .79). Further, the 5-item scale was 2 items shorter and more valid (.57 vs .46) and reliable (.79 vs. .74) than the 7-item instrument. A reduced number of items selected from the 12-item HIV Risk Questionnaire was therefore incorporated into a new instrument with improved psychometric characteristics in comparison to the 7-item measure currently in use.

All 5 items of the newly developed measure discriminated sufficiently between subjects at different levels of risk. For all 5 items, extremely high levels of risk were necessary for a subject to select the highest option on the four-point scale. The extremely high level of risk required for the highest scores to be endorsed is an indication of range compression. An additional response option between “A few times” and “Many times”—for example, “Several times”—may add to the measure’s precision. The range of precision of the measure is appropriate for a screening instrument.

All items in the 5-point risk measure were flagged for measurement bias or DIF. Both uniform and non-uniform DIF were identified as problems. However, the percentage of subjects with salient score changes and the minimal clinically important difference (MCID) for the risk measure have not yet been determined. Age-, ethnicity-, and gender-specific norms are therefore recommended. The mean scores and percentiles between these groups vary substantially (see Table 4). Of note, the actual number of older subjects of Other ethnicity in the sample is very low. Consequently, percentile scores are not an appropriate way to identify high risk individuals in this group, because a few subjects with high score skew the distribution. Lower thresholds for older adolescents of Other ethnicity, such as those presented for the older Black adolescents, are recommended.

Limitations

Several study limitations should be considered. A comprehensive literature review is an essential part of the process in scale development (DeVellis, 2012). This study focused on items used in practice to determine risk, and although these questions were supported by some previous studies and the CDC’s surveillance system, a comprehensive literature review was not conducted in scale development or refinement efforts. Also, racial groups were not equally represented in the sample; this is particularly true for older adolescents of Other ethnicity. The most important limitation is the lack of longitudinal data inclusive of HIV status to use as a criterion in determining the predictive validity of the measures analyzed.

Conclusions

These findings, combined with the fact that a less reliable 7-item measure is currently used to measure risk for HIV infection, have immediate implications for practice. A 90th percentile cut-off score for high risk resulted in similar numbers (respectively 499 and 496 for the 5-item and 7-item scales) being identified as high risk, but the agreement between the scales in categorizing adolescents’ risk levels was only moderate. In addition, the extremely high level of risk required for the highest score option on the four-point scale to be endorsed (as reflected by the mean standardized score of 11.07) is alarming. For most domains of behavior, the range of standardized scores is between -3 and 3. In total, 563 participants selected the highest response option on at least one item. This means that in a sample primarily comprised of high school students of low socioeconomic status in South Africa, 14.5% indicated that they engaged in behavior much more risky than the behavior of their peers. Although the measurement error on any single item is higher than the measurement error for the total scale, it is interesting that the percentage of high risk individuals using the 90th percentile cut-off (12.9%) is close to the percentage of participants who endorsed the highest response option on at least one item (14.5%).

VCT is not done routinely in schools in South Africa. In light of the relatively high percentage of adolescents that engage in risky behavior for HIV infection, this policy should be revisited. The newly developed 5-item measure offers a way to identify adolescents at high risk, a first step in providing targeted preventive interventions to the high risk adolescent population. Use of the 5-item screener, including age-, ethnicity-, and gender-specific percentile-based norms, may dramatically improve prevention effectiveness, and will enable additional validation studies and further refinement of this and other risk measures. In addition, the behavior dimension included in the 5-item measure is not different from other approaches used to determine risk for HIV infection. Consequently, it is possible that other measures may also include item bias for various groups. Item bias should therefore be investigated for instruments or questions used to determine HIV risk.

 

References

Baker, F. B. (1985). The Basics of Item Response Theory. Portsmouth, NH: Heinemann Educational Books.
Centers for Disease Control and Prevention. Surveillance Summaries, MMWR 2010; 59 (No.SS-5).
Crane, P.K., Gibbons, L.E., Jolley, L., Van Belle, G. (2006). Differential Item functioning analysis with ordinal Logistic Regression Techniques: DIF Detect and difwithpar. Medical Care, 44(11 Supp 3), S115-S123.
Crane, P.K., Van Belle, G., Larson, E.B. (2004). Test bias in a cognitive Test: Differential Item Functioning in the CASI. Statistics in Medicine, 23, 241-256.
Choi, S.W., Gibbons, L.E., Crane, P.K. (2011). lordif: An R Package for Detecting differential Item functioning Using Iterative Hybrid Ordinal Logistic Regression/Item Response Theory and Monte Carlo Simulations. Journal of Statistical Software, 39 (8), 1-28.
DeVellis, R. F. (2012). Scale Development – Theory and Applications (3rd ed.). Los Angeles: Sage
Du Randt, R.H., Smith, J.A., Kreiter, S.R., Krowchuk, D.P. (1999). The Relationship Between Early Age of Onset of Initial Substance Use and Engaging in Multiple Health Risk Behaviors Among Young Adolescents. Pediatric Adolescent Medicine, 153: pp286 – 291
Guttmacher, S., Weitsman, B., Kapadia, F., & Weinberg, S. (2002). Classroom-based Surveys of Adolescent Risk Taking Behaviors: Reducing the Bias of Absenteism. American Journal of Public Health, 92 (2): pp235 -237.
Nunnally, J.C.& Bernstein, I. H. (1994). Psychometric Theory (3rd ed.). New York: McGraw- Hill.
Hudson, W.W. (1982). The Clinical Measurement Package: A Field Manual. Homewood, IL: Dorsey Press.
Reckase, M. D. (1979). Unifactor latent trait models applied to multifactor tests: Results and implications. Journal of Educational Statistics, 4, 207-230.
Reeve, B. B., Hays, R. D., Bjorner, J. B., Cook, K. F., Crane, P. K., Teresi, J. A., et al. (2007).
Psychometric evaluation and calibration of Health-Related Quality of Life item banks plans for the Patient-Reported Outcomes Measurement Information System (PROMIS). Medical Care, 45(5 Suppl 1), S22-S31.
Samejima, F. (1969). Calibration of latent ability using a response pattern of graded scores.
Psychometrika Monograph Supplement, 17.
Steinberg, L., & Thissen, D. (1996). Uses of item response theory and the testlet concept in the measurement of psychopathology. Psychological Methods, 1, 81-97.
Thissen, D., Chen, W.H., & Bock, R. D. (2003). MULTILOG 7.03 [computer software].
Lincolnwood, IL: Scientific Software International.
Van Zyl, M.A., Barney R., & Pahl, K. (Under Review). VCT and Celebrity Based HIV/AIDS Prevention Education: A Pilot Program Implemented in Cape Town Secondary Schools.
Wainer, H., & Thissen, D. (1996). How is reliability related to the quality of test scores? What is the effect of local dependence on reliability? Educational Measurement: Issues and Practice, 15, 22-29.
Yen, W. (1993). Scaling performance assessments: Strategies for managing local item dependence. Journal of Educational Measurement, 30, 187-213.