LINKING COMPLEX EMERGENCY RESPONSE AND TRANSITION INITIATIVE

 

 

 

 

CERTI Crisis and Transition Tool Kit   Cross-Cultural Assessment Of Trauma-Related Mental Illness                                                                   Research Report Summary                             Paul Bolton MD MPH ScM                      The Johns Hopkins University                                                                  Lincoln Ndogoni                       World Vision International       August 2000

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

EXECUTIVE SUMMARY .……………………………………………………………………1

Project Objectives……………………………………………………………………………1

Methods………………………………………………………………………………………..1

Results…………………………………………………………………………………………3

Discussion…………………………………………………………………………………….5

Recommendations…………………………………………………………………………...7

 

STATEMENT OF SUPPORT………………………………………………………………,..8

 

ACKNOWLEDGMENTS……………………………………………………………………...8

 

INTRODUCTION…………………………………………………………………………….…9

Background to Johns Hopkins University-World Vision Collaboration………….…9

Background to WV’s Interventions in Rwanda……………………………..…………..10

Background and Overview of Project Design…………………………………………..10

 

PROJECT OBJECTIVES………………………………...…………………………………..11

 

METHODS……………………………………………………………………………………..11

Selection of Instruments……………….…………………………………………………..12

Ethnographic Mini-Study…………………………………………………………………...13

Finalizing the English version of the Instrument………………………………………15

Translation…………………………………………………………………………………….16

Instrument Testing…………………………………………………………………………..16

Sampling and Survey Procedure………………………………………………………….17

Quantitative Analysis………………………………………………………………………..18

 

RESULTS………………………………………………………………………………………22

Study Site………………………………………………………………………………….…..22

Results of the Ethnographic Study…………………………………………………….…22

Validity and reliability of survey instruments…………………………………………...27

Survey findings……………………………………………………………………………....29

 

DISCUSSION……………………………………………………………………………….…34

Process……………………………………………………………………………………..…34

Results…………………………………………………………………………………………36

 

RECOMMENDATIONS……………………………………………………………………....37

 

REFERENCES………………………………………………………………………………..39

 

APPENDIX A: ASSESSMENT INSTRUMENT (ENGLISH VERSION…………………40

 

APPENDIX B: SHORT REVIEW OF ETHNOGRAPHIC METHODS USED…  ……...47

 

APPENDIX C: VERBAL CONSENT FORM……………………………………………....49

 

APPENDIX D: SUGGESTED TIME-LINE FOR SUBSEQUENT STUDIES………...….51

 

APPENDIX E: DSM-IV CRITERIA FOR MAJOR DEPRESSIVE EPISODE……..……54

 

APPENDIX F: DSM-BASED ALGORITHM FOR DIAGNOSING DEPRESSION..…. ..56

 

APPENDIX G: ROC CURVES  ……………………………………………………………..58

 

APPENDIX H: TESTING CONSTRUCT VALIDITY USING FACTOR ANALYSIS……62

 

APPENDIX I: STRATIFIED LOGISTIC REGRESSION ANALYSES………………..….64

 

1.1 Project Objectives 

 

1.          To create an instrument adaptation and validation process which can be used by non-governmental organizations (NGOs) and others to quantitatively assess the mental health burden of trauma at the population level across cultures and situations.

 

2.          To use this instrument and process to assess part of the mental health burden of trauma on a civilian population in Rwanda.               

 

3.          To use the resulting data to assess the need for interventions, form the baseline for an intervention process, and (at a future date) to plan the form of such an intervention.

 

4.          Current methods to assess mental health across cultures require resources and time not available to NGOs and many of the populations they serve, and are therefore research tools only.  In this study we have attempted to develop a method useful for NGOs because it requires only training and existing resources.

 

1.2 Methods

 

1.2.1 Overview

 

The method we tested in Rwanda is designed for use with various clinical mental health indicators.  For this first trial we chose to study Depression as an indicator of the effects of mental trauma, and to study only adults.  We assessed Depression using a version of the well-known Hopkins Symptom Checklist (HSCL), which has been used among many populations.  We chose Depression because there is agreement on its basic form, it represents the most severely affected persons, and because it has been studied in a variety of situations and cultures.  We chose it over Posttraumatic Stress Disorder (the other major clinical indicator of the severe effects of trauma) because it occurs in situations other than trauma, and so our findings would have implications beyond populations affected by trauma.  Controversy continues about the use of these and other Western clinical mental health indicators in non-Western cultures.  This is partly due to the failure of workers using these indicators to adequately validate them prior to use, thereby leaving the question of their appropriateness unanswered.  Therefore, a vital component of this method is preliminary investigation of the local validity of these indicators. 

                                         

The method we developed has 8 main stages:

 

1.      Collecting ethnographic data on local perceptions of mental health.

2.      Analyzing these data for evidence that Western indicators of mental problems are appropriate. 

3.      If so, using these data to adapt existing questionnaires that measure these indicators.

4.      Testing the validity of these questionnaires and these indicators.

5.      Using the resulting instrument in a community-based survey of a random population sample. 

6.      Analyzing the survey data to

(a)   assess the local prevalence and characteristics of the selected mental health problems;

(b)   further test the validity and reliability of the instruments and indicators.

7.      Using the ethnographic and survey data to design appropriate interventions.

8.      Repeating the survey after the intervention to measure impact.             

 

This report describes the first 6 stages in detail, and our experience in fielding testing these stages among a population in Rwanda.

 

1.2.2 Ethnographic Study

                                                                                                                 

The purpose of the ethnographic study was to determine if local people experienced Depression as a result of trauma, and to learn the names and symptoms of comparable ‘Depression-like’ illnesses.  If local people experience the symptoms of Depression as a result of trauma, this would be evidence that Depression occurs among this population.  Knowledge about local perceptions of Depression would enable us to work with local people to adapt and validate existing Depression instruments (in this case the HSCL).  Data gathered in the ethnographic study is also useful in designing interventions.  We trained PSSP staff in three ethnographic methods - free listing, key informant interviewing and pile sorts - which they then used to conduct the study.  

 

PSSP staff used free listing to develop a list of problems resulting from the genocide in 1994, according to local informants.  This list contained the local names and short descriptions on significant mental health issues - as perceived by local people - and the importance of these issues relative to other problems.  From this list we identified Depression-like illnesses for detailed investigation using key informant interviews.  These are longer interviews with local people that provide more information than the shorter free listing.  The results of these interviews enabled us to determine whether Depression occurred in this population, and how it is understood.

 

We anticipated that respondents might link classical Depression symptoms with other symptoms.  We used pile sorts to determine if these symptoms might comprise important local variations in Depression symptomatology, or were merely individual opinions.  The pile sort results enabled us to determine which symptoms were potentially important and should be added to the HSCL.

 

Interviewers also conducted a free list exercise asking respondents to list the important tasks that local people do regularly.  We used the results to create sets of questions (separate for men and women) on functional disability.  These questions were added to the HSCL to explore the relationship with Depression, which is known to be associated with reduced function in other populations.   

 

1.2.3 Translation

 

After adaptation using the ethnographic results, we had an assessment instrument consisting of basic demographic data, the adapted HSCL, and a community-specific questionnaire on function.  The instrument was translated from English to Kinyarwanda using a combination of group and translation-backtranslation methods.  During the translation we constantly cross-checked the results with the ethnographic data.  This was to ensure that the translation used terms which were truly part of the local vocabulary, since word usage frequently varies across regions. 

 

1.2.4 Validity Study

 

Prior to the main survey we conducted a mini-survey to study the validity of the instrument.  We reasoned that the local syndrome identified in the ethnographic study as most similar to Depression should be highly correlated with Depression itself if Depression occurs in this population and is accurately diagnosed by the instrument, and if the local syndrome is accurately diagnosed by local people.  We asked key informants in the community to identify local people who have and who don’t have the local syndrome.  Blinded interviewers then interviewed these people using the instrument, and also asked if the respondents felt they had the syndrome.  We compared diagnoses of the local syndrome by key informants and the respondents themselves with Depression diagnoses using the instrument.

 

1.2.5 Survey and Analysis

 

The instrument was then used in a random survey of the adult population.  We selected 5 ‘Sectors’ from each of the rural Communes of Kinzenze and Butamwa, based on geographical spread.  Within these Sectors we then selected a simple random sample for interview.  Therefore, the survey results do not refer to the Communes overall, but only to the selected Sectors. 

 

Interviewers came from Kigali because local people with a high school education who could read and write were not available.  PSSP staff who had already received training and data gathering experience in the ethnographic study acted as supervisors.  As well as assisting the interviewers they checked on all refusals and re-interviewed 10% of respondents.  This provided a quality control measure, and data to test the reliability of the instrument over time (test-retest reliability). 

 

We used the data to conduct further analyses of the validity of the Depression concept and the instrument, and to test the instrument reliability.  We calculated the prevalence of Depression among the population as an indicator of need for specific interventions.  We explored the relationships between ability to perform specific tasks and Depression and other respondent characteristics, using correlation analyses.  Using a summary scale of functional disability we also explored the relationship between Depression and scores on this scale, using logistic regression.  Finally, we developed a Depression scale and calculated a cut-off score that can be used for diagnosing Depression (instead of using the DSM criteria) and is most sensitive and specific for detecting associated functional disability.        

 

1.3 Results

 

The ethnographic study showed that local people experience all the DSM^[1] diagnostic symptoms of Depression as a result of the 1994 genocide.  This supports the occurrence of Depression among this population.  They also experience other symptoms associated with Depression, three of which were confirmed by the pile sort exercise and incorporated into the Depression questionnaire.  Local people do not organize symptoms into an entity similar to Depression that we could use for direct comparison.  The closest syndrome to Depression is Agahinda gakabije, or severe grief.  In the validity study we therefore compared local diagnosis of this syndrome (by informants and respondents) with Depression diagnosis by the instrument.  We found that people with Depression form a sub-group of those with severe grief.  This is consistent with experience elsewhere that grief triggers Depression in a proportion of cases (Craig, 1996).  It provides some additional evidence that Depression occurs among this population and can be diagnosed by the instrument.     

 

We used the questionnaire to survey 368 and 72 adults in selected Sectors of Kinzenze and Butamwa Communes respectively.  66% of respondents were female in Kinzenze and 79% in Butamwa (other sources have found that Rwanda is now 70% female (Dabelstein, 1996)).  In the Kinzenze Sectors 17.9% met the DSM criteria for Depression (see Appendix E for criteria) and 41.8% described themselves as having severe grief.  In the Butamwa Sectors the corresponding figures were 5.6% and 31.9%.  Studies in other parts of Africa and the world have found prevalences of Depression between 0.8-5.8% (Weissman, et al, 1996; Bhagwanjee et al, 1998).  This suggests that the Butamwa data may be close to the background level of Depression in this population, whereas the higher rate in the Kinzenze Sectors may reflect the more severe genocide experience of that Commune.

 

Internal consistency reliability (agreement between similar questions) was very good for all the Depression questions (Cronbach’s alpha =0.87), both the original HSCL questions and the local symptoms identified in the pile sorts.  This supported their significance as part of the local expression of Depression.   The internal consistency of the male and female function questions was also good (Cronbach’s alpha = 0.82 and 0.82 respectively).

 

Overall, Depression is strongly associated with reduced function.  Grief is not associated with reduced function when the effects of Depression are accounted for.  When we studied these relationships between Depression, grief and function among various subgroups we found that the association between Depression and function is greatest in those with more education and who are older.  These associations are not significantly different for men and women.

 

Among the non-depressed both sexes most frequently experience difficulties with tasks requiring more energy.  This may reflect physical ailments and under-nutrition (during the study there was a drought and food shortage).  Depression is particularly associated with increased difficulties in those tasks necessary for the family’s well-being:  labor and earning among men; and washing clothes, cleaning house, and caring for children among women.  For each of these tasks most of the cases among the depressed group are associated with the Depression itself, and Depression accounts for 17-48% of all significant difficulty in the population. 

 

We created a simple score of Depression for each respondent by adding their responses to the Depression questions.  We used the validity and reliability results to decide which Depression questions were useful and should be included in the scale, and which should not be used.  In the same way we also created a function score for each respondent.  Since both Depression and function questions use a Likert scale of responses with higher numbers representing increasing severity, in both scales higher scores represent increasing Depression or dysfunction.  We measured the reliability of these scales over time by comparing scores on the first and second interviews for 37 respondents who were re-interviewed.  The Pearson correlations between the first and second interviews were 0.671 for the Depression scale and 0.574 for the function questions, which is adequate.  Using ROC analysis we calculated a cut-off score of 30.5 on the Depression scale.  At this score Sensitivity = 68.3% and Specificity = 71.8% for detecting significant functional disability and 98.5% and 79.1%, respectively for detecting Depression based on the DSM criteria.  This scale and cut-off score can be used in future to assess Depression and its effects on function among this population. 

 

1.4 Discussion

 

This study was the first field trial of a method to assess the burden of mental problems across cultures.  Existing methods require time and resources beyond the means of most NGOs, and in many cases the necessary gold standards are just not available.  Our approach, was to understand how local people view mental health so that we could enlist their assistance in answering these questions. 

 

For this first trial we worked in Rwanda because of our interest in transitional populations, and because World Vision has a psychosocial program there.  For transitional populations affected by war (and genocide) the two most severe mental health problems are Depression and PTSD, which commonly occur together (Engdahl et al, 1998. Shalev et al, 1998. Peltzer, 1998).  Resources prevented us from investigating more than one disorder so we focused on Depression, although we could have investigated PTSD or any mental illness or health issue using this method.  We chose Depression because - unlike PTSD - it is also common in countries without experience of war (Weissman et al, 1996).  Therefore our results could have relevance beyond the scope of transitional populations. 

 

The field trial proceeded smoothly and we believe demonstrated that it is possible to train local staff and conduct a rapid ethnographic and quantitative study in a very short time with resources currently available to many NGOs and other organizations - the only additional requirements are training and a commitment to understanding local communities.  Studies like this one are best done at the beginning of programs, even before the interventions have been decided:  Ethnographic methods provide a lot of general information about these communities which can be used to plan a program, as well as an effective way to meet local people and build trust.  For example, the first free listing exercise provides information on all the community’s problems that can guide an NGO in setting up all programs, not just programs for mental health.

 

We trained local staff in instrument preparation, data collection and qualitative data analysis.  We did not train them in quantitative data analysis and interpretation, because of time and resource limitations.  In future, this training can be provided to selected staff with computer skills. Eventually the quantitative analysis section of this method will consist of the following:

 

§         Calculation of correlations between Depression and local illness

§         Generating and interpreting Cronbach’s alpha scores and item analysis

§         Calculation of prevalences

§         Creation of simple function and Depression scales

§         Calculating cut-off scores on the Depression scale most appropriate for detecting functional disability.

 

Our results suggest that Depression occurs among this population even though it is not recognized locally as a distinct syndrome.  The validity and internal reliability of the instrument were also good although the test-retest reliability were adequate only.  This combination suggests that moods may vary between tests, or that there is reactivity to the survey which is expressed in the second interview.  However, this was not severe enough to invalidate using the instrument.

 

Depression rates were much lower than grief, and there was a large difference between the two areas:  17.9% of adults depressed in the Kinzenze Sectors and 5.6% in Butamwa.  A study of Depression in 10 countries (not including Africa) found prevalences ranging between 0.8-5.8% (Weissman, et al, 1996), which was consistent with results from other parts of Africa (Bhagwanjee et al, 1998).  This suggests that the Butamwa data may be close to the background level of Depression in this population, whereas the higher rate in Kinzenze may reflect the more severe genocide experience in that Commune.

 

Even people with severe grief, in the absence of depression do not suffer significantly reduced function.  The more general implication may be that even severe levels of distress in those without mental illness do not greatly affect function.  To test this hypothesis these methods should be used in similar research among other populations, and to examine other mental illnesses (such as PTSD).  If confirmed, it would support a shift from current approaches which broadly address trauma experience and grief, to focusing on those who have developed mental illness as a result

 

Among both men and women depression is associated with dysfunction in tasks important to the care of the family, including children.  Among those depressed most of the difficulty with these tasks was associated with the depression itself, and with a significant proportion of the difficulty reported by the entire sample.  This is remarkable, considering the many other problems and diseases that cause dysfunction in this part of Africa.  It also carries implications for social and economic development, since the earning power and well-being of the family are compromised.  If Depression is a direct cause, then Depression in an adult will significantly affect the whole family, and treatment will be required before the family can fully take advantage of available social and economic resources to improve their situation. 

 

Since this was a cross-sectional study we could not prove that Depression causes dysfunction.  It is possible that primary dysfunction could produce Depression, or that there are mixed effects.  Research is required to sort this out.  The most efficient way to do this would be a study of effective interventions specifically for depression.  Concomitant improvement in depression and function with depression treatments would then demonstrate the link.  This type of research would also demonstrate whether function can be improved by treating depression, and could be used to test new interventions appropriate for areas with limited resources.  If Depression is shown to be the cause of dysfunction, to be treatable, and that function improves as a result, then the prevalence of Depression becomes a developmental issue and should be assessed whenever high rates are suspected, such as after a war or disaster.  We consider that the measurement method we have described here provides a means of conducting this type of research. 

  

During the survey interviewers noted that those subsequently diagnosed as depressed were obviously distressed, and very keen to talk about their experiences.  Interviewers reported that these respondents were grateful for the opportunity to speak about their problems, and many said that this was the first time they had discussed them openly with anyone.  PSSP staff also noted that these persons were not picked up by their existing outreach program, despite the program’s good coverage.  The same was true of persons diagnosed with Depression during the validity study.  This may be due to the reclusive behavior of the depressed, as expressed by the respondents and noted by the interviewers.  Whatever the cause, it suggests that current programs are not reaching those most in need of assistance.  These are the people whose mental distress is the most severe, and whose reduced function is most likely to affect their own well-being and that of the community.

 

1.5 Recommendations

 

For further assessment

 

1.      World Vision and Johns Hopkins University should repeat this assessment procedure with other populations;  to assess need and build up a composite picture of mental illness across Africa, and to continue to refine and simplify the method.  Future changes should include assessing exposure to events and their relationship to mental illness, assessing personal and community factors which may mediate reactions to trauma, and including in the ethnographic study investigating community suggestions for addressing these issues.   Improvements should also include assessments for other mental problems (our study found good evidence for frequent occurrence of PTSD), and for children.  Future field trials should include training in quantitative analysis.

 

2.      Current programs do not appear to reach those most in need of mental health assistance. World Vision should form a technical advisory group composed of World Vision staff and experts in the field of mental health issues.  This group should review these results and consider how WV and other NGOs can best help persons with Depression, given existing NGO resources.  Communities should contribute to this process, and particularly to the review of proposals for feasibility, acceptability and implementation.

 

3.      World Vision Rwanda should use the recommendations in 2. to assist those identified in the survey as depressed, and screen for others with Depression.  The survey should also be repeated in other areas in which WV Rwanda suspects a need and is planning a mental health intervention, to assess the level of need.

 

4.      Finally, we must emphasize that the free listing and validity study both revealed that poverty and lack of people are more pressing problems for most people than mental and emotional issues.  One of the positive features of this assessment method is that it should enable the NGO to put mental health in the context of other problems.  WV Rwanda should use information to prioritize all their interventions for Kinzenze and Butamwa Communes, and repeat the free listing in other areas prior to planning interventions. 

 

For new research

 

1.      Likely interventions resulting from the technical advisory group (and other sources) should be tested using standard research protocols.  Such protocols could be designed and executed which could investigate three outstanding issues:

§         To find  effective treatment for depression, given limited resources and large group of affected persons.

§         The nature of the cause-effect relationship between depression (and other mental illness) and function.

§         To determine if improvement in depression results in improvement in function and how much.

 

2.      To repeat (or expand) these protocols to other mental illnesses associated with significant dysfunction.

                 

                                                                                                     

This research was funded in part by the US Agency for International Development (USAID) under Cooperative Agreement HRN-A-00-96-9006 with The Johns Hopkins University (JHU).  These funds support research and consultancy costs.  World Vision also provided financial support, as well as staff and logistic support. 

 

 

We would like to acknowledge the following organizations and individuals for their contributions to the research: Bill Lyerly of USAID and Tom Ventimiglia of World Vision, without whose support this project would not have been possible, the staff of the World Vision Rwanda Psychosocial Support Program and our interviewers and interviewees.  Also Bill Eaton and Bill Weiss of Johns Hopkins University, and Nancy Mock of Tulane University, all of whom gave invaluable advice on the design and execution of this project.

 

4. INTRODUCTION

 

4.1 Background to Johns Hopkins University-World Vision Collaboration