This article has Open Peer Review reports available.
Attenuating posttraumatic distress with omega-3 polyunsaturated fatty acids among disaster medical assistance team members after the Great East Japan Earthquake: The APOP randomized controlled trial
© Matsuoka et al; licensee BioMed Central Ltd. 2011
Received: 20 May 2011
Accepted: 16 August 2011
Published: 16 August 2011
On March 11, 2011, a magnitude 9.0 earthquake, the most powerful ever recorded in Japan, and a massive tsunami struck off the coast of the Sanriku region. A Disaster Medical Assistance Team, a mobile medical team with specialized training that is deployed during the acute phase of a disaster, was dispatched to areas with large-scale destruction and multiple injured and sick casualties. Previous studies have reported critical incident stress (i.e. posttraumatic stress disorder symptoms and depressive symptoms) among rescue workers as well as the need for screening and prevention for posttraumatic stress disorder. So far we have shown in an open trial that posttraumatic stress disorder symptoms in critically injured patients can be reduced by taking omega-3 fatty acids intended to stimulate hippocampal neurogenesis.
This study is designed to determine the effectiveness of attenuating posttraumatic distress with omega-3 polyunsaturated fatty acids among Disaster Medical Assistance Team members after the Great East Japan Earthquake, and is named the APOP randomized controlled trial which is currently ongoing. First, we will provide psycho-education on posttraumatic distress, which is common in responders to the Disaster Medical Assistance Team members deployed to the disaster area. Second, observational research will be conducted to evaluate critical incident stress following the completion of medical activities. Third, team members who provide consent to participate in the intervention research will be randomly divided into a group given an omega-3 fatty acid supplement and a group not given the supplements. Outcome will be evaluated at 12 weeks after the supplements are shipped to the team members.
Measures that address critical incident stress in disaster responders are important, but there is no substantial evidence that links such measures with prevention of posttraumatic stress disorder. Thus, any confirmation through this study that the intake of omega-3 fatty acid supplements serves as a simple preventative measure for critical incident stress will be of great significance.
UMIN Clinical Trials Registry, UMIN000005367
On March 11, 2011 at 14:46, a magnitude 9.0 earthquake, the most powerful ever recorded in Japan, and a massive tsunami struck off the coast of the Sanriku region, leaving over 20,000 dead or missing. The earthquake and subsequent tsunami, now known as the Great East Japan Earthquake, was the worst disaster Japan has experienced since World War II, causing psychological trauma among the survivors as well as critical incident stress among the rescue workers.
Even veteran responders with medical expertise deployed to disaster areas may experience significant psychological effects from exposure to the tragic circumstances they witness. More specifically, they may experience a variety of psychological reactions, including irritability, difficulty sleeping, feeling that the situation and work at the disaster site are unreal, recounting disaster efforts, nightmares, avoidance or reluctance to talk about people and objects that trigger memories of disaster areas, feelings of powerlessness in being unable to do anything, strong feelings of self-reproach, and anger. In fact, a study on the effects of critical incident stress in firefighters found an association between work-related psychological trauma and the onset of posttraumatic stress disorder (PTSD) . Moreover, in a study of 355 medical care personnel sent to aid trauma victims of an airline crash, 13.5% developed PTSD within 18 months of the crash . Similarly, in a study of 207 aid workers deployed to the site of the September 11 terrorist attack in New York in 2001, 16.7% developed PTSD and 21.7% developed depression at 13 months after the attack . Appropriate evaluation of critical incident stress and screening and prevention of secondary psychiatric illness are thus crucial tasks.
In the pathogenesis of PTSD, fear memories become excessively consolidated and extinction learning does not progress . Kitamura recently found that the period of hippocampus-dependent fear memory is longer in mice with decreased hippocampal neurogenesis and shorter in mice with active hippocampal neurogenesis , indicating that the level of hippocampal neurogenesis is a crucial factor in determining the period of hippocampal-dependent fear memory. This finding suggests that the fear memories characteristic to PTSD may be controlled by aptly regulating hippocampal neurogenesis . We are currently conducting a randomized controlled trial different from the one described herein to investigate the preventive effectiveness of omega-3 fatty acids for preventing PTSD in physically injured patients (ClinicalTrials.gov Identifier: NCT00671099) since these fatty acids have been confirmed to enhance hippocampal neurogenesis in animal studies [7, 8]. The open preliminary trial found that post-trial PTSD symptoms were significantly alleviated in injured patients who took the omega-3 fatty acids .
The present study aims to determine the effectiveness of attenuating posttraumatic distress with omega-3 polyunsaturated fatty acids among Disaster Medical Assistance Team (DMAT) members who are deployed during the acute disaster phase following the Great East Japan Earthquake. This study named the APOP randomized controlled trial aims to (1) provide psychoeducation on posttraumatic distress common among rescue workers to DMAT members dispatched to disaster areas, (2) assess critical incident stress among the DMAT members following completion of their medical duties, and (3) recruit these DMAT members to a 12-week study investigating the effects of omega-3 fatty acids in reducing stress, with consenting participants randomly allocated to either an omega-3 acid fatty acid supplementation group or a non-supplementation group. The efficacy of omega-3 fatty acids in reducing critical incident stress (PTSD symptoms or depressive symptoms) at 12 weeks will be examined.
The present study is a randomized clinical trial that will compare an intervention group that receives psychoeducation and omega-3 fatty acid supplementation with a parallel control group that receives psychoeducation only.
The DMAT service was established by the Ministry of Health, Labour and Welfare of Japan in April 2005 and operates from the Disaster Medical Center of the National Hospital Organization. DMAT members are physicians, nurses, and operational coordination staff (medical or clerical staff who are neither physicians nor nurses) who are dispatched as a mobile medical team with specialized training that is capable of acting during the acute phase (roughly within 48 hours) of a large-scale disaster and in the event there are multiple injured or sick casualties. Following the Great East Japan Earthquake, DMAT activities commenced on the same day, namely March 11, and concluded on March 22. Recruited DMAT members deployed to the disaster area met the following inclusion criteria: 1) aged 18 years or older; 2) a native Japanese speaker or non-native speaker with Japanese conversational abilities; and 3) physically and psychologically capable of understanding and providing consent for study participation. The exclusion criterion was regular intake of warfarin for at least 3 months before deployment.
Estimation of Sample Size
The required sample size for intervention research was estimated at 48 cases per group. Based on our previous research [9, 10], we estimated that the mean of improvement in the Impact of Event Scale-Revised (IES-R) score as a primary outcome measure would be 10 (SD = 15) for the intervention group and 0 (SD = 15) for the non-intervention group. We set α level at .05 and β at .10. This brought us to our required sample size estimation of 48 cases per group. This study set case numbers above that required, with consideration given in its design for the following: the sample would be recruited from a population different from that of previous studies (i.e. medical assistance members); the control group would receive psychoeducation; and the actual participant number was estimated. Thus, we allowed up to 150 cases for the intervention group and 300 cases for the control group.
Omega-3 fatty acid supplementation in the intervention group
In line with previous research , participants are taking 7 capsules per day, each containing 320 mg of oil following return from their duties in the disaster area. The omega-3 fatty acid composition of each capsule is 70% docosahexaenoic acid and 7% eicosapentaenoic acid. Each capsule is placed in a brown 500-ml polyethylene container with a wide opening. Participants were instructed to take the capsules after eating and additionally told that they may take a full day's dosage at one time. Participants will be contacted to ensure regular capsule intake and safety monitoring at 2, 4, 8, and 12 weeks from the start of the intervention. Whenever inquiries are received from participants, necessary information will be provided to them.
A placebo capsule was not prepared. A leaflet on psychoeducation about posttraumatic distress focusing on critical incident stress was provided to participants and they will be contacted about their situation at 2, 4, 8, and 12 weeks. Whenever inquiries are received from participants, necessary information will be provided to them.
In regard to participant enrollment and group assignment, DMAT members who returned the informed consent form for the omega-3 fatty acid intervention research have been enrolled as participants. Central registration involved assigning participants to groups according to an assignment diagram developed by trial statisticians. Core investigators were single-blinded, and participants were randomly allocated to either the omega-3 acid group or the control group using block randomization. The participants were stratified by sex, and randomization was conducted by permuted block method using a four-person block. Sex was an adjustment factor, as previous studies show that the prevalence of PTSD and major depressive disorder are higher in women than in men . Omega-3 fatty acid capsules were individually shipped to the address designated by each participant following random assignment. All information on case assignments will be disclosed after the final follow-up on participants has been completed and all data secured.
The following describes the explanation and information used in obtaining consent from eligible participants. Participation in the study is voluntary. There is no penalty for declining to participate. Participants may withdraw from the study at anytime without penalty. Information provided covered the following areas: reasons for selection, the names and occupational titles of the researchers, the meaning, purpose, method, research period, expected benefits of participation, protection of privacy, possibility of patents from the study, possible risks or unpleasant physical adverse effects, disclosure of results related to the study, the publication of results without participant identifiers, possible risks, affiliated organizations of researchers and their relationships with the organizations, methods of data use, and period of data preservation.
Basic information on the participants
Basic information obtained from the participants comprised name, sex, age, hospital affiliation, contact information, e-mail address, height, weight, occupation, marital status, number of children, highest education completed, smoking and drinking habits, years of experience, experience of deployment to disaster areas, use of omega-3 supplements, dietary habits, physical illness, and previous history of physical and psychiatric illnesses.
Information about traumatic events
Participants were surveyed about the following items, in addition to those variables identified as risk factors for PTSD in previous research [2, 13]: period of deployment, stress prior to deployment, injury during deployment, experience of saving a child during deployment, experience of contact with corpses, fears of radiation, duration of time spent watching earthquake news reports, and current subjective physical symptoms.
Peritraumatic Distress Inventory
The Peritraumatic Distress Inventory (PDI) is a 13-item questionnaire, developed by Brunet et al , for quantification of fear and sense of helplessness in the trauma cycle (the period during and directly after a traumatic experience). Previous studies have shown that one set of the PDI items predict PTSD symptoms . With permission from creators Brunet and Marmar, we previously created the Japanese version and confirmed its validity and reliability [16, 17].
Impact of Event Scale- Revised
The Impact of Event Scale-Revised (IES-R) is a self-reporting questionnaire about PTSD symptoms that was developed in the U.S. It is the most widely used measure internationally in all forms of disaster-area research . The IES-R is composed of 22 items on the three largest symptoms in the diagnostic criteria of PTSD, namely re-experiencing, avoidance, and increased physiological arousal. Respondents rate symptoms experienced in the previous week. The validity and reliability of the Japanese version of the IES-R has been confirmed .
The Center for Epidemiologic Studies Depression Scale
The Center for Epidemiologic Studies Depression Scale (CES-D) is a self-reporting questionnaire on depression that was developed by Radloff et al . The higher the total score is to the maximum score of 60, the more severe the depressive state. The cut-off score for a mood disorder is considered to be 16 points. Validity and reliability of the Japanese version have been confirmed .
Kessler K6 Scale
The Kessler 6 Scale (K6) is a self-reporting questionnaire designed to effectively screen for psychiatric disorders and mood and anxiety disorders, where respondents rate their condition for the last month . Validity and reliability of the Japanese version has been confirmed . An adequate cut-off score on the K6 for serious mental illness is 0-12 vs. 13 or more .
Resilience Scale and Resilience Scale-Short Version
The 25-item Resilience Scale (RS) and its shortened 14-item version (RS-14) are self-reporting questionnaires developed by Wagnild and Young for quantitative evaluation of resilience . Among European and U.S. scales for resilience, its reliability and validity are considered the most established. We created Japanese versions of the RS and RS-14 with the permission of Wagnild and confirmed their reliability and validity . The present study used the short RS-14 version.
Follow-up Assessment Schedule
Summary of outcome measures of the APOP clinical trial
Past medical history
Detailed information about disaster-related event
The total score on the IES-R at 12 weeks after shipment of the supplements is the primary outcome measure.
Total scores on each of the CES-D, the RS-14, and the K6 at 12 weeks after shipment of the supplements constitute the secondary outcome measure.
Safety Management and Study Monitoring
Safety of the intervention is evaluated by the presence of an adverse event during the observation period. The investigators will contact the participants regarding the presence of any adverse events at 2, 4, 8, and 12 weeks after the start of the omega-3 fatty acid supplementation intervention. When an adverse event occurs, the investigators will rate the degree of the event as either "mild", "moderate", or "severe".
The principal investigator will assess the circumstances surrounding the occurrence of a serious adverse event and/or an event that may affect the future of the investigation. Cases will be reported to an independent data safety monitoring board and the company providing the trial capsules, and related information will be shared with them. The blinding of cases may be discontinued as deemed necessary and information gathered so that the causes behind the occurrence may be investigated. The ethics committee of the facility may also be notified.
The investigation will cease when (1) discontinuation of the study is recommended by the data safety monitoring board due to an adverse event or side effect that makes continuation of the investigation difficult or (2) the principal investigator decides not to continue implementation.
All analyses were conducted according to the intention-to-treat principle. Analysis of covariance (ANCOVA) will be used to obtain differences between the means, 95% confidence interval values, and P values. Covariates for ANCOVA are sex, age, and IES-R scores at baseline. A two-tailed test will be used, with the α level set at .05%. Evaluation by regression models will be conducted as necessary. Validity of the results will be evaluated through sensitivity analysis and filling missing data.
Analysis of the secondary outcome measure will be conducted to add to discussion of the results of the primary outcome measure. Adjustment will not be conducted for data duplication because secondary statistical analyses are exploratory. A two-tailed test will be used, with the α level set at .05%. Evaluation by regression models will be conducted as necessary. Validity of the results will be evaluated through sensitivity analysis and filling missing data.
Time periods during the study
Research will be conducted from April 1 to September 30, 2011. Participant registration for observational research was from April 2 to 22, and participant registration for intervention research was from April 2 to 12. Follow ups are to be conducted after the omega-3 supplement shipments until August 31.
The present study protects the rights and welfare of participants in the spirit of ethical guidelines outlined under the Declaration of Helsinki. The study further respects the ethical principles of the Ministry of Health, Labour, and Welfare of Japan. Confidence can be assured in the ethics, safety, scientific rigor, and reliability of the research. Personal information obtained in the course of the research will be strictly secured to avoid external leaks. Because the study is a dietary intervention, no special compensation will be paid in the event of health damage directly related to the research. The research plan (2010-32) was deliberated upon and approved by the Ethics Committee of the National Disaster Medical Center on April 1, 2011.
Declines in physical and mental health due to critical incident stress in disaster aid workers or rescue workers has been demonstrated in previous research, but specific, adequate measures to counter critical incident stress have not been developed. The development of measures that can realistically be practiced by large numbers of aid workers is extremely important. Six years have passed since the DMAT service was established and little examination of critical incident stress among DMAT members has been conducted thus far. This study is designed to understand the phenomenon of critical incident stress among DMAT members and conduct the APOP clinical trial. The trial will provide omega-3 acid supplements to DMAT members stationed in all regions of Japan as a method to promote mental health without requiring individualized care from a mental health professional.
The use of self-reporting questionnaires while inferior to that of a clinical interview as an assessment of the APOP study research outcomes for PTSD and depressive symptoms, it is a reasonable assessment method given this type of emergency situation. We are currently implementing separate random comparative trials to prevent PTSD in physically injured patients (ClinicalTrials.gov Identifier: NCT00671099) and are evaluating PTSD through structured clinical interviews. The APOP study was designed at a time of crisis, 1 week after the earthquake occurred, and recruiting sufficient participants was considered difficult if a placebo group were to be used. Another limitation of the study is that fatty acid composition of red blood cell membranes could not be measured to confirm intake compliance of the omega-3 fatty acid supplements. With these limitations in mind, we do believe the results of the APOP clinical trial will be of importance: natural and man-made disasters occur across the globe and omega-3 fatty acid supplementation, if found to be efficacious for preventing critical incident stress, could contribute to maintaining the mental health of disaster relief workers in the future.
Acknowledgements and Funding
The authors would like to thank Professor Kaoru Inokuchi for generous financial support. We also thank Dr. Hiroko Noguchi, Dr. Hisayoshi Kondo, Mr. Masayuki Ichihara for coordination with participants and Mss. Kyoko Akutsu and Yumiko Kamoshida for data management and Ms. Hiroko Hamatani for preparation of bottled supplements. Professors Yasuhiro Otomo and Takeshi Terao and Dr. Katsumi Ikeshita joined this study as a member of the data and safety monitoring board. All of the supplements used in the study were supplied by Kentech Co., Ltd., Toyama, Japan. This work was supported by CREST, the Japan Science and Technology Agency. Japan Science and Technology Agency had no role in the study design and conduct, in the collection, analysis and interpretation of the data, or in the preparation, review, and approval of the manuscript.
- Bryant RA, Guthrie RM: Maladaptive self-appraisals before trauma exposure predict posttraumatic stress disorder. J Consult Clin Psychol. 2007, 75: 812-815.View ArticlePubMedGoogle Scholar
- Epstein RS, Fullerton CS, Ursano RJ: Posttraumatic Stress Disorder Following an Air Disaster: A Prospective Study. Am J Psychiatry. 1998, 155: 934-938.View ArticlePubMedGoogle Scholar
- Fullerton CS, Ursano RJ, Wang L: Acute Stress Disorder, Posttraumatic Stress Disorder, and Depression in Disaster or Rescue Workers. Am J Psychiatry. 2004, 161: 1370-1376. 10.1176/appi.ajp.161.8.1370.View ArticlePubMedGoogle Scholar
- Ressler KJ, Mayberg HS: Targeting abnormal neural circuits in mood and anxiety disorders: from the laboratory to the clinic. Nat Neurosci. 2007, 10: 1116-1124. 10.1038/nn1944.View ArticlePubMedPubMed CentralGoogle Scholar
- Kitamura T, Saitoh Y, Takashima N, Murayama A, Niibori Y, Ageta H, Sekiguchi M, Sugiyama H, Inokuchi K: Adult neurogenesis modulates the hippocampus-dependent period of associative fear memory. Cell. 2009, 139: 814-827. 10.1016/j.cell.2009.10.020.View ArticlePubMedGoogle Scholar
- Matsuoka Y: Clearance of fear memory from the hippocampus through neurogenesis by omega-3 fatty acids: a novel preventive strategy for posttraumatic stress disorder?. Biopsychosoc Med. 2011, 5: 3-10.1186/1751-0759-5-3.View ArticlePubMedPubMed CentralGoogle Scholar
- Beltz BS, Tlusty MF, Benton JL, Sandeman DC: Omega-3 fatty acids upregulate adult neurogenesis. Neurosci Lett. 2007, 415: 154-158. 10.1016/j.neulet.2007.01.010.View ArticlePubMedPubMed CentralGoogle Scholar
- Kawakita E, Hashimoto M, Shido O: Docosahexaenoic acid promotes neurogenesis in vitro and in vivo. Neuroscience. 2006, 139: 991-997. 10.1016/j.neuroscience.2006.01.021.View ArticlePubMedGoogle Scholar
- Matsuoka Y, Nishi D, Yonemoto N, Hamazaki K, Hashimoto K, Hamazaki T: Omega-3 fatty acids for secondary prevention of posttraumatic stress disorder after accidental injury: An open-label pilot study. J Clin Psychopharmacol. 2010, 217-219. 30Google Scholar
- Matsuoka Y, Nishi D, Nakajima S, Yonemoto N, Hashimoto K, Noguchi H, Homma M, Otomo Y, Kim Y: The Tachikawa cohort of motor vehicle accident study investigating psychological distress: design, methods and cohort profiles. Soc Psychiatry Psychiatr Epidemiol. 2009, 44: 341-10.1007/s00127-009-0496-4.View ArticleGoogle Scholar
- Hamazaki T, Sawazaki S, Itomura M, Asaoka E, Nagao Y, Nishimura N, Yazawa K, Kuwamori T, Kobayashi M: The effect of docosahexaenoic acid on aggression in young adults. A placebo-controlled double-blind study. J Clin Invest. 1996, 97: 1129-1133. 10.1172/JCI118507.View ArticlePubMedPubMed CentralGoogle Scholar
- Kessler RC, Sonnega A, Bromet E, Hughes M, Nelson CB: Posttraumatic stress disorder in the National Comorbidity Survey. Arch Gen Psychiatry. 1995, 52: 1048-1060.View ArticlePubMedGoogle Scholar
- Schlenger WE, Caddell JM, Ebert L, Jordan BK, Rourke KM, Wilson D, Thalji L, Dennis JM, Fairbank JA, Kulka RA: Psychological reactions to terrorist attacks: findings from the National Study of Americans' Reactions to September 11. Jama. 2002, 288: 581-588. 10.1001/jama.288.5.581.View ArticlePubMedGoogle Scholar
- Brunet A, Weiss DS, Metzler TJ, Best SR, Neylan TC, Rogers C, Fagan J, Marmar CR: The Peritraumatic Distress Inventory: a proposed measure of PTSD criterion A2. Am J Psychiatry. 2001, 158: 1480-1485. 10.1176/appi.ajp.158.9.1480.View ArticlePubMedGoogle Scholar
- Simeon D, Greenberg J, Knutelska M, Schmeidler J, Hollander E: Peritraumatic reactions associated with the World Trade Center disaster. Am J Psychiatry. 2003, 160: 1702-1705. 10.1176/appi.ajp.160.9.1702.View ArticlePubMedGoogle Scholar
- Nishi D, Matsuoka Y, Noguchi H, Sakuma K, Yonemoto N, Yanagita T, Homma M, Kanba S, Kim Y: Reliability and validity of the Japanese version of the Peritraumatic Distress Inventory. Gen Hosp Psychiatry. 2009, 31: 75-79. 10.1016/j.genhosppsych.2008.09.002.View ArticlePubMedGoogle Scholar
- Nishi D, Matsuoka Y, Yonemoto N, Noguchi H, Kim Y, Kanba S: Peritraumatic Distress Inventory as a predictor of post-traumatic stress disorder after a severe motor vehicle accident. Psychiatry and Clinical Neurosciences. 2010, 64: 149-156. 10.1111/j.1440-1819.2010.02065.x.View ArticlePubMedGoogle Scholar
- Weiss DS, Marmar CR: The Impact of Event Scale Revised. Assessing psychological trauma and PTSD. Edited by: Wilson JP, Keane TM. 1997, New York: The Guilford Press, 399-411.Google Scholar
- Asukai N, Kato H, Kawamura N, Kim Y, Yamamoto K, Kishimoto J, Miyake Y, Nishizono-Maher A: Reliability and validity of the Japanese-language version of the impact of event scale-revised (IES-R-J): four studies of different traumatic events. J Nerv Ment Dis. 2002, 190: 175-182. 10.1097/00005053-200203000-00006.View ArticlePubMedGoogle Scholar
- Radloff LS: The CES-D scale: a self-report depression scale for research in the general population. Applied Psychological Measurement. 1977, 1: 385-401. 10.1177/014662167700100306.View ArticleGoogle Scholar
- Shima S, Shikano T, Kitamura T: A new self-report depression scale (in Japanese). Seishinigaku. 1985, 27: 717-723.Google Scholar
- Kessler RC, Andrews G, Colpe LJ, Hiripi E, Mroczek DK, Normand SL, Walters EE, Zaslavsky AM: Short screening scales to monitor population prevalences and trends in non-specific psychological distress. Psychol Med. 2002, 32: 959-976. 10.1017/S0033291702006074.View ArticlePubMedGoogle Scholar
- Furukawa TA, Kessler RC, Slade T, Andrews G: The performance of the K6 and K10 screening scales for psychological distress in the Australian National Survey of Mental Health and Well-Being. Psychol Med. 2003, 33: 357-362. 10.1017/S0033291702006700.View ArticlePubMedGoogle Scholar
- Kessler RC, Barker PR, Colpe LJ, Epstein JF, Gfroerer JC, Hiripi E, Howes MJ, Normand SL, Manderscheid RW, Walters EE, Zaslavsky AM: Screening for serious mental illness in the general population. Arch Gen Psychiatry. 2003, 60: 184-189. 10.1001/archpsyc.60.2.184.View ArticlePubMedGoogle Scholar
- Wagnild GM, Young HM: Development and psychometric evaluation of the Resilience Scale. J Nurs Meas. 1993, 1: 165-178.PubMedGoogle Scholar
- Nishi D, Uehara R, Kondo M, Matsuoka Y: Reliability and validity of the Japanese version of the Resilience Scale and its short version. BMC Res Notes. 2010, 3: 310-10.1186/1756-0500-3-310.View ArticlePubMedPubMed CentralGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://0-www.biomedcentral.com.brum.beds.ac.uk/1471-244X/11/132/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.