The Trauma Resilient Community Institutional Assessment

The Stakeholder Advisory Committee (SAC), a committee made of organizational representatives and researchers, chose the mezzo-level research topics that analysts studied and which groups this information should be collected from. 

The focus of study for the mezzo-level research was on agencies and organizations that serve children and their staff. The SAC asked analysts to include the following kinds of organizations (called child-serving organizations in the report) when they collected data: 

• Winston-Salem / Forsyth County Schools, 
• child care providers, 
• home visitation programs, 
• social service providers (for example Department of Social Services and the Department of Public Health), and
• medical service providers.

The topics that the SAC asked analysts to study were: 

• general information about what Forsyth County is like, especially in relation to things that could impact experiences of trauma; 
• the number and kinds of ACEs staff at child-serving agencies have experienced themselves; 
• how many staff members at child-serving agencies reported having been discriminated against; 
• how (and the extent to which) staff at child-serving agencies had been impacted by COVID-19;
• how much staff at child-serving agencies experience stress or trauma from their work with clients who have experienced trauma; 
• how much staff at child-serving agencies know about the TRC model; 
• how well the work environment, staff practices, and leadership practices at child-serving agencies line up with the TRC model; and
• how staff at child-serving agencies have experienced or observed structural violence and racial justice practices where they work. 

To gather general information about Forsyth County, the SAC asked analysts to read and summarize reports that had already been written by other agencies describing the community that were asset-based and / or conducted by researchers who identified as Black, Indigenous, and People of Color (BIPOC) or by organizations lead by people who identified as BIPOC in the community. More details on how this was done can be found in the technical report. 

Analysts worked with the SAC and researchers from the Center for Trauma Resilient Communities (CTRC) to design a survey for organizations to take; this survey gathered information about staff experiences of ACEs, discrimination, COVID-19, and stress from working with clients; knowledge about the TRC model; and how well organizations’ environment, staff practices, and leadership practices lined up with the TRC model. It included several series of questions, called scales, that have been tested in other communities to measure things that may otherwise be difficult to measure with just one or two questions, like what style of leadership exists in an organization or what the social environment of an agency is like. 

The following scales were used in this survey: 

• The Pediatric ACEs and Related Life Events Screener (PEARLS) for adults was used to measure experiences of ACEs;
• the Secondary Traumatic Stress Scale Survey was used to measure symptoms of stress that staff may experience from working with clients with trauma; 
• the Environmental Assessment Survey was used to measure the physical and social environments of the organization and the seven commitments of the TRC model; 
• the Trauma-Informed Staff Practice Survey was used to measure how well staff practices lined up with the TRC model; and 
• the Global Transformational Leadership Scale was used to measure transformational leadership, a style of leadership that is aligned with the TRC model and important for making organizational changes. 

Analysts reached out to leaders of the organizations identified by the SAC and asked them to invite their staff to participate in the survey online. More information on how the survey was designed and analyzed and  who answered the survey and the organizations they worked for can be found in the technical report. 

Analysts were not able to find any scales for measuring structural violence or racial justice practices for staff working at organizations. To gather this information analysts conducted phone and video call in-depth interviews with staff at the organizations identified by the SAC as well as staff at some of the nonprofits and organizations that help coordinate early childhood work in Forsyth County. (The SAC recommended those additional organizations for the interviews.) To conduct the interviews, analysts asked organizational leaders to invite or recommend staff members to participate in the interviews. More information on how the interviews were conducted and who participated can be found in the technical report.

Information Gathered from the Surveys

To analyze the information gathered in the surveys, analysts used instructions provided by the researchers who wrote the scales to calculate the scores. They also ran tests to make sure that people answered questions consistently, a sign that the scales were working as expected and made a note in the report if something about how people answered the questions in the scale seemed unusual. (The statistical results of these tests are also included in Appendix C of the technical report.) If someone answering a survey skipped a question in a particular scale, their answers on that scale were not used, but the information they provided on other scales was used. 

When comparing how scores were different for different groups of people answering the survey, analysts ran tests to make sure that they were at least 95% sure that the differences in scores they saw were real differences between those groups that were not the result of random chance. Information from specific groups was only included in graphs in the technical report if analysts were at least 95% sure that differences existed between one or more groups. For example, some graphs may include information about responses by income while others do not; this information was only included if these kinds of differences were found between at least two income groups. If there is more than one group, the groups that were different from each other are described in the text under the graph. In the same way, analysts only described differences in key findings, presentations, and other materials if they were at least 95% sure that the differences were large enough to not be the result of random chance. (Specific statistical tests and their outputs are available in Appendix A of the technical report.) More information about how the survey data was analyzed can be found in the technical report. 

Information Gathered from Interviews

Analysts asked the staff members they interviewed one open-ended question about structural violence, and the rest of the interview questions were based on a survey instrument, the “Racial Justice Assessment Tool” developed by the Western States Center, that helps organizations understand how much their organization is doing to address racial equity across different areas, like workplace culture and policies. Recordings from these interviews were transcribed, and analysts reviewed the transcriptions to identify major themes. All of the major themes reported were agreed on by two analysts. More detailed information on how interviews were analyzed is available in the technical report.

• These surveys and interviews were conducted in the late spring and early summer of 2021. Organizations and their staff may have changed environments and practices since then, especially since CTRC has continued to work in the community.
• The people who ended up taking surveys and participating in interviews for this project were not selected randomly. The survey instrument was very long and dealt with sensitive issues; some of the questions may not have been easy for respondents to understand or relevant to their workplaces, which may have made some people more likely to skip questions or not finish the survey than others. We can’t know if experiences and perspectives of staff members who participated are the same as those who did not participate. 
• Analysts were only able to study how people answered questions differently by looking at one part of the respondents’ identities at a time (for example, looking at how questions were answered by field of work or level of education). However, analysts were not able to measure or control for cases where these identities may overlap. (In the case of the earlier example, people with doctoral degrees may also be more likely to work in pediatric offices. And, in cases where analysts saw that people with doctoral degrees and people employed in pediatric offices answered questions in similar ways, analysts were not able to tell if people answering the surveys with doctoral degrees were actually more likely to give a particular kind of answer because of their education or if it was really because they were more likely to work in pediatric offices and the real difference is because of something about working in a pediatric office.)
• When organizations and staff were invited to participate in surveys and interviews, they were told that the answers given by their organizations would not be identified in public reports. Analysts did notice that there were consistent differences in how staff from different organizations answered questions, and participants across the whole survey gave a wide range of answers to questions. Scores for individual organizations are likely different from the total scores for all of the participants. As long as an organization had enough staff members participate so that analysts can report on their survey results without violating the privacy of anyone who answered the survey, individual organizations will receive follow-up reports with information just for their organizations. But, those reports will not be made public by Forsyth Futures. 
• For the interviews about racial equity practices, some participants were not familiar with the TRC model or with some of the terms used, and the researchers provided clarifications. This may have impacted their responses.

In response to a request from the expanded advisory committee, Forsyth Futures analysts created diagrams to communicate how data, money, and influence were distributed among stakeholders involved in the mezzo-level analysis.

The diagram above shows how data was shared and distributed across stakeholders in the mezzo-level assessment. Forsyth Futures staff collected data from organizational staff, analyzed that data, and then shared it with organizational staff members, academic partners, the Stakeholder Advisory Committee, Community Partners for Change, Crossnore, and MDC. Crossnore and MDC shared that data with the Kate B. Reynolds Charitable Trust.

The diagram above shows the distribution of funding among stakeholders for the mezzo-level assessment. Funding was distributed from the Kate B. Reynolds Charitable Trust to Crossnore. Crossnore distributed that funding to Forsyth Futures and academic partners from the Center for Trauma Resilient Communities. Forsyth Futures distributed funding to some organization staff by compensating interview participants for their time and entering child care providers into a drawing to win gift cards. Interview participants were compensated for their time because of the increased time commitment associated with participating in a 45 minute to an hour long interview, and child care providers were provided with an opportunity for incentives because past work with child care providers has indicated that completing surveys is disproportionately burdensome for respondents working in those settings.

The diagram above shows the distribution of influence among stakeholders for the mezzo-level assessment. Influence was defined as the ability to meaningfully impact decisions that were being made about the project. In their role as a funder, the Kate B. Reynolds Charitable Trust had the ability to exert influence over Crossnore. As the key convener, Crossnore had the ability to exert influence over the initial Stakeholder Advisory Committee convened to advise on the project. As the grantee supervising Forsyth Futures as a subcontractor, Crossnore had the ability to exert influence over Forsyth Futures. The initial Stakeholder Advisory Committee and academic partners at the Center for Trauma Resilient Communities provided feedback and advice that influenced the methodological decisions made by Forsyth Futures. Staff at surveyed organizations and members of the Community Partners for Change, which convened after the research was completed, did not have the ability to exert influence in this project.