Organizational Change Process

Background

The QIC-WD team engaged with the Department of Milwaukee Child Protective Services (DMCPS) in 2017 to conduct a thorough needs assessment to understand the root causes underlying a high turnover rate among Initial Assessment Specialists. In 2016, the overall turnover rate, including both external turnover and internal movement, was just under 49%, with a little over 11% due to internal movement and 37% due external turnover. The Organizational Social Context (OSC) baseline survey showed that there were several aspects of both culture and climate that were in need of improvement, including: 1) a hierarchical decision-making structure, 2) high role conflict and change resistance among staff, and 3) low sense of personal accomplishment and cooperation among staff. Focus groups with supervisors and caseworkers and a root cause analysis with the project’s implementation team confirmed these issues with culture and climate. The QIC-WD team and the steering committee chose to adopt Availability, Responsiveness, and Continuity (ARC) as the best intervention to meet the needs and priorities of DMCPS. The theory of change was that improvements in shared leadership and staff empowerment would lead to improvements in organizational culture and climate, job satisfaction, turnover, and case practice. 

ARC is an organizational change process aimed at improving organizational culture and climate and the availability, responsiveness, and continuity of services in human service organizations. ARC teams (made up of workers and supervisors) assess problems impacting the organization and use data to develop proposals for change to submit to the Organizational Action Team (or OAT, comprised of leadership). Upon approval by the OAT team, proposed initiatives are implemented and evaluated. Between 2019 and 2021, there were a total of six ARC teams formed to address issues in the organization. 

Workforce Demographics

Among DMCPS team members in the study sample, 76% were female, 54% were White, 37% were African American, 8% identified as Hispanic/Latinx, 6% identified as bi-racial or other, 1% were Asian, and less than 1% identified as American Indian/Alaskan Native. The average age was 36 years and the average tenure in child welfare was just under 10 years. The majority (54%) had bachelor’s degrees, of which 20% were in social work. An additional 28% had master’s degrees, of which 15% were in social work. 

Evaluation Findings

An evaluation of the implementation of ARC was accomplished through surveys administered quarterly to ARC Team members (N=130 responses), ARC Team leaders (N=31 responses), and OAT Team members (N=37 responses) from 2019 to 2021 (N=198 total responses). Ratings of ARC meeting quality were high, ranging from 86% to 94% endorsement of quality. Adherence to ARC principles ranged from 80% to 95%, with the lowest ratings in 2020 at the height of the pandemic when adaptations were being developed. Ratings of satisfaction with the initiative were more variable, ranging from 70% to 82%, with differences by years (lowest in 2020) and by group (lowest for OAT team). 

The outcomes evaluation utilized a mixed-methods (quantitative surveys and qualitative interviews and focus groups), time series design for this agency-wide implementation of ARC. The evaluation focused on three key areas: 1) general organizational culture and climate domains, 2) ARC-specific domains, and 3) turnover. 

General organizational culture and climate was measured using the OSC scale at four points in time: 2018 (N=109), 2019 (N=89), 2020 (N=81), and 2021 (N=71). Scores for Milwaukee prior to the intervention were all categorized as needing improvement, with the exception of organizational commitment. Over the course of the intervention, OSC scores improved from falling within the most extreme range of needing improvement to normal ranges for every dimension. 

The impact of ARC on other target outcomes was evaluated using a variety of standardized scales in quarterly surveys to ARC and OAT team members, with a total sample of 106 responses over three years. Data indicate that perceptions of procedural justice, psychological safety, and conflict resolution improved for both ARC team members and leaders but declined for the OAT from 2019 to 2021, suggesting that the experience of leadership involved in ARC was different from caseworkers and supervisors. The impact of ARC on perceptions of leadership, supervisor support, peer support, organization support, and diversity support was evaluated through workforce-wide surveys administered each year. There were improvements in scores from 2019 to 2020 for several dimensions, including a decrease in authoritarian leadership and an improvement in supervisor, organizational, and diversity support. All areas of support were highest in 2020, suggesting that DMCPS responded favorably to the challenges of the pandemic. There was a decrease in perceptions of peer support, which was also confirmed by a separate survey on the impact of COVID-19 on the organization. Another set of domains evaluated through the annual workforce survey included individual and organization openness to innovation/change and grit, all of which showed improvements over the three years. Qualitative interviews and focus groups with a total of 39 participants confirmed improvements in psychological safety and leadership support; however, they noted the need for continued focus on supervision, diversity, and inclusive decision making with transparent communication. Many who were not directly involved in an ARC team expressed feeling “in the dark” about the initiative and ARC team projects. Those involved in the ARC and OAT teams identified a need to consider sustainability of key pieces of the initiative (e.g., psychological safety) and sustainability of the specific projects beyond the typical ARC processes.

The final area of outcomes evaluation was turnover. The analysis of turnover data from 2016 (two years prior to QIC-WD involvement) to 2021 showed a decrease in overall turnover from 36.4% to 14.6%. ARC team members were half as likely to resign or retire as other employees and more than double the percentage of ARC team members were promoted or transferred teams. Much of the turnover (39%) was within the first year of employment and suggests the need for Milwaukee to work on recruitment and selection processes. 

Overall, despite challenges with implementation related to training fidelity and the impact of the pandemic on specific intervention activities, there were positive changes in culture and climate as well as turnover associated with the ARC intervention period. The evaluation was limited by the single case study design and potential confounds of the pandemic and key leadership changes, but results are promising for the potential for ARC to impact a public child welfare agency with significant challenges. 

The Team

This project would not have been possible without the partnership of DMCPS. They dedicated staff time to support the initiative and its evaluation. They were led by Site Implementation Coordinator Lilia Figueroa, preceded by Laurice McGinnis. Clinton Herbert provided support with accessing and understanding agency data.

The QIC-WD would like to acknowledge the work of the team members who contributed to the implementation and evaluation of ARC. QIC-WD team members included Becky Antle, Lisa Merkel-Holguin, Megan Paul, Maggie Thompson, Ashley Long, Jonathan Litt, and Robert Blagg. Special thanks to the ARC specialist, Tony Hemmelgarn.