Making the Client’s Voice Heard in Our Data
15 Oct 2019
“How do we know if what we are doing is actually helping people?”
This is a question we ask ourselves every day at SYC as we seek to positively affect the lives of our clients. Naturally, this is followed up with deeper questions around how we measure our impact and what factors contributed towards the achievement of positive outcomes.
This is so important to us that we previously wrote a whole article on the value of measuring outcomes and understanding our impact. In a follow-up to that piece, we want to highlight how we make sure that the voice of our clients is heard in our data collection. We think it is vital to have that subjective view from our clients on the progress they are making on their journey to independence. This complements our other data collection to guide our understanding of how effective our services are and influence our continuous improvement and new service design.
Historically, we measured outcomes based on the requirements of individual program funding agreements and separately gathered client feedback. While this met the needs of our funders it did not capture all the outcomes achieved by our clients that contributed to them achieving independence. In 2015 we worked to bring these two things together in our custom client management system and a key component of that was the Personal Resilience Survey (PRS).
The PRS was developed based on our experience using positive psychology principles. It allows us to measure an individual’s resilience based on their strengths and capacity to manage challenges across four focus areas (Home, Wellbeing, Learning, and Work).
Clients self-assess their level of subjective resilience when they start working with us and every three months after that. When someone seeks out our services they are likely to be experiencing difficulties in their life, such as lacking housing or being disengaged from school. At this stage they are less likely to feel like they have the resilience and strength to deal with those issues. Case managers use the results to help the client develop their goals for the next three months and dig deeper into their progress or their declines in resilience. By working to address those issues and achieve their goals we build their capacity and, ultimately, help them realise they are resilient with the strength to manage the challenges in their life.
Over time we can learn about the experiences of the individual and their feelings in a direct way. When a client completes the survey, we are provided a score. Higher scores indicate a greater level of resilience, a greater amount of strengths and higher capacity to manage challenges. Through our service delivery we expect the PRS score to improve with the client feeling that they have improved levels of resilience and the ability to live an independent life.
The Story So Far
Since implementing the PRS, 925 clients have used the tool and we are seeing some interesting trends to aid service delivery.
One of the key things we have noticed is clients are entering our services with lower levels of resilience. Since implementing the PRS in 2015 initial PRS scores are 9% lower. As more services have adopted the tool, more clients who were experiencing crisis have been surveyed.
The declines are even more stark when we look at what is driving the decrease in overall resilience. Compared to when the tool was first introduced clients have significantly worse views of their strengths and less confidence in their ability to deal with challenges around Learning and Work. When clients are in crisis, such as experiencing homelessness or exiting the justice system, addressing their housing is their initial primary goal which can cloud their view of their resilience in other areas.
We can also look at the data based on demographic characteristics. For instance, First Nations clients had a far more positive view of their sense of self, their future, and their purpose scoring 9% higher than non-First Nations clients at entry. Young men were also more likely to have a negative attitude towards work, scoring 6% lower. These insights help to direct case management and think about how services or staffing may need to adapt to improve these figures and close the gap between different cohorts.
Beyond high level views on the trends or patterns in our data the PRS also allows us to tell detailed client journeys. For instance, one of our clients in our homelessness services was referred to our Independent Living Skills (ILS) program. They had struggled with looking after themselves, eating a healthy diet, isolation, and maintaining their property, particularly clutter. This was reflected in their PRS scores. They had a low level of resilience, driven by low levels of wellbeing, a poor view of their strengths and limited ability to deal with challenges.
With support from their ILS worker the client learnt how to care for their property, prepare more nutritious meals, re-engage with health supports, and get back on track to complete their study and move into work. Removing some of these barriers saw improvements in their overall PRS with each focus area and their overall strengths and ability to deal with challenges improving. There are still improvements to be made but the client now feels like they are more independent and have a safe place to stay thanks to the support to connect with services and learn important skills.
The PRS also provides the opportunity for clients to reflect on the services SYC provides and how they have helped them. The feedback is an important reminder of the impact working with SYC has had on their life. Being able to connect the client’s resilience score with their feedback is a powerful tool to measure our impact and ensure that the voice of the client is firmly present in our data collection and analysis.
I’ve been able to have a bed to sleep on and food to eat. With this I was able to concentrate at TAFE and go back to my studies.