To help you understand these concepts and ideas and how Government and not-for-profits are putting them into action SYC has developed this handy explainer.

Troy presenting at AIHW Panel

SYC Manager of Data Analysis and Insights, presenting on the future of data in not-for-profit human services.

Data Linkage

Data linkage is a method used to bring information about the same person from different sources together to create a new, richer dataset. It helps with mapping the journeys people take through various services over time and it is an important part of moving away from program-centric views to developing deeper understandings about the actual lives of individuals and families.

At the AIHW workshop many data linkage projects were discussed. For instance, AIHW were able to link data across the Youth Justice, Child Protection and Specialist Homelessness Service systems to discover that 590 young people had accessed each of those services from 2011 to 2015. Compared to young people who only interacted with specialist homelessness services these young people were more likely to have substance abuse or mental health issues, more likely to be Aboriginal and Torres Strait Islander people, likely to receive more days of support and more support periods from specialist homelessness services agencies, but less likely to receive long-term accommodation.

Big Data

Big data refers to the act of gathering and storing the large amounts of information that are generated daily. This data can be structured, which can be easily stored in databases in an ordered manner and includes things like GPS data, and unstructured, which has no clear format and includes things like social media posts and YouTube videos.

Big data is characterised by the three Vs: data coming in at a higher volume, with wider variety, and greater velocity. For example, more than 500 hours of video are uploaded to YouTube every minute. The data is so large, complex, and updated constantly that traditional data processing software cannot manage it, requiring the development of new techniques. Other considerations are also emerging, including variability (think of something trending on social media causing its popularity to surge and then drop) and veracity (how truthful is the data being collected).

In human services this could mean analysing the data generated by students engaging with online learning platforms to identify young people who are disengaged or at-risk of dropping out of school.

Data Mining

Understandably big data is big, which means it needs new techniques and tools to work with it to find useful insights. Data mining is one way to work with big data sets to find anomalies, patterns and correlations. The results of data mining activities can be used to predict outcomes or describe why something happened. It involves using various models to uncover shared similarities or groupings in historical data (looking for clusters or anomalies) or what has an influence on something else happening (predictive modelling and the relationship between one variable and many others).

For instance, data mining could be used to help prioritise responses to crisis call back services by looking through tens of thousands of text messages for unexpected relationships between certain words and the need for emergency aid.

Predictive Analytics

When programs measure data they are usually based on “lag indicators”, which are effectively measures of what has already happened. To understand what might happen those lag indicators can be used in the field of predictive analytics. Using current and historical data, predictive analytics uses techniques like machine learning and statistical algorithms to identify the likelihood of future outcomes, extending our knowledge from what has happened to include what is likely to happen in the future.

Predictive analytics and simulations are a key part of the future of human service delivery. They can assist the providers of services, like SYC, in their decision-making around what type of intervention should be delivered at what stage of a client’s journey by a specific staff member for a certain period before trying the next intervention if the first thing does not work. For instance, predictive analytics could identify the likelihood of a renter in social housing going into rent arrears in the future and their expected ability to pay it back and avoid eviction. Support workers could then follow a series of recommended steps to intervene early with those individuals while management could allocate staff based on their ability to influence the model, through their experience working with certain types of clients to achieve positive outcomes.

Data Accessibility

Data accessibility is about making it easier to share data in a safe and ethical way to allow for things like data linkage projects to occur. In many cases, collecting data is not the issue but it is usually kept hidden away in organisational or departmental silos.

Data accessibility also encompasses the need to develop shared definitions and understandings. This is important when we want to have a shared understanding of who is a client and what the outcomes are across a range of similar services. AIHW have spearheaded the development of many National minimum data sets (NMDS), which are national agreements on the agreed mandatory data elements to make data more uniform and accessible. The benefit is in something like the Specialist Homelessness Services NMDS things like who is a ‘Specialist homelessness agency’ is, at what point someone becomes a client, and what the various service types are have all been defined. This means there is uniform data about clients’ circumstances, the assistance they receive and the outcomes that are achieved for them to inform policy design and service improvement.


Connected to all of these trends are the issues of ethics, trust, and privacy. Trust is vital for not-for-profits and the people we work with, who are often the most disadvantaged. They want to know that the information they are providing is not going to be used against them unfairly and instead will be used to benefit them. Articles on the selling of data and data breaches are now commonplace. A 2018 survey found that 49% of people believe companies are misusing their data with Government one of the most trusted organisation and social media companies the least. When trust gets too low and people have too many concerns about how private their data will remain, they become unlikely to participate or provide less accurate information. This means decisions are made without having all the right information. Maintaining trust includes only asking clients questions relevant to that program and sharing how that data has been used to improve the services they are receiving.

The future of data is clearly an exciting one and when all the factors described can come together we can tell rich stories about who is using a range of services and identify what is most likely to deliver the best outcomes for that person. As a service provider, this is the end goal we are constantly working towards, improving the lives of the people we support in the most effective way possible.