Mission Australia’s journey to data maturity

Posted on 30 Oct 2023

By Rachel Christie, Mission Australia

Award for Excellence in Evaluation 2023 Duncan Rintoul Rachel Christie Cherie Pereth
Award winning team: Mission Australia's Duncan Rintoul, Rachel Christie and Cherie Pereth with their Australian Evaluation Society 'Excellence in Evaluation Systems' award.

Sophisticated inhouse data monitoring and evaluation is a must if purpose-led organisations are to best serve the needs of those who rely on their services, writes Mission Australia’s Rachel Christie.

Mission Australia recently achieved a significant milestone by becoming the first non-government organisation (NGO) to receive the “Excellence in Evaluation Systems” award from the Australian Evaluation Society.

This marked an important moment for Australia's NGO sector and highlighted the increasing visibility of data, evaluation and learning work within our sector.

Our aspiration is that Mission Australia, along with our NGO sector colleagues, can strengthen our inhouse data, evaluative and learning maturity, capacity and practice to directly benefit every person we work with, every time they seek our help.

Previously, NGOs had to outsource many of their data collection, monitoring, learning and evaluation activities to consultants or government funders. We were required to use a range of siloed funder data collection systems.

Often, we weren’t even able to access the data we collected or use it to draw insights. We valued program evaluations only if they were written by consulting firms or academics.

Consequently, NGOs historically under-invested in developing their own internal data, monitoring, evaluation and learning capabilities.

At worst, we collectively dismissed our own abilities to deeply understand the effectiveness of our programs and services.

Moving the needle on data maturity

In recent years there has been a welcome shift.

Increased availability of digital client information systems and the necessity to use them to comply with government contracts has driven the opportunity, seized by many NGOs, to focus on building and sustaining internal capability in relation to data, monitoring, evaluation, and learning.

Those NGOs recognised the value of using data-driven insights to better understand the people they serve, improve the services they provide, and add extra evidence to their courageous voices advocating for people in need.

Mission Australia was one of the NGOs in a position to seize this opportunity.

We are a national non-denominational Christian organisation with more than 160 years' experience in standing together with Australians in need on their journey to independence.

We are a large and diverse organisation with more than 2300 employees who deliver 465 services across Australia, serving around 150,000 people annually.

Our services are focused on our goal of ending homelessness and ensuring people and communities in need can thrive.

In 2019, we were using an outdated and end-of-life client information system in our community services and were about to launch a massive technology project to modernise the way we collected and used client information in our services.

We knew that driving a culture of evidence-informed decision-making and practice was going to be a key enabler of our ability to achieve our ambitious goals in our new (at the time) 2020–2025 Strategy.

In 2020, we formed our multidisciplinary Centre for Evidence and Insights, with the objective of inspiring curiosity for evidence, leading to learning and action to increase the impact of our services.

A critical function of the centre is to implement an integrated and sustainable monitoring, evaluation and learning system. To do this, we took an organisation-wide view, deliberately choosing a path of internal capacity building, investing in:

  • the skills, mindset, culture, capacity building and professional development of our people
  • technology platforms to deliver modern, fast, near real-time data to monitor the effectiveness of our services.

Now, with a strong community services client information system in place, we can focus on high impact data analysis and evaluations, and on building the translational process to inform frontline practice, leadership thinking, decision-making, strategy and our advocacy on behalf of people in need.

RC headshot April22
Mission Australia national manager, Centre for Evidence and Insights, Rachel Christie.
"These investments have led to insights that have enhanced our efforts to deliver on our organisational purpose."

Our monitoring, evaluation and learning cycles are a culmination of Mission Australia’s journey to establish an integrated approach to evidence, insights and learning across our organisation.

Key foundational work has included developing our organisation’s outcome-focused 2020–2025 Strategy, creating measures of progress in achieving it, upgrading client information systems, improving data literacy, increasing technology automation, building reporting and visualisation capacity and scaling an organisational approach to impact measurement.

A bumpy road

On this journey, we have made mistakes.

We started designing technology and data solutions primarily to meet funder requirements. While that is important, the solutions did not stick.

Those early developments did not feel useful to our frontline staff, nor did they help facilitate continuous improvement.

While it was challenging at the time, we chose to completely reorient our thinking. Now, we have a mindset where the most important design driver of the technology is neither the technology itself nor the funder contract, but the people and processes.

That is, the highest priority is enhancing the good practice behaviours of the workforce and the processes that support them.

Recently, one of our case workers said: “I love explaining the process to my clients, as I feel like it shows them that Mission Australia cares about the outcomes and is committed to continual improvement.”

To be frank, increasing data maturity is culture change.

A huge component of our data maturity journey has been in growing a culture of curiosity for evidence within our frontline teams. Innovation, improvement and impact needs to grow from the person-to-person interactions between our workers and the people we serve.

A few ways we have helped facilitate this are by:

  • focusing on collaboration. This means we value participant, staff, community and stakeholder knowledge. This helps create a shared commitment to using findings for improvement. Wherever possible, we do this through co-design, shared decision-making and group ownership.
  • focusing on action. Our technology platforms also support action and use of data through access to fast, near real-time data to monitor the effectiveness of our services. Our suite of interactive dashboards provides insights to the right people at the right time, to inform service improvements, advocacy and decision-making.
  • embracing capacity building at scale. The likelihood that initiatives will achieve positive outcomes is increased when staff have the capacity to plan, monitor and evaluate their own programs. An example of this is our national impact measurement program that supports more than 400 services to measure client outcomes and use that data to inform their practice.
  • not prescribing one specific methodology. Rather, we apply appropriate evaluation methods for different contexts, purpose and key questions, in consultation with frontline staff and community stakeholders, to ensure methods are culturally acceptable, suitable, ethical and inclusive.

These investments have led to insights that have enhanced our efforts to deliver on our organisational purpose.

Turning raw data into impactful reporting

One recent example is our Homelessness Impact Report, with our data showing that over the past three years there has been a growing demand for Mission Australia’s homelessness and housing support services.

We were particularly concerned to see a 50% increase in people who sought help from our services after they’d become homeless, rather than when they were at risk.

We found that when people came to one of our homelessness services early, while not at risk of becoming homeless, nearly all (93%) were able to keep living in a stable home.

However, only a third of people who were actually homeless exited into a secure long-term home, because of the severe lack of safe, secure and affordable housing.

Our data confirmed that the length and intensity of support are critical factors that contribute to positive housing outcomes for people who are homeless.

Having readily available data about our services and people’s outcomes tells us what is working, and what isn’t. In this example, investing strongly in early intervention and prevention for people experiencing homelessness clearly works.

"The most effective data maturity strategies facilitate learning from evidence at all levels – from frontline workers to the boardroom."

Future solutions

Together with philanthropists and donors, drawing from these findings, we are working on a range of innovative programs in this area.

Conversely, the critical lack of affordable, appropriate housing stock is contributing to increasing levels of homelessness in Australia – so much so that the homelessness crisis has escalated into an emergency.

Our data on demand for homelessness services plays a major role in our advocacy message to governments: invest in far more social and affordable housing to meet the shortfall.

The most effective data maturity strategies facilitate learning from evidence at all levels – from frontline workers to the boardroom.

Of course, organisational learning enabled by strong data maturity is a wonderful aspiration.

Realistically, however, designing and building solutions that support this ambition in resource-constrained NGOs takes years of foundational effort towards this long-term vision.

Show me the money

A significant obstacle is money.

As a rule, government and philanthropic contracts don’t cover data maturity initiatives, data collection technology, internal capacity building or service-learning activities, although some now do include evaluations.

Usually, this type of work is “outside of scope” of program guidelines.

As a large organisation, we are scraping the barrel to fund this work from our limited reserves, but many great locally based organisations are not able to do even that.

We spend a lot of time trying to convince governments that funding the true cost of service delivery means building into service contracts the legitimate service-related costs of data, technology, monitoring, evaluation and learning activities.

We also talk to governments about minimising the cumulative administrative burden on NGOs of inconsistent data definitions and processes, and unnecessary requirements.

The more streamlined they can make their contracts’ data requirements, the more time NGO frontline staff can spend on supporting vulnerable people.

Our conversations with governments are continuing.

However, our ambition is to see an elevated, consistent approach to data, evaluation, evidence and insights across the NGO sector to better serve the people we work with.

We know from our own experience that improvements are possible.

At Mission Australia, we are proud of our continued investment in the long and sometimes painful journey to build the foundations of our internal data and technology capabilities and our monitoring, evaluation and learning cycles.

We are pleased to see many of our fellow NGO organisations on similar journeys, and to work with them collaboratively to share what we have learned and celebrate each other's successes, through forums like the Data Catalyst Network.

We are excited that governments are starting to listen, recognise the value of NGO-held data, and collaborate with NGOs on data and insights projects.

In 2023, Mission Australia was the first NGO to take home the Australian Evaluation Society’s “Excellence in Evaluation Systems” award – but we won’t be the last.

Rachel Christie is the manager of Mission Australia’s Centre for Evidence and Insights.

More opinion

Become a member of ICDA – it's free!