Overview
Arinco partnered with the Western Australian Biodiversity Science Institute (WABSI) and the Western Australian Marine Science Institution (WAMSI) to deliver a secure, scalable data platform on Azure Databricks, that enables trusted collaboration between public and private environmental stakeholders. It needed to ensure sensitive scientific data from different companies could be shared and analysed securely, while aligning to rigorous governance and compliance requirements. Arinco deployed a proof-of-value Azure Databricks solution, underpinned by best practice infrastructure-as-code, data pipelines and strict security controls. Successfully demonstrated to stakeholders, the platform has helped build trust, supported informed environmental decision-making and laid the groundwork for broader data sharing. With tools like maps, reports and forecasting, it empowers decision-makers to balance development with conservation goals.
About WAMSI and WABSI
The Western Australian Biodiversity Science Institute (WABSI) plays a vital role in biodiversity research and data sharing across WA. As a collaborative mechanism, it brings together government, industry and academic partners to improve the availability and utility of biodiversity information to support better environmental decision-making.
The Western Australian Marine Science Institution (WAMSI) is a collaborative, independent research organisation that brings together state and federal government agencies, industry and academic institutions. Its mission is to advance the understanding and sustainable management of Western Australia’s unique environment.
WABSI and WAMSI facilitates peer-reviewed scientific research that helps inform environmental decision-making across the region. By adopting an unbiased scientific approach, WABSI and WAMSI aims to help proponents, regulators and decision-makers balance development goals with conservation imperatives, ultimately seeking optimal economic, environmental, and social outcomes.
Key considerations for the SEAF project
WABSI, in collaboration with WAMSI, set out to transform how environmental data is shared and used across the state. With a vision to protect WA’s unique biodiversity while supporting sustainable development, the two organisations launched the Shared Environmental Analytics Facility (SEAF) project, a groundbreaking initiative designed to create a centralised data platform for environmental data sharing.
The aim of SEAF is to bring together data from private, public and research sectors to enable transparent, cumulative impact assessments across environmental zones. This includes data from leading mining companies as well as various government agencies and academic institutions. For the first time, stakeholders would be able to assess the cumulative environmental impact of projects within a single environmental region – a crucial step toward informed decision-making and safeguarding WA’s unique biodiversity.
No existing solution could securely consolidate and govern public and private data at the required scale. WABSI and WAMSI set out to deliver a platform that could protect privacy, ensure data accuracy and governance, and provide access to actionable insights.
The SEAF project aims to transform, on a global scale, how environmental approvals and assessments are prepared and progressed with the latest digital data available.
An industry-leading data platform
Arinco partnered with WABSI and WAMSI to develop a comprehensive and secure data platform built on Azure Databricks, designed to facilitate the secure exchange of environmental data.
Arinco leveraged best practice infrastructure-as-code for the solution, deploying it via Azure DevOps and Databricks Asset Bundles. This ensured a scalable, flexible and streamlined deployment process. A separate dedicated Databricks workspace was also provisioned for each contributing company, while a central collaboration zone was established for data transformation and analysis.
To standardise data ingestion, Arinco developed pilot notebooks to handle and process data within the collaboration zone. These notebooks automatically align incoming groundwater data to a common standard, applying data constraints to ensure reliability and integrity. This infrastructure ensures that the data is trustworthy and ready for detailed analysis, addressing key concerns around data privacy and accuracy.
The platform not only provides tools for cumulative impact assessments but also generates valuable products such as maps, reports and forecasting tools. These features enhance research and decision-making, supporting the broader environmental goals of the project. In line with the recommendations from the Independent Review of the EPBC Act, the platform enables detailed environmental analysis to assess the impact of multiple projects within a region, paving the way for broader regional adoption and more informed environmental and commercial decisions.
Initial outcomes
Following Phase 1, Arinco successfully demonstrated the value of a shared, secure data analytics platform, leading to strong stakeholder engagement and endorsement. The pilot solution was successfully built and tested, with collaboration from external parties including ASX listed companies, testing relevant data in the platform to ensure real-world applicability and performance in a corporate setting.
Key outcomes include:
- Platform delivery: The secure, scalable Databricks platform was successfully established as a proof-of-value, laying the foundation for the initiative.
- Building stakeholder trust: Strong stakeholder trust was built in the platform’s data, which is essential for driving greater data sharing and improving outcomes for all involved.
- Successful platform demo: The platform was successfully demonstrated to all stakeholders, receiving positive feedback and securing a commitment to upload additional data, positioning it as the primary source of truth for groundwater monitoring data in the region.
- Data governance and security: Best practice CI/CD, data standardisation, and access control mechanisms were implemented, ensuring robust data governance and security across the platform.
- Environmental data analysis: The platform showcased its ability to analyse shared data in alignment with environmental goals, enabling more informed decision-making.
- Enhanced environmental analysis: WABSI/WAMSI was empowered to conduct more detailed environmental analysis, with the potential for the platform’s expansion to additional regions, leading to greater environmental and economic benefits across Western Australia.
- Platform expansion and adoption: A solid foundation was laid for broader platform expansion and regional adoption, focused on delivering both improved environmental and commercial outcomes.
The SEAF initiative has garnered significant media attention, including an official launch event attended by key ministers. It has been recognised for its potential to revolutionise environmental assessments and promote biodiversity outcomes in Western Australia.
What’s Next?
With the platform’s value now proven, the next step is onboarding all available data to unlock advanced analytical use cases. This will create the most comprehensive dataset for the region, supporting deeper insights and decision-making.
As part of Phase 1, Dataworks Secure Data Engine, was successfully implemented. Future phases will include Databricks Clean Rooms, offering enhanced scalability and seamless integration with the existing Databricks environment.
