WAStD - WA Sea Turtle and Strandings Database
Overview
WAStD is a data warehouse for:
Turtle strandings in WA, as reported to the Department of Biodiversity, Conservation and Attractions, WA.
Dugong and sea snake strandings in WA.
Turtle nesting census, which counts nests and tracks on mornings after nesting nights (can involve nests, tracks, predation, nest tags, temperature loggers, egg excavations, hatchling measurements, hatchling emergence track measurements, light pollution observations).
Temperature logger asset management.
Turtle tagging observations.
WAStD is built scalable enough to accommodate other, related, data:
Cetacean (whales and dolphins) and pinniped (seals and sea lions) strandings. While an integration is technically possible, the current point of truth for these strandings is a different in-house database at DBCA with its own response protocols.
WAStD offers as main functionalities:
A user-friendly interface to trawl through the data, do QA and data exports.
An access-restricted data curation portal, through which trained and trusted data curators can enter data from paper data sheets, then proofread and QA the data.
Restricted access to the “backstage” area for trusted data consumers, where they can search, filter and export the raw data, but not change or delete them.
A RESTful API that allows authenticated users to create, update, and download the data.
WAStD integrates in the Departmental information landscape as follows:
Legacy data (starting with Turtle strandings) is manually entered from paper forms.
Legacy data living in legacy systems can be batch-uploaded to WAStD, initially as a read-only copy.
Data collected digitally can be imported automatically into WAStD.
WAStD can batch-upload its data to other corporate data warehouses.
Analytical applications anwering defined management questions (informing monitoring reports, ministerial inquries, conservation planning decisions) can be built right now consuming the WAStD API, and later refactored to consume data from departmental data warehouses, once these become the point of truth for the data.
Departmental data consumers can use the R package wastdr to access data directly from the WAStD API.
wastdr
provides working examples and extensive documentation.
Departmental business related to turtle strandings:
Documented at https://confluence.dpaw.wa.gov.au/display/MSIM/Legacy+systems
If any of the legacy systems were to experience an acute business risk – e.g. data being siloed in outdated software, insufficient database curation functionality (missing sanity checks) corrupting core departmental data, data custodians retiring or not being salaried, outdated datasheets collecting incomplete, inconsistent, incorrect data –, then a solution much like WAStD or BioSys would be required to mitigate that risk, and sufficient care had to be taken to hand over the not always fully documented, often living business knowledge from current custodians to permanent departmental staff.
The roll-out of the improvements in handling turtle strandings will cross over with existing workflows of the above mentioned, out of scope business processes.
WAStD’s design philosophy follows The Basics of Unix Philosophy
The journey so far:
April 2016: Requirements Analysis
July 2016: Implementation
August - Sept 2016: Agile iterations, weekly stakeholder workshops to refine requirements and update business processes understanding and requirements
Oct 2016: Production deployment, start of turtle stranding data entry, “dog fooding” the data entry manual, usability improvements, working on datasheets.
Nov 2016: Development of digital data capture for turtle tracks. Form revised 10 times.
Nov/Dec 2016: 2300+ tracks recorded digitally, replacing paper forms.
Dec 2016: Track app deployed to two more field teams (Karratha, Broome).
Jan 2017: Automated pipeline from digital capture to WAStD.
Jan 2017: Digital form for tracks revised 15 more times to include nest tags, egg, hatchling and logger measurements.
Feb 2017: Revised tracks form used in field.
Season 2017/2018: Six regions join digital data collection of turtle track census.
2018: Threatened species and communities.
Season 2018/19: Bar two programs, all WA regions join digital data collection of turtle track census. One program is relatively small and remote, the other lacks basic literacy among available data collectors, favouring pictogram-based and established solutions like CyberTracker.
Season 2019-20: Migration from ODK Aggregate to ODK Central, ETL via API and R packages wastdr and etlTurtleNesting.
Season 2020-21: Inclusion of turtle tagging in electronic data capture.
2022: Third party access for non-DBCA users enabled.
What is where
WAStD - DBCA staff login
WAStD data curation portal > Curators > Data Curation Portal - privileged DBCA staff
WAStD docs > Manual - public
Turtle Monitoring Program Coordinator docs - - DBCA and external authorised users
2021 Reports - daily added reports, data exports, and QA products - DBCA staff login
Turtle Nesting metadata on data catalogue - pending update. DBCA intranet.
Turtle Tagging metadata on data catalogue - pending update. DBCA intranet.
- Third party access
- Data collection administrators
- Admin: Getting started
- Admin: Prepare field packs
- GIS: Create offline background imagery
- Setup the tablet - first time
- Distribute offline background imagery
- Update the tablet before a season
- Prepare a length reference pre survey
- Providing training
- Training day: mixed real and duplicated training data
- Collector: Collect data
- Admin: Review data
- Admin: Upload data
- Where’s the data now?
- Data collection protocols and training
- Data collection development
- Data Collection Management Thevenard and Delambre
- Data entry
- Data QA
- Data consumers
- Business Analysts
- Glossary
- Aims
- Overview - Turtle monitoring
- Overview - Data management
- Business Process Turtle Strandings
- Business Process Turtle Tagging
- IT process Turtle tag asset management
- IT process Turtle tagging field data collection
- IT process Turtle tagging data curation (field and office)
- IT process Turtle tagging data analysis
- Legacy system: WAMTRAM 2
- Interim solution: ETL to WAStD
- Long term solution: New data entry tool
- Insight from tagging data
- Use case: Turtle Tagging digital data capture
- Use case: Inquiry about tagged turtle
- Gap analysis
- Business Process Turtle Tracks
- IT process Turtle track and nest count
- IT process Turtle track and nest data curation
- IT process Legacy data ETL
- IT process Aerial imagery track count
- IT process Turtle track and nest count analysis
- Legacy system: Ningaloo Track count database
- Use case: Track data collection
- Use case: Track data analysis
- Non-functional requirements
- Business Process Annual Reporting
- How it’s made - the process
- Reproducible Research
- Application maintainers
- Developer documentation