R/processing.R
process_quality_check.RdThis function passes through passive acoustic telemetry datasets through some basic quality checks (see Details). Following data processing, these provide a useful `final check' prior to analysis.
process_quality_check(acoustics, moorings, ids)A dataframe which comprises passive acoustic telemetry detection time series. This must contain the following named columns: `timestamp', a POSIXct vector which defines the time of each detection; `receiver_id', a unique identifier of each receiver; and `individual_id', a unique identifier of each individual (see dat_acoustics for an example).
A dataframe which contains passive acoustic telemetry receiver metadata. This must contain the following named columns: `receiver_id', a unique identifier for each receiver deployment; `receiver_start_date', a POSIXct vector which defines the time of each receiver's deployment; and `receiver_end_date', a POSIXct vector which defines the end of each receiver's deployment (see dat_moorings for an example). If objects of class Date are provided for `receiver_start_date' and `receiver_end_date', these are coerced to POSIXct objects with a warning.
A dataframe which contains the passive acoustic telemetry individual metadata. This must contain the following named columns: `individual_id', a unique identifier for each individual; `tag_start_date', a POSIXct vector which defines the time of each tag's deployment; and `tag_end_date', a POSIXct vector which defines the end of each tag's deployment (see dat_ids for an example). If objects of class Date are provided for `tag_start_date' and `tag_end_date', these are coerced to POSIXct objects with a warning.
For each check, the function returns a message or a warning with relevant details.
The function implements the following checks:
Valid receivers. acoustics should only contain receivers recorded in moorings; other receivers may be included in centralised databases (e.g., from other projects) and often need to be removed.
Valid detections (at receivers). Observations at receivers should occur during their deployment windows; other observations may be included in centralised databases due to receiver checks, range testing or re-deployment elsewhere.
Valid tags. acoustics is checked for any unknown tag IDs. These may be due unrecorded use of tags for receiver checking or range testing, other tagging programmes or type A false detections.
Valid detections (of tags). As for detections at receivers, all detections of tags should occur during their deployment windows.
False detections. False detections should be flagged.
process_quality_check is mainly designed to be implemented after data-processing has already taken place as a basic `final check' for common issues in passive acoustic telemetry datasets. For each check, the function returns a message or warning depending on the outcome; subsequently, the most appropriate course of action (e.g., retention versus removal of flagged observations in acoustics will depend on the context). Other important checks -- such as checking for receivers which were lost and later recovered, excluding observations during receiver servicing dates, excluding observations during tag capture events and further investigation of false detections -- may be required.
#### Prepare data
## All data have previously passed false detection filters (see glatos::false_detections())
dat_acoustics$passed_filter <- 1
## Times should be in POSIXct format
dat_moorings$receiver_start_date <- as.POSIXct(dat_moorings$receiver_start_date)
lubridate::tz(dat_moorings$receiver_start_date) <- "UTC"
dat_moorings$receiver_end_date <- as.POSIXct(dat_moorings$receiver_end_date)
lubridate::tz(dat_moorings$receiver_end_date) <- "UTC"
dat_ids$tag_start_date <- as.POSIXct(dat_ids$tag_start_date)
lubridate::tz(dat_ids$tag_start_date) <- "UTC"
## tag_end_date column needed in dat_ids
dat_ids$tag_end_date <- as.POSIXct("2020-01-01", tz = "UTC")
#### Implement process_quality_check() on processed data as a final check for any issues
process_quality_check(dat_acoustics, dat_moorings, dat_ids)
#> Check 1 (receiver identity): passed.
#> Check 2 (receiver deployment windows): passed.
#> Check 3 (tag identity): passed.
#> Check 4 (tag deployment windows): passed.
#> Check 5 (false detections) passed: acoustics$passed_filter does not contain false detections.
#### Add erroneous data to acoustics for demonstrating process_quality_check()
## Define a convenience function to add erroneous data to
# ... acoustics to demonstrate process_quality_check()
add_erroneous_row <- function(acoustics, row = nrow(acoustics), col, val) {
tmp_ls <- lapply(val, function(v) {
tmp <- acoustics[row, ]
tmp[1, col] <- v
return(tmp)
})
tmp <- dplyr::bind_rows(tmp_ls)
acoustics <- rbind(acoustics, tmp)
return(acoustics)
}
## Add erroneous receiver ids
nrw <- nrow(dat_acoustics)
acoustics_wth_errors <- add_erroneous_row(dat_acoustics,
row = nrw,
col = "receiver_id",
val = c(100, 200, 300)
)
## Add erroneous time stamps (outside receiver/individual id deployment periods )
acoustics_wth_errors <- add_erroneous_row(acoustics_wth_errors,
row = nrw,
col = "timestamp",
val = as.POSIXct(c("2019-01-01", "2019-03-01"),
tz = "UTC"
)
)
## Add erroneous individual ids
acoustics_wth_errors <- add_erroneous_row(acoustics_wth_errors,
row = nrw,
col = "individual_id",
val = c(100, 200, 300)
)
## Examine erroneous data:
utils::tail(acoustics_wth_errors, 10)
#> individual_id transmitter_id index timestamp receiver_id
#> 39241 35 A69-1303-540 19903 2017-04-12 22:12:00 57
#> 39242 35 A69-1303-540 19904 2017-04-19 23:22:00 48
#> 39243 35 A69-1303-540 19904 2017-04-19 23:22:00 100
#> 39244 35 A69-1303-540 19904 2017-04-19 23:22:00 200
#> 39245 35 A69-1303-540 19904 2017-04-19 23:22:00 300
#> 39246 35 A69-1303-540 19904 2019-01-01 00:00:00 48
#> 39247 35 A69-1303-540 19904 2019-03-01 00:00:00 48
#> 39248 100 A69-1303-540 19904 2017-04-19 23:22:00 48
#> 39249 200 A69-1303-540 19904 2017-04-19 23:22:00 48
#> 39250 300 A69-1303-540 19904 2017-04-19 23:22:00 48
#> receiver receiver_long receiver_lat receiver_depth passed_filter
#> 39241 VR2AR-546585 -5.623995 56.37436 75 1
#> 39242 VR2AR-546134 -5.643867 56.38278 145 1
#> 39243 VR2AR-546134 -5.643867 56.38278 145 1
#> 39244 VR2AR-546134 -5.643867 56.38278 145 1
#> 39245 VR2AR-546134 -5.643867 56.38278 145 1
#> 39246 VR2AR-546134 -5.643867 56.38278 145 1
#> 39247 VR2AR-546134 -5.643867 56.38278 145 1
#> 39248 VR2AR-546134 -5.643867 56.38278 145 1
#> 39249 VR2AR-546134 -5.643867 56.38278 145 1
#> 39250 VR2AR-546134 -5.643867 56.38278 145 1
#### Implement process_quality_check()
process_quality_check(acoustics_wth_errors, dat_moorings, dat_ids)
#> Warning: Check 1 (receiver identity): failed. 3 receiver identities unknown (100, 200, 300), corresponding to 3 observations in acoustics.
#> Warning: Check 3 (receiver deployment windows): failed. 2 observations in acoustics outside of receiver deployment windows.
#> Warning: Check 3 (tag identity): failed. 3 tag identities unknown (100, 200, 300), corresponding to 3 observations in acoustics.
#> Check 4 (tag deployment windows): passed.
#> Check 5 (false detections) passed: acoustics$passed_filter does not contain false detections.