Skill: evaluation metrics

Source: R/skill.R

These functions are standard model skill metrics. In patter, they support comparisons of simulated and reconstructed patterns of space use.

skill_mb(.obs, .mod, .summarise = "mean")

skill_me(.obs, .mod, .summarise = "mean")

skill_rmse(.obs, .mod)

skill_R(.obs, .mod)

skill_d(.obs, .mod)

Arguments

.obs

The 'observed' (true) pattern of space use, as a SpatRaster.

.mod

The 'modelled' (reconstructed) pattern of space use, as a SpatRaster.

.summarise

A function, passed to terra::global(), used to summarise SpatRaster values.

Value

The functions return a number.

Details

We follow the mathematical definitions in Lavender et al. (2022) Supplementary Information Sect. 3.2.1.

  • skill_mb() computes mean bias (if .summarise = "mean").

  • skill_me() computes mean error (if .summarise = "mean").

  • skill_rmse() computes root mean squared error.

  • skill_R() computes Spearman's rank correlation coefficient.

  • skill_d() computes the index of agreement.

These functions are not memory safe.

References

Lavender, E. et al. (2022). Benthic animal-borne sensors and citizen science combine to validate ocean modelling. Sci. Rep. 12: 16613. https://www.doi.org/1038/s41598-022-20254-z

See also

  • To simulate observations, see sim_*() functions (especially sim_path_walk(), sim_array() and sim_observations());

  • To translate observations into coordinates for mapping patterns of space use, see:

    • coa() to calculate centres of activity;

    • pf_filter() and associates to implement particle filtering algorithms;

  • To estimate utilisation distributions from simulated data and algorithm outputs, use map_*() functions (see map_pou(), map_dens() and map_hr());

Author

Edward Lavender

Examples

# Define template SpatRasters
mod <- obs <- terra::rast()
n <- terra::ncell(mod)
mod[] <- runif(n)
obs[] <- mod[] + rnorm(n)

# Normalise template SpatRasters
mod   <- mod / terra::global(mod, "sum")[1, 1]
obs   <- obs / terra::global(obs, "sum")[1, 1]

# Visualise 'observed' versus 'modelled' distributions
pp <- par(mfrow = c(1, 2))
terra::plot(mod)
terra::plot(obs)

par(pp)

# Calculate skill metrics
skill_mb(mod, obs)
#> [1] 1.773269e-20
skill_me(mod, obs)
#> [1] 2.487593e-05
skill_rmse(mod, obs)
#> [1] 3.117977e-05
skill_R(mod, obs)
#> [1] 0.2722464
skill_d(mod, obs)
#> [1] -0.3782236