Given a list of particle histories (or a list of file paths), this function accesses the unique particles (cells) sampled by a particle filtering (PF) algorithm (pf).
pf_access_particles_unique(
history,
use_memory_safe = FALSE,
cl = NULL,
varlist = NULL
)A pf_archive-class class object from pf, the list of particle histories (the `history' element of a pf_archive-class object) from pf or a list of file paths to particle histories.
If history is a record of file paths, use_memory_safe is a logical variable that defines whether or not to use the `memory-safe(r)' method to access unique cell samples. If specified, the function sequentially loads each file and re-defines the vector of unique particles at each time step as the unique combination of previous (unique) samples and the samples from the current time step. This may be slow. Alternatively, under the default use_memory_safe = FALSE option, the function loads each file (in parallel if specified), retaining all sampled particles, before selecting the unique particles (once) at the end of this process. This option should be faster.
(optional) Parallelisation options implemented if (a) particle histories are contained in memory or (b) particle histories are supplied as a list of file paths with use_memory_safe = FALSE. cl is (a) a cluster object from makeCluster or (b) an integer that defines the number of child processes. varlist is a character vector of variables for export (see cl_export). Exported variables must be located in the global environment. If a cluster is supplied, the connection to the cluster is closed within the function (see cl_stop). For further information, see cl_lapply and flapper-tips-parallel.
The function returns a vector of the unique particles sampled by the PF algorithm.
pf implements particle filtering.
#### Example (1): Access unique particles when 'history' exists in memory
# Access unique particles from a pf_archive object
pf_access_particles_unique(dat_dcpf_histories)
#> [1] 3243 2921 3396 3159 3320 3237 3318 3559 3722 3961 3723 3079 3475 3478 3078
#> [16] 3157 3477 3315 3317 3166 3956 3963 3406 3240 3402 3006 3801 3568 3641 3565
#> [31] 3398 3238 3887 3399 3235 3005 3889 2843 3009 3400 3075 3405 3719 3329 3397
#> [46] 3886 3637 3008 3086 3000 3635 3089 2917 3493 3247 3560 3083 3898 3977 3899
#> [61] 3978 4221 4380 4301 4063 4142 4300 4304 4858 4462 4303 4783 4545 4622 4463
#> [76] 4862 4861 4387 4547 4548 4389 4864 4228 4705 5104 4943 5106 5265 4709 4469
#> [91] 4707 5429 4552 4871 4867 5026 5029 4952 5185 5266 5345 4716 5830 5196 5273
#> [106] 5036 5034 5590 4956 4636 5513 4717 4798 5200 6069 4958 4880 5672 5752 5280
#> [121] 4560 6389 4719 5360 4959 5119 5835 5279 6314 6237 5440 5519 6240 6396 6319
#> [136] 6159 6318 5520 6238 5599 5439 6320
# Access unique particles from a list of particle histories
pf_access_particles_unique(dat_dcpf_histories$history)
#> [1] 3243 2921 3396 3159 3320 3237 3318 3559 3722 3961 3723 3079 3475 3478 3078
#> [16] 3157 3477 3315 3317 3166 3956 3963 3406 3240 3402 3006 3801 3568 3641 3565
#> [31] 3398 3238 3887 3399 3235 3005 3889 2843 3009 3400 3075 3405 3719 3329 3397
#> [46] 3886 3637 3008 3086 3000 3635 3089 2917 3493 3247 3560 3083 3898 3977 3899
#> [61] 3978 4221 4380 4301 4063 4142 4300 4304 4858 4462 4303 4783 4545 4622 4463
#> [76] 4862 4861 4387 4547 4548 4389 4864 4228 4705 5104 4943 5106 5265 4709 4469
#> [91] 4707 5429 4552 4871 4867 5026 5029 4952 5185 5266 5345 4716 5830 5196 5273
#> [106] 5036 5034 5590 4956 4636 5513 4717 4798 5200 6069 4958 4880 5672 5752 5280
#> [121] 4560 6389 4719 5360 4959 5119 5835 5279 6314 6237 5440 5519 6240 6396 6319
#> [136] 6159 6318 5520 6238 5599 5439 6320
# Supply a cluster to speed up the algorithm (for very large lists)
pf_access_particles_unique(dat_dcpf_histories$history,
cl = parallel::makeCluster(2L)
)
#> [1] 3243 2921 3396 3159 3320 3237 3318 3559 3722 3961 3723 3079 3475 3478 3078
#> [16] 3157 3477 3315 3317 3166 3956 3963 3406 3240 3402 3006 3801 3568 3641 3565
#> [31] 3398 3238 3887 3399 3235 3005 3889 2843 3009 3400 3075 3405 3719 3329 3397
#> [46] 3886 3637 3008 3086 3000 3635 3089 2917 3493 3247 3560 3083 3898 3977 3899
#> [61] 3978 4221 4380 4301 4063 4142 4300 4304 4858 4462 4303 4783 4545 4622 4463
#> [76] 4862 4861 4387 4547 4548 4389 4864 4228 4705 5104 4943 5106 5265 4709 4469
#> [91] 4707 5429 4552 4871 4867 5026 5029 4952 5185 5266 5345 4716 5830 5196 5273
#> [106] 5036 5034 5590 4956 4636 5513 4717 4798 5200 6069 4958 4880 5672 5752 5280
#> [121] 4560 6389 4719 5360 4959 5119 5835 5279 6314 6237 5440 5519 6240 6396 6319
#> [136] 6159 6318 5520 6238 5599 5439 6320
#### Example (2): Access unique particles when 'history' is a list of file paths
## Write example particle histories to file (to load)
root <- paste0(tempdir(), "/pf/")
dir.create(root)
#> Warning: '/var/folders/nl/ygb3g6tj24z06jddbqqhj6hw0000gn/T//Rtmpwb5KFV/pf' already exists
pf_args <- dat_dcpf_histories$args
pf_args$calc_distance_euclid_fast <- TRUE
pf_args$write_history <- list(file = root)
out_pf <- do.call(pf, pf_args)
#> flapper::pf() called (@ 2023-08-29 15:45:01)...
#> ... Setting up function...
#> ... Determining the set of possible starting locations (t = 1)...
#> Warning: GEOS support is provided by the sf and terra packages among others
#> ... Implementing algorithm iteratively over time steps...
#> ... ... Time = 1...
#> ... ... ... Selecting candidate starting positions for the current time step...
#> ... ... ... For each particle, getting the possible positions for the next time step...
#> Warning: GEOS support is provided by the sf and terra packages among others
#> ... ... Time = 2...
#> ... ... ... Selecting candidate starting positions for the current time step...
#> ... ... ... For each particle, getting the possible positions for the next time step...
#> Warning: GEOS support is provided by the sf and terra packages among others
#> ... ... Time = 3...
#> ... ... ... Selecting candidate starting positions for the current time step...
#> ... ... ... For each particle, getting the possible positions for the next time step...
#> Warning: GEOS support is provided by the sf and terra packages among others
#> ... ... Time = 4...
#> ... ... ... Selecting candidate starting positions for the current time step...
#> ... ... ... For each particle, getting the possible positions for the next time step...
#> Warning: GEOS support is provided by the sf and terra packages among others
#> ... ... Time = 5...
#> ... ... ... Selecting candidate starting positions for the current time step...
#> ... ... ... For each particle, getting the possible positions for the next time step...
#> Warning: GEOS support is provided by the sf and terra packages among others
#> ... ... Time = 6...
#> ... ... ... Selecting candidate starting positions for the current time step...
#> ... ... ... For each particle, getting the possible positions for the next time step...
#> Warning: GEOS support is provided by the sf and terra packages among others
#> ... ... Time = 7...
#> ... ... ... Selecting candidate starting positions for the current time step...
#> ... ... ... For each particle, getting the possible positions for the next time step...
#> Warning: GEOS support is provided by the sf and terra packages among others
#> ... ... Time = 8...
#> ... ... ... Selecting candidate starting positions for the current time step...
#> ... ... ... For each particle, getting the possible positions for the next time step...
#> Warning: GEOS support is provided by the sf and terra packages among others
#> ... ... Time = 9...
#> ... ... ... Selecting candidate starting positions for the current time step...
#> ... ... ... For each particle, getting the possible positions for the next time step...
#> Warning: GEOS support is provided by the sf and terra packages among others
#> ... ... Time = 10...
#> ... ... ... Selecting candidate starting positions for the current time step...
#> ... ... ... For each particle, getting the possible positions for the next time step...
#> Warning: GEOS support is provided by the sf and terra packages among others
#> ... ... Time = 11...
#> ... ... ... Selecting candidate starting positions for the current time step...
#> ... ... ... For each particle, getting the possible positions for the next time step...
#> Warning: GEOS support is provided by the sf and terra packages among others
#> ... ... Time = 12...
#> ... ... ... Selecting candidate starting positions for the current time step...
#> ... ... ... For each particle, getting the possible positions for the next time step...
#> Warning: GEOS support is provided by the sf and terra packages among others
#> ... ... Time = 13...
#> ... ... ... Selecting candidate starting positions for the current time step...
#> ... ... ... For each particle, getting the possible positions for the next time step...
#> Warning: GEOS support is provided by the sf and terra packages among others
#> ... ... Time = 14...
#> ... ... ... Selecting candidate starting positions for the current time step...
#> ... ... ... For each particle, getting the possible positions for the next time step...
#> Warning: GEOS support is provided by the sf and terra packages among others
#> ... ... Time = 15...
#> ... ... ... Selecting candidate starting positions for the current time step...
#> ... ... ... For each particle, getting the possible positions for the next time step...
#> Warning: GEOS support is provided by the sf and terra packages among others
#> ... ... Time = 16...
#> ... ... ... Selecting candidate starting positions for the current time step...
#> ... ... ... For each particle, getting the possible positions for the next time step...
#> Warning: GEOS support is provided by the sf and terra packages among others
#> ... ... Time = 17...
#> ... ... ... Selecting candidate starting positions for the current time step...
#> ... ... ... For each particle, getting the possible positions for the next time step...
#> Warning: GEOS support is provided by the sf and terra packages among others
#> ... ... Time = 18...
#> ... ... ... Selecting candidate starting positions for the current time step...
#> ... flapper::pf() call completed (@ 2023-08-29 15:45:01) after ~0.01 minutes.
## Access particle histories using default options (use_memory_safe = FALSE)
# Access particle histories via pf_access_history_files()
pf_access_particles_unique(pf_access_history_files(root, use_list = TRUE))
#> [1] 3243 2921 3396 3159 3320 3237 3318 3559 3722 3312 3634 3167 3397 3244 3003
#> [16] 2999 3398 2997 3647 3557 3250 3089 3713 2835 2844 3470 3951 2923 3395 3570
#> [31] 2603 3555 2598 2515 3796 4189 3472 3004 2355 3626 2360 2522 3325 3477 4508
#> [46] 2839 4432 4348 2433 2756 4185 2363 3640 2353 3164 3877 2444 3078 1963 3485
#> [61] 3316 3002 3561 3887 3959 4372 4293 4618 4776 4379 4697 4777 4458 4459 4538
#> [76] 4939 5099 5098 4621 4699 4304 5100 4383 4701 4700 4308 5266 5258 4229 5659
#> [91] 4781 5102 4468 4466 5345 5820 4628 5022 4154 5667 5430 4310 5508 5030 5586
#> [106] 5826 5027 4155 5905 5910 5108 5666 5821 3995 4869 5908 5111 5916 6071 5747
#> [121] 6226 5825 6065 5909 5915 5355 6306 6229 6159 6228 5835 5829 6151 6390 5994
#> [136] 6398 6312 5197 5917 5998 6079 6158 5840 6239 5920 5839 6078 5919 6397 5520
#> [151] 5680 5759 6399 5600 5519 5440 6240 6000
# Supply a cluster to speed up the algorithm (for very large lists)
pf_access_particles_unique(pf_access_history_files(root, use_list = TRUE),
cl = parallel::makeCluster(2L)
)
#> [1] 3243 2921 3396 3159 3320 3237 3318 3559 3722 3312 3634 3167 3397 3244 3003
#> [16] 2999 3398 2997 3647 3557 3250 3089 3713 2835 2844 3470 3951 2923 3395 3570
#> [31] 2603 3555 2598 2515 3796 4189 3472 3004 2355 3626 2360 2522 3325 3477 4508
#> [46] 2839 4432 4348 2433 2756 4185 2363 3640 2353 3164 3877 2444 3078 1963 3485
#> [61] 3316 3002 3561 3887 3959 4372 4293 4618 4776 4379 4697 4777 4458 4459 4538
#> [76] 4939 5099 5098 4621 4699 4304 5100 4383 4701 4700 4308 5266 5258 4229 5659
#> [91] 4781 5102 4468 4466 5345 5820 4628 5022 4154 5667 5430 4310 5508 5030 5586
#> [106] 5826 5027 4155 5905 5910 5108 5666 5821 3995 4869 5908 5111 5916 6071 5747
#> [121] 6226 5825 6065 5909 5915 5355 6306 6229 6159 6228 5835 5829 6151 6390 5994
#> [136] 6398 6312 5197 5917 5998 6079 6158 5840 6239 5920 5839 6078 5919 6397 5520
#> [151] 5680 5759 6399 5600 5519 5440 6240 6000
## Access particle histories using the 'memory_safe' option
# For large lists, this is likely to be slower
# ... but it may be the only option in some cases.
pf_access_particles_unique(pf_access_history_files(root, use_list = TRUE),
use_memory_safe = TRUE
)
#> [1] 3243 2921 3396 3159 3320 3237 3318 3559 3722 3312 3634 3167 3397 3244 3003
#> [16] 2999 3398 2997 3647 3557 3250 3089 3713 2835 2844 3470 3951 2923 3395 3570
#> [31] 2603 3555 2598 2515 3796 4189 3472 3004 2355 3626 2360 2522 3325 3477 4508
#> [46] 2839 4432 4348 2433 2756 4185 2363 3640 2353 3164 3877 2444 3078 1963 3485
#> [61] 3316 3002 3561 3887 3959 4372 4293 4618 4776 4379 4697 4777 4458 4459 4538
#> [76] 4939 5099 5098 4621 4699 4304 5100 4383 4701 4700 4308 5266 5258 4229 5659
#> [91] 4781 5102 4468 4466 5345 5820 4628 5022 4154 5667 5430 4310 5508 5030 5586
#> [106] 5826 5027 4155 5905 5910 5108 5666 5821 3995 4869 5908 5111 5916 6071 5747
#> [121] 6226 5825 6065 5909 5915 5355 6306 6229 6159 6228 5835 5829 6151 6390 5994
#> [136] 6398 6312 5197 5917 5998 6079 6158 5840 6239 5920 5839 6078 5919 6397 5520
#> [151] 5680 5759 6399 5600 5519 5440 6240 6000