A "Snake_case" Filter System for R.
filters
A "snake_case" filter system to R.
Installation
if (!requireNamespace("remotes")) {
install.packages("remotes")
}
remotes::install_github(
repo = "openpharma/filters",
upgrade = "never"
)
Features
library(filters)
library(magrittr)
library(random.cdisc.data)
library(rtables)
library(tern)
set.seed(1)
adsl <- radsl()
adae <- radae(adsl)
vads <- list(adsl = adsl, adae = adae)
Built-In Filters
{filters} comes with a built-in filter library. You can list them using list_all_filters().
list_all_filters()
# A tibble: 272 x 4
id title target condition
<chr> <chr> <chr> <chr>
1 COV Confirmed/Suspected COVID… ADAE ACOVFL == 'Y'
2 COVAS AEs Associated with COVID… ADAE ACOVASFL == 'Y'
3 CTC35 Grade 3-5 Adverse Events ADAE ATOXGR %in% c('3', '4', '5')
4 DSC Adverse Events Leading to… ADAE AEACN == 'DRUG WITHDRAWN'
5 DSM Adverse Events Leading to… ADAE AEACN %in% c('DOSE INCREASED',…
6 FATAL Fatal Adverse Events ADAE AESDTH == 'Y'
7 NCOV Excluding Confirmed/Suspe… ADAE ACOVFL != 'Y'
8 NCOVAS AEs not Associated with C… ADAE ACOVASFL != 'Y'
9 NFATAL Non-fatal Adverse Events ADAE AESDTH == 'N'
10 NREL Adverse Events not Relate… ADAE AREL == 'N'
# … with 262 more rows
Adding New Filters
To add a new filter use add_filter(). The last argument, condition, defines the condition to use to filter the datasets later on. It will be passed to subset() when calling apply_filter().
add_filter(
id = "CTC34",
title = "Grade 3-4 Adverse Events",
target = "ADAE",
condition = AETOXGR %in% c("4", "5")
)
Alternatively, you can use load_filters() to load filter definitions from a yaml file. The file should be structured like this:
CTC4:
title: Grade 4 Adverse Events
target: ADAE
condition: ATOXGR == "4"
TP53WT:
title: TP53 Wild Type
target: ADSL
condition: TP53 == "WILD TYPE"
file_path <- system.file("filters_eg.yaml", package = "filters")
load_filters(file_path)
You can confirm that filters haven been successfully added by using get_filter().
get_filter("CTC34")
$title
[1] "Grade 3-4 Adverse Events"
$target
[1] "ADAE"
$condition
AETOXGR %in% c("4", "5")
If you ask for a non-existing filter get_filter() will throw an error.
get_filter("GIDIS")
Error: Filter 'GIDIS' does not exist.
To overwrite an existing filter you will have to set overwrite = TRUE. Otherwise an error is thrown.
add_filter(
id = "FATAL",
title = "Fatal Adverse Events",
target = "ADAE",
condition = ATOXGR == "5"
)
Error: Filter 'FATAL' already exists. Set `overwrite = TRUE` to force overwriting the existing filter definition.
add_filter(
id = "FATAL",
title = "Fatal Adverse Events",
target = "ADAE",
condition = ATOXGR == "5",
overwrite = TRUE
)
Applying Filters to Datasets
You can use apply_filter() to filter a single dataset or a list of multiple datasets.
adsl_se <- apply_filter(adsl, "SE")
Filter 'SE' matched target ADSL.
400/400 records matched the filter condition `SAFFL == 'Y'`.
adae_ctc34_ser <- apply_filter(adae, "CTC34_SER")
Filters 'CTC34', 'SER' matched target ADAE.
216/1967 records matched the filter condition `AETOXGR %in% c('4', '5') & AESER == 'Y'`.
filtered_datasets <- apply_filter(vads, "CTC34_SER_SE")
Filter 'SE' matched target ADSL.
400/400 records matched the filter condition `SAFFL == 'Y'`.
Filters 'CTC34', 'SER' matched target ADAE.
216/1967 records matched the filter condition `AETOXGR %in% c('4', '5') & AESER == 'Y'`.
As you can see apply_filter() gives you feedback on which IDs matched the dataset. This matching is done by the name of the input dataset. It does not matter whether the dataset name is in upper or lower case or a mix of both.
ADSL <- adsl
adsl_it <- apply_filter(ADSL, "IT")
Filter 'IT' matched target ADSL.
400/400 records matched the filter condition `ITTFL == 'Y'`.
In case your dataset is not named in a standard way you can manually tell apply_filter() which dataset it is by setting the target argument.
sl <- adsl
sl_it1 <- apply_filter(sl, "IT")
No filter matched target SL.
sl_it2 <- apply_filter(sl, "IT", target = "ADSL")
Filter 'IT' matched target ADSL.
400/400 records matched the filter condition `ITTFL == 'Y'`.
Using {filters} for Generating Outputs
{filters} package works well with {rtables} and {tern} packages. See the following example of creating a table by a function:
t_ae <- function(datasets) {
anl <- merge(
x = datasets$adsl,
y = datasets$adae,
by = c("STUDYID", "USUBJID"),
all = FALSE, # inner join
suffixes = c("", "_ADAE")
)
split_fun <- drop_split_levels
lyt <- basic_table(show_colcounts = TRUE) %>%
split_cols_by(var = "ARM") %>%
add_overall_col(label = "All Patients") %>%
analyze_num_patients(
vars = "USUBJID",
.stats = c("unique", "nonunique"),
.labels = c(
unique = "Total number of patients with at least one adverse event",
nonunique = "Overall total number of events"
)
) %>%
split_rows_by(
"AEBODSYS",
child_labels = "visible",
nested = FALSE,
split_fun = split_fun,
label_pos = "topleft",
split_label = obj_label(adae$AEBODSYS)
) %>%
summarize_num_patients(
var = "USUBJID",
.stats = c("unique", "nonunique"),
.labels = c(
unique = "Total number of patients with at least one adverse event",
nonunique = "Total number of events"
)
) %>%
count_occurrences(
vars = "AEDECOD",
.indent_mods = -1L
) %>%
append_varlabels(adae, "AEDECOD", indent = 1L)
result <- build_table(
lyt,
df = datasets$adae,
alt_counts_df = datasets$adsl
)
return(result)
}
You can easily create multiple outputs with this function by applying the filters to the input datasets before passing them to t_ae().
vads %>% apply_filter("SE") %>% t_ae()
Filter 'SE' matched target ADSL.
400/400 records matched the filter condition `SAFFL == 'Y'`.
Body System or Organ Class A: Drug X B: Placebo C: Combination All Patients
Dictionary-Derived Term (N=133) (N=141) (N=126) (N=400)
——————————————————————————————————————————————————————————————————————————————————————————————————————————————————————
Total number of patients with at least one adverse event 111 (83.5%) 132 (93.6%) 119 (94.4%) 362 (90.5%)
Overall total number of events 636 755 655 2046
cl A.1
Total number of patients with at least one adverse event 63 (47.4%) 79 (56.0%) 71 (56.3%) 213 (53.2%)
Total number of events 123 144 133 400
dcd A.1.1.1.1 47 (35.3%) 63 (44.7%) 50 (39.7%) 160 (40.0%)
dcd A.1.1.1.2 42 (31.6%) 47 (33.3%) 44 (34.9%) 133 (33.2%)
cl B.1
Total number of patients with at least one adverse event 47 (35.3%) 49 (34.8%) 59 (46.8%) 155 (38.8%)
Total number of events 73 63 75 211
dcd B.1.1.1.1 47 (35.3%) 49 (34.8%) 59 (46.8%) 155 (38.8%)
cl B.2
Total number of patients with at least one adverse event 73 (54.9%) 88 (62.4%) 73 (57.9%) 234 (58.5%)
Total number of events 132 156 137 425
dcd B.2.1.2.1 44 (33.1%) 56 (39.7%) 50 (39.7%) 150 (37.5%)
dcd B.2.2.3.1 48 (36.1%) 59 (41.8%) 44 (34.9%) 151 (37.8%)
cl C.1
Total number of patients with at least one adverse event 50 (37.6%) 53 (37.6%) 42 (33.3%) 145 (36.2%)
Total number of events 62 75 62 199
dcd C.1.1.1.3 50 (37.6%) 53 (37.6%) 42 (33.3%) 145 (36.2%)
cl C.2
Total number of patients with at least one adverse event 50 (37.6%) 65 (46.1%) 50 (39.7%) 165 (41.2%)
Total number of events 67 87 63 217
dcd C.2.1.2.1 50 (37.6%) 65 (46.1%) 50 (39.7%) 165 (41.2%)
cl D.1
Total number of patients with at least one adverse event 74 (55.6%) 95 (67.4%) 72 (57.1%) 241 (60.2%)
Total number of events 120 158 112 390
dcd D.1.1.1.1 37 (27.8%) 59 (41.8%) 35 (27.8%) 131 (32.8%)
dcd D.1.1.4.2 54 (40.6%) 63 (44.7%) 48 (38.1%) 165 (41.2%)
cl D.2
Total number of patients with at least one adverse event 43 (32.3%) 54 (38.3%) 56 (44.4%) 153 (38.2%)
Total number of events 59 72 73 204
dcd D.2.1.5.3 43 (32.3%) 54 (38.3%) 56 (44.4%) 153 (38.2%)
vads %>% apply_filter("SER_SE") %>% t_ae()
Filter 'SE' matched target ADSL.
400/400 records matched the filter condition `SAFFL == 'Y'`.
Filter 'SER' matched target ADAE.
581/1967 records matched the filter condition `AESER == 'Y'`.
Body System or Organ Class A: Drug X B: Placebo C: Combination All Patients
Dictionary-Derived Term (N=133) (N=141) (N=126) (N=400)
—————————————————————————————————————————————————————————————————————————————————————————————————————————————————————
Total number of patients with at least one adverse event 93 (69.9%) 110 (78.0%) 98 (77.8%) 301 (75.2%)
Overall total number of events 248 280 246 774
cl A.1
Total number of patients with at least one adverse event 42 (31.6%) 47 (33.3%) 44 (34.9%) 133 (33.2%)
Total number of events 54 63 58 175
dcd A.1.1.1.2 42 (31.6%) 47 (33.3%) 44 (34.9%) 133 (33.2%)
cl B.1
Total number of patients with at least one adverse event 47 (35.3%) 49 (34.8%) 59 (46.8%) 155 (38.8%)
Total number of events 73 63 75 211
dcd B.1.1.1.1 47 (35.3%) 49 (34.8%) 59 (46.8%) 155 (38.8%)
cl B.2
Total number of patients with at least one adverse event 48 (36.1%) 59 (41.8%) 44 (34.9%) 151 (37.8%)
Total number of events 74 78 65 217
dcd B.2.2.3.1 48 (36.1%) 59 (41.8%) 44 (34.9%) 151 (37.8%)
cl D.1
Total number of patients with at least one adverse event 37 (27.8%) 59 (41.8%) 35 (27.8%) 131 (32.8%)
Total number of events 47 76 48 171
dcd D.1.1.1.1 37 (27.8%) 59 (41.8%) 35 (27.8%) 131 (32.8%)
(Current) Limitations
The filters you created using add_filter() only persist for the duration of your R session. That means that whenever you restart your R session you will have to re-create them. The simplest way to do so is by putting all your filter definitions inside a file filters.yml file as described above and call load_filters("path/to/filters.yml") before creating outputs.
If you pass an existing filter that does not match your target dataset no warning or error is thrown. Instead apply_filter() only tells you which filters it actually used. Thus, checking that only valid filters are passed to apply_filter() is up to you.
add_filter(
id = "INFCT",
title = "Infections and Infestations",
target = "ADAE",
condition = AEBODSYS == "INFECTIONS AND INFESTATIONS"
)
adsl_filtered <- apply_filter(adsl, "DIABP_IT")
Filter 'IT' matched target ADSL.
400/400 records matched the filter condition `ITTFL == 'Y'`.
How Does it Work?
Internally, {filters} stores the filter definitions inside the .filters environment defined in R/zzz.R. When you add a filter with add_filter() a new variable with the name of the ID is created inside this environment. This variable is a list that stores the title, target and condition as a quoted expression. When you use apply_filter() the function looks for variables in .filters matching the provided suffixes. It then maps the filters to their target datasets and finally builds a call to subset() with the dataset as first and condition for the filters as second argument. This call is then evaluated using eval() and the result is returned.