MyNixOS website logo
Description

Validate SDTM Domains.

Provides a set of tools to assist statistical programmers in validating Study Data Tabulation Model (SDTM) domain data sets. Statistical programmers are required to validate that a SDTM data set domain has been programmed correctly, per the SDTM Implementation Guide (SDTMIG) by 'CDISC' (<https://www.cdisc.org/standards/foundational/sdtmig>), study specification, and study protocol using a process called double programming. Double programming involves two different programmers independently converting the raw electronic data cut (EDC) data into a SDTM domain data table and comparing their results to ensure accurate standardization of the data. One of these attempts is termed 'production' and the other 'validation'. Generally, production runs are the official programs for submittals and these are written in 'SAS'. Validation runs can be programmed in another language, in this case 'R'.

sdtmval sdtmval website

R-CMD-check Codecov testcoverage

{sdtmval} provides a set of tools to assist statistical programmers in validating Study Data Tabulation Model (SDTM) domain data sets.

Many data cleaning steps and SDTM processes are used repeatedly in different SDTM domain validation scripts. Functionalizing these repetitive tasks allows statistical programmers to focus on coding the unique aspects of a SDTM domain while standardize their code base across studies and domains. This should lead to fewer bugs and improved code readability too. {sdtmval} features include:

  • Automating the BLFL, DY, EPOCH, SEQ, and STAT methods to create new variables

  • Imputing and formatting full and partial dates: see vignette("Dates")

  • Applying specification data such as variable labels, lengths, code mapping, and sorting

  • Importing EDC and SDTM data from .csv and .sas7bdat files

  • Writing .xpt files (convenience wrapper for haven::write_xpt())

  • Converting .Rmd files to .R scripts (convenience wrapper for knitr::purl())

  • Logging R session information for reproducibility

  • Data formatting

Installation

You can install the release version of {sdtmval} from CRAN with:

install.packages("sdtmval")

You can install the development version of {sdtmval} from GitHub with:

# install.packages("devtools")
devtools::install_github("skgithub14/sdtmval")


A typical work flow example

In this example work flow, we will import a raw EDC table and transform it into a SDTM domain table. We will use the made-up domain ‘XX’ along with some example data included in {sdtmval}.

# set-up
library(sdtmval)
library(dplyr)

domain <- "XX"

# set working directory to location of sdtmval package example data
work_dir <- system.file("extdata", package = "sdtmval")

The majority of the data needed is in the EDC form/table xx.csv. There are also visit dates in the EDC table vd.csv and study start/end dates in the SDTM table dm.sas7dbat. These can be imported using read_edc_tbls() and read_sdtm_tbls().

# read in EDC tables from the forms XX and VD
edc_tbls <- c("xx", "vd")
edc_dat <- read_edc_tbls(edc_tbls, dir = work_dir)

# read in SDTM domain DM
sdtm_tbls <- c("dm")
sdtm_dat <- read_sdtm_tbls(sdtm_tbls, dir = work_dir)

The raw data looks like this:

STUDYIDUSUBJIDVISITXXTESTCDXXORRES
Study 1Subject 1Visit 1T11
Study 1Subject 1Visit 2T10
Study 1Subject 1Visit 3T12
Study 1Subject 1Visit 3T2100
Study 1Subject 1Visit 4T3PASS
Study 1Subject 2Visit 1T11
Study 1Subject 2Visit 2T1
Study 1Subject 2Visit 3T12
Study 1Subject 2Visit 3T2200
Study 1Subject 2Visit 4T3FAIL

The next thing we will do is get the relevant information from the SDTM specification for the study. The next set of functions assumes there is a .xlsx file which contains the sheets: ‘Datasets’, ‘XX’, and ‘Codelists’:

  • ‘XX’ gives the variable information for the made-up XX domain. get_data_spec() retrieves this entire tab.

  • ‘Datasets’ contains the key variables by domain. get_key_vars() retrieves the these for desired domain.

  • ‘Codelists’ provides a table of coded/decoded values by variable for all domains. get_codelist() extracts a data frame of coded/decoded values from this sheet just for the variables in desired domain.

spec_fname <- "spec.xlsx"
spec <- get_data_spec(domain = domain, dir = work_dir, filename = spec_fname)
key_vars <- get_key_vars(domain = domain, dir = work_dir, filename = spec_fname)
codelists <- get_codelist(domain = domain, dir = work_dir, filename = spec_fname)

knitr::kable(spec)
OrderDatasetVariableLabelData TypeLength
1XXSTUDYIDStudy Identifiertext200
2XXDOMAINDomain Abbreviationtext200
3XXUSUBJIDUnique Subject Identifiertext200
4XXXXSEQSequence Numberinteger8
5XXXXTESTCDXX Test Short Nametext8
6XXXXTESTXX Test Nametext40
7XXXXORRESResult or Finding in Original Unitstext200
8XXXXBLFLBaseline Flagtext1
9XXVISITVisit Nametext200
10XXEPOCHEpochtext200
11XXXXDTCDate/Time of Measurementsdatetime19
12XXXXDYStudy Day of XXinteger8
knitr::kable(codelists)
IDTermDecoded Value
XXTESTCDT1Test 1
XXTESTCDT2Test 2
XXTESTCDT3Test 3
key_vars
#> [1] "STUDYID"  "USUBJID"  "XXTESTCD" "VISIT"

Now we will begin creating the SDTM XX domain using the EDC XX form as the basis.

First, it needs some pre-processing because there is extra white space in some of the variables. We also want to turn all NA equivalent values like "" and " " to NA for the entire data set so we have consistent handling of missing values during data processing. The function trim_and_make_blanks_NA() does both of these tasks.

sdtm_xx1 <- trim_and_make_blanks_NA(edc_dat$xx)

Next, using the codelist we retrieved earlier, we can create the XXTEST variable.

# prepare the code list so it can be used by dplyr::recode() 
xxtestcd_codelist <- codelists %>%
  filter(ID == "XXTESTCD") %>%
  select(Term, `Decoded Value`) %>%
  tibble::deframe()

# create XXTEST variable
sdtm_xx2 <- mutate(sdtm_xx1, XXTEST = recode(XXTESTCD, !!!xxtestcd_codelist))

knitr::kable(sdtm_xx2)
STUDYIDUSUBJIDVISITXXTESTCDXXORRESXXTEST
Study 1Subject 1Visit 1T11Test 1
Study 1Subject 1Visit 2T10Test 1
Study 1Subject 1Visit 3T12Test 1
Study 1Subject 1Visit 3T2100Test 2
Study 1Subject 1Visit 4T3PASSTest 3
Study 1Subject 2Visit 1T11Test 1
Study 1Subject 2Visit 2T1NATest 1
Study 1Subject 2Visit 3T12Test 1
Study 1Subject 2Visit 3T2200Test 2
Study 1Subject 2Visit 4T3FAILTest 3

In order to calculate the variables XXBLFL, EPOCH, and XXDY, we need the visit dates from the EDC VD table and the study start/end dates by subject from the SDTM DM table.

sdtm_xx3 <- sdtm_xx2 %>%
  
  # get the VISITDTC column from the EDC VD form
  left_join(edc_dat$vd, by = c("USUBJID", "VISIT")) %>%
  
  # create the XXDTC variable
  rename(XXDTC = VISITDTC) %>%
  
  # get the study start/end dates by subject (RFSTDTC, RFXSTDTC, RFXENDTC)
  left_join(sdtm_dat$dm, by = "USUBJID")

Now, we can proceed with calculating those timing variables using the create_BLFL(), create_EPOCH(), and calc_DY() functions.

sdtm_xx4 <- sdtm_xx3 %>%
  
  # XXBLFL
  create_BLFL(sort_date = "XXDTC",
              domain = domain,
              grouping_vars = c("USUBJID", "XXTESTCD")) %>%
  
  # EPOCH
  create_EPOCH(date_col = "XXDTC") %>%
  
  # XXDY
  calc_DY(DY_col = "XXDY", DTC_col = "XXDTC")
  
# check the new variables and their related columns only
sdtm_xx4 %>%
  select(USUBJID, XXTEST, XXORRES, XXDTC, XXBLFL, 
         EPOCH, XXDY, starts_with("RF")) %>%
  knitr::kable()
USUBJIDXXTESTXXORRESXXDTCXXBLFLEPOCHXXDYRFSTDTCRFXSTDTCRFXENDTC
Subject 1Test 112023-08-01NASCREENING-12023-08-022023-08-022023-08-03
Subject 1Test 102023-08-02YTREATMENT12023-08-022023-08-022023-08-03
Subject 1Test 122023-08-03NATREATMENT22023-08-022023-08-022023-08-03
Subject 1Test 21002023-08-03NATREATMENT22023-08-022023-08-022023-08-03
Subject 1Test 3PASS2023-08-04NAFOLLOW-UP32023-08-022023-08-022023-08-03
Subject 2Test 112023-08-02YSCREENING-12023-08-032023-08-032023-08-04
Subject 2Test 1NA2023-08-03NATREATMENT12023-08-032023-08-032023-08-04
Subject 2Test 122023-08-04NATREATMENT22023-08-032023-08-032023-08-04
Subject 2Test 22002023-08-04NATREATMENT22023-08-032023-08-032023-08-04
Subject 2Test 3FAIL2023-08-05NAFOLLOW-UP32023-08-032023-08-032023-08-04

Next, we will assign the sequence number using assign_SEQ() (which also sorts your data frame).

sdtm_xx5 <- assign_SEQ(sdtm_xx4, 
                       key_vars = c("USUBJID", "XXTESTCD", "VISIT"),
                       seq_prefix = domain)

# check the new variable
sdtm_xx5 %>%
  select(USUBJID, XXTESTCD, VISIT, XXDTC, XXSEQ) %>%
  knitr::kable()
USUBJIDXXTESTCDVISITXXDTCXXSEQ
Subject 1T1Visit 12023-08-011
Subject 1T1Visit 22023-08-022
Subject 1T1Visit 32023-08-033
Subject 1T2Visit 32023-08-034
Subject 1T3Visit 42023-08-045
Subject 2T1Visit 12023-08-021
Subject 2T1Visit 22023-08-032
Subject 2T1Visit 32023-08-043
Subject 2T2Visit 32023-08-044
Subject 2T3Visit 42023-08-055

Now that the bulk of the data cleaning is complete, we will convert all date columns to character columns and all NA values to "" so that our validation table matches the production table produced in SAS. To do this, we will use format_chars_and_dates().

sdtm_xx6 <- format_chars_and_dates(sdtm_xx5)

As a final step, we will assign the meta data from the spec to each column using assign_meta_data(). The meta data includes the labels for each column and their maximum allowed character lengths.

sdtm_xx7 <- sdtm_xx6 %>%
  
  # only keep columns that are domain variables and order them per the spec
  select(any_of(spec$Variable)) %>%
  
  # assign variable lengths and labels
  assign_meta_data(spec = spec)

# show the final SDTM domain
knitr::kable(sdtm_xx7)
STUDYIDUSUBJIDXXSEQXXTESTCDXXTESTXXORRESXXBLFLVISITEPOCHXXDTCXXDY
Study 1Subject 11T1Test 11Visit 1SCREENING2023-08-01-1
Study 1Subject 12T1Test 10YVisit 2TREATMENT2023-08-021
Study 1Subject 13T1Test 12Visit 3TREATMENT2023-08-032
Study 1Subject 14T2Test 2100Visit 3TREATMENT2023-08-032
Study 1Subject 15T3Test 3PASSVisit 4FOLLOW-UP2023-08-043
Study 1Subject 21T1Test 11YVisit 1SCREENING2023-08-02-1
Study 1Subject 22T1Test 1Visit 2TREATMENT2023-08-031
Study 1Subject 23T1Test 12Visit 3TREATMENT2023-08-042
Study 1Subject 24T2Test 2200Visit 3TREATMENT2023-08-042
Study 1Subject 25T3Test 3FAILVisit 4FOLLOW-UP2023-08-053
# check the meta data was assigned
labels <- colnames(sdtm_xx7) %>%
  purrr::map(~ attr(sdtm_xx7[[.]], "label")) %>%
  unlist()
lengths <- colnames(sdtm_xx7) %>%
  purrr::map(~ attr(sdtm_xx7[[.]], "width")) %>%
  unlist()
data.frame(
  column = colnames(sdtm_xx7),
  labels = labels,
  lengths = lengths
)
#>      column                              labels lengths
#> 1   STUDYID                    Study Identifier     200
#> 2   USUBJID           Unique Subject Identifier     200
#> 3     XXSEQ                     Sequence Number       8
#> 4  XXTESTCD                  XX Test Short Name       8
#> 5    XXTEST                        XX Test Name      40
#> 6   XXORRES Result or Finding in Original Units     200
#> 7    XXBLFL                       Baseline Flag       1
#> 8     VISIT                          Visit Name     200
#> 9     EPOCH                               Epoch     200
#> 10    XXDTC           Date/Time of Measurements      19
#> 11     XXDY                     Study Day of XX       8

Finally, we will write the SDTM XX domain validation table as a SAS transport file using write_tbl_to_xpt().

write_tbl_to_xpt(sdtm_xx7, filename = domain, dir = work_dir)

For each previous steps, we viewed the interim results to demonstrate the features of {sdtmval} however, {sdtmval} is designed to be used with pipe operators so that you can have one long, readable pipe. To demonstrate, we will reproduce the same results from above in one code chunk.

sdtm_xx <- edc_dat$xx %>%
  
  # pre-processing
  trim_and_make_blanks_NA() %>%
  
  # XXTEST
  dplyr::mutate(XXTEST = dplyr::recode(XXTESTCD, !!!xxtestcd_codelist)) %>%

  # get the VISITDTC column from the EDC VD form
  dplyr::left_join(edc_dat$vd, by = c("USUBJID", "VISIT")) %>%
  
  # XXDTC
  dplyr::rename(XXDTC = VISITDTC) %>%

  # get the study start/end dates by subject (RFSTDTC, RFXSTDTC, RFXENDTC)
  dplyr::left_join(sdtm_dat$dm, by = "USUBJID") %>%

  # XXBLFL
  create_BLFL(sort_date = "XXDTC",
              domain = domain,
              grouping_vars = c("USUBJID", "XXTESTCD")) %>%

  # EPOCH
  create_EPOCH(date_col = "XXDTC") %>%

  # XXDY
  calc_DY(DY_col = "XXDY", DTC_col = "XXDTC") %>%

  # XXSEQ
  assign_SEQ(key_vars = c("USUBJID", "XXTESTCD", "VISIT"),
             seq_prefix = domain) %>%

  # final formatting
  format_chars_and_dates() %>%
  dplyr::select(dplyr::any_of(spec$Variable)) %>%
  assign_meta_data(spec = spec)

# check if the two data frames are identical
identical(sdtm_xx, sdtm_xx7)
#> [1] TRUE
Metadata

Version

0.4.1

License

Unknown

Platforms (75)

    Darwin
    FreeBSD
    Genode
    GHCJS
    Linux
    MMIXware
    NetBSD
    none
    OpenBSD
    Redox
    Solaris
    WASI
    Windows
Show all
  • aarch64-darwin
  • aarch64-genode
  • aarch64-linux
  • aarch64-netbsd
  • aarch64-none
  • aarch64_be-none
  • arm-none
  • armv5tel-linux
  • armv6l-linux
  • armv6l-netbsd
  • armv6l-none
  • armv7a-darwin
  • armv7a-linux
  • armv7a-netbsd
  • armv7l-linux
  • armv7l-netbsd
  • avr-none
  • i686-cygwin
  • i686-darwin
  • i686-freebsd
  • i686-genode
  • i686-linux
  • i686-netbsd
  • i686-none
  • i686-openbsd
  • i686-windows
  • javascript-ghcjs
  • loongarch64-linux
  • m68k-linux
  • m68k-netbsd
  • m68k-none
  • microblaze-linux
  • microblaze-none
  • microblazeel-linux
  • microblazeel-none
  • mips-linux
  • mips-none
  • mips64-linux
  • mips64-none
  • mips64el-linux
  • mipsel-linux
  • mipsel-netbsd
  • mmix-mmixware
  • msp430-none
  • or1k-none
  • powerpc-netbsd
  • powerpc-none
  • powerpc64-linux
  • powerpc64le-linux
  • powerpcle-none
  • riscv32-linux
  • riscv32-netbsd
  • riscv32-none
  • riscv64-linux
  • riscv64-netbsd
  • riscv64-none
  • rx-none
  • s390-linux
  • s390-none
  • s390x-linux
  • s390x-none
  • vc4-none
  • wasm32-wasi
  • wasm64-wasi
  • x86_64-cygwin
  • x86_64-darwin
  • x86_64-freebsd
  • x86_64-genode
  • x86_64-linux
  • x86_64-netbsd
  • x86_64-none
  • x86_64-openbsd
  • x86_64-redox
  • x86_64-solaris
  • x86_64-windows