Testbench for Univariate Time Series Cleaning.
cleanTS
cleanTS package focuses on developing a tool for making the process of cleaning large datasets simple and efficient. Currently it solely focuses on data cleaning for univariate time series data. The package is integrated with already developed and deployed tools for missing value imputation. It also provides a way for visualizing data at different resolutions, allowing micro-scale visualization. The ultimate goal is the creation of a handy software tool that deals with the problems, processes, analysis and visualization of big data time series, with minimum human intervention.
cleanTS()checks the data for missing and duplicate timestamps, performs missing value imputation and removes anomalies/outliers from the data.animate_interval()splits the data and generates an animated plot.interact_plot()is similar toanimate_interval()but creates an interactive plot which provides relatively more control over the animation.
The package can also be used using a shiny application, available at https://mayur1009.shinyapps.io/cleanTS/.
Package Documentation can be found at https://mayur1009.github.io/cleanTS/.
This project was a part of Google Summer of Code 2021.
Installation
# Install release version from CRAN
install.packages("cleanTS")
# Install development version from GitHub
devtools::install_github("Mayur1009/cleanTS")
Example
library(cleanTS)
#> Registered S3 method overwritten by 'quantmod':
#> method from
#> as.zoo.data.frame zoo
# Read sunspot.month dataset
data <- timetk::tk_tbl(sunspot.month)
print(data)
#> # A tibble: 3,177 × 2
#> index value
#> <yearmon> <dbl>
#> 1 Jan 1749 58
#> 2 Feb 1749 62.6
#> 3 Mar 1749 70
#> 4 Apr 1749 55.7
#> 5 May 1749 85
#> 6 Jun 1749 83.5
#> 7 Jul 1749 94.8
#> 8 Aug 1749 66.3
#> 9 Sep 1749 75.9
#> 10 Oct 1749 75.5
#> # ℹ 3,167 more rows
# Randomly insert missing values to simulate missing value imputation
set.seed(10)
ind <- sample(nrow(data), 100)
data$value[ind] <- NA
# Create `cleanTS` object
cts <- cleanTS(data, date_format = c("my"))
summary(cts)
#> Length Class Mode
#> clean_data 5 data.table list
#> missing_ts 0 POSIXct numeric
#> duplicate_ts 0 POSIXct numeric
#> imp_methods 4 -none- character
#> mcar_err 4 data.frame list
#> mar_err 4 data.frame list
#> outliers 4 data.table list
#> outlier_mcar_err 4 data.frame list
#> outlier_mar_err 4 data.frame list
# Cleaned Data
head(cts$clean_data)
#> time value missing_type method_used is_outlier
#> 1: 1749-01-01 58.0 <NA> <NA> FALSE
#> 2: 1749-02-01 62.6 <NA> <NA> FALSE
#> 3: 1749-03-01 70.0 <NA> <NA> FALSE
#> 4: 1749-04-01 55.7 <NA> <NA> FALSE
#> 5: 1749-05-01 85.0 <NA> <NA> FALSE
#> 6: 1749-06-01 83.5 <NA> <NA> FALSE
# Genearate animated plot
a <- animate_interval(cts, interval = "10 year")
gen.animation(a, height = 700, width = 900)
# Generate interactive plot
interact_plot(cts, interval = "10 year")