Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Choi algorithm failing due to timestamps, but they are correct #12

Open
muschellij2 opened this issue Jul 15, 2024 · 1 comment
Open

Choi algorithm failing due to timestamps, but they are correct #12

muschellij2 opened this issue Jul 15, 2024 · 1 comment

Comments

@muschellij2
Copy link
Contributor

Here is a reprex of the issues that are occurring due to specific storage types that seems to be a "bug"

library(agcounts)
library(read.gt3x)
library(actigraph.sleepr)
library(curl)
#> Using libcurl 8.4.0 with LibreSSL/3.3.6
library(dplyr)
#> 
#> Attaching package: 'dplyr'
#> The following objects are masked from 'package:stats':
#> 
#>     filter, lag
#> The following objects are masked from 'package:base':
#> 
#>     intersect, setdiff, setequal, union

Downloading and reading in the data

library(tidyr)
gt3x_file = tempfile(fileext = ".gt3x")
# https://github.com/muschellij2/Wrist-Worn-Accelerometry-Processing-Pipeline
url = "https://figshare.com/ndownloader/files/47702005"
curl::curl_download(url = url, destfile = gt3x_file)

df = read.gt3x::read.gt3x(path = gt3x_file, 
                          asDataFrame = TRUE, 
                          imputeZeroes = TRUE)
df
#> Sampling Rate: 30Hz
#> Firmware Version: 1.9.2
#> Serial Number Prefix: MOS
#>                  time     X     Y      Z
#> 1 2017-10-30 15:00:00 0.188 0.145 -0.984
#> 2 2017-10-30 15:00:00 0.180 0.125 -0.988
#> 3 2017-10-30 15:00:00 0.184 0.121 -0.984
#> 4 2017-10-30 15:00:00 0.184 0.121 -0.992
#> 5 2017-10-30 15:00:00 0.184 0.117 -0.988
#> 6 2017-10-30 15:00:00 0.184 0.125 -0.988

Here we are using last observation carried forward to match idle sleep mode

# impute idle sleep mode
sample_rate = attr(df, "sample_rate")
acceleration_max = as.numeric(attr(df, "acceleration_max"))
df = dplyr::as_tibble(df)
df = df %>% 
  # find where all zeroes/imputed zeroes
  mutate(all_zero = X == 0 & Y == 0 & Z == 0) %>% 
  # replace all 0 with NA so it can be filled  
  mutate(
    X = ifelse(all_zero, NA_real_, X),
    Y = ifelse(all_zero, NA_real_, Y),
    Z = ifelse(all_zero, NA_real_, Z)
  )
any(df$all_zero)
#> [1] TRUE

Filling in the data

df = df %>% 
  select(-all_zero) %>% 
  tidyr::fill(X, Y, Z, .direction = "down")
head(df)
#> # A tibble: 6 × 4
#>   time                    X     Y      Z
#>   <dttm>              <dbl> <dbl>  <dbl>
#> 1 2017-10-30 15:00:00 0.188 0.145 -0.984
#> 2 2017-10-30 15:00:00 0.18  0.125 -0.988
#> 3 2017-10-30 15:00:00 0.184 0.121 -0.984
#> 4 2017-10-30 15:00:00 0.184 0.121 -0.992
#> 5 2017-10-30 15:00:00 0.184 0.117 -0.988
#> 6 2017-10-30 15:00:00 0.184 0.125 -0.988

Getting activity counts

ac60 = df %>% 
  agcounts::calculate_counts(epoch = 60L)

Error

# needed for `actigraph.sleepr`
ac60 = ac60 %>%
  rename(timestamp = time)
choi_nonwear = actigraph.sleepr::apply_choi(ac60)
#> Error: Missing timestamps. Epochs should be evenly spaced from first(timestamp) to last(timestamp).

We dig a bit and see that has_missing_epochs is coming up TRUE, but this seems to be a bug

# error happens at actigraph.sleepr:::check_no_missing_timestamps
has_missing_epochs(ac60)
#> [1] TRUE

Taking the code from has_missing_epochs_ and running it shows that it fails on identical:

# fais at actigraph.sleepr:::has_missing_epochs_
epoch_len <- get_epoch_length(ac60)
epochs <- seq(first(ac60$timestamp), last(ac60$timestamp), 
              by = epoch_len)
identical(epochs, ac60$timestamp)
#> [1] FALSE

But if we use all.equal we see that this returns TRUE

all.equal(epochs, ac60$timestamp)
#> [1] TRUE

And if we try == equality we see a TRUE

all(epochs == ac60$timestamp)
#> [1] TRUE
attributes(epochs)
#> $class
#> [1] "POSIXct" "POSIXt" 
#> 
#> $tzone
#> [1] "UTC"
attributes(ac60$timestamp)
#> $class
#> [1] "POSIXct" "POSIXt" 
#> 
#> $tzone
#> [1] "UTC"

There are no different attributes and conversion to numeric we get:

identical(as.numeric(epochs), as.numeric(ac60$timestamp))
#> TRUE

We see the issue is with their type, which seems relatively minor and a byproduct

typeof(ac60$timestamp)
#> [1] "integer"
typeof(epochs)
#> [1] "double"

And this discrepancy is potentially coming from the epoch_len, but definitely comes from the dplyr::first and dplyr::last:

typeof(ac60$timestamp[1])
#> [1] "integer"
typeof(first(ac60$timestamp))
#> [1] "double"

Created on 2024-07-15 with reprex v2.1.0

Session info 
sessioninfo::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#>  setting  value
#>  version  R version 4.4.0 (2024-04-24)
#>  os       macOS Sonoma 14.4.1
#>  system   x86_64, darwin20
#>  ui       X11
#>  language (EN)
#>  collate  en_US.UTF-8
#>  ctype    en_US.UTF-8
#>  tz       America/New_York
#>  date     2024-07-15
#>  pandoc   3.2 @ /usr/local/bin/ (via rmarkdown)
#> 
#> ─ Packages ───────────────────────────────────────────────────────────────────
#>  package          * version date (UTC) lib source
#>  actigraph.sleepr * 0.2.0   2023-07-31 [1] Github (dipetkov/actigraph.sleepr@e754679)
#>  agcounts         * 0.6.8   2024-06-04 [1] local
#>  assertthat         0.2.1   2019-03-21 [1] CRAN (R 4.4.0)
#>  bit                4.0.5   2022-11-15 [1] CRAN (R 4.4.0)
#>  bit64              4.0.5   2020-08-30 [1] CRAN (R 4.4.0)
#>  blob               1.2.4   2023-03-17 [1] CRAN (R 4.4.0)
#>  bslib              0.7.0   2024-03-29 [1] CRAN (R 4.4.0)
#>  cachem             1.1.0   2024-05-16 [1] CRAN (R 4.4.0)
#>  cli                3.6.2   2023-12-11 [1] CRAN (R 4.4.0)
#>  colorspace         2.1-0   2023-01-23 [1] CRAN (R 4.4.0)
#>  curl             * 5.2.1   2024-03-01 [1] CRAN (R 4.4.0)
#>  data.table         1.15.4  2024-03-30 [1] CRAN (R 4.4.0)
#>  DBI                1.2.2   2024-02-16 [1] CRAN (R 4.4.0)
#>  digest             0.6.35  2024-03-11 [1] CRAN (R 4.4.0)
#>  dplyr            * 1.1.4   2023-11-17 [1] CRAN (R 4.4.0)
#>  evaluate           0.23    2023-11-01 [1] CRAN (R 4.4.0)
#>  fansi              1.0.6   2023-12-08 [1] CRAN (R 4.4.0)
#>  fastmap            1.2.0   2024-05-15 [1] CRAN (R 4.4.0)
#>  fs                 1.6.4   2024-04-25 [1] CRAN (R 4.4.0)
#>  generics           0.1.3   2022-07-05 [1] CRAN (R 4.4.0)
#>  GGIR               3.1-0   2024-06-28 [1] local
#>  ggplot2            3.5.1   2024-04-23 [1] CRAN (R 4.4.0)
#>  glue               1.7.0   2024-01-09 [1] CRAN (R 4.4.0)
#>  gsignal            0.3-5   2022-05-15 [1] CRAN (R 4.4.0)
#>  gtable             0.3.5   2024-04-22 [1] CRAN (R 4.4.0)
#>  htmltools          0.5.8.1 2024-04-04 [1] CRAN (R 4.4.0)
#>  htmlwidgets        1.6.4   2023-12-06 [1] CRAN (R 4.4.0)
#>  httpuv             1.6.15  2024-03-26 [1] CRAN (R 4.4.0)
#>  jquerylib          0.1.4   2021-04-26 [1] CRAN (R 4.4.0)
#>  jsonlite           1.8.8   2023-12-04 [1] CRAN (R 4.4.0)
#>  knitr              1.46    2024-04-06 [1] CRAN (R 4.4.0)
#>  later              1.3.2   2023-12-06 [1] CRAN (R 4.4.0)
#>  lattice            0.22-6  2024-03-20 [1] CRAN (R 4.4.0)
#>  lifecycle          1.0.4   2023-11-07 [1] CRAN (R 4.4.0)
#>  lubridate          1.9.3   2023-09-27 [1] CRAN (R 4.4.0)
#>  magrittr           2.0.3   2022-03-30 [1] CRAN (R 4.4.0)
#>  Matrix             1.7-0   2024-03-22 [1] CRAN (R 4.4.0)
#>  memoise            2.0.1   2021-11-26 [1] CRAN (R 4.4.0)
#>  mime               0.12    2021-09-28 [1] CRAN (R 4.4.0)
#>  munsell            0.5.1   2024-04-01 [1] CRAN (R 4.4.0)
#>  pillar             1.9.0   2023-03-22 [1] CRAN (R 4.4.0)
#>  pkgconfig          2.0.3   2019-09-22 [1] CRAN (R 4.4.0)
#>  png                0.1-8   2022-11-29 [1] CRAN (R 4.4.0)
#>  pracma             2.4.4   2023-11-10 [1] CRAN (R 4.4.0)
#>  promises           1.3.0   2024-04-05 [1] CRAN (R 4.4.0)
#>  purrr              1.0.2   2023-08-10 [1] CRAN (R 4.4.0)
#>  R.cache            0.16.0  2022-07-21 [1] CRAN (R 4.4.0)
#>  R.methodsS3        1.8.2   2022-06-13 [1] CRAN (R 4.4.0)
#>  R.oo               1.26.0  2024-01-24 [1] CRAN (R 4.4.0)
#>  R.utils            2.12.3  2023-11-18 [1] CRAN (R 4.4.0)
#>  R6                 2.5.1   2021-08-19 [1] CRAN (R 4.4.0)
#>  Rcpp               1.0.12  2024-01-09 [1] CRAN (R 4.4.0)
#>  RcppRoll           0.3.0   2018-06-05 [1] CRAN (R 4.4.0)
#>  reactable          0.4.4   2023-03-12 [1] CRAN (R 4.4.0)
#>  read.gt3x        * 1.2.0   2024-07-11 [1] local
#>  reprex             2.1.0   2024-01-11 [1] CRAN (R 4.4.0)
#>  reticulate         1.37.0  2024-05-21 [1] CRAN (R 4.4.0)
#>  rlang              1.1.3   2024-01-10 [1] CRAN (R 4.4.0)
#>  rmarkdown          2.27    2024-05-17 [1] CRAN (R 4.4.0)
#>  RSQLite            2.3.6   2024-03-31 [1] CRAN (R 4.4.0)
#>  rstudioapi         0.16.0  2024-03-24 [1] CRAN (R 4.4.0)
#>  sass               0.4.9   2024-03-15 [1] CRAN (R 4.4.0)
#>  scales             1.3.0   2023-11-28 [1] CRAN (R 4.4.0)
#>  sessioninfo        1.2.2   2021-12-06 [1] CRAN (R 4.4.0)
#>  shiny              1.8.1.1 2024-04-02 [1] CRAN (R 4.4.0)
#>  stringi            1.8.4   2024-05-06 [1] CRAN (R 4.4.0)
#>  stringr            1.5.1   2023-11-14 [1] CRAN (R 4.4.0)
#>  styler             1.10.3  2024-04-07 [1] CRAN (R 4.4.0)
#>  tibble             3.2.1   2023-03-20 [1] CRAN (R 4.4.0)
#>  tidyr            * 1.3.1   2024-01-24 [1] CRAN (R 4.4.0)
#>  tidyselect         1.2.1   2024-03-11 [1] CRAN (R 4.4.0)
#>  timechange         0.3.0   2024-01-18 [1] CRAN (R 4.4.0)
#>  utf8               1.2.4   2023-10-22 [1] CRAN (R 4.4.0)
#>  vctrs              0.6.5   2023-12-01 [1] CRAN (R 4.4.0)
#>  withr              3.0.0   2024-01-16 [1] CRAN (R 4.4.0)
#>  xfun               0.44    2024-05-15 [1] CRAN (R 4.4.0)
#>  xtable             1.8-4   2019-04-21 [1] CRAN (R 4.4.0)
#>  yaml               2.3.8   2023-12-11 [1] CRAN (R 4.4.0)
#>  zoo                1.8-12  2023-04-13 [1] CRAN (R 4.4.0)
#> 
#>  [1] /Library/Frameworks/R.framework/Versions/4.4-x86_64/Resources/library
#> 
#> ─ Python configuration ───────────────────────────────────────────────────────
#>  python:         /Users/johnmuschelli/miniconda3/bin/python3
#>  libpython:      /Users/johnmuschelli/miniconda3/lib/libpython3.11.dylib
#>  pythonhome:     /Users/johnmuschelli/miniconda3:/Users/johnmuschelli/miniconda3
#>  version:        3.11.4 (main, Jul  5 2023, 08:41:25) [Clang 14.0.6 ]
#>  numpy:          /Users/johnmuschelli/miniconda3/lib/python3.11/site-packages/numpy
#>  numpy_version:  1.25.2
#>  pygt3x:         /Users/johnmuschelli/miniconda3/lib/python3.11/site-packages/pygt3x
#>  
#>  NOTE: Python version was forced by RETICULATE_PYTHON
#> 
#> ──────────────────────────────────────────────────────────────────────────────
@muschellij2
Copy link
Contributor Author

muschellij2 commented Jul 15, 2024 •
edited
Loading

I've tracked the issue down to vctrs:vec_slice: r-lib/vctrs#1781. Workaround, must use typeof(timestamp_column) = "double" and rerun

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@muschellij2