Issues in kickoff descriptions

[matthewseymour]

In the 1999-2010 .csv.gz files, I see numerous kickoffs where the player name "D.Holland" replaces the number of yards in the kickoff description. For example:
1868,2008_04_SF_NO,...,J.Nedney kicks D.Holland yards from SF 30 to NO -3. P.Thomas to NO 26 for 29 yards (J.Ulbrich).,...

Where a "normal" kickoff description looks like: "S.Weatherford kicks 66 yards from NO 30 to SF 4. A.Rossum pushed ob at SF 27 for 23 yards (U.Young).,kickoff"

These all seem to be kicks of 73 yards.
The issue stops occurring after 2010.

[mrcaseb]

After changing the data source, this bug affects 91 plays in the seasons 1999 and 2000

library(dplyr, warn.conflicts = FALSE)
options(tibble.print_min = 30)
options(width = 150)

pbp <- nflfastR::load_pbp(1999:2000)

pbp %>% 
  dplyr::filter(stringr::str_detect(desc, "kicks [:alpha:]{1,}.[:alpha:]{1,} yards")) %>%
  dplyr::select(game_id, play_id, desc)
#> # A tibble: 91 x 3
#>    game_id        play_id desc                                                                                                                        
#>    <chr>            <dbl> <chr>                                                                                                                       
#>  1 1999_01_NE_NYJ    1875 A.Vinatieri kicks D.Holland yards from NE 30 to NYJ -3. L.Johnson to NYJ 14 for 17 yards (H.Shaw). FUMBLES (H.Shaw), recove~
#>  2 1999_01_PIT_C~    1527 K.Brown kicks D.Holland yards from PIT 30 to CLE -3. D.Chiaverini to CLE 10 for 13 yards (H.Ward).                          
#>  3 1999_02_GB_DET    1961 R.Longwell kicks D.Holland yards from GB 30 to DET -3. B.Olivo to DET 18 for 21 yards (S.McGarrahan).                       
#>  4 1999_02_SD_CIN    1023 J.Carney kicks D.Holland yards from SD 30 to CIN -3. D.Griffin to CIN 39 for 42 yards (J.Spencer).                          
#>  5 1999_03_CHI_O~    3870 M.Husted kicks D.Holland yards from OAK 30 to CHI -3. G.Milburn to CHI 24 for 27 yards (T.Mickens).                         
#>  6 1999_03_MIN_GB    2591 R.Longwell kicks D.Holland yards from GB 30 to MIN -3. D.Palmer to MIN 26 for 29 yards (C.Bradford).                        
#>  7 1999_04_CAR_W~    3198 J.Kasay kicks D.Holland yards from CAR 30 to WAS -3. B.Mitchell to WAS 15 for 18 yards (R.Mullen). PENALTY on WAS-T.Paul, O~
#>  8 1999_04_NYJ_D~    2021 J.Elam kicks D.Holland yards from DEN 30 to NYJ -3. A.Glenn to NYJ 17 for 20 yards (E.Brown).                               
#>  9 1999_05_JAX_N~    2058 J.Hall kicks D.Holland yards from NYJ 30 to JAX -3. R.Barlow to JAX 25 for 28 yards (R.Farmer). PENALTY on JAX-B.Boyer, Off~
#> 10 1999_05_SF_STL      35 J.Wilkins kicks D.Holland yards from STL 30 to SF -3. R.McQuarters to SF 19 for 22 yards (M.Morton).                        
#> 11 1999_06_DAL_N~    1934 B.Daluiso kicks D.Holland yards from NYG 30 to DAL -3. K.Mathis, Touchback.                                                 
#> 12 1999_06_MIA_NE    1313 O.Mare kicks D.Holland yards from MIA 30 to NE -3. K.Faulk pushed ob at MIA 42 for 61 yards (O.Mare). PENALTY on MIA-O.Mare~
#> 13 1999_06_MIN_D~    1225 J.Hanson kicks D.Holland yards from DET 30 to MIN -3. D.Palmer to MIN 26 for 29 yards (R.Jordan).                           
#> 14 1999_06_MIN_D~    2890 M.Berger kicks D.Holland yards from MIN 30 to DET -3. T.Fair to DET 18 for 21 yards (M.Williams).                           
#> 15 1999_07_CLE_S~    1092 J.Wilkins kicks D.Holland yards from STL 30 to CLE -3. R.Powell to CLE 40 for 43 yards (C.Clemons).                         
#> 16 1999_07_CLE_S~    3422 J.Wilkins kicks D.Holland yards from STL 30 to CLE -3. R.Powell to CLE 31 for 34 yards (T.Pelshak).                         
#> 17 1999_08_CAR_A~    1444 M.Andersen kicks D.Holland yards from ATL 30 to CAR -3. M.Bates, Touchback.                                                 
#> 18 1999_08_CHI_W~    3484 B.Conway kicks D.Holland yards from WAS 30 to CHI -3. G.Milburn pushed ob at CHI 19 for 22 yards (D.Pounds).                
#> 19 1999_08_DAL_I~    2467 D.Kight kicks D.Holland yards from IND 30 to DAL -3. J.Tucker to DAL 19 for 22 yards (M.Pollard, S.Reid).                   
#> 20 1999_08_JAX_C~    3304 S.Lindsey kicks D.Holland yards from JAX 30 to CIN -3. T.Mack to CIN 21 for 24 yards (B.McElmurry).                         
#> 21 1999_08_MIN_D~    1498 M.Berger kicks D.Holland yards from MIN 30 to DEN -3. C.Watson to DEN 14 for 17 yards (H.Morrow).                           
#> 22 1999_08_MIN_D~    2203 J.Elam kicks D.Holland yards from DEN 30 to MIN -3. D.Palmer to MIN 18 for 21 yards (D.Bowens).                             
#> 23 1999_08_MIN_D~    3254 M.Berger kicks D.Holland yards from MIN 30 to DEN -3. C.Watson to DEN 26 for 29 yards (R.Griffith).                         
#> 24 1999_08_STL_T~     922 C.Hentrich kicks D.Holland yards from TEN 30 to STL -3. R.Carpenter to STL 34 for 37 yards (P.Phenix).                      
#> 25 1999_08_TB_DET     981 J.Hanson kicks D.Holland yards from DET 30 to TB -3. Y.Murphy, Touchback.                                                   
#> 26 1999_08_TB_DET    2029 J.Hanson kicks D.Holland yards from DET 30 to TB -3. Y.Murphy to TB 17 for 20 yards (I.Uwaezuoke).                          
#> 27 1999_09_STL_D~    1503 J.Hanson kicks D.Holland yards from DET 30 to STL -3. R.Carpenter to STL 40 for 43 yards (T.Talton).                        
#> 28 1999_10_GB_DAL      35 T.Gowin kicks D.Holland yards from DAL 30 to GB -3. D.Parker, Touchback.                                                    
#> 29 1999_10_NYJ_NE    1203 J.Hall kicks D.Holland yards from NYJ 30 to NE -3. K.Faulk to NYJ 8 for 95 yards (A.Pleasant).                              
#> 30 1999_11_OAK_D~    2240 M.Husted kicks D.Holland yards from OAK 30 to DEN -3. C.Watson to DEN 41 for 44 yards (Z.Crockett).                         
#> # ... with 61 more rows

^{Created on 2021-03-02 by the reprex package (v1.0.0)}

[mrcaseb]

Checked the stat ids of the problematic plays with the below code.

The kick distance of every kick was indeed 73 yards. So if we want to fix this we just have to replace the string
kicks D.Holland yards with kicks 73 yards

pbp <- load_pbp(1999:2000)

broken <- pbp %>% 
  dplyr::filter(stringr::str_detect(desc, "kicks [:alpha:]{1,}.[:alpha:]{1,} yards")) %>%
  dplyr::select(game_id, play_id, desc, yards_gained) %>% 
  dplyr::mutate(id = paste(game_id, play_id, sep = "_"))


check_old_games_stats <- function(gameId, stat_ids){
  season <- as.integer(substr(gameId, 1, 4))
  
  path <- "https://raw.githubusercontent.com/guga31bb/nflfastR-raw/master/raw"
  
  fetched <- curl::curl_fetch_memory(glue::glue("{path}/{season}/{gameId}.rds"))
  
  if (fetched$status_code == 404) {
    return(tibble())
  } else if (fetched$status_code == 500) {
    return(tibble())
  }
  
  raw <- nflfastR:::read_raw_rds(fetched$content)

  game_json <- raw[[1]]
  
  # excluding last element since it's "crntdrv" and not an actual
  drives <- game_json$drives[-length(game_json$drives)]
  
  # list of plays
  # each play has "players" column which is a list of player stats from the play
  plays <- suppressWarnings(furrr::future_map_dfr(seq_along(drives), function(x) {
    cbind(
      "drive" = x,
      data.frame(do.call(
        rbind,
        drives[[x]]$plays
      ))[, c(1:11)]
    ) %>% dplyr::mutate(play_id = names(drives[[x]]$plays), play_id = as.integer(.data$play_id))
  }
  ))
  
  stats <- furrr::future_map_dfr(seq_along(plays$play_id), function(x) {
    dplyr::bind_rows(plays[x, ]$players[[1]], .id = "player_id") %>%
      dplyr::mutate(play_id = plays[x, ]$play_id)
  }
  ) %>%
    dplyr::mutate(
      game_id = gameId,
      sequence = as.numeric(.data$sequence),
      statId = as.numeric(.data$statId),
      play_id = as.character(.data$play_id),
      yards = as.integer(.data$yards)
    ) %>%
    dplyr::arrange(.data$play_id, .data$sequence) %>% 
    janitor::clean_names() %>% 
    dplyr::select(game_id, play_id, sequence, player_name, player_id, stat_id, yards) %>% 
    dplyr::filter(stat_id %in% stat_ids)
  stats
}

all_stats <- furrr::future_map_dfr(broken$game_id, check_old_games_stats, 42:43)

all_stats %>% 
  dplyr::mutate(id = paste(game_id, play_id, sep = "_")) %>% 
  dplyr::filter(id %in% broken$id) %>% 
  dplyr::group_by(id) %>%
  dplyr::arrange(-yards) %>%
  dplyr::slice(1) %>%
  tibble::view()