pkg_score() returns NA for "assess_license"

[paulie-of-punskas]

I noticed that pkg_score() returns NA value, even when license is found. For example, let's take "askpass":

# === load riskmetric library
library("riskmetric")
library("magrittr")

# === create testing environment
dir.create(paste0(tempdir(), "/test_riskmetric"))
dir.create(paste0(tempdir(), "/test_riskmetric/source"))
dir.create(paste0(tempdir(), "/test_riskmetric/library"))
dir.create(paste0(tempdir(), "/test_riskmetric/downloads"))

# === download and unpack files
pkgs <- c("askpass", "dplyr", "openssl")
download.packages("askpass", destdir = paste0(tempdir(), "/test_riskmetric/downloads"))

lapply(list.files(paste0(tempdir(), "/test_riskmetric/downloads"), full.names = TRUE),
       untar, 
       exdir = paste0(tempdir(), "/test_riskmetric/source"))

# === get license
askpass_license_assess <- pkg_ref(x = paste0(tempdir(), "/test_riskmetric/source/askpass")) %>% 
  pkg_assess(assessments = riskmetric::all_assessments())

askpass_license_score <- pkg_ref(x = paste0(tempdir(), "/test_riskmetric/source/askpass")) %>% 
  pkg_assess(assessments = riskmetric::all_assessments()) %>% 
  pkg_score()

It looks like that pkg_assess() correctly finds and extracts data from the license file, however information is not passed to pkg_score():

> askpass_license_assess[["license"]]
[1] "MIT + file LICENSE"
> askpass_license_score[["license"]]
[1] NA
attr(,"label")
[1] "A binary indicator of whether the package ships with an acceptable license."
attr(,"class")
[1] "pkg_score" "numeric" 

Same situation persists for other packages.

[emilliman5]

This is actually expected behavior. From the source:

#' Score a package for acceptable license
#'
#' Maps a license string to a score
#'
#' @eval roxygen_score_family("license")
#'
#' @return score of metric license
#' @export
metric_score.pkg_metric_license <- function(x, ...) {
  # defering scoring of licenses until we have a bit more consensus or guidance
  NA_real_
}

I guess, the question is what is an acceptable scoring criterium for licenses? My initial thoughts on possible criteria to generate a score:

1. Is there a license? - Probably not very informative, IIRC CRAN/BioC require a license.
2. Is there a standard license in place (e.g. MIT, GPL-2, etc)
3. Is the license compatible user's intended use case? (e.g. does the license allow for commercial use?)
4. Does this package's license conflict with the licenses of its dependencies or reverse dependencies? (this is more of a cohort metric).

The team is open to suggestions on how to implement a metric from this assessment, so long as we can reach some sort of consensus.