Improved handling of list columns with NULL entries

[sritchie73]

Updates the documentation for rbindlist(fill=TRUE) and the print method associated with NULL entries in list columns as per discussion in #4198

[sritchie73]

@mattdowle RE: printing NULL entries in list columns as "NULL" instead of "". This has a potentially undesirable side effect on the output of the tables() function, where the KEY column now shows "NULL" instead of "" for data.tables without a key (causing test 69.5 to fail).

E.g.:

> tables()
   NAME NROW NCOL MB COLS  KEY
1:   dt    3    1  0    A NULL
2:   DT    1    1  0    a NULL
Total: 0MB

Instead of:

> tables()
   NAME NROW NCOL MB COLS KEY
1:   dt    3    1  0    A 
2:   DT    1    1  0    a 
Total: 0MB

[MichaelChirico]

Same as #4196 -- just checking if this is still needed? If so we can merge to master and resume the review process (nearly 4 years later 😅).

Also if you'd like someone else to take over the PR, do feel free.

cc @ben-schwen since I know you've been working a lot on rbindlist lately.

[ben-schwen]

AFAIU this PR is only about documentation. Printing something might be better, to not confuse it with the empty string. But then we might should also start to print the empty string "". tibble btw prints it as <NULL>.

data.table(a="", b=list(""), c=list(NULL))
#         a      b      c
#    <char> <list> <list>
# 1:

tibble(a="a", b=list(""), c=list(NULL))
# # A tibble: 1 × 3
#   a     b         c     
#   <chr> <list>    <list>
# 1 a     <chr [1]> <NULL>

[MichaelChirico]

I like the idea of adding a further delimiter to distinguish NULL (sentinel value) from "NULL" (string value). It seems like at present the PR just changes from "can't distinguish NULL from """ to "can't distinguish NULL from "NULL"".

This is unavoidable, but using a delimiter at least makes it less likely that string is present in the data.

(PS, ideally this PR would be split into two separate PRs, one for the rbindlist documentation change, the other for the change in print behavior, but given the age of the PR / extended lack of review from us / small overall PR size, I'll allow the two-in-one PR in this case)

[MichaelChirico]

@ben-schwen I'll let you give final approval since there's a "mild" breaking change we should agree on (namely, anyone relying on the existing behavior not to print anything for NULL entries in print.data.table() would need to update).

[ben-schwen]

@ben-schwen I'll let you give final approval since there's a "mild" breaking change we should agree on (namely, anyone relying on the existing behavior not to print anything for NULL entries in print.data.table() would need to update).

No hard fillings on these things but definitely an improvement. I added an example to make it even easier to graph. Together with displaying column types, I guess we give now enough information to the users.