[r/python] R and Python create groups with incompatible SOMA metadata

[bkmartinjr]

Objects created by R or Python bindings should have identical metadata, but currently the R and Python packages tag SOMA objects with different and incompatible metadata tags for dataset_type, soma_encoding_version and soma_object_type.

• Python creates with Unicode strings (e.g., "dataset_type": "soma")
• R package creates objects with byte arrays (e.g., "dataset_type": b"soma")

Using TileDB-Py to inspect two arrays.

When array created by Python (array info):

Out[25]: {'dataset_type': 'soma', 'soma_encoding_version': '1', 'soma_object_type': 'SOMAExperiment'}

When array created by R (array info):

Out[23]: {'dataset_type': b'soma', 'soma_encoding_version': b'1', 'soma_object_type': b'SOMAExperiment'}

I also checked directly reading from S3, i.e., not using the tiledb:// URI, and the result is the same.

Where a "string" or "byte array" is right, I think it is reasonably clear that there is a bug here - the mandatory metadata tags should be identical no matter which ingestion system is used, and which package is used to read it back.

Side note: the current Python package seems to have a work-around for this, as it detects byte array metadata and converts it to utf-8. This is nice, but doesn't seem like the right answer, as it requires any other user of that metadata (e.g., end-user code) to do the same encoding/decoding step for any/all metadata values.

In my opinion, we should be using utf-8 everywhere (and document that in the SOMA spec), but at a minimum, we should have common behavior across all reader/writer code.

tiledbsoma.__version__              1.11.4
TileDB-Py version                   0.29.0
TileDB core version (tiledb)        2.23.0
TileDB core version (libtiledbsoma) 2.23.0
python version                      3.11.9.final.0
OS version                          Linux 6.8.0-76060800daily20240311-generic

[johnkerl]

@nguyenv @eddelbuettel is this complete now that #2819 is merged?

[bkmartinjr]

@johnkerl would also like to confirm we have a unit test that would catch this...

[johnkerl]

@bkmartinjr I'll add a cross-language test here https://github.com/single-cell-data/TileDB-SOMA/tree/1.13.0/apis/system/tests