Support weight serialization of string tensors

[dsmilkov]

Cloud AutoML models contain the vocab as a weight of dtype "string". The infra for string tensors is in core, however we don't yet have the ability to serialize the string values into the .bin file.

The proposal is to follow the protobuf spec for strings, which encodes strings as utf-8. Then use the built-in TextDecoder in JS to decode utf-8 to js strings.

The numpy array of strings in python will follow the built-in serialization (utf-8 strings separated by a null-terminating byte), which then can be converted into string[] using TextDecoder.decode(bytes).split('\u0000')

The separator code point (\u0000) and the byte length of the string tensor will be stored as metadata in the WeightsManifestEntry. We should store the separator in case we want to change it later and be backwards compatible with previously serialized files.

[dsmilkov]

cc @nsthorat @pyu10055 @caisq @tafsiri

[nsthorat]

Seems reasonable, you just need to encode the byte length on the other side since we assume the shape is proportional to byte length.

[dsmilkov]

Yup, clarified in the description

[tafsiri]

+1. Just clarifying that we want to store the encoded string tensors in the bin files (just like the other weights).

[pyu10055]

@dsmilkov SavedModel encode the string attributes with base64 encoding when saved as JSON format, we should probably use the same encoding for string tensors. We also need to consider other platforms that do not have TextDecoder built-in function.

[nsthorat]

If possible we should avoid base64 encoding of strings because:

• It blows out the size by 50%
• Parsing is very slow
• We’d need a special way to encode that string (e.g. in model.json). Currently weights are shipped as bytes as it is and we should reuse that infrastructure, not inventing a new way to ship string weights.

[dsmilkov]

@pyu10055 Vocabs can be large (e.g. it is common to have 10,000 english words in models), so I think we should treat them as any other weight (they are a weight after all). This allows us to decouple large weights from model.json, shard them into 4MB chucks, etc.

Continuing to use base64 for encoding "string" attributes in graph nodes SGTM.

TextDecoder is supported by all major desktop and mobile browsers we currently support with tf.js: https://caniuse.com/#feat=textencoder . If there are issues we can use a polyfill.