Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v0.5.0 Causes breaking change for certain model outputs. #19

Closed
dchoi-viant opened this issue May 18, 2023 · 3 comments
Closed

v0.5.0 Causes breaking change for certain model outputs. #19

dchoi-viant opened this issue May 18, 2023 · 3 comments
Assignees
Labels
bug Something isn't working

Comments

@dchoi-viant
Copy link
Contributor

dchoi-viant commented May 18, 2023

For further non-breaking changes, please clean up the examples directory and ensure e2e is working.

@dchoi-viant dchoi-viant added the bug Something isn't working label May 18, 2023
@dchoi-viant dchoi-viant self-assigned this May 18, 2023
@dchoi-viant dchoi-viant changed the title 0.5.0 Causes breaking change for certain model outputs. v0.5.0 Causes breaking change for certain model outputs. May 18, 2023
@dchoi-viant
Copy link
Contributor Author

dchoi-viant commented May 22, 2023

class LookupLayer(layers.Layer):
    def __init__(self, keys, values, default_value, name=None, table_name=None):
        self.keys = keys
        self.values = values
        self.default_value = default_value
        self.table_name = table_name
        super().__init__(name=name, trainable=False)

    def build(self, input_shape):
        keys_tensor = tf.constant(self.keys)
        values_tensor = tf.constant(self.values)
        kv_init = tf.lookup.KeyValueTensorInitializer(keys=keys_tensor, values=values_tensor)
        table = tf.lookup.StaticHashTable(kv_init, default_value=self.default_value, name=self.table_name)
        self.table = table

    def call(self, inputs):
        return self.table.lookup(inputs)

if __name__ == '__main__':
    # sample.csv can have columns s1, s2 for testing, with some random string values and some rows
    lookup_source = pd.read_csv('sample.csv').set_index('label')
    str_cols = ['s1', 's2']

    input = layers.Input(name='s1_input', shape=(1,), dtype=tf.float64)
    keys = lookup_source.index.to_numpy()
    lookup_layers = [LookupLayer(keys, lookup_source[c].to_numpy(dtype='str'), default_value='', name=c) for c in str_cols]

    outputs = [lookup_layer(input) for lookup_layer in lookup_layers]
    wrapped_model = models.Model(inputs=[input], outputs=outputs, name='broken_case_v0_5_0')

@dchoi-viant
Copy link
Contributor Author

dchoi-viant commented May 31, 2023

While digging through this code, there are two other issues to address:

  1. Dictionary hashes for models with wildcards may not be working as expected - changes to model dictionary may not be invalidating caches as appropriate.
  2. The local cache on the server side is no longer relevant - in commit 3f8d43f the read from local cache was removed, and now there's only a write to local and L1 backplane; this most likely seems like an oversight, or some different less obvious code path that I'm overseeing.

@dchoi-viant
Copy link
Contributor Author

Should be addressed via v0.6.0 tag.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant