-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
kafkit fails to register schema under new subject if schema_id already exists in cache #4
Comments
That's a really good point. I don't think we've run into this issue yet because we've always set the name in the Avro schema to match the subject (so we haven't been properly exercising the Your use-case is totally valid, though, to support the If you'd like to PR a solution, I'd be happy to accept it. Or I can get to it :) I'm trying to think off the top of my head of what the best way to handle this is? Add a subject-based caching layer that gets used if the subject name is provided? |
Hi! |
Hi! Thanks again for a really useful library.
I'm working on moving centrally managed schemas into the application that actually owns them, and I've stumbled on an issue with the
register_schema
method.We're using avro encoding for the keys as well as the values in the kafka messages, and in this instance the key schema is just a simple string schema. All schemas owned by the application share the same key schema.
I noticed that when testing schema creation from the application using kafkit, only the key schema for the first subject is actually created in the schema registry, the others get short circuited in the
register_schema
method here:kafkit/kafkit/registry/sansio.py
Lines 369 to 372 in 1603628
This becomes a problem when other clients rely on the key schema_id being registered under the other subject as well in the schema registry, and they now crash when they can't find a key-schema for their subjects.
Update: A quick workaround for this problem is to wipe the schema cache in the registry manually when I loop through the schemas I need to make sure exist in the confluent schema registry, like so:
I'm happy to provide a PR with a solution when I know a bit more about the rationale and use case for short circuiting the schema registering.
The text was updated successfully, but these errors were encountered: