Skip to content
This repository has been archived by the owner on Apr 1, 2024. It is now read-only.

ISSUE-9571: Avro schema incompatible with schema upload vs python producer #2158

Open
sijie opened this issue Feb 12, 2021 · 0 comments
Open

Comments

@sijie
Copy link
Member

sijie commented Feb 12, 2021

Original Issue: apache#9571


Describe the bug
Hi i just want to know if i do something wrong or if someone has identify following behavior (and found some solution for fixing it)
I create a topic and upload the Avro Schema as Yaml

pulsar/bin/pulsar-admin topics create persistent://schema/test3/foo22
/pulsar/bin/pulsar-admin schemas upload persistent://schema/test3/foo22 -f /git_files/schema/schema-validation.avro

Content of Avro Yaml Schema:
{"type": "AVRO","schema":"{\"name\":\"Validation\",\"type\":\"record\",\"fields\":[{\"name\":\"messagetimestamp\",\"type\":\"string\"}]}","properties": {}}

Schema looks like that with schemas get

{
  "version": 0,
  "schemaInfo": {
    "name": "foo22",
    "schema": {
      "name": "Validation",
      "type": "record",
      "fields": [
        {
          "name": "messagetimestamp",
          "type": "string"
        }
      ]
    },
    "type": "AVRO",
    "properties": {}
  }
}

set-is-allow-auot-update-schema is set to disable(!) to cover that nowone is “destroying” or topic
but if i know use a python producer the schema is failing with “incompatible schema”
python code:

class Validation(Record):
    messagetimestamp = String(required=True)
producer = client.create_producer(topic, schema=AvroSchema(Validation))
inputString='Test55'
producer.send(Validation(
    messagetimestamp = inputString
    ))
print("Message send")
client.close()

same behaviour with python pulsar functions .
if i turn on the auto schema update a new schema is created but total same structure:

{
  "version": 1,
  "schemaInfo": {
    "name": "foo22",
    "schema": {
      "name": "Validation",
      "type": "record",
      "fields": [
        {
          "name": "messagetimestamp",
          "type": "string"
        }
      ]
    },
    "type": "AVRO",
    "properties": {}
  }
}

Any idea why that happens and how can i avoid that multiple producer (written in Java, Python) are changing the schema all time?!
To Reproduce
Steps to reproduce the behavior:

  1. create topic
  2. upload avro Schema (with yml reference -f yml file)
  3. deactivate auto schema update
  4. create python consumer (incompatible schema)
  5. enable auto schema update
  6. check schema (schema version changed)

Expected behavior
Schema version shouldn't be updated if python schema is the same as given in the schema upload (could it be that there is some hidden character given?!)

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

1 participant