Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Destination Weaviate: Support array of known type #22530

Closed
samos123 opened this issue Feb 7, 2023 · 1 comment
Closed

Destination Weaviate: Support array of known type #22530

samos123 opened this issue Feb 7, 2023 · 1 comment
Labels
autoteam community connectors/destination/weaviate team/connectors-python type/bug Something isn't working

Comments

@samos123
Copy link
Contributor

samos123 commented Feb 7, 2023 •
edited
Loading

Currently this will be thrown when using Slack as source and Weaviate as destination:

{'error': [{'message': "invalid string property 'pending_shared' on class 'Channels': not a string, but []interface {}"}]}

I will fix this in the Weaviate destination by supporting an array of unkonwn types.

Example where is this happening: https://github.com/airbytehq/airbyte/blob/8d1a027c86b228bdb8dd4894427b04d87b4fdc6b/airbyte-integrations/connectors/source-slack/source_slack/schemas/channels.json#L50..L53
@samos123 samos123 added needs-triage type/bug Something isn't working labels Feb 7, 2023
samos123 added a commit to samos123/airbyte that referenced this issue Feb 7, 2023
@samos123
Fix issue with arrays of no data type
f8aac31 
Closes airbytehq#22530
@samos123 samos123 mentioned this issue Feb 7, 2023
Merged
37 tasks
@marcosmarxm
Copy link
Member

Thanks @samos123 I'll review soon your contribution

samos123 added a commit to samos123/airbyte that referenced this issue Feb 15, 2023
@samos123
Fix issue with arrays of no data type
bae1015 
Closes airbytehq#22530
danielduckworth pushed a commit to danielduckworth/airbyte that referenced this issue Mar 13, 2023
@samos123 @marcosmarxm
@octavia-squidington-iii @danielduckworth
🎉 Destination Weaviate: Support any string based ID and fix issues wi…
94ed4c6 
…th additionalProperties (airbytehq#22527)

* Support any string based ID

* Previously string based IDs that weren't convertable to hex would
  throw an exception. This fixes that issue by converting strings
  to 128 bit integer and converting the md5 hex into a UUID
* Update Weaviate to 1.17.3

* Fix issue with arrays of no data type

Closes airbytehq#22530

* add more testing coverage

* fix error where dataType wasn't set in weaviate

This seems to happen when records sometimes don't contain all the
properties that are defined in the schema.

This was the error observed by a user connecting Slack <> Weaviate:
property 'reactions': invalid dataType: dataType must have at least
one element, class 'Threads' not present in schema"

* add test case for additionalProperties

* Support additionalProps of object and array of obj

* Update changelog

* add weaviate icon

* fix flake tests

* add commit

* make build m1 work

* make build m1 work

* auto-bump connector version

---------

Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
autoteam community connectors/destination/weaviate team/connectors-python type/bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@samos123 @marcosmarxm @octavia-squidington-iii