Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rename transform fails on JSONB column #212

Closed
bradnewman opened this issue Dec 13, 2021 · 4 comments
Closed

Rename transform fails on JSONB column #212

bradnewman opened this issue Dec 13, 2021 · 4 comments

Comments

@bradnewman
Copy link

PGSync version: 2.1.7
Postgres version: 13.0
Elasticsearch version: opensearch:1.1.0
Redis version: 6.2.1
Python version: 3.9.7
Problem Description: When trying to add a rename transform for a JSONB column, PGSync raises an exception. Rename transforms on other columns work as expected, and the JSONB column is sync'd as expected if no rename transform is specified for it.

Error Message (if any):

  File "/home/appuser/.local/lib/python3.9/site-packages/pgsync/sync.py", line 720, in _payloads
    yield from self._sync(filters=filters, extra=extra)
  File "/home/appuser/.local/lib/python3.9/site-packages/pgsync/sync.py", line 844, in _sync
    row: dict = transform(row, self.nodes)
  File "/home/appuser/.local/lib/python3.9/site-packages/pgsync/transform.py", line 115, in transform
    data = _rename_fields(data, transform_node)
  File "/home/appuser/.local/lib/python3.9/site-packages/pgsync/transform.py", line 30, in _rename_fields
    value = _rename_fields(value, nodes[key])
  File "/home/appuser/.local/lib/python3.9/site-packages/pgsync/transform.py", line 43, in _rename_fields
    elif key in nodes.keys():
AttributeError: 'str' object has no attribute 'keys'
@toluaina
Copy link
Owner

Can you please post the rename transform?

@bradnewman
Copy link
Author

I've simplified the schema JSON to this, and the same problem occurs - unless I remove the group_tags -> tags rename transform:

[{
    "database": "db",
    "index": "groups",
    "nodes": {
        "table": "groups",
        "schema": "public",
        "columns": [
            "id",
            "name",
            "group_tags",
        ],
        "transform": {
            "rename": {
                "id": "groupId",
                "name": "displayName",
                "group_tags": "tags"
            }
        }
    }
}]

And the table is like this - though it admittedly does have more columns than this, I don't think it should matter:

CREATE TABLE public.groups (
    id uuid NOT NULL,
    name character varying NOT NULL,
    description character varying,
    group_tags jsonb
);

@toluaina
Copy link
Owner

Thanks. This should be fixed now. Can you try using the master branch.

bradnewman pushed a commit to bradnewman/pgsync that referenced this issue Dec 16, 2021
@bradnewman
Copy link
Author

I've verified #212 resolves this issue - thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants