superset metadata ingest errors

[chenp-eleos]

Affected module
The superset metadata ingest fails and not all dashboard, charts and datasets are being shown.

Describe the bug
I am using The ingest from Postgre but it also happen when tried from the superset API. there are 2 types of errors:

This error:
[2024-10-01T10:00:38.425+0000] {step.py:230} WARNING - Object type defined in def _iter() /home/airflow/.local/lib/python3.10/site-packages/metadata/ingestion/api/topology_runner.py is not an Either: ['NoneType' object has no attribute 'right']
[2024-10-01T10:00:38.427+0000] {status.py:91} WARNING - Object type defined in def _iter() /home/airflow/.local/lib/python3.10/site-packages/metadata/ingestion/api/topology_runner.py is not an Either: ['NoneType' object has no attribute 'right']
[2024-10-01T10:00:38.612+0000] {taskinstance.py:2905} ERROR - Task failed with exception
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.10/site-packages/airflow/models/taskinstance.py", line 465, in _execute_task
result = _execute_callable(context=context, **execute_callable_kwargs)
File "/home/airflow/.local/lib/python3.10/site-packages/airflow/models/taskinstance.py", line 432, in _execute_callable
return execute_callable(context=context, **execute_callable_kwargs)
File "/home/airflow/.local/lib/python3.10/site-packages/airflow/models/baseoperator.py", line 400, in wrapper
return func(self, *args, **kwargs)
File "/home/airflow/.local/lib/python3.10/site-packages/airflow/operators/python.py", line 235, in execute
return_value = self.execute_callable()
File "/home/airflow/.local/lib/python3.10/site-packages/airflow/operators/python.py", line 252, in execute_callable
return self.python_callable(*self.op_args, **self.op_kwargs)
File "/home/airflow/.local/lib/python3.10/site-packages/openmetadata_managed_apis/workflows/ingestion/common.py", line 208, in metadata_ingestion_workflow
workflow.raise_from_status()
File "/home/airflow/.local/lib/python3.10/site-packages/metadata/workflow/workflow_status_mixin.py", line 134, in raise_from_status
raise err
File "/home/airflow/.local/lib/python3.10/site-packages/metadata/workflow/workflow_status_mixin.py", line 131, in raise_from_status
self.raise_from_status_internal(raise_warnings)
File "/home/airflow/.local/lib/python3.10/site-packages/metadata/workflow/ingestion.py", line 150, in raise_from_status_internal
raise WorkflowExecutionError(
metadata.config.common.WorkflowExecutionError: Superset reported errors: Superset Summary: [3 Records, [4 Updated Records, 0 Warnings, 1 Errors, 0 Filtered]

and many Errors like this:

Failed to ingest CreateDashboardDataModelRequest [7] due to api request failure: For column data type array, arrayDataType must not be null
{status.py:91} WARNING - Failed to ingest CreateDashboardDataModelRequest [18] due to api request failure: For column data type array, arrayDataType must not be null

To Reproduce

Just created a superset metadata ingest.

Expected behavior
not getting any error and complete ingest of all metedata

Version:

• OpenMetadata version: 1.5.3
• superset version: 4.0.1

@ayush-shah asked me to open this issue when asking why it happens in the slack channel.
It is really important to us so I wil l appriciate your prompt assistance here. thanks.

[firewall413]

Same issue here!

Importing using Postgres connection using admin credentials

• OpenMetadata version: 1.5.4
• Superset version: 4.0.0

We did not have this issue with OM 1.4.x