-
Notifications
You must be signed in to change notification settings - Fork 4.5k
Closed
Description
What happened?
Steps to reproduce:
- Create a table in a database (e.g. Postgres) and insert rows
CREATE TABLE my_test (ITEM VARCHAR(25), LAST_TS DATE);
INSERT INTO my_test (ITEM, LAST_TS) VALUES ('item1', '2025-06-15');
INSERT INTO my_test (ITEM, LAST_TS) VALUES ('item2', '2025-06-16');
INSERT INTO my_test (ITEM, LAST_TS) VALUES ('item3', NULL);- Run the following python code
import logging
import apache_beam as beam
from apache_beam.options.pipeline_options import PipelineOptions
from apache_beam.io.jdbc import ReadFromJdbc
logging.basicConfig(level=logging.INFO)
options = PipelineOptions()
driver = 'org.postgresql.Driver'
jdbc_url = 'jdbc:postgresql://localhost:5432/<your_db>'
username = 'postgres'
password = ''
with beam.Pipeline(options=options) as p:
_ = (
p
| 'Read from jdbc' >> ReadFromJdbc(
table_name="my_test",
driver_class_name=driver,
jdbc_url=jdbc_url,
username=username,
password=password)
)
| beam.Map(print))The following error is raised when running the pipeline.
File "../sdks/python/apache_beam/typehints/schemas.py", line 926, in to_language_type
return Timestamp(seconds=int(value.seconds), micros=int(value.micros))
^^^^^^^^^^^^^^^^^^
TypeError: int() argument must be a string, a bytes-like object or a real number, not 'NoneType'
There is a workaround to put the following lines before the "with" statement.
from apache_beam.coders.row_coder import LogicalType
from apache_beam.typehints.schemas import MillisInstant
LogicalType.register_logical_type(MillisInstant)Issue Priority
Priority: 2 (default / most bugs should be filed as P2)
Issue Components
- Component: Python SDK
- Component: Java SDK
- Component: Go SDK
- Component: Typescript SDK
- Component: IO connector
- Component: Beam YAML
- Component: Beam examples
- Component: Beam playground
- Component: Beam katas
- Component: Website
- Component: Infrastructure
- Component: Spark Runner
- Component: Flink Runner
- Component: Samza Runner
- Component: Twister2 Runner
- Component: Hazelcast Jet Runner
- Component: Google Cloud Dataflow Runner