Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Issue] Error while running simpler_model: I will skip this one | Error: HTTPSConnectionPool(host='api.github.com', port=443): Max retries exceeded with url #4

Open
kgolubic opened this issue Mar 20, 2023 · 3 comments
Open

[Issue] Error while running simpler_model: I will skip this one | Error: HTTPSConnectionPool(host='api.github.com', port=443): Max retries exceeded with url #4

kgolubic opened this issue Mar 20, 2023 · 3 comments

Comments

@kgolubic
Copy link

When the simpler_model gets to around 1000 stargazers, I get the time-out error.

Is there a setting to change the number of retires or to do something else so that it can be run to the end?

There are 1 unused configuration paths:
- models.dbt_project.example
2023-03-20 12:54:08 +0100 - dagster - DEBUG - __ASSET_JOB - 959fd084-49dc-4785-b9ce-4d435f8dc8e7 - stargazers_with_user_info - Completed 900 of 1212 stargazers.
2023-03-20 12:54:20 +0100 - dagster.daemon.SensorDaemon - INFO - Not checking for any runs since no sensors have been started.
2023-03-20 12:54:27 +0100 - dagster - INFO - __ASSET_JOB - 959fd084-49dc-4785-b9ce-4d435f8dc8e7 - stargazers_with_user_info - 4800 tokens left
2023-03-20 12:55:07 +0100 - dagster.builtin - WARNING - [WARNING]: Configuration paths exist in your dbt_project.yml file which do not apply to any resources.
There are 1 unused configuration paths:
- models.dbt_project.example
2023-03-20 12:55:20 +0100 - dagster.daemon.SensorDaemon - INFO - Not checking for any runs since no sensors have been started.
2023-03-20 12:55:53 +0100 - dagster - DEBUG - __ASSET_JOB - 959fd084-49dc-4785-b9ce-4d435f8dc8e7 - stargazers_with_user_info - Completed 1000 of 1212 stargazers.
2023-03-20 12:56:20 +0100 - dagster.daemon.SensorDaemon - INFO - Not checking for any runs since no sensors have been started.
2023-03-20 12:56:21 +0100 - dagster.builtin - WARNING - [WARNING]: Configuration paths exist in your dbt_project.yml file which do not apply to any resources.
There are 1 unused configuration paths:
- models.dbt_project.example
2023-03-20 12:57:20 +0100 - dagster.daemon.SensorDaemon - INFO - Not checking for any runs since no sensors have been started.
I ran into an error - I will skip this one | Error: HTTPSConnectionPool(host='api.github.com', port=443): Max retries exceeded with url: /users/HerrMuna (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7f8e1cc260a0>, 'Connection to api.github.com timed out. (connect timeout=15)'))
2023-03-20 12:57:36 +0100 - dagster.builtin - WARNING - [WARNING]: Configuration paths exist in your dbt_project.yml file which do not apply to any resources.
There are 1 unused configuration paths:
- models.dbt_project.example
I ran into an error - I will skip this one | Error: HTTPSConnectionPool(host='api.github.com', port=443): Max retries exceeded with url: /users/HamletTanyavong (Caused by SSLError(OSError(24, 'Too many open files')))
I ran into an error - I will skip this one | Error: HTTPSConnectionPool(host='api.github.com', port=443): Max retries exceeded with url: /users/rempas (Caused by SSLError(OSError(24, 'Too many open files')))
I ran into an error - I will skip this one | Error: HTTPSConnectionPool(host='api.github.com', port=443): Max retries exceeded with url: /users/timmy-time (Caused by SSLError(OSError(24, 'Too many open files')))
I ran into an error - I will skip this one | Error: HTTPSConnectionPool(host='api.github.com', port=443): Max retries exceeded with url: /users/xxidbr9 (Caused by SSLError(OSError(24, 'Too many open files')))
I ran into an error - I will skip this one | Error: HTTPSConnectionPool(host='api.github.com', port=443): Max retries exceeded with url: /users/xshadowlegendx (Caused by SSLError(OSError(24, 'Too many open files')))
I ran into an error - I will skip this one | Error: HTTPSConnectionPool(host='api.github.com', port=443): Max retries exceeded with url: /users/amerikan (Caused by SSLError(OSError(24, 'Too many open files')))
I ran into an error - I will skip this one | Error: HTTPSConnectionPool(host='api.github.com', port=443): Max retries exceeded with url: /users/vpavicic (Caused by SSLError(OSError(24, 'Too many open files')))
I ran into an error - I will skip this one | Error: HTTPSConnectionPool(host='api.github.com', port=443): Max retries exceeded with url: /users/guzmud (Caused by SSLError(OSError(24, 'Too many open files')))
I ran into an error - I will skip this one | Error: HTTPSConnectionPool(host='api.github.com', port=443): Max retries exceeded with url: /users/asampal (Caused by SSLError(OSError(24, 'Too many open files')))
I ran into an error - I will skip this one | Error: HTTPSConnectionPool(host='api.github.com', port=443): Max retries exceeded with url: /users/ogiste (Caused by SSLError(OSError(24, 'Too many open files')))
I ran into an error - I will skip this one | Error: HTTPSConnectionPool(host='api.github.com', port=443): Max retries exceeded with url: /users/brandonheath (Caused by SSLError(OSError(24, 'Too many open files')))
I ran into an error - I will skip this one | Error: HTTPSConnectionPool(host='api.github.com', port=443): Max retries exceeded with url: /users/57-Wolve (Caused by SSLError(OSError(24, 'Too many open files')))
I ran into an error - I will skip this one | Error: HTTPSConnectionPool(host='api.github.com', port=443): Max retries exceeded with url: /users/sguzman (Caused by SSLError(OSError(24, 'Too many open files')))
I ran into an error - I will skip this one | Error: HTTPSConnectionPool(host='api.github.com', port=443): Max retries exceeded with url: /users/DataBoyTX (Caused by SSLError(OSError(24, 'Too many open files')))
I ran into an error - I will skip this one | Error: HTTPSConnectionPool(host='api.github.com', port=443): Max retries exceeded with url: /users/AlpriElse (Caused by SSLError(OSError(24, 'Too many open files')))
I ran into an error - I will skip this one | Error: HTTPSConnectionPool(host='api.github.com', port=443): Max retries exceeded with url: /users/mgilbir (Caused by SSLError(OSError(24, 'Too many open files')))
2023-03-20 12:58:20 +0100 - dagster.daemon.SensorDaemon - INFO - Not checking for any runs since no sensors have been started.
I ran into an error - I will skip this one | Error: HTTPSConnectionPool(host='api.github.com', port=443): Max retries exceeded with url: /users/Krazybug (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7f8e1cb47cd0>, 'Connection to api.github.com timed out. (connect timeout=15)'))
2023-03-20 12:58:46 +0100 - dagster.builtin - WARNING - [WARNING]: Configuration paths exist in your dbt_project.yml file which do not apply to any resources.
There are 1 unused configuration paths:
- models.dbt_project.example

Then there is a large number of Max retries errors and then at one point there are new errors:

Exception while writing logger call to event log: (sqlite3.OperationalError) unable to open database file
[SQL: INSERT INTO event_logs (run_id, event, dagster_event_type, timestamp, step_key, asset_key, partition) VALUES (?, ?, ?, ?, ?, ?, ?)]
[parameters: ('959fd084-49dc-4785-b9ce-4d435f8dc8e7', '{"__class__": "EventLogEntry", "dagster_event": {"__class__": "DagsterEvent", "event_specific_data": {"__class__": "StepOutputData", "metadata_entrie ... (16019 characters truncated) ... gazers_with_user_info", "timestamp": 1679313767.2596595, "user_message": "Yielded output \\"result\\" of type \\"DataFrame\\". (Type check passed)."}', 'STEP_OUTPUT', '2023-03-20 12:02:47.259660', 'stargazers_with_user_info', None, None)]
(Background on this error at: https://sqlalche.me/e/14/e3q8)
Exception while writing logger call to event log: (sqlite3.OperationalError) unable to open database file
[SQL: INSERT INTO event_logs (run_id, event, dagster_event_type, timestamp, step_key, asset_key, partition) VALUES (?, ?, ?, ?, ?, ?, ?)]
[parameters: ('959fd084-49dc-4785-b9ce-4d435f8dc8e7', '{"__class__": "EventLogEntry", "dagster_event": {"__class__": "DagsterEvent", "event_specific_data": {"__class__": "StepFailureData", "error": {"__cl ... (7691 characters truncated) ... step_key": "stargazers_with_user_info", "timestamp": 1679313767.2749624, "user_message": "Execution of step \\"stargazers_with_user_info\\" failed."}', 'STEP_FAILURE', '2023-03-20 12:02:47.274962', 'stargazers_with_user_info', None, None)]
(Background on this error at: https://sqlalche.me/e/14/e3q8)
2023-03-20 13:02:47 +0100 - dagster - ERROR - __ASSET_JOB - 959fd084-49dc-4785-b9ce-4d435f8dc8e7 - 15841 - RUN_FAILURE - Execution of run for "__ASSET_JOB" failed. An exception was thrown during execution.

sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) unable to open database file
[SQL: INSERT INTO event_logs (run_id, event, dagster_event_type, timestamp, step_key, asset_key, partition) VALUES (?, ?, ?, ?, ?, ?, ?)]
[parameters: ('959fd084-49dc-4785-b9ce-4d435f8dc8e7', '{"__class__": "EventLogEntry", "dagster_event": {"__class__": "DagsterEvent", "event_specific_data": {"__class__": "StepFailureData", "error": {"__cl ... (7691 characters truncated) ... step_key": "stargazers_with_user_info", "timestamp": 1679313767.2749624, "user_message": "Execution of step \\"stargazers_with_user_info\\" failed."}', 'STEP_FAILURE', '2023-03-20 12:02:47.274962', 'stargazers_with_user_info', None, None)]
(Background on this error at: https://sqlalche.me/e/14/e3q8)

Stack Trace:
  File "/home/kruno/repos/fake-star-detector/venv/lib/python3.8/site-packages/dagster/_core/execution/api.py", line 990, in pipeline_execution_iterator
    for event in pipeline_context.executor.execute(pipeline_context, execution_plan):
  File "/home/kruno/repos/fake-star-detector/venv/lib/python3.8/site-packages/dagster/_core/executor/in_process.py", line 41, in execute
    yield from iter(
  File "/home/kruno/repos/fake-star-detector/venv/lib/python3.8/site-packages/dagster/_core/execution/api.py", line 1101, in __iter__
    yield from self.iterator(
  File "/home/kruno/repos/fake-star-detector/venv/lib/python3.8/site-packages/dagster/_core/execution/plan/execute_plan.py", line 114, in inner_plan_execution_iterator
    for step_event in check.generator(
  File "/home/kruno/repos/fake-star-detector/venv/lib/python3.8/site-packages/dagster/_core/execution/plan/execute_plan.py", line 358, in dagster_event_sequence_for_step
    yield step_failure_event_from_exc_info(
  File "/home/kruno/repos/fake-star-detector/venv/lib/python3.8/site-packages/dagster/_core/execution/plan/objects.py", line 124, in step_failure_event_from_exc_info
    return DagsterEvent.step_failure_event(
  File "/home/kruno/repos/fake-star-detector/venv/lib/python3.8/site-packages/dagster/_core/events/__init__.py", line 802, in step_failure_event
    return DagsterEvent.from_step(
  File "/home/kruno/repos/fake-star-detector/venv/lib/python3.8/site-packages/dagster/_core/events/__init__.py", line 413, in from_step
    log_step_event(step_context, event)
  File "/home/kruno/repos/fake-star-detector/venv/lib/python3.8/site-packages/dagster/_core/events/__init__.py", line 292, in log_step_event
    step_context.log.log_dagster_event(
  File "/home/kruno/repos/fake-star-detector/venv/lib/python3.8/site-packages/dagster/_core/log_manager.py", line 405, in log_dagster_event
    self.log(level=level, msg=msg, extra={DAGSTER_META_KEY: dagster_event})
  File "/home/kruno/repos/fake-star-detector/venv/lib/python3.8/site-packages/dagster/_core/log_manager.py", line 420, in log
    self._log(level, msg, args, **kwargs)
  File "/usr/lib/python3.8/logging/__init__.py", line 1589, in _log
    self.handle(record)
  File "/usr/lib/python3.8/logging/__init__.py", line 1599, in handle
    self.callHandlers(record)
  File "/usr/lib/python3.8/logging/__init__.py", line 1661, in callHandlers
    hdlr.handle(record)
  File "/usr/lib/python3.8/logging/__init__.py", line 954, in handle
    self.emit(record)
  File "/home/kruno/repos/fake-star-detector/venv/lib/python3.8/site-packages/dagster/_core/log_manager.py", line 286, in emit
    handler.handle(dagster_record)
  File "/usr/lib/python3.8/logging/__init__.py", line 954, in handle
    self.emit(record)
  File "/home/kruno/repos/fake-star-detector/venv/lib/python3.8/site-packages/dagster/_core/instance/__init__.py", line 187, in emit
    self._instance.handle_new_event(event)
  File "/home/kruno/repos/fake-star-detector/venv/lib/python3.8/site-packages/dagster/_core/instance/__init__.py", line 1839, in handle_new_event
    self._event_storage.store_event(event)
  File "/home/kruno/repos/fake-star-detector/venv/lib/python3.8/site-packages/dagster/_core/storage/event_log/sqlite/sqlite_event_log.py", line 243, in store_event
    conn.execute(insert_event_statement)
  File "/home/kruno/repos/fake-star-detector/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1380, in execute
    return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS)
  File "/home/kruno/repos/fake-star-detector/venv/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 334, in _execute_on_connection
    return connection._execute_clauseelement(
  File "/home/kruno/repos/fake-star-detector/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement
    ret = self._execute_context(
  File "/home/kruno/repos/fake-star-detector/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context
    self._handle_dbapi_exception(
  File "/home/kruno/repos/fake-star-detector/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2124, in _handle_dbapi_exception
    util.raise_(
  File "/home/kruno/repos/fake-star-detector/venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 211, in raise_
    raise exception
  File "/home/kruno/repos/fake-star-detector/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context
    self.dialect.do_execute(
  File "/home/kruno/repos/fake-star-detector/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
    cursor.execute(statement, parameters)

The above exception was caused by the following exception:
sqlite3.OperationalError: unable to open database file

Stack Trace:
  File "/home/kruno/repos/fake-star-detector/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context
    self.dialect.do_execute(
  File "/home/kruno/repos/fake-star-detector/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
    cursor.execute(statement, parameters)

The above exception occurred during handling of the following exception:
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) unable to open database file
[SQL: INSERT INTO event_logs (run_id, event, dagster_event_type, timestamp, step_key, asset_key, partition) VALUES (?, ?, ?, ?, ?, ?, ?)]
[parameters: ('959fd084-49dc-4785-b9ce-4d435f8dc8e7', '{"__class__": "EventLogEntry", "dagster_event": {"__class__": "DagsterEvent", "event_specific_data": {"__class__": "StepOutputData", "metadata_entrie ... (16019 characters truncated) ... gazers_with_user_info", "timestamp": 1679313767.2596595, "user_message": "Yielded output \\"result\\" of type \\"DataFrame\\". (Type check passed)."}', 'STEP_OUTPUT', '2023-03-20 12:02:47.259660', 'stargazers_with_user_info', None, None)]
(Background on this error at: https://sqlalche.me/e/14/e3q8)

Stack Trace:
  File "/home/kruno/repos/fake-star-detector/venv/lib/python3.8/site-packages/dagster/_core/execution/plan/execute_plan.py", line 269, in dagster_event_sequence_for_step
    for step_event in check.generator(step_events):
  File "/home/kruno/repos/fake-star-detector/venv/lib/python3.8/site-packages/dagster/_core/execution/plan/execute_step.py", line 384, in core_dagster_event_sequence_for_step
    for evt in _type_check_and_store_output(step_context, user_event):
  File "/home/kruno/repos/fake-star-detector/venv/lib/python3.8/site-packages/dagster/_core/execution/plan/execute_step.py", line 434, in _type_check_and_store_output
    for output_event in _type_check_output(step_context, step_output_handle, output, version):
  File "/home/kruno/repos/fake-star-detector/venv/lib/python3.8/site-packages/dagster/_core/execution/plan/execute_step.py", line 287, in _type_check_output
    yield DagsterEvent.step_output_event(
  File "/home/kruno/repos/fake-star-detector/venv/lib/python3.8/site-packages/dagster/_core/events/__init__.py", line 771, in step_output_event
    return DagsterEvent.from_step(
  File "/home/kruno/repos/fake-star-detector/venv/lib/python3.8/site-packages/dagster/_core/events/__init__.py", line 413, in from_step
    log_step_event(step_context, event)
  File "/home/kruno/repos/fake-star-detector/venv/lib/python3.8/site-packages/dagster/_core/events/__init__.py", line 292, in log_step_event
    step_context.log.log_dagster_event(
  File "/home/kruno/repos/fake-star-detector/venv/lib/python3.8/site-packages/dagster/_core/log_manager.py", line 405, in log_dagster_event
    self.log(level=level, msg=msg, extra={DAGSTER_META_KEY: dagster_event})
  File "/home/kruno/repos/fake-star-detector/venv/lib/python3.8/site-packages/dagster/_core/log_manager.py", line 420, in log
    self._log(level, msg, args, **kwargs)
  File "/usr/lib/python3.8/logging/__init__.py", line 1589, in _log
    self.handle(record)
  File "/usr/lib/python3.8/logging/__init__.py", line 1599, in handle
    self.callHandlers(record)
  File "/usr/lib/python3.8/logging/__init__.py", line 1661, in callHandlers
    hdlr.handle(record)
  File "/usr/lib/python3.8/logging/__init__.py", line 954, in handle
    self.emit(record)
  File "/home/kruno/repos/fake-star-detector/venv/lib/python3.8/site-packages/dagster/_core/log_manager.py", line 286, in emit
    handler.handle(dagster_record)
  File "/usr/lib/python3.8/logging/__init__.py", line 954, in handle
    self.emit(record)
  File "/home/kruno/repos/fake-star-detector/venv/lib/python3.8/site-packages/dagster/_core/instance/__init__.py", line 187, in emit
    self._instance.handle_new_event(event)
  File "/home/kruno/repos/fake-star-detector/venv/lib/python3.8/site-packages/dagster/_core/instance/__init__.py", line 1839, in handle_new_event
    self._event_storage.store_event(event)
  File "/home/kruno/repos/fake-star-detector/venv/lib/python3.8/site-packages/dagster/_core/storage/event_log/sqlite/sqlite_event_log.py", line 243, in store_event
    conn.execute(insert_event_statement)
  File "/home/kruno/repos/fake-star-detector/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1380, in execute
    return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS)
  File "/home/kruno/repos/fake-star-detector/venv/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 334, in _execute_on_connection
    return connection._execute_clauseelement(
  File "/home/kruno/repos/fake-star-detector/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement
    ret = self._execute_context(
  File "/home/kruno/repos/fake-star-detector/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context
    self._handle_dbapi_exception(
  File "/home/kruno/repos/fake-star-detector/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2124, in _handle_dbapi_exception
    util.raise_(
  File "/home/kruno/repos/fake-star-detector/venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 211, in raise_
    raise exception
  File "/home/kruno/repos/fake-star-detector/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context
    self.dialect.do_execute(
  File "/home/kruno/repos/fake-star-detector/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
    cursor.execute(statement, parameters)

The above exception was caused by the following exception:
sqlite3.OperationalError: unable to open database file

Stack Trace:
  File "/home/kruno/repos/fake-star-detector/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context
    self.dialect.do_execute(
  File "/home/kruno/repos/fake-star-detector/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
    cursor.execute(statement, parameters)

2023-03-20 13:03:10 +0100 - dagster.builtin - WARNING - [WARNING]: Configuration paths exist in your dbt_project.yml file which do not apply to any resources.
There are 1 unused configuration paths:
- models.dbt_project.example
2023-03-20 13:03:20 +0100 - dagster.daemon.SensorDaemon - INFO - Not checking for any runs since no sensors have been started.

image
@rachfop
Copy link

rachfop commented Mar 20, 2023

I had some success modifying this value, but then I ran into more issue.

@yuhan
Copy link
Contributor

yuhan commented Mar 20, 2023

Thanks for raising this issue. I commented a fix here: #2 (comment)

@kgolubic
Copy link
Author

I've tried to switch to Postgres. It still stalls around 1000 stars, and I get the error message Max retries exceeded with URL:

I've switched to the Docker version, and it works so I'll use it for now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@yuhan @kgolubic @rachfop