docker compose connection issues #9
Comments
|
This will block any progress in openjournals/joss-reviews#4038 Hope you can help me out. |
|
Hey! I mentioned the issue about dependencies above because I believe that performing the bootstrap checks for CrateDB will address this issue. I made the mistake of not recognizing these bootstrap checks as a more essential part of system setup. In the code block you posted, it shows the Please look into the bootstrap checks here and then try running the test script again. Edit: I should probably also add a stop condition to the test script so that the tests don't run if |
|
Running after |
I tried to run
./test/all-unix.shwithout much success.bash test/all-unix.shresults in=== STARTING ADaPT-ML AND RUNNING ALL TESTS === label-studio-web is up-to-date Starting modelling-mlflow-db ... Starting modelling-mlflow-db ... done crate-db is up-to-date Starting dp-mlflow-db ... done Starting modelling-mlflow-deploy ... Starting label-studio-dev ... done Starting modelling-mlflow-deploy ... done Starting modelling-mlflow ... done Starting dp-mlflow ... done dp-mlflow-db /entrypoint.sh mysqld Up (health: starting) 3306/tcp, 33060/tcp, 33061/tcp modelling-mlflow-db /entrypoint.sh mysqld Up (health: starting) 3306/tcp, 33060/tcp, 33061/tcp Waiting for MLflow databases (5 sec)... dp-mlflow-db /entrypoint.sh mysqld Up (health: starting) 3306/tcp, 33060/tcp, 33061/tcp modelling-mlflow-db /entrypoint.sh mysqld Up (health: starting) 3306/tcp, 33060/tcp, 33061/tcp Waiting for MLflow databases (5 sec)... dp-mlflow-db /entrypoint.sh mysqld Up (health: starting) 3306/tcp, 33060/tcp, 33061/tcp modelling-mlflow-db /entrypoint.sh mysqld Up (health: starting) 3306/tcp, 33060/tcp, 33061/tcp Waiting for MLflow databases (5 sec)... dp-mlflow-db /entrypoint.sh mysqld Up (health: starting) 3306/tcp, 33060/tcp, 33061/tcp modelling-mlflow-db /entrypoint.sh mysqld Up (health: starting) 3306/tcp, 33060/tcp, 33061/tcp Waiting for MLflow databases (5 sec)... crate-db /docker-entrypoint.sh crat ... Restarting dp-mlflow-db /entrypoint.sh mysqld Up (health: starting) 3306/tcp, 33060/tcp, 33061/tcp modelling-mlflow-db /entrypoint.sh mysqld Up (health: starting) 3306/tcp, 33060/tcp, 33061/tcp Waiting for MLflow databases (5 sec)... crate-db /docker-entrypoint.sh crat ... Restarting Waiting for MLflow databases (5 sec)... Checking for exited containers... Startup complete. Traceback (most recent call last): File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 174, in _new_conn conn = connection.create_connection( File "/usr/local/lib/python3.8/site-packages/urllib3/util/connection.py", line 73, in create_connection for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM): File "/usr/local/lib/python3.8/socket.py", line 918, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno -2] Name or service not known During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 699, in urlopen httplib_response = self._make_request( File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 394, in _make_request conn.request(method, url, **httplib_request_kw) File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 239, in request super(HTTPConnection, self).request(method, url, body=body, headers=headers) File "/usr/local/lib/python3.8/http/client.py", line 1256, in request self._send_request(method, url, body, headers, encode_chunked) File "/usr/local/lib/python3.8/http/client.py", line 1302, in _send_request self.endheaders(body, encode_chunked=encode_chunked) File "/usr/local/lib/python3.8/http/client.py", line 1251, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "/usr/local/lib/python3.8/http/client.py", line 1011, in _send_output self.send(msg) File "/usr/local/lib/python3.8/http/client.py", line 951, in send self.connect() File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 205, in connect conn = self._new_conn() File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 186, in _new_conn raise NewConnectionError( urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f29a923d340>: Failed to establish a new connection: [Errno -2] Name or service not known During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.8/site-packages/crate/client/http.py", line 473, in _request response = self.server_pool[next_server].request( File "/usr/local/lib/python3.8/site-packages/crate/client/http.py", line 154, in request return self.pool.urlopen( File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 783, in urlopen return self.urlopen( File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 783, in urlopen return self.urlopen( File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 783, in urlopen return self.urlopen( [Previous line repeated 7 more times] File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 755, in urlopen retries = retries.increment( File "/usr/local/lib/python3.8/site-packages/urllib3/util/retry.py", line 574, in increment raise MaxRetryError(_pool, url, error or ResponseError(cause)) urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='crate-db', port=4200): Max retries exceeded with url: /_sql (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f29a923d340>: Failed to establish a new connection: [Errno -2] Name or service not known')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1802, in _execute_context self.dialect.do_execute( File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 719, in do_execute cursor.execute(statement, parameters) File "/usr/local/lib/python3.8/site-packages/crate/client/cursor.py", line 53, in execute self._result = self.connection.client.sql(sql, parameters, File "/usr/local/lib/python3.8/site-packages/crate/client/http.py", line 396, in sql content = self._json_request('POST', self.path, data=data) File "/usr/local/lib/python3.8/site-packages/crate/client/http.py", line 522, in _json_request response = self._request(method, path, data=data) File "/usr/local/lib/python3.8/site-packages/crate/client/http.py", line 513, in _request self._drop_server(next_server, ex_message) File "/usr/local/lib/python3.8/site-packages/crate/client/http.py", line 581, in _drop_server raise ConnectionError( crate.client.exceptions.ConnectionError: No more Servers available, exception from last server: HTTPConnectionPool(host='crate-db', port=4200): Max retries exceeded with url: /_sql (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f29a923d340>: Failed to establish a new connection: [Errno -2] Name or service not known')) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "./ls/sample_tasks.py", line 56, in <module> main() File "./ls/sample_tasks.py", line 33, in main tasks_df = pd.read_sql(random_query, DATABASE_IP) File "/usr/local/lib/python3.8/site-packages/pandas/io/sql.py", line 628, in read_sql return pandas_sql.read_query( File "/usr/local/lib/python3.8/site-packages/pandas/io/sql.py", line 1579, in read_query result = self.execute(*args) File "/usr/local/lib/python3.8/site-packages/pandas/io/sql.py", line 1424, in execute return self.connectable.execution_options().execute(*args, **kwargs) File "<string>", line 2, in execute File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/deprecations.py", line 401, in warned return fn(*args, **kwargs) File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 3139, in execute return connection.execute(statement, *multiparams, **params) File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1274, in execute return self._exec_driver_sql( File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1578, in _exec_driver_sql ret = self._execute_context( File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1845, in _execute_context self._handle_dbapi_exception( File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2026, in _handle_dbapi_exception util.raise_( File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 207, in raise_ raise exception File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1802, in _execute_context self.dialect.do_execute( File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 719, in do_execute cursor.execute(statement, parameters) File "/usr/local/lib/python3.8/site-packages/crate/client/cursor.py", line 53, in execute self._result = self.connection.client.sql(sql, parameters, File "/usr/local/lib/python3.8/site-packages/crate/client/http.py", line 396, in sql content = self._json_request('POST', self.path, data=data) File "/usr/local/lib/python3.8/site-packages/crate/client/http.py", line 522, in _json_request response = self._request(method, path, data=data) File "/usr/local/lib/python3.8/site-packages/crate/client/http.py", line 513, in _request self._drop_server(next_server, ex_message) File "/usr/local/lib/python3.8/site-packages/crate/client/http.py", line 581, in _drop_server raise ConnectionError( sqlalchemy.exc.OperationalError: (crate.client.exceptions.ConnectionError) No more Servers available, exception from last server: HTTPConnectionPool(host='crate-db', port=4200): Max retries exceeded with url: /_sql (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f29a923d340>: Failed to establish a new connection: [Errno -2] Name or service not known')) [SQL: SELECT id, txt FROM example_data ORDER BY RANDOM() LIMIT 30;] (Background on this error at: https://sqlalche.me/e/14/e3q8) === STARTING LABEL STUDIO TESTS WITH EXAMPLE DATA === Try sampling some tasks... ERROR: Task sampling failed. === STARTING DATA PROGRAMMING TESTS WITH EXAMPLE DATA === Testing multiclass... ERROR: Experiment run failed to complete. === STARTING MODELLING TESTS WITH EXAMPLE DATA === Testing multiclass with LM test data... ERROR: Experiment run failed to complete. b40e23f0ae4cba0c0f476f8490a04d4f4c5299cb72249cef17024f3debc46528 === STARTING MODEL DEPLOYMENT TESTS WITH EXAMPLE DATA === Testing multiclass predictions... {"message":"Oops! ExampleModel did not predict successfully. Please check that the requested data points have the \n features corresponding to the features the model was trained on, and that the database is available.\n Details: (crate.client.exceptions.ConnectionError) No more Servers available, exception from last server: HTTPConnectionPool(host='crate-db', port=4200): Max retries exceeded with url: /_sql (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fd46685e730>: Failed to establish a new connection: [Errno -2] Name or service not known'))\n[SQL: \n SELECT id, txt_use FROM example_data WHERE id IN ('18', '24', '22', '19', '15');\n ]\n(Background on this error at: http://sqlalche.me/e/14/e3q8)"} ERROR: Getting predictions failed. test-deploy-network === TESTING COMPLETE ===Software
Hardware
<style type="text/css"> .first {font-weight: bold; margin-left: none; padding-right: 1em;vertical-align: top; } .second {padding-left: 1em; width: 100%; vertical-align: center; } .id {font-family: monospace;} .indented {margin-left: 2em; border-left: dotted thin #dde; padding-bottom: 1em; } .node {border: solid thin #ffcc66; padding: 1em; background: #ffffcc; } .node-unclaimed {border: dotted thin #c3c3c3; padding: 1em; background: #fafafa; color: red; } .node-disabled {border: solid thin #f55; padding: 1em; background: #fee; color: gray; } </style> <title>aaron-x390</title>The text was updated successfully, but these errors were encountered: