Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Running the custom kernel example - error when starting kernel from JupyterLab #57

Open
rathboma opened this issue Feb 1, 2022 · 15 comments
Open

Running the custom kernel example - error when starting kernel from JupyterLab #57

rathboma opened this issue Feb 1, 2022 · 15 comments

Comments

@rathboma
Copy link

rathboma commented Feb 1, 2022 •
edited
Loading

When going through the custom kernel example everything starts correctly in k8s, but when starting the kernel from the notebook it produced a 500 HTTP error:

image

Seems like the error is some version incompatibility with enterprise gateway. Here are the logs from the gateway:

Error: TypeError: __init__() got an unexpected keyword argument 'kernel_id'

 HTTPServerRequest(protocol='http', host='jupytergateway-elastic-with-custom-kernels.default:8888', method='POST', uri='/api/kernels', version='HTTP/1.1', remote_ip='172.17.0.1')
Traceback (most recent call last):
File "/opt/conda/lib/python3.9/site-packages/tornado/web.py", line 1704, in _execute
result = await result
File "/opt/conda/lib/python3.9/site-packages/enterprise_gateway/services/kernels/handlers.py", line 90, in post
await super(MainKernelHandler, self).post()
File "/opt/conda/lib/python3.9/site-packages/jupyter_server/services/kernels/handlers.py", line 47, in post
kernel_id = await km.start_kernel(kernel_name=model["name"], path=model.get("path"))
File "/opt/conda/lib/python3.9/site-packages/enterprise_gateway/services/kernels/remotemanager.py", line 156, in start_kernel
kernel_id = await super(RemoteMappingKernelManager, self).start_kernel(*args, **kwargs)
File "/opt/conda/lib/python3.9/site-packages/jupyter_server/services/kernels/kernelmanager.py", line 211, in start_kernel
kernel_id = await ensure_async(self.pinned_superclass.start_kernel(self, **kwargs))
File "/opt/conda/lib/python3.9/site-packages/jupyter_server/utils.py", line 189, in ensure_async
result = await obj
File "/opt/conda/lib/python3.9/site-packages/jupyter_client/multikernelmanager.py", line 194, in _async_start_kernel
await fut
File "/opt/conda/lib/python3.9/site-packages/jupyter_client/multikernelmanager.py", line 165, in _add_kernel_when_ready
await kernel_awaitable
File "/opt/conda/lib/python3.9/site-packages/jupyter_client/utils.py", line 34, in ensure_async
return await obj
File "/opt/conda/lib/python3.9/site-packages/enterprise_gateway/services/kernels/remotemanager.py", line 365, in start_kernel
await super(RemoteKernelManager, self).start_kernel(**kwargs)
File "/opt/conda/lib/python3.9/site-packages/jupyter_client/manager.py", line 362, in _async_start_kernel
raise e
File "/opt/conda/lib/python3.9/site-packages/jupyter_client/manager.py", line 351, in _async_start_kernel
await ensure_async(self._launch_kernel(kernel_cmd, **kw))
File "/opt/conda/lib/python3.9/site-packages/jupyter_client/utils.py", line 34, in ensure_async
return await obj
File "/opt/conda/lib/python3.9/site-packages/enterprise_gateway/services/kernels/remotemanager.py", line 425, in _launch_kernel
proxy = await self.process_proxy.launch_process(kernel_cmd, **kwargs)
File "/opt/conda/lib/python3.9/site-packages/enterprise_gateway/services/processproxies/kubeflow.py", line 49, in launch_process
await super(KubeflowProcessProxy, self).launch_process(kernel_cmd, **kwargs)
File "/opt/conda/lib/python3.9/site-packages/enterprise_gateway/services/processproxies/container.py", line 81, in launch_process
self.local_proc = self.launch_kernel(kernel_cmd, **kwargs)
File "/opt/conda/lib/python3.9/site-packages/enterprise_gateway/services/processproxies/processproxy.py", line 456, in launch_kernel
return launch_kernel(cmd, **kwargs)
File "/opt/conda/lib/python3.9/site-packages/jupyter_client/launcher.py", line 169, in launch_kernel
raise ex
File "/opt/conda/lib/python3.9/site-packages/jupyter_client/launcher.py", line 157, in launch_kernel
proc = Popen(cmd, **kwargs)
TypeError: __init__() got an unexpected keyword argument 'kernel_id'
[E 220201 17:06:22 web:2239] 500 POST /api/kernels (172.17.0.1) 23.14ms

This seems related to this issue: jupyter-server/enterprise_gateway#1024

Seems like it's using jupyter-client 7.1.0, which might be the problem? I'll try to confirm
@gaocegege
Copy link
Collaborator

Can you please show the image tag of the gateway and kernel?

@ycsong1212
Copy link

ycsong1212 commented Mar 10, 2022 •
edited
Loading

Hello, could I know whether you solve this problem? I meet the same issue, I check the version of jupyter_client is 7.1.0. Could changing the version of jupyter_client work? I try to deploy with enterprise-gateway alone instead of elastic-jupyter-operator, that works. I'll appreciate it if you could help.

@ycsong1212
Copy link

Can you please show the image tag of the gateway and kernel?

Hello, Mr. Gao. I meet the same question. I try the demo as quick-start but it doesn't work. The tags of my gateway and kernel are both 2.6.0. I want to know whether some version conflict exists. I have tried deploying only with enterprise-gateway and it works. I am interested in what kind of differences between your images and theirs. I am sorry I cannot access your image information and I do not obtain the answer in your code. I'll appreciate it if you could help.

@gaocegege
Copy link
Collaborator

OK I will verify it today.

@gaocegege
Copy link
Collaborator

@ycsong1212 Did you follow the guide here? https://github.com/skai-x/elastic-jupyter-operator/tree/master/docs#remote-kernels

Remote kernel or Remote kernels with custom configuration

@gaocegege
Copy link
Collaborator

Or this https://github.com/skai-x/elastic-jupyter-operator/blob/master/docs/quick-start.md

@ycsong1212
Copy link

@ycsong1212 Did you follow the guide here? https://github.com/skai-x/elastic-jupyter-operator/tree/master/docs#remote-kernels

Remote kernel or Remote kernels with custom configuration

I think I do. First I apply crds by prepare.yaml and then make deploy. I tried elastic-with-custom-kernels demo. I apply four crds under examples/elastic-with-custom-kernels directory. Then I port-forward my notebook pod. When I launch kernel, kernel error occurs. Such error is reported in gateway container.

@DianaDai
Copy link

I have same error， how can i fix it?

@yElaine
Copy link

yElaine commented Jul 7, 2022

I have same error...
image

image

gateway log error message:
kubectl logs jupytergateway-elastic-with-custom-kernels-75bf76cc5b-85tbw -f

[I 220707 10:14:18 web:2239] 200 GET /api/kernels (10.244.2.XX) 6.00ms
[D 2022-07-07 10:14:19.932 EnterpriseGatewayApp] Found kernel python3 in /opt/conda/share/jupyter/kernels
[D 2022-07-07 10:14:19.932 EnterpriseGatewayApp] Found kernel python-kubernetes in /usr/local/share/jupyter/kernels
[I 220707 10:14:19 web:2239] 200 GET /api/kernelspecs (10.244.2.XX) 6.92ms
[I 220707 10:14:29 web:2239] 200 GET /api/kernels (10.244.2.XX) 5.81ms
[D 2022-07-07 10:14:46.055 EnterpriseGatewayApp] RemoteMappingKernelManager.start_kernel: python-kubernetes, kernel_username: jovyan
[D 2022-07-07 10:14:46.061 EnterpriseGatewayApp] Instantiating kernel 'Elastic Python Kernel on Kubernetes' with process proxy: enterprise_gateway.services.processproxies.kubeflow.KubeflowProcessProxy
[D 2022-07-07 10:14:46.062 EnterpriseGatewayApp] Instantiating kernel 'Elastic Python Kernel on Kubernetes' with kernel provisioner: local-provisioner
[D 2022-07-07 10:14:46.063 EnterpriseGatewayApp] Starting kernel: ['kubeflow-launcher', '--verbose', '--RemoteProcessProxy.kernel-id', '1bd1e838-e1bb-4692-94da-d02d0006fa17', '--RemoteProcessProxy.port-range', '0..0', '--RemoteProcessProxy.response-address', '10.244.0.YY:8877', '--RemoteProcessProxy.public-key', 'MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQC2Sw9kvggrPA/2UnCVv9boPK3I9NWLiP9ILMjn+4V+T0FXo6vfNYwl04muPBt7a5ScpiVaPXVg3J9xedBeqJuWxEhY2s9+ZqFJ6UQQ5TU8n1T8elRlCJYZaOTywN2qNPLx3SYg1U5slLOiO79wGyeq/ASBTPcze4v3rBBa0UjS9wIDAQAB', '--kernel-template-name', 'jupyterkerneltemplate-elastic-with-custom-kernels', '--kernel-template-namespace', 'default']
[D 2022-07-07 10:14:46.063 EnterpriseGatewayApp] Launching kernel: 'Elastic Python Kernel on Kubernetes' with command: ['kubeflow-launcher', '--verbose', '--RemoteProcessProxy.kernel-id', '1bd1e838-e1bb-4692-94da-d02d0006fa17', '--RemoteProcessProxy.port-range', '0..0', '--RemoteProcessProxy.response-address', '10.244.0.YY:8877', '--RemoteProcessProxy.public-key', 'MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQC2Sw9kvggrPA/2UnCVv9boPK3I9NWLiP9ILMjn+4V+T0FXo6vfNYwl04muPBt7a5ScpiVaPXVg3J9xedBeqJuWxEhY2s9+ZqFJ6UQQ5TU8n1T8elRlCJYZaOTywN2qNPLx3SYg1U5slLOiO79wGyeq/ASBTPcze4v3rBBa0UjS9wIDAQAB', '--kernel-template-name', 'jupyterkerneltemplate-elastic-with-custom-kernels', '--kernel-template-namespace', 'default']
[W 2022-07-07 10:14:46.063 EnterpriseGatewayApp] Shared namespace has been configured. All kernels will reside in EG namespace: default
[D 2022-07-07 10:14:46.063 EnterpriseGatewayApp] BaseProcessProxy.launch_process() env: {'SHELL': '/bin/bash', 'KUBERNETES_SERVICE_PORT_HTTPS': '443', 'EG_MIRROR_WORKING_DIRS': 'false', 'KUBERNETES_SERVICE_PORT': '443', 'EG_NAMESPACE': 'default', 'HOSTNAME': 'jupytergateway-elastic-with-custom-kernels-75bf76cc5b-85tbw', 'LANGUAGE': 'en_US.UTF-8', 'EG_SHARED_NAMESPACE': 'true', 'EG_PORT': '8888', 'EG_LOG_LEVEL': 'DEBUG', 'EG_NAME': 'jupytergateway-elastic-with-custom-kernels', 'EG_KERNEL_WHITELIST': "'python-kubernetes'", 'NB_UID': '1000', 'JUPYTERGATEWAY_ELASTIC_WITH_CUSTOM_KERNELS_SERVICE_HOST': '10.105.28.ZZ', 'PWD': '/usr/local/bin', 'EG_CULL_IDLE_TIMEOUT': '10', 'JUPYTERGATEWAY_ELASTIC_WITH_CUSTOM_KERNELS_PORT_8888_TCP_PORT': '8888', 'EG_DEFAULT_KERNEL_NAME': 'python_kubernetes', 'EG_ENABLE_TUNNELING': 'False', 'EG_KERNEL_LAUNCH_TIMEOUT': '60', 'HOME': '/home/jovyan', 'LANG': 'en_US.UTF-8', 'KUBERNETES_PORT_443_TCP': 'tcp://10.MM.0.1:443', 'EG_LIST_KERNELS': 'True', 'EG_SSH_PORT': '2122', 'NB_GID': '100', 'KG_PORT': '8888', 'JUPYTERGATEWAY_ELASTIC_WITH_CUSTOM_KERNELS_SERVICE_PORT': '8888', 'EG_CULL_CONNECTED': 'False', 'EG_PORT_RETRIES': '0', 'KG_IP': '0.0.0.0', 'EG_PORT_RANGE': '0..0', 'EG_CULL_INTERVAL': '10', 'JUPYTERGATEWAY_ELASTIC_WITH_CUSTOM_KERNELS_SERVICE_PORT_GATEWAY': '8888', 'JUPYTERGATEWAY_ELASTIC_WITH_CUSTOM_KERNELS_PORT_8888_TCP': 'tcp://10.105.28.ZZ:8888', 'EG_IP': '0.0.0.0', 'SHLVL': '0', 'CONDA_DIR': '/opt/conda', 'JUPYTERGATEWAY_ELASTIC_WITH_CUSTOM_KERNELS_PORT_8888_TCP_ADDR': '10.105.28.ZZ', 'KUBERNETES_PORT_443_TCP_PROTO': 'tcp', 'KG_PORT_RETRIES': '0', 'KUBERNETES_PORT_443_TCP_ADDR': '10.MM.0.1', 'EG_KERNEL_IMAGE': 'ghcr.io/skai-x/jupyter-kernel-py:2.6.0', 'NB_USER': 'jovyan', 'KUBERNETES_SERVICE_HOST': '10.MM.0.1', 'LC_ALL': 'en_US.UTF-8', 'KUBERNETES_PORT': 'tcp://10.MM.0.1:443', 'KUBERNETES_PORT_443_TCP_PORT': '443', 'PATH': '/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', 'JUPYTERGATEWAY_ELASTIC_WITH_CUSTOM_KERNELS_PORT': 'tcp://10.105.28.ZZ:8888', 'EG_KERNEL_CLUSTER_ROLE': 'enterprise-gateway-controller', 'DEBIAN_FRONTEND': 'noninteractive', 'JUPYTERGATEWAY_ELASTIC_WITH_CUSTOM_KERNELS_PORT_8888_TCP_PROTO': 'tcp', 'KERNEL_LAUNCH_TIMEOUT': '40', 'KERNEL_USERNAME': 'jovyan', 'KERNEL_GATEWAY': '1', 'KERNEL_POD_NAME': 'jovyan-1bd1e838-e1bb-4692-94da-d02d0006fa17', 'KERNEL_SERVICE_ACCOUNT_NAME': 'default', 'KERNEL_NAMESPACE': 'default', 'KERNEL_IMAGE': 'ghcr.io/skai-x/jupyter-kernel-py:2.6.0', 'KERNEL_EXECUTOR_IMAGE': 'ghcr.io/skai-x/jupyter-kernel-py:2.6.0', 'KERNEL_UID': '1000', 'KERNEL_GID': '100', 'EG_MIN_PORT_RANGE_SIZE': '1000', 'EG_MAX_PORT_RANGE_RETRIES': '5', 'KERNEL_ID': '1bd1e838-e1bb-4692-94da-d02d0006fa17', 'KERNEL_LANGUAGE': 'python', 'EG_IMPERSONATION_ENABLED': 'False'}
[E 2022-07-07 10:14:46.063 EnterpriseGatewayApp] Failed to run command:
['kubeflow-launcher', '--verbose', '--RemoteProcessProxy.kernel-id', '1bd1e838-e1bb-4692-94da-d02d0006fa17', '--RemoteProcessProxy.port-range', '0..0', '--RemoteProcessProxy.response-address', '10.244.0.YY:8877', '--RemoteProcessProxy.public-key', 'MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQC2Sw9kvggrPA/2UnCVv9boPK3I9NWLiP9ILMjn+4V+T0FXo6vfNYwl04muPBt7a5ScpiVaPXVg3J9xedBeqJuWxEhY2s9+ZqFJ6UQQ5TU8n1T8elRlCJYZaOTywN2qNPLx3SYg1U5slLOiO79wGyeq/ASBTPcze4v3rBBa0UjS9wIDAQAB', '--kernel-template-name', 'jupyterkerneltemplate-elastic-with-custom-kernels', '--kernel-template-namespace', 'default']
PATH='/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin'
with kwargs:
{'kernel_id': '1bd1e838-e1bb-4692-94da-d02d0006fa17', 'stdin': -1, 'stdout': None, 'stderr': None, 'cwd': None, 'start_new_session': True}

[E 2022-07-07 10:14:46.063 EnterpriseGatewayApp] init() got an unexpected keyword argument 'kernel_id'
Traceback (most recent call last):
File "/opt/conda/lib/python3.9/site-packages/jupyter_client/manager.py", line 351, in _async_start_kernel
await ensure_async(self._launch_kernel(kernel_cmd, **kw))
File "/opt/conda/lib/python3.9/site-packages/jupyter_client/utils.py", line 34, in ensure_async
return await obj
File "/opt/conda/lib/python3.9/site-packages/enterprise_gateway/services/kernels/remotemanager.py", line 425, in _launch_kernel
proxy = await self.process_proxy.launch_process(kernel_cmd, **kwargs)
File "/opt/conda/lib/python3.9/site-packages/enterprise_gateway/services/processproxies/kubeflow.py", line 49, in launch_process
await super(KubeflowProcessProxy, self).launch_process(kernel_cmd, **kwargs)
File "/opt/conda/lib/python3.9/site-packages/enterprise_gateway/services/processproxies/container.py", line 81, in launch_process
self.local_proc = self.launch_kernel(kernel_cmd, **kwargs)
File "/opt/conda/lib/python3.9/site-packages/enterprise_gateway/services/processproxies/processproxy.py", line 456, in launch_kernel
return launch_kernel(cmd, **kwargs)
File "/opt/conda/lib/python3.9/site-packages/jupyter_client/launcher.py", line 169, in launch_kernel
raise ex
File "/opt/conda/lib/python3.9/site-packages/jupyter_client/launcher.py", line 157, in launch_kernel
proc = Popen(cmd, **kwargs)
TypeError: init() got an unexpected keyword argument 'kernel_id'
[E 220707 10:14:46 web:1789] Uncaught exception POST /api/kernels (10.244.2.XX)
HTTPServerRequest(protocol='http', host='jupytergateway-elastic-with-custom-kernels.default:8888', method='POST', uri='/api/kernels', version='HTTP/1.1', remote_ip='10.244.2.XX')
Traceback (most recent call last):
File "/opt/conda/lib/python3.9/site-packages/tornado/web.py", line 1704, in _execute
result = await result
File "/opt/conda/lib/python3.9/site-packages/enterprise_gateway/services/kernels/handlers.py", line 90, in post
await super(MainKernelHandler, self).post()
File "/opt/conda/lib/python3.9/site-packages/jupyter_server/services/kernels/handlers.py", line 47, in post
kernel_id = await km.start_kernel(kernel_name=model["name"], path=model.get("path"))
File "/opt/conda/lib/python3.9/site-packages/enterprise_gateway/services/kernels/remotemanager.py", line 156, in start_kernel
kernel_id = await super(RemoteMappingKernelManager, self).start_kernel(*args, **kwargs)
File "/opt/conda/lib/python3.9/site-packages/jupyter_server/services/kernels/kernelmanager.py", line 211, in start_kernel
kernel_id = await ensure_async(self.pinned_superclass.start_kernel(self, **kwargs))
File "/opt/conda/lib/python3.9/site-packages/jupyter_server/utils.py", line 189, in ensure_async
result = await obj
File "/opt/conda/lib/python3.9/site-packages/jupyter_client/multikernelmanager.py", line 194, in _async_start_kernel
await fut
File "/opt/conda/lib/python3.9/site-packages/jupyter_client/multikernelmanager.py", line 165, in _add_kernel_when_ready
await kernel_awaitable
File "/opt/conda/lib/python3.9/site-packages/jupyter_client/utils.py", line 34, in ensure_async
return await obj
File "/opt/conda/lib/python3.9/site-packages/enterprise_gateway/services/kernels/remotemanager.py", line 365, in start_kernel
await super(RemoteKernelManager, self).start_kernel(**kwargs)
File "/opt/conda/lib/python3.9/site-packages/jupyter_client/manager.py", line 362, in _async_start_kernel
raise e
File "/opt/conda/lib/python3.9/site-packages/jupyter_client/manager.py", line 351, in _async_start_kernel
await ensure_async(self._launch_kernel(kernel_cmd, **kw))
File "/opt/conda/lib/python3.9/site-packages/jupyter_client/utils.py", line 34, in ensure_async
return await obj
File "/opt/conda/lib/python3.9/site-packages/enterprise_gateway/services/kernels/remotemanager.py", line 425, in _launch_kernel
proxy = await self.process_proxy.launch_process(kernel_cmd, **kwargs)
File "/opt/conda/lib/python3.9/site-packages/enterprise_gateway/services/processproxies/kubeflow.py", line 49, in launch_process
await super(KubeflowProcessProxy, self).launch_process(kernel_cmd, **kwargs)
File "/opt/conda/lib/python3.9/site-packages/enterprise_gateway/services/processproxies/container.py", line 81, in launch_process
self.local_proc = self.launch_kernel(kernel_cmd, **kwargs)
File "/opt/conda/lib/python3.9/site-packages/enterprise_gateway/services/processproxies/processproxy.py", line 456, in launch_kernel
return launch_kernel(cmd, **kwargs)
File "/opt/conda/lib/python3.9/site-packages/jupyter_client/launcher.py", line 169, in launch_kernel
raise ex
File "/opt/conda/lib/python3.9/site-packages/jupyter_client/launcher.py", line 157, in launch_kernel
proc = Popen(cmd, **kwargs)
TypeError: init() got an unexpected keyword argument 'kernel_id'
[E 220707 10:14:46 web:2239] 500 POST /api/kernels (10.244.2.XX) 20.42ms
[I 220707 10:14:49 web:2239] 200 GET /api/kernels (10.244.2.XX) 5.90ms

@gaocegege
Copy link
Collaborator

I did not reproduce it before. I will take a look ASAP but not sure if I can reproduce this time.

@gaocegege
Copy link
Collaborator

TypeError: init() got an unexpected keyword argument 'kernel_id'

It looks like a version problem related to enterprise gateway.

@EugeneTorap
Copy link
Contributor

@gaocegege Let's try to fix the issue

@andyzheung
Copy link

This repo is not maintained?

@gaocegege
Copy link
Collaborator

This repo is not maintained?

The org and repository belong to Tencent Cloud. And I am not sure if there is actively maintained.

@Gamattention
Copy link

When going through the custom kernel example everything starts correctly in k8s, but when starting the kernel from the notebook it produced a 500 HTTP error:

image

Seems like the error is some version incompatibility with enterprise gateway. Here are the logs from the gateway:

Error: TypeError: __init__() got an unexpected keyword argument 'kernel_id'

 HTTPServerRequest(protocol='http', host='jupytergateway-elastic-with-custom-kernels.default:8888', method='POST', uri='/api/kernels', version='HTTP/1.1', remote_ip='172.17.0.1')
Traceback (most recent call last):
File "/opt/conda/lib/python3.9/site-packages/tornado/web.py", line 1704, in _execute
result = await result
File "/opt/conda/lib/python3.9/site-packages/enterprise_gateway/services/kernels/handlers.py", line 90, in post
await super(MainKernelHandler, self).post()
File "/opt/conda/lib/python3.9/site-packages/jupyter_server/services/kernels/handlers.py", line 47, in post
kernel_id = await km.start_kernel(kernel_name=model["name"], path=model.get("path"))
File "/opt/conda/lib/python3.9/site-packages/enterprise_gateway/services/kernels/remotemanager.py", line 156, in start_kernel
kernel_id = await super(RemoteMappingKernelManager, self).start_kernel(*args, **kwargs)
File "/opt/conda/lib/python3.9/site-packages/jupyter_server/services/kernels/kernelmanager.py", line 211, in start_kernel
kernel_id = await ensure_async(self.pinned_superclass.start_kernel(self, **kwargs))
File "/opt/conda/lib/python3.9/site-packages/jupyter_server/utils.py", line 189, in ensure_async
result = await obj
File "/opt/conda/lib/python3.9/site-packages/jupyter_client/multikernelmanager.py", line 194, in _async_start_kernel
await fut
File "/opt/conda/lib/python3.9/site-packages/jupyter_client/multikernelmanager.py", line 165, in _add_kernel_when_ready
await kernel_awaitable
File "/opt/conda/lib/python3.9/site-packages/jupyter_client/utils.py", line 34, in ensure_async
return await obj
File "/opt/conda/lib/python3.9/site-packages/enterprise_gateway/services/kernels/remotemanager.py", line 365, in start_kernel
await super(RemoteKernelManager, self).start_kernel(**kwargs)
File "/opt/conda/lib/python3.9/site-packages/jupyter_client/manager.py", line 362, in _async_start_kernel
raise e
File "/opt/conda/lib/python3.9/site-packages/jupyter_client/manager.py", line 351, in _async_start_kernel
await ensure_async(self._launch_kernel(kernel_cmd, **kw))
File "/opt/conda/lib/python3.9/site-packages/jupyter_client/utils.py", line 34, in ensure_async
return await obj
File "/opt/conda/lib/python3.9/site-packages/enterprise_gateway/services/kernels/remotemanager.py", line 425, in _launch_kernel
proxy = await self.process_proxy.launch_process(kernel_cmd, **kwargs)
File "/opt/conda/lib/python3.9/site-packages/enterprise_gateway/services/processproxies/kubeflow.py", line 49, in launch_process
await super(KubeflowProcessProxy, self).launch_process(kernel_cmd, **kwargs)
File "/opt/conda/lib/python3.9/site-packages/enterprise_gateway/services/processproxies/container.py", line 81, in launch_process
self.local_proc = self.launch_kernel(kernel_cmd, **kwargs)
File "/opt/conda/lib/python3.9/site-packages/enterprise_gateway/services/processproxies/processproxy.py", line 456, in launch_kernel
return launch_kernel(cmd, **kwargs)
File "/opt/conda/lib/python3.9/site-packages/jupyter_client/launcher.py", line 169, in launch_kernel
raise ex
File "/opt/conda/lib/python3.9/site-packages/jupyter_client/launcher.py", line 157, in launch_kernel
proc = Popen(cmd, **kwargs)
TypeError: __init__() got an unexpected keyword argument 'kernel_id'
[E 220201 17:06:22 web:2239] 500 POST /api/kernels (172.17.0.1) 23.14ms

This seems related to this issue: jupyter-server/enterprise_gateway#1024

Seems like it's using jupyter-client 7.1.0, which might be the problem? I'll try to confirm

I encountered the same issue, have you resolved it?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
8 participants
@rathboma @gaocegege @DianaDai @andyzheung @EugeneTorap @ycsong1212 @yElaine @Gamattention