Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

TypeError: __init__() got an unexpected keyword argument 'kernel_id' #1024

Closed
LeeMoonCh opened this issue Jan 15, 2022 · 2 comments
Closed

TypeError: __init__() got an unexpected keyword argument 'kernel_id' #1024

LeeMoonCh opened this issue Jan 15, 2022 · 2 comments

Comments

@LeeMoonCh
Copy link
Contributor

Description

I have an CDH6.3.2 environment.When I want use Jupyter Enterprise Gateway to execute pyspark code on Yarn cluster Mode,I got a Error: TypeError: init() got an unexpected keyword argument 'kernel_id'!
CHD environment ： Master --> 192.168.159.101 slave1 --> 192.168.159.102 slave2 --> 192.168.159.103.
Jupyter Enterprise Gateway installed on Master. I started it by:jupyter enterprisegateway --ip=192.168.159.101 --port_retries=0 --debug.

Then use

jupyter notebook --gateway-url=http://192.168.159.101:8888--GatewayClient.http_user=hdfs

command on slave1.

Then I new a spark_python_yarn_cluster kernel. I got this:

[D 2022-01-15 16:46:25.420 EnterpriseGatewayApp] Instantiating kernel 'Spark - Python (YARN Cluster Mode)' with kernel provisioner: local-provisioner
 [D 2022-01-15 16:46:25.422 EnterpriseGatewayApp] Starting kernel: ['/var/lib/hadoop-hdfs/miniconda3/share/jupyter/kernels/spark_python_yarn_cluster/bin/run.sh', '--RemoteProcessProxy.kernel-id', 'a2be5e1f-0012-4a93-ba74-1e6b9aee03a8', '--RemoteProcessProxy.response-address', '192.168.159.101:46805', '--RemoteProcessProxy.port-range', '0..0', '--RemoteProcessProxy.spark-context-initialization-mode', 'lazy']
 [D 2022-01-15 16:46:25.422 EnterpriseGatewayApp] Launching kernel: 'Spark - Python (YARN Cluster Mode)' with command: ['/var/lib/hadoop-hdfs/miniconda3/share/jupyter/kernels/spark_python_yarn_cluster/bin/run.sh', '--RemoteProcessProxy.kernel-id', 'a2be5e1f-0012-4a93-ba74-1e6b9aee03a8', '--RemoteProcessProxy.response-address', '192.168.159.101:46805', '--RemoteProcessProxy.port-range', '0..0', '--RemoteProcessProxy.spark-context-initialization-mode', 'lazy']
 [D 2022-01-15 16:46:25.422 EnterpriseGatewayApp] BaseProcessProxy.launch_process() env: {'PATH': '/var/lib/hadoop-hdfs/miniconda3/bin:/var/lib/hadoop-hdfs/miniconda3/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin', 'KERNEL_LAUNCH_TIMEOUT': '40', 'KERNEL_USERNAME': 'hdfs', 'KERNEL_WORKING_DIR': '/var/lib/hadoop-hdfs', 'SPARK_HOME': '/opt/cloudera/parcels/CDH/lib/spark', 'PYSPARK_PYTHON': '/var/lib/hadoop-hdfs/miniconda3/bin/python', 'PYTHONPATH': '/var/lib/hadoop-hdfs/miniconda3/lib/python3.7/site-packages:/opt/cloudera/parcels/CDH/lib/spark/python:/opt/cloudera/parcels/CDH/lib/spark/lib/py4j-0.10.7-src.zip', 'SPARK_YARN_USER_ENV': 'PYTHONUSERBASE=/var/lib/hadoop-hdfs/miniconda3,PYTHONPATH=/var/lib/hadoop-hdfs/miniconda3/lib/python3.7/site-packages:/opt/cloudera/parcels/CDH/lib/spark/python:/opt/cloudera/parcels/CDH/lib/spark/lib/py4j-0.10.7-src.zip,PATH=/var/lib/hadoop-hdfs/miniconda3/bin:/var/lib/hadoop-hdfs/miniconda3/bin:/var/lib/hadoop-hdfs/miniconda3/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin', 'SPARK_OPTS': '--master yarn --deploy-mode cluster --name ${KERNEL_ID:-ERROR__NO__KERNEL_ID} --conf spark.yarn.submit.waitAppCompletion=false', 'LAUNCH_OPTS': '', 'KERNEL_GATEWAY': '1', 'EG_MIN_PORT_RANGE_SIZE': '1000', 'EG_MAX_PORT_RANGE_RETRIES': '5', 'KERNEL_ID': 'a2be5e1f-0012-4a93-ba74-1e6b9aee03a8', 'KERNEL_LANGUAGE': 'python', 'EG_IMPERSONATION_ENABLED': 'False'}
 [E 2022-01-15 16:46:25.422 EnterpriseGatewayApp] Failed to run command:
     ['/var/lib/hadoop-hdfs/miniconda3/share/jupyter/kernels/spark_python_yarn_cluster/bin/run.sh', '--RemoteProcessProxy.kernel-id', 'a2be5e1f-0012-4a93-ba74-1e6b9aee03a8', '--RemoteProcessProxy.response-address', '192.168.159.101:46805', '--RemoteProcessProxy.port-range', '0..0', '--RemoteProcessProxy.spark-context-initialization-mode', 'lazy']
         PATH='/var/lib/hadoop-hdfs/miniconda3/bin:/var/lib/hadoop-hdfs/miniconda3/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin'
         with kwargs:
     {'kernel_id': 'a2be5e1f-0012-4a93-ba74-1e6b9aee03a8', 'stdin': -1, 'stdout': None, 'stderr': None, 'cwd': None, 'start_new_session': True}
     
 [E 2022-01-15 16:46:25.422 EnterpriseGatewayApp] __init__() got an unexpected keyword argument 'kernel_id'
     Traceback (most recent call last):
       File "/var/lib/hadoop-hdfs/miniconda3/lib/python3.7/site-packages/jupyter_client/manager.py", line 71, in wrapper
         out = await method(self, *args, **kwargs)
       File "/var/lib/hadoop-hdfs/miniconda3/lib/python3.7/site-packages/jupyter_client/manager.py", line 380, in _async_start_kernel
         await ensure_async(self._launch_kernel(kernel_cmd, **kw))
       File "/var/lib/hadoop-hdfs/miniconda3/lib/python3.7/site-packages/jupyter_client/utils.py", line 34, in ensure_async
         return await obj
       File "/var/lib/hadoop-hdfs/miniconda3/lib/python3.7/site-packages/enterprise_gateway/services/kernels/remotemanager.py", line 422, in _launch_kernel
         proxy = await self.process_proxy.launch_process(kernel_cmd, **kwargs)
       File "/var/lib/hadoop-hdfs/miniconda3/lib/python3.7/site-packages/enterprise_gateway/services/processproxies/yarn.py", line 103, in launch_process
         self.local_proc = self.launch_kernel(kernel_cmd, **kwargs)
       File "/var/lib/hadoop-hdfs/miniconda3/lib/python3.7/site-packages/enterprise_gateway/services/processproxies/processproxy.py", line 220, in launch_kernel
         return launch_kernel(cmd, **kwargs)
       File "/var/lib/hadoop-hdfs/miniconda3/lib/python3.7/site-packages/jupyter_client/launcher.py", line 169, in launch_kernel
         raise ex
       File "/var/lib/hadoop-hdfs/miniconda3/lib/python3.7/site-packages/jupyter_client/launcher.py", line 157, in launch_kernel
         proc = Popen(cmd, **kwargs)
     TypeError: __init__() got an unexpected keyword argument 'kernel_id'
 [I 2022-01-15 16:46:25.424 EnterpriseGatewayApp] KernelSpecCache: adding/updating kernelspec: spark_python_yarn_cluster
 [D 2022-01-15 16:46:25.424 EnterpriseGatewayApp] Serving kernel resource from: /var/lib/hadoop-hdfs/miniconda3/share/jupyter/kernels/spark_python_yarn_cluster
 [I 220115 16:46:25 web:2243] 200 GET /kernelspecs/spark_python_yarn_cluster/logo-64x64.png (192.168.159.102) 61.59ms
 [E 220115 16:46:25 web:1793] Uncaught exception POST /api/kernels (192.168.159.102)
     HTTPServerRequest(protocol='http', host='192.168.159.101:8888', method='POST', uri='/api/kernels', version='HTTP/1.1', remote_ip='192.168.159.102')
     Traceback (most recent call last):
       File "/var/lib/hadoop-hdfs/miniconda3/lib/python3.7/site-packages/tornado/web.py", line 1704, in _execute
         result = await result
       File "/var/lib/hadoop-hdfs/miniconda3/lib/python3.7/site-packages/tornado/gen.py", line 769, in run
         yielded = self.gen.throw(*exc_info)  # type: ignore
       File "/var/lib/hadoop-hdfs/miniconda3/lib/python3.7/site-packages/enterprise_gateway/services/kernels/handlers.py", line 91, in post
         yield super(MainKernelHandler, self).post()
       File "/var/lib/hadoop-hdfs/miniconda3/lib/python3.7/site-packages/tornado/gen.py", line 762, in run
         value = future.result()
       File "/var/lib/hadoop-hdfs/miniconda3/lib/python3.7/site-packages/tornado/gen.py", line 769, in run
         yielded = self.gen.throw(*exc_info)  # type: ignore
       File "/var/lib/hadoop-hdfs/miniconda3/lib/python3.7/site-packages/notebook/services/kernels/handlers.py", line 51, in post
         kernel_id = yield maybe_future(km.start_kernel(kernel_name=model['name']))
       File "/var/lib/hadoop-hdfs/miniconda3/lib/python3.7/site-packages/tornado/gen.py", line 762, in run
         value = future.result()
       File "/var/lib/hadoop-hdfs/miniconda3/lib/python3.7/site-packages/enterprise_gateway/services/kernels/remotemanager.py", line 156, in start_kernel
         kernel_id = await super(RemoteMappingKernelManager, self).start_kernel(*args, **kwargs)
       File "/var/lib/hadoop-hdfs/miniconda3/lib/python3.7/site-packages/notebook/services/kernels/kernelmanager.py", line 176, in start_kernel
         kernel_id = await maybe_future(self.pinned_superclass.start_kernel(self, **kwargs))
       File "/var/lib/hadoop-hdfs/miniconda3/lib/python3.7/site-packages/jupyter_client/multikernelmanager.py", line 216, in _async_start_kernel
         await fut
       File "/var/lib/hadoop-hdfs/miniconda3/lib/python3.7/site-packages/jupyter_client/multikernelmanager.py", line 170, in _add_kernel_when_ready
         await kernel_awaitable
       File "/var/lib/hadoop-hdfs/miniconda3/lib/python3.7/site-packages/jupyter_client/utils.py", line 34, in ensure_async
         return await obj
       File "/var/lib/hadoop-hdfs/miniconda3/lib/python3.7/site-packages/enterprise_gateway/services/kernels/remotemanager.py", line 364, in start_kernel
         await super(RemoteKernelManager, self).start_kernel(**kwargs)
       File "/var/lib/hadoop-hdfs/miniconda3/lib/python3.7/site-packages/jupyter_client/manager.py", line 79, in wrapper
         raise e
       File "/var/lib/hadoop-hdfs/miniconda3/lib/python3.7/site-packages/jupyter_client/manager.py", line 71, in wrapper
         out = await method(self, *args, **kwargs)
       File "/var/lib/hadoop-hdfs/miniconda3/lib/python3.7/site-packages/jupyter_client/manager.py", line 380, in _async_start_kernel
         await ensure_async(self._launch_kernel(kernel_cmd, **kw))
       File "/var/lib/hadoop-hdfs/miniconda3/lib/python3.7/site-packages/jupyter_client/utils.py", line 34, in ensure_async
         return await obj
       File "/var/lib/hadoop-hdfs/miniconda3/lib/python3.7/site-packages/enterprise_gateway/services/kernels/remotemanager.py", line 422, in _launch_kernel
         proxy = await self.process_proxy.launch_process(kernel_cmd, **kwargs)
       File "/var/lib/hadoop-hdfs/miniconda3/lib/python3.7/site-packages/enterprise_gateway/services/processproxies/yarn.py", line 103, in launch_process
         self.local_proc = self.launch_kernel(kernel_cmd, **kwargs)
       File "/var/lib/hadoop-hdfs/miniconda3/lib/python3.7/site-packages/enterprise_gateway/services/processproxies/processproxy.py", line 220, in launch_kernel
         return launch_kernel(cmd, **kwargs)
       File "/var/lib/hadoop-hdfs/miniconda3/lib/python3.7/site-packages/jupyter_client/launcher.py", line 169, in launch_kernel
         raise ex
       File "/var/lib/hadoop-hdfs/miniconda3/lib/python3.7/site-packages/jupyter_client/launcher.py", line 157, in launch_kernel
         proc = Popen(cmd, **kwargs)
     TypeError: __init__() got an unexpected keyword argument 'kernel_id'

some config:

export SPARK_HOME=/opt/cloudera/parcels/CDH/lib/spark
export HADOOP_CONF_DIR=/etc/hadoop/conf

My kernel.json:

{
    "language":"python",
    "display_name":"Spark - Python (YARN Cluster Mode)",
    "metadata":{
        "process_proxy":{
            "class_name":"enterprise_gateway.services.processproxies.yarn.YarnClusterProcessProxy"
        }
    },
    "env":{
        "SPARK_HOME":"/opt/cloudera/parcels/CDH/lib/spark",
        "PYSPARK_PYTHON":"/var/lib/hadoop-hdfs/miniconda3/bin/python",
        "PYTHONPATH":"/var/lib/hadoop-hdfs/miniconda3/lib/python3.7/site-packages:/opt/cloudera/parcels/CDH/lib/spark/python:/opt/cloudera/parcels/CDH/lib/spark/lib/py4j-0.10.7-src.zip",
        "SPARK_YARN_USER_ENV":"PYTHONUSERBASE=/var/lib/hadoop-hdfs/miniconda3,PYTHONPATH=/var/lib/hadoop-hdfs/miniconda3/lib/python3.7/site-packages:/opt/cloudera/parcels/CDH/lib/spark/python:/opt/cloudera/parcels/CDH/lib/spark/lib/py4j-0.10.7-src.zip,PATH=/var/lib/hadoop-hdfs/miniconda3/bin:$PATH",
        "SPARK_OPTS":"--master yarn --deploy-mode cluster --name ${KERNEL_ID:-ERROR__NO__KERNEL_ID} --conf spark.yarn.submit.waitAppCompletion=false",
        "LAUNCH_OPTS":""
    },
    "argv":[
        "/var/lib/hadoop-hdfs/miniconda3/share/jupyter/kernels/spark_python_yarn_cluster/bin/run.sh",
        "--RemoteProcessProxy.kernel-id",
        "{kernel_id}",
        "--RemoteProcessProxy.response-address",
        "{response_address}",
        "--RemoteProcessProxy.port-range",
        "{port_range}",
        "--RemoteProcessProxy.spark-context-initialization-mode",
        "lazy"
    ]
}

Environment

  • Enterprise Gateway Version 2.5.0
  • Platform: CDH6.3.2
  • Others ：jupyter_client 7.1.1 ,jupyter_server 1.13.3,jupyterlab 3.2.8

Do has some problems?
@LeeMoonCh
Copy link
Contributor Author

I use jupyter_client 6.1.12 fix this problem!

@kevin-bates
Copy link
Member

Hi @LeeMoonCh - yes, that was the correct adjustment. EG doesn't support jupyter_client 7 at this time. We patched the 2.5.1 release to cap jupyter_client < 7, so I would recommend moving to the current release (2.5.2). Thank you.

@rathboma rathboma mentioned this issue Feb 1, 2022
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@kevin-bates @LeeMoonCh