Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

no streamparse_run in virtualenv_root #258

Closed
lcdevelop opened this issue May 14, 2016 · 8 comments
Closed

no streamparse_run in virtualenv_root #258

lcdevelop opened this issue May 14, 2016 · 8 comments
Labels
invalid

Comments

@lcdevelop
Copy link

lcdevelop commented May 14, 2016 •
edited by dan-blanchard
Loading

I installed 3.0.0.dev3, and submit wordcount, but the log said can't find

2016-05-14 09:27:35.648 o.a.s.util [ERROR] Async loop died!
java.lang.RuntimeException: Error when launching multilang subprocess

        at org.apache.storm.utils.ShellProcess.launch(ShellProcess.java:89) ~[storm-core-1.0.1.jar:1.0.1]
        at org.apache.storm.task.ShellBolt.prepare(ShellBolt.java:131) ~[storm-core-1.0.1.jar:1.0.1]
        at org.apache.storm.daemon.executor$fn__7953$fn__7966.invoke(executor.clj:792) ~[storm-core-1.0.1.jar:1.0.1]
        at org.apache.storm.util$async_loop$fn__625.invoke(util.clj:482) [storm-core-1.0.1.jar:1.0.1]
        at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?]
        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_77]
Caused by: java.io.IOException: Cannot run program "/tmp/virtualenvs3//wordcount/bin/streamparse_run" (in directory "/data/apache-storm-1.0.1/storm-local/supervisor/stormdist/wordcount-2-1463189235/resources"): error=2, 没有那个文件或目录
        at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048) ~[?:1.8.0_77]
        at org.apache.storm.utils.ShellProcess.launch(ShellProcess.java:82) ~[storm-core-1.0.1.jar:1.0.1]
        ... 5 more
Caused by: java.io.IOException: error=2, 没有那个文件或目录
        at java.lang.UNIXProcess.forkAndExec(Native Method) ~[?:1.8.0_77]
        at java.lang.UNIXProcess.<init>(UNIXProcess.java:248) ~[?:1.8.0_77]
        at java.lang.ProcessImpl.start(ProcessImpl.java:134) ~[?:1.8.0_77]
        at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029) ~[?:1.8.0_77]
        at org.apache.storm.utils.ShellProcess.launch(ShellProcess.java:82) ~[storm-core-1.0.1.jar:1.0.1]
        ... 5 more

what can I do?
@Darkless012
Copy link
Contributor

Can you describe how you installed Streamparse 3.0.0.dev3 in virtualenv?
And why it is looking at /tmp/ folder? Some OS has /tmp/ folders in RAM memory, which could be a problem.

This was referenced May 24, 2016
Merged
Closed
@dan-blanchard
Copy link
Member

@warmheartli, could you please share your config.json file?

It looks to me like something strange is going on because the path it's trying to run in the end is /tmp/virtualenvs3//wordcount/bin/streamparse_run (note the two slashes next to each other).

@tdhopper
Copy link
Contributor

tdhopper commented Jun 6, 2016 •
edited
Loading

I'm seeing something similar, I think.

config.json is:

{
    "serializer": "json",
    "topology_specs": "topologies/",
    "virtualenv_specs": "virtualenvs/",
    "envs": {
        "vagrant": {
            "user": "vagrant",
            "nimbus": "dp-local-vm0",
            "workers": ["dp-local-vm1",
                        "dp-local-vm2"
                    ],
            "log": {
                "path": "",
                "max_bytes": 1000000,
                "backup_count": 10,
                "level": "info"
            },
            "virtualenv_root": "/data/virtualenvs"
        }
    }
}

sparse submit looks ok:

16:27 $ sparse submit
[dp-local-vm1] Executing task '_create_or_update_virtualenv'
[dp-local-vm2] Executing task '_create_or_update_virtualenv'
[dp-local-vm2] virtualenv not found in /data/virtualenvs, creating one.
[dp-local-vm2] run: virtualenv /data/virtualenvs/wordcount
[dp-local-vm1] virtualenv not found in /data/virtualenvs, creating one.
[dp-local-vm1] run: virtualenv /data/virtualenvs/wordcount
[dp-local-vm2] out: New python executable in /data/virtualenvs/wordcount/bin/python
[dp-local-vm1] out: New python executable in /data/virtualenvs/wordcount/bin/python
[dp-local-vm2] out: Installing setuptools, pip, wheel...done.
[dp-local-vm2] out:

[dp-local-vm2] Uploading requirements.txt to temporary file.
[dp-local-vm2] run: mktemp /tmp/streamparse_requirements-XXXXXXXXX.txt
[dp-local-vm2] out: /tmp/streamparse_requirements-haB8FHh31.txt
[dp-local-vm2] out:

[dp-local-vm1] out: Installing setuptools, pip, wheel...done.
[dp-local-vm1] out:

[dp-local-vm1] Uploading requirements.txt to temporary file.
[dp-local-vm1] run: mktemp /tmp/streamparse_requirements-XXXXXXXXX.txt
[dp-local-vm2] put: virtualenvs/wordcount.txt -> /tmp/streamparse_requirements-haB8FHh31.txt
[dp-local-vm2] Updating virtualenv: wordcount
[dp-local-vm2] run: pip install -r /tmp/streamparse_requirements-haB8FHh31.txt --exists-action w
[dp-local-vm1] out: /tmp/streamparse_requirements-a0S7Q1V46.txt
[dp-local-vm1] out:

[dp-local-vm1] put: virtualenvs/wordcount.txt -> /tmp/streamparse_requirements-a0S7Q1V46.txt
[dp-local-vm1] Updating virtualenv: wordcount
[dp-local-vm1] run: pip install -r /tmp/streamparse_requirements-a0S7Q1V46.txt --exists-action w
[dp-local-vm2] out: Collecting streamparse (from -r /tmp/streamparse_requirements-haB8FHh31.txt (line 1))
[dp-local-vm2] out: /data/virtualenvs/wordcount/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#snimissingwarning.
[dp-local-vm2] out:   SNIMissingWarning
[dp-local-vm2] out: /data/virtualenvs/wordcount/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
[dp-local-vm2] out:   InsecurePlatformWarning
[dp-local-vm1] out: Collecting streamparse (from -r /tmp/streamparse_requirements-a0S7Q1V46.txt (line 1))
[dp-local-vm1] out: /data/virtualenvs/wordcount/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#snimissingwarning.
[dp-local-vm1] out:   SNIMissingWarning
[dp-local-vm1] out: /data/virtualenvs/wordcount/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
[dp-local-vm1] out:   InsecurePlatformWarning
[dp-local-vm2] out: Collecting contextlib2 (from streamparse->-r /tmp/streamparse_requirements-haB8FHh31.txt (line 1))
[dp-local-vm2] out:   Using cached contextlib2-0.5.3-py2.py3-none-any.whl
[dp-local-vm2] out: Collecting requests (from streamparse->-r /tmp/streamparse_requirements-haB8FHh31.txt (line 1))
[dp-local-vm1] out: Collecting contextlib2 (from streamparse->-r /tmp/streamparse_requirements-a0S7Q1V46.txt (line 1))
[dp-local-vm2] out:   Using cached requests-2.10.0-py2.py3-none-any.whl
[dp-local-vm1] out:   Using cached contextlib2-0.5.3-py2.py3-none-any.whl
[dp-local-vm1] out: Collecting requests (from streamparse->-r /tmp/streamparse_requirements-a0S7Q1V46.txt (line 1))
[dp-local-vm2] out: Collecting jinja2 (from streamparse->-r /tmp/streamparse_requirements-haB8FHh31.txt (line 1))
[dp-local-vm2] out:   Using cached Jinja2-2.8-py2.py3-none-any.whl
[dp-local-vm1] out:   Using cached requests-2.10.0-py2.py3-none-any.whl
[dp-local-vm2] out: Collecting six>=1.5 (from streamparse->-r /tmp/streamparse_requirements-haB8FHh31.txt (line 1))
[dp-local-vm1] out: Collecting jinja2 (from streamparse->-r /tmp/streamparse_requirements-a0S7Q1V46.txt (line 1))
[dp-local-vm2] out:   Using cached six-1.10.0-py2.py3-none-any.whl
[dp-local-vm2] out: Collecting invoke>=0.8 (from streamparse->-r /tmp/streamparse_requirements-haB8FHh31.txt (line 1))
[dp-local-vm1] out:   Using cached Jinja2-2.8-py2.py3-none-any.whl
[dp-local-vm1] out: Collecting six>=1.5 (from streamparse->-r /tmp/streamparse_requirements-a0S7Q1V46.txt (line 1))
[dp-local-vm2] out:   Using cached invoke-0.12.2-py2.py3-none-any.whl
[dp-local-vm1] out:   Using cached six-1.10.0-py2.py3-none-any.whl
[dp-local-vm2] out: Collecting prettytable (from streamparse->-r /tmp/streamparse_requirements-haB8FHh31.txt (line 1))
[dp-local-vm1] out: Collecting invoke>=0.8 (from streamparse->-r /tmp/streamparse_requirements-a0S7Q1V46.txt (line 1))
[dp-local-vm2] out: Collecting simplejson (from streamparse->-r /tmp/streamparse_requirements-haB8FHh31.txt (line 1))
[dp-local-vm1] out:   Using cached invoke-0.12.2-py2.py3-none-any.whl
[dp-local-vm1] out: Collecting prettytable (from streamparse->-r /tmp/streamparse_requirements-a0S7Q1V46.txt (line 1))
[dp-local-vm2] out: Collecting fabric (from streamparse->-r /tmp/streamparse_requirements-haB8FHh31.txt (line 1))
[dp-local-vm1] out: Collecting simplejson (from streamparse->-r /tmp/streamparse_requirements-a0S7Q1V46.txt (line 1))
[dp-local-vm2] out:   Using cached Fabric-1.11.1-py2-none-any.whl
[dp-local-vm2] out: Collecting MarkupSafe (from jinja2->streamparse->-r /tmp/streamparse_requirements-haB8FHh31.txt (line 1))
[dp-local-vm1] out: Collecting fabric (from streamparse->-r /tmp/streamparse_requirements-a0S7Q1V46.txt (line 1))
[dp-local-vm2] out: Collecting paramiko<2.0,>=1.10 (from fabric->streamparse->-r /tmp/streamparse_requirements-haB8FHh31.txt (line 1))
[dp-local-vm1] out:   Using cached Fabric-1.11.1-py2-none-any.whl
[dp-local-vm1] out: Collecting MarkupSafe (from jinja2->streamparse->-r /tmp/streamparse_requirements-a0S7Q1V46.txt (line 1))
[dp-local-vm1] out: Collecting paramiko<2.0,>=1.10 (from fabric->streamparse->-r /tmp/streamparse_requirements-a0S7Q1V46.txt (line 1))
[dp-local-vm2] out:   Using cached paramiko-1.17.0-py2.py3-none-any.whl
[dp-local-vm2] out: Collecting pycrypto!=2.4,>=2.1 (from paramiko<2.0,>=1.10->fabric->streamparse->-r /tmp/streamparse_requirements-haB8FHh31.txt (line 1))
[dp-local-vm1] out:   Using cached paramiko-1.17.0-py2.py3-none-any.whl
[dp-local-vm2] out: Collecting ecdsa>=0.11 (from paramiko<2.0,>=1.10->fabric->streamparse->-r /tmp/streamparse_requirements-haB8FHh31.txt (line 1))
[dp-local-vm1] out: Collecting pycrypto!=2.4,>=2.1 (from paramiko<2.0,>=1.10->fabric->streamparse->-r /tmp/streamparse_requirements-a0S7Q1V46.txt (line 1))
[dp-local-vm2] out:   Using cached ecdsa-0.13-py2.py3-none-any.whl
[dp-local-vm2] out: Installing collected packages: contextlib2, requests, MarkupSafe, jinja2, six, invoke, prettytable, simplejson, pycrypto, ecdsa, paramiko, fabric, streamparse
[dp-local-vm1] out: Collecting ecdsa>=0.11 (from paramiko<2.0,>=1.10->fabric->streamparse->-r /tmp/streamparse_requirements-a0S7Q1V46.txt (line 1))
[dp-local-vm1] out:   Using cached ecdsa-0.13-py2.py3-none-any.whl
[dp-local-vm1] out: Installing collected packages: contextlib2, requests, MarkupSafe, jinja2, six, invoke, prettytable, simplejson, pycrypto, ecdsa, paramiko, fabric, streamparse
[dp-local-vm2] out: Successfully installed MarkupSafe-0.23 contextlib2-0.5.3 ecdsa-0.13 fabric-1.11.1 invoke-0.12.2 jinja2-2.8 paramiko-1.17.0 prettytable-0.7.2 pycrypto-2.6.1 requests-2.10.0 simplejson-3.8.2 six-1.10.0 streamparse-2.1.4
[dp-local-vm1] out: Successfully installed MarkupSafe-0.23 contextlib2-0.5.3 ecdsa-0.13 fabric-1.11.1 invoke-0.12.2 jinja2-2.8 paramiko-1.17.0 prettytable-0.7.2 pycrypto-2.6.1 requests-2.10.0 simplejson-3.8.2 six-1.10.0 streamparse-2.1.4
[dp-local-vm2] out:

[dp-local-vm2] run: rm /tmp/streamparse_requirements-haB8FHh31.txt
[dp-local-vm1] out:

[dp-local-vm1] run: rm /tmp/streamparse_requirements-a0S7Q1V46.txt
Cleaning from prior builds...
Creating topology JAR...
JAR created: /Users/tdhopper/repos/distil/distil-ml-storm-test/wordcount/_build/wordcount-0.0.1-SNAPSHOT.jar
Removing _resources temporary directory...done
Deploying "wordcount" topology...
Warning: Permanently added '192.168.100.100' (ECDSA) to the list of known hosts.
ssh tunnel to Nimbus dp-local-vm0:6627 established.
Uploading topology jar /Users/tdhopper/repos/distil/distil-ml-storm-test/wordcount/_build/wordcount-0.0.1-SNAPSHOT.jar to assigned location: /var/storm/nimbus/inbox/stormjar-373770a7-bf06-45d8-be21-3d2c7e174351.jar
Uploaded 6965/6965 bytes
Routing Python logging to None.
Submitting wordcount topology to nimbus...done

However, in the log I have

java.lang.RuntimeException: Error when launching multilang subprocess

    at org.apache.storm.utils.ShellProcess.launch(ShellProcess.java:89) ~[storm-core-1.0.1.jar:1.0.1]
    at org.apache.storm.spout.ShellSpout.open(ShellSpout.java:88) ~[storm-core-1.0.1.jar:1.0.1]
    at org.apache.storm.daemon.executor$fn__7885$fn__7900.invoke(executor.clj:601) ~[storm-core-1.0.1.jar:1.0.1]
    at org.apache.storm.util$async_loop$fn__625.invoke(util.clj:482) [storm-core-1.0.1.jar:1.0.1]
    at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?]
    at java.lang.Thread.run(Thread.java:745) [?:1.8.0_91]
Caused by: java.io.IOException: Cannot run program "/data/virtualenvs/wordcount/bin/streamparse_run" (in directory "/var/storm/supervisor/stormdist/wordcount-9-1465244905/resources"): error=2, No such file or directory
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048) ~[?:1.8.0_91]
    at org.apache.storm.utils.ShellProcess.launch(ShellProcess.java:82) ~[storm-core-1.0.1.jar:1.0.1]
    ... 5 more
Caused by: java.io.IOException: error=2, No such file or directory
    at java.lang.UNIXProcess.forkAndExec(Native Method) ~[?:1.8.0_91]
    at java.lang.UNIXProcess.<init>(UNIXProcess.java:248) ~[?:1.8.0_91]
    at java.lang.ProcessImpl.start(ProcessImpl.java:134) ~[?:1.8.0_91]
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029) ~[?:1.8.0_91]
    at org.apache.storm.utils.ShellProcess.launch(ShellProcess.java:82) ~[storm-core-1.0.1.jar:1.0.1]
    ... 5 more
2016-06-06 20:39:20.495 o.a.s.util [ERROR] Halting process: ("Worker died")
java.lang.RuntimeException: ("Worker died")
    at org.apache.storm.util$exit_process_BANG_.doInvoke(util.clj:341) [storm-core-1.0.1.jar:1.0.1]
    at clojure.lang.RestFn.invoke(RestFn.java:423) [clojure-1.7.0.jar:?]
    at org.apache.storm.daemon.worker$fn__8558$fn__8559.invoke(worker.clj:765) [storm-core-1.0.1.jar:1.0.1]
    at org.apache.storm.daemon.executor$mk_executor_data$fn__7773$fn__7774.invoke(executor.clj:271) [storm-core-1.0.1.jar:1.0.1]
    at org.apache.storm.util$async_loop$fn__625.invoke(util.clj:494) [storm-core-1.0.1.jar:1.0.1]
    at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?]
    at java.lang.Thread.run(Thread.java:745) [?:1.8.0_91]

streamparse_run is nowhere to be seen:

(wordcount) vagrant@dp-local-vm1:~$ ls -lh /data/virtualenvs/wordcount/bin/
total 3.3M
-rw-rw-r-- 1 vagrant vagrant 2.1K Jun  6 20:28 activate
-rw-rw-r-- 1 vagrant vagrant 1.1K Jun  6 20:28 activate.csh
-rw-rw-r-- 1 vagrant vagrant 2.2K Jun  6 20:28 activate.fish
-rw-rw-r-- 1 vagrant vagrant 1.2K Jun  6 20:28 activate_this.py
-rwxrwxr-x 1 vagrant vagrant  255 Jun  6 20:28 easy_install
-rwxrwxr-x 1 vagrant vagrant  255 Jun  6 20:28 easy_install-2.7
-rwxrwxr-x 1 vagrant vagrant  235 Jun  6 20:28 fab
-rwxrwxr-x 1 vagrant vagrant  245 Jun  6 20:28 inv
-rwxrwxr-x 1 vagrant vagrant  245 Jun  6 20:28 invoke
-rwxrwxr-x 1 vagrant vagrant  227 Jun  6 20:28 pip
-rwxrwxr-x 1 vagrant vagrant  227 Jun  6 20:28 pip2
-rwxrwxr-x 1 vagrant vagrant  227 Jun  6 20:28 pip2.7
-rwxrwxr-x 1 vagrant vagrant 3.2M Jun  6 20:28 python
lrwxrwxrwx 1 vagrant vagrant    6 Jun  6 20:28 python2 -> python
lrwxrwxrwx 1 vagrant vagrant    6 Jun  6 20:28 python2.7 -> python
-rwxrwxr-x 1 vagrant vagrant 2.3K Jun  6 20:28 python-config
-rwxrwxr-x 1 vagrant vagrant  246 Jun  6 20:28 sparse
-rwxrwxr-x 1 vagrant vagrant  246 Jun  6 20:28 streamparse
-rwxrwxr-x 1 vagrant vagrant  234 Jun  6 20:28 wheel

@tdhopper
Copy link
Contributor

tdhopper commented Jun 6, 2016

I'm using streamparse HEAD locally. I ssh-ed into my VMs and installed it manually in the virtualenvs, and it seems to have fixed the problem, I think.

@tdhopper
Copy link
Contributor

tdhopper commented Jun 6, 2016

Solution for me is adding git+git://github.com/Parsely/streamparse.git to my virtualenv requirements file.

@dan-blanchard
Copy link
Member

Solution for me is adding git+git://github.com/Parsely/streamparse.git to my virtualenv requirements file.

Ah, really good point there. I would also bet that explicitly setting the streamparse version to 3.0.0.dev3 in your requirements would also work. By default it just puts streamparse without a version in there.

It would be nice if we could come up with a way for sparse submit to update whatever streamparse version you have in your requirements to be the version you're submitting with for compatibility.

@tdhopper
Copy link
Contributor

tdhopper commented Jun 7, 2016

@dan-blanchard: Thanks. That's what I assumed sparse submit would be doing, but it makes sense that can't happen automatically.

@dan-blanchard
Copy link
Member

Closing, since this isn't really an issue on our end.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
invalid
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@tdhopper @dan-blanchard @Darkless012 @lcdevelop