Import issue when the locustfile.py contains importing self-defined class sentence

[Callie-C-Wang]

Hi,
I see some import issue when I tried to run load test with the locust command line.
Here is my folder hierachy.

root/
folder1/
pythonClass1.py
folder2/
pythonClass2.py
folder3/
loadTest.py

The pythonClass1.py and pythonClass2.py is normal python class file including Class and some functions.
The loadTest.py file is just like locustfile.py file in the official website

In my load test file loadTest.py, I hope I could reuse the code in pythonClass1 so I try to import pythonClass1 with the following import sentence:

          from ..folder1.pythonClass1 import PythonClass1

when I run the locust command in the root directory

          locust -f folder3/loadTest.py

errors occured

It throws out "Attempted relative import in non-package" and the error like below.

I used the same import sentence for my other simple tests, not load test. it works fine.
When I run my simple tests, I just use "python -m ....." then no such import error occured.
How can I solve this problem when I run "locust -f ...."?

[2016-08-25 15:03:40,180] Unknown/ERROR/stderr: Traceback (most recent call last):
[2016-08-25 15:03:40,180] Unknown/ERROR/stderr: File "C:\Python27\Scripts\locust-script.py", line 9, in
[2016-08-25 15:03:40,180] Unknown/ERROR/stderr:
[2016-08-25 15:03:40,180] Unknown/ERROR/stderr: load_entry_point('locustio==0.7.5', 'console_scripts', 'locust')()
[2016-08-25 15:03:40,180] Unknown/ERROR/stderr: File "C:\Python27\lib\site-packages\locust\main.py", line 349, in main
[2016-08-25 15:03:40,181] Unknown/ERROR/stderr:
[2016-08-25 15:03:40,181] Unknown/ERROR/stderr: docstring, locusts = load_locustfile(locustfile)
[2016-08-25 15:03:40,181] Unknown/ERROR/stderr: File "C:\Python27\lib\site-packages\locust\main.py", line 321, in load_locustfile
[2016-08-25 15:03:40,183] Unknown/ERROR/stderr:
[2016-08-25 15:03:40,183] Unknown/ERROR/stderr: imported = import(os.path.splitext(locustfile)[0])
[2016-08-25 15:03:40,183] Unknown/ERROR/stderr: File "C:\Python27\lib\site-packages\gevent\builtins.py", line 93, in import
[2016-08-25 15:03:40,183] Unknown/ERROR/stderr:
[2016-08-25 15:03:40,183] Unknown/ERROR/stderr: result = _import(_args, *_kwargs)
[2016-08-25 15:03:40,184] Unknown/ERROR/stderr: File "c:\CodeSpace\TestScripts\apitest\src\test\python\testscripts\load\ImportIssue.py", line 7, in
[2016-08-25 15:03:40,184] Unknown/ERROR/stderr:
[2016-08-25 15:03:40,184] Unknown/ERROR/stderr: from ..folder1.pythonClass1 import PythonClass1
[2016-08-25 15:03:40,184] Unknown/ERROR/stderr: File "C:\Python27\lib\site-packages\gevent\builtins.py", line 93, in import
[2016-08-25 15:03:40,184] Unknown/ERROR/stderr:
[2016-08-25 15:03:40,184] Unknown/ERROR/stderr: result = _import(_args, *_kwargs)
[2016-08-25 15:03:40,184] Unknown/ERROR/stderr: ValueError
[2016-08-25 15:03:40,184] Unknown/ERROR/stderr: :
[2016-08-25 15:03:40,184] Unknown/ERROR/stderr: Attempted relative import in non-package
[2016-08-25 15:03:40,184] Unknown/ERROR/stderr:

[v-dev]

I have what seems to be a similar issue. My import statement looks like this:

from payload.payloads import auth_with_wallet_payload

And I get:

...
[2016-08-28 14:18:29,384] threesixty/ERROR/stderr: result = _import(*args, **kwargs)
[2016-08-28 14:18:29,384] threesixty/ERROR/stderr: ImportError
[2016-08-28 14:18:29,384] threesixty/ERROR/stderr: :
[2016-08-28 14:18:29,384] threesixty/ERROR/stderr: No module named payload

[Callie-C-Wang]

I think the error is due to my misunderstanding of module concept.
The problem can be solved in this way.

1. in code level
   create a --init--.py file for each folder. (--init--.py means _ _ init _ _.py without any space between _ )
   this mean to tell python that this folder is a module.
   then put some import sentence in this --init--.py file, this means this module is telling everybody else that: I have this class in my module.

e.g. Below is the folder hierachy
root/
root/folder1/
root/folder1/pythonClass1.py
root/folder1/ --init--.py
root/folder2/
root/folder2/pythonClass2.py
root/folder2/ --init--.py
root/folder3/
root/folder3/loadTest.py

put "from pythonClass1 import CLASSA" in root/folder1/ --init--.py
put "from pythonClass2 import CLASSB" in root/folder2/ --init--.py

2)in command line level
you have to launch the locust test in some folder which is above both folder 3 and folder 2 and folder 1.
And the import sentence in loadTest.py is depending on what directory you are in when you run the locust command.

E.g. we are in root/ directory.
Then the import sentence in loadTest.py should be like
from folder1.pythonClass1 import CLASSA
from folder2.pythonClass2 import CLASSB

And you should launch the locust in the following command(still in root/ directory):
root> python -m locust.main -f folder3/loadTest.py

then it should work.

loadTest.py is a file similar like locustfile.py in the official site example.

[v-dev]

Ah ok thanks. I already had an __init__.py in my "package" directories? but didn't know I had to run using python -m locust.main -f /path/to/locustfile.py... instead of locust -f /path/to/locustfile.py... as was provided in the quickstart documentation.

[mjone219]

I have the same issue, but have not had any luck. I have no problem getting my relative imports to work when running a script off the module itself, but when trying to execute the same module with locust.io I always get the error 'Parent module '' not loaded, cannot perform relative import'. I followed Callie-C-Wang's advice and added all of my imports into my init.py files, as well verified that each folder is a python package (contains an _ init _.py file). My folder structure looks something like the following:

• com
  • example
    • loadtests
      • group
        • test.py
    • utility
      • tech
        • import.py

My thought is that it is some sort of effect from how locust.io sets the entry point for the test (test.py in the above example), that is causing it not to acknowledge modules upstream (ie. import.py). However it has no problem importing modules downstream. Any help would be greatly appreciated (currently running Python 3.4).

[V7theory]

@mjone219 I'm betting you've figured this out by now, considering it's been several months, but in case you haven't:

I was having the same issue while trying to run the locust file while my terminal was in the same dir as the locust file, but as soon as I stepped back (cd ..) out to the main path and ran the file it was able to find the modules in directories above it.

[rambangaru]

I think the error is due to my misunderstanding of module concept.
The problem can be solved in this way.

1. in code level
   create a --init--.py file for each folder. (--init--.py means _ _ init _ _.py without any space between _ )
   this mean to tell python that this folder is a module.
   then put some import sentence in this --init--.py file, this means this module is telling everybody else that: I have this class in my module.

e.g. Below is the folder hierachy
root/
root/folder1/
root/folder1/pythonClass1.py
root/folder1/ --init--.py
root/folder2/
root/folder2/pythonClass2.py
root/folder2/ --init--.py
root/folder3/
root/folder3/loadTest.py

put "from pythonClass1 import CLASSA" in root/folder1/ --init--.py
put "from pythonClass2 import CLASSB" in root/folder2/ --init--.py

2)in command line level
you have to launch the locust test in some folder which is above both folder 3 and folder 2 and folder 1.
And the import sentence in loadTest.py is depending on what directory you are in when you run the locust command.

E.g. we are in root/ directory.
Then the import sentence in loadTest.py should be like
from folder1.pythonClass1 import CLASSA
from folder2.pythonClass2 import CLASSB

And you should launch the locust in the following command(still in root/ directory):
root> python -m locust.main -f folder3/loadTest.py

then it should work.

loadTest.py is a file similar like locustfile.py in the official site example.

@Callie-C-Wang It works fine for me. Thank you. Is there any way I can assign weight to them. For Example Class A can take 80% and Class B as 20%. Like we do with @task(5)

[lokeshsaini164]

python -m locust.main -f /path/to/locustfile.py using this command i am not getting any import error.
but i am not getting output using this, its just running and coming out even i am not getting any error message also...?
could someone please help me in resolving this issue?

[heyman]

@lokeshsaini164:
We don't use Github issues as a support forum. If you believe you've found an actual bug/issue in Locust, you can open a new issue where you describe how to reproduce the issue.

For general help, use Stackoverflow or Slack. For greater chance of success when asking for help there I advice you to put some effort into describing your problem better.

[obriat]

I think the error is in the dockerfile build: WORKDIR /home/locust makes python search local module in this folder.

I bypass this problem by adding a new mount in my docker-compose file:
- ./modules:/home/locust/modules

where a folder name modules contains my local modules each in a folder with empty __init__.py file and a class file.

The correct answer should be to change the dockerfile or the documentation in order that the tests file and their modules should be mount in the working directory:

FROM python:3.8

COPY . /build
RUN cd /build && pip install . && rm -rf /build

EXPOSE 8089 5557

RUN useradd --create-home locust
USER locust
RUN mkdir -p /home/locust/tests
WORKDIR /home/locust/tests
ENTRYPOINT ["locust"]

# turn off python output buffering
ENV PYTHONUNBUFFERED=1

docker-compose.yml

version: '3'

services:
  master:
    image: locustio/locust
    ports:
     - "8089:8089"
    volumes:
      - ./:/home/locust/tests
    command: -f  /home/locust/tests/locustfile.py --master -H http://master:8089
  
  worker:
    image: locustio/locust
    volumes:
      - ./:/home/locust/tests
    command: -f  /home/locust/tests/locustfile.py --worker --master-host master

[YeungHoiChiu]

We can setting WORKDIR and import module use the absolute path.
e.g
1)Create a new docker image like follow:

FROM locustio/locust
WORKDIR /mnt/locust

2)Set workdir in docker-compose.yml

version: '3'

services:
  master:
    working_dir: /mnt/locust
    image: my-locust
    ports:
      - "8089:8089"
    volumes:
      - ./:/mnt/locust
    command: -f ./locust_test/test_case/mytest.py --master --modern-ui

  worker:
    working_dir: /mnt/locust
    image: my-locust
    volumes:
      - ./:/mnt/locust
    command: -f ./locust_test/test_case/mytest.py --worker --master-host master

• Change import code

from folder1.pythonClass1 import PythonClass1