Documentation #11
Documentation #11
Conversation
dgbaenar
commented
Sep 19, 2024
dgbaenar
commented
- Adding documentation for evaluate retrieval and generation
- Readme updated
- Docker compose updated and working
…ose file working with the README updated
|
|
||
| RUN pip install --no-cache-dir -r requirements.txt | ||
|
|
||
| EXPOSE 8000 | ||
|
|
||
| CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"] |
There was a problem hiding this comment.
Why is this being removed?
There was a problem hiding this comment.
I placed it in the docker compose so that the run was not executed when the container is built. Now the command is run after all the containers are up.
app/data/resources_summarized.csv
Outdated
There was a problem hiding this comment.
I don't think this file should be commited to git
There was a problem hiding this comment.
File removed from the PR
docker-compose.yml
Outdated
| ports: | ||
| - "8000:8000" | ||
| volumes: | ||
| - ./data:/app/data | ||
| - ./data:/app/app/data |
There was a problem hiding this comment.
Because of the absolute imports I use during development and run uvicorn from the root of the project. Since the project is inside app, and the uvicorn command runs inside the workdir I specify in the dockerfile, I have to include the app folder for the uvicorn to work app.main:app --host 0.0.0.0.0 --port 8000 --reload
There was a problem hiding this comment.
But you could set the workdir to / in the Dockerfile before running uvicorn, so that it is run the same way as during development
There was a problem hiding this comment.
Fixed and removed the double app folder
| -t 10 | ||
| --host 0.0.0.0 | ||
| --port 8080 | ||
| -m models/Meta-Llama-3.1-8B-Instruct-F16-Q5_K_M.gguf -n 450 -c 2048 -t 10 --host 0.0.0.0 --port 8080 |
There was a problem hiding this comment.
-t 10 should be documented, and I am not sure it is really beneficial without parallel execution. Have you tested if there is a latency difference with t=4 for example?
There was a problem hiding this comment.
I added the -t explanation to documentation in the main README.md. Also specified that the parallel parameter -np is optional and also added the explanation of how to use it
