Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

server: support reverse proxy setup with custom route #6325

Merged
merged 1 commit into from
Mar 27, 2024

Conversation

EZForever
Copy link
Contributor

As of b2536, the server's builtin UI uses absolute routes for API and static files, which works fine, but will break if the server is behind a reverse proxy (e.g. nginx) under custom route.

# This will work, given that server is running at 127.0.0.1:8080
location / {
    proxy_pass http://127.0.0.1:8080/;
    # ...
}

# This will not work; the interface does not show up
location /llama/ {
    proxy_pass http://127.0.0.1:8080/;
    # ...
}

This PR modifies UI code to use relative routes instead. API URL is also made configurable in chat.js, allowing it to be used independently.

server: public: support custom `api_url`, default to relative base path
@phymbert
Copy link
Collaborator

Thanks for looking into this. This will work but this is annoying we need to rebuild cpp static files in binary format each time, unable to identify changes you have done there, so I cannot approve alone.

@phymbert phymbert requested a review from ngxson March 26, 2024 13:51
@ngxson
Copy link
Collaborator

ngxson commented Mar 26, 2024

LGTM, let's wait to see if the CI pass then we can merge.

@EZForever
Copy link
Contributor Author

These failed tests don't make sense; I don't think a UI code change could cause the Metal backend to fail or make a Nix build last for 6 hours.

@phymbert
Copy link
Collaborator

These failed tests don't make sense; I don't think a UI code change could cause the Metal backend to fail or make a Nix build last for 6 hours.

Yes, we are facing a lack of resources on the macOS github runners randomly.

@phymbert phymbert merged commit 0642b22 into ggerganov:master Mar 27, 2024
54 of 56 checks passed
hodlen pushed a commit to hodlen/llama.cpp that referenced this pull request Apr 1, 2024
server: public: support custom `api_url`, default to relative base path
hodlen pushed a commit to hodlen/llama.cpp that referenced this pull request Apr 3, 2024
server: public: support custom `api_url`, default to relative base path
tybalex pushed a commit to rubra-ai/tools.cpp that referenced this pull request Apr 17, 2024
server: public: support custom `api_url`, default to relative base path
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants