Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[serving] Make startup model default devices configurable #69

Merged
merged 1 commit into from
Mar 11, 2022
Merged

[serving] Make startup model default devices configurable #69

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion serving/src/main/java/ai/djl/serving/ModelServer.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -423,7 +423,7 @@ String mapModelUrl(Path path) {
if (engine == null) {
return null;
}
return modelName + "::" + engine + ":*=" + url;
return modelName + "::" + engine + ':' + configManager.getLoadOnDevices() + '=' + url;
} catch (MalformedURLException e) {
throw new AssertionError("Invalid path: " + path, e);
} catch (IOException e) {
Expand Down
10 changes: 10 additions & 0 deletions serving/src/main/java/ai/djl/serving/util/ConfigManager.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,7 @@ public final class ConfigManager {
private static final String MAX_REQUEST_SIZE = "max_request_size";
private static final String MODEL_STORE = "model_store";
private static final String MODEL_URL_PATTERN = "model_url_pattern";
private static final String LOAD_ON_DEVICES = "load_on_devices";
private static final String PLUGIN_FOLDER = "plugin_folder";

// Configuration which are not documented or enabled through environment variables
Expand Down Expand Up @@ -266,6 +267,15 @@ public String getLoadModels() {
return prop.getProperty(LOAD_MODELS);
}

/**
* Returns the devices the default model will be loaded on at startup.
*
* @return the devices the default model will be loaded on at startup
*/
public String getLoadOnDevices() {
return prop.getProperty(LOAD_ON_DEVICES, "*");
}

/**
* Returns the CORS allowed origin setting.
*
Expand Down