Add install-runner and uninstall-runner to manage Docker Model Runner by doringeman · Pull Request #48 · docker/model-cli

doringeman · 2025-05-08T16:29:54Z

$ make install
$ docker model install-runner --help
Usage:  docker model install-runner

Install Docker Model Runner

Options:
      --gpu           Enable GPU support
      --port uint16   Docker container port for Docker Model Runner (default 12434)
$ docker model uninstall-runner --help
Usage:  docker model uninstall-runner

Uninstall Docker Model Runner

Options:
      --images   Remove model runner images
      --models   Remove model storage

To try it out:

$ export _MODEL_RUNNER_TREAT_DESKTOP_AS_MOBY=yes # if on DD
$ docker model install-runner
latest: Pulling from docker/model-runner Digest: sha256:554abef288f3ed304403a7fabd93502b4ee0a1245c2412a5ee911b074d6f4076
Status: Image is up to date for docker/model-runner:latest

Successfully pulled docker/model-runner:latest
Starting model runner container docker-model-runner...
$ docker model ls
MODEL NAME  PARAMETERS  QUANTIZATION  ARCHITECTURE  MODEL ID  CREATED  SIZE

Some future enhancements:

Add docker model update-runner to ensure Docker Model Runner is running the latest version.
Add "variant" and "version" flags to docker model install-runner.

xenoscopic

Looks good like it's mostly there, some things blocked by me, I'll try to get those to you today.

commands/install_runner.go

Signed-off-by: Dorin Geman <dorin.geman@docker.com>

Signed-off-by: Jacob Howard <jacob.howard@docker.com>

Most systems won't have NVIDIA GPUs and this behavior better aligns with what's done in Docker Desktop. Signed-off-by: Jacob Howard <jacob.howard@docker.com>

Signed-off-by: Jacob Howard <jacob.howard@docker.com>

This commit also adds some uninstall functionality. Signed-off-by: Jacob Howard <jacob.howard@docker.com>

Also add a mechanism to treat Docker Desktop as Moby for testing. Signed-off-by: Jacob Howard <jacob.howard@docker.com>

Signed-off-by: Jacob Howard <jacob.howard@docker.com>

xenoscopic · 2025-05-13T01:50:31Z

commands/compose.go

+				setenv("URL", modelRunner.URL("/engines/v1/"))
+			} else if kind == desktop.ModelRunnerEngineKindMoby || kind == desktop.ModelRunnerEngineKindCloud {
+				// TODO: Find a more robust solution in Moby-like environments.
+				setenv("URL", "http://172.17.0.1:12434/engines/v1/")


Hey @glours and @ndeloof - we're trying to figure out the best way to make the DMR + Compose integration work with the standalone Docker Model Runner containers that we're now supporting on Docker CE. Since Compose projects don't attach their service containers to the host bridge network by default, using the 172.17.0.1 address doesn't work. I'm trying to figure out a more elegant solution without requiring users to change their Compose YAML. Do you have any ideas about a good convention that we could use here?

The best idea I have at the moment is just to default to http://host.docker.internal:12434/engines/v1/ and require a host.docker.internal:host-gateway mapping for every container needing the standalone model runner.

Signed-off-by: Jacob Howard <jacob.howard@docker.com>

doringeman

Nice, thank you!
(I can't approve because it's my pull request.)

commands/uninstall-runner.go

doringeman · 2025-05-13T10:35:16Z

commands/uninstall-runner.go

+		},
+		ValidArgsFunction: completion.NoComplete,
+	}
+	c.Flags().BoolVar(&models, "models", false, "Remove model storage")


nit:

Suggested change

c.Flags().BoolVar(&models, "models", false, "Remove model storage")

c.Flags().BoolVar(&models, "models", false, "Remove Docker Model Runner storage (docker-model-runner-models volume)")

?

Clarified this one a little differently since the volume name isn't guaranteed.

Co-authored-by: Dorin-Andrei Geman <doringeman@gmail.com>

Signed-off-by: Jacob Howard <jacob.howard@docker.com>

xenoscopic reviewed May 8, 2025

View reviewed changes

commands/install_runner.go Outdated Show resolved Hide resolved

commands/install_runner.go Outdated Show resolved Hide resolved

commands/install_runner.go Outdated Show resolved Hide resolved

xenoscopic mentioned this pull request May 10, 2025

[AIE-186] Add detection of Model Runner context #50

Merged

xenoscopic reviewed May 10, 2025

View reviewed changes

commands/install_runner.go Outdated Show resolved Hide resolved

doringeman added 2 commits May 12, 2025 20:07

Add docker model install-runner

c50ddf1

Signed-off-by: Dorin Geman <dorin.geman@docker.com>

Add docker model uninstall-runner

74ed2ed

Signed-off-by: Dorin Geman <dorin.geman@docker.com>

doringeman force-pushed the install-runner branch from 6b81632 to 74ed2ed Compare May 12, 2025 17:08

xenoscopic added 13 commits May 12, 2025 11:31

commands: rename install-runner.go for consistency

e8537b9

Signed-off-by: Jacob Howard <jacob.howard@docker.com>

install: restrict platforms on which installation is supported

cf09faf

Signed-off-by: Jacob Howard <jacob.howard@docker.com>

install: use CLI to construct Docker API client

5c2ebfc

Signed-off-by: Jacob Howard <jacob.howard@docker.com>

install: make non-GPU install the default

b5f5037

Most systems won't have NVIDIA GPUs and this behavior better aligns with what's done in Docker Desktop. Signed-off-by: Jacob Howard <jacob.howard@docker.com>

install: switch to JSONMessage type for status updates

06b6444

Signed-off-by: Jacob Howard <jacob.howard@docker.com>

install: use strong typing for port argument

9ab08aa

Signed-off-by: Jacob Howard <jacob.howard@docker.com>

install: switch to final image name / tags

a3884dd

Signed-off-by: Jacob Howard <jacob.howard@docker.com>

install: extract common definitions and fix image names

da2a0ac

Signed-off-by: Jacob Howard <jacob.howard@docker.com>

install: refactor and extract install code into standalone package

d2b3074

This commit also adds some uninstall functionality. Signed-off-by: Jacob Howard <jacob.howard@docker.com>

install: auto-invoke install-runner when required

fbe96b7

Also add a mechanism to treat Docker Desktop as Moby for testing. Signed-off-by: Jacob Howard <jacob.howard@docker.com>

compose: enable support for standalone installs

288bdc4

Signed-off-by: Jacob Howard <jacob.howard@docker.com>

all: rename DMR_HOST to MODEL_RUNNER_HOST

99dc177

Signed-off-by: Jacob Howard <jacob.howard@docker.com>

nit: avoid Printf whenever possible

4f6be97

Signed-off-by: Jacob Howard <jacob.howard@docker.com>

xenoscopic approved these changes May 13, 2025

View reviewed changes

xenoscopic reviewed May 13, 2025

View reviewed changes

xenoscopic added 4 commits May 12, 2025 21:40

nit: wrap errors

fd935cc

Signed-off-by: Jacob Howard <jacob.howard@docker.com>

uninstall: simplify image removal error handling

6895c5e

Signed-off-by: Jacob Howard <jacob.howard@docker.com>

compose: fix support for new context awareness

4199c7f

Signed-off-by: Jacob Howard <jacob.howard@docker.com>

compose: fix formatting verb

de7a6af

Signed-off-by: Jacob Howard <jacob.howard@docker.com>

xenoscopic force-pushed the install-runner branch from 847f9ca to de7a6af Compare May 13, 2025 03:40

doringeman commented May 13, 2025

View reviewed changes

xenoscopic and others added 2 commits May 13, 2025 07:01

Update commands/uninstall-runner.go

b8b852e

Co-authored-by: Dorin-Andrei Geman <doringeman@gmail.com>

uninstall: use image name constants and clarify uninstall behavior

59cbdd5

Signed-off-by: Jacob Howard <jacob.howard@docker.com>

xenoscopic merged commit 769e2a3 into docker:main May 13, 2025
1 check passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Comments

Add install-runner and uninstall-runner to manage Docker Model Runner#48

Add install-runner and uninstall-runner to manage Docker Model Runner#48
xenoscopic merged 21 commits intodocker:mainfrom
doringeman:install-runner

doringeman commented May 8, 2025 •

edited by xenoscopic

Loading

Uh oh!

xenoscopic left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

xenoscopic May 13, 2025

Uh oh!

xenoscopic May 13, 2025

Uh oh!

doringeman left a comment

Uh oh!

Uh oh!

doringeman May 13, 2025

Uh oh!

xenoscopic May 13, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	c.Flags().BoolVar(&models, "models", false, "Remove model storage")
	c.Flags().BoolVar(&models, "models", false, "Remove Docker Model Runner storage (docker-model-runner-models volume)")

Comments

Conversation

doringeman commented May 8, 2025 • edited by xenoscopic Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

xenoscopic left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

xenoscopic May 13, 2025

Choose a reason for hiding this comment

Uh oh!

xenoscopic May 13, 2025

Choose a reason for hiding this comment

Uh oh!

doringeman left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

doringeman May 13, 2025

Choose a reason for hiding this comment

Uh oh!

xenoscopic May 13, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

doringeman commented May 8, 2025 •

edited by xenoscopic

Loading