Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug]: ollama tests are flakey #2965

Closed
stevenh opened this issue Feb 3, 2025 · 2 comments · Fixed by #2966
Closed

[Bug]: ollama tests are flakey #2965

stevenh opened this issue Feb 3, 2025 · 2 comments · Fixed by #2966
Labels
bug An issue with the library

Comments

@stevenh
Copy link
Contributor

stevenh commented Feb 3, 2025

Testcontainers version

0.35.next

Using the latest Testcontainers version?

Yes

Host OS

Ubuntu 24.04.1

Host arch

AMD64

Go version

1.23.5

Docker version

N/A

Docker info

N/A

What happened?

CI test randomly failed

Relevant log output

FAIL TestOllama/Pull_and_Run_Model (4.45s)
      ollama_test.go:47: 
          	Error Trace:	/home/runner/work/testcontainers-go/testcontainers-go/modules/ollama/ollama_test.go:47
          	Error:      	Received unexpected error:
          	            	container exec create: Error response from daemon: container 5ec3858c0074fe66a2912ef3e889ba0a87313c617786b507d975a685f649980e is not running
          	Test:       	TestOllama/Pull_and_Run_Model
      --- FAIL: TestOllama/Pull_and_Run_Model (4.45s)
  
FAIL TestOllama/Commit_to_image_including_model (7.23s)
  2025/02/03 15:43:56 🐳 Creating container for image ollama/ollama:0.1.25-a8f2
  2025/02/03 15:43:56 ✅ Container created: 0d8aba62e943
  2025/02/03 15:43:56 🐳 Starting container: 0d8aba62e943
  2025/02/03 15:43:56 ✅ Container started: 0d8aba62e943
  2025/02/03 15:43:56 ⏳ Waiting for container id 0d8aba62e943 image: ollama/ollama:0.1.25-a8f2. Waiting for: &{Port:11434/tcp timeout:0xc000540128 PollInterval:100ms skipInternalCheck:false}
  2025/02/03 15:43:56 🔔 Container is ready: 0d8aba62e943
      ollama_test.go:70: 
          	Error Trace:	/home/runner/work/testcontainers-go/testcontainers-go/modules/ollama/ollama_test.go:93
          	            				/home/runner/work/testcontainers-go/testcontainers-go/modules/ollama/ollama_test.go:70
          	Error:      	"{\"models\":[]}" does not contain "all-minilm"
          	Test:       	TestOllama/Commit_to_image_including_model
  2025/02/03 15:43:59 🐳 Stopping container: 0d8aba62e943
  2025/02/03 15:44:00 ✅ Container stopped: 0d8aba62e943
  2025/02/03 15:44:00 🐳 Terminating container: 0d8aba62e943
  2025/02/03 15:44:00 🚫 Container terminated: 0d8aba62e943
  2025/02/03 15:44:00 🐳 Stopping container: 5ec3858c0074
  2025/02/03 15:44:00 ✅ Container stopped: 5ec3858c0074
  2025/02/03 15:44:00 🐳 Terminating container: 5ec3858c0074
  2025/02/03 15:44:00 🚫 Container terminated: 5ec3858c0074
      --- FAIL: TestOllama/Commit_to_image_including_model (7.23s)

----


FAIL TestOllama/Commit_to_image_including_model (re-run 5) (5.50s)
  2025/02/03 15:44:47 🐳 Creating container for image ollama/ollama:0.1.25-349b
  2025/02/03 15:44:47 ✅ Container created: 6891c9c55757
  2025/02/03 15:44:47 🐳 Starting container: 6891c9c55757
  2025/02/03 15:44:47 ✅ Container started: 6891c9c55757
  2025/02/03 15:44:47 ⏳ Waiting for container id 6891c9c55757 image: ollama/ollama:0.1.25-349b. Waiting for: &{Port:11434/tcp timeout:0xc00042c878 PollInterval:100ms skipInternalCheck:false}
  2025/02/03 15:44:47 🔔 Container is ready: 6891c9c55757
      ollama_test.go:70: 
          	Error Trace:	/home/runner/work/testcontainers-go/testcontainers-go/modules/ollama/ollama_test.go:93
          	            				/home/runner/work/testcontainers-go/testcontainers-go/modules/ollama/ollama_test.go:70
          	Error:      	"{\"models\":[]}" does not contain "all-minilm"
          	Test:       	TestOllama/Commit_to_image_including_model
  2025/02/03 15:44:50 🐳 Stopping container: 6891c9c55757
  2025/02/03 15:44:51 ✅ Container stopped: 6891c9c55757
  2025/02/03 15:44:51 🐳 Terminating container: 6891c9c55757
  2025/02/03 15:44:51 🚫 Container terminated: 6891c9c55757
  2025/02/03 15:44:51 🐳 Stopping container: b4a9297b3d1c
  2025/02/03 15:44:51 ✅ Container stopped: b4a9297b3d1c
  2025/02/03 15:44:51 🐳 Terminating container: b4a9297b3d1c
  2025/02/03 15:44:51 🚫 Container terminated: b4a9297b3d1c
      --- FAIL: TestOllama/Commit_to_image_including_model (5.50s)
  
FAIL TestOllama (re-run 5) (6.38s)
  2025/02/03 15:44:45 github.com/testcontainers/testcontainers-go - Connected to docker: 
    Server Version: 26.1.3
    API Version: 1.45
    Operating System: Ubuntu 24.04.1 LTS
    Total Memory: 15990 MB
    Testcontainers for Go Version: v0.36.0
    Resolved Docker Host: unix:///var/run/docker.sock
    Resolved Docker Socket Path: /var/run/docker.sock
    Test SessionID: ec31dd49c098f13e5968d5b708c8144dc8f94eab36a45f2a25cc956edddbfae9
    Test ProcessID: e29682e0-3b57-4862-8d2b-daaf7e309a14
  2025/02/03 15:44:45 🐳 Creating container for image ollama/ollama:0.1.25
  2025/02/03 15:44:45 🐳 Creating container for image testcontainers/ryuk:0.11.0
  2025/02/03 15:44:45 ✅ Container created: 78776de763ff
  2025/02/03 15:44:45 🐳 Starting container: 78776de763ff
  2025/02/03 15:44:45 ✅ Container started: 78776de763ff
  2025/02/03 15:44:45 ⏳ Waiting for container id 78776de763ff image: testcontainers/ryuk:0.11.0. Waiting for: &{Port:8080/tcp timeout:<nil> PollInterval:100ms skipInternalCheck:false}
  2025/02/03 15:44:45 🔔 Container is ready: 78776de763ff
  2025/02/03 15:44:45 ✅ Container created: b4a9297b3d1c
  2025/02/03 15:44:45 🐳 Starting container: b4a9297b3d1c
  2025/02/03 15:44:45 ✅ Container started: b4a9297b3d1c
  2025/02/03 15:44:45 ⏳ Waiting for container id b4a9297b3d1c image: ollama/ollama:0.1.25. Waiting for: &{Port:11434/tcp timeout:0xc0004209f8 PollInterval:100ms skipInternalCheck:false}
  2025/02/03 15:44:45 🔔 Container is ready: b4a9297b3d1c
  
  FAIL Package . (6.425s) (coverage: 11.1% of statements)

Additional information

See PR failure run: https://github.com/testcontainers/testcontainers-go/actions/runs/13114348144/job/36589516824?pr=2962
@stevenh stevenh added the bug An issue with the library label Feb 3, 2025
@stevenh stevenh changed the title [Bug]: ollama tests are flaket [Bug]: ollama tests are flakey Feb 4, 2025
@stevenh
Copy link
Contributor Author

stevenh commented Feb 4, 2025

Another set of failures here: https://github.com/testcontainers/testcontainers-go/actions/runs/13132814259/job/36642322836?pr=2964

This was referenced Feb 4, 2025
Merged
Merged
@mdelapenya
Copy link
Member

Indeed. I'm taking a look at this, as it could be caused by the old version of Ollama that we use as default. It's weird that it started a few days ago, and consistently causing flakiness.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug An issue with the library
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

chore(ollama): bump default version to 0.5.7 mdelapenya/testcontainers-go
2 participants
@stevenh @mdelapenya