Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SIM-BE-Improve webrequest module #357

Closed
AgustinRamiroDiaz opened this issue Aug 5, 2024 · 8 comments
Closed

SIM-BE-Improve webrequest module #357

AgustinRamiroDiaz opened this issue Aug 5, 2024 · 8 comments

Comments

@AgustinRamiroDiaz
Copy link
Contributor

We should use a different browser and image to ensure it works on most platforms.
More info: https://github.com/ultrafunkamsterdam/undetected-chromedriver

@AgustinRamiroDiaz AgustinRamiroDiaz self-assigned this Aug 5, 2024
@AgustinRamiroDiaz
Copy link
Contributor Author

@cristiam86 could you add more context on the problems we are trying to solve here?

@cristiam86
Copy link
Collaborator

sure @AgustinRamiroDiaz! The main problems that we currently have are:

  1. The docker image being used is too heavy
  2. Some platforms (some Windows versions) fail to build and execute because of all the modules and dependencies needed.
  3. The browser (chrome) used needs to be simplified to what we are doing. Bulat suggested the undetected chrome, which seems lighter and better.

@YeagerAI-Bulat
Copy link
Collaborator

  1. Maybe https://seleniumbase.io/help_docs/uc_mode/#uc-mode would be a better option

@AgustinRamiroDiaz
Copy link
Contributor Author

Extra:
After this, we wouldn't require to target a particular architecture since chromium is available in other architectures (now we depend on Chrome)
FROM --platform=linux/amd64 python:3.12.5-slim

@kp2pml30
Copy link
Collaborator

Note: here is no chrome version for linux aarch64 (which will be in docker on macos). And as far as I know selenium uses this endpoint to get compatible chrome

@YeagerAI-Bulat
Copy link
Collaborator

Can you guys test the --no-zygote option in Selenium to prevent zombie processes in webrequest container as described in this article?
I saw hundreds of zombie processes in the hosted Studio VM after someone tried running the football prediction market example contract

@kp2pml30
Copy link
Collaborator

zombie processes genvm issue yeagerai/genvm#110

unfortunately, no-zigote didn't help:

ps aux | awk '$8 == "Z" {print $0}' | wc -l

still grows

@kp2pml30
Copy link
Collaborator

Here it says that removing --no-sandbox will help, however usems are required to remove it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants