Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve docker compose setup #110

Merged
merged 4 commits into from
Jul 15, 2024
Merged

Improve docker compose setup #110

merged 4 commits into from
Jul 15, 2024

Conversation

MJYoung114
Copy link
Contributor

No description provided.

@MJYoung114 MJYoung114 requested a review from moose-code May 10, 2024 12:47
@@ -1,6 +1,7 @@
.PHONY: start
start:
docker compose up --build -d
docker stats envio-indexer
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can see the cpu usage, memory usage in the terminal

Comment on lines +71 to 73
MAX_QUEUE_SIZE: 50_000
MAX_BATCH_SIZE: 10_000
# these variables can be skips if hasura is not necessary
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reduce the batch size and queue size to limit processing. Smaller sizes here reduce indexing speed

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MJYoung114 do you think you can leave a comment on how these numbers scale against memory/cpu? It would help us benchmark requirements on the paper prior to a deployment 🙏

}


check_hasura_health
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this require hasura to be up? We don't have it enabled and likely prefer to not have it running. Should we instead consider something like psql $POSTGRES_URI -c "select max(log_index) > 0 from event_sync_state"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @stas I don't it's necessary for the postgres! I've added some new docker files here regarding hydra where I've removed the hasura and this check is no longer needed!

Copy link
Contributor

@moose-code moose-code left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@moose-code moose-code merged commit b04a4d6 into main Jul 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants