Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Library uses memory directly proportional to how many blob files exist in current container #125

Open
andrew-funderburgh opened this issue Oct 10, 2024 · 2 comments

Comments

@andrew-funderburgh
Copy link

Noticed this while logging to a huge container. My application was starting to consume around 500 MB for a simple 'hello world' app. If you log to a container with many files, (in my case a couple of million blobs), then the library ends up holding in memory a bunch of BlobItem and BlobItemProperty objects pointing to what looks like the entire container.

I can open a PR to address this but I need a little bit to finish some current work, I'm opening this as a reminder to myself and a note to others in case they run into the same thing.

@chriswill
Copy link
Owner

Not to belittle the issue, but why would you log into a huge container?

@andrew-funderburgh
Copy link
Author

Not to belittle the issue, but why would you log into a huge container?

Azure's storage account docs don't really have any guidance on whether you should limit how many blobs are in each container or how many containers you could have so it seems equally valid to have 1 container with a million blob entries or 1 million containers with 1 blob entry each. They both allow "unlimited" of each. So it seems unexpected to me that this library would behave differently in both situations.

That doesn't really answer your question... so I can't really say other maybe, "because we can?"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants