Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor repo_stats to use os.pipe #11726

Merged
merged 8 commits into from
Sep 5, 2020

Conversation

techknowlogick
Copy link
Member

@techknowlogick techknowlogick commented Jun 2, 2020

credit to @zeripath for discovering inefficiency and proposing solution.

@techknowlogick techknowlogick added pr/wip This PR is not ready for review type/refactoring Existing code has been cleaned up. There should be no new functionality. labels Jun 2, 2020
@techknowlogick techknowlogick added this to the 1.13.0 milestone Jun 2, 2020
@techknowlogick techknowlogick changed the title WIP: refactor repo_stats to use os.pipe refactor repo_stats to use os.pipe Jun 2, 2020
@techknowlogick techknowlogick removed the pr/wip This PR is not ready for review label Jun 2, 2020
@techknowlogick techknowlogick marked this pull request as ready for review June 2, 2020 04:51
@techknowlogick techknowlogick requested a review from zeripath June 2, 2020 04:51
modules/git/repo_stats.go Outdated Show resolved Hide resolved
@GiteaBot GiteaBot added the lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. label Jun 2, 2020
@zeripath
Copy link
Contributor

zeripath commented Jun 2, 2020

You've done the pipe bit correctly as far as I can see. The only thing is that you're throwing away the stderr which would be useful if there's an error so you need to keep it and merge it with the error when there's an error.

Given we have to keep the authors names and email addresses in memory I worry that there is still a potential memory problem although this is a lot better. It may be that we have to run stats by periods and then progressively update otherwise the load on initial import could be bad.

@stale
Copy link

stale bot commented Sep 5, 2020

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs during the next 2 months. Thank you for your contributions.

@stale stale bot added the issue/stale label Sep 5, 2020
@techknowlogick techknowlogick added issue/confirmed Issue has been reviewed and confirmed to be present or accepted to be implemented and removed issue/stale labels Sep 5, 2020
modules/git/command.go Outdated Show resolved Hide resolved
@GiteaBot GiteaBot added lgtm/need 1 This PR needs approval from one additional maintainer to be merged. and removed lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. labels Sep 5, 2020
Copy link
Member

@6543 6543 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

jup

@GiteaBot GiteaBot added lgtm/done This PR has enough approvals to get merged. There are no important open reservations anymore. and removed lgtm/need 1 This PR needs approval from one additional maintainer to be merged. labels Sep 5, 2020
@techknowlogick techknowlogick merged commit 9a17e28 into go-gitea:master Sep 5, 2020
@techknowlogick techknowlogick deleted the repo-stats-perf branch September 5, 2020 20:12
@go-gitea go-gitea locked and limited conversation to collaborators Nov 24, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
issue/confirmed Issue has been reviewed and confirmed to be present or accepted to be implemented lgtm/done This PR has enough approvals to get merged. There are no important open reservations anymore. type/refactoring Existing code has been cleaned up. There should be no new functionality.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants