Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow WAL inter-flush delay to be configurable #3217

Merged
merged 2 commits into from
Jul 2, 2015
Merged

Conversation

otoolep
Copy link
Contributor

@otoolep otoolep commented Jul 2, 2015

This increases the inter-flush delay to a default of 2 seconds, and also makes it configurable.

#3215

###

[data]
dir = "/var/opt/influxdb/data"
MaxWALSize = 104857600
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

need comments for each of the following lines describing what they are and what the value is

@otoolep otoolep force-pushed the config_timeouts branch 2 times, most recently from b20343e to 8c51ebf Compare July 2, 2015 14:39
@otoolep
Copy link
Contributor Author

otoolep commented Jul 2, 2015

Will cherry-pick to 0.9.1 after merge to master.

@dgnorton
Copy link
Contributor

dgnorton commented Jul 2, 2015

+1

2 similar comments
@toddboom
Copy link
Contributor

toddboom commented Jul 2, 2015

+1

@pauldix
Copy link
Member

pauldix commented Jul 2, 2015

+1

otoolep added a commit that referenced this pull request Jul 2, 2015
Allow WAL inter-flush delay to be configurable
@otoolep otoolep merged commit b2b69c6 into master Jul 2, 2015
@otoolep otoolep deleted the config_timeouts branch July 2, 2015 14:48
@ecables
Copy link

ecables commented Jul 2, 2015

Is it possible to also configure the WAL flush interval timer? What if a WAL flush cannot complete in <10 minutes? Can it default to 10 minutes, and be configurable to accommodate less capable hardware, without falling behind?

@pauldix
Copy link
Member

pauldix commented Jul 2, 2015

@ecables: It's configurable: https://github.com/influxdb/influxdb/blob/master/etc/config.sample.toml#L40

The 10 minutes is more about making sure that shards that are cold for writes eventually get flushed. It's also about making sure that if you have series that have a 10s sampling interval, enough points are grouped into each series to make the flush efficient.

It doesn't improve performance at all if you're writing to many series and you flush the wall so frequently that when it happens you only have a single point per series.

@ecables
Copy link

ecables commented Jul 2, 2015

Perfect; thanks for the explanation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants