Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update raw queries with a LIMIT to read in only what they need #2030

Closed
pauldix opened this issue Mar 20, 2015 · 0 comments
Closed

Update raw queries with a LIMIT to read in only what they need #2030

pauldix opened this issue Mar 20, 2015 · 0 comments
Assignees
@pauldix
Milestone
0.9.0

Comments

@pauldix
Copy link
Member

pauldix commented Mar 20, 2015

The underlying iterators in tx.go should read and yield data points only up to the limit if the query is a raw data query (one without an aggregate).

The reducer will then have to take the output of the mappers, combine together, sort and then put the limit. So each mapper will return up to what the limit is (assuming there is multiple mappers in a given time interval).

If there is only one mapper per time interval, they should be run sequentially until the limit is hit.
@pauldix pauldix added this to the 0.9.0 milestone Mar 20, 2015
pauldix added a commit that referenced this issue Mar 28, 2015
@pauldix
Add chunked responses and streaming of raw queries.
df18ea0 
Refactored query engine to have different processing pipeline for raw queries. This enables queries that have a large offset to not keep everything in memory. It also makes it so that queries against raw data that have a limit will only p
rocess up to that limit and then bail out.

Raw data queries will only read up to a certain point in the map phase before yielding to the engine for further processing.

Fixes #2029 and fixes #2030
pauldix added a commit that referenced this issue Mar 28, 2015
@pauldix
Add chunked responses and streaming of raw queries.
a12a58a 
Refactored query engine to have different processing pipeline for raw queries. This enables queries that have a large offset to not keep everything in memory. It also makes it so that queries against raw data that have a limit will only p
rocess up to that limit and then bail out.

Raw data queries will only read up to a certain point in the map phase before yielding to the engine for further processing.

Fixes #2029 and fixes #2030
@pauldix pauldix mentioned this issue Mar 28, 2015
Closed
@pauldix pauldix self-assigned this Mar 28, 2015
pauldix added a commit that referenced this issue Apr 1, 2015
@pauldix @otoolep
Add chunked responses and streaming of raw queries.
12a6fcb 
Refactored query engine to have different processing pipeline for raw queries. This enables queries that have a large offset to not keep everything in memory. It also makes it so that queries against raw data that have a limit will only p
rocess up to that limit and then bail out.

Raw data queries will only read up to a certain point in the map phase before yielding to the engine for further processing.

Fixes #2029 and fixes #2030
@pauldix pauldix closed this as completed in 6c46a5c Apr 2, 2015
pauldix added a commit that referenced this issue Apr 4, 2015
@pauldix
Add chunked responses and streaming of raw queries.
958ce63 
Refactored query engine to have different processing pipeline for raw queries. This enables queries that have a large offset to not keep everything in memory. It also makes it so that queries against raw data that have a limit will only p
rocess up to that limit and then bail out.

Raw data queries will only read up to a certain point in the map phase before yielding to the engine for further processing.

Fixes #2029 and fixes #2030
pauldix added a commit that referenced this issue Apr 6, 2015
@pauldix
Add chunked responses and streaming of raw queries.
f2cca73 
Refactored query engine to have different processing pipeline for raw queries. This enables queries that have a large offset to not keep everything in memory. It also makes it so that queries against raw data that have a limit will only p
rocess up to that limit and then bail out.

Raw data queries will only read up to a certain point in the map phase before yielding to the engine for further processing.

Fixes #2029 and fixes #2030
pauldix added a commit that referenced this issue Apr 7, 2015
@pauldix
Add chunked responses and streaming of raw queries.
57a3e0a 
Refactored query engine to have different processing pipeline for raw queries. This enables queries that have a large offset to not keep everything in memory. It also makes it so that queries against raw data that have a limit will only p
rocess up to that limit and then bail out.

Raw data queries will only read up to a certain point in the map phase before yielding to the engine for further processing.

Fixes #2029 and fixes #2030
pauldix added a commit that referenced this issue Apr 9, 2015
@pauldix @otoolep
Add chunked responses and streaming of raw queries.
7db3724 
Refactored query engine to have different processing pipeline for raw queries. This enables queries that have a large offset to not keep everything in memory. It also makes it so that queries against raw data that have a limit will only p
rocess up to that limit and then bail out.

Raw data queries will only read up to a certain point in the map phase before yielding to the engine for further processing.

Fixes #2029 and fixes #2030
pauldix added a commit that referenced this issue Apr 9, 2015
@pauldix @otoolep
Add chunked responses and streaming of raw queries.
4ea1c40 
Refactored query engine to have different processing pipeline for raw queries. This enables queries that have a large offset to not keep everything in memory. It also makes it so that queries against raw data that have a limit will only p
rocess up to that limit and then bail out.

Raw data queries will only read up to a certain point in the map phase before yielding to the engine for further processing.

Fixes #2029 and fixes #2030
pauldix added a commit that referenced this issue Apr 10, 2015
@pauldix @otoolep
Add chunked responses and streaming of raw queries.
4f66251 
Refactored query engine to have different processing pipeline for raw queries. This enables queries that have a large offset to not keep everything in memory. It also makes it so that queries against raw data that have a limit will only p
rocess up to that limit and then bail out.

Raw data queries will only read up to a certain point in the map phase before yielding to the engine for further processing.

Fixes #2029 and fixes #2030
pauldix added a commit that referenced this issue Apr 10, 2015
@pauldix @otoolep
Add chunked responses and streaming of raw queries.
2a3ed63 
Refactored query engine to have different processing pipeline for raw queries. This enables queries that have a large offset to not keep everything in memory. It also makes it so that queries against raw data that have a limit will only p
rocess up to that limit and then bail out.

Raw data queries will only read up to a certain point in the map phase before yielding to the engine for further processing.

Fixes #2029 and fixes #2030
mark-rushakoff pushed a commit that referenced this issue Jan 11, 2019
@ischolten
Merge pull request #2030 from influxdata/dataLoaders/listening-error-…
59d2d1c 
…state

fix(ui/dataLoaders): Separate the error state from the data not found
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@pauldix pauldix

Labels
None yet
Projects
None yet
Milestone
0.9.0
Development

Successfully merging a pull request may close this issue.

Add chunked responses and streaming of raw queries influxdata/influxdb
1 participant
@pauldix