Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update dependency org.questdb:questdb to v8 #194

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Update dependency org.questdb:questdb to v8 #194

wants to merge 1 commit into from

Conversation

renovate[bot]
Copy link

@renovate renovate bot commented Jul 28, 2024 •
edited
Loading

This PR contains the following updates:

Package Change Age Adoption Passing Confidence
org.questdb:questdb (source) 6.2 -> 8.1.2 age adoption passing confidence

Release Notes

questdb/questdb (org.questdb:questdb)

v8.1.2

Compare Source

October has arrived, and with it the fall. According to an interesting character, it's also Libra ♎ season, a phase of balance. It works for us, as this release restores the balance within QuestDB's production reliability. Chasing maximum performance and cutting-edge features is our favourite game. But every now and then, it's essential to double-back and smooth out the edges so that the overall experience presents consistent excellence. And, well, we'll keep improving performance too.

Download, upgrade, and prepare for the seasonal feasts. 🎃

Breaking Changes 💥
  • Logging level adjustments: Some log messages previously logged at the INFO level have been changed to DEBUG. This reduction in log verbosity may impact monitoring systems relying on these messages. Adjust logging configurations as necessary.
New features 🐣

  • Web Console UI tabs: The QuestDB Web Console features tabs for better query management. Tabs are archived upon closing and can be brought back from the history section as needed.

  • ILP client row cancellation: The ILP client now supports cancelling the current row during data ingestion. This allows for greater control when writing data and helps prevent incomplete or erroneous data from being committed.

  • New financial function spread_bps: Computes the spread in basis points between two numerical values.

  • greatest and least functions: Determine the maximum or minimum value from a list of expressions. These functions simplify comparisons across multiple columns or expressions.

  • Interval functions for date filters: Implemented new interval functions today(), tomorrow(), and yesterday() that return intervals representing the entire day. These functions make it easier to filter queries based on whole-day periods without manually specifying time ranges.

  • New aggregate function string_distinct_agg: Aggregates unique string values into a single, concatenated string with a specified delimiter. This is useful for creating summaries of distinct values in a dataset.

  • ILP traffic to Prometheus: Introduced metrics to monitor ILP (Influx Line Protocol) traffic volumes over both TCP and HTTP. These metrics help in tracking data ingestion rates and diagnosing network-related issues.

  • RedHat OpenShift Compatible Docker Image: Beginning with this release, QuestDB Docker images with a -rhel suffix are now compatible with RedHat's OpenShift container orchestration platform.

New integrations

  • Bento by Warpstream Labs: In the words of Bento they provide: Fancy stream processing made operationally mundane. Indeed, and now even fancier with a fresh QuestDB component.

  • Redpanda Connect: Formerly Benthos. Comes with a QuestDB output component that can be used as a sink for your stream processing data. Read our docs for more information.

Performance improvements 🚀

  • Static metadata cache: Implemented a static metadata cache to improve performance by reducing the overhead associated with frequent metadata access. This enhancement speeds up query compilation and execution.

  • Optimized partition reloads: Reduced unnecessary reloads of TableReader partitions, which enhances query performance, especially in environments with high data ingestion rates and frequent queries.

  • Improved parallel GROUP BY execution: Optimized parallel GROUP BY operations by eliminating redundant bytecode generation. This results in faster execution times for complex aggregation queries.

  • Faster Web Console queries: You'll notice improved performance during the highligh and execution of queries with SQL comments in the editor.

Improvements 🛠️

  • Enhanced log compatibility: Added a configuration option to control verbose log levels, improving compatibility with log readers and monitoring tools that require specific log formats or verbosity levels.

  • Detailed ILP error messages: Enhanced the server logs to include specific error messages indicating the exact line that caused a receive buffer overflow during ILP ingestion. This aids in troubleshooting and correcting data formatting issues.

Bug fixes 🐛

  • ILP client retry behaviour: Fixed an issue where ILP clients would retry data insertion when encountering invalid UUID values. The client now correctly handles bad UUIDs without unnecessary retries, improving ingestion stability.

  • Server crash with first(varchar) and last(varchar): Resolved a critical issue where using first(varchar) or last(varchar) functions could cause the server to crash. These functions now operate reliably with VARCHAR data types.

  • Deduplication key crash fix: Addressed a potential crash when writing data with deduplication enabled on VARCHAR or STRING columns as keys.

  • Parquet byte column support: Added support for RleDictionary encoding of BYTE columns in the read_parquet() function. This enhances compatibility with Parquet files that use this encoding scheme.

  • Handling comment-only queries: Fixed an issue where SQL queries containing only comments were not properly processed. The server now correctly handles such queries without errors.

  • UUID and IPv4 in coalesce and switch: Corrected the lack of support for UUID and IPv4 literals in the coalesce() function and SWITCH statements. These functions now fully support these data types.

  • Parallel GROUP BY race condition: Fixed a race condition that could occur during parallel GROUP BY operations involving a large number of groups.

  • Symbol to integer comparison fix: Resolved a NullPointerException that occurred when comparing SYMBOL columns to integer values. Comparisons between these data types now function correctly.

  • Responsive health check endpoint: Improved the responsiveness of the health check HTTP endpoint when the server reaches its memory limit. This ensures that health checks remain reliable under memory pressure.

List of PRs

New Contributors

Full Changelog: questdb/questdb@8.1.1...8.1.2

v8.1.1

Compare Source

QuestDB 8.1.1

Back to school... Baaack to school...

Lunch? Check. Pants? Check (phew!) New QuestDB release? Oh yeah.

Let's go! 📚 ✏️ 🥪

Breaking changes 💥

We have changed the language in our SNAPSHOT related SQL statements. We introduce CHECKPOINT.

Apply it before taking a backup.

Instead of executing:

 SNAPSHOT PREPARE

You will now execute:

CHECKPOINT CREATE

When the backup is complete, instead of executing:

SNAPSHOT COMPLETE

You will execute:

  CHECKPOINT RELEASE

Both commands are supported for now, but we encourage you to migrate to the new system.

We have also introduced a fix for integer overflow, which now widens the type of the result.

For example, the following will be automatically widened to LONG (64bit) to avoid numeric overflow:

select 799990090 * 100000

Improvements 🚀

  • General improvements to database stability
  • Out-of-the-box ingress performance improved via reduced resource contention
  • SQL execution performance improvements for cases like ORBER BY <long>, and GROUP BY with virtual columns such as now()
  • Certain ASOF SQL queries have improved their performance by orders of magnitude
  • Support for VARCHAR and STRING as deduplication keys.

Pull requests

New Contributors

Full Changelog: questdb/questdb@8.1.0...8.1.1

v8.1.0

Compare Source

QuestDB 8.1.0 has arrived.

In this release, QuestDB's foundation takes a second step towards our next generation architecture.

Our prior release introduced JSON extraction.

And now, by popular demand, QuestDB meets Apache Parquet.

New Features 🐣
  • Parquet read support Use the read_parquet() function to read Apache Parquet files.
  • Improved SAMPLE BY:
    • FROM-TO syntax: Specify result ranges, with support for prefilling and postfilling data. Also can correct misaligned buckets.
    • Parallel SAMPLE BY: Now supports FILL(VALUE) and FILL(NULL).
  • Snapshot recovery: Added a trigger file to initiate snapshot recovery, improving database resilience and ease-of-use.
Performance 🚀
  • Improved ILP ingress latency
  • SQL Performance:
    • Speedup for small transaction writing by 50-100%.
    • Enhanced performance for the like/ilike operator on symbol columns.
    • Speed improvements for regexp_replace(varchar) with simple patterns.
  • Aggregation queries: Aggregation queries are now faster.
Bug Fixes 🐛

  • SQL:

    • Fixed spurious "unsupported operation" errors.
    • Corrected wrong results from parallel WHERE and GROUP BY for some function keys.
    • Fixed incorrect results from parallel GROUP BY with a single varchar function key.
    • Addressed issues with some window functions double counting rows.

  • Core:

    • Prevented table suspension under memory pressure.
    • Fixed Charts when using together with OAuth.

Pull requests

New Contributors

Full Changelog: questdb/questdb@8.0.3...8.1.0

v8.0.3

Compare Source

Summer time! 🌞 For some, that's sunshine, beaches, lotion, all that fun stuff. Or, well, hiding inside near the air conditioner. It's hot! For QuestDB, it's another set of improvements. This release contains a bundle of smaller improvements, with new features laying down major groundwork. And - of course - the usual motion of performance improvements. Hot indeed. 🔥

Upgrade! Apply liberal amounts of sun screen!

Breaking Changes 💥

  • QuestDB no longer supports FreeBSD. We test major browser versions on each release. That means everything we write needs to account for multiple OS paradigms. FreeBSD has very small market share, and more often than not we'd trip over edge cases. To speed things up, we've made the decision to drop support. This means FreeBSD is no longer tested during development.

  • NFS is an unsupported file system. NFS has been unsupported for some time, but only warnings were presented to the user. QuestDB will now refuse to start if running on an NFS disk. We recommend using ZFS. For a full list of supported filesystems, check out the Capacity Planning documentation.

New Features 🐣

  • JSON Support: The json_extract() function is our first step towards robust JSON support. This function leverages the wonderful simdjson library written by Daniel Lemire. Simply store documents in a VARCHAR column, and then use the function to extract JSON fields into values. There is some performance overhead versus our native format, so we recommend using it to as a way to test a schema before putting the data into native columns. Read more in the documentation.

  • New financial SQL Functions: mid price function and spread function for financial calculations. Simple. But effective.

  • Smarter Web Console: Vague errors aren't helpful. The Web Console will now help you to troubleshoot common database configuration problems, with clear visual indicators to help you identify and resolve issues.

  • More parallel execution: Enhanced GROUP BY operations and parallel execution, particularly benefiting queries with aggregate functions such as avg() and sum().

  • HTTP Basic Authentication: QuestDB Open Source now offers Basic Auth to protect its HTTP endpoints. See how to configure this here.

Performance 🚀

  • JIT-optimized IN(numeric values) operator for faster query execution.

  • Optimized aggregate queries with the LAST function on designated timestamps.

  • Speed improvements for self-join queries on symbol columns and more efficient pattern lookups for short ASCII patterns.

Bug Fixes 🐛

  • Fixes for various issues including VARCHAR column queries via sqlx::postgres, invalid column errors in GROUP BY with JOIN, and more.

  • Improved error handling, such as preventing ugly error messages when the log directory does not exist and ensuring REST interface returns HTTP 408 on timeouts.

  • Ensured consistent escaping of VARCHAR fields in CSV exports.

  • Improved compatibility with InfluxDB clients and integrations, by supporting Basic Authentication.

Pull requests

New Contributors

Full Changelog: questdb/questdb@8.0.1...8.0.3

v8.0.2

Compare Source

v8.0.1

Compare Source

What's Changed

Full Changelog: questdb/questdb@8.0.0...8.0.1

v8.0.0

Compare Source

This one's a big one!

In brief, to get you to the goods quicker:

  • New VARCHAR data type over STRING default for ++compression & ++performance
  • 50% improvement over already fast SQL query performance
  • Data compression via system-level ZFS, previously Enterprise-only

And more...

Upgrade as soon as you can. Or start fresh and build out your use case.

If you're still reading, onwards for more detail, breaking changes, plus the PR breakdown.

New features 🐣
  • VARCHAR: Introduces the VARCHAR data type and a new type driver API. VARCHAR offers UTF8 encoded strings of variable length as opposed to the UTF16 and fixed length encoding found in STRING, for better compression and significantly improved performance.
    • Maximum size of a single VARCHAR field size is 268 MB.
    • A size of a VARCHAR column in a single partition is limited by 218 TB or disk size, whichever is smaller.
    • STRING is now considered a legacy type.
  • Data compression: Data compression for OSS! Full support for the ZFS file system and system-level data compression.
  • ALTER COLUMN type: Use ALTER TABLE SQL to convert column types.
    • Migrate between STRING, SYMBOL, and VARCHAR data types.
Breaking changes 💥
  • GLIBC upgrade: Upgraded GLIBC to a minimum of 2.28-amd64 and 2.33-aarch64.
  • ILP and TextImport changes: ILP and TextImport now create VARCHAR instead of STRING columns.
  • Jemalloc allocator: The default allocator is now jemalloc. Later updates include disabling jemalloc as the default allocator and removing jemalloc from no-jre builds.
Performance 🚀
  • SQL performance improvement: Average SQL performance improves by 50%. 🔥
  • Memory allocation limit: New memory allocation limit prevents OS OOM handler terminations in many cases.
Bug fixes 🥾
  • SQL: Fixed issues with SAMPLE BY and GROUP BY not grouping when used with column aliases. Fixed WHERE filters with FLOAT columns null comparison.
  • Core: Addressed trailing boundary processing of multipart HTTP requests. Fixed hour parsing and formatting in the 1-12 and 1-24 conventions.
Pull requests :octocat:

New Contributors

Full Changelog: questdb/questdb@7.4.2...8.0.0

v7.4.2

Compare Source

Choice is essential. But isn't it nice when a clear, happy path emerges?

To that tune, we're pleased to announce QuestDB 7.4.2, with the usual mix of features, fixes and improvements.

In focus this week is the refinement of our updated ingestion clients. Built atop the InfluxDB Line Protocol (ILP) and leveraging HTTP instead of TCP, our clients and core database now offer a primary ingestion path that is robust, responsive and fast. They're available in popular languages, including Rust, Python, Java, Go, C & C++, and more.

It is quicker, cleaner and clearer to both get started and refine.

New Features 🐣

Fresh goods.

  • Added bind variable support for long256 and IPv4 operators, milliseconds and microseconds support to datediff and dateadd.

  • Enabled FILL(LINEAR) to support ALIGN TO CALENDAR.

Performance 🚀

Always a category unto its own.

  • ILP: Optimized data ingestion for tables with a large number of columns.

  • Core: Improved hash table lookup for small string keys and optimized memory usage to reduce OS kills.

  • SQL: Faster SQL COPY and REST API CSV import, optimized binary search corner cases, and fixed the stuck parallel GROUP BY queries via better query timeout handling.

Bug Fixes 🥾

Squished, squashed & kiboshed.

  • ILP: Addressed 'maximum buffer size exceeded' in ILP client and made config adjustments to disable individual flush types and support auto_flush_bytes. also corrected client processing of data received before TLS close_notification.

  • SQL: Fixed incorrect results from lower, upper, substring functions when executed in parallel, erroneous SAMPLE BY behavior mixing asterisks, errors in CASE statements when ELSE branch is missing, incorrect results in GROUP BY queries, and SAMPLE BY queries with functions in the SELECT clause.

    Also fixed a bug for negative upper bound in BETWEEN operator, issues with null string insertions as designated timestamps, and inaccurate double comparisons. Finally, resolved stuck parallel group by queries due to incorrect timeout handling.

  • Core: Addressed a rare segmentation fault during o3 commit of a non-WAL table, potential error rebuilding nascent data index, and incorrect partition size after an update.

New Contributors 🎉

@​hoangsvit and @​sivukhin made their first contributions. Thanks, both of you. 🙏

Pull requests

New Contributors

Full Changelog: questdb/questdb@7.4.0...7.4.2

[v7.4.0](https://redir

Configuration

📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

  • If you want to rebase/retry this PR, check this box

This PR was generated by Mend Renovate. View the repository job log.

@renovate renovate bot force-pushed the renovate/org.questdb-questdb-8.x branch from a685e03 to 0269930 Compare September 15, 2024 12:11
@renovate renovate bot force-pushed the renovate/org.questdb-questdb-8.x branch from 0269930 to b83c5e3 Compare October 13, 2024 09:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
0 participants