Changelog entries for 11.3.0

[hanefi]

In this release, I tried something different. I experimented with adding the PR number and title to the changelog right before each changelog entry. This way, it is easier to track where a particular changelog entry comes from. After reviews are over, I plan to remove those lines with PR numbers and titles.

I went through all the PRs that are merged after 11.2.0 release and came up with a list of PRs that may need help with changelog entries. You can see details on PRs grouped in several sections below.

PRs with missing entries

The following PRs below do not have a changelog entry. If you think that this is a mistake, please share it in this PR along with a suggestion on what the changelog item should be.

PR #6846 : fix 3 flaky tests in failure schedule
PR #6844 : Add CPU usage to citus_stat_tenants
PR #6833 : Fix citus_stat_tenants period updating bug
PR #6787 : Add more tests for ddl coverage
PR #6842 : Add build-cdc-* temporary directories to .gitignore
PR #6841 : Add build-cdc-* temporary directories to .gitignore
PR #6840 : Bump Citus to 12.0devel
PR #6824 : Fixes flakiness in multi_metadata_sync test
PR #6811 : Backport identity column improvements to v11.2
PR #6830 : In run_test.py actually return worker_count
PR #6825 : Fixes flakiness in multi_cluster_management test
PR #6816 : Refactor run_test.py
PR #6817 : Explicitly disallow local rels when inserting into dist table
PR #6821 : Rename citus stats tenants
PR #6822 : Add some more tests for initial sql support
PR #6819 : Fix flakyness in citus_split_shard_by_split_points_deferred_drop
PR #6814 : Make python-regress based tests runnable with run_test.py
PR #6813 : Fix flaky multi_mx_schema_support test
PR #6720 : Convert columnar tap tests to pytest
PR #6812 : Revoke statistics permissions from public and grant them to pg_monitor
PR #6769 : Citus stats tenants guc
PR #6807 : Fix the incorrect (constant) value passed to pointer-to-bool parameter, pass a NULL as the value is not used
PR #6797 : Attribute local queries and cached plans on local execution
PR #6796 : Parse the annotation string correctly
PR #6762 : Add logs to citus_stats_tenants
PR #6773 : Add initial sql support for distributed tables that don't have a shard key
PR #6792 : Disentangle MERGE planning code from the modify-planning code path
PR #6761 : Citus stats tenants collector view
PR #6791 : Make 8 more tests runnable multiple times via run_test.py
PR #6786 : Refactor some of the planning code to accommodate a new planning path for MERGE SQL
PR #6789 : Rename AllRelations.. functions to AllDistributedRelations..
PR #6788 : Actually skip arbitrary_configs_router & nested_execution for AllNullDistKeyDefaultConfig
PR #6783 : Add a config for arbitrary config tests where all the tables are null-shard-key tables
PR #6784 : Fix attach partition: citus local to null distributed
PR #6782 : Add an arbitrary config test heavily based on multi_router_planner_fast_path.sql
PR #6781 : Decide what to do with router planner error at one place
PR #6778 : Support partitioning for dist tables with null dist keys
PR #6766 : fix pip lock file
PR #6764 : Make workerCount configurable for regression tests
PR #6745 : Add support for creating distributed tables with a null shard key
PR #6696 : This implements MERGE phase-III
PR #6767 : Add pytest depedencies to Pipfile
PR #6760 : Decide core distribution params in CreateCitusTable
PR #6759 : Add multi_create_fdw into minimal_schedule
PR #6743 : Replace CITUS_TABLE_WITH_NO_DIST_KEY checks with HasDistributionKey()
PR #6751 : Stabilize single_node.sql and others that report illegal node removal
PR #6742 : Refactor CreateDistributedTable()
PR #6747 : Remove unused lock functions
PR #6744 : Fix multiple output version arbitrary config tests
PR #6741 : Stabilize single node tests
PR #6740 : Fix string eval bug in migration files check
PR #6736 : Make run_test.py and create_test.py importable without errors
PR #6734 : Don't blanket ignore flake8 E402 error
PR #6737 : Fixes bookworm packaging pipeline problem
PR #6735 : Fix run_test.py on python 3.9
PR #6733 : MERGE: In deparser, add missing check for RETURNING clause.
PR #6714 : Remove auto_explain workaround in citus explain hook for ALTER TABLE
PR #6719 : Fix flaky test
PR #6718 : Add more powerfull dependency tracking to run_test.py
PR #6710 : Install non-vulnerable cryptography package
PR #6711 : Support compilation and run tests on latest PG versions
PR #6700 : Add auto-formatting and linting to our python code
PR #6707 : Allow multi_insert_select to run repeatably
PR #6708 : Fix flakyness in failure_create_distributed_table_non_empty
PR #6698 : Miscellaneous cleanup
PR #6704 : Update README for 11.2
PR #6703 : Fix dubious ownership error from git
PR #6690 : Bump Citus to 11.3devel

Too long changelog entries

The following PRs have changelog entries that are too long to fit in a single line. I'd expect authors to supply at changelog entries in DESCRIPTION: lines that are at most 78 characters. If you want to supply multi-line changelog items, you can have multiple lines that start with DESCRIPTION: instead.

PR #6837 : fixes update propagation bug when citus_set_coordinator_host is called more than once
PR #6738 : Identity column implementation refactorings
PR #6756 : Schedule parallel shard moves in background rebalancer by removing task dependencies between shard moves across colocation groups.
PR #6793 : Add a GUC to disallow planning the queries that reference non-colocated tables via router planner
PR #6726 : fix memory leak during altering distributed table with a lot of partition and shards
PR #6722 : fix memory leak during distribution of a table with a lot of partitions
PR #6693 : prevent memory leak during ConvertTable with a lot of partitions

Empty changelog entries.

The following PR had an empty DESCRIPTION: line. This generates an empty changelog line that needs to be removed manually. Please either provide a short entry, or remove DESCRIPTION: line completely.

PR #6810 : Make CDC decoder an independent extension
PR #6827 : Makefile changes to build CDC in builddir for pgoutput and wal2json.

[codecov]

Codecov Report

Merging #6856 (8650101) into main (59ccf36) will increase coverage by 0.00%.
The diff coverage is n/a.

❗ Current head 8650101 differs from pull request most recent head 037ea72. Consider uploading reports for the commit 037ea72 to get more accurate results

@@           Coverage Diff           @@
##             main    #6856   +/-   ##
=======================================
  Coverage   93.26%   93.27%           
=======================================
  Files         269      269           
  Lines       57457    57456    -1     
=======================================
+ Hits        53589    53593    +4     
+ Misses       3868     3863    -5     

[pinodeca]

I like having the PR lnk in the change log, makes it so easy to click through for clarity or when intrigued.

[onurctirtir]

Too long changelog entries

The following PRs have changelog entries that are too long to fit in a single line. I'd expect authors to supply at changelog entries in DESCRIPTION: lines that are at most 78 characters. If you want to supply multi-line changelog items, you can have multiple lines that start with DESCRIPTION: instead.

I think we were ok with having changelog entries longer than 78, by making use of multiple lines?

[onurctirtir]

Empty changelog entries.

The following PR had an empty DESCRIPTION: line. This generates an empty changelog line that needs to be removed manually. Please either provide a short entry, or remove DESCRIPTION: line completely.

Two PRs listed in this section don't need to show up in CHANGELOG anyway

[onurctirtir]

PRs with missing entries

The following PRs below do not have a changelog entry. If you think that this is a mistake, please share it in this PR along with a suggestion on what the changelog item should be.

None of those PRs require changelog entry except the Merge Phase-III one and the ones related to tenant statistics.

• We should definitely have a more descriptive entry for MERGE.
• We should have a comprehensive changelog entry for tenant statistics work: the GUC, views, UDFs (if any) ..

[rajeshkt78]

I don't see the Changelog for CDC (Change Data Capture) PR in the Changelog..
#6623
#6810
#6827

[marcocitus]

Leaving a unrelated/semi-related note here: We probably need to bump the citus_columnar version as well even if there were no changes.

[hanefi]

I don't see the Changelog for CDC (Change Data Capture) PR in the Changelog.. #6623 #6810 #6827

We expect the changelog entry to be in the same line that starts with DESCRIPTION: . However #6810 and #6827 does not have that information on the same line. However, I do not really understand why #6623 was skipped. I guess we should have that.

The pull request template is DESCRIPTION: PR description that will go into the change log, up to 78 characters. Maybe it is not clear that we want the description on the same line.

How about a single line at the very beginning:

* Introduces CDC implementation for Citus using logical replication (#6623, #6810, #6827)

[tejeswarm]

@hanefi @onurctirtir

PR #6792 : Disentangle MERGE planning code from the modify-planning code path
Simple refactoring of the code, no need of PR

PR #6786 : Refactor some of the planning code to accommodate a new planning path for MERGE SQL
Simple refactoring of the code, no need of PR

PR #6696 : This implements MERGE phase-III
Added

PR #6733 : MERGE: In deparser, add missing check for RETURNING clause.
Added

[aykut-bozkurt]

I fixed following too long entries:
#6837, #6726, #6722, #6693

[hanefi]

PR #6733 : MERGE: In deparser, add missing check for RETURNING clause.
Added

The added changelog entry was: RETURNING is not supported in MERGE, so, it must be NULL

@tejeswarm I do not think that this one deserves an entry in the changelog as there is no supported MERGE command with returning clause as of today and it is impossible right now to get the error message.

Thanks for adding a nice entry for #6696 though. Appreciated.

[onurctirtir]

Is this ready for reviews?

Also, as Marco mentioned, should we bump columnar version too?

Leaving a unrelated/semi-related note here: We probably need to bump the citus_columnar version as well even if there were no changes.

[onderkalaci]

why is this at the top of the change log? I'd probably move down a bit

[hanefi]

I ordered items lexicographically and moved one important entry to the top of the list. Where do you want it to be placed exactly?

[onurctirtir]

I see what Onder means, let's move this to right after

Allows parallel shard moves in background rebalancer (#6756)

[onurctirtir]

I will push a commit to a separate branch to suggest a different ordering for the entries

[onurctirtir]

12fc492

[hanefi]

I forgot to cherry pick your comment Onur. Now I did it and the changelog should be ok.

[hanefi]

Is this ready for reviews?

It is ready now.

Also, as Marco mentioned, should we bump columnar version too?

I can (and will) do that in another PR.

[onurctirtir]

Also, as Marco mentioned, should we bump columnar version too?

I can (and will) do that in another PR.

Thank you. We should probably come up with a clear procedure about that.

• Should we bump schema version on minor schema versions too (-1, -2 ..) or only on major updates?
• Should we reflect this in release automation scripts? etc.
• ...

[hanefi]

A final update:

• I added one more changelog entry for a newly backported commit in 8650101
• I cherry picked Onur's commit that reorders some items as suggested by Onder. 5f03124
• I opened 2 PRs that update columnar versions on release-11.3 and main branches. Prepare release branch for 11.3.0 #6897 and Bump columnar to 11.3 #6898
• I see 3 failures on packaging tests that are not really related to my changes in the PR. I hope the connection timeouts for the PGDG repositories get resolved soon and the tests start to pass again.

[onurctirtir]

We don't need to have this for 12.0 because non-transactional metadata sync is introduced in this release already?

[onurctirtir]

Also, should we update the release date for 12.0?

[hanefi]

I had a chat with Aykut, and merged the non-transactional fix with the related s919fe72.

I also set the date to today in 037ea72