Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Load SLRU segments on demand #331

Merged
merged 8 commits into from
Jan 31, 2024
Merged

Conversation

knizhnik
Copy link

@knizhnik knizhnik force-pushed the load_slru_on_demand branch 3 times, most recently from fefc07a to eb34fd7 Compare December 16, 2023 08:58
@knizhnik knizhnik force-pushed the load_slru_on_demand branch from 4f6e1fc to c04811e Compare December 27, 2023 12:06
Copy link
Contributor

@hlinnaka hlinnaka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we also need to do on-demand download in SlruPhysicalWritePage()? If not, why not?


static SMgrRelationData dummy_smgr_rel = {0};

/* If page is greather than latest written page, then do not try to download segment from server */
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/* If page is greather than latest written page, then do not try to download segment from server */
/* If page is greater than latest written page, then do not try to download segment from server */

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@knizhnik
Copy link
Author

Do we also need to do on-demand download in SlruPhysicalWritePage()? If not, why not?

Before writing page Postgres first cake if it exists:

	/* Create the current segment file, if necessary */
	if (!SimpleLruDoesPhysicalPageExist(CommitTsCtl, pageno))
	{
		int			slotno;

		LWLockAcquire(CommitTsSLRULock, LW_EXCLUSIVE);
		slotno = ZeroCommitTsPage(pageno, false);
		SimpleLruWritePage(CommitTsCtl, slotno);
		Assert(!CommitTsCtl->shared->page_dirty[slotno]);
		LWLockRelease(CommitTsSLRULock);
	}

and SimpleLruDoesPhysicalPageExist tries to load page from PS if it doesn't exists.

There are few more cases in slru.c where SlruInternalWritePage is called (except SimpleLruWritePage) but it is done for pages present in the slot and they should be checked and downloaded before.

@knizhnik knizhnik force-pushed the load_slru_on_demand branch from 438f36e to 745bae3 Compare January 17, 2024 09:42
@knizhnik knizhnik force-pushed the load_slru_on_demand branch from 745bae3 to 433b83c Compare January 24, 2024 21:15
@knizhnik knizhnik merged commit f7ea954 into REL_16_STABLE_neon Jan 31, 2024
@knizhnik knizhnik deleted the load_slru_on_demand branch January 31, 2024 13:41
tristan957 pushed a commit that referenced this pull request Feb 5, 2024
* Load SLRU segments on demand

refer #8763

* Fix errors in downloading SLRU segments

* Fix build problems

* Undo occcasional changes

* Remove unintenmded changes

* Fix smgr_read_slru_segment

* Determine SLRU kind in extension

* Use ctl->PagePrecedes for SLRU page comparison in SimpleLruDownloadSegment to address wraparround

---------

Co-authored-by: Konstantin Knizhnik <knizhnik@neon.tech>
tristan957 pushed a commit that referenced this pull request Feb 5, 2024
* Load SLRU segments on demand

refer #8763

* Fix errors in downloading SLRU segments

* Fix build problems

* Undo occcasional changes

* Remove unintenmded changes

* Fix smgr_read_slru_segment

* Determine SLRU kind in extension

* Use ctl->PagePrecedes for SLRU page comparison in SimpleLruDownloadSegment to address wraparround

---------

Co-authored-by: Konstantin Knizhnik <knizhnik@neon.tech>
tristan957 pushed a commit that referenced this pull request Feb 6, 2024
* Load SLRU segments on demand

refer #8763

* Fix errors in downloading SLRU segments

* Fix build problems

* Undo occcasional changes

* Remove unintenmded changes

* Fix smgr_read_slru_segment

* Determine SLRU kind in extension

* Use ctl->PagePrecedes for SLRU page comparison in SimpleLruDownloadSegment to address wraparround

---------

Co-authored-by: Konstantin Knizhnik <knizhnik@neon.tech>
tristan957 pushed a commit that referenced this pull request May 10, 2024
* Load SLRU segments on demand

refer #8763

* Fix errors in downloading SLRU segments

* Fix build problems

* Undo occcasional changes

* Remove unintenmded changes

* Fix smgr_read_slru_segment

* Determine SLRU kind in extension

* Use ctl->PagePrecedes for SLRU page comparison in SimpleLruDownloadSegment to address wraparround

---------

Co-authored-by: Konstantin Knizhnik <knizhnik@neon.tech>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants