Maintain last written LSN for each page to enable prefetch on vacuum,…

[knizhnik]

… delete and other massive update operations

[hlinnaka]

Note: this was done in upstream commit 7bf713d, but with a check for XLogHintBitIsNeeded:

@@ -8838,6 +8837,9 @@ heap_xlog_visible(XLogReaderState *record)
 
                PageSetAllVisible(page);
 
+               if (XLogHintBitIsNeeded())
+                       PageSetLSN(page, lsn);
+
                MarkBufferDirty(buffer);
        }
        else if (action == BLK_RESTORED)

I believe we need to do it always in Neon, so unfortunately we still need to carry a patch here. But let's move this to the same location, above the MarkBufferDirty.

[knizhnik]

Done, but please notice that in upstream it is committed with XLogHintBitIsNeeded() check and we are running neon now with wal_log_hints=off

[MMeent]

I'm not super happy with the memory overhead of tracking so many pages, but that's fine for now.

[knizhnik]

Please notice that there is eviction algorithm for this cache: we do not need to keep information about all pages i memory. Current default cache size is 128k entries - few Mb.

[MMeent]

Please notice that there is eviction algorithm for this cache: we do not need to keep information about all pages i memory. Current default cache size is 128k entries - few Mb.

Those "few MB" is what I'm not super happy about. In low-memory systems (like our free tier - 256MB) every MB counts.

For now, I think this is OK, but I think we need to revisit this in the future.