fix(search-indexer): Only calculate folder hash on initial iteration

[RunarVestmann]

Only calculate folder hash on initial iteration

What

• Now the folder hash is only calculated once
• Doubled the chunk size on dev for when fetching entries from Contentful to speed things up

Why

• Folder hash was getting calculated for each paginated sync response (it only really needs to be calculated once)

Checklist:

• I have performed a self-review of my own code
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• Formatting passes locally with my changes
• I have rebased against main before asking for a review

Summary by CodeRabbit

• New Features

  • Increased the chunk size for fetching entries in the development environment, enhancing data retrieval efficiency.
  • Introduced a new optional property, folderHash, to improve synchronization tracking and management.

• Bug Fixes

  • Streamlined the sync process by adding checks to prevent unnecessary computations related to the folderHash, improving overall efficiency.
  • Refined the conditions for identifying delta updates in sync operations, enhancing control over synchronization behavior.

[coderabbitai]

Walkthrough

The pull request introduces several changes across multiple files, primarily focusing on enhancing synchronization processes and configuration settings. Key modifications include increasing the CONTENTFUL_ENTRY_FETCH_CHUNK_SIZE for the development environment, refining the logic for handling folder hashes in the CmsSyncService, and adding a new folderHash property to the SyncOptions interface. These changes aim to improve data fetching efficiency and streamline synchronization operations.

Changes

File Path	Change Summary
apps/services/search-indexer/infra/search-indexer-service.ts, charts/islandis/values.dev.yaml	Increased CONTENTFUL_ENTRY_FETCH_CHUNK_SIZE.dev from '20' to '40'.
libs/cms/src/lib/search/cmsSync.service.ts	Improved handling of folderHash to avoid unnecessary sync calls by checking if it is defined.
libs/content-search-indexer/src/lib/indexing.service.ts	Added folderHash to options in importer.doSync method to enhance synchronization context.
libs/content-search-indexer/types/src/index.ts	Introduced a new optional property folderHash in SyncOptions interface to track folder-specific data.
libs/cms/src/lib/search/contentful.service.ts	Modified sync operation logic to specifically recognize 'fromLast' as a delta update.

Possibly related PRs

• feat(search-indexer): Nested document resolution #15949: The changes in the ContentfulService class involve enhancements to data retrieval from Contentful, which may relate to the adjustments made in the search-indexer-service.ts for fetching entries, as both involve optimizing how data is handled and retrieved from Contentful.

Suggested reviewers

• mannipje

---

Recent review details

Configuration used: .coderabbit.yaml
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 2796cf8 and 93db59c.

Files selected for processing (1)

• libs/content-search-indexer/src/lib/indexing.service.ts (2 hunks)

Files skipped from review as they are similar to previous changes (1)

• libs/content-search-indexer/src/lib/indexing.service.ts

---

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share

• X
• Mastodon
• Reddit
• LinkedIn

Tips

Chat

There are 3 ways to chat with CodeRabbit:

• Review comments: Directly reply to a review comment made by CodeRabbit. Example:
  -- I pushed a fix in commit <commit_id>, please review it.
  -- Generate unit testing code for this file.
  • Open a follow-up GitHub issue for this discussion.
• Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
  -- @coderabbitai generate unit testing code for this file.
  -- @coderabbitai modularize this function.
• PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
  -- @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
  -- @coderabbitai read src/utils.ts and generate unit testing code.
  -- @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
  -- @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

• @coderabbitai pause to pause the reviews on a PR.
• @coderabbitai resume to resume the paused reviews.
• @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
• @coderabbitai full review to do a full review from scratch and review all the files again.
• @coderabbitai summary to regenerate the summary of the PR.
• @coderabbitai resolve resolve all the CodeRabbit review comments.
• @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
• @coderabbitai help to get help.

Other keywords and placeholders

• Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
• Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
• Add @coderabbitai anywhere in the PR title to generate the title automatically.

Documentation and Community

• Visit our Documentation for detailed information on how to use CodeRabbit.
• Join our Discord Community to get help, request features, and share feedback.
• Follow us on X/Twitter for updates and announcements.

[svanaeinars]

LGTM

[codecov]

Codecov Report

Attention: Patch coverage is 0% with 14 lines in your changes missing coverage. Please review.

Project coverage is 36.66%. Comparing base (bc0f27b) to head (93db59c).
Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
libs/cms/src/lib/search/cmsSync.service.ts	0.00%	13 Missing ⚠️
libs/cms/src/lib/search/contentful.service.ts	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #16084      +/-   ##
==========================================
- Coverage   36.66%   36.66%   -0.01%     
==========================================
  Files        6748     6748              
  Lines      138729   138733       +4     
  Branches    39405    39407       +2     
==========================================
  Hits        50862    50862              
- Misses      87867    87871       +4     

Flag	Coverage Δ
air-discount-scheme-web	0.00% <ø> (ø)
application-api-files	57.59% <ø> (ø)
application-core	71.29% <ø> (+0.08%)	⬆️
application-system-api	41.50% <0.00%> (-0.01%)	⬇️
application-template-api-modules	23.47% <ø> (+0.01%)	⬆️
application-templates-accident-notification	22.14% <ø> (ø)
application-templates-car-recycling	3.12% <ø> (ø)
application-templates-criminal-record	26.63% <ø> (ø)
application-templates-driving-license	18.63% <ø> (ø)
application-templates-estate	12.34% <ø> (ø)
application-templates-example-payment	25.41% <ø> (ø)
application-templates-financial-aid	14.34% <ø> (ø)
application-templates-general-petition	23.68% <ø> (ø)
application-templates-health-insurance	26.62% <ø> (ø)
application-templates-inheritance-report	6.45% <ø> (ø)
application-templates-marriage-conditions	15.23% <ø> (ø)
application-templates-mortgage-certificate	43.96% <ø> (ø)
application-templates-parental-leave	30.15% <ø> (ø)
application-types	6.71% <ø> (ø)
application-ui-components	1.28% <ø> (ø)
application-ui-shell	21.29% <ø> (ø)
auth-react	22.80% <ø> (ø)
clients-charge-fjs-v2	24.11% <ø> (ø)
cms	0.43% <0.00%> (-0.01%)	⬇️
cms-translations	39.12% <0.00%> (-0.02%)	⬇️
contentful-apps	5.76% <ø> (ø)
financial-aid-backend	56.52% <ø> (ø)
financial-aid-shared	19.03% <ø> (ø)
island-ui-core	28.57% <ø> (ø)
judicial-system-api	18.61% <ø> (ø)
judicial-system-backend	54.71% <0.00%> (-0.02%)	⬇️
judicial-system-web	28.66% <ø> (ø)
services-auth-personal-representative	47.94% <ø> (ø)
services-user-notification	47.16% <0.00%> (-0.02%)	⬇️
web	1.85% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
libs/cms/src/lib/search/contentful.service.ts	6.60% <0.00%> (ø)
libs/cms/src/lib/search/cmsSync.service.ts	38.92% <0.00%> (-1.08%)	⬇️

---

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bc0f27b...93db59c. Read the comment docs.

[datadog-island-is]

Datadog Report

All test runs dc6a406 🔗

✅ 34 Total Test Services: 0 Failed, 32 Passed
🔻 Test Sessions change in coverage: 3 decreased, 1 increased (+0.1%), 137 no change

Test Services

This report shows up to 10 services

Service Name	Failed	Known Flaky	New Flaky	Passed	Skipped	Total Time	Code Coverage Change	Test Service View
air-discount-scheme-web	0	0	0	2	0	7.62s	1 no change	Link
api	0	0	0	4	0	3.1s	1 no change	Link
api-domains-license-service	0	0	0	0	0	620.26ms	1 no change	Link
application-api-files	0	0	0	12	0	5.72s	1 no change	Link
application-core	0	0	0	90	0	18.17s	1 increased (+0.1%)	Link
application-system-api	0	0	0	111	2	3m 23.64s	1 no change	Link
application-template-api-modules	0	0	0	109	0	1m 49.34s	1 no change	Link
application-templates-accident-notification	0	0	0	98	0	18.83s	1 no change	Link
application-templates-criminal-record	0	0	0	2	0	10.8s	1 no change	Link
application-templates-driving-license	0	0	0	13	0	17.16s	1 no change	Link

🔻 Code Coverage Decreases vs Default Branch (3)

• cms-translations - jest 48.65% (-0.03%) - Details
• judicial-system-backend - jest 58.09% (-0.02%) - Details
• services-user-notification - jest 69.85% (-0.01%) - Details