Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sync Engine with support for syncing bookmarks #355

Merged
merged 190 commits into from
May 26, 2023
Merged

Sync Engine with support for syncing bookmarks #355

merged 190 commits into from
May 26, 2023

Conversation

ayoy
Copy link
Contributor

@ayoy ayoy commented May 16, 2023

Please review the release process for BrowserServicesKit here.

Required:

Task/Issue URL: https://app.asana.com/0/0/1204606874917765/f
iOS PR: duckduckgo/iOS#1724
macOS PR: duckduckgo/macos-browser#1203
What kind of version bump will this require?: Major

Optional:

Tech Design URL:

Description:
Changes to bookmarks:

  • Updated Bookmarks model to v3 with added isPendingDeletion and modifiedAt
  • Bookmarks are now soft-deleted at all times. When sync is off, a database cleaner runs on schedule
    and removes soft-deleted bookmarks.
  • Client UIs are updated to not display soft-deleted bookmarks

Changes to Sync:

  • Replaced isAuthenticated with SyncAuthState enum (inactive, setupNewAccount, addNewDevice, active)
  • Added SyncScheduling and DataProviding public protocols. Made Crypting public too.
  • Added SyncMetadata Core Data model, SyncMetadataStore class and SyncFeatureEntity class representing a syncable model.
  • Sync is triggered from clients by calling scheduler APIs. Scheduler decides internally when to start sync.
  • SyncQueue is an actor and it can only run a single sync operation at a time.

New module – SyncDataProviders:

  • For now only contains BookmarkProvider, it’s an implementation of DataProviding for handling bookmarks.
    The logic is fully reused in iOS and macOS apps.

Recommended reading order:

  1. BookmarkEntity.swift, BookmarkUtils.swift, BookmarksDatabaseCleaner.swift
    (other changes under Bookmarks models are related to iOS app UI)
  2. DDGSyncing.swift, DDGSync.swift, ProductionDependencies.swift
  3. SyncScheduler.swift, SyncQueue.swift
  4. BookmarksProvider.swift
  5. The rest.

Steps to test this PR:
To speed up testing sync you may want to set SyncScheduler.appLifecycleEventsDebounceInterval to 1.

  1. Run client apps from develop and import some bookmarks.
  2. Switch to dominik/sync-engine branches on both clients and run the app again. Make sure that the app didn’t crash at startup due to failed migration.
  3. Enable sync on one of the devices. Then enable sync on another device by adding it to the first device’s account.
  4. Make sure that devices’ data was merged, i.e. added device contains both local and remote bookmarks and vice-versa.
  5. Make changes to data and verify that they are synced according to the design.

OS Testing:

  • iOS 14
  • iOS 15
  • iOS 16
  • macOS 10.15
  • macOS 11
  • macOS 12

Internal references:

Software Engineering Expectations
Technical Design Template

ayoy added 30 commits April 7, 2023 20:34
while true {
let bookmarksPendingDeletion = fetchBookmarksPendingDeletion(context)
if bookmarksPendingDeletion.isEmpty {
print("No bookmarks pending deletion")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't these be os logs? :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, I forgot to update them, let me handle this now :)

Copy link
Collaborator

@bwaresiak bwaresiak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great job!

@ayoy ayoy assigned ayoy and unassigned bwaresiak May 26, 2023
@ayoy ayoy merged commit 52aed5b into main May 26, 2023
@ayoy ayoy deleted the dominik/sync-engine branch May 26, 2023 09:02
samsymons added a commit that referenced this pull request Jun 19, 2023
* main: (45 commits)
  SecureVault Keychain updates (#382)
  Reattach orphaned bookmarks to root folder when reordering (#383)
  Allow for asynchronous sync initialization (#381)
  Drop regex support in param stripping (#367)
  mac promo exp2 - add support for share link message (#375)
  Revert accidentally pushed change
  Remove test that was dependant on the keychain
  Add item to keychain in the background (#380)
  macOS in-context signup updates (#359)
  Fixes the format in a log call (#379)
  Add EventMapping to DDGSync dependencies and call it from handleUnauthenticated (#376)
  Autofill hash migration updates + Bitwarden hotfix (#372)
  Autofill password generation support for iOS (#361)
  Update autofill to 7.1.0 (#370)
  Sync Engine with support for syncing bookmarks (#355)
  Centralised Messaging for User Scripts (#299)
  Update TLD accounts query to be more specific (#368)
  Bump Tests/BrowserServicesKitTests/Resources/privacy-reference-tests (#366)
  Adds a different hash Account per each macOS build kind (#363)
  adjust os_log parameter name using disfavoredOverload (#349)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants