Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Salesforce] Add custom objects and fields support via Advanced Sync Rules #1938

Merged
merged 10 commits into from
Dec 5, 2023

Conversation

moxarth-rathod
Copy link
Contributor

@moxarth-rathod moxarth-rathod commented Nov 30, 2023

Relates to #1859

This PR contains the following Salesforce connector changes:

  • Include 1:1 Salesforce object representation in ElasticSearch doc
  • Ingest all custom objects and their associated fields
  • Ingest specific objects and fields by Advanced Sync Rules

Refer this Advanced Sync Rules discussion documentation for the schema: link

Checklists

Pre-Review Checklist

  • this PR has a meaningful title
  • this PR links to all relevant github issues that it fixes or partially addresses
  • if there is no GH issue, please create it. Each PR should have a link to an issue
  • this PR has a thorough description
  • Covered the changes with automated tests
  • Tested the changes locally
  • Added a label for each target release version (example: v7.13.2, v7.14.0, v8.0.0)

Release Note

This version of the Salesforce connector will change the elastic search document structure. The previous version uses the custom mapping for the salesforce objects but now we do 1:1 representation of Salesforce object in elastic search. The new document structure will be as following:

{
  "_id:: <Id>,
  "_timestamp": <LastModifiedDate>,
  "type": <type>,
  "owner": <ownerName>,
  "owner_url": <OwnerURL>,
  "title": <objectName>,
  "description": <description>
  "CreatedDate": <CreatedDate>,
  "Name": <objectName>,
  "Description": <description>,
  ...
}

Copy link
Member

@seanstory seanstory left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better! Left some suggestions to finish removing duplicate properties but to default to the Salesforce names for fields, rather than our own names.

Copy link
Member

@seanstory seanstory left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@moxarth-rathod moxarth-rathod merged commit ff0e91e into main Dec 5, 2023
@moxarth-rathod moxarth-rathod deleted the moxarth/salesforce-advanced-sync-rules branch December 5, 2023 17:24
Copy link

github-actions bot commented Dec 5, 2023

💔 Failed to create backport PR(s)

The backport operation could not be completed due to the following error:
There are no branches to backport to. Aborting.

The backport PRs will be merged automatically after passing CI.

To backport manually run:
backport --pr 1938 --autoMerge --autoMergeMethod squash

@leemthompo
Copy link
Contributor

@seanstory is this a breaking change?

@seanstory
Copy link
Member

@leemthompo yes, the structure changes would break any existing search experience or client code that relied on specific field names in the documents. Since this connector was not yet GA, PM determined that such a change was acceptable.

@leemthompo
Copy link
Contributor

Cool just wondering if we need to explicitly call it out as such. cc @ppf2

@ppf2
Copy link
Member

ppf2 commented Dec 19, 2023

I think it can be helpful to add something to the release notes, maybe under enhancements instead of breaking change (since the connector is not GA)?

What will happen when upgrading to 8.12? Will the existing search index fields be migrated to the new schema or will users have to recreate the connector and search index once on 8.12 and re-sync? @seanstory

@leemthompo
Copy link
Contributor

cc @danajuratoni

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants