Not adding Bot or BotVersion

[marioconcilio]

Issue verification check :

• does the current repository is fully deployable at the commit sha given to the from parameter of the command ?

What is the problem

---

We are looking into using sgd to automate our deployments. During testing, we noticed that neither bots nor botVersions are being added to the package, although they have changed.

What is the parameter and their value you used

sfdx sgd:source:delta \
    --from $(git merge-base issue/219 master) \
    --to HEAD \
    --output changed_sources \
    --api-version=53.0 \
    --source sgd \

What is the expected result

<?xml version="1.0" encoding="UTF-8"?>
<Package xmlns="http://soap.sforce.com/2006/04/metadata">
    <types>
        <members>Test</members>
        <members>Test.xml</members>
        <name>ApexClass</name>
    </types>
    <types>
        <members>BR_WhatsApp_Bot</members>
        <name>Bot</name>
    </types>
    <types>
        <members>BR_WhatsApp_Bot.v25</members>
        <name>BotVersion</name>
    </types>
    <version>53.0</version>
</Package>

What is the actual result

<?xml version="1.0" encoding="UTF-8"?>
<Package xmlns="http://soap.sforce.com/2006/04/metadata">
    <types>
        <members>Test</members>
        <members>Test.xml</members>
        <name>ApexClass</name>
    </types>
    <version>53.0</version>
</Package>

Steps to reproduce

---

1. Branch off from master
2. Add a new bot and a botVersion to the sfdx project.
3. Commit the new bot-meta.xml and botVersion-meta.xml files
4. Run SGD with the following parameters (comparison copied from the 3rd topic of the Diff between branches section:

--from $(git merge-base issue/219 master) # common ancestor
--to HEAD
--output changed_sources
--api-version=53.0

Execution context

---

Operating System: MacOs Catalina (Darwin Kernel Version 19.4.0; root:xnu-6153.101.6~15/RELEASE_X86_640)

node version: v16.13.1

git version: git version 2.24.3 (Apple Git-128)

sfdx version: sfdx-cli/7.129.0 darwin-x64 node-v16.13.1

sgd plugin version: sfdx-git-delta 4.11.0

Optional more information

---

git diff --name-status --no-renames $(git merge-base issue/219 master) HEAD

Output:

A       sgd/reproduction/playground/bots/BR_WhatsApp_Bot/BR_WhatsApp_Bot.bot-meta.xml
A       sgd/reproduction/playground/bots/BR_WhatsApp_Bot/v25.botVersion-meta.xml
A       sgd/reproduction/playground/classes/Test.cls
A       sgd/reproduction/playground/classes/Test.xml

[marioconcilio]

I tried to push my example to https://github.com/scolladon/sfdx-git-delta-reproduction-playground, but I have no access.
BTW, that is a great plugin, nice job!

[scolladon]

Hi @marioconcilio !

Thanks for this very well documented issue !
Thanks for your help to make this plugin better !

You should now have access to the reproduction playground, if you could create a branch there (issues/219 per example) and put just what it needs to have to isolate the issue it would be great.

I'll look at it soon and let you know here my findings !

Thanks for your compliments about the plugin, it is always very nice to hear that, thank you from the whole team

[marioconcilio]

Hi @scolladon! Thanks for the prompt response!
I cloned the repo and created the branch, but I still don't have permission to push my changes. Can you double-check it, please?

[scolladon]

You should have receive an email to accept the access to the repo
here is the link

[marioconcilio]

Oops, my mistake! I've just pushed the new branch. I am at your disposal to test the new solution and clarify any doubts.

Thanks again for your attention!

[scolladon]

Thanks for providing data to isolate the behaviour

I am trying to compare what is in the reproduction-playground with what the doc says

Bot and BotVersion are already in our metadata file so they should be handled (and they are but not with this form).

What I can see with this issue and the reproduction playground is that it is expected Bot and BotVersion to be retrieved each in their own files.
What looks weird to me is how to deduce to which Bot the BotVersion is related to?
Because the name contains only the version number and not the Bot to which it is related (the file does not contains any reference either). Then the name would be taken from the subfolder where it is located I guess ?

But when I look at the doc I can see only one file (.bot) containing the botVersion definition.

But ! the Bot suffix and Directory location section can also be understood as if there is 2 different files, one of the bot definition and one for the bot version, with the same file extension... (weird)

@marioconcilio if you have any experience with that, answers to those questions, documentations, please share with us.
And also, how did you to get those files ? with which api version ? Does the result is the same on every API version which support Bot and BotVersion metadata ?

[mehdicherf]

@scolladon I've performed a few tests, and I confirm that a bot is actually composed of several files (one .bot-meta.xml and one or several .botVersion-meta.xml). I believe the doc is misleading regarding the suffix of the BotVersion.
Those files are located within the same sub-folder (named as the bot) inside the bot folder.

For example, I've created 3 bots in a scratch org, and here is the retrieved metadata after running a sfdx force:source:pull:

• 3 subfolders (1 per bot) are created within the bot folder:
  
• each subfolder is composed of several files (one .bot-meta.xml, and one .botVersion-meta.xml per version)
  

[marioconcilio]

Hi @scolladon, sorry for the late response.

What looks weird to me is how to deduce to which Bot the BotVersion is related to?
Because the name contains only the version number and not the Bot to which it is related (the file does not contains any reference either). Then the name would be taken from the subfolder where it is located I guess ?

The botVersion is related to the Bot that is on the same subfolder. It is just as @mehdisfdc stated. The documentation is indeed misleading regarding the bot suffix.

And also, how did you to get those files ? with which api version ? Does the result is the same on every API version which support Bot and BotVersion metadata ?

Bots are retrieved using the following command:
sfdx force:source:retrieve -m Bot:BR_WhatsApp_Bot

This retrieves the bot-meta.xml file and all its version files (botVersion-meta.xml):

FULL NAME            TYPE        PROJECT PATH
───────────────────  ──────────  ────────────────────────────────────────────────────────────────────────
BR_WhatsApp_Bot      Bot         force-app/main/default/bots/BR_WhatsApp_Bot/BR_WhatsApp_Bot.bot-meta.xml
v25                  BotVersion  force-app/main/default/bots/BR_WhatsApp_Bot/v25.botVersion-meta.xml

Additionally, it is possible to retrieve a single bot version running:
sfdx force:source:retrieve -m BotVersion:BR_WhatsApp_Bot.v25

This retrieves a single file representing the bot version (botVersion-meta.xml):

FULL NAME  TYPE        PROJECT PATH
─────────  ──────────  ───────────────────────────────────────────────────────────────────
v25        BotVersion  force-app/main/default/bots/BR_WhatsApp_Bot/v25.botVersion-meta.xml

[scolladon]

Hi @marioconcilio !

I just created a new version (currently under review).

Could you try it (how to here) and give feedback ?

[marioconcilio]

Hi @scolladon!

I tested and it worked! The plugin added the bot and botVersion to the changed sources.
There is just one detail - and I forgot to mention: botVersions cannot be deployed alone, they need the bot file as well, just like it happens with fieldTranslations (they need the objectTranslation too, and the sgd plugin automatically includes them).

Can you please make sgd automatically include bot files whenever a botVersion is added to the package?

[scolladon]

Hi @marioconcilio !

I just added this behaviour in the PR #226
Ready for you to check how it works.
I added a commit in the reproduction-playground issue/219 branch to isolate the behaviour

Let me know !

[marioconcilio]

Hi @scolladon!

I just tested and the bot-meta.xml files are being copied to the right place when running with the --generate-delta flag. However, the package.xml is wrong when we only add/change a botVersion (botVersion-meta.xml files)

<types>
    <members>BRAcquisitionWhatsAppBot</members>
    <members>BR_WhatsApp_Bot</members>
    <members>Sam_Web_IT</members>
    <name>bots</name>
</types>
<types>
    <members>BRAcquisitionWhatsAppBot.v5</members>
    <members>BR_WhatsApp_Bot.v26</members>
    <members>Sam_Web_IT.v12</members>
    <name>BotVersion</name>
</types>

The expected output is:

<types>
    <members>BRAcquisitionWhatsAppBot</members>
    <members>BR_WhatsApp_Bot</members>
    <members>Sam_Web_IT</members>
    <name>Bot</name>
</types>
<types>
    <members>BRAcquisitionWhatsAppBot.v5</members>
    <members>BR_WhatsApp_Bot.v26</members>
    <members>Sam_Web_IT.v12</members>
    <name>BotVersion</name>
</types>

When we change only the bot (bot-meta.xml files), the package is generated accordingly:

<types>
    <members>BR_WhatsApp_Bot</members>
    <name>Bot</name>
</types>

Please let me know when it is ready to test!

[scolladon]

@marioconcilio nice catch 👁️ !!

Thanks for the review I have pushed a fix to address that.
Could you try again and let me know?
I have also pushed my test data in the reproduction playground

[marioconcilio]

It is working perfectly fine @scolladon !

I tested creating a new bot, and the sgd:

• copied the bot and botVersion files to the changed sources folder accordingly
• created a valid package.xml with Bot and BotVersion types

I also tested creating just a new bot version, and the sgd:

• copied the bot and the new botVersion files to the changed sources folder accordingly
• created a valid package.xml with Bot and the new BotVersion types

Additionally, I tested deleting one single bot version, and the sgd:

• created a valid destructiveChanges.xml with the BotVersion type to delete

Finally, I tested deleting the entire bot, and the sgd:

• created a valid destructiveChanges.xml with the Bot and the BotVersion to delete

In summary, sfdx-git-delta is working:

• to create a new Bot ✅
• to create a new BotVersion ✅
• to delete a single BotVersion ✅
• to delete an entire Bot (w/ its BotVersions) ✅

Thanks a lot for this huge fix! I am looking forward to seeing this new version published!

[scolladon]

Thank you very much for the QA and all the help !

[mehdicherf]

Thank you @scolladon for the code, and thank you @marioconcilio for find the issue and testing the new version! The plugin is getting better every week 😄

[scolladon]

New version (4.12.0) available !