Global substitutions vs personalization substitutions

[gavrichards]

Issue Summary

I'm sending an email using a transactional template, passing in the content as it is a generic template, and sending it to multiple recipients, and I don't want each recipient to see who else it is going to. Each recipient will see a personalized "Hello {{first_name}}" above the content of the message.

Say I call addGlobalSubstitutions on a Mail object, with properties such as title, content and footer. I don't want these to vary between the recipients.

I then add a Personalization for each receipient, and on each one I call addSubstitution with a key of first_name and the value being the person's name.

What is then happening is the global substitutions are being ignored, and the emails are sending but there is no title, content or footer, just the "Hello {name}" part with the person's name.

Is it expected behaviour that by adding substitutions to personalizations, the global substitutions are not used at all? I was hoping that they would individually override, so the first person would get title, content, footer, and first_name (with their name), the next person the same but with their name, etc, etc.

Technical details:

• sendgrid-php Version: 7.2.1
• PHP Version: 7.2.8

[edwardrbaker]

I am experiencing something similar but with custom_args.
I am intending to add a global custom_args object, but the args are only added to the first Personalization. I have an example test that shows this, and I'm curious if it's intentional or not.

    public function testCustomArgsInNewMessage()
    {        
        $email = new \SendGrid\Mail\Mail();

        $customArgs = [
            "marketing2" => "true",
            "transactional2" => "false",
            "category" => "name"
        ];
        $email->addCustomArgs($customArgs);
        $json = json_encode($email->jsonSerialize());
        
        $this->assertEquals(count($email->getCustomArgs()), 3);
        $this->assertEquals(count($email->getCustomArgs(0)), 0); // this assertion fails; why is this 3?
    }

[gavrichards]

@erbaker Have you tried using addGlobalCustomArgs instead?

[edwardrbaker]

@gavrichards yeah that did it _

This does seem not quite as intuitive, but I think I understand why it was done that way. Thanks!!

[thinkingserious]

Hi @gavrichards,

Generally, I recommend folks to follow our transactional template example.

However, if you can either provide the code or the request body, I can try to reproduce. Thanks!

With Best Regards,

Elmer

[gavrichards]

@thinkingserious but the example doesn't make use of global substitutions which is the key here.

I've cleaned up a version of my code for you to try. Thanks!

$fromEmail = 'from@example.com';
$fromName = 'Example McExampleFace';
$subject = 'Subject';
$text = 'Text email';
$templateId = 'abc123';
$globalSubstitutions = [
    'subject' => $subject,
    'content' => nl2br($text),
];
$to = [
    'gav@example.com',
    'sarah@example.com',
    'dave@example.com',
];
$substitutions = [
    'first_name' => [
        'Gav',
        'Sarah',
        'Dave',
    ],
];

$sendgrid = new Sendgrid($apiKey);

$mail = new SendGrid\Mail\Mail(
    new SendGrid\Mail\From($fromEmail, $fromName),
    null,
    new SendGrid\Mail\Subject($subject)
);

$mail
    ->addContent('text/plain', $text);

$mail
    ->setTemplateId($templateId);

$mail
    ->addGlobalSubstitutions($globalSubstitutions);

$i = 0;
foreach($to as $address)
{
    $personalization = new SendGrid\Mail\Personalization();
    $personalization
        ->addTo(new SendGrid\Mail\To($address));
   
    foreach($substitutions as $key => $vals) {
        $personalization
            ->addSubstitution($key, $vals[$i]);
    }

    $mail
        ->addPersonalization($personalization);

    $i++;
}

$sendgrid
    ->send($mail);

[thinkingserious]

Hello @gavrichards,

I think I see the issue, can you try building your mail objects without passing parameters to the constructor like so?

When you use the constructor with arguments we create an initial Personalization object behind the scenes at index 0.

Also, looking at the request body may also offer some clues.

Please take a look and let me know the results. Thanks!

With Best Regards,

Elmer

[gavrichards]

@thinkingserious thanks - I thought the arguments in the constructor were required, has that changed at some point, or was I incorrect in the first place?
I'll give this a try and report back.

[thinkingserious]

I believe in previous releases they were required.

[gavrichards]

@thinkingserious
I've changed the constructor part to:

$this->mail = new SendGrid\Mail\Mail();

$this->mail
    ->setFrom($fromEmail, $fromName);

$this->mail
    ->setSubject($this->subject);

I then used the technique for inspecting the request body, which resulted in the following (I have redacted the values for privacy):

{
    "personalizations": [
        null,
        {
            "to": [
                {
                    "email": "[redacted]"
                }
            ],
            "dynamic_template_data": {
                "first_name": "[redacted]"
            }
        },
        {
            "to": [
                {
                    "email": "[redacted]"
                }
            ],
            "dynamic_template_data": {
                "first_name": "[redacted]"
            }
        },
        {
            "to": [
                {
                    "email": "[redacted]"
                }
            ],
            "dynamic_template_data": {
                "first_name": "[redacted]"
            }
        },
        {
            "to": [
                {
                    "email": "[redacted]"
                }
            ],
            "dynamic_template_data": {
                "first_name": "[redacted]"
            }
        },
        {
            "to": [
                {
                    "email": "[redacted]"
                }
            ],
            "dynamic_template_data": {
                "first_name": "[redacted]"
            }
        }
    ],
    "from": {
        "name": "Aiir",
        "email": "[redacted]"
    },
    "reply_to": {
        "email": "[redacted]"
    },
    "subject": "Subject line here",
    "content": [
        {
            "type": "text\/plain",
            "value": "[redacted]"
        }
    ],
    "template_id": "[redacted]",
    "categories": [
        "[redacted]"
    ],
    "substitutions": {
        "subject": "Subject line here",
        "preheader": "[redacted]",
        "title": "[redacted]",
        "content": "[redacted]",
        "footer": "[redacted]"
    }
}

It looks like there is still a blank initial personalization, where it says null.

[thinkingserious]

@gavrichards,

Could you please verify which version of this SDK you are using?

Also, as a quick workaround, you could add a personalizationIdex of 0 to attach to that default Personalization object.

[gavrichards]

@thinkingserious 7.2.1. Does the latest version offer any fixes for this?

Where would I use the personalizationindex of 0 that I'm not already?

[thinkingserious]

@gavrichards,

I was thinking that perhaps you were on a version < 7. In the current version, I'm not seeing this behavior (a null personalization) in our tests, so perhaps I need to expand that test. I must be missing some edge case.

With regards to the Personalization index, I was referring to the code you first provided 3 days ago. In your loop, for the first personalization object you build, instead of building a new object, you would add those values to the first personalization. This might be more clear if you take a look at the request object generated by that code example.

[gavrichards]

@thinkingserious What I'm confused about is you said:

When you use the constructor with arguments we create an initial Personalization object behind the scenes at index 0.

So I took all arguments out of the constructor, presuming therefore that it doesn't create an initial Personalization object.

Therefore my code from a few days ago should work ok, where I create the first personalization object in the loop? But it is that that is resulting in the first personalization being null.

[thinkingserious]

My apologies for the confusion @gavrichards.

What I'm trying to say is that you are correct, it should work if you take all the arguments out of the constructor. So I think there may be a bug. I am suggesting that you revert to your original code, with the caveat I mentioned, as a workaround while we debug.

I hope that helps!

[gavrichards]

@thinkingserious I've tried this, and while the first personalization isn't null anymore, the substitutions for that personalization aren't there.

e.g.

$address = 'andy@example.com';
$subs = [
  'first_name' => 'Andy'
];
$mail->addTo(new SendGrid\Mail\To($address), null, $subs, 0);

Results:

    "personalizations": [
        {
            "to": [
                {
                    "email": "andy@example.com"
                }
            ]
        }, ...

Subsequent personalizations do have their substitutions, so it appears to just be an issue with the first personalization.

[thinkingserious]

Thanks for the follow up @gavrichards,

It looks like there is a logic bug somewhere. The workaround I'd suggest until we get this sorted is to manually pass in the JSON object like so.

Thank again for taking the time and having the patience to work through this. All of the details you provided should help us pinpoint the issue.

With Best Regards,

Elmer

[lfolco]

This doesn't seem to be fixed? I am sending the following data in my request and the global substitution is not working:

{
    "personalizations": [
        {
            "to": [
                {
                    "email": "user1@example.com"
                }
            ],
            "substitutions": {
                "-EHASH-": "Subscriber1Hash"
            },
            "custom_args": {
                "subscriber_id": "39"
            }
        },
        {
            "to": [
                {
                    "email": "user2@example.com"
                }
            ],
            "substitutions": {
                "-EHASH-": "Subscriber2Hash"
            },
            "custom_args": {
                "subscriber_id": "40"
            }
        }
    ],
    "from": {
        "name": "Owner",
        "email": "me@laurafolco.com"
    },
    "subject": "Test email",
    "content": [
        {
            "type": "text\/html",
            "value": "<p>Here is a test email"
        }
    ],
    "categories": [
        "author-initiated-update-notification"
    ],
    "custom_args": {
        "author_ids": "20",
        "product_id": "96",
        "version_id": "40",
        "content_id": "20"
    },
    "substitutions": {
        "-YEAR-": "2021"
    }
}

When I get the email, I see the text -YEAR- instead of the value.