HeaderReroutingAgent
This sample implements a RoutingAgent. This code executes when the message has been received from the Transport Front-End, put into the queue and is in the process of being categorized by the Hub Transport service.
This agent has been written to intercept messages that have a specific header set and re-route them via a separate Send Connector.
The organization had a need to avoid routing messages from mass-mailing domains via the default path (MX record of the recipient) and via Exchange Online, which at this time doesn't support outbound High-Volume Email Traffic. The matching condition here is checking for the presence of one specific header, "X-HeaderReroutingAgent-Target" which must contain the desired routing domain (i.e. ACS.TONIOLO.CLOUD) that has to match the Send Connector.
In Exchange Server there is a Send Connector, in this case matching ACS.TONIOLO.CLOUD, configured to use Azure Communication Services Email as an upstream Smart Host.
Other SMTP compliant services can be used.
It must be noted that for intra-organization messages, this Transport Agent can be used to route messages to Office 365 HVE, which allows internal mass mailing.
The agent offers two separate methods, one that handles the Routing Domain override, and another that manages the removal of empty message headers, given services like Azure Communication Services Email might not accept empty message headers. At this time the triggering condition is the same, so any message for which routing will be overridden, will also get empty headers stripped out.
This means the agent implements Conditional Routing capabilities in Exchange Server, thus allowing to separate traffic between different ACS or HVE services, targeting different subscriptions or tenants.
It must be noted that this Transport Agent is capable of intercepting messages regardless of the protocol used for generating the outbound message (Exchange Web Services, SMTP, Outlook, OWA, ActiveSync, Outlook, Mobile, etc.) and likewise will be able to process messages regardless of how the Exchange server received them (TLS-based connector, IP-based connector, SMTP submission, anonymous connector, basic authentication, pickup directory).
This, as such, can provide way to leverage specific egress or mass-mailing services, like Azure Communication Services, even from applications that cannot directly interact with it or that are not natively compatible, and transparently for the application owners and developers.
Please take into consideration that the agent does not process messages even if the header is set if:
- The message is a Non-Delivery Report or a Delivery Status Notification, or if it's a System Message
- The agent will not process messages that had been already processed, this is done to prevent mail loops
This apply to all the capabilities of the Transport Agent (routing override, P1/P2 update, header removal).
First you need to configure your Azure Communication Services or SMTP compliant service. For Azure Communication Services Email please refer to the official documentation here.
Once you've followed the article you shall have your username (in the format AzureCommunicationServicesResourceName|EntraApplicationID|EntraTenantID) and the password (the client secret for the registered application).
You then need you to create a new Send Connector and set smtp.azurecomm.net and the username/password retrieved at the previous step as shown in the following image. Note that the Exchange Admin Centre doesn't allow setting a target port (587, in this case) for the Send Connector. ACS however work also with port 25, therefore you can skip updating the port to 587.
At this point you need to log on to the Exchange Management Shell and update the newly create Send Connector to use port 587, that is used in Azure Communication Services (and other SMTP-compliant services) for client submission. The image below shows the Get- command to allow viewing all the properties and their configuration; use the Set-SendConnector to modify the "Port" attribute.
Now all pre-requisites will be met and the Transport Agent will be able to re-route messages.
First you need to configure your High Volume Email service in the Office 365 Admin Center. Please refer to the official documentation here.
Once you've followed the article you shall have your username and the password associated to the email address of choice.
You then need you to create a new Send Connector and set smtp-hve.office365.com and the username/password retrieved at the previous step as shown in the following image. Note that the Exchange Admin Centre doesn't allow setting a target port (587, in this case) for the Send Connector. For HVE you have to mandatorily perform this step as port 587 must be used.
At this point you need to log on to the Exchange Management Shell and update the newly create Send Connector to use port 587, that is used in HVE (and other SMTP-compliant services) for client submission. The image below shows the Get- command to allow viewing all the properties and their configuration; use the Set-SendConnector to modify the "Port" attribute.
Now all pre-requisites will be met and the Transport Agent will be able to re-route messages.
There are at this point multiple ways to make the agent trigger, as it just requires the header to be set with appropriate value. This can be done directly at origin (i.e. on the business applications that generates the message) or for a greater flexibility this can be done with Transport Rules.
Transport Rules offers the ability to trigger on multiple conditions, can be mutually exclusive, allows for exceptions and cater for consecutive processing and overlap, allowing for the most flexibly.
The example below shows how to redirect messages from NoReply@Toniolo.cloud.
It is possible to target different routing domains based on different conditions, therefore allowing to trigger separate Azure Communication Services Email (or compliant services).
Microsoft Exchange Servers performing the Edge role don't have the same capabilities of Mailbox servers. Specifically, it is not possible to create rules that use the "From" (The sender is) condition, and the only option is to rely either on the FromAddressContainsWords or the FromAddressMatchesPatterns attributes to scope the Transport Rule.
You want to use the FromAddressContainsWords condition when you want to match a substring. For example, if you set the condition to match "noreply@toniolo.cloud", this will also match "test-noreply@toniolo.cloud", "123noreply@toniolo.cloud", and "string.noreply@toniolo.cloud". You want to be careful and evaluate your existing traffic prior to using this condition, as otherwise the rule might catch unwanted senders.
On the other hand, FromAddressMatchesPatterns is RegEx based and therefore offers more flexibility for pattern matching. With this you can replicate the "From" preficate by prepending the string to be matched with "^" and terminating the string with "$". This will allow you to scope the rule tightly and match exact addresses. For example, if you set the matching string to "^noreply@toniolo.cloud$", then only this one address will be matched.
In Hybrid deployments this Transport Agent can be used in conjunction to Conditional mail routing in Exchange Online or Centralized Mail Transport. In such case messages that match a certain criteria can be sent from Exchange On-Line to Exchange On-Premises for further processing, and the Transport Agent can override the egress path to leverage SMTP compliant services such as Azure Communication Services Email.
For this example, a new test account with SMTP Address Cloud_NoReply@Toniolo.cloud has been created in Exchange Online, this just to simplify testing as messages can be submitted via OWA.
For the example, the header X-HeaderReroutingAgent-Target is set via Transport Rule and the same Transpor Rule also redirect the message via a newly created, custom, Outbound Connector.
The Outbound Connector is set to use the On-Premises Exchange Server as a smart-host and to use TLS validation.
At this stage, when Exchange On-Premises receives the message, the Transport Agent detects the presence of the header and overrides the routing domain to the value in the header (in this case ACS.TONIOLO.CLOUD). As described in the Initial Configuration this domain is associated to a specific Send Connector that uses ACS as upstream smart-host.
In this way, in Hybrid Deployments, it is possible to seamlessly redirect messages to Azure Communication Services regardless of their origin (Cloud or On-prem) and irrespective to how the line-of-business application sends the message (GRAPH API, EWS, Outlook, SMTP, etc.).
This approach also has the added benefit of allowing messages to be evaluated against Data-Loss Prevention Policies that might exist.
As the Transport Rules executes as part of the Transport Pipeline, the priority of agents can be changed. It is imperative, for the correct execution of this process, that the "Transport Rule Agent" executes before the custom one.
Considering a scenario where the P1 Sender (envelope) might different from the P2 Sender (message), a new control header has been implemented. This transport agent offers the capability to remediate the P1 Sender (to be set to the P2 value) or the P2 Sender (to be set to the P1 value), addressing effectively the mismatch and ensuring the two are aligned.
This can be achieved by ensuring that the message contains the "X-HeaderReroutingAgent-P1P2MismatchAction" header and this has any of the following case-insensitive values:
- "UseP1"; overrides the P2 Sender (message) with the value set on P1 (envelope)
- "UseP2"; overrides the P1 Sender (envelope) with the value set on P2 (message)
- "None"; takes no action, similarly to when the "X-HeaderReroutingAgent-P1P2MismatchAction" is missing.
This becomes prominently useful when the Transport Agent has to reroute messages to a SMTP-compatible services (such as Azure Communication Services Email) which validates the sender validity based on P1 (message envelope, the value of the MAIL FROM SMTP verb). Often times line of business applications, such as alerting and moniutoring systems, might be omitting the P1 Sender altogether, or might have a value such as application@localhostname or application@localdomain with even non-routable domains and for which bulk-mail services would fail validation.
Very similarly to what described in the earlier paragraph, even this header can be set directly in the client application (if supported by the application) or stamped by a Transport Rule. An example of rule follows:
Having faced a scenario where the P1 Sender was not set at all, or where it was not possible to set the P1 Sender to the desired value as a consequence of lack of customization on the service/application sending the messages, a new control header has been implemented.
If the header "X-HeaderReroutingAgent-ForceP1" is present, and its value is set to a valid SMTP Address, the Transport Agent will override the P1 Sender to the value provided via the header. This works both in scenarios where the original sender is set to an invalid value or when the original sender is missing.
As depicted for the earlier scenarios, this header can be set directly in the source system (i.e. Line of Business Application) or can be set, for convenience, via Transport Rules. An example follows.
This Transport Agent implements Event Logging for its operation. This has been chosen at it diminishes the chances of crashing the transport service in scenarios such as when a log file (if logging to text) could be locked by another process (i.e. anti-virus) or not accessible (i.e. missing permissions or non-existing path).
Event Logs would be recorded in the Application Event Log on the server; the agent would try to register a Source matching its name (HeaderReroutingAgent), if this is not possible it will try to fall back on the DLL name (TransportAgents) and, in case that is not possible as only applications running elevated has the privilege to do so, will fall back to logging entries as "Application" which should be already present and registered.
If you want to make sure the Transport Agent is capable or logging events under its own, separate, source for a quicker identification of the relevant events, please create the source manually on an elevated PowerShell prompt.
This can be done with the following command.
New-EventLog -LogName Application -Source HeaderReroutingAgent
Alternatively, if you want all different agents to log their events on a single source, you can create the TransportAgents one as follows.
New-EventLog -LogName Application -Source TransportAgents
By default this Transport Agent has debug logging disabled. Verbose/Debug log can be abled by inserting a the necessary key in the Windows Registry. As Exchange Transport runs as Network Service, the registry hive and key have to be created under HKEY_USERS\S-1-5-20.
The Registry Path that the Agent checks is the following:
HKEY_USERS\S-1-5-20\Software\TransportAgents\HeaderReroutingAgent
The key(s) that controls the behaviour is instead:
DebugEnabled
DebugEnabled-OverrideRoutingDomain
DebugEnabled-OverrideSenderAddress
DebugEnabled-RemoveAllHeaders
If the path or key(s) are missing, Debug logging is disabled, and the Transport Agent will only log Warnings and Errors. Likewise if the keys are misconfigured (i.e. have a value which is neither True nor False), logging will remain disabled. Only when the value is set to "True" debug logging will be enabled and all the information will be logged.
Note: Please note that the ability of leverage per-method logging is available only if DebugEnabled is either False or not set. DebugEnabled overrides any per-method logging preference and therefore will enable verbose logging across the whole Transport Agent.
For example:
DebugEnabled == false && DebugEnabled-OverrideRoutingDomain == true: logging is enabled only on OverrideRoutingDomain
DebugEnabled == null && DebugEnabled-OverrideSenderAddress == true: logging is enabled only on DebugEnabled-OverrideSenderAddress
DebugEnabled == true: logging is enabled across all methods
DebugEnabled == true && DebugEnabled-OverrideRoutingDomain == false: logging is enabled across all methods
DebugEnabled == false: logging is disabled
Example of the registry configuration follows.
- Copy the DLL to the server (i.e. F:\Transport Agents); Make sure the Exchange accounts have access to the folder
- Install the transport agent
Install-TransportAgent -Name HeaderReroutingAgent -TransportAgentFactory "TransportAgents.HeaderReroutingAgent" -AssemblyPath "F:\Transport Agents\TransportAgents.dll"
- Enable the chosen Transport Agent
Enable-TransportAgent HeaderReroutingAgent
- Exit from Exchange Management Shell
- Restart the MSExchangeTransport service
Restart-Service MSExchangeTransport
- Disable the agent
Disable-TransportAgent HeaderReroutingAgent
- Remove the agent
Uninstall-TransportAgent HeaderReroutingAgent
- Exit from Exchange Management Shell
- Restart the MSExchangeTransport service
Restart-Service MSExchangeTransport