[BUG] Channel Step sequenceNumber is not respected when POSTing

[prestontw]

Describe the bug
When posting a channel through the channels API, steps' sequenceNumber has lower priority than the type of step it is, leading to a misordering of steps.

To Reproduce
Setup steps (if required). Small reproducing repo is available here. The following steps will use this repo.
Example:

1. Install docker-compose, jq, Mirth Connect Administrator, and git
2. git clone https://github.com/prestontw/mirth-sequencenumber-repro.git
3. Within the repository, run docker compose up -d to start up the Mirth server within a docker container
4. Launch the Mirth Administrator (on localhost:8443)

Steps to reproduce the behavior:

1. Import the channel example_hl7_to_fhir_channel.xml
2. Note that the intermediate javascript log step is the second step (has sequence number 1)

3. Copy the channel locally using save-channel.sh in the directory
4. Note that, after formatting, there is no difference between this file and the previous version.
5. Note that the intermediate logging step has sequenceNumber: 1:

          "com.mirth.connect.plugins.javascriptstep.JavaScriptStep": [
          {
            "@version": "3.11.0",
            "name": "Intermediate js logging step",
            "sequenceNumber": 1,

6. Delete the channel

7. Import the channel to Mirth over the channel API with import-channel.sh
8. Refresh, open the channel, view the Source transformation
9. Note that the intermediate logging step is now a much later step

Expected behavior
After importing with import-channel.sh, the intermediate logging step should retain its original position in the transformation.

Actual behavior
After importing with import-channel.sh, the intermediate logging step is in a new position. Notably, it is now grouped with the other javascript step.

Screenshots
(pasted inline above)

Environment (please complete the following information):

• Host OS: MacOS
• Docker image: custom image with FHIR extension based on FROM nextgenhealthcare/connect:3.11
• Connect Version 3.11

Workaround(s)
Convert all of the steps to a common format, such as JS. This removes a lot of the benefits of the Mirth GUI, though.

Additional Context
Reproducing repo is available at https://github.com/prestontw/mirth-sequencenumber-repro

[cturczynskyj]

Tested this locally and it looks like this only happens when exporting the channel as JSON. For the time being if you stick to XML it should work properly, keeping your transformer steps in the correct order.

[prestontw]

@cturczynskyj is there a way to import and export the channel in xml format over the API? From the examples I've seen, it's only been using json.

[cturczynskyj]

@prestontw Yes, you can set the accept parameter to "application/xml". In your example code you are using curl so you can add the curl parameter -H "accept: application/xml".

[prestontw]

@cturczynskyj thank you. I have made changes in the repo above (prestontw/mirth-sequencenumber-repro@1e0f641) to reflect your hint. However, now I'm seeing that the Scripts of the channel are now losing their line endings, leading the return message of the preprocessing script being part of the comment, leading to preprocessing not actually returning the message:

I can work around this by not having a comment on the first line, but it would lead to issues if I have a comment anywhere at all. It also removes the newlines in general JS steps, leading to hard-to-edit code:

Exporting the channels in JSON was a workaround for XML not respecting newlines.

Do you have any recommendations? I'm on MacOS 11.4. The curious thing to me is that importing the channel through the GUI respects line endings. What is the difference between using the API and using the GUI? I know the GUI is implemented over the API, so it's unexpected to me that there would be a difference.

[prestontw]

It looks like it's an issue exporting the channel over the api: https://github.com/prestontw/mirth-sequencenumber-repro/blob/1e0f64171519b2f795bda2d1c85f885498207b28/saved-channel.xml#L1585 doesn't have a newline separating the comment from the return message;.

[cturczynskyj]

This may be how Curl is handling the POST body. I see you are using -d but I think you may also need to use $ before your body as well, like $'your_xml_body', to include new lines in the XML. Alternatively, you may be able to just use --data-binary instead of -d which should send the information exactly with no processing.

[prestontw]

Alternatively, you may be able to just use --data-binary instead of -d which should send the information exactly with no processing.

This worked, thank you so much @cturczynskyj ! This fixes the issue I had with exporting/importing over xml. Should we leave this issue open since the issue with importing over JSON still remains?

[cturczynskyj]

Glad you got it working with XML 👍

And, yes, we'll leave this issue open for now.

[prestontw]

@cturczynskyj one other note is that I can reproduce this with XML format too---if I reorder the steps, it imports them in that order rather than the order specified by sequenceNumber. If you're curious, I can try to copy to the linked repo, though it should be pretty quick to reproduce in any repo.