Booting from UART is extremely inconsistent

[AlexandreTunstall]

Booting the same program over and over again seems to work consistently.
But most programs I tried for some unknown reason do not work.

For example, the following RV64GC program boots from UART:

.set DW_APB_UART, 0x10000000

_init:
    lla t0, int
    csrw mtvec, t0
_start:
    li x10, DW_APB_UART
    csrrs x11, mhartid, x0
    addi x11, x11, '0
loop:
    sb x11, 0(x10)
    j loop

int:
    li x11, 'I
    j loop

It outputs a stream of 0, 1, 2, 3, and 4, suggesting that the code at int never runs.

After removing the mtvec write, the program no longer boots:

.set DW_APB_UART, 0x10000000

_init:
    lla t0, int
_start:
    li x10, DW_APB_UART
    csrrs x11, mhartid, x0
    addi x11, x11, '0
loop:
    sb x11, 0(x10)
    j loop

int:
    li x11, 'I
    j loop

When I try to boot it, I receive a newline, (C)StarFive, another newline, and then the boot ROM seems to restart listening for a program over XMODEM, except that it no longer boots the working program until I press the reset button.

But if I instead assemble the broken program for RV64I_Zicsr, it boots normally and I get the expected 01234 output again.

This strange behaviour doesn't occur when booting from a microSD card.

---

I am using vf2-recover to transfer the program.
Could this be the result of incompatibility between my XMODEM client and the boot ROM's XMODEM client.

If not, what exactly is expected of programs booted from UART?