Trying to achieve max bandwidth results in (some) packets not being sent

[birkenfeld]

Basically, we identified smoltcp-rs/smoltcp#319 as not smoltcp's fault.

If you will have time to check this out with a reproducer, I will boil down my code to the bare minimum.

[dtjones190]

I had an issue where when I compiled in release mode only every other ping was being responded to.
I ended up adding a Data Synchronization Barrier in the send method of Eth and that fixed it.
See the following change:
dtjones190@39d9d9b#diff-b4aea3e418ccdb71239b96952d9cddb6

And the breadcrumbs that led me there:
https://community.st.com/s/question/0D50X00009Xkgff/stm32f746-ethernet-dma-transmit-problem
https://docs.rs/cortex-m/0.5.8/cortex_m/asm/fn.dsb.html

[thalesfragoso]

@birkenfeld Are you using a F7 or F4 ? If using a F7, could you try the master branch to see if your problem persists ? This might also be caused by the lack of compiler_fence/fence in the proper places (even on F4s), I will hopefully address this on a PR soon.

Edit: Also, could you see if #16 applies to your case ?

[birkenfeld]

I'm using an F429. Thanks for the suggestions, I will try master and stuff from #16 when I'm back at work.

[thalesfragoso]

@birkenfeld You should probably test #17 instead of master now. Let me know if you find any problems.

[birkenfeld]

Unfortunately, I tested with current master, with the "fence" feature both disabled and enabled, and could reproduce the problem.
I was using HCLK 180 MHz (see the PR I just submitted), but I see the same with 168 MHz.

Is #16 still relevant? Is there an example how to put the buffers in SRAM2?

[thalesfragoso]

Is #16 still relevant? Is there an example how to put the buffers in SRAM2?

Yes, it's still relevant, you can see an example here: rust-embedded/cortex-m-rt#282, but you should use SRAM2/3 instead, also, beware that cortex-m-rt won't initialize these extra sections, so you need to do so, probably in #[pre_init]. Not sure if that will fix the problem, but it can at least reduce contention, which seems important in your use case.

Could you provide the code (a reduced one, preferably) so we can see if we can reproduce it ? That together with your memory.x.

[birkenfeld]

Will do; should be ready later today.

[thalesfragoso]

Oh, I just realized that the f4xx-hal does enable the data cache, maybe you could try disabling it and see how it goes, however, for performance reasons, the best solution is indeed to leave the data cache enabled and use SRAM2/3.

[birkenfeld]

I've put up the reproducing example as #21, I've not yet experimented with dcache.

[thalesfragoso]

Ok, update to this issue:

Right now smoltcp puts the socket in a Waiting state after sending the ARP request, which causes the socket to stall indefinitely if we don't get a response, this isn't intended and a fixed is needed to make it re-send the ARP after some specific timeout.

Now, there is still one more question, do we ever send the ARP packet ? Because we tell smoltcp that we do (Edit: that wasn't the case, see smoltcp-rs/smoltcp#369), if we don't then there is a bug on our side (although it shouldn't alone cause the indefinitely stall). Another possibility is that the packet gets lost somewhere since the problem appears when dealing with a saturated link.

Edit: Also, after talking to adam, I came to the conclusion that caching shouldn't be a problem on the F4, since it only caches the flash.

[thalesfragoso]

Closing this as the problem was tracked down in smoltcp, issue here smoltcp-rs/smoltcp#319, and fixes here smoltcp-rs/smoltcp#368 and here smoltcp-rs/smoltcp#369.