Testing

[nickray]

The stm32-rs crates are depended on by many. Perhaps we should store their outputs with git lfs or similar, and run regression tests against current last release (or HEAD?) of stm32-rs?

[jessebraham]

I took a swing at this last night. Unfortunately, it seems (according to this comment and others) that GitHub doesn't allow pushing files to LFS on public forks unless the original repo already has LFS objects, so I immediately hit a bit of a wall.

Basic idea was to store all of the known-good patched SVDs in tests/svds using LFS, then Travis can pull down the stm32-rs repo and patch its SVDs using the current build of svdtools. I added a basic script that diffs the SVDs for each device.

I also am (temporarily) cloning from your stm32-rs fork to pull down the changes made to the Makefile when splitting out svdtools, but we'd want to check out a tag or a specific commit once those changes have been merged.

Basically just documenting my attempt, my testing branch can be found here for reference:
https://github.com/jessebraham/svdtools/tree/testing

[nickray]

I'm not quite sure using LFS is the right approach. Do we really want any non-example SVDs in this repo? Whether directly or via LFS.

I was thinking we might add a directory that users of the repository can PR a self-contained script into, that does whatever, with some notion of limited resource use, assuming network access, in a gitignored subdirectory. Basically offload the work on users 😇.

Ideally, these scripts would re-use the CI setups of the dependent repositories, so maybe they would git checkout themselves, install proposed HEAD of svdtools (via flit install --symlink --python <virtual env of repo>), and run their own CI regression scripts?

In the case of STM32, maybe we can patch something together using:

• https://stm32.agg.io/rs/ (contains patched SVDs, e.g. https://stm32.agg.io/rs/stm32f103.svd.patched)
• https://github.com/stm32-rs/stm32-rs-nightlies

I'm not quite sure which version of the SVDs stm32.agg.io contains (last published or nightly), @adamgreig?

[nickray]

The other question is which CI to use. I've not used it, but heard good things about CircleCI's local CI. For my personal projects I've been using builds.sr.ht recently since it's "just" scripts on a variety of distributions.

[adamgreig]

I'm not quite sure which version of the SVDs stm32.agg.io contains (last published or nightly)

There's a cronjob that runs every 15min to git pull && make html, so they're basically last published. There's no technical reason the stm32-rs-nightlies repo couldn't also include the generated patched SVDs alongside the source code, which might be simpler, but some people have raised concerns about redistributing modified SVDs so I've shied away from it (on GitHub) so far.

The other question is which CI to use.

I used GitHub Actions for the stm32-rs-nightlies which was pretty simple in the end, just having this file exist makes everything work: https://github.com/stm32-rs/stm32-rs/blob/master/.github/workflows/nightlies.yaml

[brainstorm]

(...) but some people have raised concerns about redistributing modified SVDs so I've shied away from it (on GitHub) so far.

Why is that @adamgreig? Ideally if we keep patching against an official rust-embedded fork of https://github.com/posborne/cmsis-svd, we could potentially reach a (very?) desirable "steady state" where all SVD files were good/final with no need for further patching?

... or perhaps I'm too blue-eyed when looking at this whole SVD problem overall for the first time? ;)

[adamgreig]

I appreciate you taking a fresh look at the problem!

The concern around redistribution of modified SVDs is about their copyright. I don't know how significant the concern is.

I think a big concern with trying to reach a steady state where all the SVD files are good is the scope of the task (for example, stm32-rs is only one vendor, is not even halfway done, but is already 91 SVD files, some 40000 lines of YAML to patch from 72 contributors over 1400 commits), but another problem is that there's a lot of decisions to make about how the represent the hardware and how to handle edge cases and inconsistencies, where reasonable people might disagree. The working group itself is nominally vendor-neutral, and there are already projects for several vendors - STM32, nRF, ATSAMD, etc - which seem better suited to host their patched SVDs.

Plus, unfortunately, vendors just keep releasing new chips, so the work is never really complete.

[brainstorm]

I appreciate you taking a fresh look at the problem!

The concern around redistribution of modified SVDs is about their copyright. I don't know how significant the concern is.

Gotcha, I'd err on the side of asking for forgiveness rather than permission here, but I understand the concern.

I think a big concern with trying to reach a steady state where all the SVD files are good is the scope of the task (for example, stm32-rs is only one vendor, is not even halfway done, but is already 91 SVD files, some 40000 lines of YAML to patch from 72 contributors over 1400 commits), but another problem is that there's a lot of decisions to make about how the represent the hardware and how to handle edge cases and inconsistencies, where reasonable people might disagree. The working group itself is nominally vendor-neutral, and there are already projects for several vendors - STM32, nRF, ATSAMD, etc - which seem better suited to host their patched SVDs.

Woah, indeed it's a huge task. When the scope is so large I'd go for "bazaar" approach rather than "cathedral" building as it is now. People will contribute fixes/patches to those SVDs that are being currently used and incrementally patch as issues are found. Larger scale engineering around this problem can be a daunting task indeed if carried on by just a handful of people :-S

Plus, unfortunately, vendors just keep releasing new chips, so the work is never really complete.

Oh, yeah, this is never over :) ... OTOH, MSP430's, STM32F0's, etc... are still being used and SVDs should not change much. Bleeding edge SVDs will keep trickling in over time anyway.