Crank: fix and test package build for linux

[rtsuk]

So that the --device option works on Linux

[rtsuk]

I think this is fixed now, due to @lilyinstarlight's work, but let's leave it open until a Linux user gets a device to test it.

[tjkirch]

It took a few days, but I figured out how to get the Hello World example working on a Playdate from Linux. Here are the crankstart-specific things I ran into:

• It was easy to figure out that I needed the arm-none-eabi-* tools thanks to lilyinstarlight's change, but it still failed to build with --device because stdint.h had include_next <stdint.h> and couldn't find that stdint.h. Eventually I figured out that installing cross-arm-none-eabi-newlib (that's the package on Void Linux) was good enough to fix it, though I'm not sure it's the correct fix. [edit: yes, that's the correct package, per Playdate docs.]
• That build ran fine on the simulator, but gave "pdxinfo file not found" on the Playdate. I confirmed the file was there, compared it to Season 1 games' pdxinfo files, and everything seemed fine, so I flailed around a bit... and found that by renaming Hello World.pdx to HelloWorld.pdx (removing the space) it worked fine. I tried using the simulator's "Run Game on Device" and encoding the space in various ways, but nothing worked other than renaming it. If there's no obvious issue, maybe crankstart should rename the example? [edit: fixed in 1.13.1!]
• crank run --device ... and crank build --device --run ... both just hang forever, with no indication of anything happening on the terminal or the Playdate screen. strace shows it checking/writing the pdx files, writing \ndatadisk\n to /dev/TTYACM0, then it gets stuck in a loop calling statx on /dev/ttyACM0, which I believe is from this code that waits for the device to disappear. The tty device doesn't seem to disappear on my system, even when booted into Data Disk, so I'm not sure what it's waiting for. I've been using the simulator instead, but it'd be good to understand this. [edit: see Improve Linux experience for crank run --device #36]

There were also another couple non-crankstart-specific oddities that I'll document just in case someone else finds this.

• When using the simulator to upload a game to the device, it was sitting at "Waiting for Playdate data disk..." for several minutes before failing. My system doesn't auto-mount removable devices, so I had to do that manually. (I know there are tools to automate this.) [edit: see Improve Linux experience for crank run --device #36]
• After it's mounted, it will copy the game and unmount the device, but you still have to press A on the Playdate to exit Data Disk mode and let the game try to run; it doesn't say that anywhere. [edit: see Improve Linux experience for crank run --device #36]

[rtsuk]

At least on the Mac, the Playdate SDK provides all the tools and headers needed. Perhaps the SDK path environmental variable isn't correct for where it is installed?

As for the space in package names, that worked in the past. Does the C-API example in the SDK have a space?

[tjkirch]

At least on the Mac, the Playdate SDK provides all the tools and headers needed.

I checked the contents of the Mac SDK and confirmed the Linux SDK has the same contents, at a high level. They include the Playdate-specific headers and pdc/pdutil. However, neither includes a toolchain for the target architecture. I just found the below section of "Inside Playdate with C" that explains it. You probably have Xcode installed, and the Playdate SDK installer handles the rest, on Mac. It might be useful to copy the requirements for other OSes to crank's requirements list.

From https://sdk.play.date/1.13.0/Inside%20Playdate%20with%20C.html#_prerequisites -

To build games for the Playdate in C you will need to use a native C compiler for the Simulator and the ARM embedded compiler for the Playdate hardware. Native C compilers come with the standard development tools on the platform: Xcode, Visual Studio, Linux Development packages. However, the embedded ARM compiler will also need to be installed.

MacOS: The Playdate SDK installer installs the ARM embedded compiler in /Applications/ARM automatically.

Linux: Install the arm-none-eabi-newlib package (naming varies based on distro).

Windows: Install the Windows ARM embedded compiler gcc-arm-none-eabi from [developer.arm.com](https://developer.arm.com/).

---

Perhaps the SDK path environmental variable isn't correct for where it is installed?

I'm pretty sure it's correct, since crank uses it to find the SDK location, and I got crank building PDXs OK.

> ls $PLAYDATE_SDK_PATH 
 C_API     'Designing for Playdate'        Disk      'Inside Playdate'              'Inside Playdate.html'   README.md   SDK_LICENSE.md   bin
 CoreLibs  'Designing for Playdate.html'   Examples  'Inside Playdate with C.html'   PlaydateSDK.docset      Resources   VERSION.txt      setup.sh

As for the space in package names, that worked in the past. Does the C-API example in the SDK have a space?

Yes, several do, like 3d library, Hello World, and Sprite Game. I tried Hello World as a test, and the difference is that the cmake/makefile included with the SDK creates the target directory as hello_world.pdx with no space, as opposed to Hello World.pdx with a space from crank/crankstart.

The C_API Hello World (becoming hello_world.pdx) did run fine on my Playdate. crank uses target.metadata.name from Crank.toml to build the pdx name, and when I changed it to a name with no spaces, that build worked fine on the Playdate, too. As I mentioned, I couldn't find a path that the Playdate would accept to run a pdx with a space in the name, but maybe there's a way of escaping it. [edit: this was fixed in 1.13.1!]

One other random note: for me, even after removing spaces, the simulator's Upload Game to Device isn't working; it gives the "pdxinfo file not found." error I mentioned earlier, even though I confirmed the pdxinfo file was there and there were no spaces anywhere. It doesn't matter if the pdx is from a C_API example or crank. However, selecting Run Game On Device and passing giving the path works fine, or the equivalent with pdutil. I looked at what the simulator is doing using strace, and after everything is uploaded to the device, it appear to be writing run /Games/\n to the device, without a full game path, but I'm not sure I'm interpreting it correctly...
[edit] this last issue is a known bug in SDK 1.13.0, fixed in 1.13.1.

[tjkirch]

To summarize and clarify the above thoughts, I think I'd recommend the following changes:

• In the crank requirements, add a link to the Inside Playdate with C prerequisites so people know that they need to install that toolchain, and how to do so. [edit: see Update README #34]
• Either
  • change crankstart's Crank.toml so that target.metadata.name has no space, and consider documenting the effect it has, or
  • change crank to create target directories without spaces, regardless of the project name; I think this happens here in make_source_dir.

[edit: no longer necessary, fixed in 1.13.1!]

I'd be happy to do a PR for those if you think I'm on track.

I'm less sure what to think about this loop that waits for /dev/ttyACM0 to disappear, which never happens on my system. I currently only have a dev setup in Linux; I'm assuming it does disappear on Mac or you'd see the same thing. (Windows uses simpler code for run_target that just calls pdutil.) There are comments in this section that indicate hard-won victories over complexity in the Unix case, so I assume it can't be simplified down to pdutil calls like Windows, but I wonder if it's as simple as removing the wait loop in the Linux case.
[edit: see #36]

Are there any other Linux users of crank who could check whether crank run hangs, and relatedly, whether /dev/ttyACM0 disappears after crank initiates Data Disk mode during crank run?

[rtsuk]

• Adding the requirements would be great.
• As long as the space still appears in the game name, changes here for compatibility are fine.
• pdutil did not exist on the Mac when I wrote that part of crank, so it would be fine now to switch all platforms over to pdutil. I can test it on the Mac for you if you don't have one. Perhaps pdutil does some magic to unmount the volume but keep the serial device present.

I think it would be best to approach these as three separate PRs.

[tjkirch]

As long as the space still appears in the game name, changes here for compatibility are fine.

Yeah, I wouldn't want to change the name of someone's game, of course, just in the system-level artifact where necessary. Do you think it'd make sense to change spaces to underscores programmatically, just for the name of the pdx / directory?

(Of course, that makes me wonder what other characters are invalid - Inside Playdate doesn't mention any rules - but I think it'd be fine to just fix compatibility with spaces for now.)

it would be fine now to switch all platforms over to pdutil

I'll give this a shot, thanks.

[tjkirch]

it would be fine now to switch all platforms over to pdutil.

Oh, heh, pdutil on Linux doesn't have an install subcommand, so that install logic in crank can't be fully replaced yet.

If Mac pdutil has an install subcommand, that case could probably be simplified, and just keep the logic for Linux.

---

I just did some testing, and the following was required to get --run to work on Linux:

• Commenting out the code that waits for /dev/ttyACM* to disappear, since it doesn't.
• Sleeping before issuing the pdutil run. I first tried matching the Windows case, sleeping for 500ms, but it wasn't enough; 1s was enough. (I'm not sure of a specific marker to watch for rather than just timing; could run pdutil as a canary, but that seems ugly to me.)

I think the following would also be helpful:

• Using umount rather than eject for the data path, since eject tries to issue a physical eject to the USB device, which gives an error.
• Printing a request to press 'A' on the Playdate after we eject/umount, since it's not obvious for beginners.
• Printing a message before waiting for the data path / games dir, for systems that don't auto-mount devices, so the user knows when to mount. I figured out this timing by setting RUST_LOG to see when it's waiting, but I think it could be made clearer.

I'm happy to work on PRs if you think this sounds good.

[lilyinstarlight]

• Using umount rather than eject for the data path, since eject tries to issue a physical eject to the USB device, which gives an error.

That was actually specifically intended, but I didn't have a playdate at the time I wrote that to know that it produces an error. Changing it to umount is probably a good idea (does that work without root for devices that were mounted via udisks2?)

I'm happy to work on PRs if you think this sounds good.

I am not @rtsuk but as the person who initially added any Linux support, that all sounds fabulous to me. Thank you so much for working on it!

[tjkirch]

Changing it to umount is probably a good idea (does that work without root for devices that were mounted via udisks2?)

@lilyinstarlight I'm not familiar with udisks2 - I use the (simpler? dumber?) route of the user option in fstab. I'd be interested to hear confirmation before I did anything. I'm working on the other things first anyway, since the eject error is harmless - it does still unmount the device.

[tjkirch]

As long as the space still appears in the game name, changes here for compatibility are fine.

I'm happy to report that with 1.13.1, spaces are no longer an issue! I can send Hello World.pdx through crank just fine, or run the simulator, select Run Game, and input /Games/Hello World.pdx and it runs just fine, neither of which worked before the 1.13.1 update.