Refactor platformio fix ide

[ppenguin]

Description of changes

• Add platformio as a python module
• (planned/consider?) modifying core.nix and chrootenv.nix to use the python module as a dependency, thereby mirroring upstream structure and facilitating fixing the IDE (vscode extension)

Rationale

The current nixpkgs version does not work with the IDE (vscode extension), because the extension calls python -m platformio as executable to interact with the "backend server".

The goal would be to have the package platformio-core expose both a python environment with the required modules, most importantly platformio within the env, (possibly also accepting an optional argument for "extra pyPackages" (or just an overridden pkgs.python3 instance passed in?)) and the pio executable.

This package would then (for the "normal usecase") be instantiated in an FHS nix-shell or devShell e.g. loaded via direnv, and be automatically picked up by a started vscode that has the platformio-ide extension.

Probably the best way to implement this is to have a platformio python3 package (already implemented as part of this PR), and use it as a dependecy in the "normal" nix packages. This probably means that most of the "heavy lifting" will only be present in the expression for the python module, and the others would be highly simplified and "just" take care of instantiating their respective environments.

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 23.11 Release Notes (or backporting 23.05 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

[ppenguin]

@mogorman @makefu

I tested the python package with the vscode extension in this nix-shell, and (for the first time) the extension appears to correctly start the server and interact with it.

{ pkgs ? import /home/jeroen/devel/github.com/ppenguin/nixpkgs {} }:

let
  envname = "platformio-fhs";
  mypython = pkgs.python3.withPackages (ps: with ps; [ platformio ]);
in
(pkgs.buildFHSUserEnv {
  name = envname;
  targetPkgs = pkgs: (with pkgs; [
    arduino-cli avrdude
    platformio-core
    mypython
    zsh
  ]);
  runScript = "zsh";
}).env

The above would use a different python env for platformio-core, basically having two separate instances of the main functionality, hence the refactoring proposal.

For further integration (see above refactoring ideas) I could use some help, since I'm not entirely sure how to correctly handle outputs offering a complete env in the best way (maybe just the current chrootenv.nix with additional python env exposure?)

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/how-to-use-platformio-with-vscode/15805/8

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/how-to-build-patched-avr-gcc-with-patched-avr-libc/21468/2

[delan]

Thanks for working on this! I don’t really have the expertise to help land this, but with this patch I can now use platformio with vscode to build a rp2040 project. I had to tweak my shell.nix slightly due to #263201:

{ pkgs ? import (builtins.fetchTarball {
    # NixOS/nixpkgs#237313 = ppenguin:refactor-platformio-fix-ide
    url = "https://github.com/NixOS/nixpkgs/archive/3592b10a67b518700002f1577e301d73905704fe.tar.gz";
  }) {},
}:
let
  envname = "platformio-fhs";
  mypython = pkgs.python3.withPackages (ps: with ps; [ platformio ]);
in
(pkgs.buildFHSUserEnv {
  name = envname;
  targetPkgs = pkgs: (with pkgs; [
    platformio-core
    mypython
  ]);
  # NixOS/nixpkgs#263201, NixOS/nixpkgs#262775, NixOS/nixpkgs#262080
  runScript = "env LD_LIBRARY_PATH= bash";
}).env

I’ve updated the wiki in the meantime.

[ppenguin]

@wegank unstale please, this issue needs support; As-is platformIO cannot be used on NixOS so should either be fixed or removed.

[CodePidgy]

Is this pull request still being considered/reviewed? It really sucks not being able to use PIO at all on NixOS

[ppenguin]

Is this pull request still being considered/reviewed? It really sucks not being able to use PIO at all on NixOS

Yes, but you can use it already, just use this PR for the PIO package, see also the wiki

I'm still waiting for one of the original package authors to chime in, since I've currently neither the time nor the skills to land this PR in a consistent way (i.e. using correct passthrough's and what not to make sure both vscode and standalone function uses the same pyenv, a correct FHS, etc.)

[spector700]

@ppenguin I can not get the example shell to work, is there a way to do this?
Also, I'm trying to use the Arduino framework for the esp32, but it can never find Arduino.h. I don't know if this issue is related.

[CodePidgy]

I also could never get it to work, but that could very well be due to my inexperience with Nix. I never asked as I was too nervous and StackOverflow has ruined my confidence in stuff like this. I also never really wanted to use it in a shell, but rather have it so that when I rebuild it simply installs this PR instead of pulling the current version, this is really important to me but I don't understand Nix enough to have implemented it after reading through the documentation on how to implement it like this. My current Nix config is public on my profile if curious

[ppenguin]

@spector700 @CodePidgy

I made a quick&dirty fix in my env flake: https://github.com/ppenguin/nixenvs/blob/main/dev/devshell-pio-arduino-fhs.nix

I'm short on time to do it well, also I haven't really tested it, but avrdude and pio run in the shell and code starts the pio plugin.

Run int like so:
.envrc

#!/usr/bin/env bash

PRJROOT="$(git rev-parse --show-toplevel)"
FLAKE=github:ppenguin/nixenvs

# avoid a "load loop" of direnv when the new fhs env is entered
# https://github.com/direnv/direnv/issues/992
if [ -z "$IN_NIX_SHELL" ]; then
	use flake $FLAKE\#pio-arduino-fhs
fi

export PRJROOT