arduino-cli compile can no longer specify .ino file

[janjongboom]

Bug Report

Current behavior

On v0.9 of the CLI we can compile a specific ino file via:

$ arduino-cli compile --fqbn arduino:mbed:nano33ble my-sketch.ino

On v0.11 this fails with:

Error during build: unable to find a sketch file in directory XXX

And there seems no way to specify the .ino file anymore. Is there any replacement?

Environment

• CLI version (output of arduino-cli version): arduino-cli Version: 0.11.0 Commit: 0296f4d
• OS and platform: macOS Catalina

[rsora]

Hi @janjongboom,
We solved a couple of bugs regarding the sketch file or folder parsing from command line here #690

It could be a regression, can you give us more details in order to reproduce the issue?
What is the sketch folder structure you are trying to compile?

[janjongboom]

@rsora I have a sketch folder (e.g. here-is-my-sketch) and an .ino with a different name (hello.ino). I could previously compile this fine by specifying the hello.ino file at the end of my compile command. This no longer works in 0.11.

[cmaglie]

The folder containing the sketch must have the same name as the main .ino file and this is due to the way the sketches made by multiple .ino files are preprocessed to obtain the final .cpp file to compile.

For example, if you have a project like this:

~/Arduino/GPS_Tracker/GPS_Module.ino
~/Arduino/GPS_Tracker/GPS_Tracker.ino    <-- main file named as the containing folder
~/Arduino/GPS_Tracker/LCD.ino
~/Arduino/GPS_Tracker/SD.ino

all the .ino files are concatenated together in alphabetic order, except for the main file GPS_Tracker.ino that is always the first -> this is the reason why we need the containing folder to be the same name as the main sketch.

Here you can find the relevant documentation about how a sketch structure https://arduino.github.io/arduino-cli/latest/sketch-specification/#primary-sketch-file

It may be a problem to rename your folders or the sketch .ino so they match?
We may like this requirement or not, but it has always been like this from day 1 of Arduino, changing it now may turn out to be a huge compatiblity mess...

[xplsek01]

@cmaglie I am trying to understand your arguments but if I specify directly what I want to compile as a cmd parameter I would expect compilation of that file and not getting error (Error during build: unable to find a sketch file in directory ...).

I also experience troubles with your last changes. My project in Gitlab repository has different name than the main *.ino file.

I was trying to use 0.10 version via install.sh script but this script checks for updates and install the 0.11 version anyway. It is also not good. If I use install script from version 0.10 I would expect that the 0.10 will be installed. But it is only my opinion...

[janjongboom]

@xplsek01 I heard back from Arduino and you can get older versions through CDN. I had the same issue as you with the versions. Here are the links for 0.9.x (I assume 0.10 has similar ones):

• Linux 64 bit: https://downloads.arduino.cc/arduino-cli/arduino-cli_0.9.0_Linux_64bit.tar.gz
• Linux 32 bit: https://downloads.arduino.cc/arduino-cli/arduino-cli_0.9.0_Linux_32bit.tar.gz
• Linux ARM 64 bit: https://downloads.arduino.cc/arduino-cli/arduino-cli_0.9.0_Linux_ARM64.tar.gz
• Linux ARM 32 bit: https://downloads.arduino.cc/arduino-cli/arduino-cli_0.9.0_Linux_ARMv7.tar.gz
• Windows 64 bit: https://downloads.arduino.cc/arduino-cli/arduino-cli_0.9.0_Windows_64bit.zip
• Windows 32 bit: https://downloads.arduino.cc/arduino-cli/arduino-cli_0.9.0_Windows_32bit.zip
• Mac OSX: https://downloads.arduino.cc/arduino-cli/arduino-cli_0.9.0_macOS_64bit.tar.gz

[rsora]

@janjongboom and @xplsek01
We recently fixed our install script to support a "version" argument, please see the issue #766

[davideshay]

I'd like to add that this whatever change caused this also seems to have broken other useful features. I used to have very good Eclipse IDE integration via a simple Makefile script that called the arduino-cli. In this scenario, the subdirectory was the same name as the main "sketch", but the main sketch had a .cpp extension, so that eclipse would handle it well and do all of the appropriate edit-time checking, etc. Now that is is broken and it claims it can't find a sketch.

Not understanding why the default behavior can't be as described -- subdir name same as the sketch, with an extension of ".ino", but more advanced users can specify other parameters (that USED to work just fine).

Happy to provide a copy of my Makefile if that would help.

[drstevenhale]

Another vote here for specifying the file to be complied. I don't even use .ino files, since they break compatibility with editors that know how to handle .cpp files. Previously it worked just fine for those that provided their own main.cpp file rather than relying on the .ino wrapper.

[matthijskooijman]

The easy workaround for third-party IDE integration as described above would be to add an empty .ino file matching the directory name, to indicate to arduino-cli that this is indeed an Arduino sketch, but then just keep all your code in .cpp files so you can edit those files with your IDE optimally.

[matthijskooijman]

Also note that the change mentioned in the first post seems like a regression in arduino-cli, but this should be viewed in the context of the entire Arduino ecosystem: Before, arduino-cli allowed compiling sketches that the Java IDE would reject. IMHO, the concept of a "Sketch" should be consistent between all tools, so the change in arduino-cli was really a fix to make arduino-cli consistent with all other tools in the ecosystem.

Of course, the notion of what constitutes a sketch can be changed in all Arduino tools, see #948 for discussion about this.

[thegecko]

Just came across this regression myself. The offending code is here:

https://github.com/arduino/arduino-cli/pull/690/files#diff-76bc1d481fc32adae0d5a4ee3b2ccdf0R46

Prior to this, a full sketch path (including .ino file) could be specified. After this change it always ignores any sketch file passed and forces it to expect the sketch to be the same name as the containing folder.

Can this be fixed?

[ubidefeo]

@thegecko
like @matthijskooijman explains in the comment just before yours, arduino-cli allows you to do a lot of things but we still need to be consistent with our whole concept of Sketch.
A Sketch is not the .ino file itself but the whole folder and including (if present) a sketch.json file withe board/port association and (in the future) more things related to libraries, versions and so on.
Before beginning to work on this project I often found myself struggling with the fact that the main file should be contained by a folder with its name, but then made peace with it once I learned what the structure of a sketch is supposed to be.
Believe me when I say that a lot of thinking goes behind defining these workflows :)

Because many Arduino sketches only contain a single .ino file, it's easy to think of that file as the sketch. However, it is the folder that is the sketch. The reason is that sketches may consist of multiple code files and the folder is what groups those files into a single program.

https://arduino.github.io/arduino-cli/latest/sketch-specification/

[janjongboom]

So this behavior broke again in the 0.13 release. As we have zero control over the folder name where the .ino.bin is in (as it's downloaded) we make a copy of the ino.bin to foldername.ino.bin - which we created as a response to this issue. This worked fine in 0.11 and 0.12, and now this no longer works:

Error during Upload: retrieving build artifacts: autodetect build artifact: multiple build artifacts found: 'ei-custom-keywords-local-deployment-arduino-nano-33-ble-sense-1601039830533.ino.bin' and 'firmware.ino.bin'

I realize that this is technically not a 1.0 release of the CLI, but the constant breaking of the API is very annoying as there's zero heads-up on our end and the only way we realize this is from complaining customers.

[electronicsguy]

I find this requirement confusing as well. If we explicitly specify a different file, why not use that as the base file? It took me hours today till I finally figured out the error. (as an aside: why not add a message to the not found error, saying something like do you have a file named the same as your repo?)

[silvanocerza]

@electronicsguy are you using the latest release of the CLI?

It seems to work correctly for me on Linux.

~/Arduino 
$ arduino-cli version
arduino-cli alpha Version: 0.18.3 Commit: d710b642 Date: 2021-07-05T07:52:34Z

~/Arduino 
$ tree MySketch 
MySketch
├── MySketch.ino
└── test.h

0 directories, 2 files

~/Arduino 
$ arduino-cli compile -b arduino:avr:uno ./MySketch/test.h
Sketch uses 444 bytes (1%) of program storage space. Maximum is 32256 bytes.
Global variables use 9 bytes (0%) of dynamic memory, leaving 2039 bytes for local variables. Maximum is 2048 bytes.

The only issue that I know of with compile and paths is #1285 and it's Windows only.
If you have further problems please open a new issue with steps to reproduce it, even if it's a just an error message that is not clear enough.

[matthijskooijman]

@silvanocerza I suspect that @electronicsguy means a sketch with multiple .ino files e.g. A/A.ino and A/B.ino and expects that if you pass A/B.ino to arduino-cli, it will use that file as the primary file (i.e. when merging .ino files for compilation, but B.ino first rather than A.ino.

@electronicsguy I think the reasoning for the current behavior is that a sketch is now compiled in the same way regardless of how you specify the path to the sketch (remember that the sketch is the directory, not just a single .ino file). So it seems that there is a tradeoff between your surprise (When I specify a .ino file, I am surprised that another .ino file is taken as the primary file) vs the surprise that would be introduced when your suggestion is implemented (When I specify another .ino file within the same sketch, the sketch compiles differently or fails to compile).

[silvanocerza]

That works too:

~/Arduino 
$ arduino-cli compile -b arduino:avr:uno ./MySketch/MySketch2.ino 
Sketch uses 444 bytes (1%) of program storage space. Maximum is 32256 bytes.
Global variables use 9 bytes (0%) of dynamic memory, leaving 2039 bytes for local variables. Maximum is 2048 bytes.

As @matthijskooijman says the Sketch is the directory, not a single .ino file. If you specify a path to a single file the CLI uses the parent folder that contains that file to infer the main .ino file, instead by specifiying a folder it uses that path as the Sketch.
In both cases the main .ino file must always be the one that matches the Sketch name, if a matching .ino file is not found that's not a valid Sketch.

See the Sketch Specification docs:
https://arduino.github.io/arduino-cli/latest/sketch-specification/#primary-sketch-file

[electronicsguy]

@silvanocerza do you still have a dummy MySketch.ino there? If so yes it'll work. It won't without it, though AFAIK.

[electronicsguy]

@matthijskooijman Yes, I see the reasoning behind dumbing down things for beginners. Wonder why it can't understand that we have provided a file explicitly, so just use that.

[matthijskooijman]

@electronicsguy I thought you were arguing that a file that has two files (e.g. MySketch.ino and MySketch2.ino) would be compiled differently depending on which file you pass to arduino-cli, but now I think you are arguing that it should be possible to omit MySketch.ino, right? IOW, to have a sketch where there is no .ino file named after the directory? If the latter, I suggest chiming in at #948 which was specifically opened for that discussion.

(as an aside: why not add a message to the not found error, saying something like do you have a file named the same as your repo?)

Maybe this part is still good to discuss here, but it seems that in a quick test (with git master version of arduino-cli), the error message seems ok already?

matthijs@grubby:~$ tree MySketch/
MySketch/
└── MySketch2.ino

matthijs@grubby:~$ arduino-cli compile MySketch/MySketch2.ino 
Error during build: opening sketch: no valid sketch found in /home/matthijs/MySketch: missing /home/matthijs/MySketch/MySketch.ino

But maybe I'm doing something different than you meant?

[electronicsguy]

@matthijskooijman You understood correctly. Apologies if I was vague. Yes, I do mean having a sketch directory without a file with the same name. I guess it's ok to do it so, it just wasn't clear to me when I got the error. The error message simply says "no valid sketch found". Unless I go through the spec which says it must be so, I had no idea that you needed to have a file with the same name as the sketch. It was only when I was using arduino-cli in github-ci that this error was exposed. This is not a requirement when using standalone Arduino IDE.

[silvanocerza]

Yes, there have been some enhancements in certain errors reporting, there's still lots of improvements we can do to show clearer errors to users, we're working on it though.

By the way as of now there are no plans on changing the Sketch main file specification, I understand it creates confusion but it's not our main focus right now. Compilation works so am happy with it, we'll see in the future.

[per1234]

This is not a requirement when using standalone Arduino IDE.

This is a universal requirement across all official Arduino development software. If you attempt to open Matthijs's sketch in the Arduino IDE, you will get a dialog:

The file "MySketch2.ino" needs to be inside a sketch folder named "MySketch2".
Create this folder, move the file, and continue?

[sezanzeb]

but the main sketch had a .cpp extension, so that eclipse would handle it well and do all of the appropriate edit-time checking, etc.

I guess it's somewhat unrelated to the original problem, but since this has been mentioned here and since I wanted to have it as .cpp files as well, this of how it works now:

project
├─ project.ino # includes "source/main.cpp"
└─ source
   ├─ stuff.cpp # contains various other functions
   └─ main.cpp # contains setup and loop, includes "stuff.cpp"

I can't call that folder src because it compiles it recursively and prevents me from getting includes to work the way I want it. Docs: https://arduino.github.io/arduino-cli/0.21/sketch-specification/#src-subfolder. "It can be used to bundle libraries with the sketch in order to make it a self-contained project.", then why is it not called "lib"...