All GIT Versions Higher than 3.0.2 Can't Find the Arduino Preferences.txt File

[jgrass3]

Basic Infos

• This issue complies with the issue POLICY doc.
• I have read the documentation at readthedocs and the issue is not addressed there.
• I have tested that the issue is present in current master branch (aka latest git).
• I have searched the issue tracker for a similar issue.
• If there is a stack dump, I have decoded it.
• I have filled out all fields below. (Some not needed)

Platform

• Hardware: ESP8266
• Core Version: 3.1.1
• Development Env: Arduino IDE
• Operating System: Windows 11

Settings in IDE

• Module: Adafruit Feather HUZZAH ESP8266
• Flash Mode: Only Sketch
• Flash Size: 4MB
• lwip Variant: v2 Lower Memory
• Reset Method: nodemcu
• Flash Frequency: 40Mhz
• CPU Frequency: 80Mhz
• Upload Using: SERIAL
• Upload Speed: 115200

Problem Description

Using any GIT version higher than 3.0.2 results in the compiler not being able to find my "preferences.txt" file. The file is always in the same place, in D:\Documents\ArduinoData, yet newer GIT versions can't find it. However, all versions 3.0.2 and lower, have no problem finding the file.

#include <Arduino.h>

Irrelevant since NO sketch will compile.

Debug Messages

Arduino: 1.8.19 (Windows Store 1.8.57.0) (Windows 10), Board: "Adafruit Feather HUZZAH ESP8266, 80 MHz, Flash, Disabled (new aborts on oom), Disabled, All SSL ciphers (most compatible), 32KB cache + 32KB IRAM (balanced), Use pgm_read macros for IRAM/PROGMEM, 4MB (FS:2MB OTA:~1019KB), v2 Lower Memory, Disabled, None, Only Sketch, 115200"

C:\Program Files\WindowsApps\ArduinoLLC.ArduinoIDE_1.8.57.0_x86__mdqgnx93n4wtt\arduino-builder -dump-prefs -logger=machine -hardware C:\Program Files\WindowsApps\ArduinoLLC.ArduinoIDE_1.8.57.0_x86__mdqgnx93n4wtt\hardware -hardware D:\Documents\ArduinoData\packages -tools C:\Program Files\WindowsApps\ArduinoLLC.ArduinoIDE_1.8.57.0_x86__mdqgnx93n4wtt\tools-builder -tools C:\Program Files\WindowsApps\ArduinoLLC.ArduinoIDE_1.8.57.0_x86__mdqgnx93n4wtt\hardware\tools\avr -tools D:\Documents\ArduinoData\packages -built-in-libraries C:\Program Files\WindowsApps\ArduinoLLC.ArduinoIDE_1.8.57.0_x86__mdqgnx93n4wtt\libraries -libraries D:\Documents\Arduino\libraries -fqbn=esp8266:esp8266:huzzah:xtal=80,vt=flash,exception=disabled,stacksmash=disabled,ssl=all,mmu=3232,non32xfer=fast,eesz=4M2M,ip=lm2f,dbg=Disabled,lvl=None____,wipe=none,baud=115200 -ide-version=10819 -build-path C:\Users\JOEGUA~1\AppData\Local\Temp\arduino_build_756055 -warnings=default -build-cache C:\Users\JOEGUA~1\AppData\Local\Temp\arduino_cache_195117 -prefs=build.warn_data_percentage=75 -prefs=runtime.tools.python3.path=D:\Documents\ArduinoData\packages\esp8266\tools\python3\3.7.2-post1 -prefs=runtime.tools.python3-3.7.2-post1.path=D:\Documents\ArduinoData\packages\esp8266\tools\python3\3.7.2-post1 -prefs=runtime.tools.mklittlefs.path=D:\Documents\ArduinoData\packages\esp8266\tools\mklittlefs\3.1.0-gcc10.3-e5f9fec -prefs=runtime.tools.mklittlefs-3.1.0-gcc10.3-e5f9fec.path=D:\Documents\ArduinoData\packages\esp8266\tools\mklittlefs\3.1.0-gcc10.3-e5f9fec -prefs=runtime.tools.xtensa-lx106-elf-gcc.path=D:\Documents\ArduinoData\packages\esp8266\tools\xtensa-lx106-elf-gcc\3.1.0-gcc10.3-e5f9fec -prefs=runtime.tools.xtensa-lx106-elf-gcc-3.1.0-gcc10.3-e5f9fec.path=D:\Documents\ArduinoData\packages\esp8266\tools\xtensa-lx106-elf-gcc\3.1.0-gcc10.3-e5f9fec -prefs=runtime.tools.mkspiffs.path=D:\Documents\ArduinoData\packages\esp8266\tools\mkspiffs\3.1.0-gcc10.3-e5f9fec -prefs=runtime.tools.mkspiffs-3.1.0-gcc10.3-e5f9fec.path=D:\Documents\ArduinoData\packages\esp8266\tools\mkspiffs\3.1.0-gcc10.3-e5f9fec -verbose D:\Documents\Arduino\Mini_Temp_Sensor_Feather_V6\Mini_Temp_Sensor_Feather_V6.ino

C:\Program Files\WindowsApps\ArduinoLLC.ArduinoIDE_1.8.57.0_x86__mdqgnx93n4wtt\arduino-builder -compile -logger=machine -hardware C:\Program Files\WindowsApps\ArduinoLLC.ArduinoIDE_1.8.57.0_x86__mdqgnx93n4wtt\hardware -hardware D:\Documents\ArduinoData\packages -tools C:\Program Files\WindowsApps\ArduinoLLC.ArduinoIDE_1.8.57.0_x86__mdqgnx93n4wtt\tools-builder -tools C:\Program Files\WindowsApps\ArduinoLLC.ArduinoIDE_1.8.57.0_x86__mdqgnx93n4wtt\hardware\tools\avr -tools D:\Documents\ArduinoData\packages -built-in-libraries C:\Program Files\WindowsApps\ArduinoLLC.ArduinoIDE_1.8.57.0_x86__mdqgnx93n4wtt\libraries -libraries D:\Documents\Arduino\libraries -fqbn=esp8266:esp8266:huzzah:xtal=80,vt=flash,exception=disabled,stacksmash=disabled,ssl=all,mmu=3232,non32xfer=fast,eesz=4M2M,ip=lm2f,dbg=Disabled,lvl=None____,wipe=none,baud=115200 -ide-version=10819 -build-path C:\Users\JOEGUA~1\AppData\Local\Temp\arduino_build_756055 -warnings=default -build-cache C:\Users\JOEGUA~1\AppData\Local\Temp\arduino_cache_195117 -prefs=build.warn_data_percentage=75 -prefs=runtime.tools.python3.path=D:\Documents\ArduinoData\packages\esp8266\tools\python3\3.7.2-post1 -prefs=runtime.tools.python3-3.7.2-post1.path=D:\Documents\ArduinoData\packages\esp8266\tools\python3\3.7.2-post1 -prefs=runtime.tools.mklittlefs.path=D:\Documents\ArduinoData\packages\esp8266\tools\mklittlefs\3.1.0-gcc10.3-e5f9fec -prefs=runtime.tools.mklittlefs-3.1.0-gcc10.3-e5f9fec.path=D:\Documents\ArduinoData\packages\esp8266\tools\mklittlefs\3.1.0-gcc10.3-e5f9fec -prefs=runtime.tools.xtensa-lx106-elf-gcc.path=D:\Documents\ArduinoData\packages\esp8266\tools\xtensa-lx106-elf-gcc\3.1.0-gcc10.3-e5f9fec -prefs=runtime.tools.xtensa-lx106-elf-gcc-3.1.0-gcc10.3-e5f9fec.path=D:\Documents\ArduinoData\packages\esp8266\tools\xtensa-lx106-elf-gcc\3.1.0-gcc10.3-e5f9fec -prefs=runtime.tools.mkspiffs.path=D:\Documents\ArduinoData\packages\esp8266\tools\mkspiffs\3.1.0-gcc10.3-e5f9fec -prefs=runtime.tools.mkspiffs-3.1.0-gcc10.3-e5f9fec.path=D:\Documents\ArduinoData\packages\esp8266\tools\mkspiffs\3.1.0-gcc10.3-e5f9fec -verbose D:\Documents\Arduino\Mini_Temp_Sensor_Feather_V6\Mini_Temp_Sensor_Feather_V6.ino

Using board 'huzzah' from platform in folder: D:\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.1.1

Using core 'esp8266' from platform in folder: D:\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.1.1

"D:\\Documents\\ArduinoData\\packages\\esp8266\\tools\\python3\\3.7.2-post1/python3" -I "D:\\Documents\\ArduinoData\\packages\\esp8266\\hardware\\esp8266\\3.1.1/tools/mkbuildoptglobals.py" "C:\\Program Files\\WindowsApps\\ArduinoLLC.ArduinoIDE_1.8.57.0_x86__mdqgnx93n4wtt" 10819 "C:\\Users\\JOEGUA~1\\AppData\\Local\\Temp\\arduino_build_756055" "C:\\Users\\JOEGUA~1\\AppData\\Local\\Temp\\arduino_build_756055/core/build.opt" "D:\\Documents\\Arduino\\Mini_Temp_Sensor_Feather_V6/Mini_Temp_Sensor_Feather_V6.ino.globals.h" "D:\\Documents\\ArduinoData\\packages\\esp8266\\hardware\\esp8266\\3.1.1\\cores\\esp8266/CommonHFile.h"

*** File preferences.txt not found on Windows

*** 'preferences.txt' file missing from well known locations.

exit status 1

Error compiling for board Adafruit Feather HUZZAH ESP8266.

[risidoro81]

Hi, same issue with latest 3.1.1
I think the error is in %USERPROFILE%\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.1.1\tools\mkbuildoptglobals.py line 466:

return [fqfn, fqfn2]

it must be

return [fqfn, None]

because fqfn2 path does not exist if we reach that else clause (probably lines 464 and 466 should be swapped)

I modified line 466 by hand and I can now compile correctly.

Bye

[jgrass3]

Your reply is posted with the issue so I think we’re covered and the developer will see your comments there. Thanks for this. Jguarrasi From: Isidoro Russo ***@***.***> Sent: Monday, January 16, 2023 5:25 PM To: esp8266/Arduino ***@***.***> Cc: jgrass3 ***@***.***>; Author ***@***.***> Subject: Re: [esp8266/Arduino] All GIT Versions Higher than 3.0.2 Can't Find the Arduino Preferences.txt File (Issue #8811) Hi, same issue with latest 3.1.1 I think the error is in %USERPROFILE%\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.1.1\tools\mkbuildoptglobals.py line 466: return [fqfn, fqfn2] it must be return [fqfn, None] because fqfn2 path does not exist if we reach that else clause (probably lines 464 and 466 should be exchanged) I modified line 466 by hand and I can now compile correctly. I don't know what is the best ways to report the issue to the developer though... Bye — Reply to this email directly, view it on GitHub<#8811 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AQOHZAEZYTGBSFBN73BL3HLWSXDCDANCNFSM6AAAAAAT5FDZWA>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[mhightower83]

@risidoro81 I think I have your issues resolved with PR#8812

@jgrass3 I don't think that PR will resolve your issue. Your installation looks strange to me. Your installation has moved the Arduino sketch folder to D:\Documents\Arduino which is easy to change through the File->Preferences on the IDE; however, the ArduinoData folder has also moved to D:\Documents\ArduinoData. How did you make this change?

[jgrass3]

Hi, thanks for the update. I believe that when you install Arduino IDE from the Microsoft Store, you have the option to put the App in whatever location you want. And so the ArduinoData directory goes with the installation. And since it is a Store App, you can go into the Apps section of Windows Settings and move it wherever you want. I typically move most of my Apps to my D: drive which has alot of storage vice my C: boot drive which is small. And remember, I never had a problem with any of the older versions of the Git. Something changed in the latest versions.

[mhightower83]

@mcspr I think there are two different issues in this thread. I think what @risidoro81 describes is fixed.

RE: The origin issue report by @jgrass3, based on the content of "Debug messages" there is some unusual placement of files. I think his issue may need PR#8814

[cgalvan21]

I had a similar problem with an "alternate" location for the preferences file. My Arduino Data folder is on OneDrive. I edited line 455 of C:\Users\xxx\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.1.1\tools\mkbuildoptglobals.py" so that it reads 'fqfn = os.path.expanduser("~\OneDrive\Documents\ArduinoData\preferences.txt")'. Note the addition of "\OneDrive". It worked for me.

Credit to @risidoro81 for steering me in the right direction.

[martysm]

Same issue.
return [fqfn, fqfn2]
to
return [fqfn, None]
worked for me too.

[04alband]

For some reason, I've just bumped into this issue as well. I agree with what I think has been said (not sure I've followed everything so apologies if I'm behind the curve). There's a logical issue with the return values of find_preferences_txt around lines 464-466, and then also an issue with how unusual locations of preferences.txt are handled.

From what I've seen though it looks like these should both have been addressed already? However, I re-installed 3.1.1 and I still seem to have mkbuildoptglobals.py with both issues. Either way, I'd like to add the following to the discussion...

Similar to @cgalvan21, my ArduinoData folder is in OneDrive and resolves to D:\David\OneDrive\Documents\ArduinoData\preferences.txt. In response to @mhightower83's question about how it would have ended up there, it's because I moved my default Documents folder location (one can easily move such folders around by Right click > Properties > Location > Change Location). The Arduino IDE (app store version 1.8.19 on Win 10) clearly respected this and so has put the ArduinoData folder and thus preferences.txt in that location.

So I sought a hopefully more reliable way of finding where the IDE has put the preferences folder. I examined with processmonitor how the IDE was accessing the preferences file, and, shortly before accessing preference.txt, it queries the HKCU\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\User Shell Folders\Personal registry key to find the location of the user's Documents folder:

I've therefore changed my version of mkbuildoptglobals.py to do the same thing in order to resolve the location of preferences.txt using the following:

import winreg
shfKey = winreg.OpenKey(winreg.HKEY_CURRENT_USER, r'SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\User Shell Folders')
docs = winreg.QueryValueEx(shfKey, "Personal")[0]
fqfn = os.path.join(docs, 'ArduinoData\preferences.txt')

This is working for me, though I'm not sure if it's still relevant with the latest version of mkbuildoptglobals.py, which I don't seem to have. So just adding that idea for anyone else with this issue, or maybe it might be relevant to incorporate properly.