MegaCoreX dynamic fuses and bootloader

[MCUdude]

Hi!
After you @valeros released the latest atmelmegaavr package I started working on the fuses/bootloader script, and it wasn't that much work really. As mentioned earlier, I am NOT a (professional) Python programmer. The scrips work, but they are probably terrible because I'm happy as long as they just do what I want.

First, here is the fuses script I've come up with:

import sys
import os

from SCons.Script import ARGUMENTS, COMMAND_LINE_TARGETS, Import, Return

Import("env")

def get_WDTCFG_fuse():
    return 0x00

def get_BODCFG_fuse(bod):
    if bod == "4.3v":
        return 0xF4
    elif bod == "2.6v":
        return 0x54
    elif bod == "1.8v":
        return 0x14
    else: # bod disabled
        return 0x00

def get_OSCCFG_fuse(f_cpu, oscillator):
    if (f_cpu == "20000000L" or f_cpu == "10000000L" or f_cpu == "5000000L") and oscillator == "internal":
        return 0x02
    else:
        return 0x01

def get_TCD0CFG_fuse():
    return 0x00

def get_SYSCFG0_fuse(eesave, rstpin, uart):
    eesave_bit = 1 if eesave == "yes" else 0
    if rstpin == "gpio":
        if uart == "no_bootloader":
            rstpin_bit = 0
        else:
            rstpin_bit = 1
    else:
        rstpin_bit = 1
    return 0xC0 | rstpin_bit << 3 | eesave_bit

def get_SYSCFG1_fuse():
    return 0x06

def get_APPEND_fuse():
    return 0x00

def get_BOOTEND_fuse(uart):
    if uart == "no_bootloader":
        return 0x00
    else:
        return 0x02

def get_LOCKBIT_fuse():
    return 0xC5

def print_fuses_info(fuse_values, fuse_names, lock_fuse):
    print("\nSelected fuses:")
    print("------------------------")
    for idx, value in enumerate(fuse_values):
        if value:
            print("[fuse%d / %s = 0x%s]" % (idx, fuse_names[idx], value))
    if lock_fuse:
        print("[lfuse / LOCKBIT = %s]" % lock_fuse)
    print("------------------------\n")

board = env.BoardConfig()
platform = env.PioPlatform()
core = board.get("build.core", "")

target = (
    board.get("build.mcu").lower()
    if board.get("build.mcu", "")
    else env.subst("$BOARD").lower()
)

fuses_section = "fuse_values"
if "bootloader" in COMMAND_LINE_TARGETS or "UPLOADBOOTCMD" in env:
    fuses_section = "bootloader"

board_fuses = board.get(fuses_section, {})

# Note: the index represents the fuse number
fuse_values = [
    board_fuses.get("WDTCFG", ""),
    board_fuses.get("BODCFG", ""),
    board_fuses.get("OSCCFG", ""),
    "",  # reserved
    board_fuses.get("TCD0CFG", ""),
    board_fuses.get("SYSCFG0", ""),
    board_fuses.get("SYSCFG1", ""),
    board_fuses.get("APPEND", ""),
    board_fuses.get("BOOTEND", ""),
]

fuse_names = (
    "WDTCFG ",
    "BODCFG ",
    "OSCCFG ",
    "       ",
    "TCD0CFG",
    "SYSCFG0",
    "SYSCFG1",
    "APPEND ",
    "BOOTEND",
    "LOCKBIT"
)

lock_fuse = board_fuses.get("LOCKBIT", "0xC5")


if not board_fuses and "FUSESFLAGS" not in env and core not in ("MegaCoreX"):
    sys.stderr.write(
        "Error: Dynamic fuses generation for %s / %s is not supported."
        " Please specify fuses in platformio.ini\n" % (target, env.subst("$BOARD"))
    )
    env.Exit(1)

if core in ("MegaCoreX"):
    f_cpu = board.get("build.f_cpu", "16000000L").upper()
    oscillator = board.get("hardware.oscillator", "internal").lower()
    bod = board.get("hardware.bod", "2.6v").lower()
    uart = board.get("hardware.uart", "no_bootloader").lower()
    eesave = board.get("hardware.eesave", "yes").lower()
    rstpin = board.get("hardware.rstpin", "reset").lower()

    # Guard that prevents the user from turning the reset pin
    # into a GPIO while using a bootloader
    if uart != "no_bootloader":
        rstpin = "reset"

    print("\nTARGET CONFIGURATION:")
    print("------------------------")
    print("Target = %s" % target)
    print("Clock speed = %s" % f_cpu)
    print("Oscillator = %s" % oscillator)
    print("BOD level = %s" % bod)
    print("Save EEPROM = %s" % eesave)
    print("Reset pin mode = %s" % rstpin)
    print("------------------------")

    fuse_values[0] = fuse_values[0] or '%.2X' % get_WDTCFG_fuse()
    fuse_values[1] = fuse_values[1] or '%.2X' % get_BODCFG_fuse(bod)
    fuse_values[2] = fuse_values[2] or '%.2X' % get_OSCCFG_fuse(f_cpu, oscillator)
    fuse_values[4] = fuse_values[4] or '%.2X' % get_TCD0CFG_fuse()
    fuse_values[5] = fuse_values[5] or '%.2X' % get_SYSCFG0_fuse(eesave, rstpin, uart)
    fuse_values[6] = fuse_values[6] or '%.2X' % get_SYSCFG1_fuse()
    fuse_values[7] = fuse_values[7] or '%.2X' % get_APPEND_fuse()
    fuse_values[8] = fuse_values[8] or '%.2X' % get_BOOTEND_fuse(uart)

env.Append(
    FUSESUPLOADER="avrdude",
    FUSESUPLOADERFLAGS=[
        "-p",
        "$BOARD_MCU",
        "-C",
        '"%s"'
        % os.path.join(env.PioPlatform().get_package_dir(
            "tool-avrdude-megaavr") or "", "avrdude.conf"),
    ],
    SETFUSESCMD="$FUSESUPLOADER $FUSESUPLOADERFLAGS $UPLOAD_FLAGS $FUSESFLAGS",
)

env.Append(
    FUSESFLAGS=[
        "-Ufuse%d:w:0x%s:m" % (idx, value) for idx, value in enumerate(fuse_values) if value
    ]
)

if lock_fuse:
    env.Append(FUSESFLAGS=["-Ulock:w:%s:m" % lock_fuse])

if int(ARGUMENTS.get("PIOVERBOSE", 0)):
    env.Append(FUSESUPLOADERFLAGS=["-v"])

if not env.BoardConfig().get("upload", {}).get("require_upload_port", False):
    # upload methods via USB
    env.Append(FUSESUPLOADERFLAGS=["-P", "usb"])
else:
    env.AutodetectUploadPort()
    env.Append(FUSESUPLOADERFLAGS=["-P", '"$UPLOAD_PORT"'])

if env.subst("$UPLOAD_PROTOCOL") != "custom":
    env.Append(FUSESUPLOADERFLAGS=["-c", "$UPLOAD_PROTOCOL"])
else:
    print(
        "Warning: The `custom` upload protocol is used! The upload and fuse flags may "
        "conflict!\nMore information: "
        "https://docs.platformio.org/en/latest/platforms/atmelavr.html"
        "#overriding-default-fuses-command\n"
    )

print_fuses_info(fuse_values, fuse_names, lock_fuse)

fuses_action = env.VerboseAction("$SETFUSESCMD", "Setting fuses...")

Return("fuses_action")

Note that many of the functions are empty and only return a constant. This is on purpose so it's easier to extend the functionality later (other megaavr cores etc.).

Here is the bootloader script:

# Copyright 2019-present PlatformIO <contact@platformio.org>
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

import sys
import os

from SCons.Script import Import, Return

Import("env")

board = env.BoardConfig()
platform = env.PioPlatform()
core = board.get("build.core", "")


def get_suitable_optiboot_binary(framework_dir, board_config):
    uart = board_config.get("hardware.uart", "no_bootloader").upper()
    if uart == "UART0":
      uart = "UART0_DEF"
    elif uart == "UART1":
      uart = "UART1_DEF"
    elif uart == "UART2":
      uart = "UART2_DEF"
    elif uart == "UART3":
      uart = "UART3_DEF"

    bootloader_led = board_config.get("bootloader.led_pin", "A7").upper()
    bootloader_speed = board_config.get("bootloader.speed", env.subst("$UPLOAD_SPEED"))
    bootloader_file = "Optiboot_mega0_%s_%s_%s.hex" % (
        uart, bootloader_speed, bootloader_led)

    bootloader_path = os.path.join(
        framework_dir, "bootloaders", "optiboot", "bootloaders", "mega0",
        bootloader_speed, bootloader_file
    )

    return bootloader_path


framework_dir = ""
if env.get("PIOFRAMEWORK", []):
    framework_dir = platform.get_package_dir(platform.frameworks[env.get(
        "PIOFRAMEWORK")[0]]["package"])

#
# Bootloader processing
#

bootloader_path = board.get("bootloader.file", "")
if core == "MegaCoreX":
    if not os.path.isfile(bootloader_path):
        bootloader_path = get_suitable_optiboot_binary(framework_dir, board)
else:
    if not os.path.isfile(bootloader_path):
        bootloader_path = os.path.join(framework_dir, "bootloaders", bootloader_path)

    if not board.get("bootloader", {}):
        sys.stderr.write("Error: missing bootloader configuration!\n")
        env.Exit(1)

if not os.path.isfile(bootloader_path):
    bootloader_path = os.path.join(framework_dir, "bootloaders", bootloader_path)

if board.get("hardware.uart", "no_bootloader").lower() == "no_bootloader":
    sys.stderr.write("Error: board_hardware.bootloader = no bootloader selected! \n")
    env.Exit(1)

if not os.path.isfile(bootloader_path) and "BOOTFLAGS" not in env:
    sys.stderr.write("Error: Couldn't find bootloader image %s\n" % bootloader_path)
    env.Exit(1)

env.Append(
    BOOTUPLOADER="avrdude",
    BOOTUPLOADERFLAGS=[
        "-p",
        "$BOARD_MCU",
        "-C",
        '"%s"'
        % os.path.join(env.PioPlatform().get_package_dir(
            "tool-avrdude-megaavr") or "", "avrdude.conf"),
    ],
    BOOTFLAGS=['-Uflash:w:"%s":i' % bootloader_path],
    UPLOADBOOTCMD="$BOOTUPLOADER $BOOTUPLOADERFLAGS $UPLOAD_FLAGS $BOOTFLAGS",
)

if not env.BoardConfig().get("upload", {}).get("require_upload_port", False):
    # upload methods via USB
    env.Append(BOOTUPLOADERFLAGS=["-P", "usb"])
else:
    env.AutodetectUploadPort()
    env.Append(FUSESUPLOADERFLAGS=["-P", '"$UPLOAD_PORT"'])

if env.subst("$UPLOAD_PROTOCOL") != "custom":
    env.Append(BOOTUPLOADERFLAGS=["-c", "$UPLOAD_PROTOCOL"])
else:
    print(
        "Warning: The `custom` upload protocol is used! The upload and fuse flags may "
        "conflict!\nMore information: "
        "https://docs.platformio.org/en/latest/platforms/atmelavr.html"
        "#overriding-default-bootloader-command\n"
    )

fuses_action = env.SConscript("fuses.py", exports="env")

bootloader_actions = [
    fuses_action,
    env.VerboseAction("$UPLOADBOOTCMD", "Uploading bootloader"),
]

Return("bootloader_actions")

I removed the board_bootloard.pinsparameter to make the setup procedure more similar to my other cores. You can instead use board_hardware.uart = uart0 #or uart0_def or uart0_alt

There are some minor issues we need to work out though.

• First, on the megaAVR-0 microcontrollers, the internal/external oscillator option is no longer done in the fuse bits, but rather by writing to a register in the user application setup procedure. In Arduino IDE, this is triggered by a macro in the boards.txt file; completely invisible for the user. It would be great if PlatformIO could do the same thing: Define USE_EXTERNAL_OSCILLATOR somewhere if board_hardware.oscillator = external is present in platformio.ini. I know this could be added to the build flags, but it would be slightly more elegant if it could be hidden just like in the Arduino IDE. This will harmonize with the way my other cores work and behave.
• Second, We need to figure out a way to give the generated hex file an offset of 0x200 (512 bytes) when the bootloader is in use (board_hardware.uart != no_bootloader present in platformio.ini). If not, it won't be possible to use the bootloader to upload new programs.
• Third, it is currently no possibility to upload using the bootloader. Even though I specify upload_protocol = arduino, PlatformIO forces the USB flag and mutes the provided upload port.
  I think this can be fixed by adding arduino as a programmer here:
  

  platform-atmelmegaavr/builder/main.py

  Line 200 in 3c2f7d8

  if upload_protocol == "jtag2updi":

  
  However arduino does not require use_1200bps_touch or wait_for_upload_port. Below is the error I'm getting with an umodified main.py file:

[env:ATmega3209_optiboot]
platform = atmelmegaavr
framework = arduino
board = ATmega3209
board_build.f_cpu = 16000000L
board_hardware.bod = 4.3v
board_build.variant = 48pin-standard
upload_protocol = arduino
upload_port = /dev/cu.usbserial-1420

$ pio run -t upload -e ATmega3209_optiboot -v
...
avrdude -v -p atmega3209 -C "/Users/hans/.platformio/packages/tool-avrdude-megaavr/avrdude.conf" -c arduino -b 115200 -P usb -U flash:w:.pio/build/ATmega3209_optiboot/firmware.hex:i

...

         Using Port                    : usb
         Using Programmer              : arduino
         Overriding Baud Rate          : 115200
avrdude: ser_open(): can't open device "usb": No such file or directory

avrdude done.  Thank you.

*** [upload] Error 1

However, Modifying the main.py file by replacing line 200 - 203 with the following code works:

    # jtag2updi and arduino are the only protocols that requires serial port
    if upload_protocol == "jtag2updi":
        upload_options = env.BoardConfig().get("upload", {})
        for opt in ("require_upload_port", "use_1200bps_touch", "wait_for_upload_port"):
            upload_options[opt] = True
    elif upload_protocol == "arduino":
        upload_options = env.BoardConfig().get("upload", {})
        upload_options["require_upload_port"] = True
        upload_options["use_1200bps_touch"] = False
        upload_options["wait_for_upload_port"] = False
        env.Append(UPLOADERFLAGS=["-D"])

Note that I've added the -D flag because this is required if you're using the arduino protocol. If not, you're getting an erase error.

[MCUdude]

@valeros any thoughts?

[valeros]

Hi @MCUdude ! I've pushed your development with slight modification, including fixes for the issues your specified above. It'd be great if you could test new functionality using the platform for dev branch. Thanks!

[MCUdude]

I'm not completely done testing, since there are some issues that are preventing me. First, I'm getting the incorrect device signature in Avrdude again. I've manually deleted the tool-avrdude-megaavr package. Here's the output:

$ pio run -t fuses -e fuses_bootloader
Processing fuses_bootloader (platform: https://github.com/platformio/platform-atmelmegaavr.git#develop; board: ATmega3209)
--------------------------------------------------------------------------------------------------------------------------------------------------
Tool Manager: Installing platformio/tool-avrdude-megaavr @ ~1.60300.0
Tool Manager: tool-avrdude-megaavr @ 1.60300.191015 has been installed!
Verbose mode can be enabled via `-v, --verbose` option
CONFIGURATION: https://docs.platformio.org/page/boards/atmelmegaavr/ATmega3209.html
PLATFORM: Atmel megaAVR (1.2.0+sha.daf7155) > ATmega3209
HARDWARE: ATMEGA3209 16MHz, 4KB RAM, 32KB Flash
PACKAGES: 
 - tool-avrdude-megaavr 1.60300.191015 (6.3.0) 
 - toolchain-atmelavr 1.70300.191015 (7.3.0)
LDF: Library Dependency Finder -> http://bit.ly/configure-pio-ldf
LDF Modes: Finder ~ chain, Compatibility ~ soft
Found 0 compatible libraries
Scanning dependencies...
No dependencies
Building in release mode

TARGET CONFIGURATION:
------------------------
Target = atmega3209
Clock speed = 16000000L
Oscillator = internal
BOD level = 2.7v
Save EEPROM = yes
Reset pin mode = reset
------------------------

Selected fuses:
------------------------
[fuse0 / WDTCFG = 0x0]
[fuse1 / BODCFG = 0x0]
[fuse2 / OSCCFG = 0x1]
[fuse4 / TCD0CFG = 0x0]
[fuse5 / SYSCFG0 = 0xc9]
[fuse6 / SYSCFG1 = 0x6]
[fuse7 / APPEND = 0x0]
[fuse8 / BOOTEND = 0x2]
[lfuse / LOCKBIT = 0xc5]
------------------------

Setting fuses...

avrdude: AVR device initialized and ready to accept instructions

Reading | ################################################## | 100% 0.07s

avrdude: Device signature = 0x1e9531
avrdude: Expected signature for ATmega3209 is 1E 95 53
         Double check chip, or use -F to override this check.

avrdude done.  Thank you.

*** [fuses] Error 1

I'm not sure what's causing this. I was able to sneak in the -F flag to continue my testing.

Setting fuses works fine. However, bootloading not so much:

error: Couldn't find bootloader image bootloaders/bootloaders/optiboot/bootloaders/mega0/115200/Optiboot_mega0_UART3_DEF_115200_A7.hex

The corerct path is bootloaders/optiboot/bootloaders/mega0/115200/Optiboot_mega0_UART3_DEF_115200_A7.hex I believe.

Another thing. I'm pretty sure all fuses are supposed to be in capital letters. At least, that's what Microchip uses in Atmel Studio and in their datasheets

[MCUdude]

About the 0x200 offset. I think we should use the upload_protocol value as the indicator, not board_hardware.uart. There's no real reason to specify what exact UART port on your target that's used for serial uploads.

[env:Upload_UART]
platform = https://github.com/platformio/platform-atmelmegaavr.git
framework = arduino
board = ATmega3209
board_build.f_cpu = 16000000L
board_hardware.bod = 4.3v
board_build.variant = 48pin-standard
upload_protocol = arduino
upload_port = /dev/cu.usbserial-1420

In this example, since upload_protocol = arduino, it means that we're using a bootloader, and thus 0x200 is needed. It's also a benefit since upload_protocol has to be defined anyways if you're using a bootloader for uploads.
Another 0x200 related bug; 0x200 is actually added when no bootloader is used, so it's the other way around. But again upload_protocol would be better This is wrong. I got my platformio.ini settings wrong due to the board_hardware.uart parameter confusion I've caused myself.

[valeros]

First, I'm getting the incorrect device signature in Avrdude again. I've manually deleted the tool-avrdude-megaavr package.

Thanks for testing! I see you didn't specified framework = arduino in your configuration. That explains most of the issues. I suppose we should allow using your boards without any framework? In this case, where users should get bootloader binaries? Arduino packages are optional, if users don't run any example with Arduino framework, they won't be downloaded.

Another thing. I'm pretty sure all fuses are supposed to be in capital letters. At least, that's what Microchip uses in Atmel Studio and in their datasheets.

Thanks, now I see why you used %.2X, should be fixed now.

About the 0x200 offset. I think we should use the upload_protocol value as the indicator, not board_hardware.uart.

Thanks, fixed as well.

[MCUdude]

I see you didn't specified framework = arduino in your configuration. That explains most of the issues.

🤦 Of course! I'm being sloppy, sorry! This fixed the issues I faced.

There's one tiny formatting thing I'd like to have fixed:

Selected fuses:
------------------------
[fuse0 / WDTCFG = 0x00]
[fuse1 / BODCFG = 0x00]
[fuse2 / OSCCFG = 0x01]
[fuse4 / TCD0CFG = 0x00]
[fuse5 / SYSCFG0 = 0xC9]
[fuse6 / SYSCFG1 = 0x06]
[fuse7 / APPEND = 0x00]
[fuse8 / BOOTEND = 0x02]
[lfuse / LOCKBIT = 0xc5]
------------------------

You can probably guess what it is? Add a space in front of the = character to get everything aligned 👍

[MCUdude]

Another question @valeros. Is it possible to get access to a parameter defined in one environment in another environment?
Take this example:

[env:environment1]
upload_protocol = xplainedmini_updi

[env:environment2]
upload_protocol = ${environment1.upload_protocol}

This would be very convenient in the platformio.ini template I'm creating for MegaCoreX

EDIT:
I figured it out!

[env:environment1]
upload_protocol = xplainedmini_updi

[env:environment2]
upload_protocol = ${env:environment1.upload_protocol}

[valeros]

Thanks, the formatting should be fine now.

[MCUdude]

Awesome!

Here are the README and platformio.ini template I've created for MegaCoreX users:

https://github.com/MCUdude/MegaCoreX/blob/master/PlatformIO.md

[MCUdude]

Are we ready for a new atmelmegaavr release?

[valeros]

Here are the README and platformio.ini template I've created for MegaCoreX users:

Is there any chance you could port that page to PlatformIO docs? Something similar to the atmelavr docs you added earlier.

Are we ready for a new atmelmegaavr release?

Once the issue #10 is resolved.

[MCUdude]

Once the issue #10 is resolved.

Sure! I do have an Uno Wifi Rev2, so iI can help out with the testing this evening.

[MCUdude]

@valeros are we ready for a new release? I don't think there will be any significant changes in this repo for a while since pretty much all MegaCoreX features are implemented, and the bugs regarding Uno Wifi Rev2/Nano Every (#10) are fixed.

[valeros]

Thanks, released.