(Question) Separate config file for custom keymaps

[jfernandez]

AFAIK toshy only looks for configs in a single file, and there are some special sections where you are supposed to drop custom keymaps. I want to keep my custom mappings in a separate file to put in my dotfiles git repo.

Is it possible to do this?

[RedBearAK]

@jfernandez

The tl;dr version of the answer to this is that you can back up your config file (toshy_config.py) and sync it between multiple machines, although at times this may fail if you do a new install or upgrade of Toshy on one machine and another machine is too far behind.

There is a much longer answer:

I certainly don't have the skills to have written the original keymapper that Kinto was using (xkeysnail), then someone else forked it into "keyszer" and fixed a bunch of little issues and added some functionality. What I've mainly done on the keymapper side is come up with a strategy to support to absolute mess of the Wayland compositor landscape, which has required nearly a dozen different methods to get the window context for app-specific keymapping. Eventually I forked keyszer into xwaykeyz, but other than the Wayland stuff it's still pretty much the same.

In keyszer, there is an include() function call in the API that theoretically lets you do something similar to "sourcing" a file with shell script commands from another shell script. However I am enough of a sub-genius that I have to rely on VSCode's helpful syntax highlighting and auto-completion to make sure I'm not making silly mistakes in the config file, which is full of all sorts of custom functions and lists, and needs to call on all sorts of API functions in the keymapper. I had so much trouble trying to work with the include() function to separate parts of the config into different files that I eventually gave up on it. This is when I came up with the "editable slices" idea, to allow parts of the config file to be upgraded when reinstalling Toshy, while user customizations of that same file are retained.

If I was a lot smarter there could be much better separation between the user's "config" and the rest of the keymapper, but the reality of how the keymapper was designed means the config file is just another part of the "program" and there are a lot of dynamic things going on in the config file, in addition to just static remaps of shortcuts. In fact Toshy's config is far more dynamic than the original Kinto config, with automatic identification of keyboard types as one of the main examples. That particular example allows the use of different keyboard types on the same machine.

So what the installer does is back up any existing Toshy folder into a timestamped subfolder in ~/.config/ (except for the Python virtual environment inside it, which can be quite large) and then it literally creates an entirely new folder at ~/.config/toshy/ from scratch, including a new Python virtual environment. This was essential for keeping the environment consistent and making sure that Toshy would always work as it was evolving. After that it pulls the content from the "slices" in your old config file and splices your customizations into the new default config file. So far this has been working well in the real world, but I know it's kind of annoying to have to look through a very large file to find those editable sections. That's why I have the ASCII art and "tags" like user_apps on each section, for easier navigation of the config file.

In the end, all you really need to do is keep the installed version of Toshy kind of consistent across machines, and sync the ~/.config/toshy/toshy_config.py file between them. As long as you do things in your customizations like restrict remaps that should only happen in a specific desktop environment (GNOME/KDE/Xfce, etc.) with an if statement or correct conditional, it should be possible to use the same config file on any number of different machines without conflicts. With a small caveat for machine-specific hardware/media key remaps, if you have any. This is typically only necessary on laptops. But there's no built-in way to identify a specific machine and restrict a keymap to just being active on that machine. It's all Python so there's probably a simple way to do that with a standard Python module by accessing the machine's serial number or something.

If you think you can help move the keymapper toward having better config/code separation without losing functionality, I'm always open to concrete suggestions or contributions. Meanwhile, this is the best answer/explanation I can offer.

[RedBearAK]

Just a follow-up on this, the techniques I've tried to get a machine-specific ID number have required superuser permissions, so I'm a little confused about how KDE's "About" panel shows me a serial number without needing a password. It appears to be the "product_serial".

Anyway, it would be possible to just stash the board serial, product serial, or product uuid in a text file, after reading it with superuser permissions from some command like this:

sudo cat /sys/class/dmi/id/board_serial
sudo cat /sys/class/dmi/id/product_serial
sudo cat /sys/class/dmi/id/product_uuid

Then that text file could be read from the config file to enforce a machine-specific if statement to make a keymap only exist on one machine. Thus making a single config file usable on multiple laptops that require different hardware/media key remaps.

This is some example code GPT 4o just provided to safely read the file and use the unique value. Looks pretty good.

def read_value_from_file(filepath):
    """
    Safely reads a single value from a file.
    
    Args:
        filepath (str): The path to the file containing the value.
    
    Returns:
        str: The value from the file, or None if the file doesn't exist or cannot be read.
    """
    try:
        with open(filepath, "r") as file:
            return file.read().strip()  # Strip to remove any extra newlines or spaces
    except FileNotFoundError:
        print(f"File not found: {filepath}")
    except PermissionError:
        print(f"Permission denied: Unable to read file {filepath}")
    except Exception as e:
        print(f"An unexpected error occurred: {e}")
    return None

# Example usage
unique_value_file = "/path/to/your/unique_value_file"
stored_value = read_value_from_file(unique_value_file)

# Use the value in an `if` statement
if stored_value == "SPECIFIC_VALUE":
    print("Value matches. Executing code...")
else:
    print("Value does not match or file could not be read.")

[RedBearAK]

I found a simpler solution to grabbing a unique machine ID, which should work for most Linux systems that anyone would ever run Toshy on. It's a file that DBus uses to uniquely identify a physical system. This same ID is also usually copied or symlinked for use by systemd, but should exist on systems that don't have systemd, like Void Linux. As long as DBus is there.

This has been added to the default config files.

# Global variable to store the local machine ID at runtime, for machine-specific keymaps.
# Allows syncing a single config file between different machines without overlapping the
# hardware/media key overrides, or any other machine-specific customization.
# Get the ID for each machine from /var/lib/dbus/machine-id for use in `if` conditions.
MACHINE_ID = None


def read_machine_id():
    """Reads the machine ID from the D-Bus or systemd file and stores it in a global variable."""
    global MACHINE_ID
    paths = ["/var/lib/dbus/machine-id", "/etc/machine-id"]

    for path in paths:
        try:
            with open(path, "r") as f:
                MACHINE_ID = f.read().strip()
                obfuscated_ID = f"{MACHINE_ID[:5]} ... {MACHINE_ID[-5:]}"
                debug(f"Machine ID successfully read from {path} (obfuscated): '{obfuscated_ID}'")
                return  # Exit after successfully reading the ID
        except FileNotFoundError:
            continue  # Try the next file if this one is missing
        except PermissionError:
            error(f"Permission denied when trying to read {path}.")
            return
        except Exception as e:
            error(f"Unexpected error occurred while reading {path}: {e}")
            return

    # If no valid machine ID is found
    error("Error: Could not retrieve a valid machine ID from known paths.")


read_machine_id()

With this in place, it should be possible to use the machine ID from a specific system to restrict a keymap to being active or loading into memory only on that physical system. For example, you could something like this:

my_Acer_Aspire_laptop = '3210b7e9a6ba462689bf633a03d7e21a'

if MACHINE_ID == my_Acer_Aspire_laptop:
    keymap(usual arguments for a keymap() call go here)

To see what your machine ID is, cat one of these files:

cat /var/lib/dbus/machine-id
cat /etc/machine-id

If both files exist, they should contain the same value.

The files are both user-readable, so they aren't particularly secret. I'm still thinking about whether it would make sense to hash the value, but that would make it more difficult for the user to find the correct value to store in the variable or use in the if condition, unless I also make a script/command to get the same hashed value.

[RedBearAK]

I put the code for the machine ID in a separate Python module (lib/machine_context.py) and changed it to produce a shortened hash of the DBus machine ID. The same Python module is then executed by the terminal command toshy-machine-id to reveal the unique ID for your system. That ID can then be stored in your config file in an appropriately named variable, and used to control whether a modmap or keymap becomes active on a particular system.

For example, for my Acer Aspire laptop:

my_Acer_Aspire_laptop = 'abcd1234'

if MACHINE_ID == my_Acer_Aspire_laptop:
    debug("Activating keymap for Acer Aspire laptop keyboard hardware/media keys.")
    keymap("User hardware keys - Acer Aspire", {

        C("CapsLock"):          toggle_and_show_capslock_state, # Show CapsLock state notification
        C("NumLock"):           toggle_and_show_numlock_state,  # Show NumLock state notification

        # My hardware Sleep key on function row - BIOS set to make hardware functions require Fn key
        C("Sleep"):                 None,                           # Disable hardware Sleep key (F1 + Fn)
        # My brightness keys on function row - BIOS set to make hardware functions require Fn key
        C("F3"):                    C("Brightnessdown"),            # Turn F3 into brightness down button
        C("F4"):                    C("Brightnessup"),              # Turn F4 into brightness up button
        # My volume keys on function row - BIOS is set to make hardware functions require Fn key
        C("F9"):                    C("Mute"),                      # Turn F9 into mute button
        C("F10"):                   C("Volumedown"),                # Turn F10 into volume down button
        C("F11"):                   C("Volumeup"),                  # Turn F11 into volume up button
        # Reverse the behavior of the volume keys to be function keys when I use Fn key
        C("Mute"):                  C("F9"),                        # Turn mute button into F9
        C("Volumedown"):            C("F10"),                       # Turn volume down button into F10
        C("Volumeup"):              C("F11"),                       # Turn volume up button into F11

    }, when = lambda ctx:
        cnfg.screen_has_focus and
        matchProps(not_clas=remoteStr)(ctx)
    )

In this example I chose to also put a line inside the if block that prints out a debug line about the machine-specific keymap being activated. This would only appear on that specific system, and only when using toshy-debug to see verbose logging in the terminal. On any other machine, that keymap will never enter the memory or be available. A different keymap or modmap could do entirely different things on another physical system.

The machine ID is reliable up to the point where you reinstall Linux on the same system from scratch, or do something else to reset the DBus/systemd ID. It's not as permanent as the physical hardware serial or product UUID.

The MACHINE_ID variable is now considered part of the "environment" in the config file, but is kept separate from the typical environmental info revealed by toshy-env, since it is linked to one specific system instead of just being generic information about the login session.