Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Full stateless mode when working with single instances and other improvements for 1.0 #80

Merged
merged 22 commits into from
Oct 26, 2022

Conversation

natefoo
Copy link
Member

@natefoo natefoo commented Oct 20, 2022

Stateless

Instead of the state dir, you now just provide the path to a Galaxy config via one of --config-file, $GRAVITY_CONFIG_FILE, or $GALAXY_CONFIG_FILE. This removes the need for configstate.yaml entirely, but it is still used if you provide a state dir instead of a config file, for multi-instance support (but this is the far less common use case). got rid of that too.

We'll need to update Galaxy's startup scripts and the galaxyproject.galaxy Ansible role to undo the addition of $GRAVITY_STATE_DIR to the venv's activate script since it's no longer desirable to set it.

I recommend setting $GRAVITY_CONFIG_FILE so you don't have to provide the path every time, but I don't think we need to do this automatically like we did with the state dir var. That said, if you run as root, it'll automatically look in a few places, as mentioned in the docs:

  • /etc/galaxy/gravity.yml
  • /etc/galaxy/galaxy.yml
  • /etc/galaxy/gravity.d/*.y(a?)ml

systemd is the default process manager when running as root. It remains supervisor when running as an unprivileged user, but systemd user mode is also supported.

Dropped subcommands

Because there's no more state, there's no more registering, so all of the commands to do with that are gone, e.g. register, deregister, rename, etc.

Multi-Instance

If the value of gravity is a list instead of a dict, you can have multiple Gravity configs in a single file. Previously this was supported by registering multiple config files, which had to be stored in the configstate, but that's 100% gone now.

Couple of things still to do here:

  • Support for multiple --config-file options so that you can have multiple files done
  • If you set an instance_name then (especially with systemd) it should always use that when generating unit names (right now it only does if there are more than one instance configured), because otherwise configs will clobber eachother when you run update separately with different configs. There's a TODO about this in test_operations.py as well. done

Reports

Support has been added for running Galaxy Reports via Gravity. Config options (including proxy URL prefix) are in the documentation. Addresses galaxyproject/galaxy#14591.

Including via !include

Support for !include is dropped since it never worked properly anyway (see #78), but the same split-config-file functionality is now possible using the galaxy_config_file option in the gravity section of the config.

AttributeDict and config.attribs

Back when I wrote Gravity I thought the idea of a dict that was also accessible by attributes was a great idea. Then I made the same objects have class attributes, properties, and all kinds of other stuff and it caused a whole bunch of headaches. In addition, the attribs section of the config was a concept that was held over from the configstate days. Now everything has either been moved to ConfigFile or Service and attribs and AttributeDict are both dropped. ConfigFile and Service are both now pydantic model classes so it's now much better defined what their attributes are.

Other stuff

  • Show the Galaxy version in galaxyctl list --version
  • Add a print-only --no-exec flag to galaxyctl exec for debugging/informational purposes.
  • Update the documentation for 1.0.0.

that can be set in the Gravity config. Improvments for fully stateless
mode. Move the logic from whether supervisord should be started from the
commands to the supervisor PM.
set, or from anywhere if $GALAXY_CONFIG_FILE is set.
Related: class vars are never a good idea, even when you think maybe you
have a case for one.
… /etc/galaxy.

Also, support setting multiple config files via the command line option
or env var.
…temd service when an instance name is set, even if there is only one instance.
@natefoo
Copy link
Member Author

natefoo commented Oct 24, 2022

The flakiness of the test seems to be new with the most recent changes, but I have no idea how it could be caused by these changes.

@natefoo
Copy link
Member Author

natefoo commented Oct 24, 2022

Turns out it was sometimes writing both the original and the force-updated files in the same kernel cycle (between timer interrupts), so the mtime really was the same both before and after. Hooray for Linux being fast, I guess?

@natefoo natefoo merged commit cbf5d74 into galaxyproject:release_1.x Oct 26, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant