Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactoring: replacing singularity with apptainer in variable names #1366

Open
wants to merge 12 commits into
base: dev
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion BrainPortal/app/controllers/bourreaux_controller.rb
Original file line number Diff line number Diff line change
Expand Up @@ -623,7 +623,7 @@ def bourreau_params #:nodoc:
:cms_default_queue, :cms_extra_qsub_args, :cms_shared_dir, :workers_instances,
:workers_chk_time, :workers_log_to, :workers_verbose, :help_url, :rr_timeout, :proxied_host,
:spaced_dp_ignore_patterns, :support_email, :system_from_email, :external_status_page_url,
:docker_executable_name, :docker_present, :singularity_executable_name, :singularity_present,
:docker_executable_name, :docker_present, :apptainer_executable_name, :apptainer_present,
:small_logo, :large_logo, :license_agreements
)
end
Expand Down
8 changes: 4 additions & 4 deletions BrainPortal/app/controllers/tool_configs_controller.rb
Original file line number Diff line number Diff line change
Expand Up @@ -206,7 +206,7 @@ def update #:nodoc:
@tool_config.container_image_userfile_id = other_tc.container_image_userfile_id
@tool_config.container_exec_args = other_tc.container_exec_args.presence
@tool_config.container_index_location = other_tc.container_index_location
@tool_config.singularity_overlays_specs = other_tc.singularity_overlays_specs
@tool_config.apptainer_overlays_specs = other_tc.apptainer_overlays_specs
@tool_config.extra_qsub_args = other_tc.extra_qsub_args
@tool_config.cloud_disk_image = other_tc.cloud_disk_image
@tool_config.cloud_vm_user = other_tc.cloud_vm_user
Expand Down Expand Up @@ -243,7 +243,7 @@ def update #:nodoc:
container_image_userfile_id containerhub_image_name
container_engine container_index_location container_exec_args
inputs_readonly
singularity_overlays_specs singularity_use_short_workdir
apptainer_overlays_specs apptainer_use_short_workdir
boutiques_descriptor_path
)
)
Expand Down Expand Up @@ -304,8 +304,8 @@ def tool_config_params #:nodoc:
:version_name, :description, :tool_id, :bourreau_id, :env_array, :script_prologue, :script_epilogue,
:group_id, :ncpus, :container_image_userfile_id, :containerhub_image_name, :container_index_location,
:inputs_readonly,
:container_engine, :extra_qsub_args, :singularity_overlays_specs, :container_exec_args,
:singularity_use_short_workdir,
:container_engine, :extra_qsub_args, :apptainer_overlays_specs, :container_exec_args,
:apptainer_use_short_workdir,
:boutiques_descriptor_path,
# The configuration of a tool in a VM managed by a
# ScirCloud Bourreau is defined by the following
Expand Down
6 changes: 3 additions & 3 deletions BrainPortal/app/models/bourreau.rb
Original file line number Diff line number Diff line change
Expand Up @@ -80,11 +80,11 @@ def docker_present? #:nodoc:
alias docker_present docker_present? #:nodoc:


def singularity_present? #:nodoc:
singularity_executable_name.present?
def apptainer_present? #:nodoc:
apptainer_executable_name.present?
end

alias singularity_present singularity_present? #:nodoc:
alias apptainer_present apptainer_present? #:nodoc:



Expand Down
138 changes: 69 additions & 69 deletions BrainPortal/app/models/cluster_task.rb

Large diffs are not rendered by default.

50 changes: 25 additions & 25 deletions BrainPortal/app/models/sing_bindmount_data_provider.rb
Original file line number Diff line number Diff line change
Expand Up @@ -22,22 +22,22 @@

# This class implements a Data Provider which fetches
# files out of a single filesystem file. The
# implementation requires Singularity 3.2 or better to
# be installed on the host, as well as a singularity
# implementation requires Apptainer 1.1 or Singularity 3.7 and better
# be installed on the host, as well as an Apptainer
# container image that contains the basic Linux
# commands and the 'rsync' command too.
#
# For the moment the DP is read-only. Allowing
# write access would require a mechanism in CBRAIN
# to prevent more than one singularity process to
# to prevent more than one Apptainer process to
# be accessing the mounted file at any one time.
#
# Bind mounts are performed using a command such as:
#
# singularity shell -B filesystem.img:/my/mount/point:image-src=/inside/dir,ro sing_squashfs.simg
# apptainer shell -B filesystem.img:/my/mount/point:image-src=/inside/dir,ro sing_squashfs.simg
#
# where:
# 1- sing_squashfs.simg is a singularity image file,
# 1- sing_squashfs.simg is an Apptainer image file,
# 2- filesystem.img is either a ext3 or squashfs filesystem,
# 3- /inside/dir is a path inside that filesystem,
# 4- /my/mount/point is a path where the data will be mounted under
Expand All @@ -51,7 +51,7 @@
#
# The attribute 'remote_dir' must be a full path to the location of
# the filesystem image 'filesystem.img' (even though it is not a 'dir').
# The singularity image file will be expected to be called 'sing_squashfs.simg'
# The apptainer image file will be expected to be called 'sing_squashfs.simg'
# and be located in the same subdirectory as 'filesystem.img'.
class SingBindmountDataProvider < SshDataProvider

Expand All @@ -64,14 +64,14 @@ class SingBindmountDataProvider < SshDataProvider
# TODO: Change this is the DP is ever made writable!
BROWSE_CACHE_EXPIRATION = 2.months #:nodoc:

# This is the basename of the singularity image
# This is the basename of the Apptainer image
# we use to access the remote filesystem; we
# expect this image to be installed in the same
# directory that contains it. Its minimum
# requirements are that 1) basic UNIX commands
# exist on it 2) the rsync command is installed
# in it too.
SINGULARITY_IMAGE_BASENAME = 'sing_squashfs.simg'
APPTAINER_IMAGE_BASENAME = 'sing_squashfs.simg'

# We use this to point to the directory INSIDE the container
# were the root of the data is stored.
Expand All @@ -91,7 +91,7 @@ def read_only=(val) #:nodoc:

# This returns the category of the data provider
def self.pretty_category_name #:nodoc:
"Singularity Bind Mount"
"Apptainer Bind Mount"
end

# This uses the new CBRAIN capability of registering
Expand All @@ -109,20 +109,20 @@ def impl_is_alive? #:nodoc:
# Raise an exception with a message indicating what is wrong with the config.
# This method is not part of the official method API
def check_remote_config! #:nodoc:
# Check we have one singularity image file
remote_cmd = "cd #{self.real_remote_dir.to_s.bash_escape} && test -f #{SINGULARITY_IMAGE_BASENAME} && echo OK-Exists"
# Check we have one apptainer image file
remote_cmd = "cd #{self.real_remote_dir.to_s.bash_escape} && test -f #{APPTAINER_IMAGE_BASENAME} && echo OK-Exists"
text = self.remote_bash_this(remote_cmd)
# The following check will also make sure the remote shell is clean!
cb_error "No installed singularity image #{SINGULARITY_IMAGE_BASENAME}, or remote shell is unclean" unless text =~ /\AOK-Exists\s*\z/
cb_error "No installed apptainer image #{APPTAINER_IMAGE_BASENAME}, or remote shell is unclean" unless text =~ /\AOK-Exists\s*\z/

# Check we have the remote filesystem file
remote_cmd = "test -f #{self.remote_dir.to_s.bash_escape} && echo OK-Exists"
cb_error "Filesystem file '#{self.remote_dir}' not found" unless text =~ /\AOK-Exists\s*\z/

# Check we have singularity version 3.7 or better
# Check we have apptainer 1.1 or better or singularity version 3.7 or better
remote_cmd = "(singularity --version 2>/dev/null || apptainer --version 2>/dev/null)"
text = self.remote_bash_this(remote_cmd)
cb_error "Can't find singularity version number on remote host" unless text =~ /^((singularity|apptainer) version )?(\d+)\.(\d+)/
cb_error "Can't find apptainer version number on remote host" unless text =~ /^((singularity|apptainer) version )?(\d+)\.(\d+)/
_, _, tool, major, minor = Regexp.last_match.to_a
major = major.to_i
minor = minor.to_i
Expand All @@ -134,7 +134,7 @@ def check_remote_config! #:nodoc:

# Check that inside the container, the containerized path exists
checkdir = "test -d #{self.containerized_path.bash_escape} && echo OK-Exists"
text = remote_in_sing_bash_this(checkdir)
text = remote_in_apptainer_bash_this(checkdir)
cb_error "No path '#{self.containerized_path}' inside container" unless text =~ /\AOK-Exists\s*\z/

# Well, we passed all the tests
Expand Down Expand Up @@ -179,7 +179,7 @@ def impl_provider_list_all(user=nil, browse_path=nil) #:nodoc:

file_infos = Rails.cache.fetch(cache_key, :expires_in => BROWSE_CACHE_EXPIRATION) do
dir = Pathname.new(self.containerized_path) + browse_path.to_s
text = remote_in_sing_stat_all(dir,".",true)
text = remote_in_apptainer_stat_all(dir,".",true)
stat_reports_to_fileinfos(text)
end

Expand Down Expand Up @@ -211,7 +211,7 @@ def impl_provider_collection_index(userfile, directory = :all, allowed_types = :
type_opt = allowed_types == [ :regular ] ? "f" :
allowed_types == [ :directory ] ? "d" :
nil # we can still filter for other combinations on Ruby side
text = remote_in_sing_stat_all(basedir, subdir, one_level, type_opt)
text = remote_in_apptainer_stat_all(basedir, subdir, one_level, type_opt)
file_infos = stat_reports_to_fileinfos(text)

# Apply more complex filters if necessary
Expand Down Expand Up @@ -268,7 +268,7 @@ def impl_provider_rename(userfile,newname) #:nodoc:
# Returns true if we have to use 'ssh' to
# connect to the remote server. Returns false
# when we can optimize requests by running
# singularity locally. The local situation is
# apptainer locally. The local situation is
# detected pretty much like in the Smart DP
# module: if the hostname is the same as *remote_host*
# or *alternate_host*, and if the *remote_dir* exists
Expand All @@ -294,16 +294,16 @@ def provider_is_remote #:nodoc:
@provider_is_remote
end

def singularity_exec_prefix #:nodoc:
"cd #{self.real_remote_dir.to_s.bash_escape} && singularity -s exec #{self.local_bind_opt} #{SINGULARITY_IMAGE_BASENAME}"
def apptainer_exec_prefix #:nodoc:
"cd #{self.real_remote_dir.to_s.bash_escape} && singularity -s exec #{self.local_bind_opt} #{APPTAINER_IMAGE_BASENAME}"
end

def local_bind_opt #:nodoc:
"-B #{self.filesystem_image_basename.bash_escape}:/cbrain:image-src=/,ro"
end

def remote_rsync_command #:nodoc:
"#{singularity_exec_prefix} rsync"
"#{apptainer_exec_prefix} rsync"
end

def real_remote_dir #:nodoc:
Expand Down Expand Up @@ -333,12 +333,12 @@ def rsync_over_ssh_prefix
rsync
end

def remote_in_sing_bash_this(com) #:nodoc:
newcom = "#{singularity_exec_prefix} bash -c #{com.bash_escape}"
def remote_in_apptainer_bash_this(com) #:nodoc:
newcom = "#{apptainer_exec_prefix} bash -c #{com.bash_escape}"
remote_bash_this(newcom)
end

def remote_in_sing_stat_all(basedir, subdir, one_level = true, find_type = nil) #:nodoc:
def remote_in_apptainer_stat_all(basedir, subdir, one_level = true, find_type = nil) #:nodoc:
max_depth = one_level ? "-maxdepth 1" : ""
type_opt = find_type ? "-type #{find_type}" : ""
# Linux 'stat' command formats:
Expand All @@ -354,7 +354,7 @@ def remote_in_sing_stat_all(basedir, subdir, one_level = true, find_type = nil)
# Linux 'find' command format:
find_format = "E=%y,%m,%s,%U,%u,%G,%g,%A@,%T@,%C@,%p\\n"
com = "cd #{basedir.to_s.bash_escape} && find #{subdir.to_s.bash_escape} #{max_depth} #{type_opt} -printf \"#{find_format}\""
remote_in_sing_bash_this(com)
remote_in_apptainer_bash_this(com)
end

# Given a text file report such as this:
Expand Down
Loading
Loading