Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

improve update.py #1536

Merged
merged 107 commits into from
Jul 21, 2022
Merged
Show file tree
Hide file tree
Changes from 98 commits
Commits
Show all changes
107 commits
Select commit Hold shift + click to select a range
6e5b7a3
remove unused import
fabianegli Apr 28, 2022
d5c1112
use early return pattern
fabianegli Apr 28, 2022
789b76d
bug fix: variable name
fabianegli Apr 28, 2022
e753d9f
use f string instead of format
fabianegli Apr 28, 2022
5764ead
do not use f-strings unnecessarily
fabianegli Apr 28, 2022
e3d24b3
use nested unpacking
fabianegli Apr 29, 2022
2491a03
wrapping f-strings
fabianegli Apr 29, 2022
af726e6
say updates complete only after updates
fabianegli Apr 29, 2022
7f892f2
use a list comprehension
fabianegli Apr 29, 2022
0c2089b
fail if the arguments are incompatible
fabianegli Apr 29, 2022
81a1147
adopt isort
fabianegli Apr 29, 2022
64d27e7
remove unused import
fabianegli Apr 29, 2022
0b58244
use import from
fabianegli Apr 29, 2022
4293b0c
use list comprehension
fabianegli Apr 29, 2022
b170827
let tempfile chose a temporary directory
fabianegli Apr 29, 2022
e9e92f2
collect parameter test prompt+sha
fabianegli Apr 29, 2022
89a115b
collect parameter test module+all
fabianegli Apr 29, 2022
2e44600
raise UserWarning instead of return
fabianegli Apr 29, 2022
6e0ab52
remove unused import
fabianegli Apr 29, 2022
de5dcb7
click doesn't handle mutually exclusive flags
fabianegli May 5, 2022
19f3728
Fix #1419
drpatelh Apr 21, 2022
36e5469
Black
drpatelh Apr 21, 2022
e100be0
Handle missing key, log, call after web builder
ewels May 5, 2022
143c041
Deep copy of schema not needed
ewels May 5, 2022
4cc3b04
Format strings
ewels May 5, 2022
5edb359
print include statement when installing a module
Apr 21, 2022
b0794f7
resolved rebase merge conflict
fabianegli May 12, 2022
dccef6c
remove remove stringtie special case
Apr 21, 2022
2536266
Update actions_ci.py
njspix Apr 26, 2022
1876107
resolve rebase mergeconflicts
Apr 25, 2022
2e713dd
run tests with pytest
Apr 26, 2022
b55aeca
remove test workflow check
Apr 26, 2022
41e2232
add code tests
Apr 26, 2022
20dd418
fix linting
Apr 26, 2022
3622615
apply comments from PR review
Apr 26, 2022
d88a6cd
fix linting
Apr 26, 2022
3f2a1de
apply comments from PR
Apr 27, 2022
f3dc187
fix test
Apr 27, 2022
e7fdd53
always set profile
Apr 27, 2022
005143a
do not abbreviate env var
Apr 27, 2022
9f48c90
add new test
Apr 27, 2022
d59497b
update CHANGELOG
Apr 27, 2022
5a61003
specify test files as test_*.py in pytest.ini
Apr 28, 2022
98f0239
apply comments from PR review
Apr 28, 2022
86dabbf
check if profile is available
Apr 29, 2022
37750dc
modify linting
mirpedrol Apr 29, 2022
a0ac5a1
Add nf-core/modules folder warning
mirpedrol May 2, 2022
e45dbfd
Add valid profile check
mirpedrol May 2, 2022
cfa2e1c
fix linting
mirpedrol May 2, 2022
5dce262
avoid using shell=True when checking profile
mirpedrol May 2, 2022
930601d
Apply suggestions from code review
mirpedrol May 2, 2022
0d4f813
remove unnecessary libraryes
mirpedrol May 2, 2022
b4343ae
replace a weird with a normal dash
fabianegli May 4, 2022
c9acfb5
add documentation for modules mulled command
mirpedrol May 6, 2022
9fd0352
add documentation for modules test command
mirpedrol May 6, 2022
1afd36b
Add biocontainers reference
mirpedrol May 6, 2022
90ec221
First stab at writing a central utils function for GitHub API calls.
ewels Mar 29, 2022
e0f984f
Remove superfluous dict() call
ewels Mar 30, 2022
464510e
Fix sync pytests
ewels Mar 30, 2022
fb2c4cf
Better debug logging.
ewels Mar 30, 2022
d37dbff
Try to rewrite GitHub API calls to use a single shared Session
ewels Apr 13, 2022
605dc8a
Use new class more in the sync code
ewels Apr 13, 2022
002deff
Nearly fix the tests
ewels Apr 13, 2022
4cbbee8
Better exception handling / logging for gh auth failure
ewels May 10, 2022
de429f7
Use XDG_CONFIG_HOME or ~/.config/nf-core/ for the requests cache dir.
ewels May 10, 2022
c0f179d
resolve merge conflicts after rebase Changelog
ewels May 10, 2022
1490a0b
Apply suggestions from code review
ewels May 11, 2022
7227dad
Update nf_core/sync.py
ewels May 11, 2022
ffa278b
Merge branch 'dev' into fabianegli-update-refactor
fabianegli May 12, 2022
6ef8255
Fix wrong merge conflict resolution
fabianegli May 12, 2022
3dfbacc
if-else instead of if-if
fabianegli May 12, 2022
b60a424
collect more parameter checks
fabianegli May 12, 2022
3e203a4
Merge branch 'nf-core:master' into fabianegli-update-refactor
fabianegli May 25, 2022
97fb34e
Merge branch 'dev' into fabianegli-update-refactor
fabianegli May 25, 2022
690268a
Merge branch 'fabianegli-update-refactor' of github.com:fabianegli/to…
fabianegli May 25, 2022
869305e
Merge branch 'dev' into fabianegli-update-refactor
fabianegli May 25, 2022
c793838
Merge branch 'dev' into fabianegli-update-refactor
fabianegli May 27, 2022
e0cb22c
Merge branch 'dev' into fabianegli-update-refactor
fabianegli Jun 1, 2022
3f92853
Merge branch 'dev' into fabianegli-update-refactor
fabianegli Jun 1, 2022
944955c
Merge branch 'fabianegli-update-refactor' of github.com:fabianegli/to…
fabianegli Jun 1, 2022
caad096
fix missing import after merge conflict resolution
fabianegli Jun 1, 2022
038597b
Merge branch 'dev' of github.com:nf-core/tools into fabianegli-update…
ErikDanielsson Jul 20, 2022
1c06c03
Fix failing test due to merge
ErikDanielsson Jul 20, 2022
bdee1d5
Move --all or not code to methods
ErikDanielsson Jul 20, 2022
ea61cb3
Move more stuff to methods
ErikDanielsson Jul 20, 2022
fe42c54
Refactor dry run stuff into methods
ErikDanielsson Jul 20, 2022
d51e097
Always move files to temp dir first, to make sure the installation wo…
ErikDanielsson Jul 20, 2022
2686e9a
Move mod json diff to method
ErikDanielsson Jul 20, 2022
b8cfed3
Move the update method
ErikDanielsson Jul 20, 2022
100d311
fix variable name
fabianegli Jul 20, 2022
4522d69
More misc. refactoring
ErikDanielsson Jul 20, 2022
685d301
Merge branch 'fabianegli-update-refactor' of github.com:fabianegli/to…
ErikDanielsson Jul 20, 2022
7df4f57
Fix tests
ErikDanielsson Jul 20, 2022
e4f6e9b
Don't pass install folder as a list
ErikDanielsson Jul 20, 2022
1949497
Fix install dir issue in ModulesJson
ErikDanielsson Jul 20, 2022
9d2c29c
Update CHANGELOG.md
ErikDanielsson Jul 20, 2022
9145b6c
Update CHANGELOG.md
ErikDanielsson Jul 21, 2022
9cdc79c
Fix short flag of '--show-diff' and update CHANGELOG
ErikDanielsson Jul 21, 2022
be5cf77
some docstring improvements
fabianegli Jul 21, 2022
f39d97d
use questionary.path for file name input
fabianegli Jul 21, 2022
8fddf53
break too long line
fabianegli Jul 21, 2022
d949be1
ensure diff file exists after diff file setup
fabianegli Jul 21, 2022
88ff09f
Fix changelog
ErikDanielsson Jul 21, 2022
c3a0ed9
Merge branch 'fabianegli-update-refactor' of github.com:fabianegli/to…
ErikDanielsson Jul 21, 2022
6c79bd0
Merge branch 'dev' of github.com:nf-core/tools into fabianegli-update…
ErikDanielsson Jul 21, 2022
ee59392
Update docs
ErikDanielsson Jul 21, 2022
22f9ca7
Merge branch 'dev' of github.com:nf-core/tools into fabianegli-update…
ErikDanielsson Jul 21, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@
- Command `nf-core modules test` obtains module name suggestions from installed modules ([#1624](https://github.com/nf-core/tools/pull/1624))
- Add `--base-path` flag to `nf-core modules` to specify the base path for the modules in a remote. Also refactored `modules.json` code. ([#1643](https://github.com/nf-core/tools/issues/1643))
- Rename methods in `ModulesJson` to remove explicit reference to `modules.json`
- Fix inconsistencies in the `--show-diff` flag `nf-core modules update`. Refactor `nf-core modules update` ([#1536](https://github.com/nf-core/tools/pull/1536))
fabianegli marked this conversation as resolved.
Show resolved Hide resolved

## [v2.4.1 - Cobolt Koala Patch](https://github.com/nf-core/tools/releases/tag/2.4) - [2022-05-16]

Expand Down
2 changes: 1 addition & 1 deletion nf_core/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -504,7 +504,7 @@ def install(ctx, tool, dir, prompt, force, sha):
help="Preview / no preview of changes before applying",
)
@click.option(
"-p",
"-D",
"--save-diff",
type=str,
metavar="<filename>",
Expand Down
7 changes: 3 additions & 4 deletions nf_core/modules/install.py
Original file line number Diff line number Diff line change
Expand Up @@ -85,11 +85,10 @@ def install(self, module):
current_version = modules_json.get_module_version(module, self.modules_repo.fullname)

# Set the install folder based on the repository name
install_folder = [self.dir, "modules"]
install_folder.extend(os.path.split(self.modules_repo.fullname))
install_folder = os.path.join(self.dir, "modules", self.modules_repo.fullname)

# Compute the module directory
module_dir = os.path.join(*install_folder, module)
module_dir = os.path.join(install_folder, module)

# Check that the module is not already installed
if (current_version is not None and os.path.exists(module_dir)) and not self.force:
Expand Down Expand Up @@ -135,7 +134,7 @@ def install(self, module):

# Print include statement
module_name = "_".join(module.upper().split("/"))
log.info(f"Include statement: include {{ {module_name} }} from '.{os.path.join(*install_folder, module)}/main'")
log.info(f"Include statement: include {{ {module_name} }} from '.{os.path.join(install_folder, module)}/main'")

# Update module.json with newly installed module
modules_json.update(self.modules_repo, module, version)
Expand Down
5 changes: 2 additions & 3 deletions nf_core/modules/modules_json.py
Original file line number Diff line number Diff line change
Expand Up @@ -301,7 +301,7 @@ def check_up_to_date(self):
base_path = contents["base_path"]

modules_repo = nf_core.modules.modules_repo.ModulesRepo(remote_url=remote, base_path=base_path)
install_folder = os.path.split(modules_repo.fullname)
install_dir = os.path.join(self.dir, "modules", modules_repo.fullname)

for module, entry in modules.items():
sha = entry.get("git_sha")
Expand All @@ -313,8 +313,7 @@ def check_up_to_date(self):
)
remove_from_mod_json[repo].append(module)
continue
module_dir = [self.dir, "modules", *install_folder]
if not modules_repo.install_module(module, module_dir, sha):
if not modules_repo.install_module(module, install_dir, sha):
if repo not in remove_from_mod_json:
remove_from_mod_json[repo] = []
log.warning(f"Could not install module '{module}' in '{repo}' - removing from modules.json")
Expand Down
8 changes: 7 additions & 1 deletion nf_core/modules/modules_repo.py
Original file line number Diff line number Diff line change
Expand Up @@ -284,7 +284,7 @@ def install_module(self, module_name, install_dir, commit):
return False

# Copy the files from the repo to the install folder
shutil.copytree(self.get_module_dir(module_name), os.path.join(*install_dir, module_name))
shutil.copytree(self.get_module_dir(module_name), os.path.join(install_dir, module_name))

# Switch back to the tip of the branch
self.checkout_branch()
Expand Down Expand Up @@ -338,6 +338,12 @@ def get_module_git_log(self, module_name, depth=None, since="2021-07-07T00:00:00
commits = ({"git_sha": commit.hexsha, "trunc_message": commit.message.partition("\n")[0]} for commit in commits)
return commits

def get_latest_module_version(self, module_name):
"""
Returns the latest commit in the repository
"""
return list(self.get_module_git_log(module_name, depth=1))[0]["git_sha"]

def sha_exists_on_branch(self, sha):
"""
Verifies that a given commit sha exists on the branch
Expand Down
Loading