Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove paskino channel requirement #204

Closed
wants to merge 8 commits into from
Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 9 additions & 11 deletions .github/workflows/README.md
Original file line number Diff line number Diff line change
@@ -1,25 +1,25 @@
# GitHub Actions

## Building the Conda Package: [conda_build_and_publish](https://github.com/TomographicImaging/iDVC/blob/master/.github/workflows/conda_build_and_publish.yml)
This github action builds and tests the conda package, by using the [conda-package-publish-action](https://github.com/paskino/conda-package-publish-action)
## Building the Conda Package: [conda_build_and_publish](./conda_build_and_publish.yml)
This github action builds and tests the conda package, by using the [conda-package-publish-action](https://github.com/TomographicImaging/conda-package-publish-action)

When pushing to master *all* variants are built and tested.

When making an [annotated](https://git-scm.com/book/en/v2/Git-Basics-Tagging) tag, *all* variants are built, tested and published to the [ccpi conda channel for idvc](https://anaconda.org/ccpi/idvc/files). This includes linux, windows and macOS versions.

When opening or modifying a pull request to master, a single variant is built and tested, but not published. This variant is `python=3.7` and `numpy=1.18`.

## Building/Publishing Documentation: [docs_build_and_publish](https://github.com/TomographicImaging/iDVC/blob/master/.github/workflows/docs_build_and_publish.yml)
## Building/Publishing Documentation: [conda_build_and_publish](./conda_build_and_publish.yml)

This github action builds and optionally publishes the documentation located in [docs/source](https://github.com/TomographicImaging/iDVC/tree/master/docs/source).
This github action builds and optionally publishes the documentation located in [docs/source](../../docs/source).

The github action has two jobs:

1. [build](https://github.com/TomographicImaging/iDVC/blob/master/.github/workflows/docs_build_and_publish.yml#L12):
1. [docs](./conda_build_and_publish.yml#L29):
- builds the documentation with sphinx
- uses upload-artifact to upload the html files which may then be used by **publish**

2. [publish](https://github.com/TomographicImaging/iDVC/blob/master/.github/workflows/docs_build_and_publish.yml#L27):
2. [publish](./conda_build_and_publish.yml#L42):
- uses download-artifact to retrieve the built html files
- pushes the html files to the gh-pages branch

@@ -28,17 +28,17 @@ When opening or modifying a pull request to master, `build` is run, but not `pub
When pushing to master or tagging, the documentation is built *and* published (both the `build` and `publish` jobs are run).

### Viewing Built Documentation
The `build` job builds the documentation and uploads it as an [artifact](https://github.com/TomographicImaging/iDVC/blob/master/.github/workflows/docs_build_and_publish.yml#L21),
The `build` job builds the documentation and uploads it as an [artifact](./conda_build_and_publish.yml#L37),
in a folder named `DocumentationHTML`.
This can be found by going to the ‘Actions’ tab, and selecting the appropriate run of `.github/workflows/docs_build_and_publish.yml`.
This can be found by going to the ‘Actions’ tab, and selecting the appropriate run of [conda_build_and_publish.yml](./conda_build_and_publish.yml).

When viewing the summary for the run of the action, there is an `Artifact` section at the bottom of the page.
Clicking on `DocumentationHTML` allows you to download a zip folder containing the built html files.
This allows you to preview the documentation site before it is published.

### Publication of the Documentation
The documentation is hosted on the [github site](https://tomographicimaging.github.io/iDVC/) associated with the repository.
This is built from the [gh-pages branch](https://github.com/TomographicImaging/iDVC/tree/gh-pages).
This is built from the [gh-pages branch](https://github.com/TomographicImaging/iDVC/tree/gh-pages).

If you are an admin of the repository, you are able to see the settings for the site by going to `Settings->Pages`.

@@ -52,5 +52,3 @@ To get the action to work I first had to:
3. [Set the source](https://github.com/TomographicImaging/iDVC/settings/pages) for our github pages to be the gh-pages branch.

I followed the examples on the [sphinx build action page](https://github.com/marketplace/actions/sphinx-build), specifically this [example workflow](https://github.com/ammaraskar/sphinx-action-test/blob/master/.github/workflows/default.yml)


62 changes: 52 additions & 10 deletions .github/workflows/conda_build_and_publish.yml
Original file line number Diff line number Diff line change
@@ -1,27 +1,69 @@
name: conda_build

name: conda
on:
release:
types: [published]
push:
branches: [ master ]
tags:
- '**'
branches: [master]
tags: ['**']
pull_request:
branches: [ master ]

branches: [master]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v1
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: publish-to-conda
uses: paskino/conda-package-publish-action@v1.4.4
uses: TomographicImaging/conda-package-publish-action@v2
with:
subDir: 'recipe'
channels: '-c conda-forge -c ccpi -c paskino'
channels: '-c conda-forge -c ccpi'
AnacondaToken: ${{ secrets.ANACONDA_TOKEN }}
publish: ${{ github.event_name == 'push' && startsWith(github.event.ref, 'refs/tags') }}
test_all: ${{(github.event_name == 'push' && startsWith(github.event.ref, 'refs/tags')) || (github.ref == 'refs/heads/master')}}
convert_win: false
convert_osx: false
test_pyver: 3.9
test_npver: 1.22
docs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: ammaraskar/sphinx-action@master
with:
docs-folder: "docs/"
pre-build-command: python -m pip install --upgrade pip; pip install sphinx_rtd_theme
# Create an artifact of the html output.
- uses: actions/upload-artifact@v3
with:
name: DocumentationHTML
path: docs/build/html/
publish:
runs-on: ubuntu-latest
needs: docs
if: github.ref == 'refs/heads/master'
steps:
- name: Download artifact of the html output.
uses: actions/download-artifact@v3
with:
name: DocumentationHTML
path: docs/build/html/
- name: Commit documentation changes
run: |
git clone https://github.com/TomographicImaging/iDVC.git --branch gh-pages --single-branch gh-pages
cp -r docs/build/html/* gh-pages/
cd gh-pages
touch .nojekyll
git config --local user.email "action@github.com"
git config --local user.name "GitHub Action"
git add .
git commit -m "Update documentation" -a || true
# The above command will fail if no changes were present, so we ignore
# that.
- name: Push changes
uses: ad-m/github-push-action@master
with:
branch: gh-pages
directory: gh-pages
github_token: ${{ secrets.GITHUB_TOKEN }}
54 changes: 0 additions & 54 deletions .github/workflows/docs_build_and_publish.yml

This file was deleted.

10 changes: 5 additions & 5 deletions docs/source/installation.rst
Original file line number Diff line number Diff line change
@@ -5,9 +5,9 @@ Currently the app is available to install on windows. Linux and MacOS versions a

Installing the App
==================
1. Install miniconda: https://docs.conda.io/en/latest/miniconda.html
2. Open an anaconda prompt (miniconda) and type...
3. ``conda create --name idvc idvc=22.3.0 -c ccpi -c paskino -c conda-forge -c defaults --override-channels`` (or replace 22.3.0 with the latest version number)
1. Install miniconda: https://docs.conda.io/en/latest/miniconda.html
2. Open an anaconda prompt (miniconda) and type...
3. ``conda create --name idvc idvc=22.3.0 -c ccpi -c conda-forge -c defaults --override-channels`` (or replace 22.3.0 with the latest version number)
4. ``activate idvc``
5. ``idvc``

@@ -19,7 +19,7 @@ If you have previously installed the app, to get an updated version:

1. Open anaconda prompt (miniconda) and type…
2. ``activate idvc`` (note that when you created your environment, if you called it something else you need to replace 'idvc' with your environment name.)
3. ``conda install idvc=22.3.0 -c ccpi -c paskino -c conda-forge -c defaults --override-channels`` (or replace 22.3.0 with the latest version number)
3. ``conda install idvc=22.3.0 -c ccpi -c conda-forge -c defaults --override-channels`` (or replace 22.3.0 with the latest version number)
4. Then use: ``idvc`` to open up the app, as normal

Installing the DVC Executable Only
@@ -29,7 +29,7 @@ Alternatively, if you would only like to install the dvc executable and not the

1. Install miniconda<https://docs.conda.io/en/latest/miniconda.html>
2. Open an anaconda prompt (miniconda) and type….
3. ``conda create --name dvc-core ccpi-dvc -c ccpi -c paskino -c conda-forge -c defaults --override-channels``
3. ``conda create --name dvc-core ccpi-dvc -c ccpi -c conda-forge -c defaults --override-channels``
4. ``activate dvc-core``
5. ``dvc``

1 change: 0 additions & 1 deletion environments/idvc22.3.0.linux.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
name: idvc
channels:
- ccpi
- paskino
- conda-forge
- defaults
dependencies:
1 change: 0 additions & 1 deletion environments/idvc22.3.0.windows.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
name: idvc
channels:
- ccpi
- paskino
- conda-forge
- defaults
dependencies:
1 change: 0 additions & 1 deletion recipe/dev_environment.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
name: idvc_dev
channels:
- ccpi
- paskino
- conda-forge
dependencies:
- python
1 change: 0 additions & 1 deletion recipe/idvc_environment.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
name: idvc
channels:
- ccpi
- paskino
- conda-forge
dependencies:
- idvc
28 changes: 13 additions & 15 deletions src/idvc/io.py
Original file line number Diff line number Diff line change
@@ -30,7 +30,7 @@
cilNumpyCroppedReader,
cilNumpyResampleReader,
cilTIFFResampleReader,
cilTIFFCroppedReader,
cilTIFFCroppedReader,
Converter)
from eqt.threading import Worker
from PySide2 import QtCore, QtGui
@@ -103,8 +103,8 @@ def createImageData(main_window, image_files, output_image, *finish_fn_args, inf
elif file_extension in ['tif', 'tiff', '.tif', '.tiff']:
createProgressWindow(main_window, "Converting", "Converting Image")
# filenames, reader, output_image, convert_numpy = False, image_info = None, progress_callback=None
image_worker = Worker(loadTif, image_files, output_image, convert_numpy=convert_numpy,

image_worker = Worker(loadTif, image_files, output_image, convert_numpy=convert_numpy,
image_info=info_var, resample=resample, crop_image=crop_image, target_size=target_size,
origin=origin, target_z_extent=target_z_extent)

@@ -124,7 +124,7 @@ def createImageData(main_window, image_files, output_image, *finish_fn_args, inf
error_text = "Error reading file: ({filename})".format(
filename=image)
displayFileErrorDialog(
main_window, message=error_text, title=error_title,
main_window, message=error_text, title=error_title,
detailed_message='File format is not supported. Accepted formats include: .mhd, .mha, .npy, .tif, .raw')
return

@@ -141,7 +141,7 @@ def createImageData(main_window, image_files, output_image, *finish_fn_args, inf
if finish_fn is not None:
rif = partial(runIfFinishedCorrectly, main_window=main_window, finish_fn=finish_fn, *finish_fn_args, **finish_fn_kwargs)
image_worker.signals.result.connect(rif)

main_window.threadpool = QThreadPool()
main_window.threadpool.start(image_worker)
print("Started worker")
@@ -186,7 +186,7 @@ def displayFileErrorDialog(main_window, message, title, detailed_message):
title : str
The title of the dialog
detailed_message : str
The detailed message to display in the dialog
The detailed message to display in the dialog

'''
msg = QMessageBox(main_window)
@@ -200,14 +200,14 @@ def displayFileErrorDialog(main_window, message, title, detailed_message):
def displayErrorDialogFromWorker(main_window, error):
'''This is a new version of displayFileErrorDialog that takes an error object as an argument.
This function is meant to be used with the Worker class, which passes the error object to the error signal.

The error object is a tuple containing (exctype, value, traceback.format_exc())
https://github.com/paskino/qt-elements/blob/b34e7886f7e395683bbb618cc925ede8426fe8cd/eqt/threading/QtThreading.py#L83
https://github.com/TomographicImaging/eqt/blob/b34e7886f7e395683bbb618cc925ede8426fe8cd/eqt/threading/QtThreading.py#L83

Additionally, this function does not make use of the main_window.e.ErrorMessage() function of ErrorObserver.

Example Usage:
Suppose you have a Worker, any error that occurs in the worker will emit the error signal, which can be
Suppose you have a Worker, any error that occurs in the worker will emit the error signal, which can be
connected to this function.

ff = partial(displayErrorDialogFromWorker, main_window)
@@ -491,7 +491,7 @@ def loadTif(*args, **kwargs):

# time.sleep(0.1) #required so that progress window displays
# progress_callback.emit(10)

if resample:
reader = cilTIFFResampleReader()
reader.SetFileName(filenames)
@@ -503,7 +503,7 @@ def loadTif(*args, **kwargs):
print("Spacing ", output_image.GetSpacing())
header_length = reader.GetFileHeaderLength()
print("Length of header: ", header_length)

shape = reader.GetStoredArrayShape()
if not reader.GetIsFortran():
shape = shape[::-1]
@@ -525,7 +525,7 @@ def loadTif(*args, **kwargs):
if image_info is not None:
image_info['sampled'] = False
image_info['cropped'] = True

reader.AddObserver(vtk.vtkCommand.ProgressEvent, partial(
getProgress, progress_callback=progress_callback))
reader.SetFileName(filenames)
@@ -563,7 +563,7 @@ def loadTif(*args, **kwargs):
reader.Update()

shape = reader.GetStoredArrayShape()

progress_callback.emit(80)

image_data = reader.GetOutput()
@@ -1021,5 +1021,3 @@ def save_tiff_stack_as_raw(filenames: list, output_fname: str, progress_callback
slice_data = Converter.vtk2numpy(reader.GetOutput())
f.write(slice_data.tobytes())
progress_callback.emit(int(start_progress + (end_progress - start_progress) * (filenames.index(el) / steps)))