Merge pull request #272 from MannLabs/develop

Develop
MannLabs · Jul 18, 2023 · 14fcf38 · 14fcf38
2 parents 8a6b7dd + 21134d1
commit 14fcf38
Show file tree

Hide file tree

Showing 18 changed files with 257 additions and 91 deletions.
diff --git a/.github/workflows/cla_assistant.yml b/.github/workflows/cla_assistant.yml
diff --git a/.github/workflows/publish_and_release.yml b/.github/workflows/publish_and_release.yml
@@ -6,6 +6,10 @@ on:
 
 name: Publish on PyPi and release on GitHub
 
+env:
+  REGISTRY: ghcr.io
+  IMAGE_NAME: MannLabs/alphatims
+
 jobs:
   Version_Bumped:
     runs-on: ubuntu-latest
@@ -193,6 +197,49 @@ jobs:
         uses: pypa/gh-action-pypi-publish@master
         with:
           password: ${{ secrets.PYPI_API_TOKEN }}
+
+  Create_Docker_Release:
+    # Adapted from wfondrie/mokapot's docker actions
+    runs-on: ubuntu-latest
+    needs: [Create_Linux_Release]
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v3
+
+      - name: Set up Python
+        uses: actions/setup-python@v4
+        with:
+          python-version: '3.9'
+
+      - name: Build Wheel
+        run: |
+          python setup.py bdist_wheel
+
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v2
+
+      - name: Login to the GitHub Container Registry
+        uses: docker/login-action@v1
+        with:
+          registry: ${{ env.REGISTRY }}
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Extract metadata for Docker
+        id: meta
+        uses: docker/metadata-action@v3
+        with:
+          images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
+
+      - name: Push the Docker image to the GHCR
+        uses: docker/build-push-action@v3
+        with:
+          context: .
+          push: true
+          platforms: linux/amd64,linux/arm64
+          tags: ${{ steps.meta.outputs.tags }}
+          labels: ${{ steps.meta.outputs.labels }}
+
   Test_PyPi_Release:
     name: Test_PyPi_version_on_${{ matrix.os }}
     runs-on: ${{ matrix.os }}

diff --git a/Dockerfile b/Dockerfile
@@ -0,0 +1,22 @@
+
+# FROM python:3.9-slim
+FROM --platform=linux/amd64 python:3.9-bullseye
+
+RUN apt-get update && apt-get install -y build-essential gcc python3-dev
+
+RUN adduser worker
+USER worker
+WORKDIR /home/worker
+
+COPY --chown=worker:worker dist/*.whl /home/worker
+
+# RUN python3 -m pip install ".[plotting-stable]" # Image is 1.6gb with plotting
+# The size is reduced to 847 mb without it.
+RUN python3 -m pip install --disable-pip-version-check --no-cache-dir --user /home/worker/*.whl
+RUN ls /home/worker/.local/lib/python3.9/site-packages/alphatims/ext/timsdata.so
+RUN chmod 777 /home/worker/.local/lib/python3.9/site-packages/alphatims/ext/timsdata.so
+
+RUN python3 -m pip cache purge
+ENV PATH="/home/worker/.local/bin:${PATH}"
+
+ENTRYPOINT [ "alphatims" ]
diff --git a/README.md b/README.md
@@ -20,7 +20,7 @@ AlphaTims is an open-source Python package that provides fast accession and visu
      * [**Installation issues**](#installation-issues)
   * [**Test data**](#test-data)
     * [**Test sample**](#test-sample)
-    * [**LC**](#lc)  
+    * [**LC**](#lc)
     * [**DDA**](#dda)
     * [**DIA**](#dia)
   * [**Usage**](#usage)
@@ -66,6 +66,7 @@ There are three different types of installation possible:
 * [**One-click GUI installer:**](#one-click-gui) Choose this installation if you only want the GUI and/or keep things as simple as possible.
 * [**Pip installer:**](#pip) Choose this installation if you want to use AlphaTims as a Python package in an existing Python 3.8 environment (e.g. a Jupyter notebook). If needed, the GUI and CLI can be installed with pip as well.
 * [**Developer installer:**](#developer) Choose this installation if you are familiar with CLI tools, [conda](https://docs.conda.io/en/latest/) and Python. This installation allows access to all available features of AlphaTims and even allows to modify its source code directly. Generally, the developer version of AlphaTims outperforms the precompiled versions which makes this the installation of choice for high-throughput experiments.
+* [**Docker:**](#docker) Use this installation if you want to use a container based workflow. This is usefull to preserve a clean environment or when running multiple tools that might have conflicting dependencies.
 
 ***IMPORTANT: While AlphaTims is mostly platform independent, some calibration functions require [Bruker libraries](alphatims/ext) which are only available on Windows and Linux.***
 
@@ -179,6 +180,14 @@ The following steps are optional, but make working with AlphaTims slightly more
   When `zsh` is the default terminal instead of `bash`, replace `~/.bashrc` with `~/.zshrc`. On Windows, the command `where alphatims` can be used to find the location of the binary executable. This path can then be (permanently) added to Windows' path variable.
 * When using Jupyter notebooks and multiple conda environments direcly from the terminal, it is recommended to `conda install nb_conda_kernels` in the conda base environment. Hereafter, running a `jupyter notebook` from the conda base environment should have a `python [conda env: alphatims]` kernel available, in addition to all other conda kernels in which the command `conda install ipykernel` was run.
 
+### Docker
+
+(WIP)
+
+```shell
+docker pull ghcr://MannLabs/alphatims:latest
+```
+
 ### Installation issues
 
 See the general [troubleshooting](#troubleshooting) section.
@@ -318,6 +327,7 @@ Common installation/usage issues include:
 * **GUI does not open.** In some cases this can be simply because of using an incompatible (default) browser. AlphaTims has been tested with Google Chrome and Mozilla Firefox. Windows IE and Windows Edge compatibility is not guaranteed.
 * **When older Bruker files need to be processed as well,** the [legacy dependencies](requirements/requirements_legacy.txt) are also needed. However, note that this requires [Microsoft Visual C++](https://visualstudio.microsoft.com/visual-cpp-build-tools) to be manually installed (on Windows machines) prior to AlphaTims installation! To include the legacy dependencies, install AlphaTims with `pip install "alphatims[legacy]"` or `pip install "alphatims[legacy]" --upgrade` if already pre-installed.
 * **When installed through `pip`, the GUI cannot be started.** Make sure you install AlphaTims with `pip install "alphatims[plotting-stable]"` to include the GUI with stable dependancies. If this was done and it still fails to run the GUI, a possible fix might be to run `pip install panel==0.10.3` after AlphaTims was installed.
+* **Some external libraries are missing.** On some OS, there might be libraries missing. As an exmaple, the following error message might pop up: `OSError: libgomp.so.1: cannot open shared object file: No such file or directory`. This can be solved by installing those manually, e.g. on Linux: `apt-get install libgomp1`.
 
 ---
 ## How it works

diff --git a/alphatims/__init__.py b/alphatims/__init__.py
@@ -2,7 +2,7 @@
 
 
 __project__ = "alphatims"
-__version__ = "1.0.7"
+__version__ = "1.0.8"
 __license__ = "Apache"
 __description__ = "A Python package to index Bruker TimsTOF raw data for fast and easy accession and visualization"
 __author__ = "Sander Willems, Eugenia Voytik"

diff --git a/alphatims/bruker.py b/alphatims/bruker.py
@@ -29,18 +29,8 @@
         "timsdata.so"
     )
 else:
-    logging.warning(
-        "WARNING: "
-        "No Bruker libraries are available for this operating system. "
-        "Mobility and m/z values need to be estimated. "
-        "While this estimation often returns acceptable results with errors "
-        "< 0.02 Th, huge errors (e.g. offsets of 6 Th) have already been "
-        "observed for some samples!"
-    )
-    logging.info("")
     BRUKER_DLL_FILE_NAME = ""
 
-
 def init_bruker_dll(bruker_dll_file_name: str = BRUKER_DLL_FILE_NAME):
     """Open a bruker.dll in Python.
 
@@ -996,6 +986,19 @@ def __init__(
             This is ignored if the polarity is dropped.
             Default is True.
         """
+
+        #Log a warning if there was not a valid DLL filename
+        if BRUKER_DLL_FILE_NAME=="":
+            logging.warning(
+                "WARNING: "
+                "No Bruker libraries are available for this operating system. "
+                "Mobility and m/z values need to be estimated. "
+                "While this estimation often returns acceptable results with errors "
+                "< 0.02 Th, huge errors (e.g. offsets of 6 Th) have already been "
+                "observed for some samples!"
+            )
+            logging.info("")
+
         if bruker_d_folder_name.endswith("/"):
             bruker_d_folder_name = bruker_d_folder_name[:-1]
         logging.info(f"Importing data from {bruker_d_folder_name}")
@@ -1982,7 +1985,7 @@ def index_precursors(
         precursor_offsets[-1] = len(precursor_order)
         offset = precursor_offsets[1]
         offsets = precursor_order[offset:]
-        counts = np.empty(len(offsets) + 1, dtype=np.int)
+        counts = np.empty(len(offsets) + 1, dtype=np.int64)
         counts[0] = 0
         counts[1:] = np.cumsum(
             self.quad_indptr[offsets + 1] - self.quad_indptr[offsets]
@@ -2058,22 +2061,23 @@ def index_precursors(
             trimmed_spectrum_intensity_values
         )
 
-    def save_as_mgf(
+    def save_as_spectra(
         self,
         directory: str,
         file_name: str,
         overwrite: bool = False,
         centroiding_window: int = 5,
         keep_n_most_abundant_peaks: int = -1,
+        mgf: bool = True
     ):
-        """Save profile spectra from this TimsTOF object as an mgf file.
+        """Save profile spectra from this TimsTOF object as an spectrum file.
 
         Parameters
         ----------
         directory : str
-            The directory where to save the mgf file.
+            The directory where to save the spectrum file.
         file_name : str
-            The file name of the  mgf file.
+            The file name of the spectrum file.
         overwrite : bool
             If True, an existing file is truncated.
             If False, nothing happens if a file already exists.
@@ -2090,7 +2094,7 @@ def save_as_mgf(
         Returns
         -------
         str
-            The full file name of the mgf file.
+            The full file name of the spectrum file.
         """
         full_file_name = os.path.join(
             directory,
@@ -2127,30 +2131,36 @@ def save_as_mgf(
         mobilities = self.mobility_values[
             self.precursors.ScanNumber.values.astype(np.int64)
         ]
-        with open(full_file_name, "w") as infile:
-            logging.info(f"Exporting profile spectra to {full_file_name}...")
-            for index in alphatims.utils.progress_callback(
-                range(1, self.precursor_max_index)
-            ):
-                start = spectrum_indptr[index]
-                end = spectrum_indptr[index + 1]
-                title = (
-                    f"index: {index}, "
-                    f"intensity: {intensities[index - 1]:.1f}, "
-                    f"mobility: {mobilities[index - 1]:.3f}, "
-                    f"average_mz: {average_mzs[index - 1]:.3f}"
-                )
-                infile.write("BEGIN IONS\n")
-                infile.write(f'TITLE="{title}"\n')
-                infile.write(f"PEPMASS={mono_mzs[index - 1]:.6f}\n")
-                infile.write(f"CHARGE={charges[index - 1]}\n")
-                infile.write(f"RTINSECONDS={rtinseconds[index - 1]:.2f}\n")
-                for mz, intensity in zip(
-                    self.mz_values[spectrum_tof_indices[start: end]],
-                    spectrum_intensity_values[start: end],
-                ):
-                    infile.write(f"{mz:.6f} {intensity}\n")
-                infile.write("END IONS\n")
+        logging.info(f"Exporting profile spectra to {full_file_name}...")
+        if mgf:
+            save_as_mgf(
+                full_file_name,
+                spectrum_indptr,
+                intensities,
+                mobilities,
+                average_mzs,
+                mono_mzs,
+                charges,
+                rtinseconds,
+                spectrum_tof_indices,
+                spectrum_intensity_values,
+                self.precursor_max_index,
+                self.mz_values,
+            )
+        else:
+            save_as_spectra(
+                full_file_name,
+                spectrum_indptr,
+                intensities,
+                mobilities,
+                average_mzs,
+                mono_mzs,
+                charges,
+                rtinseconds,
+                spectrum_tof_indices,
+                spectrum_intensity_values,
+                self.mz_values,
+            )
         logging.info(
             f"Succesfully wrote {self.precursor_max_index - 1:,} "
             f"spectra to {full_file_name}."
@@ -2326,7 +2336,7 @@ class PrecursorFloatError(TypeError):
 
 
 @alphatims.utils.pjit(
-    signature_or_function="void(i8,i8[:],i8[:],i8[:],u4[:],u2[:],u4[:],f8[:],i8[:],i8[:])"
+    # signature_or_function="void(i8,i8[:],i8[:],i8[:],u4[:],u2[:],u4[:],f8[:],i8[:],i8[:])"
 )
 def set_precursor(
     precursor_index: int,
@@ -3291,3 +3301,69 @@ def filter_tof_to_csr(
                 tof_value = tof_indices[idx]
         indptr.append(len(values))
     return np.array(indptr), np.array(values), np.array(columns)
+
+
+def save_as_mgf(
+    full_file_name,
+    spectrum_indptr,
+    intensities,
+    mobilities,
+    average_mzs,
+    mono_mzs,
+    charges,
+    rtinseconds,
+    spectrum_tof_indices,
+    spectrum_intensity_values,
+    precursor_max_index,
+    mz_values,
+):
+    with open(full_file_name, "w") as infile:
+        for index in alphatims.utils.progress_callback(
+            range(1, precursor_max_index)
+        ):
+            start = spectrum_indptr[index]
+            end = spectrum_indptr[index + 1]
+            title = (
+                f"index: {index}, "
+                f"intensity: {intensities[index - 1]:.1f}, "
+                f"mobility: {mobilities[index - 1]:.3f}, "
+                f"average_mz: {average_mzs[index - 1]:.3f}"
+            )
+            infile.write("BEGIN IONS\n")
+            infile.write(f'TITLE="{title}"\n')
+            infile.write(f"PEPMASS={mono_mzs[index - 1]:.6f}\n")
+            infile.write(f"CHARGE={charges[index - 1]}\n")
+            infile.write(f"RTINSECONDS={rtinseconds[index - 1]:.2f}\n")
+            for mz, intensity in zip(
+                mz_values[spectrum_tof_indices[start: end]],
+                spectrum_intensity_values[start: end],
+            ):
+                infile.write(f"{mz:.6f} {intensity}\n")
+            infile.write("END IONS\n")
+
+
+def save_as_spectra(
+    full_file_name,
+    spectrum_indptr,
+    intensities,
+    mobilities,
+    average_mzs,
+    mono_mzs,
+    charges,
+    rtinseconds,
+    spectrum_tof_indices,
+    spectrum_intensity_values,
+    mz_values,
+):
+    with h5py.File(full_file_name, "w") as infile:
+        infile["indptr"] = spectrum_indptr[1:]
+        infile["fragment_mzs"] = mz_values[
+            spectrum_tof_indices
+        ]
+        infile["fragment_intensities"] = spectrum_intensity_values
+        infile["precursor_rt"] = rtinseconds
+        infile["precursor_charge"] = charges
+        infile["precursor_intensity"] = intensities
+        infile["precursor_mobility"] = mobilities
+        infile["precursor_average_mz"] = average_mzs
+        infile["precursor_monoisotopic_mz"] = mono_mzs