2022.09

• Moved Qiita's code base and plugins to SLURM (from Torque). The plugins updated are: qiita-spots/qp-woltka, biocore/mg-scripts, qiita-spots/qp-spades, qiita-spots/qp-meta, qiita-spots/qp-fastp-minimap2.
• Pinned the paramiko version to < 2.9 as newer versions were causing issues with older systems.
• Pinned the scipy version to < 1.8 to avoid issues with the biom-format library.
• Updates to the INSTALL instructions (thank you @aliu104 !)

2022.07

• Users can opt-in to get emails when their jobs change status in their User Information preferences.
• Added BIOM artifact archiving to the system; this unlinks artifacts from the main processing but leaves them in the system in case they are needed in the future.
• Added qiime2.2022.02 to the system; which updated these plugins: qp-qiime2, qtp-biom, qtp-diversity, qtp-visualization
• Users can now select multiple artifacts for analysis qp-qiime2, which gives access to new commands like PCoA biplots.
• qtp-sequencing now uses fqtools to count the number of sequences in fastq/fastq.gz files as part as the artifact summary.
• Artifact summaries can now be updated qiita-spots #3205.
• Added to the internal Sequence Processing Pipeline the CHM13 genome so human studies are now filtered by GRCh38 genome + PhiX and CHM13 genome.

2022.05

• Added Artifact.iter() for easier Artifact interactions.
• Fixed the SCP Qiita transfer; this was the problem and solution.
• Added User.email to the Admin Job Processing listing.

2022.04

• Moved from Python 3.6 to 3.9.
• Added support for Pandas 1.4.0, details here.
• Updated all available JavaScript libraries, details here.
• Users can select which metadata to use when creating a new analysis. By default only overlapping metadata in all studies is selected.
• Now we can fully delete users in the backend.
• Updated documentation to reflect the new EMPO version 2.
• Fixed outstanding issues to add default workflow to a preparation, details here.
• Fixed the following issues: 3183, 3182, 3170, 3193.
• We deprecated the use of specimen_id from Qiita; this is no longer required in the backend or the GUI.
• Moved qp-fastp-minimap2 to per sample parallelization. Now an iSeq processing takes ~20min, while before it took close to 2hrs.
• Fixed the following issues qp-knight-lab-processing #15, qp-knight-lab-processing #16, qp-knight-lab-processing #17, qp-knight-lab-processing #19, mg-scripts #60, mg-scripts #62 from the Knight Lab Sequence Processing Pipeline.

2021.11

• Upgrading PostgreSQL from 9.5 to 13.4 as 9.5 is no longer supported
• Updated SortMeRNA within qt-meta to filter RNA reads to run as job arrays to speed up processing. A full NovaSeq run will now take ~15hrs vs. ~161hrs.
• Added qiime2.2021.11 to the system; which updated these plugins: qp-qiime2, qtp-biom, qtp-diversity, qtp-visualization.
• Jobs no longer need to be linked to studies or analyses; this allows us to create general purpose jobs like "admin like" jobs that can be fully traced in Qiita.
• Added a new user level: "wet-lab admin" so they can start "admin like" jobs in the system without having to be an actual admin.
• Added a new plugin: "qp-knight-lab-processing" that provides the general Knight Lab sequence processing: from BCL to per_sample_FASTQ to the upload folders in Qiita.
• Added "Oxford_Nanopore" as a valid "platform" with "GridION" as valid "instrument_model" to the system; this allows submission of this data to EBI-ENA.
• Allow chucked download of metadata files in analyses; this allows to process large meta-analysis (like those for The Microsetta Initiative) without worker blockage.
• Added to the qp-qiime2 plugin the possibility of filtering tables based on system available "FeatureData[Sequence]"; to start we added 90/100/150 bps bloom tables.
• Now we can instantiate a study via their title (Study.from_title); this will facilitate orchestration with qebil.
• Speed up Study listing for admins and general users; the admin study display came down from 20 to 2 seconds.
• Fixed the following issues: 3142, 3149, 3150, 3119, and 3160.

2021.09

• Updated the qp-deblur plugin to version 2021.09 addressing a bug in fragment insertion parsing and caching; more information.
• Double the number of possible connections for the Qiita database: 100 -> 200 simultaneous connections.
• Added a new data type: "Job Output Folder" and artifact type definition: "job-output-folder" to initially support admin-only standalone commands in Qiita.
• The study listing is now sorted by descending study id and then ascending number of available artifacts.
• Removed old code from the Sample Information update method that regenerated all preparations in that study - this is no longer necessary as the per preparation sample information files are built on the fly.
• Fixed a bug that did not copy the raw files when adding a new artifact via the CLI.
• Fixed the following issues: #3124, #3122, #3134.

2021.07

• Added a new "Add Default Workflow" button to the preparation tab so we automatically add all the "recommended" steps in a preparation based on our workflows.
• New Preparation Listing GUI + searching within preparation types, thank you @AmandaBirmingham!
• We limit the parameters displayed to the user per command based on which have already been run successfully; for example if a preparation has already a "Trim 100" job, this option will not be shown.
• Re-added Coveralls to Qiita and removed codecov.
• We are limiting the number of samples in a preparation file to 800.
• Added User.update_email which allows to update a Users email.
• Fixed the following issues: #3113, and #3079.

2021.05

• Replaced vis.js for cytoscape.js to display the processing networks.
• The commands available to users, originally only filtered by input type, are now also limited by the preparation type. The options are taken from the recommended workflows.
• Added a new spades assembly pipeline for "Genome Isolate".
• Fixed the following issues: #3070, #3089, #2968, #3102, and #3079.

2021.03

• Fixed issue that left behind non gz per sample FASTQ files.
• Recommended Workflows are now stored in the database.
• Added a new button only for owners and admins within the Study page to display a list of all the Analyses generated with that study; helpful to clean up Studies and for general information.
• The Qiita CI now runs as GitHub Actions; moving away from Travis CI.
• Prep information file object now stores its creation and modification timestamps.
• Improved creation time for all information files via the to_dataframe() method.
• Split the "other" category of the storage stats plot (https://qiita.ucsd.edu/stats/) into "other" and "biom" so biom can be its own category.
• Added a processing_jobs property to qiita_db.software.Command to easily retrieve all jobs in the system that have ran the given command.
• Fixed the following issues: #3068, #3072, #3076, and #3070.

2021.01

• Moved the qiita repo from biocore to qiita-spots.
• Created the Qiita portal for the Cancer Microbiome.
• The EBI-ENA code now verifies that the sample information file has a description column; this wasn't previously required because it was automatically prefilled by the QIIME 1 mapping file.
• Now it is possible to download the per preparation sample information file and the sample-preparation summary.
• Added a faster metagenomic/metatranscriptomic adaptor and host removal step based on fastp and minimap2. The previous version, using atropos and bowtie2 for QC host filtering, is now deprecated.
• Added qiime2.2020.11 to the system; which updated these plugins: qp-qiime2, qtp-biom, qtp-diversity, qtp-visualization.
• Added WoL tree for phylogenetic analyses (/projects/wol/release/databases/qiime2/phylogeny.qza) with per-genome WoL artifacts.
• Fixed the following issues: #3060, #3049, and #2751.