Negative genes input, neutral genes output, CI deploy refactor, and o…

…ther frontend additions (#49)

@ChristopherMancuso please look at the negative genes input and neutral
gene results portions. You'll have to run the stack locally.

@falquaddoomi please take a look at the deployment workflows. I haven't
tested these on a fork yet, and haven't generated the secrets yet
either.

- format workflow files with prettier
- CI deploy the backend and frontend in sequence, in single workflow, so
that they are deployed close together in time. make frontend deployment
dependent on function deploy success or skip.
- add link to old geneplexus
- add "negatives" to analysis input type, and "neutral gene info" to
analysis output type
- tweak select component prop names and styles, and update in-situ usage
of them to match
- tweak table component. pass row to custom render func, fix expansion
css, more descriptive filter labels, fix filter bug
- add note about old geneplexus to about page and home page
- add "neutrals" tab in analysis results page with basic table view of
neutral gene info
- add negative genes input text box on new analysis page, and
incorporate into "check genes" functionality
- add redirect link on 404 page to old site
- update cloud functions to take negatives as input and return neutral
gene info as output

---------

Co-authored-by: Faisal Alquaddoomi <falquaddoomi@gmail.com>

.github/workflows/helper-deploy-func.yaml → .github/workflows/deploy-func.yaml

@@ -1,4 +1,4 @@
-name: Cloud Function Deploy Helper
+name: Deploy cloud function
 
 on:
  workflow_call:
@@ -21,7 +21,6 @@ on:
  func-memory-mb:
  required: true
  type: number
-
  func-runtime:
  default: python311
  type: string
@@ -35,77 +34,78 @@ on:
  default: us-central1
  type: string
  func-data-local-path:
- default: 'data'
+ default: data
  type: string
- description: 'Path under func-src-dir where the GCS archive is extracted, default ./data'
+ description: Path under func-src-dir where the GCS archive is extracted, default ./data
  secrets:
  JSON_GCLOUD_SERVICE_ACCOUNT_JSON:
  required: true
+
 jobs:
  deploy-cloud-func-helper:
- runs-on: 'ubuntu-latest'
+ runs-on: ubuntu-latest
  env:
- DATA_ARCHIVE_HASH: 'none'
+ DATA_ARCHIVE_HASH: none
 
  steps:
- - uses: 'actions/checkout@v4'
+ - name: Checkout repo
+ uses: actions/checkout@v4
 
  - name: Authenticate to GCP
- uses: 'google-github-actions/auth@v2'
+ uses: google-github-actions/auth@v2
  with:
  # workload_identity_provider: 'projects/123456789/locations/global/workloadIdentityPools/my-pool/providers/my-provider'
  # service_account: 'cloud-function-deployer@gap-som-dbmi-geneplx-app-p0n.iam.gserviceaccount.com'
  credentials_json: ${{ secrets.JSON_GCLOUD_SERVICE_ACCOUNT_JSON }}
 
  - name: Set up Cloud SDK
- uses: 'google-github-actions/setup-gcloud@v2'
+ uses: google-github-actions/setup-gcloud@v2
  with:
- version: '>= 363.0.0'
+ version: ">= 363.0.0"
 
- - name: Get hash of the data archive for this function
+ - name: Get hash of data archive for func
  run: |
  gsutil ls -L ${{ inputs.func-data-gcs-url }} | \
  grep "Hash (crc32c)" | \
  awk '{printf "DATA_ARCHIVE_HASH=%s",$3}' >> "$GITHUB_ENV"
 
- # if the cached files aren't present, this will defer caching the files
- # until the end of a successful run of this workflow. the next time the
- # workflow runs, it'll retrieve the cached files from the previous run
- # and skip the download step.
- # the hash of the current datafile is included as part of the key, so
- # that fresh data will be fetched if the datafile has changed in GCS.
+ # if cached files aren't present, defer caching files until end of
+ # successful run. next time workflow runs, it'll retrieve cached files
+ # from previous run and skip download step. hash of current datafile
+ # included as part of key, so fresh data will be fetched if datafile has
+ # changed in GCS.
  - name: Cache existing data folder
- id: cache-existing-data
+ id: cache
  uses: actions/cache@v4
  with:
  path: ${{ inputs.func-src-dir }}/${{ inputs.func-data-local-path }}
  key: ${{ inputs.func-name }}-data-${{ env.DATA_ARCHIVE_HASH }}
 
  - name: Download function data from GCS
- if: steps.cache-existing-data.outputs.cache-hit != 'true'
+ if: steps.cache.outputs.cache-hit != 'true'
  run: |
  gsutil cp ${{ inputs.func-data-gcs-url }} /tmp/data.tar.gz
  mkdir -p ${{ inputs.func-src-dir }}/${{ inputs.func-data-local-path }}
  tar -xvf /tmp/data.tar.gz -C ${{ inputs.func-src-dir }}
  rm /tmp/data.tar.gz
 
  - name: Check filesystem status
- run: |
- find ${{ inputs.func-src-dir }}
+ run: find ${{ inputs.func-src-dir }}
 
- - if: runner.debug == '1'
+ - name: SSH debug
+ if: runner.debug == '1'
  uses: mxschmitt/action-tmate@v3
 
  - name: Deploy function '${{ inputs.func-name }}' to GCP
  run: |
  gcloud functions deploy ${{ inputs.func-name }} \
-  --gen2 \
-  --runtime=${{ inputs.func-runtime }} \
-  --project=${{ inputs.project-id }} \
-  --region=${{ inputs.region }} \
-  --source=${{ inputs.func-src-dir }} \
-  --entry-point=${{ inputs.func-entrypoint }} \
-  --trigger-http \
-  --allow-unauthenticated \
-  --memory=${{ inputs.func-memory-mb }}MB \
-  --service-account=${{ inputs.func-svc-acct }}
+ --gen2 \
+ --runtime=${{ inputs.func-runtime }} \
+ --project=${{ inputs.project-id }} \
+ --region=${{ inputs.region }} \
+ --source=${{ inputs.func-src-dir }} \
+ --entry-point=${{ inputs.func-entrypoint }} \
+ --trigger-http \
+ --allow-unauthenticated \
+ --memory=${{ inputs.func-memory-mb }}MB \
+ --service-account=${{ inputs.func-svc-acct }}

.github/workflows/deploy.yaml

@@ -0,0 +1,96 @@
+name: Deploy backend functions and frontend app
+
+on:
+ workflow_dispatch:
+ push:
+ branches:
+ - main
+
+jobs:
+ path-changes:
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout code
+ uses: actions/checkout@v4
+
+ - uses: dorny/paths-filter@v3
+ id: changes
+ with:
+ filters: |
+ convert_ids:
+ - "functions/convert_ids/convert_ids_deploy/**"
+ ml:
+ - "functions/ml/ml_deploy/**"
+ outputs:
+ convert_ids: ${{ steps.changes.outputs.convert_uds }}
+ ml: ${{ steps.changes.outputs.ml }}
+
+ deploy-convert-ids:
+ needs: path-changes
+ if: ${{ needs.path-changes.outputs.convert_ids == 'true' }}
+ uses: ./.github/workflows/deploy-func.yaml
+ with:
+ func-name: "gpz-convert-ids"
+ func-src-dir: "functions/convert_ids/convert_ids_deploy"
+ func-entrypoint: "convert_ids"
+ func-memory-mb: 1024
+ func-data-gcs-url: "gs://geneplexus-func-data/convert-ids/convert-ids_data.tar.gz"
+ secrets: inherit
+
+ deploy-ml:
+ needs: path-changes
+ if: ${{ needs.path-changes.outputs.ml == 'true' }}
+ uses: ./.github/workflows/deploy-func.yaml
+ with:
+ func-name: "gpz-ml"
+ func-src-dir: "functions/ml/ml_deploy"
+ func-entrypoint: "ml"
+ func-memory-mb: 8192
+ func-data-gcs-url: "gs://geneplexus-func-data/ml/ml_data.tar.gz"
+ secrets: inherit
+
+ deploy-frontend:
+ needs:
+ - deploy-convert-ids
+ - deploy-ml
+ # https://github.com/actions/runner/issues/491#issuecomment-850884422
+ if: |
+ always() &&
+ (needs.deploy-convert-ids.result == 'success' || needs.deploy-convert-ids.result == 'skipped') &&
+ (needs.deploy-ml.result == 'success' || needs.deploy-ml.result == 'skipped')
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout code
+ uses: actions/checkout@v4
+
+ - name: Set up Bun
+ uses: oven-sh/setup-bun@v1
+
+ - name: Install packages
+ run: bun install
+ working-directory: frontend
+
+ - if: runner.debug == '1'
+ uses: mxschmitt/action-tmate@v3
+
+ - name: Build
+ run: bun run build
+ working-directory: frontend
+
+ - name: Deploy to Netlify
+ uses: nwtgck/actions-netlify@v3.0
+ with:
+ publish-dir: .frontend/dist
+ production-branch: main
+ production-deploy: true
+ deploy-message: Deploy from GitHub Actions
+ enable-pull-request-comment: false
+ enable-commit-comment: false
+ enable-commit-status: true
+ overwrites-pull-request-comment: false
+ enable-github-deployment: false
+ fails-without-credentials: true
+ env:
+ NETLIFY_AUTH_TOKEN: ${{ secrets.NETLIFY_AUTH_TOKEN }}
+ NETLIFY_SITE_ID: ${{ secrets.NETLIFY_SITE_ID }}
+ timeout-minutes: 1

.github/workflows/test-frontend.yaml

@@ -3,8 +3,8 @@ name: Test Frontend
 on:
  pull_request:
  paths:
- - 'frontend/**'
- - 'package.json'
+ - "frontend/**"
+ - "package.json"
 
 defaults:
  run:
@@ -16,7 +16,7 @@ jobs:
  runs-on: ubuntu-latest
  steps:
  - name: Checkout code
- uses: actions/checkout@v3
+ uses: actions/checkout@v4
 
  - name: Set up Bun
  uses: oven-sh/setup-bun@v1
@@ -35,7 +35,7 @@ jobs:
  runs-on: ubuntu-latest
  steps:
  - name: Checkout code
- uses: actions/checkout@v3
+ uses: actions/checkout@v4
 
  - name: Set up Bun
  uses: oven-sh/setup-bun@v1
@@ -54,7 +54,7 @@ jobs:
  runs-on: ubuntu-latest
  steps:
  - name: Checkout code
- uses: actions/checkout@v3
+ uses: actions/checkout@v4
 
  - name: Set up Bun
  uses: oven-sh/setup-bun@v1

.gitignore

@@ -1,4 +1,4 @@
-*DS_Store
+.DS_Store
 functions/ml/ml_deploy/data/
 functions/convert_ids/convert_ids_deploy/data/
 

README.md

@@ -1 +1,3 @@
 # GenePlexus App v2.0
+
+[![Cloud Function Deploy Helper](https://github.com/krishnanlab/geneplexus-app-v2/actions/workflows/deploy.yaml/badge.svg)](https://github.com/krishnanlab/geneplexus-app-v2/actions/workflows/deploy.yaml) [![Netlify Status](https://api.netlify.com/api/v1/badges/aae668a9-01fa-4998-9158-60c92d994598/deploy-status)](https://app.netlify.com/sites/molevolvr/deploys) 

frontend/.env

@@ -1,4 +1,5 @@
 VITE_TITLE=GenePlexus
 VITE_DESCRIPTION=GenePlexus enables researchers to predict novel genes similar to genes of interest based on their patterns of connectivity in human genome-scale networks.
-VITE_URL=
+VITE_URL=https://geneplexus.net
+VITE_OLD_URL=https://geneplexus-old.net
 VITE_API=https://us-central1-gap-som-dbmi-geneplx-app-p0n.cloudfunctions.net

frontend/.prettierrc

@@ -13,6 +13,7 @@
  "^./",
  "^../"
  ],
+ "importOrderParserPlugins": ["typescript", "jsx", "importAssertions"],
  "cssDeclarationSorterOrder": "smacss",
  "jsdocCapitalizeDescription": false,
  "htmlWhitespaceSensitivity": "strict"

frontend/README.md

@@ -25,55 +25,3 @@ This project was scaffolded using Vite, and has the following key features:
 ## Usage
 
 See the "testbed" page for an overview of what formatting/elements/components/etc. you can use and how.
-
-## Background
-
-- **[HTML](https://developer.mozilla.org/en-US/docs/Web/HTML/Reference)**.
- Static markup that defines the "content" of a page/document.
- Consists of nested [tags](https://developer.mozilla.org/en-US/docs/Web/HTML/Element) with [attributes](https://developer.mozilla.org/en-US/docs/Web/HTML/Attributes) that become the [DOM](https://developer.mozilla.org/en-US/docs/Glossary/DOM).
-
-- **[CSS](https://developer.mozilla.org/en-US/docs/Web/css/Reference)**.
- [Selectors](https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_selectors) and [styles](https://developer.mozilla.org/en-US/docs/Web/CSS/Reference) that determine how the content of your page looks and is laid out.
- [Modules](https://vitejs.dev/guide/features.html#css-modules) should be used to keep styles scoped to a particular file and avoid selector name collisions.
-
-- **[JavaScript](https://developer.mozilla.org/en-US/docs/Web/JavaScript)**.
- A dynamic, loosely-typed, general-purpose language that runs in a browser when a user visits a page.
- It can dynamically manipulate the document/styles, make network requests, and interact with the browser in [a multitude of other ways](https://developer.mozilla.org/en-US/docs/Learn/JavaScript/Client-side_web_APIs).
-
-_These 👆 are the only things that a browser can natively understand_ (and assets like images, videos, etc.).
-Anything else 👇 is abstraction or convenience built on top of them, and must first go through a "compilation"/"build" step to transform it into one of the three core things.
-
-- **[React](https://react.dev/learn)**.
- A widely popular and plugin-rich JavaScript library that makes it easier to generate HTML on the page, keep it in sync with data, handle events, and more.
- In general, it makes code more readable, robust, and re-usable.
- The [JSX](https://react.dev/learn/writing-markup-with-jsx) syntax is used to write dynamic content in a more declarative, clear way.
- Pieces of content are split into cleanly separated "components".
- A component is a generic term for any function that returns HTML to be generated, and it can range from low level (e.g. button, select dropdown, table) to high-level (e.g. tab group, section, page).
-
-- **[TypeScript](https://en.wikipedia.org/wiki/TypeScript)**.
- A superset of JavaScript that makes it strongly, statically typed so you can catch errors before even running the code.
- Adds some additional overhead/complexity, but is especially essential for apps dealing with complex data structures.
-
-- **[Vite](https://vitejs.dev/)**.
- A multi-purpose tool that handles coordination of the various other technologies.
- Most notably, it handles running the app locally for development (with fast automatic refresh), transpiling TypeScript into plain JavaScript, and "compiling" the app into a production/browser-ready bundle.
-
-- **[Node](https://nodejs.org/en/)**.
- An environment for running JavaScript locally instead of in a browser.
- Has APIs that browsers cannot have, such as filesystem access, and lacks some of the APIs browsers do have, such as functions to manipulate the DOM (there is no DOM or browser).
- That is, not all Node code can run in the browser, and vice-versa.
- Vite and other packages that do things locally are ultimately intended to run on top of this.
-
-- **[Bun](https://bun.sh/)**.
- A very new tool that aims to be an all-in-one replacement for Node, Yarn, Vite, and many other tools.
- Since it is so new, in this project will only use it as a runtime (replacement for Node) and package manager (replacement for Yarn).
- If you encounter issues, install [Node](https://nodejs.org/en) (`v18` or later) and try running the above commands with `npm` instead of `bun`.
- Do not use functionality that is Bun-only (not "backwards-compatible" with Node).
-
-- **[ESLint](https://eslint.org/)**.
- A tool that checks JavaScript code for common pitfalls, mostly focused on things that _affect functionality_ rather than code aesthetics.
-
-- **[Prettier](https://prettier.io/)**.
- A tool that formats JavaScript code to make it look pretty.
- Useful for enforcing consistency, avoiding bike-shedding, and writing code more quickly without worrying about formatting.
- _Only affects code aesthetics_, and should not affect functionality.

frontend/public/404.html

@@ -7,8 +7,9 @@
  console.debug("With state:", history.state);
  const { origin, pathname, search, hash } = location;
  sessionStorage.redirectPath = pathname + search + hash;
+ // don't fail redirection all together if state exceeds session storage limit
  try {
- // remove entries that could exceed storage limit
+ // remove entries that will likely exceed storage limit
  delete history.state.usr.results;
  sessionStorage.redirectState = JSON.stringify(history.state);
  } catch (error) {