Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PTV-1810 fix obj input 1 #3320

Merged
merged 5 commits into from
Aug 14, 2023
Merged

PTV-1810 fix obj input 1 #3320

merged 5 commits into from
Aug 14, 2023

Conversation

briehl
Copy link
Member

@briehl briehl commented Jul 27, 2023

Description of PR purpose/changes

This is the first of probably 3 or 4 PRs related to some fixes for PTV-1810.
The short version is with the recent updates to that bug, input objects are stored in app cells as UPAs and passed to the app framework that way. The problem here is that apps don't always expect UPAs (ALTHOUGH THEY SHOULD). Older ones can expect object names, or references of the form ws_name/obj_name.

The goal of this chain of PRs is to update the Narrative backend code to detect those requirements and parse UPAs back into lesser, worse formats for apps that want them. Backward compatibility and all that.

This will mainly require updating biokbase.app_util.transform_object_value - an old bit of code that, with the lightest poking, reveals ancient misguided decisions enabled by Python's "anything goes!" flexibility around input types. Again, the goal isn't to fully fix all of that, which would be a very large effort, but to put in an effective patch without creating more chaos. Currently, that function implicitly expects that an object name is provided, and transforms based on that. This update will explicitly expect that any object-ish string (name, reference, reference-path, UPA) is provided, and transform based on the given parameter spec.

Sounds simple, but it's less so.

PR strategy:

  1. (this PR) move the system_variable and strict_system_variable functions to their own module to avoid what will become circular dependencies in the next PR.
  2. Make use of the UPA API to detect whether inputs are UPAs or not (currently some hand-crafted code in there), add missing tests.
  3. Adjust the rest of that code to fit our needs, add thorough unit and integration tests.

Jira Ticket / Issue

Related Jira ticket: https://kbase-jira.atlassian.net/browse/PTV-1810

  • Added the Jira Ticket to the title of the PR (e.g. DATAUP-69 Adds a PR template)

Testing Instructions

  • Details for how to test the PR:
  • Tests pass locally and in GitHub Actions
  • Changes available by spinning up a local narrative and navigating to X to see Y

Dev Checklist:

  • My code follows the guidelines at https://sites.google.com/lbl.gov/trussresources/home?authuser=0
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • [n/a] (JavaScript) I have run Prettier and ESLint on changed code manually or with a git precommit hook
  • (Python) I have run Black and Flake8 on changed Python code manually or with a git precommit hook
  • Any dependent changes have been merged and published in downstream modules

Updating Version and Release Notes (if applicable)

@codecov
Copy link

codecov bot commented Jul 27, 2023

Codecov Report

Merging #3320 (4580595) into develop (3bccc58) will decrease coverage by 23.61%.
Report is 5 commits behind head on develop.
The diff coverage is 94.11%.

Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff             @@
##           develop   #3320       +/-   ##
===========================================
- Coverage    29.22%   5.62%   -23.61%     
===========================================
  Files          495     496        +1     
  Lines        50482   50486        +4     
===========================================
- Hits         14755    2841    -11914     
- Misses       35727   47645    +11918     
Files Changed Coverage Δ
src/biokbase/narrative/viewers.py 0.00% <0.00%> (ø)
src/biokbase/narrative/system.py 94.59% <94.59%> (ø)
src/biokbase/narrative/app_util.py 84.22% <100.00%> (-0.91%) ⬇️
src/biokbase/narrative/jobs/appmanager.py 91.94% <100.00%> (+0.02%) ⬆️
src/biokbase/narrative/jobs/batch.py 98.82% <100.00%> (ø)
src/biokbase/narrative/jobs/jobmanager.py 95.37% <100.00%> (ø)
src/biokbase/narrative/upa.py 100.00% <100.00%> (ø)
src/biokbase/narrative/widgetmanager.py 94.77% <100.00%> (+0.01%) ⬆️

... and 326 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 38873c4...4580595. Read the comment docs.

@briehl
Copy link
Member Author

briehl commented Jul 27, 2023

Codacy is whining about assert being used throughout test code. Which is what it's for. Ugh.

@@ -162,6 +99,31 @@ def _untransform(transform_type, value):


def app_param(p):
"""
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a bit of notes here that weren't exactly intended for this PR, though still helpful.

@@ -0,0 +1,64 @@
import os
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the bulk of the change - migrate system_variable and strict_system_variable over here.

Really, these should be separate functions - get_workspace, get_workspace_id, etc. But that's out of scope for this little project. It was originally done with strings to map directly to various app spec requests.

@@ -0,0 +1,90 @@
import os
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Redid the tests here as pytest style, since we're using pytest anyway and it would be nice to migrate there.

@@ -5,20 +5,19 @@
"""

import re

from .app_util import system_variable
from .system import system_variable

external_tag = "&"


Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The various typing things also weren't exactly intended to pop into this PR, but they're benign and generally helpful.

This was referenced Jul 28, 2023
Comment on lines 62 to 63
else:
return None
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can just do return None here - no need for final else.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right. Fixed.

if var == "workspace":
return os.environ.get("KB_WORKSPACE_ID", None)
elif var == "workspace_id":
ws_name = os.environ.get("KB_WORKSPACE_ID", None)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't os.environ.get("blahblahblah") return None by default if it can't find the value?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup. I really didn't modify that code, just copy-pasted. I'll make it better.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed up. Also swapped BaseException over to Exception for those try/catch blocks. That #TODO in there still holds - we should do some better error trapping there.

@sonarcloud
Copy link

sonarcloud bot commented Aug 11, 2023

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 1 Code Smell

No Coverage information No Coverage information
0.0% 0.0% Duplication

@ialarmedalien ialarmedalien merged commit 317dc0d into develop Aug 14, 2023
6 of 10 checks passed
@ialarmedalien ialarmedalien deleted the PTV-1810-fix-obj-input branch August 14, 2023 17:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants