Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

15257 - Fix statements serialization issue #1115

Merged
merged 17 commits into from
Feb 27, 2023

Conversation

seeker25
Copy link
Collaborator

Issue #:
bcgov/entity#15257

Description of changes:
Use cattrs instead of pydantic. Minor fix in migration for upgrading sqlalchemy in the future.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of the sbc-pay license (Apache 2.0).

@codecov
Copy link

codecov bot commented Feb 16, 2023

Codecov Report

Merging #1115 (361333d) into main (28ff797) will increase coverage by 0.07%.
The diff coverage is 91.30%.

@@            Coverage Diff             @@
##             main    #1115      +/-   ##
==========================================
+ Coverage   91.54%   91.62%   +0.07%     
==========================================
  Files         153      163      +10     
  Lines        9842    10947    +1105     
==========================================
+ Hits         9010    10030    +1020     
- Misses        832      917      +85     
Flag Coverage Δ
eventlistenerqueue 81.81% <ø> (-0.48%) ⬇️
payadmin ∅ <ø> (?)
payapi 93.93% <91.15%> (+0.03%) ⬆️
paymentjobs 79.93% <ø> (+0.97%) ⬆️
paymentreconciliationsqueue 91.64% <100.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
pay-api/src/pay_api/services/bcol_service.py 84.44% <9.09%> (-3.80%) ⬇️
pay-api/src/pay_api/models/__init__.py 100.00% <100.00%> (ø)
pay-api/src/pay_api/models/invoice.py 100.00% <100.00%> (ø)
pay-api/src/pay_api/models/payment.py 97.16% <100.00%> (+0.90%) ⬆️
pay-api/src/pay_api/models/payment_account.py 97.61% <100.00%> (+0.64%) ⬆️
pay-api/src/pay_api/models/payment_line_item.py 100.00% <100.00%> (ø)
pay-api/src/pay_api/resources/account.py 93.44% <100.00%> (ø)
pay-api/src/pay_api/services/payment.py 97.02% <100.00%> (-0.05%) ⬇️
pay-api/src/pay_api/utils/converter.py 100.00% <100.00%> (ø)
pay-api/src/pay_api/utils/enums.py 100.00% <100.00%> (ø)
... and 28 more

…hedule, we can get that from the invoice instead. More cleanup.
@seeker25 seeker25 added the bug Something isn't working label Feb 16, 2023
@seeker25 seeker25 requested a review from thorwolpert February 16, 2023 08:45
@seeker25
Copy link
Collaborator Author

Still needs to be tested, but seems to pass all the unit tests.

@seeker25 seeker25 marked this pull request as ready for review February 25, 2023 20:35
@seeker25 seeker25 temporarily deployed to dev February 25, 2023 21:31 — with GitHub Actions Inactive
@seeker25 seeker25 temporarily deployed to dev February 25, 2023 21:49 — with GitHub Actions Inactive
@seeker25 seeker25 temporarily deployed to dev February 25, 2023 21:56 — with GitHub Actions Inactive
@seeker25 seeker25 temporarily deployed to dev February 25, 2023 22:55 — with GitHub Actions Inactive
@seeker25
Copy link
Collaborator Author

seeker25 commented Feb 25, 2023

I've done some comparison on the payload versus marshmallow, added a fix for removing None fields. Seems to be ok.

image

50,000~ invoices took 42s second total to download a CSV, with two like filters

It took 1.3 minutes to download a CSV with the details search parameter entered (worst performance case).

Date filter range only:
52,000 invoices - 42s

Note our limit is 60,000 - and some of the more advanced filters could take longer (details for example, details will be fixed in the future when we move the column from jsonb to it's own table).

I've upped the gunicorn timeout to 100s just incase. We'll have to update the route timeouts to 100s as well.

I checked all the columns in the master transaction list, they seem to be all populating:

image

…r doesn't work).

Put in note regarding payment accounts needing one row.
Copy link
Collaborator

@kialj876 kialj876 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work 👍

One optional comment. If you want we can make that change afterwards as its currently not using the config file anyways

pay-api/docker-entrypoint.sh Outdated Show resolved Hide resolved
# Conflicts:
#	pay-api/docker-entrypoint.sh
@sonarqubecloud
Copy link

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

@seeker25 seeker25 temporarily deployed to dev February 27, 2023 19:56 — with GitHub Actions Inactive
@seeker25 seeker25 merged commit 19bc8f4 into bcgov:main Feb 27, 2023
@seeker25
Copy link
Collaborator Author

seeker25 commented Mar 1, 2023

@kialj876 Not sure what you mean it's not using that config file anyways? nvm wrong ticket I was thinking of

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants