Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

21464 - Merge EFT into main branch #1552

Merged
merged 94 commits into from
May 31, 2024
Merged

Conversation

seeker25
Copy link
Collaborator

Issue #:
bcgov/entity#21464

Description of changes:

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of the sbc-pay license (Apache 2.0).

seeker25 and others added 30 commits March 5, 2024 12:22
* Update docker files to python 3.12.2

* Update CI flows

* update requirements

* Update grcpiostatus and protobuf

* Add python 3.12 to test CI

* Add in more python 3.12

* Fix flake8

* Get the bulk of the unit tests working, upgrade marshmellow, upgrade flask, upgrade sqlalchemy - still need to fix "tracing", still need to fix sql-continuum, still need to fix exception handling, still need to fix JWT library flask-oidc-jwt (doesn't work with Flask 3.0 without update)

* Small cleanup

* remove autoflush param, didn't mean for it to be in there

* Update requirements to pull from github, where we fixed the library

* Clear out some old TODOs

* clear out old todo

* remove todo

* old todo

* todo removal

* Remove todo

* change default database name

* Use more modern database

* nuke tracer for now

* add in ci for queue_python_upgrade

* Fix linting

* remove setup.cfg changes

* fix unit test

* Small unit test fix

* use utcnow instead of now.

* Move created on to utcnow

* unit test fixes

* Small unit test tweak

* use datetime.now(tz=timezone.utc)
* Squashed commit of the following:

commit e1c58f9
Author: Rodrigo Barraza <hello@rod.dev>
Date:   Tue Mar 5 08:42:01 2024 -0800

    Cleanup auto_save

commit 9b8a9a0
Author: Rodrigo Barraza <hello@rod.dev>
Date:   Tue Mar 5 08:33:27 2024 -0800

    Remove committing

commit a05e210
Author: Rodrigo Barraza <hello@rod.dev>
Date:   Tue Mar 5 08:30:07 2024 -0800

    Reference and Receipt creation

commit edb6e51
Author: Rodrigo Barraza <hello@rod.dev>
Date:   Tue Mar 5 08:18:47 2024 -0800

    Updates to EFT Service

    Handling invoice reference creation

* Reverting reconciliation changes

* PR feedback updates
* Updates to EFT Service

Handling invoice reference creation

* Reference and Receipt creation

* Remove committing

* Cleanup auto_save

* Revert EFT service

* PR feedback updates

Bringing back auto_save
* update api and payment-jobs for EFT cfs_account create

* add updated requirements.txt from PR#18263

* isort fix
* Update requirements.txt

* Fix CI

* Changes to convert project to new queues

* Remove nats from workflows

* remove nats from docker compose

* remove q_cli

* Remove more NATS

* Remove more nats

* Gun events-listener, combining the queues for less tech debt

* More nats removal

* Move payment-reconciliations into pay-queue

* Change project structure of queues

* Add in tests from events listener

* Getting rid of some infrastructure and tech debt we don't need

* Enhance gcp_queue_publisher

* Wire up ftp poller to new queues

* Add in enum, change util func

* Small tweaks for gcp_queue_publisher

* Put in adhoc test for publishing to queue

* Add in PaymentToken dataclass shared across apps

* shorten enums

* Replace queue messages in mailer in payment-jobs

* Add more MessageTypes to enum

* More enum

* Convert to PUB

* missing comma

* Add in pub

* Rework entire PUB

* Clean up clean up everybody clean up!

* Small fix for ftp-poller

* Fix ref

* Holy moly third try it must be friday

* fix lint

* remove unused receipt method

* payment jobs updates, makefile for pay-queue

* linting fixes, requirements fixes

* Dependencies are being funny with this one - will need to relook at it. This should work for now.

* Update old references

* Add in pay jobs CI

* directory change

* Use pip 24.0.0

* Flask-Migrate upgrade + disable compare_type

* Small tweaks for jobs, still some failing tests

* Use scalars to fix where conditions when there are tuples, as those cause exceptions

* Fix CI

* Fix unit test for pay-api

* Lint fixes and others

* Update CI

* fix deps

* update reqs

* Take out tracing for BCOL-API

* more tracing removal

* 1 more spot for tracing

* Fix linting, move pubsub to local, had issues referencing it as a library

* Update requirements

* More test / lint fixes

* Cleanup conftest

* more cleanup

* Some unit tests working, more to go

* More cleanup

* 4 unit tests to go

* Fix eft reconciliation tests

* Lint fixes more unit test fixes

* Unit tests should be all passing

* Remove redundant comment

* Cleanup

* Lint fix

* Add in code coverage for pay-queue

* Put in event listener topic

* Some cleanup

* Update requirements
* update api and payment-jobs for EFT cfs_account create

* add updated requirements.txt from PR#18263

* isort fix

* EFT - Create CFS account job - 2

* remove redundant function

* remove incorrect comment
* Migrate to Poetry

* More cleanup, add dev deps

* Init poetry, add in deps

* Docker file poetry for bcol-api

* Revert makefile changes

* Fix make files

* Docker file + Makefile update

* makefile update

* Makefile update

* Fix makefiles

* Fix pay-admin docker

* update pay queue dockerfile

* minor nudge

* one more nudge

* Fix missing port

* Fix poetry for docker

* attempt to fix CI

* attempt to fix pay-queue migrations

* attempt to fix CI #999

* Update paths

* Fix pay queues migration path

* Fix path for jobs

* rename payment-jobs -> jobs

* Update maintainer

* CI fix
Co-authored-by: Patrick Wei <44277752+pwei1018@users.noreply.github.com>
…1448)

* First run at pubsub emulation

* Use pg8000 instead of psycopg2

* Remove requirement of keys allow for unauthenticated client

* Change gunicorn to match lear

* Remove prehooks, no concepts of prehooks anymore

* Remove jaeger-client

* Fix unit tests

* Update dependencies

* Use test environment variables

* Update secret for linux

* Checkout v4

* Add in extra hosts

* bind to 0.0.0.0
* Upgrade CD flows for Payment projects.

* fixed linting.
* Fixed empty value issue.

* Fixed deployment config issue.

* Fixed deployment environment variables issue.

* Fixed the configuration.
@seeker25
Copy link
Collaborator Author

Fixing 12 codecov issues

Copy link

codecov bot commented May 31, 2024

Codecov Report

Attention: Patch coverage is 82.01439% with 175 lines in your changes are missing coverage. Please review.

Project coverage is 90.95%. Comparing base (79924ce) to head (2a3214a).
Report is 156 commits behind head on main.

Current head 2a3214a differs from pull request most recent head 29a69f6

Please upload reports for the commit 29a69f6 to get more accurate results.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1552      +/-   ##
==========================================
- Coverage   91.45%   90.95%   -0.51%     
==========================================
  Files         186      206      +20     
  Lines       11319    13355    +2036     
==========================================
+ Hits        10352    12147    +1795     
- Misses        967     1208     +241     
Flag Coverage Δ
bcolapi ?
eventlistenerqueue ?
payapi 92.57% <82.93%> (-1.17%) ⬇️
paymentjobs 82.76% <79.40%> (+2.54%) ⬆️
paymentreconciliationsqueue 91.16% <85.36%> (-0.29%) ⬇️
reportapi 90.84% <66.66%> (-0.26%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
jobs/payment-jobs/tasks/ap_task.py 96.07% <100.00%> (+0.03%) ⬆️
jobs/payment-jobs/tasks/cfs_create_account_task.py 92.68% <100.00%> (+2.43%) ⬆️
...ayment-jobs/tasks/ejv_partner_distribution_task.py 99.37% <100.00%> (+0.24%) ⬆️
jobs/payment-jobs/tasks/ejv_payment_task.py 96.61% <100.00%> (+0.14%) ⬆️
pay-api/src/pay_api/__init__.py 91.66% <100.00%> (+1.04%) ⬆️
...-api/src/pay_api/factory/payment_system_factory.py 98.68% <100.00%> (-0.02%) ⬇️
pay-api/src/pay_api/models/__init__.py 100.00% <100.00%> (ø)
pay-api/src/pay_api/models/account_fee.py 100.00% <100.00%> (ø)
pay-api/src/pay_api/models/base_model.py 96.42% <100.00%> (-3.58%) ⬇️
pay-api/src/pay_api/models/base_schema.py 95.00% <100.00%> (-5.00%) ⬇️
... and 89 more

... and 113 files with indirect coverage changes

@ochiu
Copy link
Collaborator

ochiu commented May 31, 2024

Thanks for powering through this.
Looks good to me, will check back as I know you are taking a look at some of the CI issues.

@@ -1,51 +1,49 @@
Flask-Moment==1.0.5
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will remove

Copy link

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarCloud

@seeker25 seeker25 merged commit 04476af into main May 31, 2024
33 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants