This repository has been archived by the owner on Jul 13, 2023. It is now read-only.
Schema validation refactor in endpoint #379
Assignees
Milestone
Comments
bbangert
changed the title
This was referenced May 9, 2016
Closed
bbangert
added a commit
that referenced
this issue
Jun 2, 2016
This refactor splits out the base handler and several bits of generic callbacks into a base.py module. Each web handler will now get its own module, with simplepush.py as the initial refactor candidate. Marshmallow schemas are used to validate incoming requests and format/coerce/validate data in a separate thread. This allows the validation to make AWS calls or other expensive calls that previously required punting calls back/forth between the twisted thread-pool. The SimplePush endpoints issued are now under a new spush prefix so that they can go to a separate handler entirely that has the schema validation. WebPush endpoints will similarly be sent to a new prefix. An additional toggle is stored on the user object to track whether their record was upgraded for this change so that at a later point we can expire-on-connect old user records. Issue #379
bbangert
added a commit
that referenced
this issue
Jun 2, 2016
This refactor splits out the base handler and several bits of generic callbacks into a base.py module. Each web handler will now get its own module, with simplepush.py as the initial refactor candidate. Marshmallow schemas are used to validate incoming requests and format/coerce/validate data in a separate thread. This allows the validation to make AWS calls or other expensive calls that previously required punting calls back/forth between the twisted thread-pool. The SimplePush endpoints issued are now under a new spush prefix so that they can go to a separate handler entirely that has the schema validation. WebPush endpoints will similarly be sent to a new prefix. An additional toggle is stored on the user object to track whether their record was upgraded for this change so that at a later point we can expire-on-connect old user records. Issue #379
bbangert
added a commit
that referenced
this issue
Jun 3, 2016
This refactor splits out the base handler and several bits of generic callbacks into a base.py module. Each web handler will now get its own module, with simplepush.py as the initial refactor candidate. Marshmallow schemas are used to validate incoming requests and format/coerce/validate data in a separate thread. This allows the validation to make AWS calls or other expensive calls that previously required punting calls back/forth between the twisted thread-pool. The SimplePush endpoints issued are now under a new spush prefix so that they can go to a separate handler entirely that has the schema validation. WebPush endpoints will similarly be sent to a new prefix. An additional toggle is stored on the user object to track whether their record was upgraded for this change so that at a later point we can expire-on-connect old user records. Issue #379
bbangert
added a commit
that referenced
this issue
Jun 3, 2016
This refactor splits out the base handler and several bits of generic callbacks into a base.py module. Each web handler will now get its own module, with simplepush.py as the initial refactor candidate. Marshmallow schemas are used to validate incoming requests and format/coerce/validate data in a separate thread. This allows the validation to make AWS calls or other expensive calls that previously required punting calls back/forth between the twisted thread-pool. The SimplePush endpoints issued are now under a new spush prefix so that they can go to a separate handler entirely that has the schema validation. WebPush endpoints will similarly be sent to a new prefix. An additional toggle is stored on the user object to track whether their record was upgraded for this change so that at a later point we can expire-on-connect old user records. Issue #379
bbangert
added a commit
that referenced
this issue
Jun 3, 2016
This refactor splits out the base handler and several bits of generic callbacks into a base.py module. Each web handler will now get its own module, with simplepush.py as the initial refactor candidate. Marshmallow schemas are used to validate incoming requests and format/coerce/validate data in a separate thread. This allows the validation to make AWS calls or other expensive calls that previously required punting calls back/forth between the twisted thread-pool. The SimplePush endpoints issued are now under a new spush prefix so that they can go to a separate handler entirely that has the schema validation. WebPush endpoints will similarly be sent to a new prefix. An additional toggle is stored on the user object to track whether their record was upgraded for this change so that at a later point we can expire-on-connect old user records. Issue #379
bbangert
added a commit
that referenced
this issue
Jun 3, 2016
This refactor splits out the base handler and several bits of generic callbacks into a base.py module. Each web handler will now get its own module, with simplepush.py as the initial refactor candidate. Marshmallow schemas are used to validate incoming requests and format/coerce/validate data in a separate thread. This allows the validation to make AWS calls or other expensive calls that previously required punting calls back/forth between the twisted thread-pool. The SimplePush endpoints issued are now under a new spush prefix so that they can go to a separate handler entirely that has the schema validation. WebPush endpoints will similarly be sent to a new prefix. An additional toggle is stored on the user object to track whether their record was upgraded for this change so that at a later point we can expire-on-connect old user records. Issue #379
bbangert
added a commit
that referenced
this issue
Jun 3, 2016
This refactor splits out the base handler and several bits of generic callbacks into a base.py module. Each web handler will now get its own module, with simplepush.py as the initial refactor candidate. Marshmallow schemas are used to validate incoming requests and format/coerce/validate data in a separate thread. This allows the validation to make AWS calls or other expensive calls that previously required punting calls back/forth between the twisted thread-pool. The SimplePush endpoints issued are now under a new spush prefix so that they can go to a separate handler entirely that has the schema validation. WebPush endpoints will similarly be sent to a new prefix. An additional toggle is stored on the user object to track whether their record was upgraded for this change so that at a later point we can expire-on-connect old user records. Issue #379
bbangert
added a commit
that referenced
this issue
Jun 28, 2016
This refactor splits out the webpush endpoint to a new URL path and uses the marshmallow schema to validate requests. The webpush endpoints are issued under a new URL endpoint to avoid conflicts with the existing. Closes #379
bbangert
added a commit
that referenced
this issue
Jun 28, 2016
This refactor splits out the webpush endpoint to a new URL path and uses the marshmallow schema to validate requests. The webpush endpoints are issued under a new URL endpoint to avoid conflicts with the existing. Closes #379
bbangert
added a commit
that referenced
this issue
Jun 28, 2016
This refactor splits out the webpush endpoint to a new URL path and uses the marshmallow schema to validate requests. The webpush endpoints are issued under a new URL endpoint to avoid conflicts with the existing. Closes #379
jrconlin
pushed a commit
that referenced
this issue
Jun 28, 2016
This refactor splits out the webpush endpoint to a new URL path and uses the marshmallow schema to validate requests. The webpush endpoints are issued under a new URL endpoint to avoid conflicts with the existing. Closes #379
bbangert
added a commit
that referenced
this issue
Jul 13, 2016
This refactor splits out the base handler and several bits of generic callbacks into a base.py module. Each web handler will now get its own module, with simplepush.py as the initial refactor candidate. Marshmallow schemas are used to validate incoming requests and format/coerce/validate data in a separate thread. This allows the validation to make AWS calls or other expensive calls that previously required punting calls back/forth between the twisted thread-pool. The SimplePush endpoints issued are now under a new spush prefix so that they can go to a separate handler entirely that has the schema validation. WebPush endpoints will similarly be sent to a new prefix. An additional toggle is stored on the user object to track whether their record was upgraded for this change so that at a later point we can expire-on-connect old user records. Issue #379
bbangert
added a commit
that referenced
this issue
Jul 13, 2016
This refactor splits out the webpush endpoint to a new URL path and uses the marshmallow schema to validate requests. The webpush endpoints are issued under a new URL endpoint to avoid conflicts with the existing. Closes #379
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Our code has slowly grown an if/else heap of validation checks over time. We should refactor to use a proper validation library, my thought is to use colander (http://docs.pylonsproject.org/projects/colander/en/latest/index.html).
We will need a slightly custom decorator for our cyclone functions that kicks the validation off to a thread, before calling back to the function if validation succeeds.
Because the validation should occur in a thread-pool, it can do additional checks all at once that we typically punt to the thread-pool for multiple times, such as:
These checks can all be implemented as custom validators in colander so that we end up with a single consistent schema of what headers/payloads/etc are required that is easier to maintain and verify.
The text was updated successfully, but these errors were encountered: