Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Copy Blueprints Implementation #2184

Merged
merged 29 commits into from
Aug 9, 2021
Merged

Conversation

ChihweiLHBird
Copy link
Member

Related issue: #2163

@ChihweiLHBird ChihweiLHBird changed the title Initial version of Blueprint.copy, with a test case. Copy Blueprints Implementation Jul 7, 2021
@ChihweiLHBird
Copy link
Member Author

@ahopkins The parameters list of the copy method breaks the codeclimate rule: "Function copy has 5 arguments (exceeds 4 allowed)."

Do you think key words parameters will be a better choice? Or can we ignore the codeclimate issue?

@ahopkins
Copy link
Member

ahopkins commented Jul 7, 2021

Ignore that. I need to configure the rules. The defaults might work for an app, but are too restrictive for a framework.

@ChihweiLHBird
Copy link
Member Author

Ignore that. I need to configure the rules. The defaults might work for an app, but are too restrictive for a framework.

Thank you. I think this PR is roughly done, how can I get some feedbacks from you and other members of the community?

@ahopkins
Copy link
Member

ahopkins commented Jul 7, 2021

Sure, I'll take a look. Mark it as ready for review.

@sanic-org/framework thoughts?

@ChihweiLHBird ChihweiLHBird marked this pull request as ready for review July 7, 2021 17:20
@ChihweiLHBird ChihweiLHBird requested a review from a team as a code owner July 7, 2021 17:20
@ChihweiLHBird
Copy link
Member Author

ChihweiLHBird commented Jul 7, 2021

Sure, I'll take a look. Mark it as ready for review.

@sanic-org/framework thoughts?

One doubt I got is that in the copy method, should we use the default values of the parameters of the method in the new blueprint if the parameters was not provided by caller? Or we don't change anything in the new blueprint and let it use the old values copied from the old blue print.

Another way to do this, by using key words parameters, we can allow user to change any attribute (methods and variables) in the new blueprint. Something like:

def copy(
        self,
        name: str,
        **kwargs
    ):
        new_bp = deepcopy(self)
        for key, value in kwargs.items():
            if hasattr(new_bp, key):
                new_bp.key = value

Do you guys think it is good idea or not?

@Tronic
Copy link
Member

Tronic commented Jul 7, 2021

Another way to do this, by using key words parameters, we can allow user to change any attribute (methods and variables) in the new blueprint. Something like:

It is better to explicitly list the parameters. This gives auto completion and error checking in IDEs, not possible if a function takes **kwargs.

@ahopkins
Copy link
Member

ahopkins commented Jul 8, 2021

There are four solutions that I can see:

Pull from kwargs

Pro: Explicit knowledge that the dev intentionally passed a value
Con: No IDE auto-completion

Keyword with None as default value

Pro: Easy to check that a value was intentionally set
Con: Does not allow passing None as a value

Keyword with carrying over existing defaults

Pro: Easy to check that a value is NOT default and set to that
Con: What if you are returning a value back to its default?

Keyword with sentinel as default

Pro: Allows for both None and orig defaults to be passed
Con: Difficult to type annotate


Therefore, I suggest a two step approach:

# ./helpers.py
class Default:
    ...


_default = Default()
class Blueprint:
    def copy(
        self,
        name: str,
        *,
        url_prefix: Optional[Union[str, Default]] = _default,
        version: Optional[Union[int, str, float, Default]] = _default,
        strict_slashes: Optional[Union[bool, Default]] = _default,
        version_prefix: Optional[Union[str, Default]] = _default,
    ):
        ...

@ChihweiLHBird
Copy link
Member Author

There are four solutions that I can see:

Pull from kwargs

Pro: Explicit knowledge that the dev intentionally passed a value
Con: No IDE auto-completion

Keyword with None as default value

Pro: Easy to check that a value was intentionally set
Con: Does not allow passing None as a value

Keyword with carrying over existing defaults

Pro: Easy to check that a value is NOT default and set to that
Con: What if you are returning a value back to its default?

Keyword with sentinel as default

Pro: Allows for both None and orig defaults to be passed
Con: Difficult to type annotate


Therefore, I suggest a two step approach:

# ./helpers.py
class Default:
    ...


_default = Default()
class Blueprint:
    def copy(
        self,
        name: str,
        *,
        url_prefix: Optional[Union[str, Default]] = _default,
        version: Optional[Union[int, str, float, Default]] = _default,
        strict_slashes: Optional[Union[bool, Default]] = _default,
        version_prefix: Optional[Union[str, Default]] = _default,
    ):
        ...

Hi Adam, thanks for the suggestion. Do you think the default class can be used globally in the framework? We can achieve similar functionality in other functions as well with it, I think.

By the way, I don't quite understand what's the Con of the third solution you mentioned. Can you explain what is the situation of returning a value back to its default? Thank you so much.

@Tronic
Copy link
Member

Tronic commented Jul 8, 2021

Derive the sentinel type from NoneType, then plain Optional should work for annotation. Seems a bit hackish though, is it a frequent need to set some value to None?

@ChihweiLHBird
Copy link
Member Author

Derive the sentinel type from NoneType, then plain Optional should work for annotation. Seems a bit hackish though, is it a frequent need to set some value to None?

I think generally, we are not able to use NoneType as a class.

@Tronic
Copy link
Member

Tronic commented Jul 8, 2021

I think generally, we are not able to use NoneType as a class.

Today I learned something new. Looks like None and a few other built in values are not proper objects (in contrast to integers and floats which are).

TypeError: type 'NoneType' is not an acceptable base type

@ahopkins
Copy link
Member

ahopkins commented Jul 8, 2021

>>> class D(type(None)):
...  ...
... 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: type 'NoneType' is not an acceptable base type

Hardcoded protection: https://github.com/python/cpython/blob/main/Objects/typeobject.c#L2207


Off the top of my head, I do not have another use case for _default. Typically object() is the standard sentinel, but typing it is hard, broken, or awkward. Hence the Default class.


is it a frequent need to set some value to None?

None is a valid value and could be reused to reset to "default". Since this is entirely new functionality, it is hard to comment on the frequency. However, creating a blueprint with strict_slashes=None is not the same as strict_slashes=False.

bp1 = Blueprint("One", strict_slashes=False)
bp2 = bp1.copy("Two", strict_slashes=None)
group = Blueprint.group(bp1, bp2, strict_slashes=True)

In this use case, the routes on bp2 would have strict_slashes=True (as long as the route definition does not override it).


By the way, I don't quite understand what's the Con of the third solution you mentioned. Can you explain what is the situation of returning a value back to its default? Thank you so much.

See the above. If we simply looked for strict_slashes is not None (which is the default valie), then bp2 would end up with strict_slashes=False. Here, bp1 changed the value, but we want to go back to the default value for bp2. Option 3 would not allow us to do this. If, however, we check that strict_slashes != _default, then we can set strict_slahes to a new value, or the original default.

@ChihweiLHBird
Copy link
Member Author

I think the natural way to do copy is keeping everything same as the old object unless the user request a change. So I would suggest we to think about a way that allows users to pass or apply these below to the attributes of the new object:

  1. None as a value.
  2. The default value of an attribute when the object is created via its constructor with a minimal parameter input.

The second can be achieved by the Default class and object @ahopkins mentioned above. But None value still conflicts with the attributes from the old objects, meaning we cannot use None as the default value of the parameters of the copy method, otherwise we don't know whether or not the None is from the user.

I guess we must take a trade off when implementing this feature, 😂

@ahopkins
Copy link
Member

ahopkins commented Jul 9, 2021

I think the natural way to do copy is keeping everything same as the old object unless the user request a change. So I would suggest we to think about a way that allows users to pass or apply these below to the attributes of the new object:

  1. None as a value.

  2. The default value of an attribute when the object is created via its constructor with a minimal parameter input.

The second can be achieved by the Default class and object @ahopkins mentioned above. But None value still conflicts with the attributes from the old objects, meaning we cannot use None as the default value of the parameters of the copy method, otherwise we don't know whether or not the None is from the user.

I guess we must take a trade off when implementing this feature, 😂

The default values of all copy args should be _default for that reason.

@ChihweiLHBird
Copy link
Member Author

@ahopkins So, _default means the value of the attributes of the old object?

@ahopkins
Copy link
Member

@ahopkins So, _default means the value of the attributes of the old object?

In this case it means "do nothing"

if version != _default:
    # set version on new blueprint

@ChihweiLHBird
Copy link
Member Author

Thank you @ahopkins @Tronic

Can you take a look and see how is it now?

Copy link
Member

@ahopkins ahopkins left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for putting this together. I think it will be a nice add. Make sure to add some unit tests.

sanic/blueprints.py Outdated Show resolved Hide resolved
sanic/blueprints.py Outdated Show resolved Hide resolved
sanic/blueprints.py Show resolved Hide resolved
sanic/blueprints.py Outdated Show resolved Hide resolved
sanic/blueprints.py Outdated Show resolved Hide resolved
ChihweiLHBird and others added 2 commits August 7, 2021 16:03
Co-authored-by: Adam Hopkins <adam@amhopkins.com>
@ChihweiLHBird ChihweiLHBird requested a review from ahopkins August 8, 2021 04:22
sanic/blueprints.py Outdated Show resolved Hide resolved
sanic/blueprints.py Outdated Show resolved Hide resolved
sanic/blueprints.py Show resolved Hide resolved
tests/test_blueprint_copy.py Outdated Show resolved Hide resolved
Copy link
Member

@ahopkins ahopkins left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay... now that I closed some of the conversations so it is easier to follow, this LGTM. Just a couple more comments and we should be good to wrap this up. Thanks for all the work on this one!! 🚀

@ChihweiLHBird ChihweiLHBird requested a review from ahopkins August 9, 2021 07:58
sanic/blueprints.py Outdated Show resolved Hide resolved
ahopkins
ahopkins previously approved these changes Aug 9, 2021
@ahopkins
Copy link
Member

ahopkins commented Aug 9, 2021

Failing test

  >           raise FinalizationError("Cannot finalize router more than once.")

On this line:

           bp6 = bp1.copy(
              name="test_bp6",
              version=6,
              with_registration=True,
  >           version_prefix="/version",
          )
  
  tests/test_blueprint_copy.py:64: 

Easy solve:

before the b6 copy, add:

app.router.reset()

@ChihweiLHBird ChihweiLHBird requested a review from ahopkins August 9, 2021 20:13
@ahopkins ahopkins merged commit e2eefaa into sanic-org:main Aug 9, 2021
@ChihweiLHBird ChihweiLHBird deleted the zhiwei/bp-copy branch August 9, 2021 23:06
@ahopkins ahopkins mentioned this pull request Sep 30, 2021
ahopkins added a commit that referenced this pull request Oct 2, 2021
* Remove unnecessary import in test_constants.py, which also fixes an error on win (#2180)

Co-authored-by: Adam Hopkins <admhpkns@gmail.com>

* Manually reset the buffer when streaming request body (#2183)

* Remove Duplicated Dependencies and PEP 517 Support (#2173)

* Remove duplicated dependencies

* Specify setuptools as the tool for generating distribution (PEP 517)

* Add `isort` to `dev_require`

* manage all dependencies in setup.py

* Execute `make pretty`

* Set usedevelop to true (revert previous change)

* Fix the handling of the end of a chunked request. (#2188)

* Fix the handling of the end of a chunked request.

* Avoid hardcoding final chunk header size.

* Add some unit tests for pipeline body reading

* Decode bytes for json serialization

Co-authored-by: L. Kärkkäinen <tronic@users.noreply.github.com>
Co-authored-by: Adam Hopkins <adam@amhopkins.com>

* Resolve regressions in exceptions (#2181)

* Update sanic-routing to fix path issues plus lookahead / lookbehind support (#2178)

* Update sanic-routing to fix path issues plus lookahead / lookbehind support

* Update setup.py

Co-authored-by: Adam Hopkins <adam@amhopkins.com>
Co-authored-by: Adam Hopkins <admhpkns@gmail.com>

* style(app,blueprints): add some type hints (#2196)

* style(app,blueprints): add some type hints

* style(app): option is Any

* style(blueprints): url prefix default value is ``""``

* style(app): backward compatible

* style(app): backward compatible

* style(blueprints): defult is None

* style(app): apply code style (black)

* Update some CC config (#2199)

* Update README.rst

* raise exception for `_static_request_handler` unknown exception; add test with custom error (#2195)

Co-authored-by: n.feofanov <n.feofanov@visionlabs.ru>
Co-authored-by: Adam Hopkins <admhpkns@gmail.com>

* Change dumps to AnyStr (#2193)

* HTTP tests (#2194)

* Fix issues with after request handling in HTTP pipelining (#2201)

* Clean up after a request is complete, before the next pipelined request.

* Limit the size of request body consumed after handler has finished.

* Linter error.

* Add unit test re: bad headers

Co-authored-by: L. Kärkkäinen <tronic@users.noreply.github.com>
Co-authored-by: Adam Hopkins <admhpkns@gmail.com>
Co-authored-by: Adam Hopkins <adam@amhopkins.com>

* Update CHANGELOG

* Log remote address if available (#2207)

* Log remote address if available

* Add tests

* Fix testing version

Co-authored-by: Adam Hopkins <adam@amhopkins.com>

* Fixed for handling exceptions of asgi app call. (#2211)

@cansarigol3megawatt Thanks for looking into this and getting the quick turnaround on this. I will 🍒 pick this into the 21.6 branch and get it out a little later tonight.

* Signals Integration (#2160)

* Update some tests

* Resolve #2122 route decorator returning tuple

* Use rc sanic-routing version

* Update unit tests to <:str>

* Minimal working version with some signals implemented

* Add more http signals

* Update ASGI and change listeners to signals

* Allow for dynamic ODE signals

* Allow signals to be stacked

* Begin tests

* Prioritize match_info on keyword argument injection

* WIP on tests

* Compat with signals

* Work through some test coverage

* Passing tests

* Post linting

* Setup proper resets

* coverage reporting

* Fixes from vltr comments

* clear delayed tasks

* Fix bad test

* rm pycache

* uncomment windows tests (#2214)

* Add convenience methods to BP groups (#2209)

* Fix bug where ws exceptions not being logged (#2213)

* Fix bug where ws exceptions not being logged

* Fix t\est

* Style: add type hints (#2217)

* style(routes): add_route argument, return typing

* style(listeners): typing

* style(views): typing as_view

* style(routes): change type hint

* style(listeners): change type hint

* style(routes): change type hint

* add some more types

* Change as_view typing

* Add some cleaner type annotations

Co-authored-by: Adam Hopkins <adam@amhopkins.com>

* Add default messages to SanicExceptions (#2216)

* Add default messages to SanicExceptions

* Cleaner exception message setting

* Copy Blueprints Implementation (#2184)

* Accept header parsing (#2200)

* Add some tests

* docstring

* Add accept matching

* Add some more tests on matching

* Add matching flags for wildcards

* Add mathing controls to accept

* Limit uvicorn 14 in testing

* Add convenience for annotated handlers (#2225)

* Split HttpProtocol parts into base SanicProtocol and HTTPProtocol subclass (#2229)

* Split HttpProtocol parts into base SanicProtocol and HTTPProtocol subclass.

* lint fixes

* re-black server.py

* Move server.py into its own module (#2230)

* Move server.py into its own module

* Change monkeypatch path on test_logging.py

* Blueprint specific exception handlers (#2208)

* Call abort() on sockets after close() to prevent dangling sockets (#2231)

* Add ability to return Falsey but not-None from handlers (#2236)

* Adds Blueprint Group exception decorator (#2238)

* Add exception decorator

* Added tests

* Fix line too long

* Static DIR and FILE resource types (#2244)

* Explicit static directive for serving file or dir


Co-authored-by: anbuhckr <36891836+anbuhckr@users.noreply.github.com>
Co-authored-by: anbuhckr <miki.suhendra@gmail.com>

* Close HTTP loop when connection task cancelled (#2245)

* Terminate loop when no transport exists

* Add log when closing HTTP loop because of shutdown

* Add unit test

* New websockets (#2158)

* First attempt at new Websockets implementation based on websockets >= 9.0, with sans-i/o features. Requires more work.

* Update sanic/websocket.py

Co-authored-by: Adam Hopkins <adam@amhopkins.com>

* Update sanic/websocket.py

Co-authored-by: Adam Hopkins <adam@amhopkins.com>

* Update sanic/websocket.py

Co-authored-by: Adam Hopkins <adam@amhopkins.com>

* wip, update websockets code to new Sans/IO API

* Refactored new websockets impl into own modules
Incorporated other suggestions made by team

* Another round of work on the new websockets impl
* Added websocket_timeout support (matching previous/legacy support)
* Lots more comments
* Incorporated suggested changes from previous round of review
* Changed RuntimeError usage to ServerError
* Changed SanicException usage to ServerError
* Removed some redundant asserts
* Change remaining asserts to ServerErrors
* Fixed some timeout handling issues
* Fixed websocket.close() handling, and made it more robust
* Made auto_close task smarter and more error-resilient
* Made fail_connection routine smarter and more error-resilient

* Further new websockets impl fixes
* Update compatibility with Websockets v10
* Track server connection state in a more precise way
* Try to handle the shutdown process more gracefully
* Add a new end_connection() helper, to use as an alterative to close() or fail_connection()
* Kill the auto-close task and keepalive-timeout task when sanic is shutdown
* Deprecate WEBSOCKET_READ_LIMIT and WEBSOCKET_WRITE_LIMIT configs, they are not used in this implementation.

* Change a warning message to debug level
Remove default values for deprecated websocket parameters

* Fix flake8 errors

* Fix a couple of missed failing tests

* remove websocket bench from examples

* Integrate suggestions from code reviews
Use Optional[T] instead of union[T,None]
Fix mypy type logic errors
change "is not None" to truthy checks where appropriate
change "is None" to falsy checks were appropriate
Add more debug logging when debug mode is on
Change to using sanic.logger for debug logging rather than error_logger.

* Fix long line lengths of debug messages
Add some new debug messages when websocket IO is paused and unpaused for flow control
Fix websocket example to use app.static()

* remove unused import in websocket example app

* re-run isort after Flake8 fixes

Co-authored-by: Adam Hopkins <adam@amhopkins.com>
Co-authored-by: Adam Hopkins <admhpkns@gmail.com>

* Account for BP with exception handler but no routes (#2246)

* Don't log "enabled" if auto-reload disabled (#2247)

Fixes #2240

Co-authored-by: Adam Hopkins <admhpkns@gmail.com>

* Smarter auto fallback (#2162)

* Smarter auto fallback

* remove config from blueprints

* Add tests for error formatting

* Add check for proper format

* Fix some tests

* Add some tests

* docstring

* Add accept matching

* Add some more tests on matching

* Fix contains bug, earlier return on MediaType eq

* Add matching flags for wildcards

* Add mathing controls to accept

* Cleanup dev cruft

* Add cleanup and resolve OSError relating to test implementation

* Fix test

* Fix some typos

* Some fixes to the new Websockets impl (#2248)

* First attempt at new Websockets implementation based on websockets >= 9.0, with sans-i/o features. Requires more work.

* Update sanic/websocket.py

Co-authored-by: Adam Hopkins <adam@amhopkins.com>

* Update sanic/websocket.py

Co-authored-by: Adam Hopkins <adam@amhopkins.com>

* Update sanic/websocket.py

Co-authored-by: Adam Hopkins <adam@amhopkins.com>

* wip, update websockets code to new Sans/IO API

* Refactored new websockets impl into own modules
Incorporated other suggestions made by team

* Another round of work on the new websockets impl
* Added websocket_timeout support (matching previous/legacy support)
* Lots more comments
* Incorporated suggested changes from previous round of review
* Changed RuntimeError usage to ServerError
* Changed SanicException usage to ServerError
* Removed some redundant asserts
* Change remaining asserts to ServerErrors
* Fixed some timeout handling issues
* Fixed websocket.close() handling, and made it more robust
* Made auto_close task smarter and more error-resilient
* Made fail_connection routine smarter and more error-resilient

* Further new websockets impl fixes
* Update compatibility with Websockets v10
* Track server connection state in a more precise way
* Try to handle the shutdown process more gracefully
* Add a new end_connection() helper, to use as an alterative to close() or fail_connection()
* Kill the auto-close task and keepalive-timeout task when sanic is shutdown
* Deprecate WEBSOCKET_READ_LIMIT and WEBSOCKET_WRITE_LIMIT configs, they are not used in this implementation.

* Change a warning message to debug level
Remove default values for deprecated websocket parameters

* Fix flake8 errors

* Fix a couple of missed failing tests

* remove websocket bench from examples

* Integrate suggestions from code reviews
Use Optional[T] instead of union[T,None]
Fix mypy type logic errors
change "is not None" to truthy checks where appropriate
change "is None" to falsy checks were appropriate
Add more debug logging when debug mode is on
Change to using sanic.logger for debug logging rather than error_logger.

* Fix long line lengths of debug messages
Add some new debug messages when websocket IO is paused and unpaused for flow control
Fix websocket example to use app.static()

* remove unused import in websocket example app

* re-run isort after Flake8 fixes

* Some fixes to the new Websockets impl
Will throw WebsocketClosed exception instead of ServerException now when attempting to read or write to closed websocket, this makes it easier to catch
The various ws.recv() methods now have the ability to raise CancelledError into your websocket handler
Fix a niche close-socket negotiation bug
Fix bug where http protocol thought the websocket never sent any response.
Allow data to still send in some cases after websocket enters CLOSING state.
Fix some badly formatted and badly placed comments

* allow eof_received to send back data too, if the connection is in CLOSING state

Co-authored-by: Adam Hopkins <adam@amhopkins.com>
Co-authored-by: Adam Hopkins <admhpkns@gmail.com>

* 21.9 release docs (#2218)

* Beging 21.9 release docs

* Add PRs to changelog

* Change deprecation version

* Update logging tests

* Bump version

* Update changelog

* Change dev install command (#2251)

Co-authored-by: Zhiwei <zhi.wei.liang@outlook.com>
Co-authored-by: L. Kärkkäinen <98187+Tronic@users.noreply.github.com>
Co-authored-by: L. Kärkkäinen <tronic@users.noreply.github.com>
Co-authored-by: Robert Palmer <robd003@users.noreply.github.com>
Co-authored-by: Ryu JuHeon <saidbysolo@gmail.com>
Co-authored-by: gluhar2006 <49654448+gluhar2006@users.noreply.github.com>
Co-authored-by: n.feofanov <n.feofanov@visionlabs.ru>
Co-authored-by: Néstor Pérez <25409753+prryplatypus@users.noreply.github.com>
Co-authored-by: Can Sarigol <56863826+cansarigol3megawatt@users.noreply.github.com>
Co-authored-by: Zhiwei <chihwei.public@outlook.com>
Co-authored-by: YongChan Cho <h3236516@gmail.com>
Co-authored-by: Zhiwei <zhiwei@sinatra.ai>
Co-authored-by: Ashley Sommer <ashleysommer@gmail.com>
Co-authored-by: anbuhckr <36891836+anbuhckr@users.noreply.github.com>
Co-authored-by: anbuhckr <miki.suhendra@gmail.com>
ChihweiLHBird pushed a commit to ChihweiLHBird/sanic that referenced this pull request Jun 1, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants