Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[pull] master from scrapy:master #22

Open
wants to merge 750 commits into
base: master
Choose a base branch
from
Open

[pull] master from scrapy:master #22

wants to merge 750 commits into from
This pull request is big! We’re only showing the most recent 250 commits.

Commits on Mar 12, 2024

  1. Merge pull request #6290 from Gallaecio/fix-skips 

    Fix WindowsRunSpiderCommandTest skip outside Windows for older Twisted
    @wRAR
    wRAR authored Mar 12, 2024
    Configuration menu
    Copy the full SHA
    95a70d3 View commit details
    Browse the repository at this point in the history

Commits on Mar 13, 2024

  1. Remove tests/requirements.txt and refactor extra deps (#6272) 

    Co-authored-by: lucasbelo777 <lucas.belo@cialdnb.com>
    @lucas-belo @lucasbelo777
    lucas-belo and lucasbelo777 authored Mar 13, 2024
    Configuration menu
    Copy the full SHA
    4460d3e View commit details
    Browse the repository at this point in the history

Commits on Mar 14, 2024

  1. Merge pull request #6231 from elramen/check_command_tests 

    Added tests for check command
    @wRAR
    wRAR authored Mar 14, 2024
    Configuration menu
    Copy the full SHA
    7255dfd View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into master

    @wRAR
    wRAR authored Mar 14, 2024
    Configuration menu
    Copy the full SHA
    5bf0e1d View commit details
    Browse the repository at this point in the history

  3. Run black.

    @wRAR
    wRAR committed Mar 14, 2024
    Configuration menu
    Copy the full SHA
    8d917c0 View commit details
    Browse the repository at this point in the history

  4. Merge pull request #6242 from cankupeli/master 

    implemented tests for branch coverage for function printSummary
    @wRAR
    wRAR authored Mar 14, 2024
    Configuration menu
    Copy the full SHA
    02b97f9 View commit details
    Browse the repository at this point in the history

Commits on Apr 1, 2024

  1. modified string's concat to f-strings (#6296)

    @igeni
    igeni authored Apr 1, 2024
    Configuration menu
    Copy the full SHA
    f7bf3f7 View commit details
    Browse the repository at this point in the history

Commits on Apr 4, 2024

  1. Handle robots.txt files not UTF-8 encoded

    @lorenzoverardo
    lorenzoverardo committed Apr 4, 2024
    Configuration menu
    Copy the full SHA
    7b37dcd View commit details
    Browse the repository at this point in the history

Commits on Apr 5, 2024

  1. Merge pull request #6298 from lorenzoverardo/bug-6292/robots.txt-non-… 

    …UTF-8

Handle robots.txt files not UTF-8 encoded
    @wRAR
    wRAR authored Apr 5, 2024
    Configuration menu
    Copy the full SHA
    5d31e89 View commit details
    Browse the repository at this point in the history

Commits on Apr 8, 2024

  1. Fix WrappedRequest.get_header raising TypeError if default is None (#… 

    …6310)
    @VMRuiz
    VMRuiz authored Apr 8, 2024
    Configuration menu
    Copy the full SHA
    48c5a8c View commit details
    Browse the repository at this point in the history

Commits on Apr 11, 2024

  1. Update practices.rst

    @Gallaecio
    Gallaecio authored Apr 11, 2024
    Configuration menu
    Copy the full SHA
    1d11ea3 View commit details
    Browse the repository at this point in the history

Commits on Apr 17, 2024

  1. chore: fix some typos in comments (#6317) 

    Signed-off-by: TechVest <techdashen@qq.com>
    @TechVest
    TechVest authored Apr 17, 2024
    Configuration menu
    Copy the full SHA
    5f67c01 View commit details
    Browse the repository at this point in the history

  2. Fix test expectations (#6316)

    kokobhara authored Apr 17, 2024
    Configuration menu
    Copy the full SHA
    b1fe97d View commit details
    Browse the repository at this point in the history

Commits on Apr 19, 2024

  1. Merge pull request #6221 from jxlil/fix/LxmlLinkExtractor 

    fix: LxmlLinkExtractor unique_list missing key
    @wRAR
    wRAR authored Apr 19, 2024
    Configuration menu
    Copy the full SHA
    a5da77d View commit details
    Browse the repository at this point in the history

Commits on Apr 29, 2024

  1. Remove the auto-generated copyright years from the docs footer. (#6322)

    @wRAR
    wRAR authored Apr 29, 2024
    Configuration menu
    Copy the full SHA
    a166e97 View commit details
    Browse the repository at this point in the history

  2. Full typing for scrapy/extensions, part 3. (#6325)

    @wRAR
    wRAR authored Apr 29, 2024
    Configuration menu
    Copy the full SHA
    57acad3 View commit details
    Browse the repository at this point in the history

  3. Typing for build_from_*. (#6326)

    @wRAR
    wRAR authored Apr 29, 2024
    Configuration menu
    Copy the full SHA
    d7da298 View commit details
    Browse the repository at this point in the history

  4. Full typing for scrapy/cmdline.py.

    @wRAR
    wRAR committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    9eea22f View commit details
    Browse the repository at this point in the history

  5. Full typing for scrapy/item.py.

    @wRAR
    wRAR committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    fc1a83e View commit details
    Browse the repository at this point in the history

  6. Full typing for scrapy/extension.py.

    @wRAR
    wRAR committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    08a265b View commit details
    Browse the repository at this point in the history

  7. Full typing for scrapy/mail.py.

    @wRAR
    wRAR committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    38020e0 View commit details
    Browse the repository at this point in the history

Commits on May 4, 2024

  1. Full typing for scrapy/pqueues.py.

    @wRAR
    wRAR committed May 4, 2024
    Configuration menu
    Copy the full SHA
    0c8e21b View commit details
    Browse the repository at this point in the history

  2. Fix MutableMapping import for Python 3.8.

    @wRAR
    wRAR committed May 4, 2024
    Configuration menu
    Copy the full SHA
    21fa076 View commit details
    Browse the repository at this point in the history

  3. Full typing for scrapy/resolver.py.

    @wRAR
    wRAR committed May 4, 2024
    Configuration menu
    Copy the full SHA
    ad35ffd View commit details
    Browse the repository at this point in the history

  4. Full typing for scrapy/robotstxt.py.

    @wRAR
    wRAR committed May 4, 2024
    Configuration menu
    Copy the full SHA
    b749db9 View commit details
    Browse the repository at this point in the history

  5. Full typing for scrapy/squeues.py.

    @wRAR
    wRAR committed May 4, 2024
    Configuration menu
    Copy the full SHA
    5f7fd2a View commit details
    Browse the repository at this point in the history

  6. Add queue typing to scrapy/core/scheduler.py.

    @wRAR
    wRAR committed May 4, 2024
    Configuration menu
    Copy the full SHA
    203fa96 View commit details
    Browse the repository at this point in the history

  7. Fixes for queuelib.

    @wRAR
    wRAR committed May 4, 2024
    Configuration menu
    Copy the full SHA
    bd0d4ce View commit details
    Browse the repository at this point in the history

Commits on May 5, 2024

  1. Use the Self type hint in from_crawler/from_settings.

    @wRAR
    wRAR committed May 5, 2024
    Configuration menu
    Copy the full SHA
    1f39430 View commit details
    Browse the repository at this point in the history

  2. Full typing for scrapy/http/cookies.py.

    @wRAR
    wRAR committed May 5, 2024
    Configuration menu
    Copy the full SHA
    8a08283 View commit details
    Browse the repository at this point in the history

  3. Full typing for scrapy/linkextractors.

    @wRAR
    wRAR committed May 5, 2024
    Configuration menu
    Copy the full SHA
    c76dfc3 View commit details
    Browse the repository at this point in the history

  4. Skip coverage checks for TYPE_CHECKING blocks.

    @wRAR
    wRAR committed May 5, 2024
    Configuration menu
    Copy the full SHA
    4b47a5d View commit details
    Browse the repository at this point in the history

  5. Fix Python 3.8.

    @wRAR
    wRAR committed May 5, 2024
    Configuration menu
    Copy the full SHA
    40e4a59 View commit details
    Browse the repository at this point in the history

Commits on May 6, 2024

  1. Indicate that Selector.type can be json (#6334)

    @kumar-sanchay
    kumar-sanchay authored May 6, 2024
    Configuration menu
    Copy the full SHA
    aa025d7 View commit details
    Browse the repository at this point in the history

  2. Small fix for _get_slot_key().

    @wRAR
    wRAR committed May 6, 2024
    Configuration menu
    Copy the full SHA
    2cba789 View commit details
    Browse the repository at this point in the history

  3. Merge pull request #6336 from wRAR/typing-cookies 

    Full typing for scrapy/http/cookies.py.
    @wRAR
    wRAR authored May 6, 2024
    Configuration menu
    Copy the full SHA
    07e31b9 View commit details
    Browse the repository at this point in the history

  4. Merge pull request #6337 from wRAR/typing-linkextractors 

    Full typing for scrapy/linkextractors.
    @wRAR
    wRAR authored May 6, 2024
    Configuration menu
    Copy the full SHA
    8a083fb View commit details
    Browse the repository at this point in the history

  5. Merge pull request #6335 from wRAR/typing-self 

    Use the Self type hint in from_crawler/from_settings.
    @wRAR
    wRAR authored May 6, 2024
    Configuration menu
    Copy the full SHA
    d338982 View commit details
    Browse the repository at this point in the history

  6. Merge pull request #6333 from wRAR/typing-scrapy 

    Full typing for scrapy/*.py
    @wRAR
    wRAR authored May 6, 2024
    Configuration menu
    Copy the full SHA
    563ecbe View commit details
    Browse the repository at this point in the history

  7. Small typing improvements in scrapy/core/downloader/contextfactory.py.

    @wRAR
    wRAR committed May 6, 2024
    Configuration menu
    Copy the full SHA
    c4d2748 View commit details
    Browse the repository at this point in the history

  8. Full typing for smaller download handlers.

    @wRAR
    wRAR committed May 6, 2024
    Configuration menu
    Copy the full SHA
    ec4d407 View commit details
    Browse the repository at this point in the history

  9. Full typing for scrapy/core/downloader/handlers/ftp.py.

    @wRAR
    wRAR committed May 6, 2024
    Configuration menu
    Copy the full SHA
    e8e13eb View commit details
    Browse the repository at this point in the history

  10. Full typing for scrapy/core/downloader/handlers/http10.py.

    @wRAR
    wRAR committed May 6, 2024
    Configuration menu
    Copy the full SHA
    af3e38a View commit details
    Browse the repository at this point in the history

  11. More typing for scrapy/core/downloader/handlers/http11.py.

    @wRAR
    wRAR committed May 6, 2024
    Configuration menu
    Copy the full SHA
    045387e View commit details
    Browse the repository at this point in the history

  12. Re-run pre-commit.

    @wRAR
    wRAR committed May 6, 2024
    Configuration menu
    Copy the full SHA
    a828da9 View commit details
    Browse the repository at this point in the history

Commits on May 7, 2024

  1. Merge pull request #6341 from wRAR/typing-downloader 

    More typing for scrapy/core/downloader
    @wRAR
    wRAR authored May 7, 2024
    Configuration menu
    Copy the full SHA
    c9bac7a View commit details
    Browse the repository at this point in the history

  2. Update MANIFEST.in.

    @wRAR
    wRAR committed May 7, 2024
    Configuration menu
    Copy the full SHA
    6bbfb53 View commit details
    Browse the repository at this point in the history

Commits on May 8, 2024

  1. Merge pull request #6344 from wRAR/fix-twinecheck 

    Update MANIFEST.in.
    @wRAR
    wRAR authored May 8, 2024
    Configuration menu
    Copy the full SHA
    4300a1d View commit details
    Browse the repository at this point in the history

  2. Closes #6342. Setting METAREFRESH_IGNORE_TAGS to [‘noscript’] by default

    @aisha-partha
    aisha-partha committed May 8, 2024
    Configuration menu
    Copy the full SHA
    180bc9b View commit details
    Browse the repository at this point in the history

  3. Closes #6342. Update documentation on METAREFRESH_IGNORE_TAGS default… 

    … value change
    @aisha-partha
    aisha-partha committed May 8, 2024
    Configuration menu
    Copy the full SHA
    3590a1f View commit details
    Browse the repository at this point in the history

  4. Update METAREFRESH_IGNORE_TAGS default value in documentation

    @aisha-partha
    aisha-partha committed May 8, 2024
    Configuration menu
    Copy the full SHA
    2e13a9b View commit details
    Browse the repository at this point in the history

  5. Fix typo

    @Gallaecio
    Gallaecio authored May 8, 2024
    Configuration menu
    Copy the full SHA
    fe163d9 View commit details
    Browse the repository at this point in the history

Commits on May 10, 2024

  1. Merge pull request #6347 from aisha-partha/master 

    Closes #6342. Setting METAREFRESH_IGNORE_TAGS to [‘noscript’] by default
    @wRAR
    wRAR authored May 10, 2024
    Configuration menu
    Copy the full SHA
    10a843a View commit details
    Browse the repository at this point in the history

  2. Make certain args of ScrapyAgent and TunnelingAgent required (#6349)

    @kumar-sanchay
    kumar-sanchay authored May 10, 2024
    Configuration menu
    Copy the full SHA
    ae7bb84 View commit details
    Browse the repository at this point in the history

  3. Add Downloader.get_slot_key() without a spider parameter (#6352)

    @kumar-sanchay
    kumar-sanchay authored May 10, 2024
    Configuration menu
    Copy the full SHA
    c9ef520 View commit details
    Browse the repository at this point in the history

Commits on May 13, 2024

  1. Improve typing for Spider.parse(). (#6274)

    @wRAR
    wRAR authored May 13, 2024
    Configuration menu
    Copy the full SHA
    93f0628 View commit details
    Browse the repository at this point in the history

  2. Use ParamSpec for callables. (#6353)

    @wRAR
    wRAR authored May 13, 2024
    Configuration menu
    Copy the full SHA
    4ed5c5a View commit details
    Browse the repository at this point in the history

  3. Full typing for scrapy/spiders. (#6356)

    @wRAR
    wRAR authored May 13, 2024
    Configuration menu
    Copy the full SHA
    b8e333c View commit details
    Browse the repository at this point in the history

  4. Add SpiderLoggerAdapter, change Spider.logger to return SpiderLoggerA… 

    …dapter (#6324)
    @bloodforcream
    bloodforcream authored May 13, 2024
    Configuration menu
    Copy the full SHA
    b88f22c View commit details
    Browse the repository at this point in the history

  5. Merge branch 'master' into typing-request-response-cls

    @wRAR
    wRAR committed May 13, 2024
    Configuration menu
    Copy the full SHA
    d97d32c View commit details
    Browse the repository at this point in the history

Commits on May 14, 2024

  1. Merge 2.11.2 changes (#6363)

    @Gallaecio
    Gallaecio authored May 14, 2024
    Configuration menu
    Copy the full SHA
    d2f1e00 View commit details
    Browse the repository at this point in the history

Commits on May 15, 2024

  1. Allow user-defined secure cookies (#6357)

    @Gallaecio
    Gallaecio authored May 15, 2024
    Configuration menu
    Copy the full SHA
    812fd23 View commit details
    Browse the repository at this point in the history

Commits on May 16, 2024

  1. Update expectations of cookies after redirects. (#6367)

    @wRAR
    wRAR authored May 16, 2024
    Configuration menu
    Copy the full SHA
    631fc65 View commit details
    Browse the repository at this point in the history

Commits on May 19, 2024

  1. Full typing for scrapy/contracts.

    @wRAR
    wRAR committed May 19, 2024
    Configuration menu
    Copy the full SHA
    b99526b View commit details
    Browse the repository at this point in the history

Commits on May 22, 2024

  1. Merge branch 'master' into py313

    @wRAR
    wRAR committed May 22, 2024
    Configuration menu
    Copy the full SHA
    85d7458 View commit details
    Browse the repository at this point in the history

  2. Bump 3.13 to beta1.

    @wRAR
    wRAR committed May 22, 2024
    Configuration menu
    Copy the full SHA
    534a66e View commit details
    Browse the repository at this point in the history

  3. Help with building lxml on 3.13beta1.

    @wRAR
    wRAR committed May 22, 2024
    Configuration menu
    Copy the full SHA
    b6d3d90 View commit details
    Browse the repository at this point in the history

  4. Skip zstandard on 3.13.

    @wRAR
    wRAR committed May 22, 2024
    Configuration menu
    Copy the full SHA
    04bc1e6 View commit details
    Browse the repository at this point in the history

  5. docs: Remove top-level reactor imports from CrawlerProces/CrawlerRunn… 

    …er examples
    @Laerte
    Laerte committed May 22, 2024
    Configuration menu
    Copy the full SHA
    e676cd3 View commit details
    Browse the repository at this point in the history

  6. Update docs/topics/practices.rst 

    Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
    @Laerte @wRAR
    Laerte and wRAR authored May 22, 2024
    Configuration menu
    Copy the full SHA
    8210fae View commit details
    Browse the repository at this point in the history

  7. Update docs/topics/practices.rst 

    Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
    @Laerte @wRAR
    Laerte and wRAR authored May 22, 2024
    Configuration menu
    Copy the full SHA
    dc6a495 View commit details
    Browse the repository at this point in the history

Commits on May 23, 2024

  1. fix: checks

    @Laerte
    Laerte committed May 23, 2024
    Configuration menu
    Copy the full SHA
    3f66b66 View commit details
    Browse the repository at this point in the history

  2. Update tests-macos.yml

    @Laerte
    Laerte authored May 23, 2024
    Configuration menu
    Copy the full SHA
    e143dc7 View commit details
    Browse the repository at this point in the history

  3. Retrigger CI

    @Laerte
    Laerte authored May 23, 2024
    Configuration menu
    Copy the full SHA
    9d5a0d2 View commit details
    Browse the repository at this point in the history

  4. Update docs/topics/practices.rst 

    Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
    @Laerte @wRAR
    Laerte and wRAR authored May 23, 2024
    Configuration menu
    Copy the full SHA
    17e623c View commit details
    Browse the repository at this point in the history

  5. Update docs/topics/practices.rst 

    Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
    @Laerte @wRAR
    Laerte and wRAR authored May 23, 2024
    Configuration menu
    Copy the full SHA
    8ec67ca View commit details
    Browse the repository at this point in the history

  6. Update docs/topics/practices.rst 

    Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
    @Laerte @wRAR
    Laerte and wRAR authored May 23, 2024
    Configuration menu
    Copy the full SHA
    62c89aa View commit details
    Browse the repository at this point in the history

Commits on May 26, 2024

  1. Add change reactor test to CrawlerRunner

    @Laerte
    Laerte committed May 26, 2024
    Configuration menu
    Copy the full SHA
    2facdd4 View commit details
    Browse the repository at this point in the history

  2. Move path

    @Laerte
    Laerte committed May 26, 2024
    Configuration menu
    Copy the full SHA
    6cd0857 View commit details
    Browse the repository at this point in the history

Commits on May 27, 2024

  1. Merge pull request #6374 from Laerte/master 

    docs: Remove top-level reactor imports from CrawlerProces/CrawlerRunner examples
    @wRAR
    wRAR authored May 27, 2024
    Configuration menu
    Copy the full SHA
    f9a9860 View commit details
    Browse the repository at this point in the history

Commits on May 28, 2024

  1. Install typing stubs for boto3 and botocore. (#6370)

    @wRAR
    wRAR authored May 28, 2024
    Configuration menu
    Copy the full SHA
    9ba4dd3 View commit details
    Browse the repository at this point in the history

  2. Move CI from the decommissioned macos-11 to macos-latest. (#6372)

    @wRAR
    wRAR authored May 28, 2024
    Configuration menu
    Copy the full SHA
    986d1ee View commit details
    Browse the repository at this point in the history

  3. Fix overridable methods in MediaPipeline (#6368)

    @kumar-sanchay
    kumar-sanchay authored May 28, 2024
    Configuration menu
    Copy the full SHA
    cadb0dd View commit details
    Browse the repository at this point in the history

  4. Bump twinecheck deps.

    @wRAR
    wRAR committed May 28, 2024
    Configuration menu
    Copy the full SHA
    d9b5538 View commit details
    Browse the repository at this point in the history

  5. Install pre-release cffi on 3.13.

    @wRAR
    wRAR committed May 28, 2024
    Configuration menu
    Copy the full SHA
    42347de View commit details
    Browse the repository at this point in the history

  6. Skip mitmproxy and Pillow on 3.13.

    @wRAR
    wRAR committed May 28, 2024
    Configuration menu
    Copy the full SHA
    e6e9fd7 View commit details
    Browse the repository at this point in the history

  7. Help with building lxml on 3.13beta1 for checks too.

    @wRAR
    wRAR committed May 28, 2024
    Configuration menu
    Copy the full SHA
    5755e22 View commit details
    Browse the repository at this point in the history

  8. Skip uvloop and bpython on 3.13.

    @wRAR
    wRAR committed May 28, 2024
    Configuration menu
    Copy the full SHA
    1be8aee View commit details
    Browse the repository at this point in the history

Commits on May 29, 2024

  1. Add support for multiple referer policy tokens.

    Fabian Schneebauer committed May 29, 2024
    Configuration menu
    Copy the full SHA
    0d58af8 View commit details
    Browse the repository at this point in the history

  2. Add spec link to scrapy/spidermiddlewares/referer.py 

    Co-authored-by: Adrián Chaves <adrian@chaves.io>
    @0xdeb @Gallaecio
    0xdeb and Gallaecio authored May 29, 2024
    Configuration menu
    Copy the full SHA
    62a028b View commit details
    Browse the repository at this point in the history

  3. Merge pull request #6381 from 0xdeb/referer-policy-tokens 

    Add support for multiple referer policy tokens.
    @wRAR
    wRAR authored May 29, 2024
    Configuration menu
    Copy the full SHA
    469e8a2 View commit details
    Browse the repository at this point in the history

Commits on May 30, 2024

  1. Merge pull request #6371 from wRAR/typing-contracts 

    Full typing for scrapy/contracts.
    @wRAR
    wRAR authored May 30, 2024
    Configuration menu
    Copy the full SHA
    a011fa6 View commit details
    Browse the repository at this point in the history

Commits on May 31, 2024

  1. Misc typing improvements. (#6384)

    @wRAR
    wRAR authored May 31, 2024
    Configuration menu
    Copy the full SHA
    b4293e8 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into typing-request-response-cls

    @wRAR
    wRAR authored May 31, 2024
    Configuration menu
    Copy the full SHA
    b950ed7 View commit details
    Browse the repository at this point in the history

  3. Add parameters to typing.Dict.

    @wRAR
    wRAR committed May 31, 2024
    Configuration menu
    Copy the full SHA
    da42e8f View commit details
    Browse the repository at this point in the history

  4. Add parameters to typing.List.

    @wRAR
    wRAR committed May 31, 2024
    Configuration menu
    Copy the full SHA
    98c755e View commit details
    Browse the repository at this point in the history

  5. Add parameters to typing.Tuple.

    @wRAR
    wRAR committed May 31, 2024
    Configuration menu
    Copy the full SHA
    4164e63 View commit details
    Browse the repository at this point in the history

  6. Add parameters to typing.IO.

    @wRAR
    wRAR committed May 31, 2024
    Configuration menu
    Copy the full SHA
    70c56fa View commit details
    Browse the repository at this point in the history

  7. Add parameters to misc generics.

    @wRAR
    wRAR committed May 31, 2024
    Configuration menu
    Copy the full SHA
    751c91e View commit details
    Browse the repository at this point in the history

  8. Use a TypedDict for the verbose cookie form.

    @wRAR
    wRAR committed May 31, 2024
    Configuration menu
    Copy the full SHA
    859a77e View commit details
    Browse the repository at this point in the history

Commits on Jun 2, 2024

  1. Add parameters to some of typing.Callable.

    @wRAR
    wRAR committed Jun 2, 2024
    Configuration menu
    Copy the full SHA
    019f23e View commit details
    Browse the repository at this point in the history

Commits on Jun 3, 2024

  1. Don't run callbacks of requests from get_media_requests(). (#6386)

    @wRAR
    wRAR authored Jun 3, 2024
    Configuration menu
    Copy the full SHA
    492c3bc View commit details
    Browse the repository at this point in the history

  2. Merge pull request #6385 from wRAR/typing-generics-collections 

    Add parameters to various generics.
    @Gallaecio
    Gallaecio authored Jun 3, 2024
    Configuration menu
    Copy the full SHA
    2b9e32f View commit details
    Browse the repository at this point in the history

Commits on Jun 5, 2024

  1. Full typing for scrapy/pipelines. (#6387)

    @wRAR
    wRAR authored Jun 5, 2024
    Configuration menu
    Copy the full SHA
    e56b425 View commit details
    Browse the repository at this point in the history

  2. Handle AttributeError: 'NoneType' in contract parsing (#6388)

    @suvanbanerjee
    suvanbanerjee authored Jun 5, 2024
    Configuration menu
    Copy the full SHA
    3f76853 View commit details
    Browse the repository at this point in the history

  3. Add parameters to iterable generics, replace generators with iterables.

    @wRAR
    wRAR committed Jun 5, 2024
    Configuration menu
    Copy the full SHA
    2e21421 View commit details
    Browse the repository at this point in the history

  4. Bump typing deps.

    @wRAR
    wRAR committed Jun 5, 2024
    Configuration menu
    Copy the full SHA
    de146ad View commit details
    Browse the repository at this point in the history

  5. Use typing.Coroutine instead of types.CoroutineType.

    @wRAR
    wRAR committed Jun 5, 2024
    Configuration menu
    Copy the full SHA
    262c10d View commit details
    Browse the repository at this point in the history

  6. Add mssing __future__ imports.

    @wRAR
    wRAR committed Jun 5, 2024
    Configuration menu
    Copy the full SHA
    480a11b View commit details
    Browse the repository at this point in the history

  7. Add pyupgrade.

    @wRAR
    wRAR committed Jun 5, 2024
    Configuration menu
    Copy the full SHA
    feb0b8f View commit details
    Browse the repository at this point in the history

Commits on Jun 6, 2024

  1. Document missing parts of response.json method

    @Laerte
    Laerte committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    144ff6c View commit details
    Browse the repository at this point in the history

  2. Merge pull request #6394 from Laerte/master 

    Document missing parts of response.json method
    @wRAR
    wRAR authored Jun 6, 2024
    Configuration menu
    Copy the full SHA
    d9d7bd1 View commit details
    Browse the repository at this point in the history

  3. Add a comment about pyupgrade and scrapy/__init__.py.

    @wRAR
    wRAR committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    23b1214 View commit details
    Browse the repository at this point in the history

  4. Merge pull request #6391 from wRAR/typing-generics-iterables 

    Add parameters to iterable generics, replace generators with iterables.
    @wRAR
    wRAR authored Jun 6, 2024
    Configuration menu
    Copy the full SHA
    b24ecca View commit details
    Browse the repository at this point in the history

  5. Merge branch 'master' into typing-request-response-cls

    @wRAR
    wRAR committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    1268b23 View commit details
    Browse the repository at this point in the history

  6. Merge pull request #6143 from wRAR/typing-request-response-cls 

    Improve type hints for copy() and replace() in Request and Response.
    @wRAR
    wRAR authored Jun 6, 2024
    Configuration menu
    Copy the full SHA
    a4778d2 View commit details
    Browse the repository at this point in the history

Commits on Jun 7, 2024

  1. Remove the Python version check from scrapy/__init__.py.

    @wRAR
    wRAR committed Jun 7, 2024
    Configuration menu
    Copy the full SHA
    ed3a7ac View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into pyupgrade

    @wRAR
    wRAR committed Jun 7, 2024
    Configuration menu
    Copy the full SHA
    5850b8f View commit details
    Browse the repository at this point in the history

  3. Merge pull request #6392 from wRAR/pyupgrade 

    Add pyupgrade.
    @wRAR
    wRAR authored Jun 7, 2024
    Configuration menu
    Copy the full SHA
    163e7d9 View commit details
    Browse the repository at this point in the history

Commits on Jun 10, 2024

  1. Deprecate scrapy.utils.defer.process_chain_both(). (#6397)

    @wRAR
    wRAR authored Jun 10, 2024
    Configuration menu
    Copy the full SHA
    ddc98fe View commit details
    Browse the repository at this point in the history

  2. Add parameters to most Deferred in scrapy/core. (#6395)

    @wRAR
    wRAR authored Jun 10, 2024
    Configuration menu
    Copy the full SHA
    1282ddf View commit details
    Browse the repository at this point in the history

Commits on Jun 17, 2024

  1. Removing empty example reference (#6402) 

    Co-authored-by: Michael Duane Mooring <mikeumus@gmail.com>
    @wRAR @mikeumus
    wRAR and mikeumus authored Jun 17, 2024
    Configuration menu
    Copy the full SHA
    365c9e6 View commit details
    Browse the repository at this point in the history

  2. Unpin markupsafe in extra-deps. (#6403)

    @wRAR
    wRAR authored Jun 17, 2024
    Configuration menu
    Copy the full SHA
    a364560 View commit details
    Browse the repository at this point in the history

Commits on Jun 19, 2024

  1. flake8-debugger

    @wRAR
    wRAR committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    d132190 View commit details
    Browse the repository at this point in the history

  2. flake8-string-format

    @wRAR
    wRAR committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    a617e04 View commit details
    Browse the repository at this point in the history

  3. flake8-comprehensions

    @wRAR
    wRAR committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    1c70d3e View commit details
    Browse the repository at this point in the history

  4. flake8-docstrings

    @wRAR
    wRAR committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    1ef9c33 View commit details
    Browse the repository at this point in the history

  5. flake8-bugbear

    @wRAR
    wRAR committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    3d8dbd5 View commit details
    Browse the repository at this point in the history

  6. Split ignores into blocks.

    @wRAR
    wRAR committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    13d3b1a View commit details
    Browse the repository at this point in the history

Commits on Jun 20, 2024

  1. Merge pull request #6406 from wRAR/more-flake8 

    Add some flake8 plugins
    @wRAR
    wRAR authored Jun 20, 2024
    Configuration menu
    Copy the full SHA
    0e78ac6 View commit details
    Browse the repository at this point in the history

Commits on Jun 21, 2024

  1. Apply grammar fixes (#6411)

    @mlmsmith
    mlmsmith authored Jun 21, 2024
    Configuration menu
    Copy the full SHA
    326e323 View commit details
    Browse the repository at this point in the history

Commits on Jun 25, 2024

  1. Add flake8-type-checking. (#6413)

    @wRAR
    wRAR authored Jun 25, 2024
    Configuration menu
    Copy the full SHA
    d08f559 View commit details
    Browse the repository at this point in the history

Commits on Jun 26, 2024

  1. Add parameteres to most Deferred instances. (#6414)

    @wRAR
    wRAR authored Jun 26, 2024
    Configuration menu
    Copy the full SHA
    e47110f View commit details
    Browse the repository at this point in the history

  2. Bump twine in twinecheck. (#6416)

    @wRAR
    wRAR authored Jun 26, 2024
    Configuration menu
    Copy the full SHA
    96d6519 View commit details
    Browse the repository at this point in the history

  3. Remove an obsolete import. (#6415)

    @wRAR
    wRAR authored Jun 26, 2024
    Configuration menu
    Copy the full SHA
    41e15e9 View commit details
    Browse the repository at this point in the history

Commits on Jul 1, 2024

  1. Use CallbackT for Request.callback. (#6422)

    @wRAR
    wRAR authored Jul 1, 2024
    Configuration menu
    Copy the full SHA
    558b1d1 View commit details
    Browse the repository at this point in the history

Commits on Jul 4, 2024

  1. Documentation improvements (#6429)

    @mlmsmith
    mlmsmith authored Jul 4, 2024
    Configuration menu
    Copy the full SHA
    d8ecd28 View commit details
    Browse the repository at this point in the history

Commits on Jul 9, 2024

  1. Remove top-level imports that install the reactor from scrapy.extensi… 

    …ons.telnet. (#6432)
    @wRAR
    wRAR authored Jul 9, 2024
    Configuration menu
    Copy the full SHA
    ceedb02 View commit details
    Browse the repository at this point in the history

  2. add CLOSESPIDER_PAGECOUNT_NO_ITEM to CloseSpider extension

    @guillermo-bondonno
    guillermo-bondonno committed Jul 9, 2024
    Configuration menu
    Copy the full SHA
    0b8604b View commit details
    Browse the repository at this point in the history

  3. restore mitmproxy-dhparam

    @guillermo-bondonno
    guillermo-bondonno committed Jul 9, 2024
    Configuration menu
    Copy the full SHA
    a44818a View commit details
    Browse the repository at this point in the history

Commits on Jul 10, 2024

  1. refactor changes on closespider.py and improve test

    @guillermo-bondonno
    guillermo-bondonno committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    d6352f9 View commit details
    Browse the repository at this point in the history

  2. update docs

    @guillermo-bondonno
    guillermo-bondonno committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    59782d7 View commit details
    Browse the repository at this point in the history

  3. fix docs detail

    @guillermo-bondonno
    guillermo-bondonno committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    8646d2e View commit details
    Browse the repository at this point in the history

Commits on Jul 11, 2024

  1. Bump tool versions.

    @wRAR
    wRAR committed Jul 11, 2024
    Configuration menu
    Copy the full SHA
    129dbfa View commit details
    Browse the repository at this point in the history

  2. Bump the Python version for RTD.

    @wRAR
    wRAR committed Jul 11, 2024
    Configuration menu
    Copy the full SHA
    4356868 View commit details
    Browse the repository at this point in the history

  3. Bump pylint.

    @wRAR
    wRAR committed Jul 11, 2024
    Configuration menu
    Copy the full SHA
    3c9c1a3 View commit details
    Browse the repository at this point in the history

  4. Merge pull request #6435 from wRAR/update-tools 

    Update tool versions
    @Gallaecio
    Gallaecio authored Jul 11, 2024
    Configuration menu
    Copy the full SHA
    7a0a34b View commit details
    Browse the repository at this point in the history

  5. improve test_closespider_pagecount_no_item and MaxItemsSpider

    @guillermo-bondonno
    guillermo-bondonno committed Jul 11, 2024
    Configuration menu
    Copy the full SHA
    a40d528 View commit details
    Browse the repository at this point in the history

  6. improve test_closespider_pagecount_no_item and corresponding test spider

    @guillermo-bondonno
    guillermo-bondonno committed Jul 11, 2024
    Configuration menu
    Copy the full SHA
    5f0fad1 View commit details
    Browse the repository at this point in the history

  7. add test_closespider_pagecount_no_item_with_pagecount

    @guillermo-bondonno
    guillermo-bondonno committed Jul 11, 2024
    Configuration menu
    Copy the full SHA
    9cdbcb4 View commit details
    Browse the repository at this point in the history

Commits on Jul 12, 2024

  1. clean closespider_pagecount_no_item docs section 

    Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
    @guillermo-bondonno @wRAR
    guillermo-bondonno and wRAR authored Jul 12, 2024
    Configuration menu
    Copy the full SHA
    026d606 View commit details
    Browse the repository at this point in the history

  2. Merge pull request #6434 from guillermo-bondonno/feature/closespider_… 

    …pagecount_no_item

Feature/closespider pagecount no item
    @wRAR
    wRAR authored Jul 12, 2024
    Configuration menu
    Copy the full SHA
    06f9c28 View commit details
    Browse the repository at this point in the history

Commits on Jul 24, 2024

  1. Tutorial edits (#6440)

    @mlmsmith
    mlmsmith authored Jul 24, 2024
    Configuration menu
    Copy the full SHA
    e376c0b View commit details
    Browse the repository at this point in the history

Commits on Jul 31, 2024

  1. Do not suggest logging dropped items twice (#6448) 

    Co-authored-by: Kevin Lloyd Bernal <kevinoxy@gmail.com>
    @Gallaecio @BurnzZ
    Gallaecio and BurnzZ authored Jul 31, 2024
    Configuration menu
    Copy the full SHA
    03a15ce View commit details
    Browse the repository at this point in the history

Commits on Aug 1, 2024

  1. Proofread the commands documentation (#6449)

    @mlmsmith
    mlmsmith authored Aug 1, 2024
    Configuration menu
    Copy the full SHA
    b9ef132 View commit details
    Browse the repository at this point in the history

Commits on Aug 11, 2024

  1. Merge branch 'master' into py313

    @wRAR
    wRAR committed Aug 11, 2024
    Configuration menu
    Copy the full SHA
    1e68d3c View commit details
    Browse the repository at this point in the history

  2. Revert cffi and Pillow restrictions.

    @wRAR
    wRAR committed Aug 11, 2024
    Configuration menu
    Copy the full SHA
    70756fd View commit details
    Browse the repository at this point in the history

Commits on Aug 19, 2024

  1. minor changes to items section of docs (#6462)

    @mlmsmith
    mlmsmith authored Aug 19, 2024
    Configuration menu
    Copy the full SHA
    af15bd1 View commit details
    Browse the repository at this point in the history

Commits on Aug 20, 2024

  1. Revert uvloop restrictions.

    @wRAR
    wRAR committed Aug 20, 2024
    Configuration menu
    Copy the full SHA
    c21c4a1 View commit details
    Browse the repository at this point in the history

Commits on Aug 23, 2024

  1. Typing fixes and updates. (#6460)

    @wRAR
    wRAR authored Aug 23, 2024
    Configuration menu
    Copy the full SHA
    5794071 View commit details
    Browse the repository at this point in the history

Commits on Aug 26, 2024

  1. Allow yielding items from start_requests (#6417) 

    Co-authored-by: Georgiy Zatserklianyi <george.zatseklyany@gmail.com>
Co-authored-by: Adrián Chaves <adrian@chaves.io>
Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
    @GeorgeA92 @Gallaecio @wRAR
    4 people authored Aug 26, 2024
    Configuration menu
    Copy the full SHA
    6ce0342 View commit details
    Browse the repository at this point in the history

Commits on Aug 27, 2024

  1. Add support for meta in Spider Contracts

    @Laerte
    Laerte committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    b85e5a6 View commit details
    Browse the repository at this point in the history

  2. Update docs/topics/contracts.rst

    @Laerte
    Laerte authored Aug 27, 2024
    Configuration menu
    Copy the full SHA
    f68f29d View commit details
    Browse the repository at this point in the history

  3. update docs

    @Laerte
    Laerte committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    3c2a9fa View commit details
    Browse the repository at this point in the history

  4. Merge branch 'master' of github.com:Laerte/scrapy

    @Laerte
    Laerte committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    4f9b234 View commit details
    Browse the repository at this point in the history

  5. Update scrapy/contracts/default.py

    @Laerte
    Laerte authored Aug 27, 2024
    Configuration menu
    Copy the full SHA
    ddbdfeb View commit details
    Browse the repository at this point in the history

Commits on Aug 28, 2024

  1. Merge pull request #6469 from Laerte/master 

    Add support for meta in Spider Contracts
    @wRAR
    wRAR authored Aug 28, 2024
    Configuration menu
    Copy the full SHA
    c9d85fa View commit details
    Browse the repository at this point in the history

  2. Refactor genspider slightly so template variables can be overridden (#… 

    …6470)
    @CloCkWeRX
    CloCkWeRX authored Aug 28, 2024
    Configuration menu
    Copy the full SHA
    67ab8d4 View commit details
    Browse the repository at this point in the history

Commits on Sep 9, 2024

  1. Remove debug log message from _schedule_request method

    @LucasSD
    LucasSD committed Sep 9, 2024
    Configuration menu
    Copy the full SHA
    f260f81 View commit details
    Browse the repository at this point in the history

Commits on Sep 10, 2024

  1. Merge pull request #6475 from LucasSD/remove-debug-log-message 

    Remove log pollution from core.engine signal handler (#6433)
    @wRAR
    wRAR authored Sep 10, 2024
    Configuration menu
    Copy the full SHA
    ae967d1 View commit details
    Browse the repository at this point in the history

Commits on Sep 12, 2024

  1. Revert "Revert uvloop restrictions." 

    This reverts commit c21c4a1.
    @wRAR
    wRAR committed Sep 12, 2024
    Configuration menu
    Copy the full SHA
    b3f562d View commit details
    Browse the repository at this point in the history

  2. Revert bpython restrictions.

    @wRAR
    wRAR committed Sep 12, 2024
    Configuration menu
    Copy the full SHA
    ee9ee2d View commit details
    Browse the repository at this point in the history

Commits on Sep 20, 2024

  1. Fix expectations for get_func_args() on 3.13.

    @wRAR
    wRAR committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    e139d22 View commit details
    Browse the repository at this point in the history

Commits on Oct 2, 2024

  1. Ignore SyntaxError as well when SPIDER_LOADER_WARN_ONLY is set to True ( 

    #6484)
    @mmoriniere
    mmoriniere authored Oct 2, 2024
    Configuration menu
    Copy the full SHA
    46cddc6 View commit details
    Browse the repository at this point in the history

Commits on Oct 7, 2024

  1. Remove the installation dependency on setuptools.

    @wRAR
    wRAR committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    8c133fc View commit details
    Browse the repository at this point in the history

Commits on Oct 8, 2024

  1. Use the 3.13 release.

    @wRAR
    wRAR committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    df6c51a View commit details
    Browse the repository at this point in the history

  2. Remove the beta block.

    @wRAR
    wRAR committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    29bb869 View commit details
    Browse the repository at this point in the history

  3. Don't use types-setuptools.

    @wRAR
    wRAR committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    87651fd View commit details
    Browse the repository at this point in the history

  4. Merge pull request #6487 from wRAR/drop-setuptools-dep 

    Remove the installation dependency on setuptools.
    @wRAR
    wRAR authored Oct 8, 2024
    Configuration menu
    Copy the full SHA
    7ec6b7e View commit details
    Browse the repository at this point in the history

Commits on Oct 9, 2024

  1. update gh-action-pypi-publish

    @rettinghaus
    rettinghaus authored Oct 9, 2024
    Configuration menu
    Copy the full SHA
    5ef5474 View commit details
    Browse the repository at this point in the history

Commits on Oct 10, 2024

  1. Merge pull request #6489 from rettinghaus/pypa_action 

    update gh-action-pypi-publish
    @wRAR
    wRAR authored Oct 10, 2024
    Configuration menu
    Copy the full SHA
    9736e49 View commit details
    Browse the repository at this point in the history

Commits on Oct 16, 2024

  1. Drop Python 3.8 Support (#6472)

    @vsevolodbreus
    vsevolodbreus authored Oct 16, 2024
    Configuration menu
    Copy the full SHA
    5391663 View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'origin/master' into py313

    @wRAR
    wRAR committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    845b1ff View commit details
    Browse the repository at this point in the history

  3. Revert uvloop restrictions.

    @wRAR
    wRAR committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    9bd5e5b View commit details
    Browse the repository at this point in the history

  4. Misc typing improvements. (#6494)

    @wRAR
    wRAR authored Oct 16, 2024
    Configuration menu
    Copy the full SHA
    f65e64a View commit details
    Browse the repository at this point in the history

Commits on Oct 17, 2024

  1. Fixed typos (#6497)

    @julian-st
    julian-st authored Oct 17, 2024
    Configuration menu
    Copy the full SHA
    c8e87ab View commit details
    Browse the repository at this point in the history

  2. Remove --keep-runtime-typing from pyupgrade.

    @wRAR
    wRAR committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    c9095ef View commit details
    Browse the repository at this point in the history

  3. Reorder unions with None.

    @wRAR
    wRAR committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    7196a11 View commit details
    Browse the repository at this point in the history

  4. Merge pull request #6498 from wRAR/remove-runtime-typing 

    Remove --keep-runtime-typing from pyupgrade.
    @wRAR
    wRAR authored Oct 17, 2024
    Configuration menu
    Copy the full SHA
    1138a5c View commit details
    Browse the repository at this point in the history

  5. Small 3.7 and 3.8 cleanup.

    @wRAR
    wRAR committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    7e07d48 View commit details
    Browse the repository at this point in the history

  6. Drop Reppy.

    @wRAR
    wRAR committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    5759b3f View commit details
    Browse the repository at this point in the history

  7. Remove dead links to the Reppy doc from the release notes.

    @wRAR
    wRAR committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    677e977 View commit details
    Browse the repository at this point in the history

  8. Merge pull request #6499 from wRAR/3.8-cleanup 

    Additional Python 3.7 and 3.8 cleanup, including Reppy support.
    @wRAR
    wRAR authored Oct 17, 2024
    Configuration menu
    Copy the full SHA
    6d65708 View commit details
    Browse the repository at this point in the history

Commits on Oct 21, 2024

  1. Filter test-time warnings. (#6501)

    @wRAR
    wRAR authored Oct 21, 2024
    Configuration menu
    Copy the full SHA
    04d0411 View commit details
    Browse the repository at this point in the history

Commits on Oct 22, 2024

  1. Merge remote-tracking branch 'origin/master' into py313

    @wRAR
    wRAR committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    fffacb9 View commit details
    Browse the repository at this point in the history

  2. Bump pyftpdlib to the version supporting Python 3.13 on Windows.

    @wRAR
    wRAR committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    d10c58f View commit details
    Browse the repository at this point in the history

  3. Merge pull request #6166 from Gallaecio/py313 

    Add Python 3.13 to CI
    @wRAR
    wRAR authored Oct 22, 2024
    Configuration menu
    Copy the full SHA
    b4bad97 View commit details
    Browse the repository at this point in the history

Commits on Oct 29, 2024

  1. Explictly set html_baseurl on RTD. (#6507)

    @wRAR
    wRAR authored Oct 29, 2024
    Configuration menu
    Copy the full SHA
    0523e16 View commit details
    Browse the repository at this point in the history

  2. Remove code for unsupported Twisted. (#6510)

    @wRAR
    wRAR authored Oct 29, 2024
    Configuration menu
    Copy the full SHA
    fcb5ab6 View commit details
    Browse the repository at this point in the history

  3. Updated deprecated ast.NameConstant with ast.Constant #6305 (#6463)

    @ThunderMind2019
    ThunderMind2019 authored Oct 29, 2024
    Configuration menu
    Copy the full SHA
    5bbf812 View commit details
    Browse the repository at this point in the history

  4. Fixes for Twisted Version Check and Typing Issues (#6511)

    @Rohitkr117
    Rohitkr117 authored Oct 29, 2024
    Configuration menu
    Copy the full SHA
    65ecd5d View commit details
    Browse the repository at this point in the history

  5. Added ignore statements for Windows specific typing issues (#6516)

    @Rohitkr117
    Rohitkr117 authored Oct 29, 2024
    Configuration menu
    Copy the full SHA
    12b087b View commit details
    Browse the repository at this point in the history

  6. Deprecate scrapy.twisted_version (#6512) 

    * Deprecate scrapy.twisted_version

* fix: typing

* remove typing

* raise default exception if attribute is not found

* remove redudant ()

* add tests

* rollback exception raised

* add filterwarnings again

* change order

* lint
    @Laerte
    Laerte authored Oct 29, 2024
    Configuration menu
    Copy the full SHA
    d2bdbad View commit details
    Browse the repository at this point in the history

Commits on Oct 31, 2024

  1. Deprecation removals. (#6500) 

    * Deprecation removals.

* Clean up the default pytest filterwarnings.

* Remove test_get_images_old().

* Redo boto-requiring test filtering.

* Remove an unused function.

* Improve the Crawler.crawl() error message.

* Fix the test.
    @wRAR
    wRAR authored Oct 31, 2024
    Configuration menu
    Copy the full SHA
    d85c39f View commit details
    Browse the repository at this point in the history

Commits on Nov 2, 2024

  1. Documentation added for Spider State in extensions.rst (#6522) 

    * Documentation added for Spider State in extensions.rst

* Made correction in documentation for Spiderstate

* Added appropriate intro for Spider state extension

* Added reference for spiderstate extension

* Added Spiderstate extension hyperlink refrence in jobs.rst
    @Rohitkr117
    Rohitkr117 authored Nov 2, 2024
    Configuration menu
    Copy the full SHA
    7701e59 View commit details
    Browse the repository at this point in the history

Commits on Nov 4, 2024

  1. Run and fix linkcheck. (#6524)

    @wRAR
    wRAR authored Nov 4, 2024
    Configuration menu
    Copy the full SHA
    ce5a132 View commit details
    Browse the repository at this point in the history

  2. Update the outdated Item docstring. (#6427) 

    * Update the outdated Item doscstring.

* Fix the reference links in items.html.
    @wRAR
    wRAR authored Nov 4, 2024
    Configuration menu
    Copy the full SHA
    e7f5ae0 View commit details
    Browse the repository at this point in the history

  3. Deprecate unused scrapy utils (#6519) 

    * Added deprecation warnings for unused Scrapy.utils

* Grammatical corrections

* Exceptions class connected

* Deprecation of ScrapyJSONDecoder

* request_authenticate function deprecation

* Making all warning similar

* Added ignore statements for deprecation warning in tests

* Missing stacklevel attr. added

* Added Deprecation message
    @Rohitkr117
    Rohitkr117 authored Nov 4, 2024
    Configuration menu
    Copy the full SHA
    d215669 View commit details
    Browse the repository at this point in the history

Commits on Nov 5, 2024

  1. Replace Slot.throttle with Request.meta['dont_throttle']

    @Gallaecio
    Gallaecio committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    f57fc45 View commit details
    Browse the repository at this point in the history

  2. Fix docs example

    @Gallaecio
    Gallaecio committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    5862216 View commit details
    Browse the repository at this point in the history

  3. Add the missing import to the docs example

    @Gallaecio
    Gallaecio committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    b244ea7 View commit details
    Browse the repository at this point in the history

Commits on Nov 7, 2024

  1. dont_throttle → autothrottle_dont_adjust_delay

    @Gallaecio
    Gallaecio committed Nov 7, 2024
    Configuration menu
    Copy the full SHA
    2a4b7fe View commit details
    Browse the repository at this point in the history

  2. Refactor the docs

    @Gallaecio
    Gallaecio committed Nov 7, 2024
    Configuration menu
    Copy the full SHA
    dc3ebb6 View commit details
    Browse the repository at this point in the history

Commits on Nov 8, 2024

  1. Merge pull request #6527 from Gallaecio/dont-throttle-meta 

    Replace Slot.throttle with Request.meta['dont_throttle']
    @kmike
    kmike authored Nov 8, 2024
    Configuration menu
    Copy the full SHA
    bcef965 View commit details
    Browse the repository at this point in the history

Commits on Nov 11, 2024

  1. Address some previously ignored pylint messages. (#6531)

    @wRAR
    wRAR authored Nov 11, 2024
    Configuration menu
    Copy the full SHA
    b042ad2 View commit details
    Browse the repository at this point in the history

  2. Remove build_from_settings().

    @wRAR
    wRAR committed Nov 11, 2024
    Configuration menu
    Copy the full SHA
    eda3a89 View commit details
    Browse the repository at this point in the history

  3. Deprecate scrapy.extensions.feedexport.build_storage() and simplify _… 

    …get_storage().
    @wRAR
    wRAR committed Nov 11, 2024
    Configuration menu
    Copy the full SHA
    83d4939 View commit details
    Browse the repository at this point in the history

Commits on Nov 12, 2024

  1. Add tests for MediaPipeline.from_crawler() and related code.

    @wRAR
    wRAR committed Nov 12, 2024
    Configuration menu
    Copy the full SHA
    f796d87 View commit details
    Browse the repository at this point in the history

  2. Add from_crawler() to components that only had from_settings().

    @wRAR
    wRAR committed Nov 12, 2024
    Configuration menu
    Copy the full SHA
    499e7e8 View commit details
    Browse the repository at this point in the history

  3. Deprecate MiddlewareManager.from_settings().

    @wRAR
    wRAR committed Nov 12, 2024
    Configuration menu
    Copy the full SHA
    eda1a8a View commit details
    Browse the repository at this point in the history

  4. Deprecate build_from_crawler() calling from_settings().

    @wRAR
    wRAR committed Nov 12, 2024
    Configuration menu
    Copy the full SHA
    8700a5b View commit details
    Browse the repository at this point in the history

Commits on Nov 13, 2024

  1. Enhancement: Update docs to include IgnoreRequest details (#6506) 

    Co-authored-by: Adrián Chaves <adrian@chaves.io>
    @RJPalmer @Gallaecio
    RJPalmer and Gallaecio authored Nov 13, 2024
    Configuration menu
    Copy the full SHA
    261c4b6 View commit details
    Browse the repository at this point in the history

  2. Modernize the media pipeline initialization API.

    @wRAR
    wRAR committed Nov 13, 2024
    Configuration menu
    Copy the full SHA
    28fafbb View commit details
    Browse the repository at this point in the history

  3. Prefer crawler.settings over settings in media pipelines.

    @wRAR
    wRAR committed Nov 13, 2024
    Configuration menu
    Copy the full SHA
    6aa4d2b View commit details
    Browse the repository at this point in the history

Commits on Nov 14, 2024

  1. Extract duplicated code.

    @wRAR
    wRAR committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    2ad5f0c View commit details
    Browse the repository at this point in the history

  2. Address PR feedback.

    @wRAR
    wRAR committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    929d665 View commit details
    Browse the repository at this point in the history

  3. Merge pull request #6540 from wRAR/build_from_settings 

    Remove build_from_settings() and deprecate from_settings() code
    @wRAR
    wRAR authored Nov 14, 2024
    Configuration menu
    Copy the full SHA
    ab5cb7c View commit details
    Browse the repository at this point in the history

  4. Added failing test cases to tests/test_contracts.py and fixed corresp… 

    …onding methods + removed pylint comments
    @nicklaustrup
    nicklaustrup committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    bfcee45 View commit details
    Browse the repository at this point in the history

  5. Merge pull request #6542 from nicklaustrup/Tests-contracts-swallowed-… 

    …exceptions

Fix - return in finally can swallow exception (#6505)
    @wRAR
    wRAR authored Nov 14, 2024
    Configuration menu
    Copy the full SHA
    30fb54f View commit details
    Browse the repository at this point in the history

Commits on Nov 15, 2024

  1. Verified PyPI releases (a.k.a. PEP740)

    @dangra
    dangra committed Nov 15, 2024
    Configuration menu
    Copy the full SHA
    dc4d6d1 View commit details
    Browse the repository at this point in the history

  2. Merge pull request #6543 from scrapy/pep740-attestations 

    Verified PyPI releases (a.k.a. PEP740)
    @wRAR
    wRAR authored Nov 15, 2024
    Configuration menu
    Copy the full SHA
    87b2300 View commit details
    Browse the repository at this point in the history

  3. Commit mitmproxy-dhparam.pem.

    @wRAR
    wRAR committed Nov 15, 2024
    Configuration menu
    Copy the full SHA
    feea3a0 View commit details
    Browse the repository at this point in the history

  4. Merge pull request #6545 from wRAR/mitmproxy-dhparam 

    Commit mitmproxy-dhparam.pem.
    @wRAR
    wRAR authored Nov 15, 2024
    Configuration menu
    Copy the full SHA
    212e848 View commit details
    Browse the repository at this point in the history

Commits on Nov 18, 2024

  1. 2.12 release notes (#6226) 

    * Cover 2.12 in the release notes up to 9bb973d

* Add one more highlight

* Better merge of the news entries.

* Cover 2.12 in the release notes up to 642af40.

* Cover 2.12 in the release notes up to 7a0a34b.

* Cover 2.12 in the release notes up to b4bad97.

* Add not yet merged PRs #6463, #6507, #6511 to the 2.12 release notes.

* Cover 2.12 in the release notes up to d85c39f, small fixes.

* Cover 2.12 in the release notes up to d215669.

* Cover #6527 in the release notes.

* Address PR feedback.

* Cover recent PRs.

* Finalize the 2.12.0 release notes, small additional fixes.

---------

Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
    @Gallaecio @wRAR
    Gallaecio and wRAR authored Nov 18, 2024
    Configuration menu
    Copy the full SHA
    10089c6 View commit details
    Browse the repository at this point in the history

  2. Bump version: 2.11.2 → 2.12.0

    @wRAR
    wRAR committed Nov 18, 2024
    Configuration menu
    Copy the full SHA
    b1f9e56 View commit details
    Browse the repository at this point in the history

  3. Fix a typo that broke PDF builds.

    @wRAR
    wRAR committed Nov 18, 2024
    Configuration menu
    Copy the full SHA
    efb53aa View commit details
    Browse the repository at this point in the history

Commits on Nov 19, 2024

  1. Integrating configs into pyproject.toml (#6547)

    @swayam0322
    swayam0322 authored Nov 19, 2024
    Configuration menu
    Copy the full SHA
    8c23da9 View commit details
    Browse the repository at this point in the history

Commits on Nov 24, 2024

  1. Add tests for DOWNLOADER_CLIENT_TLS_METHOD, remove dead code.

    @wRAR
    wRAR committed Nov 24, 2024
    Configuration menu
    Copy the full SHA
    4dcc04b View commit details
    Browse the repository at this point in the history

Commits on Nov 25, 2024

  1. Merge pull request #6560 from wRAR/tls-method-tests 

    Add tests for DOWNLOADER_CLIENT_TLS_METHOD, remove dead code.
    @wRAR
    wRAR authored Nov 25, 2024
    Configuration menu
    Copy the full SHA
    37be9da View commit details
    Browse the repository at this point in the history

Commits on Dec 9, 2024

  1. Add ruff with basic rules. (#6565)

    @wRAR
    wRAR authored Dec 9, 2024
    Configuration menu
    Copy the full SHA
    cc146b9 View commit details
    Browse the repository at this point in the history

  2. Made path absolute to enable running pytest from a different director… 

    …y. (#6567)
    @emeryberger
    emeryberger authored Dec 9, 2024
    Configuration menu
    Copy the full SHA
    5680bee View commit details
    Browse the repository at this point in the history

  3. Add flake8-bugbear rules to ruff.

    @wRAR
    wRAR committed Dec 9, 2024
    Configuration menu
    Copy the full SHA
    c184f12 View commit details
    Browse the repository at this point in the history

  4. Add flake8-comprehensions and flake8-debugger rules to ruff.

    @wRAR
    wRAR committed Dec 9, 2024
    Configuration menu
    Copy the full SHA
    e53d6f0 View commit details
    Browse the repository at this point in the history

  5. Add pydocstyle/flake8-docstrings rules to ruff.

    @wRAR
    wRAR committed Dec 9, 2024
    Configuration menu
    Copy the full SHA
    d6bf146 View commit details
    Browse the repository at this point in the history

  6. Add flake8-type-checking rules to ruff.

    @wRAR
    wRAR committed Dec 9, 2024
    Configuration menu
    Copy the full SHA
    d47f142 View commit details
    Browse the repository at this point in the history

  7. Fix building docs.

    @wRAR
    wRAR committed Dec 9, 2024
    Configuration menu
    Copy the full SHA
    0d7a5e7 View commit details
    Browse the repository at this point in the history

Commits on Dec 10, 2024

  1. Remove flake8.

    @wRAR
    wRAR committed Dec 10, 2024
    Configuration menu
    Copy the full SHA
    ba30f64 View commit details
    Browse the repository at this point in the history

  2. Tidy up noqa comments.

    @wRAR
    wRAR committed Dec 10, 2024
    Configuration menu
    Copy the full SHA
    f4d8d6d View commit details
    Browse the repository at this point in the history

  3. Merge pull request #6576 from wRAR/ruff-rules-1 

    Add flake8 plugin rules to ruff
    @wRAR
    wRAR authored Dec 10, 2024
    Configuration menu
    Copy the full SHA
    b423e97 View commit details
    Browse the repository at this point in the history

  4. Ruff: migrate pyupgrade and bandit, enable some other rules (#6577)

    @wRAR
    wRAR authored Dec 10, 2024
    Configuration menu
    Copy the full SHA
    cde0845 View commit details
    Browse the repository at this point in the history