Project alive?

[mvhconsult]

Last commit was 15 months ago. Are the maintainers still interested?

[joeharrison714]

I'm curious about this as well. Is there a newer preferred method for running spiders?

[ilias-ant]

+1 here! Would be great if the maintainers shared some thoughts on the future of this project!

p.s. All things considered, this is a great application!

[mxdev88]

I'm also interested by this. Would be great to get input from maintainers. Thanks.

[Princeofthebow]

I up this as well - any news on the matter? Alternatively what do suggest to be the best way to schedule spiders and crawls? The only valid alternative appears to be Scrapy Cloud

[Digenis]

I haven't been using scrapyd for some years now
and continuing development without actually using it
feels like taking decisions for others with nothing mine at stake.

Of course I can still provide input, ideas and comments
but no large scale testing.

This projects needs maintainers
but most of all it needs what I didn't have even back then:
more users with diverse setups
willing to try out alpha versions and give feedback.

[my8100]

@Digenis
Would you consider releasing a (beta) version at least containing PR #326 to support basic auth first?
I think it’s a significant enhancement in recent years.

[mxdev88]

This projects needs maintainers

How many maintainers does the project have today? Indeed, there are few outstansding PRs that would be interesting to have merged.

[Digenis]

@my8100,

#326 was followed by #337 which is still open.
From what I see in my comments,
you can merge it,
unless you have something to add.

1.3 has one last outstanding feature (#325)
so that would be an alpha release.
I think we can drop #140 from 1.3
and give the users full control, probably through #197
although I'm not sure we can do this for 1.3,
because #187 would need to be fixed too.

If there there are users willing to give feedback for 1.3-alpha
we can go on with it.
Then I estimate it would take 1-2 weeks to beta
and maybe 2-3 weeks to a release candidate.

Regarding 1.4, we'd postpone all its issues for 1.5
and use 1.4 only to bump and drop requirements' versions
because it's been some time since releasing and they are outdated.
Even if we target experimental support for py3.10 (twisted is onto it)
this would probably take a week or less.

---

@mxdev88,

Me and @my8100 have rights to the this repo and ticketing system
but zyte (F.K.A. scrapinghub) has full control of the github project
and the copyright to the scrapy name and logo.
The code is licensed under BSD 3-clause.

[jpmckinney]

Hi! I am a new maintainer.

I've copied parts of #389 (comment) to #364

I've reviewed the PRs and issues, answering and closing over 50 of them (they were mostly support requests).

I've reviewed and approved 2 PRs. I'll follow up in #364 (after my vacation) to get a minimal 1.3 alpha published.

Also, scrapyd-client is nearly ready for a release.

To avoid confusion, yes, the project is alive.

[azzamsa]

Hi.

Is Zyte still "maintaining" this repo?
I don't see any of Zyte's employees making a commit to this repo since late 2018.

From jpmckinney's LinkedIn, I don't think he is a Zyte employee too.
Is Scrapy Cloud is not based on this project?

I couldn't check Digenis info, so I can't make any assumption.

[Gallaecio]

Is Zyte still "maintaining" this repo?

Not actively, no. @pawelmhm and I are around in case we are needed, but @jpmckinney has effectively taken over, and has been doing a great job at it.

Is Scrapy Cloud is not based on this project?

It is not. I don’t think it has even been.

[azzamsa]

Thank you for the "official" clarification :)