Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pixiv Fanbox Downloader? #402

Closed
NHOrus opened this issue Aug 23, 2018 · 32 comments
Closed

Pixiv Fanbox Downloader? #402

NHOrus opened this issue Aug 23, 2018 · 32 comments

Comments

@NHOrus
Copy link
Contributor

NHOrus commented Aug 23, 2018

Possible option to get images from people whom I patron?

@Nandaka
Copy link
Owner

Nandaka commented Aug 24, 2018

I never use that feature, how does it works?

@NHOrus
Copy link
Contributor Author

NHOrus commented Aug 24, 2018

In simplest words, Patreon, but directly linked to pixiv account of all participants.

Do you need test pages?

@Nandaka
Copy link
Owner

Nandaka commented Aug 24, 2018

Checked one of the pixiv fanbox, I can get the ajax call for retriving the posts
https://www.pixiv.net/ajax/fanbox/creator?userId=648285
image

and the post details:
https://www.pixiv.net/ajax/fanbox/post?postId=123882
image

both with empty body element, because I'm not the patron.

Do you have samples for those messages?

@Nandaka
Copy link
Owner

Nandaka commented Aug 24, 2018

Got one with open post: https://www.pixiv.net/ajax/fanbox/post?postId=481
Looks like the body element is not structured, are you want to download the imageForShare or coverImageUrl?

@NHOrus
Copy link
Contributor Author

NHOrus commented Aug 24, 2018

originalUrl from body - type:image - body - images array
also, url from body - type:file - body - files array

Two examples are included.
post102135.txt
post132421.txt

@Nandaka
Copy link
Owner

Nandaka commented Aug 24, 2018

any referer need to be passed, or you can just download the file based on the current cookies value?

@NHOrus
Copy link
Contributor Author

NHOrus commented Aug 24, 2018

If tests are correct - 403 when not logged in, image from empty page with just URL when logged in, cookies are enough.

@NHOrus
Copy link
Contributor Author

NHOrus commented Aug 24, 2018

... That was poorly worded.
Using any link results in 403 when not logged in
Link provides image when logged in, even not from pixiv page.

So, cookies.
Any other way I can help?

@NHOrus
Copy link
Contributor Author

NHOrus commented Aug 25, 2018

Maybe release beta3 before this?

@NHOrus
Copy link
Contributor Author

NHOrus commented Aug 27, 2018

https://www.pixiv.net/ajax/fanbox/index is the start url for the process

Following URLs look like https://www.pixiv.net/ajax/fanbox/supporting_latest_post?maxPublishedDatetime=2018-07-06%2018%3A35%3A12&maxId=96862&limit=10 and https://www.pixiv.net/ajax/fanbox/supporting_latest_post?maxPublishedDatetime=2018-07-23%2022%3A17%3A59&maxId=109759&limit=10

Attaching JSON files that were returned to me.
index.txt
supporting_latest_post_1.txt
supporting_latest_post_2.txt

@NHOrus
Copy link
Contributor Author

NHOrus commented Aug 27, 2018

Alternatively, it could be better to get list of people supported from https://www.pixiv.net/ajax/fanbox/support to extract userID from, then iterate over all https://www.pixiv.net/ajax/fanbox/creator?userId=15521131 and get from nextUrl link in the form of https://www.pixiv.net/ajax/fanbox/creator/post?userId=15521131&maxPublishedDatetime=2018-05-01%2022%3A11%3A59&maxId=26783&limit=10 with more posts. Triple indirection, but works generically.

nextURL.txt
creator.txt
page.txt

@NHOrus
Copy link
Contributor Author

NHOrus commented Sep 27, 2018

Let me be overly direct. What I need to donate for you to consider working on this?

@Nandaka
Copy link
Owner

Nandaka commented Sep 28, 2018

Time? a bit busy with real works, so might not be possible for me to create new features.

@Nandaka
Copy link
Owner

Nandaka commented Oct 5, 2018

@NHOrus is it possible for you to send you username/password to my mail, so I can test the feature, as I don't follow any fanbox (or do they have 0 yen plan?)

@NHOrus
Copy link
Contributor Author

NHOrus commented Oct 5, 2018

Well, hope it works successfully.

Nandaka added a commit that referenced this issue Oct 5, 2018
Nandaka added a commit that referenced this issue Oct 5, 2018
@Nandaka
Copy link
Owner

Nandaka commented Oct 5, 2018

You can try this one.
pixivutil20181005-alpha1.zip

Please check the changelog.

@NHOrus
Copy link
Contributor Author

NHOrus commented Oct 5, 2018

It downloads one image from post when there's multiple. Because it tries to save them into same filename and overwrite is disabled.

@Nandaka
Copy link
Owner

Nandaka commented Oct 5, 2018

Check your filenamemangaformat in config.ini. It is used to generate the images filename.
https://github.com/Nandaka/PixivUtil2/blob/master/PixivUtil2.py#L1865

As for the cover image, it uses the filenameFormat.
https://github.com/Nandaka/PixivUtil2/blob/master/PixivUtil2.py#L1812

I'm using %urlFilename% to make it unique, else you can try the combination of %image_id% and %page_index% or %page_number%?

The given filename for the images are randomized from the API, fyi.

@Nandaka
Copy link
Owner

Nandaka commented Oct 5, 2018

also, I forgot to add the post id to the cover image, so for now it will shows only the original filename...

@NHOrus
Copy link
Contributor Author

NHOrus commented Oct 5, 2018

filenameformat = %member_id%\%image_id% - %title%
filenamemangaformat = %member_id%\%image_id% - %title%_%page_index%

I will try with %urlFilename%

@NHOrus
Copy link
Contributor Author

NHOrus commented Oct 5, 2018

Or with %page_number%, file names end with _

@NHOrus
Copy link
Contributor Author

NHOrus commented Oct 5, 2018

%page_number% and %page_index% don't work, only %urlFilename% does

@NHOrus
Copy link
Contributor Author

NHOrus commented Oct 5, 2018

with %title%_%urlFilename% I see stuff like 109759_p8_bvEkWxw2K8MXWTf9NWUtDGj7.png
Something got concatenated somewhere?

@Nandaka
Copy link
Owner

Nandaka commented Oct 5, 2018

I'm using %urlFilename% - %title% and it does append the title in the back.
image

109759_p8_bvEkWxw2K8MXWTf9NWUtDGj7.png is the %urlFilename% portion.

@NHOrus
Copy link
Contributor Author

NHOrus commented Oct 5, 2018

I completely misunderstood what %urlFilename% does, I expected only hash-looking thing for that.
Thank you.

Nandaka added a commit that referenced this issue Oct 5, 2018
@Nandaka
Copy link
Owner

Nandaka commented Oct 5, 2018

updated for cover filename, now it will include the post id, but without the _p.
pixivutil20181006-alpha2.zip

@NHOrus
Copy link
Contributor Author

NHOrus commented Oct 5, 2018

Will test it a bit later.

@NHOrus
Copy link
Contributor Author

NHOrus commented Oct 6, 2018

Well, it appears to be working correctly.

Thank you very much. If I encounter any bugs, I will open new issue.

@1pang2yik
Copy link

15884334814562571777248211318936
hey i have some trouble

@1pang2yik
Copy link

Error at doLogin(): (<class 'mechanize._response.httperror_seek_wrapper'>, <httperror_seek_wrapper (urllib2.HTTPError instance) at 0x33732d0 whose wrapped object = <closeable_response at 0x33b2bc0 whose fp = <response_seek_wrapper at 0x33b22d8 whose wrapped object = <closeable_response at 0x33b2670 whose fp = <socket._fileobject object at 0x033A7DF0>>>>>, <traceback object at 0x03361A58>)
Cannot Login!

@1pang2yik
Copy link

and if i remove cookies from config.ini it will show

Server reply: {u'captcha': u'Complete the reCAPTCHA verification'}
press enter to exit.

@Nandaka
Copy link
Owner

Nandaka commented May 3, 2020

see #505

35122 pushed a commit to 35122/PixivUtil2 that referenced this issue Oct 30, 2020
35122 pushed a commit to 35122/PixivUtil2 that referenced this issue Oct 30, 2020
35122 pushed a commit to 35122/PixivUtil2 that referenced this issue Oct 30, 2020
35122 pushed a commit to 35122/PixivUtil2 that referenced this issue Oct 30, 2020
35122 pushed a commit to 35122/PixivUtil2 that referenced this issue Oct 30, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants