Remove retriving bookmarks function

[YukihoAA]

for #1159

removing retriving bookmarks infomation to prevent ban.

it works fine for tags mode but idk why code get bookmarks seperatly before download...

[Nandaka]

It is to skip the post if the bookmark limit is defined when you download by tags. If you remove this code, then it will download all images when you set a limit.

If you want to skip this bookmark data retrieval, then just set the limit to 0/None from the console input.

https://github.com/Nandaka/PixivUtil2/blob/master/PixivUtil2.py#L366
https://github.com/Nandaka/PixivUtil2/blob/master/PixivUtil2.py#L371

[YukihoAA]

@Nandaka

bookmark skipping and grouping works fine for image mode because there is skipping function in PixivImageHandler.py L174

The reason why i had to remove this function, retriving bookmark data cause IP ban from pixiv, and we actually get bookmarkdata
in image processing so don't need to get image data before it.

it makes actually faster download because it skips getting bookmark data for posts which already in database

[Nandaka]

But if the image doesn't exists in the db, it still need to get the bookmark data. Only if you are registered premium user that have the data from the json response, I think?

PixivUtil2/PixivBrowserFactory.py

Line 832 in b723feb

# only premium support server-side filtering for bookmark count

fyi, bookmark data is empty from server

and it is only available when you load the page, which I think it is more expensive to run? and you still need to get the page anyway if you want to filter by bookmark count.

Maybe you can add a logic to skip this you are premium user? e.g. if use_bookmark_data and not self._isPremium:

PixivUtil2/PixivBrowserFactory.py

Line 867 in b723feb

if use_bookmark_data:

[YukihoAA]

I'm not using premium now and I'm using bookmark count skip feature.

pixivutil gets bookmark data who does not using premium.

It was an ideal codes to use bookmark count skipping without being IP ban. issue #1159
if i use 20220825 version, like if image skipped 120 in one time, pixiv will give me IP ban.

I tried to add 2 second wait to every loop for bookmark retriving function using wait(result, self.__config)
but does not worked properly and took too many times (because, it will double wait for images which will be downloaded) :(

so my Idea was just using functions in PixivImageHandler to make some delay to prevent ban.

so.. this code will use more CPU time but less Physical time... for wait()

this is not perfect solution for it so just for reference

[Nandaka]

can you confirm after running it for some time? because after the change, then it will always fetch the image page and then skipping it if the bookmark count is lower ==> which in turn, currently it will cause 429 error due to it request too many image page too often.

[YukihoAA]

In case v20220825: will got baned after 120~180 image skipped by retriving bookmarks with 5 sec delay

After edited: it worked fine (~2hr running) with 5 sec delay.

fine with 4 sec delay (tested 20min)

I run same tag and option that i used before so almost all images was skipped.

2 sec delay is not working both case.

[YukihoAA]

3sec is seems fine too (1hr test) - @Nandaka

[Nandaka]

sorry for the late reply, I was just back. Anyway, it is merged now