Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CSS Selector Bridge not finding links while browser does #3612

Closed
LarsStegman opened this issue Aug 13, 2023 · 3 comments
Closed

CSS Selector Bridge not finding links while browser does #3612

LarsStegman opened this issue Aug 13, 2023 · 3 comments
Labels
Bug-Report Confirmed bug report

Comments

@LarsStegman
Copy link
Contributor

Describe the bug
I want to use the CSS Selector Bridge to generate a feed from this https://www.deschakelbarendrecht.nl/search page, however the bridge is not able to find the elements with the links, while finding it in the browser works fine. I checked that the articles are not loaded asynchronously, they are already present if you download the page through curl.

To Reproduce
Steps to reproduce the behavior:

  1. Set up the bridge with these parameters: action=display&bridge=CssSelectorBridge&home_page=https://www.deschakelbarendrecht.nl/search&url_selector=a.block.relative&url_pattern=&content_selector=&content_cleanup=&title_cleanup=&limit=100&format=Html
  2. go to the generated feed
  3. The following error will be shown: No results for URL selector

Expected behaviour
A feed is generated with links to the articles

Additional context
I am running the tool in a docker container. The lack of debugging in the logs makes it very difficult to find the cause.

@LarsStegman LarsStegman added the Bug-Report Confirmed bug report label Aug 13, 2023
@dvikan
Copy link
Contributor

dvikan commented Aug 13, 2023

the url https://www.deschakelbarendrecht.nl/search actually doesnt contain the data you are looking for.

yes i checked.

@LarsStegman
Copy link
Contributor Author

Oh, that's a dumb mistake by me. I thought I did see the data when I requested it through curl. Strange. Thanks!

@LarsStegman
Copy link
Contributor Author

I think they require a cookie before they also send the contents, that's why it worked through curl before.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug-Report Confirmed bug report
Projects
None yet
Development

No branches or pull requests

2 participants