Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update scraped_page references when it's refreshed #1151

Merged
merged 1 commit into from
Nov 7, 2024

Conversation

wintonzheng
Copy link
Contributor

@wintonzheng wintonzheng commented Nov 7, 2024

Important

Update refresh() in scraper.py to update all relevant ScrapedPage attributes when refreshed, ensuring consistency with refreshed data.

  • Behavior:
    • Update refresh() in scraper.py to update all relevant attributes of ScrapedPage when refreshed.
    • Attributes updated include elements, id_to_css_dict, id_to_element_dict, id_to_frame_dict, id_to_element_hash, hash_to_element_ids, element_tree, element_tree_trimmed, screenshots, html, and extracted_text.
    • Ensures screenshots are retained if not available in refreshed data.

This description was created by Ellipsis for e2bc0ba. It will automatically update as commits are pushed.

Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Looks good to me! Reviewed everything up to e2bc0ba in 16 seconds

More details
  • Looked at 31 lines of code in 1 files
  • Skipped 0 files when reviewing.
  • Skipped posting 1 drafted comments based on config settings.
1. skyvern/webeye/scraper/scraper.py:244
  • Draft comment:
    Consider updating the private attributes (_browser_state, _clean_up_func, _scrape_exclude) in the refresh method to ensure consistency with the refreshed page.
  • Reason this comment was not posted:
    Comment did not seem useful.

Workflow ID: wflow_NGQO0HZr1ZuOMClc


You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

@wintonzheng wintonzheng merged commit c531f9b into main Nov 7, 2024
2 checks passed
@wintonzheng wintonzheng deleted the shu/ensure_scrape_before_data_extraction branch November 7, 2024 00:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant