-
Notifications
You must be signed in to change notification settings - Fork 357
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: Added get_public_url method to KeyValueStore #572
Conversation
This looks like the right approach. You should use the |
@janbuchar Thanks for the review and hint :) I have updated the PR with storage_dir |
src/crawlee/base_storage_client/_base_key_value_store_client.py
Outdated
Show resolved
Hide resolved
src/crawlee/base_storage_client/_base_key_value_store_client.py
Outdated
Show resolved
Hide resolved
Co-authored-by: Jan Buchar <Teyras@gmail.com>
Co-authored-by: Jan Buchar <Teyras@gmail.com>
src/crawlee/base_storage_client/_base_key_value_store_client.py
Outdated
Show resolved
Hide resolved
@akshay11298 sorry for the broken CI. I confirm that code quality checks are passing on my machine. |
Co-authored-by: Jan Buchar <Teyras@gmail.com>
@janbuchar @vdusek found the cause of the error. It was the slash used when creating the path manually. Used os.path.join to fix the error Edit: |
@janbuchar @vdusek Now the test passes. urlparse was returning netloc in windows and path in mac/linux |
src/crawlee/base_storage_client/_base_key_value_store_client.py
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, @akshay11298, for solving it. I added one more test, test_get_public_url_raises_for_non_existing_key,
and updated the method a bit so that it passes. LGTM.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Description
Issues
KeyValueStore.get_public_url
#514Testing
Checklist