Migrate to use Apify client #6
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ig to be passed to Axios later
…jection, KVS output retrieval tested only string
…cs determined by input or plan to prevent blunt timeout error, have the item retrieval logic be connected to run status, clean return value
…API interaction logic
…ed API interaction
matyascimbulka
approved these changes
Aug 27, 2025
Collaborator
matyascimbulka
left a comment
matyascimbulka
left a comment
There was a problem hiding this comment.
It looks great. Thank you for taking care of this issues and bringing it across the finish line.
drobnikj
requested changes
Sep 1, 2025
components/apify/actions/scrape-single-url/scrape-single-url.mjs
Outdated
Show resolved
Hide resolved
| paidPlan: { | ||
| propDefinition: [ | ||
| apify, | ||
| "paidPlan", |
Member
There was a problem hiding this comment.
As we discussed in person, let's keep the current behaviour with waiting unlimited and keep Pipedream kills the workflow if the timeout is exceeded.
components/apify/actions/scrape-single-url/scrape-single-url.mjs
Outdated
Show resolved
Hide resolved
…aset item, remove extra input fields connected to WCC run
Author
|
@drobnikj I'll change the target branch to |
oleksandravalko
changed the base branch from
feat/migrate-to-apify-client
to
master
oleksandravalko
added
enhancement
matyascimbulka
added a commit
that referenced
this pull request
Sep 24, 2025
* feat(apify): Show list of built tags in actor run action * feat(apify): Allow selecting actor search source for run Actor action and change how Actor or task name is displayed * feat(apif): Remove wait for finish prop in run Actor action * fix(apify): Fix PR issues * fix(apif): Fix PR issues * fix(scrape-single-url): return the only dataset item after the run is finished (#3) * fix(scrape-single-url): return the only dataset item after the run is finished * fix(scrape-single-url): version up * fix(scrape-single-url): version up * fix(scrape-single-url): version up * fix(scrape-single-url): introduce a job status constant, expand a list of terminal statuses to stop the loop * fix(scrape-single-url): import constants from package, decrease delay in between calls * Migrate to use Apify client (#6) * feat(apify): Replace Axios with Apify client * fix(general): adding custom headers to client()=> preserve whole config to be passed to Axios later * feat(general): add linter script * fix(apify-get-dataset-items): a function for getting items and parsing of a result * fix(apify-run-actor): working sync and async, dynamic input schema injection, KVS output retrieval tested only string * fix(general): change maxResults for limit as an input field * fix(run-task-sync): move items retrieval to the component, add waitSecs determined by input or plan to prevent blunt timeout error, have the item retrieval logic be connected to run status, clean return value * fix(apify-scrape-single-url): incorporate timeouts, rework the whole API interaction logic * fix(apify-set-key-value-store-record): detection of content type, fixed API interaction * fix(apify-scrape-single-url): remove waiting timeout, return only dataset item, remove extra input fields connected to WCC run * fix(apify-run-actor): success message * fix(apify-run-task-synchronously): remove waiting for run to finish timeout * fix(app): remove paidPlan input filed config --------- Co-authored-by: Matyas Cimbulka <matyas.cimbulka@apify.com> * fix(apify-get-dataset-items) 6: change input parameters (#8) * chore(apify): Bump component versions * chore: Sync upstream repo (#9) * feat(apify): Prefill values from the input schema (#7) * Revert "chore: Sync upstream repo (#9)" This reverts commit cd804ba. * Revert "chore(apify): Bump component versions" This reverts commit 6040822 which for some reason bumped version of the wrong components. * fix(apify): Fix build tag * fix(apify): Address issues in run task synchronously action * feat(apify): Add default crawler type to scrape single url * fix(apify): Address issues from PR * chore(apify): Change component versions * fix(apify): Fix run Actor action * fix(apify): Fix run get dataset items action * fix(apify): Fix typos for PR --------- Co-authored-by: Oleksandra Valko <oleksandra.valko@apify.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
WHY
I've completed my overview of suggested switch to Apify client. As you may notice, this shift has affected methods and their usage dramatically. Code at this stage works as expected, but I might not thinks about all of possible scenarios. So, please give them your own test drive. Cheers!