feat: add basic support for setStatusMessage
#1790
Conversation
|
|
||
| const log = async () => { | ||
| const currentStats = this.stats.calculate(); | ||
| await client.setStatusMessage?.(`Crawled ${this.handledRequestsCount}/${currentStats.requestsTotal} pages.`); |
There was a problem hiding this comment.
we wanted to have the messages a bit smarter, our initial thoughts were this:
- as long as the crawler runs normally, we could write something like
Crawling 5/123 pages, 8 requests failed, 5 results in default dataset., we’d have to define what means the “normally”, e.g. as long as there are no hard failures (requests that reached all retries), we could consider normal - when we see higher error rate, we could print something about the errors, e.g. the most common error, maybe something from the retry histogram or statistics object?
- maybe some stats about average response time?
https://apifier.slack.com/archives/CG3EJESUX/p1671462495090689
There was a problem hiding this comment.
It's a bit updated / more granular now, but the final format is still WIP - imo in this case, less is more - status message should inform you about the run status as fast as possible - having long, convoluted messages filled with data isn't the way to go (again, just my opinion, I've seen the Slack thread)
B4nan
changed the title
setStatusMessage
|
Let's release a new version so the changes can be implemented in the |
|
Let me check the E2E tests first, I'd like to see how this works on the platform. |
|
I was expecting the status message to be printed to the console logs, which apparently it is not. I'd like the behaviour to be the same regardless of the storage being used, it should always log to console, and if apify client is used, it will also make the API call. Other than that, it seems to be working fine. |
|
Well, we haven't released yet, this is still doable. I would argue that the output will be a bit duplicit on the Platform (seeing the new status message both in the log and the status message pill), but it's probably better than having different logs locally and on the platform 🤔 Let's log from Crawlee directly and noop the statusMessage in the local storage implementations |
The log should be the main source of truth, and only there you can see the history - status message changes over time. |
|
Yeah I don't mind logging from crawlee. I think it sounds sane to have |
|
pr in here #1808 |
|
Agree with log everywhere. |
No description provided.