Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Data] "Rows Outputted" is 0 in Data dashboard #44635

Closed
bveeramani opened this issue Apr 10, 2024 · 0 comments · Fixed by #48745
Closed

[Data] "Rows Outputted" is 0 in Data dashboard #44635

bveeramani opened this issue Apr 10, 2024 · 0 comments · Fixed by #48745
Assignees
@scottjlee
Labels
bug Something that is supposed to be working; but isn't data Ray Data-related issues P1 Issue that should be fixed within a few weeks

Comments

@bveeramani
Copy link
Member

What happened + What you expected to happen

I ran the repro script twice and expected to see 10 rows outputted, but instead I saw 0.

image

Versions / Dependencies

2.10

Reproduction script

import ray
import time


def sleep(row):
    time.sleep(1)
    return row

ray.data.range(10, override_num_blocks=2).map(sleep).materialize()

Issue Severity

Low: It annoys or frustrates me.
@bveeramani bveeramani added bug Something that is supposed to be working; but isn't P2 Important issue, but not time-critical data Ray Data-related issues labels Apr 10, 2024
@gvspraveen gvspraveen added P1 Issue that should be fixed within a few weeks and removed P2 Important issue, but not time-critical labels Oct 8, 2024
This was referenced Nov 8, 2024
Merged
Merged
dentiny pushed a commit to dentiny/ray that referenced this issue Dec 7, 2024
@scottjlee @dentiny
[Data] [Dashboard] Fix Rows Outputted being zero on Ray Data Dashbo…
962fac1 
…ard (ray-project#48745)

## Why are these changes needed?

Currently, there are some cases where the `Rows Outputted` value on the
Ray Job page's `Ray Data Overview` section says "0", even after the
dataset execution completes. The root cause of the bug is that we clear
iteration/execution metrics after the dataset completes. This was
previously used to "reset" the metrics to 0 after dataset completion, so
that the last emitted value would not persist on the dashboard, even
after the job finishes. Now that we display rates on the dashboard, this
hack is no longer needed, and we can skip the metrics clearing.

Fixed result:
<img width="1860" alt="Screenshot at Nov 14 12-11-24"
src="https://github.com/user-attachments/assets/35061b3f-9359-412b-8ab2-f4bcce412994">

## Related issue number

Closes ray-project#44635

## Checks

- [x] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [x] I've run `scripts/format.sh` to lint the changes in this PR.
- [ ] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [x] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [x] Unit tests
   - [ ] Release tests
   - [ ] This PR is not tested :(

---------

Signed-off-by: Scott Lee <sjl@anyscale.com>
Signed-off-by: hjiang <dentinyhao@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@scottjlee scottjlee

Labels
bug Something that is supposed to be working; but isn't data Ray Data-related issues P1 Issue that should be fixed within a few weeks
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[Data] [Dashboard] Fix Rows Outputted being zero on Ray Data Dashboard scottjlee/ray
3 participants
@gvspraveen @scottjlee @bveeramani