Skip to content

Commit

Permalink
[Data] Fix logging output from write_xxx APIs (#48096)
Browse files Browse the repository at this point in the history
## Why are these changes needed?
Followup to #47942. Fixes the
formatting of the output logged after write operation finishes, which
incorrectly printed out the entire `dataclasses.field` object.

Previous logging:
```
2024-10-17 17:26:48,396 INFO datasink.py:103 -- Write operation succeeded. Aggregated write results:                                                
        Field(name='num_rows',type=<class 'int'>,default=0,default_factory=<dataclasses._MISSING_TYPE object at 0x104ad5970>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),_field_type=_FIELD): 100
        Field(name='size_bytes',type=<class 'int'>,default=0,default_factory=<dataclasses._MISSING_TYPE object at 0x104ad5970>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),_field_type=_FIELD): 800
```

Fixed logging:
```
2024-10-17 17:30:52,491 INFO datasink.py:103 -- Write operation succeeded. Aggregated write results:                                                
        - num_rows: 100
        - size_bytes: 800
```

## Related issue number


## Checks

- [x] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [x] I've run `scripts/format.sh` to lint the changes in this PR.
- [ ] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [x] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [x] Unit tests
   - [ ] Release tests
   - [ ] This PR is not tested :(

Signed-off-by: Scott Lee <sjl@anyscale.com>
  • Loading branch information
scottjlee authored Oct 18, 2024
1 parent d5fa9a0 commit 09cf5bb
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion python/ray/data/datasource/datasink.py
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ def on_write_complete(self, write_result_blocks: List[Block]) -> WriteResult:
aggregated_results_str = ""
for k in fields(aggregated_write_results.__class__):
v = getattr(aggregated_write_results, k.name)
aggregated_results_str += f"\t{k}: {v}\n"
aggregated_results_str += f"\t- {k.name}: {v}\n"

logger.info(
f"Write operation succeeded. Aggregated write results:\n"
Expand Down

0 comments on commit 09cf5bb

Please sign in to comment.