Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change WDS index version representation to integer. #4708

Merged
merged 4 commits into from
Mar 14, 2023

Conversation

mzient
Copy link
Contributor

@mzient mzient commented Mar 13, 2023

Category:

Refactoring (Redesign of existing code that doesn't affect functionality)

Description:

It changes the representation of the WDS index from a string to integer. It improves performance and it would help with comparisons.

Additional information:

Affected modules and functionalities:

WDS reader

Key points relevant for the review:

Tests:

dali/test/python/reader/test_webdataset_*.py

  • Existing tests apply
  • New tests added
    • Python tests
    • GTests
    • Benchmark
    • Other
  • N/A

Checklist

Documentation

  • Existing documentation applies
  • Documentation updated
    • Docstring
    • Doxygen
    • RST
    • Jupyter
    • Other
  • N/A

DALI team only

Requirements

  • Implements new requirements
  • Affects existing requirements
  • N/A

REQ IDs: N/A

JIRA TASK: DALI-2557

Signed-off-by: Michał Zientkiewicz <mzient@gmail.com>
@mzient
Copy link
Contributor Author

mzient commented Mar 13, 2023

!build

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [7567149]: BUILD STARTED

@JanuszL JanuszL self-assigned this Mar 13, 2023
Signed-off-by: Michał Zientkiewicz <mzient@gmail.com>
Signed-off-by: Michał Zientkiewicz <mzient@gmail.com>
@@ -24,6 +24,7 @@
#include <utility>
#include <vector>
#include "dali/core/bitmask.h"
#include "dali/core/version_util.h"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not include that in .cc file?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will do.

Signed-off-by: Michał Zientkiewicz <mzient@gmail.com>
@dali-automaton
Copy link
Collaborator

CI MESSAGE: [7567523]: BUILD STARTED

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [7567523]: BUILD PASSED

#ifndef DALI_CORE_VERSION_UTIL_H_
#define DALI_CORE_VERSION_UTIL_H_

namespace dali {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The contents of this file are (mostly) copied from core/util.h - these functions are not generic enough to warrant a place in util.h. The differences:

  • GetVersionNumber renamed to MakeVersionNumber
  • Added a default value for patch
  • Made value a template V * instead of fixed int *.
  • Added a customizable invalid_value

@jantonguirao jantonguirao self-assigned this Mar 14, 2023
@mzient mzient merged commit ebd85f0 into NVIDIA:main Mar 14, 2023
aderylo pushed a commit to zpp-dali-2022/DALI that referenced this pull request Mar 17, 2023
… utilties. (NVIDIA#4708)

* Change WDS index version representation to integer.
* Use a common utility for generating an integer version number.
* Move version utilities to a standalone header
* Make version utilities more generic

Signed-off-by: Michał Zientkiewicz <mzient@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants