Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add missing standardize_image key to metadata #2946

Merged
merged 2 commits into from
Jan 19, 2023
Merged

Conversation

jppgks
Copy link
Contributor

@jppgks jppgks commented Jan 17, 2023

A Ludwig model from before #2408 failed to preprocess_for_prediction:

File /workspaces/ludwig/ludwig/features/image_feature.py:479, in ImageFeatureMixin._finalize_preprocessing_parameters(preprocessing_parameters, encoder_type, column)
    475 assert isinstance(num_channels, int), ValueError("Number of image channels needs to be an integer")
    477 average_file_size = np.mean(sample_num_bytes) if sample_num_bytes else None
--> 479 standardize_image = preprocessing_parameters["standardize_image"]
    480 if standardize_image == "imagenet1k" and num_channels != 3:
    481     warnings.warn(
    482         f"'standardize_image=imagenet1k' is defined only for 'num_channels=3' but "
    483         f"detected 'num_channels={num_channels}'.  For this situation setting 'standardize_image=None'.",
    484         RuntimeWarning,
    485     )

KeyError: 'standardize_image'

@github-actions
Copy link

github-actions bot commented Jan 17, 2023

Unit Test Results

         6 files  ±  0           6 suites  ±0   4h 50m 42s ⏱️ - 8m 35s
  3 892 tests ±  0    3 817 ✔️ ±  0    74 💤 ±  0  1 ±0 
11 532 runs   - 95  11 327 ✔️  - 78  204 💤  - 17  1 ±0 

For more details on these failures, see this check.

Results for commit 42efd48. ± Comparison against base commit d83693a.

♻️ This comment has been updated with latest results.



def _is_image_feature(feature_config: FeatureConfigDict):
preproc = feature_config.get(PREPROCESSING, {})
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we just be checking feature_config[TYPE] == IMAGE instead?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like feature type information is not available on the training set metadata

@jppgks jppgks merged commit 1213a54 into master Jan 19, 2023
@jppgks jppgks deleted the compat-image-preproc branch January 19, 2023 11:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants