Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Modify metadata_register API for support individual metric propagation. #1387

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

yigithanyigit
Copy link
Contributor

In current state of this API it can only propagate explicitly given features, In other words giving an identifier doesn't work.

For example; for propagating PSNR:
psnr_y, psnr_cb, psnr_cr should be given.

With these changes, propagating individual features and propagating with identifier possible.

@yigithanyigit yigithanyigit reopened this Sep 12, 2024
@kylophone
Copy link
Collaborator

I'm not sure if we need an API like this yet (?). I will hold off on the reviewing this PR until we figure that out. In the meantime, I will reply to your patch in ffmpeg-devel.

@yigithanyigit
Copy link
Contributor Author

Thanks for the comment,

I found some problems about this patch and I wanted to add for future reference.

After some tests I realized this patch doesn't work expected in some identifiers (VIF, ADM).
The main reason of the problem those identifiers has some conditional features.

For example:

if (!s->debug) return err;
err |= vmaf_feature_collector_append_with_dict(feature_collector,
s->feature_name_dict, "integer_adm", score, index);
err |= vmaf_feature_collector_append_with_dict(feature_collector,
s->feature_name_dict, "integer_adm_num", score_num, index);
err |= vmaf_feature_collector_append_with_dict(feature_collector,
s->feature_name_dict, "integer_adm_den", score_den, index);
err |= vmaf_feature_collector_append_with_dict(feature_collector,
s->feature_name_dict, "integer_adm_num_scale0", scores[0], index);
err |= vmaf_feature_collector_append_with_dict(feature_collector,
s->feature_name_dict, "integer_adm_den_scale0", scores[1], index);
err |= vmaf_feature_collector_append_with_dict(feature_collector,
s->feature_name_dict, "integer_adm_num_scale1", scores[2], index);
err |= vmaf_feature_collector_append_with_dict(feature_collector,
s->feature_name_dict, "integer_adm_den_scale1", scores[3], index);
err |= vmaf_feature_collector_append_with_dict(feature_collector,
s->feature_name_dict, "integer_adm_num_scale2", scores[4], index);
err |= vmaf_feature_collector_append_with_dict(feature_collector,
s->feature_name_dict, "integer_adm_den_scale2", scores[5], index);
err |= vmaf_feature_collector_append_with_dict(feature_collector,
s->feature_name_dict, "integer_adm_num_scale3", scores[6], index);
err |= vmaf_feature_collector_append_with_dict(feature_collector,
s->feature_name_dict, "integer_adm_den_scale3", scores[7], index);

if (!s->debug) return err;
const double score_num =
(double)vif.scale[0].num + (double)vif.scale[1].num +
(double)vif.scale[2].num + (double)vif.scale[3].num;
const double score_den =
(double)vif.scale[0].den + (double)vif.scale[1].den +
(double)vif.scale[2].den + (double)vif.scale[3].den;
const double score =
score_den == 0.0 ? 1.0f : score_num / score_den;
err |= vmaf_feature_collector_append_with_dict(feature_collector,
s->feature_name_dict, "integer_vif", score, index);
err |= vmaf_feature_collector_append_with_dict(feature_collector,
s->feature_name_dict, "integer_vif_num", score_num, index);
err |= vmaf_feature_collector_append_with_dict(feature_collector,
s->feature_name_dict, "integer_vif_den", score_den, index);
err |= vmaf_feature_collector_append_with_dict(feature_collector,
s->feature_name_dict, "integer_vif_num_scale0", vif.scale[0].num,
index);
err |= vmaf_feature_collector_append_with_dict(feature_collector,
s->feature_name_dict, "integer_vif_den_scale0", vif.scale[0].den,
index);
err |= vmaf_feature_collector_append_with_dict(feature_collector,
s->feature_name_dict, "integer_vif_num_scale1", vif.scale[1].num,
index);
err |= vmaf_feature_collector_append_with_dict(feature_collector,
s->feature_name_dict, "integer_vif_den_scale1", vif.scale[1].den,
index);
err |= vmaf_feature_collector_append_with_dict(feature_collector,
s->feature_name_dict, "integer_vif_num_scale2", vif.scale[2].num,
index);
err |= vmaf_feature_collector_append_with_dict(feature_collector,
s->feature_name_dict, "integer_vif_den_scale2", vif.scale[2].den,
index);
err |= vmaf_feature_collector_append_with_dict(feature_collector,
s->feature_name_dict, "integer_vif_num_scale3", vif.scale[3].num,
index);
err |= vmaf_feature_collector_append_with_dict(feature_collector,
s->feature_name_dict, "integer_vif_den_scale3", vif.scale[3].den,
index);
return err;

As you see those features going to append if debug enabled. However my implementation going to add those features anyway:

static const char *provided_features[] = {
"VMAF_integer_feature_adm2_score", "integer_adm_scale0",
"integer_adm_scale1", "integer_adm_scale2", "integer_adm_scale3",
"integer_adm", "integer_adm_num", "integer_adm_den",
"integer_adm_num_scale0", "integer_adm_den_scale0", "integer_adm_num_scale1",
"integer_adm_den_scale1", "integer_adm_num_scale2", "integer_adm_den_scale2",
"integer_adm_num_scale3", "integer_adm_den_scale3",
NULL
};

We should handle this somehow. Maybe we can add an option to VmafMetadataConfiguration.

Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants