Skip to content
This repository has been archived by the owner on Nov 3, 2023. It is now read-only.

[Cringe] edits to metric logging #5036

Merged
merged 1 commit into from
May 18, 2023
Merged

[Cringe] edits to metric logging #5036

merged 1 commit into from
May 18, 2023

Conversation

jxmsML
Copy link
Contributor

@jxmsML jxmsML commented May 17, 2023

Patch description
Main change:

  1. target_token
    before: target_tokens = notnull.long().sum()
    after:
        notnull = batch.label_vec.ne(self.NULL_IDX)
        ce_mask = torch.logical_and(notnull, ce_mask.view(-1, batch.label_vec.size(-1)))
        ct_mask = torch.logical_and(notnull, ct_mask.view(-1, batch.label_vec.size(-1)))
        # number of tokens in each examples for cross entropy or cringe loss.
        metric_notnull = torch.logical_or(ce_mask, ct_mask)
        target_tokens = metric_notnull.long().sum(dim=-1)
  1. logging other
    metrics:
  1. ppl_ce: ppl on tokens computed for lm nll loss
  2. ce_target_tokens: count of tokens computed for lm nll loss
  3. ct_target_tokens: count of tokens computed for cringe loss

Testing steps

Other information

@jxmsML jxmsML requested a review from jaseweston May 17, 2023 16:17
@jxmsML jxmsML merged commit dea6377 into main May 18, 2023
@jxmsML jxmsML deleted the cringe0516 branch May 18, 2023 15:43
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants