Skip to content

Commit

Permalink
[ML] Fixing categorization tokens for multi-line messages (#103007)
Browse files Browse the repository at this point in the history
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
  • Loading branch information
jgowdyelastic and kibanamachine authored Jun 29, 2021
1 parent b774e37 commit 824463a
Showing 1 changed file with 3 additions and 2 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -145,10 +145,11 @@ export function categorizationExamplesProvider({
for (let g = 0; g < sumLengths.length; g++) {
if (t.start_offset <= sumLengths[g] + g) {
const offset = g > 0 ? sumLengths[g - 1] + g : 0;
const start = t.start_offset - offset;
tokensPerExample[g].push({
...t,
start_offset: t.start_offset - offset,
end_offset: t.end_offset - offset,
start_offset: start,
end_offset: start + t.token.length,
});
break;
}
Expand Down

0 comments on commit 824463a

Please sign in to comment.