-
Notifications
You must be signed in to change notification settings - Fork 8.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
til::rle<T, S> - a run length encoded storage template #8794
Conversation
…ributeRun is now just a std::pair<> so it fits nicely in the AttrRow which is just a til::rle now. Add tests.
Known TODO list
|
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
Requests from trying to use with new buffer:
What error do you get when you remove non-const-_at |
Since the new buffer works by counting columns per I implemented it by replacing the impacted region with My current backing store grows as necessary when I do that, and it does not have a maximum size (it is a vector). |
This comment has been minimized.
This comment has been minimized.
VERIFY_ARE_EQUAL("3|2|1 1 1"sv, rle.slice(2, 7)); | ||
// within a run -> within a run | ||
VERIFY_ARE_EQUAL("3|2|1"sv, rle.slice(2, 5)); | ||
VERIFY_ARE_EQUAL("3|2|1 1"sv, rle.slice(2, 6)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh man. We have other places in the code where ""sv would be useful...
src/inc/til/rle.h
Outdated
// NOTE: This can change the size/length of the vector. | ||
void replace(size_type start_index, size_type end_index, const rle_type& new_run) | ||
{ | ||
replace(start_index, end_index, { &new_run, 1 }); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why doesn't this one _check_indicies
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It uses the non-underscored, non-private version of replace
. I could rename _replace
to _unchecked_replace
to avoid confusion.
@@ -27,8 +27,6 @@ Revision History: | |||
#include "WexTestClass.h" | |||
#endif | |||
|
|||
#pragma pack(push, 1) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
FYI: Without packing this class (and ColorType
) it's size increases from just 13 to 14 bytes.
(For some reason the size increase is much larger if you pack ColorType
, but I haven't looked into it.)
But this doesn't change the size of til::rle_pair<TextAttribute, UINT>
, as the latter UINT length
field is 4-byte aligned. This causes padding to be inserted in the rle_pair
and it's size to be 20 bytes either way.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
qq: since the rest of conhost uses SHORT for coordinates, can we consider uint16_t the max size type for an attribute run? Does that help save us a couple bytes?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I ran cacafire for 30s and then ran "cat big.txt" 10 times:
type | time cat big.txt | private working set |
---|---|---|
uint32_t |
1.54s | 36.552MB |
uint16_t |
1.50s | 36.368MB |
{ | ||
// I couldn't figure out how to attach additional info | ||
// to a failed assertion so I'm doing it this way... | ||
Log::Comment(NoThrowString().Format( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yo can use VERIFY_ARE_EQUAL(x, y, NoThrowString().Format(xxxxxxxxxxx))
😄
@check-spelling-bot ReportUnrecognized words, please review:
Previously acknowledged words that are now absentAAAAABBBBBBBCCC AAAAABCCCCCCCCC AAAAADCCCCCCCCC actionmap apos bc cang ci conpixels cplinfo crt cw cx df dh dw eb EK ev exeuwp gb gh Gravell's hc hh hk hu Kd KF KJ KU KX lk memcopy mybase nc Nx oa OI Oj Oo oq Ou Ov pb pv pw px py qi QJ qo reserialize rgch ri Rl rmdir ru smalllogo splashscreen storelogo sx sy sz td tf tl tt uk ul Unregistering vf vk wx xa xy Yw yx YZ Zc zd zh ZM zu zySome files were were automatically ignoredThese sample patterns would exclude them:
You should consider excluding directory paths (e.g. You should consider adding them to:
File matching is via Perl regular expressions. To check these files, more of their words need to be in the dictionary than not. You can use To accept these unrecognized words as correct (and remove the previously acknowledged and now absent words), run the following commands... in a clone of the git@github.com:microsoft/terminal.git repository
|
@check-spelling-bot ReportUnrecognized words, please review:
Previously acknowledged words that are now absentAAAAABBBBBBBCCC AAAAABCCCCCCCCC AAAAADCCCCCCCCC cang memcopy mybase rgchTo accept these unrecognized words as correct (and remove the previously acknowledged and now absent words), run the following commands... in a clone of the git@github.com:microsoft/terminal.git repository
✏️ Contributor please read thisBy default the command suggestion will generate a file named based on your commit. That's generally ok as long as you add the file to your commit. Someone can reorganize it later.
If the listed items are:
See the 🔬 You can test your commits without appending to a PR by creating a new branch with that extra change and pushing it to your fork. The check-spelling action will run in response to your push -- it doesn't require an open pull request. By using such a branch, you can limit the number of typos your peers see you make. 😉 🗜️ If you see a bunch of garbageIf it relates to a ... well-formed patternSee if there's a pattern that would match it. If not, try writing one and adding it to a Patterns are Perl 5 Regular Expressions - you can test yours before committing to verify it will match your lines. Note that patterns can't match multiline strings. binary-ish stringPlease add a file path to the File paths are Perl 5 Regular Expressions - you can test yours before committing to verify it will match your files.
|
@check-spelling-bot ReportUnrecognized words, please review:
Previously acknowledged words that are now absentAAAAABBBBBBBCCC AAAAABCCCCCCCCC AAAAADCCCCCCCCC cang memcopy mybase rgchTo accept these unrecognized words as correct (and remove the previously acknowledged and now absent words), run the following commands... in a clone of the git@github.com:microsoft/terminal.git repository
✏️ Contributor please read thisBy default the command suggestion will generate a file named based on your commit. That's generally ok as long as you add the file to your commit. Someone can reorganize it later.
If the listed items are:
See the 🔬 You can test your commits without appending to a PR by creating a new branch with that extra change and pushing it to your fork. The check-spelling action will run in response to your push -- it doesn't require an open pull request. By using such a branch, you can limit the number of typos your peers see you make. 😉 🗜️ If you see a bunch of garbageIf it relates to a ... well-formed patternSee if there's a pattern that would match it. If not, try writing one and adding it to a Patterns are Perl 5 Regular Expressions - you can test yours before committing to verify it will match your lines. Note that patterns can't match multiline strings. binary-ish stringPlease add a file path to the File paths are Perl 5 Regular Expressions - you can test yours before committing to verify it will match your files.
|
Introduces
til::rle<T, S>
. Stores a chosen type T in a run length encoded (compacted) format. For additional memory savings, the counter size S can be chosen as well.References
ATTR_ROW
andTextAttributeRun
are purposefully left behind as pass-throughs to not get in the way of the other hide/cleanup effort in this PR.til::rle
will be extremely useful as a column counter as we pursue NxM storage and presentation.TextBufferCellIterator
norOutputCellIterator
. It also allows a bulk insert in multiple ways viaassign
andfill
. Getting stuck on inserting cell-by-cell through those iterators is part of the performance detriment left to this issue.pmr
-ified likebitmap
for things likechafa
andcacafire
that are changing the run length frequently (and probably causing many transient allocations).PR Checklist
Detailed Description of the Pull Request / Additional comments
Validation Steps Performed
cacafire
inOpenConsole.exe
and it looked beautifulRunLengthEncodingTests.cpp
supportingtil::rle<T, S>
and derived from the concepts ofAttrRowTests.cpp
which is obsolete/removed.