Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix color codes emitting mid-UTF8 code. #312

Merged
merged 2 commits into from
Mar 21, 2023
Merged

Fix color codes emitting mid-UTF8 code. #312

merged 2 commits into from
Mar 21, 2023

Conversation

blackhole89
Copy link
Contributor

Some moving around of ANSI color code emissions in recent patches has left us in a situation where RESET codes were getting defensively emitted after every token, resulting in multibyte UTF-8 codes that are split across tokens being broken up:

image

This fixes that, while cleaning up the entire color control architecture, stopping unnecessary emission of color codes and probably making it more conducive to portability later on, e.g. to platforms where color has to be set by out-of-band API calls.

image

@blackhole89 blackhole89 requested a review from ggerganov March 20, 2023 01:58
@anzz1 anzz1 self-requested a review March 20, 2023 03:40
Copy link
Contributor

@anzz1 anzz1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everything looks good for now, but as you pointed out the
line 843: con_use_color = params_use.color;
is a bit silly design.

The params could be made a global variable, but then again making the design go in that direction would hold back future design where modularity might be wanted so I'm not sure that would be the right direction to take. It seems to me that not having any global variables is a conscious design choice.

In that sense maybe using a "global state" for the console isn't a very good long-term solution, and it would be better to have to the "console state" as a param which could be carried over with the params context wherever needed.

@blackhole89
Copy link
Contributor Author

blackhole89 commented Mar 20, 2023

Thanks for the review! I do wish we could find a more elegant solution, but it's not easy.

In that sense maybe using a "global state" for the console isn't a very good long-term solution, and it would be better to have to the "console state" as a param which could be carried over with the params context wherever needed.

The thing is, the actual console state, which the con_st variable is supposed to reflect, is in fact global. If the existing code sets the color to blue, control passes to an unrelated routine that was written without knowledge of our code, and that routine then prints to the console, the text will come out blue all the same. This is arguably a design flaw of the console itself, as a shared resource with internal state that is not easily interrogated, but doing anything about that is obviously way outside of our scope. However, given that, I'm not convinced making con_st more local on our end would amount to an improvement as far as the overall design is concerned. At most you could argue that it should be associated with a particular fd, but even that has problems as multiple fds can (and often will: setting the color on stdout will affect output to stderr) point to the same console.

Note also that we do in fact already have a piece of state that is "global" / associated with main.cpp-the-linked-module, namely static bool is_interacting. To be fair, that one was my doing as well, but it was somewhat necessitated by the requirement to have the SIGINT handler interact with the main loop (which you could argue is also downstream from the console being a stateful shared resource). Therefore, even if con_use_color and con_st are awkward, I'd argue that they don't really push the envelope.

@gjmulder gjmulder added the bug Something isn't working label Mar 20, 2023

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants