Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support larger than 1k client responses (bigger winfmts, etc) #43

Merged
merged 2 commits into from
Dec 15, 2022

Conversation

smemsh
Copy link
Contributor

@smemsh smemsh commented Sep 14, 2022

Previously the buffer size could not exceed 1k in the response from server, or it would be truncated. This included commands like, for example:

$ windows w %{i,a,c,t,n}

would easily get over 1024 chars (actually 1022 due to \n and \0) on my system. The new code can handle any size response.

Fixes #39

@smemsh smemsh force-pushed the large-cmd-responses branch from a58d08f to b795f08 Compare September 14, 2022 21:56
This patch allows those of us with larger numbers of windows and use
"windows" format strings to print stuff to wrapper scripts, to continue
to use sdorfehs when switching from ratpoison.

Before this patch, client would read only a single up-to-1k bytes buffer
to get the entire response, resulting in truncation of the message
(windows list, etc) if it was longer than 1022 bytes.  Ultimately this
was a consequence of retooling to a different IPC mechanism that
happened in a426e8b, the new code uses a single read of 1024 bytes to
get the response.

First thought was to just make the response buffer larger, but who knows
what uses people have? Can we predict the right size limit? This way it
can handle any sized response.  Server is still the one controlling size
of result buffer, and fixed 1k is still used for the "request" buffer
sent to the server process.

Second thought was, since response already has "successfulness" as the
first byte of response (and we have to process it), we could declare an
"initial response" packet that would include some bytes that specified
the response buffer size, and then client would always make a second
second request, this time with an exact size it knew in advance.  This
may have been a fine approach, but results in an extra round trip for
the common case of a single buffer.  Thought a loop would be better...

So, instead we treat the first buffer specially, by continuing to strip
the successfulness response, but then also now arranging for subsequent
reads to be O_NONBLOCK.  Presence of the first buffer means that
necessarily the server has prepared the full response, so we can keep
going until a blocked or closed connection to get the rest of it.

In testing, at response conclusion, sometimes the socket connection was
closed, and sometimes the end of the response would just block forever
waiting for another read.  Was not able to tell what differentiates
these end-states, but also found that treating them identically (as
"command finished" signals) always seems to result in the right
behavior, so it might be ok not to know why...
@smemsh smemsh force-pushed the large-cmd-responses branch from b795f08 to 3c1e6b0 Compare September 14, 2022 22:02
@smemsh
Copy link
Contributor Author

smemsh commented Sep 14, 2022

Too bad about all the #ifdef DEBUG, but PRINT_DEBUG() isn't really useful in the client (it prints to stdout) and warnx() could not be used without condition because those print to stderr, which non-debug case should not have to deal with.

I debated removing these prints after the code was working, but it took some time to develop the right points where those debug prints should be located, I had an occasional off-by-one that was difficult to track down. A new macro like PRINT_DEBUG() but printing to stderr would be better for these, or even a client flag for cli to turn this on and off, but then we would need args processing.

For now, thought it best to ask for comment on what should be done with these prints...

@smemsh
Copy link
Contributor Author

smemsh commented Sep 15, 2022

Too bad about all the #ifdef DEBUG, but PRINT_DEBUG() isn't really useful in the client (it prints to stdout)

I added a different macro WARNX_DEBUG() that uses stderr, it's only used in the one function now, but it reads a lot better while retaining the important print info for debugging, at least for me. This function is special anyways because it's for cli and used in scripts, it has to use stderr so as to be out-of-band.

This will not print anything if -DDEBUG is not defined.

Let me know if it's suitable, maybe you won't like those prints even in your debug builds.

@smemsh smemsh force-pushed the large-cmd-responses branch 2 times, most recently from b98e8eb to 38eb5f0 Compare September 15, 2022 18:33
@smemsh
Copy link
Contributor Author

smemsh commented Sep 15, 2022

now I further hid those stderr macros under -DSENDCMD_DEBUG instead of just -DDEBUG because it's probably only useful in subsystem-specific scenario just like -DINPUT_DEBUG

The client cannot use PRINT_DEBUG because it sends to stdout, which we
cannot contaminate.  We can use warn(), but don't want that to pollute
the non-DEBUG experience, and we don't want to pollute -DDEBUG, so we
add it under a new -DSENDCMD_DEBUG following the example of INPUT_DEBUG.

However, to avoid ifdefs everywhere, we'll prefer to do that in a macro.
So we just add another macro that uses stderr.  It's a bit more compact
than the one that uses stdout, this one uses __VA_ARGS__ as a straight
text macro, this should compile on BSD and Linux (only latter tested).

We could not use a modification of PRINT_DEBUG because it would have
needed fprintf(file, ...) and the way the macro was written it would
require passing ((args like this)) because it had a naked "printf args"
and relied on getting first level "()" stripped in the macro expansion.
This was a bit strange.  It does mean there's now inconsistency between
PRINT_DEBUG and WARNX_DEBUG usage (former needs '(())', latter '()') but
the latter macro is only used by one function anyways and only
appropriate for the client so usage will probably never expand.
@smemsh smemsh force-pushed the large-cmd-responses branch from 38eb5f0 to cb343a2 Compare December 15, 2022 18:24
@jcs jcs merged commit c671599 into jcs:master Dec 15, 2022
@smemsh smemsh deleted the large-cmd-responses branch December 15, 2022 20:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

large window lists are truncated
2 participants