only getting partial du console output in cloud

[jrcheli]

With AppScope 1.0.3 on an x86_64 ec2 machine running Ubuntu 18.04.6 LTS
And with a cloud instance with this System Information:

Organization ID:	blahblahblah
UI Version:	3.4.0-a45499bf/2022-03-22T08:17:15.669Z
Backend Version:	3.4.0-a45499bf/v3.4.0/2022-03-22T08:23:37.538Z

After configuring appscope to send to my stream cloud instance over tls, I was looking at console output events with stream.
LD_PRELOAD=lib/linux/x86_64/libscope.so ps -ef
LD_PRELOAD=lib/linux/x86_64/libscope.so df -h
LD_PRELOAD=lib/linux/x86_64/libscope.so du -h

The for the first two commands, all console data which was output to the local terminal made it to the cloud instance (observed by doing a live capture of the AppScope Source in Stream). The last command had some console output make it to the cloud instance but not all.

I retried with SCOPE_CONNECT_TIMEOUT_SECS=10 and it did not change the behavior.

The du command in question output quite a few lines to the console:

ubuntu@ip-10-8-107-159:~/jrc/appscope5$ SCOPE_CONNECT_TIMEOUT_SECS=10 LD_PRELOAD=lib/linux/x86_64/libscope.so du -h | wc
   3583    7166  201461

The amount that made it to the cloud instance:

ubuntu@ip-10-8-107-159:~/jrc/appscope5$ cat file | wc
   1231    2462   65533

[jrcheli]

I thought this might be explained by work done in #781.

To test this theory with 1.0.4, I disabled the Nagle algorithm by adding BIO_set_tcp_ndelay(trans->net.sock, TRUE); to src/transport.c:establishTlsSession(), and recompiled the library. Before this change, I observed the missing console with the du -h command. After making the change, I still did not receive the console message in my stream cloud instance. Huh. This makes me think that it might not be completely explained by #781, after all?

This was the command I ran, where I didn't receive any console messages at all:

SCOPE_CRIBL_CLOUD=tcp://in.logstream.<myinstance>.cribl.cloud:10090 LD_PRELOAD=lib/linux/x86_64/libscope.so du -h

But, if I added SCOPE_EVENT_FS=false to this same command, I did receive a single console message in my cloud instance of Stream. This is good, but the contents of the console message was not complete.

So is AppScope capturing all of the console information? This proves to me that we are capturing all console data. After running this command, and inspecting /tmp/eventout I could see that all of the console data is present (spread across 7 different console events):

SCOPE_EVENT_FS=false SCOPE_CRIBL_ENABLE=false SCOPE_EVENT_DEST=file:///tmp/eventout  LD_PRELOAD=lib/linux/x86_64/libscope.so du -h

So it appears that either AppScope isn't sending all the console data over tls, or Stream isn't receiving, processing, or presenting it all.

[iapaddler]

tested with 1.1. i am getting console data at a cloud connection:
SCOPE_CRIBL=tcp://in.logstream..cloud:10091 LD_PRELOAD=lib/linux/x86_64/libscope.so du -h | wc -l

[jrcheli]

I agree with iapaddler, that this ticket is no longer needed.

I changed two things from a couple of days ago:

1. I'm using this "Filter Expression" in Cribl.Cloud, to help make sure that I'm not accidentally scrolling past the console events in the live capture window:

__inputId=='appscope:in_appscope_tls' && sourcetype=='console'

2. I'm using the latest AppScope (e3a0fd1)

I think the TCP_QUICKACK (from 8c5b798) must be making the difference! woot!