Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial CoreClr Android logcat log integration. #113416

Open
wants to merge 12 commits into
base: main
Choose a base branch
from

Conversation

lateralusX
Copy link
Member

@lateralusX lateralusX commented Mar 12, 2025

Initial CoreClr Android logcat integration was removed from cf2aae6. This PR adds a unified approach re-routing runtime console logging though a central log API defined in native/minipal/log.h potential shared between components as well as runtimes.

This initial PR starts out defining a new header under native/minipal/log.h that includes a straightforward API to log formatted and raw output using different priority. It includes regular printf/vprintf style functions as well as low level none crt direct write capabilities, usable in high speed or async safe logging.

API closely mimics Mono's logging API defined in glib.h and the idea is to potentially rewire Mono's logging to use this new API going forward potentially adding more capabilities into native/minipal/log.h

Ability to log different log message using different priority is inherited from Mono and maps close to logcat priority levels, where different priorities can be filtered directly in logcat, creating virtual log streams. API includes a way to log directly to virtual standard error/output streams in case no specific need for fine grained priority-based logging exists. On platforms supporting stderr/stdout this will be a 1:1 mapping, but on platforms like logcat, stderr/stdout will be mapped to similar priority levels (error, info).

There is no buffering when calling logcat log functions, each log call will end up as a new log entry. Code that relies on ability to do multiple logging calls and expect to get a new line only when writing \n into the log buffer, won't behave as expected when using logcat. This PR adjust a couple of places where such assumptions are made, appending logging into a string buffer before passed to minipal_log_write API's. Logcat has an internal buffer limitation of 4068 bytes, buffers larger than that will be truncated. To mitigate this logging targeting Android includes a mechanism splitting up the bigger buffer into smaller chunks based on \n in log message, if there is no \n in bigger log message, message will still be logged in chunks, but each chunk ending up as a separate logcat message. Internal chunk size has been defined to 4000 bytes, making room for future internal logcat buffer reduction. Regular console logging enforces a buffer size limit as well on low level write calls, this limitation was directly inherited from previous implementation in src/coreclr/vm/util.cpp into native/minipal/log.c, if buffer exceeds chunk size, it will be split into multiple chunks where each chunk is written "as is" to underlying write function.

On Mono, formatted print style logcat implementation uses dynamic memory allocation + vsnprintf and low level __android_log_write. Current implementation in native/minipal/log.h adapt a similar schema but tries to first use a smaller stack buffer, inline with Androids internal implementation of __android_log_vprint use of 1024 bytes buffer. If log message is below 1024, implementation will match __android_log_vprint and use stack memory, but if exceeding 1024, implementation fallbacks to dynamic memory allocation, format message into dynamic memory buffer and then pass it through minipal_log_write to correctly handle max payload limitations as described above.

This initial PR adapts a smaller set of source files under src/coreclr to use the new logging API as a first step adaption. It makes sure the LOG macro as well as exception handling gets emitted through the new log API ending up in logcat. PR also adapts jit logging as well as diagnostic servers pause message, currently written directly to console.

This PR will be followed by an additional PR adapting more sources under src/coreclr/vm to use the new logging API enabling more log messages to end up in logcat on Android. Once the initial pass is done, adaption to new logging API will probably happen on a case-by-case basis.

@Copilot Copilot bot review requested due to automatic review settings March 12, 2025 14:39

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot reviewed 13 out of 13 changed files in this pull request and generated no comments.

@lateralusX lateralusX changed the title Initial CoreClr Android logcat console log integration. Initial CoreClr Android logcat log integration. Mar 12, 2025
Copy link
Contributor

Tagging subscribers to this area: @mangod9
See info in area-owners.md if you want to be subscribed.

Comment on lines +16 to +24
typedef enum
{
minipal_log_flags_fatal = 1 << 1,
minipal_log_flags_error = 1 << 2,
minipal_log_flags_warning = 1 << 3,
minipal_log_flags_info = 1 << 4,
minipal_log_flags_debug = 1 << 5,
minipal_log_flags_verbose = 1 << 6
} minipal_log_flags;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we actually need this level of log granularity. There is very little distinction between info warning debug and verbose in practice. All those message could probably be lumped under debug unless there is a specific reason to have them.

If we want them, then we should have very clear and precise definitons for when to use info vs debug vs verbose vs warning.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This mainly maps to what we currently have in Mono as well as what Android has for its different logging priority. If we would like to move Mono's logging over at some future point, we would need to at least have a representation of these different priorities or we won't have a migration path for Mono.

Other CoreCLR logging technology, like LOG and STRESS_LOG currently have even more levels, 0-10, but I agree having to many makes it unclear when to use what.

Could an option to do the following, keep the priorities in the enum in order to have a migration path for Mono, meaning it could at least use the underlying minipal_log_print/minipal_log_vprint/minipal_log_write functions directly. Reduce the set of defined functions targeting a smaller set of predefined functions primarily having fatal, error, info and verbose? I believe there is value to have at least 2 priorities of none error related levels, one that is used for more infrequent loggings that might appear under normal runtime conditions and one for dumping larger amount of data, like what the JIT compiler does (currently using the verbose priority). We could then easily map stderr to error and stdout to info, fatal is reserved for errors aborting/exiting the process.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a small clarification - the LOG and STRESSLOG levels are not really kinds of logging in the sense of the ones mentioned above. They attempt to express decadic logarithm of frequency of occurrence of those loggings. So when I choose certain level for logging output, I am limiting it to messages upto that expected frequency. It has nothing to do with fatal / error / info etc.

Copy link
Member Author

@lateralusX lateralusX Mar 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is what I did:

  • Kept all values in enum so that Mono could transition over to this API in the future, using the underlying minipal_log_print/vprint/write functions directly.
  • Reduced the number of defined logging functions down to fatal/error/info/verbose.
  • Only use the 4 predefined logging functions through this PR, there is usage of all variations in this PR.

@lateralusX lateralusX force-pushed the lateralusX/hookup-coreclr-android-logging branch from c50079c to f0162da Compare March 12, 2025 17:25
@jkotas
Copy link
Member

jkotas commented Mar 12, 2025

This PR will be followed by an additional PR adapting more sources under src/coreclr/vm to use the new logging API enabling more log messages to end up in logcat on Android.

What is an example of logging that you plan to convert? I think this PR covers it all.

@lateralusX
Copy link
Member Author

This PR will be followed by an additional PR adapting more sources under src/coreclr/vm to use the new logging API enabling more log messages to end up in logcat on Android.

What is an example of logging that you plan to convert? I think this PR covers it all.

There are still printf's in utilcode and vm folder that could be hooked up to the minipal_log API's.

@am11
Copy link
Member

am11 commented Mar 17, 2025

If we want to capture PAL debug build logs in logcat, they are in src/coreclr/pal/src/include/pal/dbgmsg.h. They help tracing code paths during development. e.g. PAL_DBG_CHANNELS="+all.all" corerun ...

@lateralusX
Copy link
Member Author

lateralusX commented Mar 17, 2025

If we want to capture PAL debug build logs in logcat, they are in src/coreclr/pal/src/include/pal/dbgmsg.h. They help tracing code paths during development. e.g. PAL_DBG_CHANNELS="+all.all" corerun ...

I have a follow up PR that adjust additional files under utilcode, pal and vm folder, I make sure to add this one into that PR as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants