Enhance Logging Infrastructure with Structured Logging and Correlation IDs

[monilpat]

Is your feature request related to a problem? Please describe.

Debugging and monitoring the current system is challenging due to inconsistent log formats and missing context, especially when dealing with distributed systems.

Describe the solution you'd like

Implement a structured logging framework that:

• Utilizes JSON format for all logs to ensure consistency
• Includes essential fields such as timestamp, severity, and correlation IDs to trace requests across services
• Supports context injection to enrich logs with relevant metadata
• Provides different log levels (DEBUG, INFO, WARN, ERROR) for flexibility in log verbosity
• Allows adding custom fields to capture application-specific data
• Offers utilities for performance logging to identify bottlenecks

Describe alternatives you've considered

• Continuing with plain text logs and using grep for search
• Manually formatting logs in JSON
• Relying solely on APM tools without structured logs

Additional context

This enhancement will greatly improve our ability to debug issues, monitor application health, and analyze system performance. It aligns with industry best practices and will facilitate smoother integration with log aggregation tools.