[WIP][SPARK-29779][SQL] Compact old event log files and cleanup - part 2

[HeartSaVioR]

What changes were proposed in this pull request?

This patch is the part 2 of SPARK-29779 and only addresses adding filter to handle SQL related events. Please refer the description of PR #27085 to see overall rationalization of this patch.

Below functionalities will be addressed in later parts:

• integrate compaction into FsHistoryProvider
• documentation about new configuration

Why are the changes needed?

One of major goal of SPARK-28594 is to prevent the event logs to become too huge, and SPARK-29779 achieves the goal. We've got another approach in prior, but the old approach required models in both KVStore and live entities to guarantee compatibility, while they're not designed to do so.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Added UTs.

[HeartSaVioR]

#27085 is the part 1 of SPARK-29779 - please review it first. Once we merged the part 1, I'll rebase this patch and remove WIP tag.

[SparkQA]

Test build #116071 has finished for PR 27086 at commit 38bffaa.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #116074 has finished for PR 27086 at commit c306711.

• This patch fails due to an unknown error code, -9.
• This patch merges cleanly.
• This patch adds no public classes.

[HeartSaVioR]

replaced by #27164