Replace timestamp with time & Add document for GraphQL API `connRawEvents` #918

sophie-cluml · 2024-12-03T07:36:44Z

Close: #917
Close: #922

Below was a description in PR draft stage:

Hi @sehkone, I’ve created this draft PR to ask for your confirmation if I’m heading in the right direction. So far, I’ve only worked on the connRawEvents GraphQL API as a single case example. If the approach aligns with your expectations, I can proceed to apply the same methodology to the remaining GraphQL APIs. Let me know your thoughts.

GraphQL Playground looks like below:

codecov · 2024-12-03T07:43:42Z

Codecov Report

Attention: Patch coverage is 69.34307% with 126 lines in your changes missing coverage. Please review.

Project coverage is 77.27%. Comparing base (ffef4d1) to head (96fb059).

Files with missing lines	Patch %	Lines
src/graphql/sysmon.rs	0.00%	61 Missing ⚠️
src/graphql/export.rs	0.00%	59 Missing ⚠️
src/ingest.rs	57.14%	3 Missing ⚠️
src/graphql/netflow.rs	0.00%	2 Missing ⚠️
src/graphql/network.rs	98.36%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #918      +/-   ##
==========================================
+ Coverage   77.17%   77.27%   +0.09%     
==========================================
  Files          32       32              
  Lines       25723    25651      -72     
==========================================
- Hits        19851    19821      -30     
+ Misses       5872     5830      -42

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

sehkone · 2024-12-04T01:42:23Z

src/graphql/network.rs

    timestamp: DateTime<Utc>,
+    /// Source IP, or source IP address in long format.


Source IP address only would suffice without Source IP

What does long format means?

It is now modified to Source IP address.. Long format was what I got from consultation with ChatGPT. I don't think we need it anymore.

sehkone · 2024-12-04T01:42:43Z

src/graphql/network.rs

    orig_addr: String,
+    /// Source port, or source port number in long format.


Source port number would suffice without Source port

sehkone · 2024-12-06T00:13:34Z

src/graphql/network/tests.rs

@@ -3217,7 +3217,7 @@ async fn union() {
        }
    }"#;
    let res = schema.execute(query).await;
-    assert_eq!(res.data.to_string(), "{networkRawEvents: {edges: [{node: {timestamp: \"2019-12-31T23:59:59+00:00\", __typename: \"BootpRawEvent\"}}, {node: {timestamp: \"2020-01-01T00:00:01+00:00\", __typename: \"SshRawEvent\"}}, {node: {timestamp: \"2020-01-01T00:00:05+00:00\", __typename: \"SmtpRawEvent\"}}, {node: {timestamp: \"2020-01-01T00:01:01+00:00\", __typename: \"ConnRawEvent\"}}, {node: {timestamp: \"2020-01-05T00:01:01+00:00\", __typename: \"RdpRawEvent\"}}, {node: {timestamp: \"2020-01-05T06:05:00+00:00\", __typename: \"DceRpcRawEvent\"}}, {node: {timestamp: \"2020-06-01T00:01:01+00:00\", __typename: \"HttpRawEvent\"}}, {node: {timestamp: \"2021-01-01T00:01:01+00:00\", __typename: \"DnsRawEvent\"}}, {node: {timestamp: \"2022-01-05T00:01:01+00:00\", __typename: \"NtlmRawEvent\"}}, {node: {timestamp: \"2023-01-05T00:01:01+00:00\", __typename: \"KerberosRawEvent\"}}, {node: {timestamp: \"2023-01-05T12:12:00+00:00\", __typename: \"FtpRawEvent\"}}, {node: {timestamp: \"2023-01-05T12:12:00+00:00\", __typename: \"MqttRawEvent\"}}, {node: {timestamp: \"2023-01-06T11:11:00+00:00\", __typename: \"TlsRawEvent\"}}, {node: {timestamp: \"2023-01-06T12:12:00+00:00\", __typename: \"LdapRawEvent\"}}, {node: {timestamp: \"2023-01-06T12:12:10+00:00\", __typename: \"SmbRawEvent\"}}, {node: {timestamp: \"2023-01-06T12:13:00+00:00\", __typename: \"NfsRawEvent\"}}, {node: {timestamp: \"2023-01-06T12:13:10+00:00\", __typename: \"DhcpRawEvent\"}}]}}");
+    assert_eq!(res.data.to_string(), "{networkRawEvents: {edges: [{node: {timestamp: \"2019-12-31T23:59:59+00:00\", __typename: \"BootpRawEvent\"}}, {node: {timestamp: \"2020-01-01T00:00:01+00:00\", __typename: \"SshRawEvent\"}}, {node: {timestamp: \"2020-01-01T00:00:05+00:00\", __typename: \"SmtpRawEvent\"}}, {node: {time: \"2020-01-01T00:01:01+00:00\", __typename: \"ConnRawEvent\"}}, {node: {timestamp: \"2020-01-05T00:01:01+00:00\", __typename: \"RdpRawEvent\"}}, {node: {timestamp: \"2020-01-05T06:05:00+00:00\", __typename: \"DceRpcRawEvent\"}}, {node: {timestamp: \"2020-06-01T00:01:01+00:00\", __typename: \"HttpRawEvent\"}}, {node: {timestamp: \"2021-01-01T00:01:01+00:00\", __typename: \"DnsRawEvent\"}}, {node: {timestamp: \"2022-01-05T00:01:01+00:00\", __typename: \"NtlmRawEvent\"}}, {node: {timestamp: \"2023-01-05T00:01:01+00:00\", __typename: \"KerberosRawEvent\"}}, {node: {timestamp: \"2023-01-05T12:12:00+00:00\", __typename: \"FtpRawEvent\"}}, {node: {timestamp: \"2023-01-05T12:12:00+00:00\", __typename: \"MqttRawEvent\"}}, {node: {timestamp: \"2023-01-06T11:11:00+00:00\", __typename: \"TlsRawEvent\"}}, {node: {timestamp: \"2023-01-06T12:12:00+00:00\", __typename: \"LdapRawEvent\"}}, {node: {timestamp: \"2023-01-06T12:12:10+00:00\", __typename: \"SmbRawEvent\"}}, {node: {timestamp: \"2023-01-06T12:13:00+00:00\", __typename: \"NfsRawEvent\"}}, {node: {timestamp: \"2023-01-06T12:13:10+00:00\", __typename: \"DhcpRawEvent\"}}]}}");


timestamp here does not need to change to time?

{node: {timestamp: \"2020-01-01T00:01:01+00:00\", __typename: \"ConnRawEvent\"}}

is changed to

{node: {time: \"2020-01-01T00:01:01+00:00\", __typename: \"ConnRawEvent\"}}

The change of timestamp to time occurs here partially only for ConnRawEvent, not for other events. It is because networkRawEvents is a GraphQL query that returns an Union type, and in this case ConnRawEvent is one of the types included in that Union type response. Since this PR targets to change ConnRawEvent, only above part of code is modified.

By the way, do you think it is better to modify timestamp to time in all event protocols all at once, in this PR?

Once we decided to changed timestamp to time, I think it would be better to complete it all at once.

I understood. I created an issue regarding timestamp-to-time as #922, and this PR has been updated to resolve that issue together at once.

sophie-cluml · 2025-01-03T03:58:47Z

@sehkone 시간 괜찮으시다면 리뷰 부탁드릴 수 있을까요? 최근 push 들은 단순 rebase들 입니다.

sophie-cluml requested a review from sehkone December 3, 2024 08:30

sehkone reviewed Dec 4, 2024

View reviewed changes

sophie-cluml force-pushed the sophie/issue-917 branch 2 times, most recently from 1a5dd56 to d5f4dad Compare December 5, 2024 09:16

sophie-cluml marked this pull request as ready for review December 5, 2024 09:20

sophie-cluml changed the title ~~[Draft] Add document for GraphQL API connRawEvents~~ Add document for GraphQL API connRawEvents Dec 5, 2024

sehkone reviewed Dec 6, 2024

View reviewed changes

sophie-cluml force-pushed the sophie/issue-917 branch 2 times, most recently from 678d34e to 1c796cc Compare December 10, 2024 06:55

sophie-cluml changed the title ~~Add document for GraphQL API connRawEvents~~ Replace timestamp with time & Add document for GraphQL API connRawEvents Dec 10, 2024

sophie-cluml force-pushed the sophie/issue-917 branch from 1c796cc to d107827 Compare December 12, 2024 03:04

sophie-cluml requested a review from sehkone December 23, 2024 08:50

sophie-cluml force-pushed the sophie/issue-917 branch from d107827 to ae203ef Compare December 23, 2024 08:51

sophie-cluml added 3 commits January 3, 2025 12:57

Rename timestamp to time

e271dc9

Rename timestamp to time in interal usage

27df618

Add document for ConnRawEvent

96fb059

sophie-cluml force-pushed the sophie/issue-917 branch from ae203ef to 96fb059 Compare January 3, 2025 03:57

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Replace timestamp with time & Add document for GraphQL API `connRawEvents` #918

Replace timestamp with time & Add document for GraphQL API `connRawEvents` #918

sophie-cluml commented Dec 3, 2024 •

edited

Loading

codecov bot commented Dec 3, 2024 •

edited

Loading

sehkone Dec 4, 2024

sehkone Dec 4, 2024

sophie-cluml Dec 5, 2024

sehkone Dec 4, 2024 •

edited

Loading

sophie-cluml Dec 5, 2024

sehkone Dec 6, 2024

sophie-cluml Dec 6, 2024

sophie-cluml Dec 6, 2024

sehkone Dec 9, 2024

sophie-cluml Dec 10, 2024

sophie-cluml commented Jan 3, 2025

		timestamp: DateTime<Utc>,
		/// Source IP, or source IP address in long format.

		orig_addr: String,
		/// Source port, or source port number in long format.

Replace timestamp with time & Add document for GraphQL API connRawEvents #918

Are you sure you want to change the base?

Replace timestamp with time & Add document for GraphQL API connRawEvents #918

Conversation

sophie-cluml commented Dec 3, 2024 • edited Loading

codecov bot commented Dec 3, 2024 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sehkone Dec 4, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sophie-cluml commented Jan 3, 2025

Replace timestamp with time & Add document for GraphQL API `connRawEvents` #918

Replace timestamp with time & Add document for GraphQL API `connRawEvents` #918

sophie-cluml commented Dec 3, 2024 •

edited

Loading

codecov bot commented Dec 3, 2024 •

edited

Loading

sehkone Dec 4, 2024 •

edited

Loading