Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add json format support for log export via faro receiver #2276

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

ravishankar15
Copy link
Contributor

PR Description

Which issue(s) this PR fixes

Fixes #1973

Notes to the Reviewer

PR Checklist

  • CHANGELOG.md updated
  • Documentation added
  • Tests updated
  • Config converters updated

@clayton-cornell clayton-cornell added the type/docs Docs Squad label across all Grafana Labs repos label Dec 12, 2024
@clayton-cornell clayton-cornell requested a review from a team December 13, 2024 18:12
@clayton-cornell
Copy link
Contributor

clayton-cornell commented Dec 13, 2024

Docs look as for now. Over to @grafana/grafana-alloy-maintainers for a code review.

@ptodev ptodev self-assigned this Dec 16, 2024
Copy link
Contributor

@ptodev ptodev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@ptodev
Copy link
Contributor

ptodev commented Dec 18, 2024

Hi @ravishankar15, would you mind rebasing your branch with main please?

Apologies for the delay with merging - I'll merge it after it's rebased to avoid further conflicts with the changelog :)

@ravishankar15
Copy link
Contributor Author

Hi @ptodev I have rebased the branch

func newFakeLogsReceiver(t *testing.T) *fakeLogsReceiver {
ctx := componenttest.TestContext(t)
require.Equal(t, http.StatusAccepted, resp.StatusCode)
require.Len(t, lr.GetEntries(), 1)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ravishankar15 Do you think the test could be flaky? The CI failed with an error on this line:

level=error test=Test_LogsExporter_Export/export_logfmt_for_exception_payload ts=15:59:24.922 msg="Error resolving stack trace frame source location" err="no sourcemap available"
level=error test=Test_LogsExporter_Export/export_logfmt_for_exception_payload ts=15:59:24.922 msg="Error resolving stack trace frame source location" err="no sourcemap available"
level=error test=Test_LogsExporter_Export/export_json_for_exception_payload ts=15:59:24.923 msg="Error resolving stack trace frame source location" err="no sourcemap available"
level=error test=Test_LogsExporter_Export/export_json_for_exception_payload ts=15:59:24.923 msg="Error resolving stack trace frame source location" err="no sourcemap available"
level=error test=TestMultipleExportersOneFails ts=15:59:24.937 msg="exporter failed with error" exporter=exporter1 err="this exporter is broken"
level=error test=TestMultipleExportersAllFail ts=15:59:24.937 msg="exporter failed with error" exporter=exporter1 err="this exporter is broken"
level=error test=TestMultipleExportersAllFail ts=15:59:24.938 msg="exporter failed with error" exporter=exporter2 err="this exporter is broken"
test=Test/format_logfmt ts=15:59:24.941 caller="ts=2024-12-18T15:59" msg=":24.941536215Z level=info msg=\"starting server\" subcomponent=server addr=127.0.0.1:46273"
test=Test/format_json ts=15:59:24.942 caller="ts=2024-12-18T15:59" msg=":24.942355859Z level=info msg=\"starting server\" subcomponent=server addr=127.0.0.1:32955"
test=Test/format_logfmt ts=15:59:24.953 caller="ts=2024-12-18T15:59" msg=":24.953015579Z level=info msg=\"terminating server\" subcomponent=server"
test=Test/format_json ts=15:59:24.954 caller="ts=2024-12-18T15:59" msg=":24.954537892Z level=info msg=\"terminating server\" subcomponent=server"
--- FAIL: Test (0.00s)
    --- FAIL: Test/format_logfmt (0.01s)
        receiver_test.go:136: 
            	Error Trace:	/drone/src/internal/component/faro/receiver/receiver_test.go:136
            	Error:      	"[]" should have 1 item(s), but has 0
            	Test:       	Test/format_logfmt
level=debug test=Test_sourceMapsStoreImpl_DownloadSuccess ts=15:59:24.955 msg="attempting to download source file" url=http://localhost:1234/foo.js
level=debug test=Test_sourceMapsStoreImpl_DownloadSuccess ts=15:59:24.955 msg="resolved absolute source map URL" url=http://localhost:1234/foo.js sourceMapURL=foo.js.map
level=debug test=Test_sourceMapsStoreImpl_DownloadSuccess ts=15:59:24.955 msg="attempting to download source map file" url=http://localhost:1234/foo.js.map
level=info test=Test_sourceMapsStoreImpl_DownloadSuccess ts=15:59:24.960 msg="successfully parsed source map" url=http://localhost:1234/foo.js.map release=123
level=debug test=Test_sourceMapsStoreImpl_DownloadError ts=15:59:24.960 msg="attempting to download source file" url=http://localhost:1234/foo.js
level=debug test=Test_sourceMapsStoreImpl_DownloadError ts=15:59:24.960 msg="failed to download source file" url=http://localhost:1234/foo.js err="unexpected status 500"
level=error test=Test_sourceMapsStoreImpl_DownloadError ts=15:59:24.960 msg="Error resolving stack trace frame source location" err="unexpected status 500"
level=debug test=Test_sourceMapsStoreImpl_DownloadHTTPOriginFiltering ts=15:59:24.961 msg="attempting to download source file" url=http://bar.com/foo.js
level=debug test=Test_sourceMapsStoreImpl_DownloadHTTPOriginFiltering ts=15:59:24.961 msg="resolved absolute source map URL" url=http://bar.com/foo.js sourceMapURL=foo.js.map
level=debug test=Test_sourceMapsStoreImpl_DownloadHTTPOriginFiltering ts=15:59:24.961 msg="attempting to download source map file" url=http://bar.com/foo.js.map
level=info test=Test_sourceMapsStoreImpl_DownloadHTTPOriginFiltering ts=15:59:24.962 msg="successfully parsed source map" url=http://bar.com/foo.js.map release=123
level=debug test=Test_sourceMapsStoreImpl_ReadFromFileSystem ts=15:59:24.962 msg="source map found on filesystem" url=/var/build/latest/foo.js.map file_path=/var/build/latest/foo.js.map
level=info test=Test_sourceMapsStoreImpl_ReadFromFileSystem ts=15:59:24.963 msg="successfully parsed source map" url=http://foo.com/foo.js release=123
level=debug test=Test_sourceMapsStoreImpl_ReadFromFileSystem ts=15:59:24.963 msg="source map not found on filesystem" url=http://foo.com/bar.js file_path=/var/build/latest/bar.js.map
level=debug test=Test_sourceMapsStoreImpl_ReadFromFileSystem ts=15:59:24.963 msg="source map found on filesystem" url=/var/build/123/foo.js.map file_path=/var/build/123/foo.js.map
level=info test=Test_sourceMapsStoreImpl_ReadFromFileSystem ts=15:59:24.963 msg="successfully parsed source map" url=http://bar.com/foo.js release=123
level=debug test=Test_sourceMapsStoreImpl_ReadFromFileSystemAndDownload ts=15:59:24.964 msg="source map found on filesystem" url=/var/build/latest/foo.js.map file_path=/var/build/latest/foo.js.map
level=info test=Test_sourceMapsStoreImpl_ReadFromFileSystemAndDownload ts=15:59:24.964 msg="successfully parsed source map" url=http://foo.com/foo.js release=123
level=debug test=Test_sourceMapsStoreImpl_ReadFromFileSystemAndDownload ts=15:59:24.964 msg="attempting to download source file" url=http://bar.com/foo.js
level=debug test=Test_sourceMapsStoreImpl_ReadFromFileSystemAndDownload ts=15:59:24.965 msg="resolved absolute source map URL" url=http://bar.com/foo.js sourceMapURL=foo.js.map
level=debug test=Test_sourceMapsStoreImpl_ReadFromFileSystemAndDownload ts=15:59:24.965 msg="attempting to download source map file" url=http://bar.com/foo.js.map
level=info test=Test_sourceMapsStoreImpl_ReadFromFileSystemAndDownload ts=15:59:24.965 msg="successfully parsed source map" url=http://bar.com/foo.js.map release=123
level=debug test=Test_sourceMapsStoreImpl_ReadFromFileSystemAndNotDownloadIfDisabled ts=15:59:24.966 msg="source map found on filesystem" url=/var/build/latest/foo.js.map file_path=/var/build/latest/foo.js.map
level=info test=Test_sourceMapsStoreImpl_ReadFromFileSystemAndNotDownloadIfDisabled ts=15:59:24.966 msg="successfully parsed source map" url=http://foo.com/foo.js release=123
level=debug test=Test_sourceMapsStoreImpl_FilepathSanitized ts=15:59:24.967 msg="source map not found on filesystem" url=http://foo.com/../../../etc/passwd file_path=/var/build/latest/etc/passwd.map
FAIL
FAIL	github.com/grafana/alloy/internal/component/faro/receiver	0.192s
ok  	github.com/grafana/alloy/internal/component/faro/receiver/internal/payload	

Copy link
Contributor Author

@ravishankar15 ravishankar15 Dec 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think its a flaky failure. We are spinning up two go routines

  1. To start the controller
  2. Inside newFakeLogsReceiver

For the controller go routine we wait for the server to be up. I believe that some delay in the newFakeLogsReceiver go routine could cause this failure. We can add a small sleep duration before doing GetEntries to fix this Your thoughts ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for looking into it! The best solution is usually to have a wait group so that we can do the check as soon as possible. If this is not practical or too much work, you could put in a require.EventuallyWithT.

Probably require.EventuallyWithT is the easiest option here :) If it keeps being flaky, we could try make it more resilient with a wait group.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup have made the changes Please take a look

Fix chan size
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/docs Docs Squad label across all Grafana Labs repos
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support JSON log format in faro.receiver
3 participants