Skip to content
This repository has been archived by the owner on Jul 2, 2024. It is now read-only.

DEVPROD-5130: Case on renderingType during log ingestion, introduce LOCAL_UPLOAD logType and fix bug related to exposing renderingType from hook #509

Closed
wants to merge 22 commits into from

Conversation

SupaJoon
Copy link
Collaborator

@SupaJoon SupaJoon commented Mar 13, 2024

DEVPROD-5130

Description

These code changes:

  • Introduce LogTypes.LOCAL_UPLOAD
  • Fix useResolveLogURLAndRenderingType to correctly return the resmoke rendering type when applicable
  • Update some components to read LogType from context instead of props
  • Update the breadcrumb component to show the test name for EVG test logs.
  • Makes some changes to how logs from S3 are fetched

Testing

I unit tested the useResolveLogURLAndRenderingType hook and then asserted the correct rendering logic is utilized in E2E tests. It's probably useful to ensure ingestLogs is called with the correct rendering type in context which means we would have to unit test LoadingPage. We should actually do this eventually because LoadingPage makes guarantees about the state of context during the log loading phase of the app (DEVPROD-5891).

Copy link

cypress bot commented Mar 13, 2024

Passing run #5353 ↗︎

0 121 1 0 Flakiness 0

Details:

update story shot
Project: Parsley Commit: cf67eec2e8
Status: Passed Duration: 03:07 💡
Started: Mar 13, 2024 8:02 PM Ended: Mar 13, 2024 8:05 PM

Review all test suite changes for PR #509 ↗︎

@SupaJoon SupaJoon changed the title DEVPROD-5130: Case on renderingType during log ingestion and fix bug TS bug related to exposing renderingType DEVPROD-5130: Case on renderingType during log ingestion, introduce LOCAL_UPLOAD logType and fix bug TS bug related to exposing renderingType Mar 13, 2024
@SupaJoon SupaJoon changed the title DEVPROD-5130: Case on renderingType during log ingestion, introduce LOCAL_UPLOAD logType and fix bug TS bug related to exposing renderingType DEVPROD-5130: Case on renderingType during log ingestion, introduce LOCAL_UPLOAD logType and fix bug related to exposing renderingType from hook Mar 13, 2024
@@ -9,10 +9,9 @@ import ResmokeRow from "../ResmokeRow";

type RowRendererFunction = (props: {
processedLogLines: ProcessedLogLines;
logType: LogTypes;
Copy link
Collaborator Author

@SupaJoon SupaJoon Mar 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed this code to access logType from LogContext.logMetadata instead of a prop since the function already accesses the context. I'm open to making this change in a separate PR.

@@ -33,6 +38,7 @@ import { getNextPage } from "./utils";

interface LogContextState {
expandedLines: ExpandedLines;
isUploadedLog: boolean;
Copy link
Collaborator Author

@SupaJoon SupaJoon Mar 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This value used to be set directly via setLogMetadata when the user uploads a log. I now represent "uploaded log" with LogType.LOCAL_UPLOAD

import { useLogContext } from "context/LogContext";
import FileDropper from "./LogDrop/FileDropper";

const LogDrop = () => {
const { hasLogs, logMetadata } = useLogContext();
Copy link
Collaborator Author

@SupaJoon SupaJoon Mar 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

logMetadata is set during the log parsing steps of FileDropper and hasLogs is set to true after successful parsing and ingestion. This makes it appropriate for LogWindow to access logMetadata directly via context.

renderingType === LogRenderingTypes.Resmoke
? LogTypes.RESMOKE_LOGS
: LogTypes.EVERGREEN_TASK_LOGS;
const logType = LogTypes.LOCAL_UPLOAD;
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I adjusted the code so LogTypes directly represents log origin and LogRenderingType represents how it should be rendered.

) : (
<LogWindow logType={logType} />
);
return hasLogs === null ? <LoadingPage logType={logType} /> : <LogWindow />;
Copy link
Collaborator Author

@SupaJoon SupaJoon Mar 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LoadingPage needs logType to save it into logMetadata. By the time LogWindow is rendered, that value is available via context.

@SupaJoon SupaJoon marked this pull request as ready for review March 27, 2024 13:51
@@ -458,17 +458,16 @@ functions:
content_type: text/plain
permissions: public-read

seed-logkeeper:
seed-bucket-data:
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update the name to help phase out Logkeeper.

echo "${YELLOW}LK_CORS_ORIGINS=http:\/\/localhost:\\\d+ LK_EVERGREEN_ORIGIN=http://localhost:8080 LK_PARSLEY_ORIGIN=http://localhost:5173 go run main/logkeeper.go --localPath $PWD/bin/_bucketdata${NC}"

echo "Create symlink in your local evergreen directory:"
echo "ln -s $PWD/bin/_bucketdata _bucketdata"
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

EVG expects bucketdata to exist in the EVG directory and Logkeeper will expect it to exist at the dir defined in --localPath

const { setWrap } = preferences;

useEffect(() => {
ingestLines(ansiLogLines, LogTypes.EVERGREEN_TASK_LOGS);
setLogMetadata({ logType: LogTypes.EVERGREEN_TASK_LOGS });
Copy link
Collaborator Author

@SupaJoon SupaJoon Mar 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I updated some components to access logType from context instead of props after data is set in the "loading" phase of the app. This decreases data management complexity. Props and context data shouldn't be used interchangeably because they can diverge.

buildID,
execution,
logType,
taskID,
});

if (loading || !task) {
const { data: testData, loading: isLoadingTest } = useQuery<
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Breadcrumb now shows the test name in the trailing breadcrumb for EVG tests.

buildID,
execution,
fileName,
isUploadedLog,
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pulled this value out of logMetadata because it maps on logMetadata.logType.

@@ -204,7 +204,11 @@ export const useResolveLogURLAndRenderingType = ({
downloadURL = rawLogURL;
if (!renderingTypeFromQuery) {
renderingType = LogRenderingTypes.Default;
} else if (renderingTypeFromQuery in LogRenderingTypes) {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This works only on non-const, number-based enums

@SupaJoon SupaJoon requested a review from a team March 27, 2024 14:43
Copy link
Collaborator

@sophstad sophstad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any chance you could open this in the new repo? The Evergreen YAML changes in particular may be a bit different and I think it would be a lot easier to centralize the review process 🙏 let me know if I can help at all!

@SupaJoon
Copy link
Collaborator Author

SupaJoon commented Apr 2, 2024

Ported this PR to evergreen-ci/ui

@SupaJoon SupaJoon closed this Apr 2, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants