feat: ✨ Metadata + license validators + bug patches

[slugb0t]

Summary by Sourcery

Implement metadata and license validation features, including re-validation through the UI, and enhance metadata processing and validation for codemeta.json and CITATION.cff files. Fix CWL file filtering issues and update build requirements to include jsonschema.

New Features:

• Introduce metadata and license validation capabilities, allowing for re-validation of metadata files and licenses through the UI.
• Add support for validating codemeta.json and CITATION.cff files using a new schema and validation endpoints.

Bug Fixes:

• Fix issues related to filtering what is considered a CWL file in the repository.

Enhancements:

• Improve metadata handling by introducing functions to convert and apply metadata from CITATION.cff and codemeta.json files to the database.
• Enhance the validation process for metadata files by integrating external validation services and storing validation messages in the database.

Build:

• Add jsonschema to the validator's requirements to support JSON schema validation.

[slugb0t]

@sourcery-ai review

[sourcery-ai]

Hey @slugb0t - I've reviewed your changes and found some issues that need to be addressed.

Blocking issues:

• The function name convertMetadataForDB was renamed but this reference wasn't updated (link)

Overall Comments:

• Consider adding more specific error messages in the validation endpoints to help users better understand and fix validation failures

Here's what I looked at during the review

• 🔴 General issues: 1 blocking issue, 2 other issues
• 🟢 Security: all looks good
• 🟢 Testing: all looks good
• 🟡 Complexity: 1 issue found
• 🟢 Documentation: all looks good

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[sourcery-ai]

issue (bug_risk): The function name convertMetadataForDB was renamed but this reference wasn't updated

This will cause a runtime error since the function was renamed to convertCodemetaForDB. Update this reference to use the new name.

[sourcery-ai]

nitpick (typo): There's a typo in 'Introducde' - should be 'Introduce'

[sourcery-ai]

nitpick (typo): Grammatical error: 'what is consider' should be 'what is considered'

[sourcery-ai]

issue (complexity): Consider extracting a reusable helper function for merging author metadata to reduce code duplication.

The author merging logic in applyCodemetaMetadata and applyCitationMetadata could be simplified by extracting a reusable helper function:

function mergeAuthors(existingAuthors, newAuthors) {
  const merged = [...existingAuthors];

  newAuthors.forEach(newAuthor => {
    const existingIdx = merged.findIndex(existing => 
      existing.familyName === newAuthor.familyName && 
      existing.givenName === newAuthor.givenName
    );

    if (existingIdx >= 0) {
      // Update existing author
      merged[existingIdx] = {
        ...merged[existingIdx],
        affiliation: merged[existingIdx].affiliation || newAuthor.affiliation || "",
        email: merged[existingIdx].email || newAuthor.email || "",
        roles: merged[existingIdx].roles || newAuthor.roles || [],
        uri: merged[existingIdx].uri || newAuthor.uri || ""
      };
    } else {
      // Add new author with defaults
      merged.push({
        ...newAuthor,
        affiliation: newAuthor.affiliation || "",
        email: newAuthor.email || "",
        roles: newAuthor.roles || [],
        uri: newAuthor.uri || ""
      });
    }
  });

  return merged;
}

This simplifies the author merging in both metadata functions to a single line:

metadata.authors = mergeAuthors(metadata.authors, convertedCodemeta.authors);

The helper function handles all the author matching, field merging and defaults in one place, making the code more maintainable while preserving the exact same functionality.

[sourcery-ai]

suggestion (code-quality): Merge nested if conditions (merge-nested-ifs)

Suggested change

	if (existingCWL) {
	// TODO: Check for removed files
	// // Check for removed files
	// const removedFiles = existingCWL.files.filter((file) => {
	// return !cwlFiles.some((newFile) => newFile.path === file.path)
	// });
	// if (removedFiles.length > 0) {
	// cwlObject.removed_files = removedFiles.map((file) => file.path);
	// }
	if (existingCWL?.contains_cwl_files) {
	cwlObject.contains_cwl_files = existingCWL.contains_cwl_files;
	}
	}
	if (existingCWL && existingCWL?.contains_cwl_files) {
	cwlObject.contains_cwl_files = existingCWL.contains_cwl_files;
	}

Explanation

Reading deeply nested conditional code is confusing, since you have to keep track of which
conditions relate to which levels. We therefore strive to reduce nesting where
possible, and the situation where two if conditions can be combined using
and is an easy win.

[sourcery-ai]

suggestion (code-quality): Avoid unneeded ternary statements (simplify-ternary)

Suggested change

	const license = licenseRequest.data.license ? true : false;
	const license = !!licenseRequest.data.license;

Explanation

It is possible to simplify certain ternary statements into either use of an || or !.
This makes the code easier to read, since there is no conditional logic.

[sourcery-ai]

suggestion (code-quality): Merge nested if conditions (merge-nested-ifs)

Suggested change

	if (metadata.authors) {
	// consola.info("metadata.authors", metadata.authors);
	// consola.info("convertedCodemeta.authors", convertedCodemeta.authors);
	// Check if authors are already in the metadata, if so update the details of the author
	if (convertedCodemeta.authors.length > 0) {
	const updatedAuthors = convertedCodemeta.authors.map((author) => {
	const foundAuthor = metadata.authors.find((newAuthor) => newAuthor?.familyName === author?.familyName && newAuthor.givenName === author.givenName);
	if (foundAuthor) {
	// consola.info("Found author:", foundAuthor);
	author.affiliation = foundAuthor.affiliation || author.affiliation || "";
	author.email = foundAuthor.email || author.email || "";
	author.roles = foundAuthor.roles || author.roles || [];
	author.uri = foundAuthor.uri || author.uri || "";
	}
	return author;
	});
	// join the updated authors with the remaining authors
	metadata.authors = updatedAuthors;
	}
	}
	if (metadata.authors && convertedCodemeta.authors.length > 0) {
	const updatedAuthors = convertedCodemeta.authors.map((author) => {
	const foundAuthor = metadata.authors.find((newAuthor) => newAuthor?.familyName === author?.familyName && newAuthor.givenName === author.givenName);
	if (foundAuthor) {
	// consola.info("Found author:", foundAuthor);
	author.affiliation = foundAuthor.affiliation || author.affiliation || "";
	author.email = foundAuthor.email || author.email || "";
	author.roles = foundAuthor.roles || author.roles || [];
	author.uri = foundAuthor.uri || author.uri || "";
	}
	return author;
	});
	// join the updated authors with the remaining authors
	metadata.authors = updatedAuthors;
	}

Explanation

Reading deeply nested conditional code is confusing, since you have to keep track of which
conditions relate to which levels. We therefore strive to reduce nesting where
possible, and the situation where two if conditions can be combined using
and is an easy win.

[sourcery-ai]

suggestion (code-quality): Merge nested if conditions (merge-nested-ifs)

Suggested change

	if (metadata.contributors) {
	// Check if contributors are already in the metadata, if so update the details of the contributor
	if (convertedCodemeta.contributors.length > 0) {
	const updatedContributors = convertedCodemeta.contributors.map((contributor) => {
	const foundContributor = metadata.contributors.find((newContributor) => newContributor.familyName === contributor.familyName && newContributor.givenName === contributor.givenName);
	if (foundContributor) {
	contributor.affiliation = foundContributor.affiliation || contributor.affiliation || "";
	contributor.email = foundContributor.email || contributor.email || "";
	contributor.roles = foundContributor.roles || contributor.roles || [];
	contributor.uri = foundContributor.uri || contributor.uri || "";
	}
	return contributor;
	});
	metadata.contributors = updatedContributors;
	}
	}
	if (metadata.contributors && convertedCodemeta.contributors.length > 0) {
	const updatedContributors = convertedCodemeta.contributors.map((contributor) => {
	const foundContributor = metadata.contributors.find((newContributor) => newContributor.familyName === contributor.familyName && newContributor.givenName === contributor.givenName);
	if (foundContributor) {
	contributor.affiliation = foundContributor.affiliation || contributor.affiliation || "";
	contributor.email = foundContributor.email || contributor.email || "";
	contributor.roles = foundContributor.roles || contributor.roles || [];
	contributor.uri = foundContributor.uri || contributor.uri || "";
	}
	return contributor;
	});
	metadata.contributors = updatedContributors;
	}

Explanation

Reading deeply nested conditional code is confusing, since you have to keep track of which
conditions relate to which levels. We therefore strive to reduce nesting where
possible, and the situation where two if conditions can be combined using
and is an easy win.

[sourcery-ai]

suggestion (code-quality): Merge nested if conditions (merge-nested-ifs)

Suggested change

	if (convertedCitation.authors) {
	// Check if the authors are already in the metadata, if so update the details of the author
	if (metadata.authors.length > 0) {
	const updatedAuthors = metadata.authors.map((author) => {
	const foundAuthor = convertedCitation.authors.find((newAuthor) => newAuthor.familyName === author.familyName && newAuthor.givenName === author.givenName);
	if (foundAuthor) {
	author.affiliation = author.affiliation || foundAuthor.affiliation || "";
	author.email = author.email || foundAuthor.email || "";
	// Remove author from convertedCitation.authors
	const index = convertedCitation.authors.indexOf(foundAuthor);
	if (index > -1) {
	convertedCitation.authors.splice(index, 1);
	}
	}
	return author;
	});
	// Apply the missings fields to the remaining convertedCitation.authors
	convertedCitation.authors = convertedCitation.authors.map((author) => {
	author.affiliation = author.affiliation || "";
	author.email = author.email || "";
	author.roles = author.roles || [];
	author.uri = author.uri || "";
	return author;
	});
	metadata.authors = updatedAuthors.concat(convertedCitation.authors);
	}
	}
	if (convertedCitation.authors && metadata.authors.length > 0) {
	const updatedAuthors = metadata.authors.map((author) => {
	const foundAuthor = convertedCitation.authors.find((newAuthor) => newAuthor.familyName === author.familyName && newAuthor.givenName === author.givenName);
	if (foundAuthor) {
	author.affiliation = author.affiliation || foundAuthor.affiliation || "";
	author.email = author.email || foundAuthor.email || "";
	// Remove author from convertedCitation.authors
	const index = convertedCitation.authors.indexOf(foundAuthor);
	if (index > -1) {
	convertedCitation.authors.splice(index, 1);
	}
	}
	return author;
	});
	// Apply the missings fields to the remaining convertedCitation.authors
	convertedCitation.authors = convertedCitation.authors.map((author) => {
	author.affiliation = author.affiliation || "";
	author.email = author.email || "";
	author.roles = author.roles || [];
	author.uri = author.uri || "";
	return author;
	});
	metadata.authors = updatedAuthors.concat(convertedCitation.authors);
	}

Explanation

Reading deeply nested conditional code is confusing, since you have to keep track of which
conditions relate to which levels. We therefore strive to reduce nesting where
possible, and the situation where two if conditions can be combined using
and is an easy win.

[sourcery-ai]

issue (code-quality): We've found these issues:

• Use f-string instead of string concatenation [×3] (use-fstring-for-concatenation)
• Simplify conditional into switch-like form [×3] (switch)

[fairdataihub-bot]

Thanks for making updates to your pull request. Our team will take a look and provide feedback as soon as possible. Please wait for any GitHub Actions to complete before editing your pull request. If you have any additional questions or concerns, feel free to let us know. Thank you for your contributions!

[fairdataihub-bot]

Thanks for closing this pull request! If you have any further questions, please feel free to open a new issue. We are always happy to help!