feat: ✨ Codefair v3.2.0

[slugb0t]

Summary by Sourcery

Introduce metadata and license validation features, enhance the UI for displaying validation results, and set up deployment for the validator service using Kamal and Docker.

New Features:

• Introduce metadata and license validation capabilities, allowing for re-validation of metadata files and licenses through the UI.
• Add support for validating codemeta.json and CITATION.cff files using a new schema and validation endpoints.

Bug Fixes:

• Fix issues related to filtering what is considered a CWL file in the repository.

Enhancements:

• Improve metadata handling by introducing functions to convert and validate metadata from codemeta.json and CITATION.cff files.
• Enhance the dashboard UI to display metadata validation results and provide options for re-validation.

CI:

• Add CI workflows for deploying the validator service using Kamal and Docker.

Deployment:

• Set up deployment configurations for the validator service using Kamal, including Docker and Azure Container Registry integrations.

Documentation:

• Update the README for the validator service to include setup and running instructions.

Tests:

• Add a new schema for codemeta.json validation and implement validation logic for CITATION.cff files.

[fairdataihub-bot]

Thank you for submitting this pull request! We appreciate your contribution to the project. Before we can merge it, we need to review the changes you've made to ensure they align with our code standards and meet the requirements of the project. We'll get back to you as soon as we can with feedback. Thanks again!

[sourcery-ai]

Reviewer's Guide by Sourcery

This pull request implements Codefair v3.2.0, introducing metadata validation capabilities and improving the handling of metadata files. The changes include a new validator service for codemeta.json and CITATION.cff files, enhanced error handling, and UI improvements for displaying validation results.

Class diagram for metadata conversion functions

classDiagram
    class MetadataConverter {
        +convertCodemetaForDB(codemetaContent, repository)
        +convertCitationForDB(citationContent, repository)
        +validateMetadata(metadataInfo, fileType, repository)
        +applyCodemetaMetadata(codemeta, metadata, repository)
        +applyCitationMetadata(citation, metadata, repository)
    }
    class DatabaseInstance {
        +codeMetadata
        +licenseRequest
        +cwlValidation
    }
    MetadataConverter --> DatabaseInstance : uses

Loading

File-Level Changes

Change	Details	Files
Added a new validator service for metadata files	Created a Flask-based validator service with endpoints for validating CWL, CITATION.cff, and codemeta.json files Implemented JSON schema validation for codemeta.json files Added Docker configuration for the validator service Set up deployment configuration using Kamal	validator/app.py validator/apis/__init__.py validator/codemeta-schema.json validator/Dockerfile validator/config/deploy.yml
Enhanced metadata validation and handling in the bot	Added validation message storage in the database Improved error handling for metadata processing Implemented separate validation for codemeta.json and CITATION.cff files Added revalidation capabilities for metadata files	bot/metadata/index.js bot/prisma/migrations/20241114193552_metadata_validation_messages/migration.sql
Added UI components for displaying metadata validation results	Created a new metadata validation results page Added validation status indicators in the dashboard Implemented revalidation controls in the UI Added API endpoints for fetching validation results	ui/pages/dashboard/[owner]/[repo]/view/metadata-validation.vue ui/server/api/[owner]/[repo]/metadata-validation/index.get.ts ui/pages/dashboard/[owner]/[repo]/index.vue

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time. You can also use
  this command to specify where the summary should be inserted.

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[fairdataihub-bot]

Thanks for making updates to your pull request. Our team will take a look and provide feedback as soon as possible. Please wait for any GitHub Actions to complete before editing your pull request. If you have any additional questions or concerns, feel free to let us know. Thank you for your contributions!

[sourcery-ai]

Hey @slugb0t - I've reviewed your changes and found some issues that need to be addressed.

Blocking issues:

• Remove debug logging statement (link)

Here's what I looked at during the review

• 🟡 General issues: 3 issues found
• 🔴 Security: 1 blocking issue, 2 other issues
• 🟢 Testing: all looks good
• 🟡 Complexity: 1 issue found
• 🟢 Documentation: all looks good

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[sourcery-ai]

🚨 suggestion (security): CWL file path validation needs to handle special characters

Add validation/sanitization for file paths that may contain special characters to prevent potential issues with file handling

Suggested change

	if (file.type === "file" && file.name.endsWith(".cwl")) {
	cwlObject.files.push(file);
	}
	// Helper function to validate file paths
	const isValidFilePath = (filename) => {
	// Check for common problematic characters and patterns
	const invalidChars = /[<>:"|?*\x00-\x1F]/g;
	const hasInvalidChars = invalidChars.test(filename);
	const hasDoubleDots = filename.includes('..');
	const hasLeadingSlash = filename.startsWith('/');
	return !hasInvalidChars && !hasDoubleDots && !hasLeadingSlash;
	};
	if (file.type === "file" && file.name.endsWith(".cwl")) {
	if (isValidFilePath(file.name)) {
	cwlObject.files.push(file);
	} else {
	console.warn(`Skipping file with invalid path: ${file.name}`);
	}
	}

[sourcery-ai]

🚨 suggestion (security): Add security headers to Flask application

Consider adding security headers like X-Content-Type-Options, X-Frame-Options, and Content-Security-Policy. These help protect against common web vulnerabilities.

Suggested implementation:

    # Only allow CORS origin for localhost:3000
    # and any subdomain of azurestaticapps.net/

    @app.after_request
    def add_security_headers(response):
        """Add security headers to each response."""
        # Prevent MIME type sniffing
        response.headers['X-Content-Type-Options'] = 'nosniff'
        # Prevent clickjacking
        response.headers['X-Frame-Options'] = 'SAMEORIGIN'
        # Enable XSS filter
        response.headers['X-XSS-Protection'] = '1; mode=block'
        # Strict CSP
        response.headers['Content-Security-Policy'] = "default-src 'self'; script-src 'self' 'unsafe-inline' 'unsafe-eval'; style-src 'self' 'unsafe-inline';"
        # HSTS (uncomment if you're using HTTPS)
        # response.headers['Strict-Transport-Security'] = 'max-age=31536000; includeSubDomains'
        return response

    CORS(

Note: The Content-Security-Policy header provided is a basic secure default. You may need to adjust it based on your application's specific needs, such as:

1. Adding additional domains to script-src if you use external JavaScript
2. Configuring img-src if you load images from external sources
3. Adding connect-src directives for your API endpoints
4. Uncommenting HSTS if you're using HTTPS (recommended for production)

[sourcery-ai]

issue (typo): Grammar issue: 'consider' should be 'considered'

Suggested change

	- Fixed issues related to filtering what is consider a cwl file in the repository.
	- Fixed issues related to filtering what is considered a cwl file in the repository.

[sourcery-ai]

issue (typo): 'api' should be capitalized as 'API'

Suggested change

	If you would like to update the api, please follow the instructions below.
	If you would like to update the API, please follow the instructions below.

[sourcery-ai]

issue (complexity): Consider extracting duplicated person handling logic into a reusable function to process both authors and contributors.

The person handling logic in convertCodemetaForDB is overly complex with duplicated code for authors and contributors. Consider extracting this into a reusable function:

function processPersons(persons, type = 'author') {
  const processed = [];

  // Process base person records
  persons?.forEach(person => {
    if (person.type === "Person" || person.type === "Organization") {
      processed.push({
        affiliation: person?.affiliation?.name || "",
        email: person?.email || "",
        familyName: person?.familyName || "",
        givenName: person?.givenName || "",
        roles: [],
        uri: person?.id || ""
      });
    }
  });

  // Process roles
  persons?.forEach(person => {
    if (person.type === "Role") {
      processed.forEach(proc => {
        const personId = type === 'author' ? person?.["schema:author"] : 
          (person?.contributor || person?.["schema:contributor"]);
        if (proc.uri === personId) {
          proc.roles.push({
            role: person.roleName || "",
            startDate: person.startDate ? convertDateToUnix(person.startDate) : null,
            endDate: person.endDate ? convertDateToUnix(person.endDate) : null
          });
        }
      });
    }
  });

  return processed;
}

This would simplify the main function:

const sortedAuthors = processPersons(codemetaContent?.author, 'author');
const sortedContributors = processPersons(codemetaContent?.contributor, 'contributor');

This refactoring preserves all functionality while making the code more maintainable and easier to test.

[sourcery-ai]

suggestion (code-quality): Avoid unneeded ternary statements (simplify-ternary)

Suggested change

	const license = licenseRequest.data.license ? true : false;
	const license = !!licenseRequest.data.license;

Explanation

It is possible to simplify certain ternary statements into either use of an || or !.
This makes the code easier to read, since there is no conditional logic.

[sourcery-ai]

suggestion (code-quality): Merge nested if conditions (merge-nested-ifs)

Suggested change

	if (metadata.authors) {
	if (convertedCodemeta.authors.length > 0) {
	const updatedAuthors = convertedCodemeta.authors.map((author) => {
	// Find a matching author in metadata
	const foundAuthor = metadata.authors.find(
	(existingAuthor) =>
	existingAuthor?.familyName === author?.familyName &&
	existingAuthor?.givenName === author?.givenName
	);
	if (foundAuthor) {
	// Merge roles, avoiding duplicates based on `role` and `startDate`
	if (!foundAuthor?.roles) {
	foundAuthor.roles = [];
	}
	const mergedRoles = [
	...foundAuthor.roles,
	...author.roles.filter(
	(newRole) =>
	!foundAuthor.roles.some(
	(existingRole) =>
	existingRole.role === newRole.role &&
	existingRole.startDate === newRole.startDate
	)
	)
	];
	// Merge and prioritize data from `author`
	return {
	...foundAuthor,
	...author,
	affiliation: author.affiliation || foundAuthor.affiliation || "",
	email: author.email || foundAuthor.email || "",
	uri: author.uri || foundAuthor.uri || "",
	roles: mergedRoles
	};
	}
	// If no match, return the current author from convertedCodemeta
	return author;
	});
	// Merge updated authors with any authors in metadata not present in convertedCodemeta
	const nonUpdatedAuthors = metadata.authors.filter(
	(existingAuthor) =>
	!convertedCodemeta.authors.some(
	(author) =>
	author.familyName === existingAuthor.familyName &&
	author.givenName === existingAuthor.givenName
	)
	);
	metadata.authors = [...nonUpdatedAuthors, ...updatedAuthors];
	}
	}
	if (metadata.authors && convertedCodemeta.authors.length > 0) {
	const updatedAuthors = convertedCodemeta.authors.map((author) => {
	// Find a matching author in metadata
	const foundAuthor = metadata.authors.find(
	(existingAuthor) =>
	existingAuthor?.familyName === author?.familyName &&
	existingAuthor?.givenName === author?.givenName
	);
	if (foundAuthor) {
	// Merge roles, avoiding duplicates based on `role` and `startDate`
	if (!foundAuthor?.roles) {
	foundAuthor.roles = [];
	}
	const mergedRoles = [
	...foundAuthor.roles,
	...author.roles.filter(
	(newRole) =>
	!foundAuthor.roles.some(
	(existingRole) =>
	existingRole.role === newRole.role &&
	existingRole.startDate === newRole.startDate
	)
	)
	];
	// Merge and prioritize data from `author`
	return {
	...foundAuthor,
	...author,
	affiliation: author.affiliation || foundAuthor.affiliation || "",
	email: author.email || foundAuthor.email || "",
	uri: author.uri || foundAuthor.uri || "",
	roles: mergedRoles
	};
	}
	// If no match, return the current author from convertedCodemeta
	return author;
	});
	// Merge updated authors with any authors in metadata not present in convertedCodemeta
	const nonUpdatedAuthors = metadata.authors.filter(
	(existingAuthor) =>
	!convertedCodemeta.authors.some(
	(author) =>
	author.familyName === existingAuthor.familyName &&
	author.givenName === existingAuthor.givenName
	)
	);
	metadata.authors = [...nonUpdatedAuthors, ...updatedAuthors];
	}

Explanation

Reading deeply nested conditional code is confusing, since you have to keep track of which
conditions relate to which levels. We therefore strive to reduce nesting where
possible, and the situation where two if conditions can be combined using
and is an easy win.

[sourcery-ai]

issue (code-quality): We've found these issues:

• Use f-string instead of string concatenation [×3] (use-fstring-for-concatenation)
• Simplify conditional into switch-like form [×3] (switch)

[fairdataihub-bot]

Thanks for closing this pull request! If you have any further questions, please feel free to open a new issue. We are always happy to help!