Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Ingestion of CMV logs into ELK #54

Open
cessda-bitbucket-importer opened this issue Apr 15, 2021 · 0 comments
Open

Ingestion of CMV logs into ELK #54

cessda-bitbucket-importer opened this issue Apr 15, 2021 · 0 comments
Labels
major task use case

Comments

@cessda-bitbucket-importer
Copy link
Contributor

Original report on BitBucket by Joshua Tetteh Ocansey (GitHub: Joshocan).

CESSDA Logging guideline can serve as a guide

Keys Points

  • Structure logging format as JSON
  • The default logging level is WARNING
  • Parameterize and contextualize logs for easier analysis ( Example are seen in the guideline above)

‌

What to log:

  • Number of metadata format validated per validation gate
  • Number of metadata format validated by predefined profile type
  • Number of metadata format validated by Url profile
  • Number of metadata format validated by upload profile
  • Number of metadata format validated per document type
  • Errors/exceptions generated during validation
  • Duration in seconds for Validating metadata
  • Other systems exceptions generated per seconds

‌
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
major task use case
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@matthew-morris-cessda @cessda-bitbucket-importer