From 93a67a1d5a9d8ed31b13912c178b627bdf357ecc Mon Sep 17 00:00:00 2001 From: mariojmdavid Date: Tue, 23 Nov 2021 18:15:14 +0000 Subject: [PATCH] small type corrections, markdown line format --- content/01.abstract.md | 7 +- content/02.0.copyright-ack.md | 11 +- content/02.document-log.md | 2 +- content/03.intro.md | 26 +- content/04.goals.md | 4 +- content/05.notational_conventions.md | 6 +- content/06.quality_criteria.md | 339 +++++++++++++-------------- content/08.annex.md | 66 +++--- 8 files changed, 229 insertions(+), 232 deletions(-) diff --git a/content/01.abstract.md b/content/01.abstract.md index 9c86933..0ddf1c7 100644 --- a/content/01.abstract.md +++ b/content/01.abstract.md @@ -1,6 +1,5 @@ # Abstract {.page_break_before} -The purpose of this document is to define a set of quality standards, -procedures and best practices to conform a Software Quality Assurance plan to -serve as a reference within the European research ecosystem related projects -for the adequate development and timely delivery of software products. +The purpose of this document is to define a set of quality standards, procedures and best practices +to conform a Software Quality Assurance plan to serve as a reference within the European research +ecosystem related projects for the adequate development and timely delivery of software products. diff --git a/content/02.0.copyright-ack.md b/content/02.0.copyright-ack.md index a6895db..c43829f 100644 --- a/content/02.0.copyright-ack.md +++ b/content/02.0.copyright-ack.md @@ -1,14 +1,13 @@ # Copyright Notice -Copyright © Members of the INDIGO-DataCloud, DEEP Hybrid-DataCloud eXtreme -DataCloud and EOSC-Synergy collaborations, 2015-2021. +Copyright © Members of the INDIGO-DataCloud, DEEP Hybrid-DataCloud eXtreme DataCloud and +EOSC-Synergy collaborations, 2015-2021. # Acknowledgements -The INDIGO-DataCloud, DEEP-Hybrid-DataCloud, eXtreme-DataCloud and -EOSC-Synergy projects have received funding from the European Union’s -Horizon 2020 research and innovation programme under grant agreement -number 653549, 777435, 777367 and 857647 respectively. +The INDIGO-DataCloud, DEEP-Hybrid-DataCloud, eXtreme-DataCloud and EOSC-Synergy projects have +received funding from the European Union’s Horizon 2020 research and innovation programme under +grant agreement number 653549, 777435, 777367 and 857647 respectively.

diff --git a/content/02.document-log.md b/content/02.document-log.md index a313a22..c11dae6 100644 --- a/content/02.document-log.md +++ b/content/02.document-log.md @@ -8,4 +8,4 @@ | v3.1 | 05/03/2020 | Add tags/names for each criteria | | v3.2 | 23/04/2020 | Add EOSC-Synergy to copyright | | v3.3 | 15/10/2020 | Fix issues: #32, #46, #47, #48, #49, #51 | -| v4.0 | 2021 | Add annex, spell check, Fix issues: #7, #57, #59, #60, #61, #63, #64 | +| v4.0 | 12/2021 | Add annex, spell check, Fix issues: #7, #35, #50, #57, #59, #60, #61, #63, #64, #65, #66 | diff --git a/content/03.intro.md b/content/03.intro.md index 1b9c5d6..dd0a074 100644 --- a/content/03.intro.md +++ b/content/03.intro.md @@ -1,14 +1,16 @@ # 1. Introduction and Purpose {.page_break_before} -This document has been tailored upon the recommendations and requirements found -in the Initial Plan for Software Management and Pilot Services deliverable -[@url:https://owncloud.indigo-datacloud.eu/index.php/s/yDklCrWjKnjutVA], -produced by the INDIGO-DataCloud project. These guidelines evolved throughout -the project’s lifetime and are being extended in the -EOSC-Synergy [@doi:10.20350/digitalCSIC/12607], -as well as as the past DEEP-Hybrid-DataCloud and eXtreme DataCloud subsequent projects. -The result is a consolidated Software Quality Assurance (SQA) baseline criteria emanated -from the European Open Science Cloud (EOSC), which aims to outline the SQA -principles to be considered in the upcoming software development efforts within -the European research community, and continuously evolve in order to be aligned -with future software engineering practices and security recommendations. +This document has been tailored upon the recommendations and requirements found in the Initial Plan +for Software Management and Pilot Services deliverable +[@url:https://owncloud.indigo-datacloud.eu/index.php/s/yDklCrWjKnjutVA], produced by the +INDIGO-DataCloud project. + +These guidelines evolved throughout the project’s lifetime and are being extended in the +EOSC-Synergy [@doi:10.20350/digitalCSIC/12607], as well as the past DEEP-Hybrid-DataCloud and +eXtreme DataCloud subsequent projects. + +The result is a consolidated Software Quality Assurance (SQA) baseline criteria emanated from the +European Open Science Cloud (EOSC), which aims to outline the SQA principles to be considered in the +upcoming software development efforts within the European research community, and continuously +evolve in order to be aligned with future software engineering practices and security +recommendations. diff --git a/content/04.goals.md b/content/04.goals.md index 9eb7e41..cf02341 100644 --- a/content/04.goals.md +++ b/content/04.goals.md @@ -9,8 +9,8 @@ 3. Promote code style standards to deliver good quality source code emphasizing its readability and reusability. -4. Improve the quality and reliability of software by covering different - testing methods at development and pre-production stages. +4. Improve the quality and reliability of software by covering different testing methods at + development and pre-production stages. 5. Propose a change-based driven scenario where all new updates in the source code are continuously validated by the automated execution of the relevant tests. diff --git a/content/05.notational_conventions.md b/content/05.notational_conventions.md index 1553dbc..34cfc9b 100644 --- a/content/05.notational_conventions.md +++ b/content/05.notational_conventions.md @@ -1,5 +1,5 @@ # 3. Notational Conventions -The keywords “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, -“SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be -interpreted as described in RFC 2119 [@https://www.rfc-editor.org/info/rfc2119]. +The keywords "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", +"RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 +[@https://www.rfc-editor.org/info/rfc2119]. diff --git a/content/06.quality_criteria.md b/content/06.quality_criteria.md index d08abc9..1d264fa 100644 --- a/content/06.quality_criteria.md +++ b/content/06.quality_criteria.md @@ -1,100 +1,96 @@ # 4. Quality Criteria {.page_break_before} -The following sections describe the quality conventions and best practices that -apply to the development phase of a software component within the EOSC -ecosystem. These guidelines ruled the software development process of the -former European Commission-funded project INDIGO-DataCloud, where they have -proved valuable for improving the reliability of software produced in the -scientific European arena. - -The next sections describe the development process driven by a change-based -strategy, followed by a continuous integration approach. Changes in the source -code, trigger automated builds to analyze the new contributions in order to -validate them before being added to the software component code base. -Consequently, software components are more eligible for being deployed in -production infrastructures, reducing the likelihood of service disruption. +The following sections describe the quality conventions and best practices that apply to the +development phase of a software component within the EOSC ecosystem. These guidelines ruled the +software development process of the former European Commission-funded project INDIGO-DataCloud, +where they have proved valuable for improving the reliability of software produced in the scientific +European arena. + +The next sections describe the development process driven by a change-based strategy, followed by a +continuous integration approach. Changes in the source code, trigger automated builds to analyze the +new contributions in order to validate them before being added to the software component code base. +Consequently, software components are more eligible for deployment in production infrastructures, +reducing the likelihood of service disruption. ## 4.1. Code Accessibility [QC.Acc] -* **[QC.Acc01]** Following the open-source model, the source code being produced MUST be open - and publicly available to promote the adoption and augment the visibility of - the software developments. +* **[QC.Acc01]** Following the open-source model, the source code being produced MUST be open and + publicly available to promote the adoption and augment the visibility of the software + developments. * **[QC.Acc02]** Source code MUST use a Version Control System (VCS). - * **[QC.Acc02.1]** It is RECOMMENDED that all software components delivered by the same - project agree on a common VCS. + * **[QC.Acc02.1]** It is RECOMMENDED that all software components delivered by the same project + agree on a common VCS. -* **[QC.Acc03]** Source code produced within the scope of a broader development project - SHOULD reside in a common organization of a version control repository - hosting service. +* **[QC.Acc03]** Source code produced within the scope of a broader development project SHOULD + reside in a common organization of a version control repository hosting service. ## 4.2. Licensing [QC.Lic] -* **[QC.Lic01]** As open-source software, source code MUST adhere to an open-source license - to be freely used, modified and distributed by others. +* **[QC.Lic01]** As open-source software, source code MUST adhere to an open-source license to be + freely used, modified and distributed by others. * **[QC.Lic01.1]** Non-licensed software is exclusive copyright by default. -* **[QC.Lic02]** License MUST be compliant with the Open Source Definition [@https://opensource.org/osd]. +* **[QC.Lic02]** License MUST be compliant with the Open Source Definition + [@https://opensource.org/osd]. - * **[QC.Lic02.1]** RECOMMENDED licenses are listed in the Open Source Initiative portal - under the Popular Licenses category [@https://opensource.org/licenses]. + * **[QC.Lic02.1]** RECOMMENDED licenses are listed in the Open Source Initiative portal under the + Popular Licenses category [@https://opensource.org/licenses]. -* **[QC.Lic03]** Licenses MUST be physically present (e.g. as a LICENSE file) in the root of - all the source code repositories related to the software component. +* **[QC.Lic03]** Licenses MUST be physically present (e.g. as a LICENSE file) in the root of all the + source code repositories related to the software component. ## 4.3. Code Style [QC.Sty] -Code style requirements pursue the correct maintenance of the source code by -the common agreement of a series of style conventions. These vary based on the -programming language being used. +Code style requirements pursue the correct maintenance of the source code by the common agreement of +a series of style conventions. These vary based on the programming language being used. -* **[QC.Sty01]** Each individual software product MUST comply with community-driven or de-facto - code style standards for the programming languages being used. +* **[QC.Sty01]** Each individual software product MUST comply with community-driven or de-facto code + style standards for the programming languages being used. * **[QC.Sty01.1]** Compliance with multiple complementary standards MAY exist. -* **[QC.Sty02]** Custom code style guidelines SHOULD be avoided, only considered in the - hypothetical event of programming languages without existing community style - standards. +* **[QC.Sty02]** Custom code style guidelines SHOULD be avoided, only considered in the hypothetical + event of programming languages without existing community style standards. - * **[QC.Sty02.1]** Custom styles MUST be documented by defining each convention and its - expected output. + * **[QC.Sty02.1]** Custom styles MUST be documented by defining each convention and its expected + output. * **[QC.Sty02.2]** Custom styles SHOULD evolve over time towards a more consistent definition. -* **[QC.Sty03]** Exceptions of individual conventions from the main definition are allowed - but SHOULD be avoided +* **[QC.Sty03]** Exceptions of individual conventions from the main definition are allowed but + SHOULD be avoided * **[QC.Sty03.1]** Absence of standard conventions SHOULD be justified and tracked. -* **[QC.Sty04]** Code style compliance testing MUST be automated and MUST be triggered for - each candidate change in the source code. +* **[QC.Sty04]** Code style compliance testing MUST be automated and MUST be triggered for each + candidate change in the source code. ## 4.4. Code metadata [QC.Met] -Metadata for the software component provides a way to achieve its full identification, -thus making software citation viable [@doi:10.7717/peerj-cs.86]. -It allows the assignment of a Digital Object Identifier (DOI) and is key -towards preservation, discovery, reuse, and attribution of the software component. +Metadata for the software component provides a way to achieve its full identification, thus making +software citation viable [@doi:10.7717/peerj-cs.86]. It allows the assignment of a Digital Object +Identifier (DOI) and is key towards preservation, discovery, reuse, and attribution of the software +component. -* **[QC.Met01]** A metadata file SHOULD exist along side the code, under its VCS. - The metadata file SHOULD be updated when needed, as is the case of a new version. +* **[QC.Met01]** A metadata file SHOULD exist along side the code, under its VCS. The metadata file + SHOULD be updated when needed, as is the case of a new version. ## 4.5. Unit Testing [QC.Uni] -Unit testing evaluates all the possible flows in the internal design of the -code, so that its behavior becomes apparent. It is a key type of testing for -early detection of failures in the development cycle. +Unit testing evaluates all the possible flows in the internal design of the code, so that its +behavior becomes apparent. It is a key type of testing for early detection of failures in the +development cycle. * **[QC.Uni01]** Minimum acceptable code coverage threshold SHOULD be 70%. * **[QC.Uni01.1]** Unit testing coverage SHOULD be higher for those sections of the code identified as critical by the developers, such as units part of a security module. - * **[QC.Uni01.2]** Unit testing coverage MAY be lower for external libraries or pieces of - code not maintained within the product’s code base. + * **[QC.Uni01.2]** Unit testing coverage MAY be lower for external libraries or pieces of code not + maintained within the product’s code base. * **[QC.Uni02]** Units SHOULD reside in the repository code base but separated from the main code. @@ -103,18 +99,18 @@ early detection of failures in the development cycle. * **[QC.Uni04]** Unit testing coverage MUST be automated. * **[QC.Uni04.1]** When working on automated testing, the use of testing doubles is - RECOMMENDED to mimic a simplistic behavior of objects and procedures. + RECOMMENDED to mimic a simplistic behavior of objects and procedures (c.f. section 4.6.). ## **PROPOSAL** ## 4.6. Test Harness [QC.Har] In software development, a test harness -[@https://searchsoftwarequality.techtarget.com/definition/test-harness], -is a collection of software and test data used by developers to unit test software models during -development. A test harness will specifically refer to test doubles, which are programs -that interact with the software being tested. Once a test harness is used to execute a test, they -can also utilize a test library to generate reports. +[@https://searchsoftwarequality.techtarget.com/definition/test-harness], is a collection of software +and test data used by developers to unit test software models during development. A test harness +will specifically refer to test doubles, which are programs that interact with the software being +tested. Once a test harness is used to execute a test, they can also utilize a test library to +generate reports. It is also a simple form of Integration Testing, where interaction and integration with external components are substituted by a Double. @@ -122,32 +118,32 @@ components are substituted by a Double. Test Double is a generic term for any case where you replace a production object for testing purposes. There are various kinds of double [@isbn:9780131495050]: -* Dummy objects are passed around but never actually used. Usually they are just used to fill +* **Dummy objects** are passed around but never actually used. Usually they are just used to fill parameter lists. -* Fake objects actually have working implementations, but usually take some shortcut which makes - them not suitable for production (an InMemoryTestDatabase is a good example). +* **Fake objects** actually have working implementations, but usually take some shortcut which makes + them not suitable for production (such as an InMemoryTestDatabase). -* Stubs provide canned answers to calls made during the test, usually not responding at all to +* **Stubs** provide canned answers to calls made during the test, usually not responding at all to anything outside what's programmed in for the test. -* Spies are stubs that also record some information based on how they were called. One form of this - might be an email service that records how many messages it was sent. +* **Spies** are stubs that also record some information based on how they were called. One form of + this might be an email service that records how many messages where sent. -* Mocks are pre-programmed with expectations which form a specification of the calls they are - expected to receive. They can throw an exception if they receive a call they don't expect and - are checked during verification to ensure they got all the calls they were expecting. +* **Mocks** are pre-programmed with expectations which form a specification of the calls they are + expected to receive. They can throw an exception if they receive a call they don't expect and are + checked during verification to ensure they got all the calls they were expecting. As such the following criteria is defined for Test Harness: -* **[QC.Har01]** When working on automated testing, the use of Test Doubles - is RECOMMENDED to mimic a simplistic behavior of objects and procedures. +* **[QC.Har01]** When working on automated testing, the use of Test Doubles is RECOMMENDED to mimic + a simplistic behavior of objects and procedures. -* **[QC.Har02]** Test Doubles SHOULD reside in the software component repository code - base but separated from the main code. +* **[QC.Har02]** Test Doubles SHOULD reside in the software component repository code base but + separated from the main code. -* **[QC.Har03]** Regression testing, that checks the conformance with previous tests, is - covered at this stage by executing the complete set of Test Doubles available. +* **[QC.Har03]** Regression testing, that checks the conformance with previous tests, is covered at + this stage by executing the complete set of Test Doubles available. * **[QC.Har04]** Test Doubles and regression, MUST be checked on change basis. @@ -184,44 +180,43 @@ design analysis or side-effects to external systems. ## 4.7. Test-Driven Development [QC.Tdd] Test-Driven Development [@isbn:9780321146533], is a software development process relying on software -requirements being converted to test cases before software is fully developed, -and tracking all software development by repeatedly testing the software against -all test cases. This is opposed to software being developed first and test cases -created later. +requirements being converted to test cases before software is fully developed, and tracking all +software development by repeatedly testing the software against all test cases. This is opposed to +software being developed first and test cases created later. -* **[QC.Tdd01]** Software requirements SHOULD be converted to test cases, and these - test cases SHOULD be checked automatically. +* **[QC.Tdd01]** Software requirements SHOULD be converted to test cases, and these test cases + SHOULD be checked automatically. ## 4.8. Documentation [QC.Doc] * **[QC.Doc01]** Documentation MUST be treated as code. - * **[QC.Doc01.1]** Version controlled, it MAY reside in the same repository where the source - code lies. + * **[QC.Doc01.1]** Version controlled, it MAY reside in the same repository where the source code + lies. -* **[QC.Doc02]** Documentation MUST use plain text format using a markup language, such as - Markdown or reStructuredText. +* **[QC.Doc02]** Documentation MUST use plain text format using a markup language, such as Markdown + or reStructuredText. - * **[QC.Doc02.1]** It is RECOMMENDED that all software components delivered by the same - project agree on a common markup language. + * **[QC.Doc02.1]** It is RECOMMENDED that all software components delivered by the same project + agree on a common markup language. -* **[QC.Doc03]** Documentation MUST be online available in a documentation repository. +* **[QC.Doc03]** Documentation MUST be online and available in a documentation repository. * **[QC.Doc03.1]** Documentation SHOULD be rendered automatically. -* **[QC.Doc04]** Documentation MUST be updated on new software versions involving any - substantial or minimal change in the behavior of the application. +* **[QC.Doc04]** Documentation MUST be updated on new software versions involving any substantial or + minimal change in the behavior of the application. * **[QC.Doc05]** Documentation MUST be updated whenever reported as inaccurate or unclear. -* **[QC.Doc06]** Documentation MUST be produced according to the target audience, varying - according to the software component specification. The identified types of - documentation and their RECOMMENDED content are: +* **[QC.Doc06]** Documentation MUST be produced according to the target audience, varying according + to the software component specification. The identified types of documentation and their + RECOMMENDED content are: * **[QC.Doc06.1]** README file * One-paragraph description of the application. - * A “Getting Started” step-by-step description on how to get a development environment running + * A *"Getting Started"* step-by-step description on how to get a development environment running (prerequisites, installation). * Automated test execution how-to. * Links to the external documentation below (production deployment, user guides). @@ -241,21 +236,21 @@ created later. * Installation and configuration guides. * Service Reference Card, with the following RECOMMENDED content: - * Brief functional description. - * List of processes or daemons. - * Init scripts and options. - * List of configuration files, location and example or template. - * Log files location and other useful audit information. - * List of ports. - * Service state information. - * List of cron jobs. + * Brief functional description. + * List of processes or daemons. + * Init scripts and options. + * List of configuration files, location and example or template. + * Log files location and other useful audit information. + * List of ports. + * Service state information. + * List of cron jobs. * Security information. * FAQs and troubleshooting. * **[QC.Doc06.4]** User * Public API documentation. - * Command-line (CLI) reference. + * Command Line Interface (CLI) reference. * **[QC.Doc07]** Documentation MUST be checked on change basis. @@ -281,12 +276,12 @@ especially effective at the source code level. * **[QC.Sec03]** Security code reviews [@https://owasp.org/www-project-code-review-guide/migrated_content] for certain vulnerabilities SHOULD be done as part of the identification of potential security - flaws in the code. Inputs SHOULD come from automated linters and manual penetration testing results. + flaws in the code. Inputs SHOULD come from automated linters and manual penetration testing + results. * **[QC.Sec04]** World-writable files or directories MUST NOT be present in the product’s configuration or logging locations. ---- **TO BE REMOVED - moved/added/adapted in the Service QA** * **[QC.Sec03]** Dynamic application security testing (DAST) @@ -310,72 +305,69 @@ especially effective at the source code level. * **[QC.Sec10]** The services delivered SHALL adhere to any extra security policies or requirements set at the operational level. ---- - ## 4.10. Code Workflow [QC.Wor] A change-based approach is accomplished with a branching model. -* **[QC.Wor01]** The main branch in the source code repository MUST maintain a working state - version of the software component. +* **[QC.Wor01]** The main branch in the source code repository MUST maintain a working state version + of the software component. - * **[QC.Wor01.1]** Main branch SHOULD be protected to disallow force pushing, thus - preventing untested and un-reviewed source code from entering the production-ready version. + * **[QC.Wor01.1]** Main branch SHOULD be protected to disallow force pushing, thus preventing + untested and un-reviewed source code from entering the production-ready version. - * **[QC.Wor01.2]** New features SHOULD only be merged in the main branch whenever the SQA - criteria is fulfilled. + * **[QC.Wor01.2]** New features SHOULD only be merged in the main branch whenever the SQA criteria + is fulfilled. * **[QC.Wor02]** New changes in the source code MUST be placed in individual branches. - * **[QC.Wor02.1]** It is RECOMMENDED to agree on a branch nomenclature, usually by - prefixing, to differentiate change types (e.g. feature, release, fix). + * **[QC.Wor02.1]** It is RECOMMENDED to agree on a branch nomenclature, usually by prefixing, to + differentiate change types (e.g. feature, release, fix). -* **[QC.Wor03]** The existence of a secondary long-term branch that contains the changes for - the next release is RECOMMENDED. +* **[QC.Wor03]** The existence of a secondary long-term branch that contains the changes for the + next release is RECOMMENDED. * **[QC.Wor03.1]** Next release changes come from the individual branches. - * **[QC.Wor03.2]** Once ready for release, changes in the secondary long-term branch are - merged into the main branch and versioned. At that point in time, main - and secondary branches are aligned. This step SHOULD mark a production - release. + * **[QC.Wor03.2]** Once ready for release, changes in the secondary long-term branch are merged + into the main branch and versioned. At that point in time, main and secondary branches are + aligned. This step SHOULD mark a production release. ## 4.11. Semantic Versioning [QC.Ver] -* **[QC.Ver01]** Semantic Versioning [@https://semver.org] specification is RECOMMENDED for - tagging the production releases. +* **[QC.Ver01]** Semantic Versioning [@https://semver.org] specification is RECOMMENDED for tagging + the production releases. ## 4.12. Code Management [QC.Man] * **[QC.Man01]** An issue tracking system facilitates structured software development. Leveraging - issues to track down both new enhancements and defects (bugs or documentation typos) - is RECOMMENDED. + issues to track down both new enhancements and defects (bugs, documentation typos) is RECOMMENDED. * **[QC.Man01.1]** In addition to monitoring the internal development, issues are the best - means for supporting users. External users SHOULD be able to create incidences based on the + means for supporting users. External users SHOULD be able to create issues based on the operational performance of the software. - * **[QC.Man01.2]** The description of an issue SHOULD be concise and state clearly the problem. - It is RECOMMENDED to add any reference to the actual problem. In the case of bugs, the - issue SHOULD be accompanied by the relevant debug information. + * **[QC.Man01.2]** The description of an issue SHOULD be concise and state clearly the problem. It + is RECOMMENDED to add any reference to the actual problem. In the case of bugs, the issue SHOULD + be accompanied by the relevant debug information. * The usage of templates for the issue description is RECOMMENDED. -* **[QC.Man02]** In social coding environments, pull requests represent the cornerstone of - collaboration. A pull request provides a place for review and discussion of the changes proposed - to be part of an existing version of the code. +* **[QC.Man02]** In social coding environments, pull or merge requests represent the cornerstone of + collaboration. A pull or merge request provides a place for review and discussion of the changes + proposed to be part of an existing version of the code. - * **[QC.Man02.1]** Pull requests SHOULD be used for every change in the codebase. + * **[QC.Man02.1]** Pull/Merge requests SHOULD be used for every change in the codebase. - * **[QC.Man02.2]** A software project SHOULD be open to external collaboration through pull requests. + * **[QC.Man02.2]** A software project SHOULD be open to external collaboration through pull/merge + requests. - * **[QC.Man02.3]** A pull request description SHOULD be concise and state clearly its purpose (e.g. if - it is fixing an observed bug or adding a new feature) + * **[QC.Man02.3]** A pull/merge request description SHOULD be concise and state clearly its + purpose (e.g. if it is fixing an observed bug or adding a new feature). - * **[QC.Man02.4]** The usage of templates for the pull request's description is RECOMMENDED. + * **[QC.Man02.4]** The usage of templates for the pull/merge request's description is RECOMMENDED. - * **[QC.Man02.5]** It is RECOMMENDED to use pull requests to address open issues. The pull request - description SHOULD make reference to the identifiers of the issues it is fixing - (to eventually close them, either manually or automatically). + * **[QC.Man02.5]** It is RECOMMENDED to use pull/merge requests to address open issues. The + pull/merge request description SHOULD make reference to the identifiers of the issues it is + fixing (to eventually close them, either manually or automatically). ## 4.13. Code Review [QC.Rev] @@ -384,49 +376,52 @@ source code [@https://owasp.org/www-project-code-review-guide/migrated_content]. last step in the change management pipeline, once the candidate change has successfully passed over the required set of change-based tests. -* **[QC.Rev01]** Code reviews MUST be done in the agreed peer review tool within the project, - with the following RECOMMENDED functionality: +* **[QC.Rev01]** Code reviews MUST be done in the agreed peer review tool within the project, with + the following RECOMMENDED functionality: - * **[QC.Rev01.1]** Allows general and specific comments on the line or lines that need to - be reviewed. + * **[QC.Rev01.1]** Allows general and specific comments on the line or lines that need to be + reviewed. * **[QC.Rev01.2]** Shows the results of the required change-based test executions. - * **[QC.Rev01.3]** Allows to prevent merges of the candidate change whenever not all the - required tests are successful. Exceptions to this rule cover the third-party or upstream - contributions which MAY use the existing mechanisms or tools for code review provided by the - target software project. This exception MUST only be allowed whenever the external revision - lifecycle does not interfere with the project deadlines. + * **[QC.Rev01.3]** Allows to prevent merges of the candidate change whenever not all the required + tests are successful. Exceptions to this rule cover the third-party or upstream contributions + which MAY use the existing mechanisms or tools for code review provided by the target software + project. This exception MUST only be allowed whenever the external revision lifecycle does not + interfere with the project deadlines. -* **[QC.Rev02]** Code reviews MUST be open and collaborative, allowing external expert revisions. +* **[QC.Rev02]** Code reviews MUST be open and collaborative, allowing external experts revisions. * **[QC.Rev03]** Code reviews SHOULD be concise and use neutral language. The areas where the reviewers MAY focus are: - * **[QC.Rev03.1]** Message description: commit message is clear, self-explanatory and - describes precisely the objectives being addressed. + * **[QC.Rev03.1]** Message description: commit message is clear, self-explanatory and describes + precisely the objectives being addressed. - * **[QC.Rev03.2]** Goal or scope: change is needed and/or addresses/fixes the whole set of objectives. + * **[QC.Rev03.2]** Goal or scope: change is needed and/or addresses/fixes the whole set of + objectives. * **[QC.Rev03.3]** Code analysis: useless statements in the code, library or modules imported but never used or code style suggestions. - * **[QC.Rev03.4]** Review of required tests: current battery of tests is sufficient for validation. + * **[QC.Rev03.4]** Review of required tests: current battery of tests is sufficient for + validation. - * **[QC.Rev03.5]** Review of documentation: whether the change SHOULD bring along a - corresponding update in the documentation. + * **[QC.Rev03.5]** Review of documentation: whether the change SHOULD bring along a corresponding + update in the documentation. * **[QC.Rev04]** Code reviews MUST be checked on change basis. -* **[QC.Rev05]** Code reviews SHOULD assess the inherent security risk of the changes, - ensuring that the security model has not been downgraded or compromised by the changes. +* **[QC.Rev05]** Code reviews SHOULD assess the inherent security risk of the changes, ensuring that + the security model has not been downgraded or compromised by the changes. ## 4.14. Automated Delivery [QC.Del] -Automated delivery comprises the build of Software into an artifact, its upload/registration into -a public repository of such artifacts and notification of successful process. +Automated delivery comprises the build of Software into an artifact, its upload/registration into a +public repository of such artifacts and notification of the success of the process. -* **[QC.Del01]** Production-ready code SHALL be built as an artifact that can be executed on a system. +* **[QC.Del01]** Production-ready code SHALL be built as an artifact that can be executed on a + system. * **[QC.Del02]** The builded artifact SHALL be uploaded and registered into a public repository of such artifacts. @@ -437,19 +432,21 @@ a public repository of such artifacts and notification of successful process. ## 4.15. Automated Deployment [QC.Dep] * **[QC.Dep01]** Production-ready code SHALL be deployed as a workable system with the minimal user -or system administrator interaction leveraging software configuration management (SCM) tools. + or system administrator interaction leveraging software configuration management (SCM) tools. * **[QC.Dep02]** A SCM module is treated as code. - * **[QC.Dep02.1]** Version controlled, it SHOULD reside in a different repository than the - source code to facilitate the distribution. -* **[QC.Dep03]** It is RECOMMENDED that all software components delivered by the same project - agree on a common SCM tool. - * **[QC.Dep03.1]** However, software products are not restricted to provide a unique - solution for the automated deployment. + * **[QC.Dep02.1]** Version controlled, it SHOULD reside in a different repository than the source + code to facilitate the distribution. + +* **[QC.Dep03]** It is RECOMMENDED that all software components delivered by the same project agree + on a common SCM tool. + + * **[QC.Dep03.1]** However, software products are not restricted to provide a unique solution for + the automated deployment. -* **[QC.Dep04]** Any change affecting the application’s deployment or operation MUST be - subsequently reflected in the relevant SCM modules. +* **[QC.Dep04]** Any change affecting the application’s deployment or operation MUST be subsequently + reflected in the relevant SCM modules. -* **[QC.Dep05]** Official repositories provided by the manufacturer SHOULD be used to host - the SCM modules, thus augmenting the visibility and promote external collaboration. +* **[QC.Dep05]** Official repositories provided by the manufacturer SHOULD be used to host the SCM + modules, thus augmenting the visibility and promote external collaboration. diff --git a/content/08.annex.md b/content/08.annex.md index 0146a69..cbba5b7 100644 --- a/content/08.annex.md +++ b/content/08.annex.md @@ -1,41 +1,41 @@ # A1. Annex {.page_break_before} -The Quality Criteria described in this document is abstract, as such the choice of tools -and services to implement their verification and the code workflow, is up to the team -or community developing and/or using a given software. +The Quality Criteria described in this document is abstract, as such the choice of tools and +services to implement their verification and the code workflow, is up to the team or community +developing and/or using a given software. -The annex describes the implementation of how the Quality Criteria detailed in this -document can be verified. The technical implementation is achieved through the -SQA as a Service (SQAaaS) platform, being developed in the framework of the EOSC-Synergy project -described in Deliverable 3.2 [@doi:10.20350/digitalCSIC/12721]). It details the architecture -and set of components of the platform. +The annex describes the implementation of how the Quality Criteria detailed in this document can be +verified. The technical implementation is achieved through the SQA as a Service (SQAaaS) platform, +being developed in the framework of the EOSC-Synergy project described in Deliverable 3.2 +[@doi:10.20350/digitalCSIC/12721]). It details the architecture and set of components of the +platform. This annnex details the code workflow, the tools and services used to achieve the verification of the Criteria. ## A1.1. Code workflow, tools and services -The code workflow shown in Figure @fig:workflow. In this real case example, Github is used for -several purposes that will be described below. The workflow start when the developer branches +The code workflow is shown in Figure @fig:workflow. It depicts a real case example, Github is used +for several purposes that will be described below. The workflow start when the developer branches the code to implement a given new feature or fix, after the implementation the Pull Request triggers a CI pipeline in the Jenkins service. In the Jenkins service, several checks are performed, both on the code (static), such as code style -linting, as well as dynamic tests such as unit and functional tests. To perform functional tests -that may include API tests in case of a service, the packages will need to be built (delivery), -and automatically deployed into a running state. In the case shown, the configuration is done with +linting, as well as dynamic tests such as unit and harness tests. To perform functional tests that +may include API tests in case of a service, the packages will need to be built (delivery), and +automatically deployed into a running state. In the case shown, the configuration is done with ansible roles and playbooks. The Pull Request is updated with the results of the tests, thus notifying the developer team that they can proceed with the review and, if approved, with the release process where the branch is -merge into the main production branch. +merge into the main production branch. The SQAaaS platform can be used to compose custom Jenkins pipelines in an easy way, in order to implement all necessary checks of the baseline criteria. ![Code workflow](images/devops.png){height="500px"}{#fig:workflow} -Table @tbl:services show the list of tools and services used for the source code management as well +Table @tbl:services shows the list of tools and services used for the source code management as well as to implement the verification of the Quality Criteria detailed in this document. The main service used for Software source code management has been Github. It is uses git as Version @@ -48,12 +48,13 @@ tools. The software is packed/built into executable artifacts that can be RPMs (case of RedHat and derivative OS), DEBs (case of Debian/Ubuntu and derivatives) and in many cases into docker images. -The artifacts are provided, in general, by public repositories and most notably Dockerhub in the case -of docker images. Other common repositories are PyPI for python SW or modules and Maven for Java. +The artifacts are provided, in general, by public repositories and most notably Dockerhub in the +case of docker images. Other common repositories are PyPI for python SW or modules and Maven for +Java. -Regarding the CI/CD automation, Jenkins pipelines can be easily composed through the SQAaaS -platform and put into the git repositories to be used by the Jenkins CI service to perform the -tests. The tools used in the CI automation are shown in section A1.2. +Regarding the CI/CD automation, Jenkins pipelines can be easily composed through the SQAaaS platform +and put into the git repositories to be used by the Jenkins CI service to perform the tests. The +tools used in the CI automation are shown in section A1.2. | Service/Tool | Usage | Criteria | Comment | |:---------------:|:------------------:|:----------:|:--------------------------------------------:| @@ -66,9 +67,10 @@ tests. The tools used in the CI automation are shown in section A1.2. | SQAaaS platform | pipeline composition | N.A. | Pipeline composition for automatic tests | | Jenkins CI service | Automated tests | N.A. | Execution of automatic tests | | Dockerhub | Docker images | **QC.Del** | Public repository of docker images | +| tox | | **QC.Sty**/**QC.Uni**/**QC.Fun**/**QC.Sec**/**QC.Doc** | Automated test framework | -Table: Tools and services used to implement the QA criteria, also shown the criteria -where applicable. {#tbl:services} +Table: Tools and services used to implement the QA criteria, also shown the criteria where +applicable. {#tbl:services} ## A1.2. Tools for CI/CDD automation @@ -76,14 +78,12 @@ This section shows the tools being used in the CI pipelines, the criteria that i applicable programming language. This list is based on the template file in . -| Tool | Criteria | Programming Language | Repo URL or documentation | Summary | -|:------------:|:----------:|:--------------------:|:-------------------------:|:-----------------:| -| pylint | **QC.Sty** | Python | | Code style | -| hadolint | **QC.Sty** | Dockerfile | | Code style | -| checkstyle | **QC.Sty** | Java | | Code style | -| jsonlint | **QC.Sty** | JSON | | Code style | -| bandit | **QC.Sec** | Python | | Static security | -| pycodestyle | **QC.Sty** | Python | | Code style | -| tox | **QC.Sty**/**QC.Uni**/**QC.Fun**/**QC.Sec**/**QC.Doc** | Python | | Automated test framework | -| licensee | **QC.Lic** | Agnostic | | Check license | - +| Tool | Criteria | Programming Language | Repo URL or documentation | Summary | +|:-----------:|:----------:|:--------------------:|:-------------------------:|:-----------------:| +| pylint | **QC.Sty** | Python | | Code style | +| hadolint | **QC.Sty** | Dockerfile | | Code style | +| checkstyle | **QC.Sty** | Java | | Code style | +| jsonlint | **QC.Sty** | JSON | | Code style | +| bandit | **QC.Sec** | Python | | Static security | +| pycodestyle | **QC.Sty** | Python | | Code style | +| licensee | **QC.Lic** | Agnostic | | Check license |