Enhanced the Github action workflows

.codecov.yml

@@ -0,0 +1,23 @@
+codecov:
+  require_ci_to_pass: yes
+
+coverage:
+  precision: 2
+  round: down
+  range: "70...100"
+
+parsers:
+  gcov:
+    branch_detection:
+      conditional: yes
+      loop: yes
+      method: no
+      macro: no
+
+comment:
+  layout: "reach,diff,flags,tree"
+  behavior: default
+  require_changes: no
+
+fixes:
+  - "nextcloud/apps/cookbook/::"

.github/actions/run-tests/README.md

@@ -0,0 +1,90 @@
+# Test scripts
+
+- [Manual usage](#manual-usage)
+- [Automatic usage](#automatic-usage)
+
+The scripts in this folder should allow both local (manual) testing of the app as well as automatic continuous integration testing. This documentation should cover both useages and flavours of these scripts .
+
+## Manual usage
+
+For Manuel usage a containerized environment is used. Here `docker` and `docker-compose` are used in order to use exactly same environment for the tests compared with the continuous integration environment.
+
+### Configuration
+
+The first step to do manual testing is to set up the doctor-compose configuration. This is necessary because the configuration is depending on the developer's machine and desires.
+
+1. The build output should be owned by the development user when running Linux. That way, all files can be access without right issues.
+2. The PHP version to be testet agains must be set accordingly. There is a default but the developmer might choose to use a different version.
+
+Just tweak the `docker-compose.yml` file according to your needs.
+
+Do not fear if you are not fluent with docker. There is not much to be done here. The sections `mysql` and `postgres` should be left as they are. Just go to `dut` and adjust the build args. Set the desired PHP version and your main user's uid there.
+
+If you are not developing on Linux, you can keep the defaults.
+
+### Folder generation
+
+The nextcloud installation needs to be placed somewhere. Also, the build result must be saved somewhere. To do so, a dedicated folder needs to be generated in this folder (as a sibling to the `docker-compose.yml` file). Call it `workdir`. Under Linux make it owned by your development user.
+
+The folder will be populated later during the build. There will be a few 100MB needed for the installation of a nextcloud instance.
+
+### Building of the image
+
+In order to run the tests, a docker image accoring to your settings needs to be generated. This is done by calling
+```
+docker-compose built dut
+```
+
+This process will take some time as a few dependencies need to be compiled into the image. Just wait for it to succeed.
+
+**Note:** When you want/need to change the test routines in the docker image, you need to rebuild the image as shown in the command above. This is only necessary if you want to change the build process, so you will most probably know what you are doing anyways.
+
+### Run a complete test
+
+The whole testing process is running completely automated. Just go to the corresponding folder where the `docker-compose.yml` file relies. There you can just call the `run-local.sh` script. It will do the following:
+
+- Fire up a set of databases
+- Remove any previous test installation
+- Cloned the nextcloud server from github including submodules in the folder created above
+- Drop all tables from these just created databases (that is reset the dabase to start from scratch)
+- Install the nextcloud test instance (create databases and folder structure)
+- Finally: Run the device under test (dut).
+
+Note that no data is preserved during such a run. If you change anything it will be overwritten.
+
+### Quicker run of unit tests
+
+To avoid the overhead of removing and reinstalling the nextcloud in each iteration, there is the option to run the unit tests only.
+
+Obviously, this can only work if the rest of the system is prepared accordingly. So you **must** hav called the `run-locally.sh` file at least once to create a basic installation.
+
+After that, by calling
+```
+docker-compose run --rm dut --test-only
+```
+you can run only the tests.
+
+Here the database is not affected and the installation is not tested as well. If some strange behaior is observed, a complete run might help.
+
+**Note:** Before publishing large changes a complete build is highly suggested.
+
+### Test coverage
+
+The test will generate code coverage reports for the developer. These can be found in plain HTML format under `workdir/nextcloud/apps/cookbook`.
+
+## Automatic usage
+
+This set of scripts is well usable using Github Actions to automate the continuous integration (CI) test. It contains a complete action that will run the test suite based on some input variables:
+
+- the PHP version
+- the database to be used
+
+During the run on the action creates a docker network and the corresponding databases. After the run, the containers are shut down.
+
+In fact, building and running the container is done by Github Actions. During the building process some dependencies and the corresponding PHP version are installed in the container as for the local build process. Also `npm` is initialised globally.
+
+The main build process involves creating a nextcloud installation, building the main app (fulfill composer and npm dependencies), preparing some database values depending on the selected database and installing the nextcloud main instance. The tests are carried out and the results are kept in the local folder.
+
+The application is installed in the folder `nextcloud/apps/cookbook`. There will also reside a set of build artefacts, that are generated during the test run.
+
+If you want to have an interactive look at the code coverage, you might want to go to [codecov.io](https://codecov.io/gh/nextcloud/cookbook). The latest information should be available there.

.github/actions/run-tests/action.yml

@@ -12,6 +12,10 @@ inputs:
     db:
         description: The db type to use
         required: true
+    allowFailure:
+        description: The test may fail
+        required: false
+        default: false
 
 runs:
     using: 'composite'
@@ -77,6 +81,7 @@ runs:
                 -e GITHUB_API_URL
                 -e GITHUB_GRAPHQL_URL
                 -e CI
+                -e ALLOW_FAILURE=${{ inputs.allowFailure }}
                 --network net_cookbook
                 -v "$GITHUB_WORKSPACE":/workdir
                 -v "$GITHUB_WORKSPACE/cookbook":/app:ro

.github/actions/run-tests/tests/install.sh

@@ -43,6 +43,9 @@ docker-php-ext-install -j$(nproc) zip
 
 docker-php-ext-install -j$(nproc) pdo pdo_mysql pdo_pgsql pdo_sqlite
 
+pecl install xdebug
+docker-php-ext-enable xdebug
+
 echo 'runner ALL=(ALL) NOPASSWD: ALL' >> /etc/sudoers
 
 npm install -g npm@latest

.github/actions/run-tests/tests/test-entrypoint.sh

@@ -2,6 +2,21 @@
 
 set -x
 
+trap 'catch $? $LINENO' EXIT
+
+catch()
+{
+	if [ "$1" != '0' ]; then
+		echo "::error line=$LINENO::Error during the test run: $1"
+
+		if [ "$ALLOW_FAILURE" = 'true' ]; then
+			exit 0
+		else
+			exit $1
+		fi
+	fi
+}
+
 cd nextcloud
 
 if [ ! "$1" = '--test-only' ]; then
@@ -121,6 +136,10 @@ echo 'Running the main tests'
 make test
 echo 'Tests finished'
 
+echo 'Copy code coverage in HTML format'
+cp -r coverage $GITHUB_WORKSPACE
+cp -r coverage-integration $GITHUB_WORKSPACE
+
 popd
 
 echo 'Shutting down temporary web server'

.github/workflows/tests.yml

@@ -69,9 +69,74 @@ jobs:
         with:
             db: ${{ matrix.database }}
             phpVersion: ${{ matrix.phpVersion }}
+            allowFailure: ${{ matrix.mayFail }}
 
       - name: Upload the log file
         uses: actions/upload-artifact@v2
         with:
             name: Nextcloud-logs (${{matrix.database}}, ${{matrix.coreVersion}}, ${{matrix.phpVersion}})
             path: nextcloud/data/nextcloud.log
+
+      - name: Upload the coverage reports to codecov.io
+        shell: bash
+        run: >-
+            bash <(curl -s https://codecov.io/bash) -cF unittests -f nextcloud/apps/cookbook/coverage.xml &&
+            bash <(curl -s https://codecov.io/bash) -cF integration -f nextcloud/apps/cookbook/coverage.integration.xml
+
+      - name: Upload the code coverage report (unit tests)
+        uses: actions/upload-artifact@v2
+        with:
+            name: Code coverage (HTML) (${{matrix.database}}, ${{matrix.coreVersion}}, ${{matrix.phpVersion}})
+            path: |
+                coverage/
+                coverage-integration/
+
+
+  source-package:
+    name: Create source code artifacts
+    needs: test
+    runs-on: ubuntu-latest
+
+    steps:
+        -   name: Checkout the nextcloud server
+            uses: actions/checkout@v2
+            with:
+                repository: nextcloud/server
+                ref: ${{ matrix.coreVersion }}
+                path: nextcloud
+        -   name: Checkout the app
+            uses: actions/checkout@v2
+            with:
+                path: nextcloud/apps/cookbook
+
+        -   name: Install the NPM dependencies
+            shell: bash
+            run: >-
+                cd nextcloud/apps/cookbook &&
+                npm install
+        -   name: Install the dependencies for composer
+            shell: bash
+            run: >-
+                cd nextcloud/apps/cookbook &&
+                make
+
+        -   name: Create the source package
+            shell: bash
+            run: >-
+                make -C nextcloud/apps/cookbook source_package_name=$(pwd)/source source
+        -   name: Publish the source package as artifact
+            uses: actions/upload-artifact@v2
+            with:
+                name: cookbook-source
+                path: source.tar.gz
+
+        -   name: Create the app store package
+            shell: bash
+            run: >-
+                make -C nextcloud/apps/cookbook appstore_package_name=$(pwd)/appstore appstore
+        -   name: Publish the app store package as artifact
+            uses: actions/upload-artifact@v2
+            with:
+                name: cookbook-appstore
+                path: appstore.tar.gz
+

Makefile

@@ -115,20 +115,21 @@ dist:
 source:
 	rm -rf $(source_build_directory)
 	mkdir -p $(source_build_directory)
-	tar cvzf $(source_package_name).tar.gz ../$(app_name) \
+	tar cvzf $(source_package_name).tar.gz \
 	--exclude-vcs \
 	--exclude="../$(app_name)/build" \
 	--exclude="../$(app_name)/js/node_modules" \
 	--exclude="../$(app_name)/node_modules" \
 	--exclude="../$(app_name)/*.log" \
 	--exclude="../$(app_name)/js/*.log" \
+	../$(app_name)
 
 # Builds the source package for the app store, ignores php and js tests
 .PHONY: appstore
 appstore:
 	rm -rf $(appstore_build_directory)
 	mkdir -p $(appstore_build_directory)
-	tar cvzf $(appstore_package_name).tar.gz ../$(app_name) \
+	tar cvzf $(appstore_package_name).tar.gz \
 	--exclude-vcs \
 	--exclude="../$(app_name)/build" \
 	--exclude="../$(app_name)/tests" \
@@ -150,8 +151,9 @@ appstore:
 	--exclude="../$(app_name)/protractor\.*" \
 	--exclude="../$(app_name)/.*" \
 	--exclude="../$(app_name)/js/.*" \
+	../$(app_name)
 
 .PHONY: test
 test: composer
-	$(CURDIR)/vendor/phpunit/phpunit/phpunit -c phpunit.xml
-	$(CURDIR)/vendor/phpunit/phpunit/phpunit -c phpunit.integration.xml
+	$(CURDIR)/vendor/phpunit/phpunit/phpunit -c phpunit.xml --coverage-clover coverage.xml --coverage-html coverage
+	$(CURDIR)/vendor/phpunit/phpunit/phpunit -c phpunit.integration.xml --coverage-clover coverage.integration.xml --coverage-html coverage-integration

phpunit.integration.xml

@@ -4,4 +4,14 @@
             <directory>./tests/Integration</directory>
         </testsuite>
     </testsuites>
+	<filter>
+		<whitelist processUncoveredFilesFromWhitelist="true" addUncoveredFilesFromWhitelist="true">
+			<directory suffix=".php">lib/</directory>
+			<!--<directory suffix=".php">lib/Controller</directory>
+			<directory suffix=".php">lib/Db</directory>
+			<directory suffix=".php">lib/Exception</directory>
+			<directory suffix=".php">lib/Migration</directory>
+			<directory suffix=".php">lib/Service</directory>-->
+		</whitelist>
+	</filter>
 </phpunit>

phpunit.xml

@@ -4,4 +4,14 @@
             <directory>./tests/Unit</directory>
         </testsuite>
     </testsuites>
+	<filter>
+		<whitelist processUncoveredFilesFromWhitelist="true" addUncoveredFilesFromWhitelist="true">
+			<directory suffix=".php">lib</directory>
+			<!--<directory suffix=".php">lib/Controller</directory>
+			<directory suffix=".php">lib/Db</directory>
+			<directory suffix=".php">lib/Exception</directory>
+			<directory suffix=".php">lib/Migration</directory>
+			<directory suffix=".php">lib/Service</directory>-->
+		</whitelist>
+	</filter>
 </phpunit>