Developer Information

Developer Information

Table of Contents

• 1. Introduction
• 2. Quick-start
• 3. Project Structure
• 4. Build and run
  • 4.1. Databases
  • 4.2. Building the jar
  • 4.3. Running the application
  • 4.4. Running the project in IntelliJ-Idea
  • 4.5. Running tests
• 5. Continuous integration

1. Introduction

In order to build the project successfully, you need to meet some preconditions:

• You need to have a docker daemon running. It is used for Flyway to migrate the schema and for jOOQ to generate classes from the database schema. It is also used to run the database integration tests against.

• Your IDE must be enabled to run Lombok to enhance the byte code based on lombok annotations. If you use eclipse, you also need to instruct it to treat the classes generated by jOOQ and JaxB as source folders. I.e.

  • With IntelliJ, install th Lombok-Plugin. IntelliJ will automatically recognize the generated class folders as source folder.

2. Quick-start

Refer to the quick start guide in order to get the project up and running with minimal configuration.

3. Project Structure

SciPaMaTo consists of two applications sharing some common aspects. SciPaMaTo-Core is the main application, SciPaMaTo-Public the public facade of data that is exposed over the internet.

The project is a multi-module project consisting of the three basic groups of modules (or projects, as they are called in the gradle world):

• SciPaMaTo-Core modules comprising the core system that is used internally by the main users feeding the system with data. The build process processes the Java/Kotlin part of the core application into an executable spring boot jar.

• SciPaMaTo-Public modules comprising the public face of the application which is used from anonymous users on the web to query the application for papers. SciPaMaTo-Public is also processed into an executable spring boot jar.

• SciPaMaTo-Common modules with common code used in both the core and the public applications.

The common modules are:

test

Module that contains classes that are commonly used by tests (in main not test)

utils

Module containing utility classes (typically static).

entity

Module containing generic entity and filter base classes.

persistence-api

Common pagination and sorting classes.

persistence-jooq

Common sorting and mapping jOOQ implementations.

persistence-jooq.test

Common test-classes concerning the jOOQ implementations.

wicket

Wicket components and common base page, panel.

The core modules are:

entity

Module containing the entity classes and projections (DTOs). Also contains the Filter classes used for searching papers as well as SearchTerm and its derivatives. In addition hosts Role specific classes.

persistence-api

Purely provides the service interfaces.

persistence-jooq

jOOQ-specific implementation of the service interfaces.

pubmed-api

Classes generated from the PubMed API dtd. Also contains the facade classes for pubmed articles used within SciPaMaTo.

sync

Module with the responsibility to push data from the core to the public database. At a later stage, there will be a workflow in SciPaMaTo-Core defining which papers are eligible for being pushed to public. Currently, all papers are pushed to SciPaMaTo public.

web

Web application. Currently also ties all modules together. This latter function could later be externalized into an integration module, so that core-web does not know the persistence implementation (only the API).

The public modules are:

entity

Currently a stub; will contain entities, DTOs, filters etc.

persistence-api

Purely provides the service interfaces.

persistence-jooq

jOOQ-specific implementation of the service interfaces.

web

Web application. Currently also ties all modules together.

4. Build and run

Run gradlew assemble in the implementation/scipamato directory. It will traverse through all modules and build the executable artifacts within:

• implementation/scipamato/core/web/build/libs/core-web-${version}.jar and

• implementation/scipamato/public/web/build/libs/public-web-${version}.jar

By default, gradle builds both SciPaMaTo-Core and SciPaMaTo-Public. You can limit the build to only one of the two applications (or explicitly build both) by calling specific gradle tasks:

SciPaMaTo-Core only	SciPaMaTo-Public only	Core and Public
gradlew :core-web:assemble	gradlew :public-web:assemble	gradlew: assemble

4.1. Databases

4.1.1. Supported Databases

While SciPaMaTo before version 0.5.0 has supported two databases (H2 and PostgreSQL), scipamato-0.5.0 and later only supports PostgreSQL.

4.1.2. Preparing the database for the core application (required once)

Note: Follow the instructions below if you have a PostgreSQL database installed directly on your operating system (i.e. not running it in a docker container)

• Install PostgreSQL and configure (I currently develop with version 15.4)

• Create the database scipamato, the administrative user scipamadmin for the Flyway migrations as well as the application user scipamato. Make sure you’re logged on as user postgres:

  • Create the database:

    createdb -E utf8 scipamato

  • Start the PostgreSQL interactive terminal psql

    CREATE USER scipamadmin WITH CREATEROLE PASSWORD 'scipamadmin';
    GRANT ALL PRIVILEGES ON DATABASE scipamato to scipamadmin WITH GRANT OPTION;

If you use a different username and/or password (and maybe you should!), you’ll need to override the property flyway.user and/or flyway.password defined in two places (unfortunately, it’s redundant for developers):

• core/web/src/main/resources/application.properties (used at runtime). You could also create your own application.properties in the same directory as the jar for that.

• core/persistence-jooq/src/main/resources/application.properties (used for the gradle build)

The Flyway scripts will create the operational user scipamato with a default password. Of course, you can change that later too and provide the changed password in your local application.properties

ℹ️	On the server, there is only one single file (application.properties) overriding the two definitions in the core-web and core-persistence-jooq modules.

4.1.3. Preparing the database for the public application (required once)

• Create the database scipamato_public, the administrative user scipamadminpub for the Flyway migrations as well as the application user scipamatopub. Make sure you’re logged on as user postgres:

  • Create the database:

    createdb -E utf8 scipamato_public

  • Start the PostgreSQL interactive terminal psql

    CREATE USER scipamadminpub WITH CREATEROLE PASSWORD 'scipamadminpub';
    GRANT ALL PRIVILEGES ON DATABASE scipamato_public to scipamadminpub WITH GRANT OPTION;

If you use a different username and/or password (and maybe you should!), you’ll need to override the property flyway.user and/or flyway.password defined in two places (unfortunately, it’s redundant):

• public/web/src/main/resources/application.properties (used at runtime). You could also create your own application.properties in the same directory as the jar for that.

• public/persistence-jooq/src/main/resources/application.properties (used for the gradle build)

The Flyway scripts will create the operational user scipamatopub with a default password. Of course, you can change that later too, and provide the changed password in your local application.properties.

4.2. Building the jar

SciPaMaTo-Core	SciPaMaTo-Public	Core and Public
gradlew :core-web:jar	gradlew :public-web:jar	gradlew jar

This runs all PostgreSQL-Flyway scripts to create the tables in the schema public and inserts some generic data. It then runs the jOOQ code generator to generate the type-safe java classes from the database.

4.3. Running the application

To run the core application from gradle:

SciPaMaTo-Core	SciPaMaTo-Public
gradlew :core-web:bootRun	gradlew :public-web:bootRun

4.4. Running the project in IntelliJ-Idea

In the Spring Dashboard, configure SciPaMaTo-Core and -Public with the respective profiles development and the module.

For SciPaMaTo Core:

Main Class

ch.difty.scipamato.core.ScipamatoCoreApplication

Profile

development

and likewise for SciPaMaTo Public:

Main Class

ch.difty.scipamato.publ.ScipamatoPublicApplication

Profile

development

4.5. Running tests

4.5.1. Running non-database tests via gradle

Run the unit tests - excluding database integration tests:

gradlew test

4.5.2. Running database tests via gradle

The database test requires some stable sample data to run the test against. We, therefore, need a dedicated database that is pre-filled with some test specific reference data.

gradlew integrationTest

or

gradlew check

Preparation of the databases (once only)

• Create the test database as user postgres:

  createdb -E utf8 scipamato_it
  createdb -E utf8 scipamato_public_it

• Start the PostgreSQL interactive terminal psql

  GRANT ALL PRIVILEGES ON DATABASE scipamato_it to scipamadmin WITH GRANT OPTION;
  GRANT ALL PRIVILEGES ON DATABASE scipamato_public_it to scipamadminpub WITH GRANT OPTION;

The project is configured to always run the Flyway migrations, and the jOOQ model generation.

Running the tests

To run all tests for both core and public (including wicket page/panel tests and integrationTests):

gradlew check

This will only succeed if you have created all four databases (core and public, each normal database, and for integration tests).

If you want to stick with unit tests (requiring only one database for core and public for jOOQ to generate the type-safe classes), you can run:

gradlew test

4.5.3. Clearing the test databases

In order to clear the test databases, do the following:

Clearing SciPaMaTo-Core	Clearing SciPaMaTo-Public	Clearing both Core and Public
gradlew :core-web:flywayClean	gradlew :public-web:flywayClean	gradlew flywayClean

5. Continuous integration

SciPaMaTo is running on Travis-CI.

Static Code Analysis is offered in SonarCloud.