scholarspace-hyrax

A Hyrax app for GW Libraries with:

two item types: GwWork and GwEtd
roles: admin, content-admin

The public application is accessible at https://scholarspace.library.gwu.edu/

Some convenient links to have handy:

Hyrax github repo
Hyrax project
Hyrax developer knowledge base

Getting started

The recommended production setup involves two servers. However, these can be the same server if needed, for example in a development environment.

Repository server, for the Fedora repository and Solr interface
Application server, for the GW ScholarSpace rails app

Currently these instructions are for an Ubuntu 16.04 repository server, and an Ubuntu 18.04 application server.

Repository server

Install Java 8
- For Ubuntu 16:
```
  sudo apt-get install openjdk-8*
```
  Verify that Java has been installed and is running Java 8:
```
  java -version
```
This should return Java version 1.8.

Install necessary Ubuntu packages:

    sudo apt-get install git postgresql libpq-dev unzip clamav-daemon curl tomcat7 libcurl4-openssl-dev libapr1-dev libaprutil1-dev

Create needed directories:

  sudo mkdir /opt/install
  sudo mkdir /opt/fedora; sudo chown tomcat7:tomcat7 /opt/fedora
  sudo mkdir /etc/fcrepo

Tomcat 7 setup

Configure Tomcat7 Java settings:

Retrieve tomcat_conf/tomcat7 file from this github repository and overwrite /etc/default/tomcat7

Make sure that the JAVA_HOME value corresponds to the correct Java installation directory. If not, update JAVA_HOME.

Solr setup

Install Solr (may require sudo):

NOTE: While GW ScholarSpace has not been tested with Solr 7.7.1, a plain Hyrax 2.5.0 instance works with Solr 7.7.1, so it is likely to work with a new instance of GW ScholarSpace. Current (as of GW ScholarSpace v1.2.0) prod and test instances are running Solr 6.4.1 and should be upgraded at a later date.

  cd /opt/install
  wget http://archive.apache.org/dist/lucene/solr/6.4.1/solr-6.4.1.tgz
  tar xzf solr-6.4.1.tgz solr-6.4.1/bin/install_solr_service.sh --strip-components=2
  sudo ./install_solr_service.sh solr-6.4.1.tgz

Verify that Solr started
```
  sudo service solr status
```

Configure a Solr core:

  sudo su - solr -c "/opt/solr/bin/solr create -c scholarspace -n /opt/install/solr/config"

Convert the new Solr core from managed-schema to schema.xml support:

  sudo mv /var/solr/data/scholarspace/conf/managed-schema /var/solr/data/scholarspace/conf/managed-schema.bak

Copy the solr/config/ contents from the samvera/hyrax repo to /var/solr/data/scholarspace/conf/ (this can be accomplished by git clone-ing the hyrax repo, making sure to check out the appropriate tag)
Restart Solr:
```
  sudo service solr restart
```

Fedora setup

Optional: Back up existing Fedora

If creating a new repository, skip this step.

Create a backup of the existing Fedora instance

    sudo mkdir /opt/fedora_backups
    sudo chown -R tomcat7:tomcat7 /opt/fedora_backups
    curl -X POST -u <FedoraUsername>:<FedoraPassword> --data "/opt/fedora_backups" yourserver.com/fcrepo/rest/fcr:backup

Verify that a backup was created in /opt/fedora_backups before proceeding

Set up a PostgreSQL database for Fedora

Create a postgreSQL database user and database for Fedora

  sudo su - postgres
  (postgres)% psql
  postgres=# create user YOURDBUSERNAME with createdb password 'YOURDBPASSWORD';
  postgres=# \q
  (postgres)% createdb -O YOURDBUSERNAME ispn
  (postgres)% exit

Note the database name for Fedora must be ispn

Edit /etc/default/tomcat7 and update these settings with the ispn database username and password, replacing the placeholders for these values:

  JAVA_OPTS="${JAVA_OPTS} -Dfcrepo.postgresql.username=<ADD ISPN DB USERNAME HERE>"
  JAVA_OPTS="${JAVA_OPTS} -Dfcrepo.postgresql.password=<ADD ISPN DB PASSWORD HERE>"

Create a Fedora settings folder

  sudo mkdir /etc/fcrepo

Copy tomcat_conf/fcrepo/infinispan.xml from the Github repo to /etc/fcrepo/infinispan.xml. Set the ownership to tomcat7:
```
  sudo chown -R tomcat7:tomcat7 /etc/fcrepo
```

Set up Fedora with audit support

Copy fcrepo WAR file to tomcat7

  cd /opt/install
  wget https://github.com/fcrepo4-exts/fcrepo-webapp-plus/releases/download/fcrepo-webapp-plus-4.7.1/fcrepo-webapp-plus-audit-4.7.1.war
  sudo cp fcrepo-webapp-plus-audit-4.7.1.war /var/lib/tomcat7/webapps/fcrepo.war

Wait for tomcat to deploy the war file before proceeding to the next step. This can be verified by watching /var/log/tomcat7/catalina.out
Copy the tomcat_conf/fcrepo-webapp/web.xml file from the Github repo to /var/lib/tomcat7/webapps/fcrepo/WEB-INF/web.xml
Ensure that tomcat7 library files are all still owned by tomcat7
```
  sudo chown -R tomcat7:tomcat7 /var/lib/tomcat7
```
Set up Fedora authentication by copying the tomcat_conf/tomcat-users.xml file from the Github repo and overwrite /etc/tomcat7/tomcat-users.xml. Edit tomcat-users.xml and replace the dummy passwords with your preferred secure passwords. (Be sure that your passwords don't contain any characters considered special characters in XML, such as <,>,&,',")
Edit /var/lib/tomcat7/webapps/fcrepo/WEB-INF/classes/config/jdbc-postgresql/repository.json to change the database name from fcrepo to ispn.
Restart Tomcat
```
  sudo service tomcat7 restart
```
Check /var/log/tomcat7/catalina.out to ensure that tomcat7 restarted and deployed fcrepo with no errors.

OPTIONAL: To restore from a Fedora backup:

  curl -X POST -u <FedoraUsername>:<FedoraPassword> --data "/opt/fedora_backups" yourserver.com/fcrepo/rest/fcr:restore

Restart tomcat and validate that the repository has been restored:

  sudo service tomcat7 restart

Optional: Add SSL to Fedora Connections

These instructions are for redirecting port 8080 traffic on Tomcat to port 8443 and running SSL using the Apache Portable Runtime (APR).

Install Tomcat dependencies

       sudo apt-get install libapr1 libapr1-dev libtcnative-1

Add the tomcat7 user to the ssl-cert group in /etc/group
```
        sudo vi /etc/group
```
Generate your SSL certificates and key using the instructions provided here: https://github.com/gwu-libraries/ssl_howto
Update the server.xml file
```
    cd /opt/install
```
Retrieve server_ssl.xml from tomcat_conf/server_ssl.xml in the GitHub repo:
```
    sudo cp tomcat_conf/server_ssl.xml /etc/tomcat7/server.xml
```

Edit /etc/tomcat7/server.xml and replace the dummy values for the following lines with your certificates and keys:

     SSLCertificateFile="/etc/ssl/certs/yourservername.cer"
     SSLCertificateChainFile="/etc/ssl/certs/yourservername.cer"
     SSLCertificateKeyFile="/etc/ssl/private/yourservername.pem"

Create a symbolic link to libtcnative1.so to address a Ubuntu/Tomcat bug

        sudo ln -sv /usr/lib/x86_64-linux-gnu/libtcnative-1.so /usr/lib/

Replace the web.xml files for Fedora with the web_ssl.xml files from the repo:
```
        cd /opt/install
```
Retrieve web_ssl.xml from tomcat_conf/fcrepo-webapp/web_ssl.xml in the GitHub repo
```
        cp tomcat_conf/fcrepo-webapp/web_ssl.xml /var/lib/tomcat7/webapps/fcrepo/WEB-INF/web.xml
```
- Restart Tomcat and test access over HTTPS
```
        sudo service tomcat7 restart
```

Application server

Start with an Ubuntu 18 server.

Install dependencies

Install necessary Ubuntu packages:

        sudo apt update
        sudo apt install git postgresql libpq-dev redis-server unzip clamav-daemon curl imagemagick libreoffice libcurl4-openssl-dev apache2 apache2-dev ffmpeg gnupg2 libxml2 libxml2-dev

Install RVM for multi-users. If the GPG signature verification fails at this step, just follow the instructions in the warning in order to fetch the public key.
```
        curl -L https://get.rvm.io | sudo bash -s stable
        source /etc/profile.d/rvm.sh
```
Edit /etc/group and add yourself (and any other users who will need to run rvm) to the rvm system group. You may then need to log back out and back in as that user.

Log out, then log back in.
Install Ruby:
```
        rvm install ruby-2.7.3
```
Install Rails:
```
        gem install rails -v 5.2.6 -N
```
Also, add export rvmsudo_secure_path=1 to your user's .bashrc file. This will avoid a warning when running rvmsudo.

Create directories

        mkdir /opt/scholarspace
        mkdir /opt/install
        mkdir /opt/xsendfile

Create the necessary users and user groups, and assign folder permissions
```
        sudo adduser --disabled-password scholarspace
```
This also creates a scholarspace group. Edit /etc/group and add users (including www-data) to the new scholarspace group. Additionally, add the scholarspace user to the rvm group.
```
        sudo chown scholarspace:scholarspace /opt/scholarspace
        sudo chown www-data:www-data /opt/xsendfile
```

Get the GW ScholarSpace code:

        sudo su - scholarspace
        cd /opt/scholarspace
        git clone https://github.com/gwu-libraries/scholarspace-hyrax.git

Check out the desired tag, where TAGNUMBER might be, for example, 1.2.0:

        cd scholarspace-hyrax
        git checkout TAGNUMBER

Install gems
```
        gem install bundler
        bundle install --without development --deployment
```
More information on the meaning of these bundle install options can be found at https://bundler.io/v2.0/man/bundle-install.1.html . For a development environment, to install development gems as well, omit the --without development option.

Return to your user account:
```
        exit
```

Create a postgresql user for scholarspace

        sudo su - postgres
        (postgres)% psql
        postgres=# create user YOURDBUSERNAME with createdb password 'YOURSFMDBPASSWORD';

The result should be:

        CREATE ROLE

Create three databases (e.g. scholarspace_dev, scholarspace_test, scholarspace_prod)

        postgres=# \q
        (postgres)% createdb -O YOURDBUSERNAME YOURDEVDBNAME
        (postgres)% createdb -O YOURDBUSERNAME YOURTESTDBNAME
        (postgres)% createdb -O YOURDBUSERNAME YOURPRODDBNAME
        (postgres)% exit

Create the database.yml file

        sudo su - scholarspace
        cd config
        cp database.yml.template database.yml

Edit database.yml to add your specific database names and credentials

Create the solr.yml file
```
        cp solr.yml.template solr.yml
```
Edit solr.yml to add the URL of the Solr instance(s).
Create the blacklight.yml file
```
        cp blacklight.yml.template blacklight.yml
```
Edit blacklight.yml to add the URL of the Solr instance(s).
Create the fedora.yml file
```
        cp fedora.yml.template fedora.yml
```
Edit fedora.yml to add the URL of the Fedora repository(/-ies). Make sure to add the password you configured in tomcat-users.xml when setting up Fedora.
Create the secure secret key. In production, put this in your environment, not in the file.
```
        cd ..
        bundle exec rake secret
```
Copy the key to your clipboard.
```
        cd config
        cp secrets.yml.template secrets.yml
```
(For a dev or test environment, paste secret keys into the secrets.yml file.) For a production environment, as root, add the following to the bottom of /etc/profile, substituting the actual key value that you copied above:
```
        export SECRET_KEY_BASE=<the secret key you generated above>
```

Run the database migrations.

        rake db:migrate RAILS_ENV=production

If you get an error about rake versions, this can be resolved with:

        gem install rake -v 13.0.3   # or other desired version

Copy the production environment configuration:

        cp config/environments/production.rb.template production.rb

Install fits.sh version 1.5.0 (check FITS for the latest 1.5.0 download). Also check the Hyrax repo to verify the latest recommended version of FITS for use with Hyrax.

        cd /usr/local/bin
        sudo wget https://github.com/harvard-lts/fits/releases/download/1.5.0/fits-1.5.0.zip
        sudo unzip fits-1.5.0.zip -d fits-1.5.0
        sudo rm fits-1.5.0.zip
        cd fits-1.5.0
        sudo chmod a+x fits*.sh

Configure the `minter-state` file path

Create a minter folder

        sudo mkdir /opt/scholarspace/scholarspace-minter
        sudo chown -R scholarspace:scholarspace /opt/scholarspace/scholarspace-minter

If an existing minter-state file exists in /tmp/minter-state copy it to the new folder

        cp /tmp/minter-state /opt/scholarspace/scholarspace-minter/

Verify that config.minter_statefile in config/initializers/hyrax.rb matches the path of the new minter directory:

        config.minter_statefile = '/opt/scholarspace/scholarspace-minter/minter-state'

Configure the tmp path

Create a tmp folder

        sudo mkdir /opt/scholarspace/scholarspace-tmp
        sudo chown -R scholarspace:scholarspace /opt/scholarspace/scholarspace-tmp

Verify that config.temp_file_base in config/initializers/hyrax.rb matches the path of the new tmp directory:

        config.temp_file_base = '/opt/scholarspace/scholarspace-tmp'

Configure path for libre-office

Verify that config.libreoffice_path in config/initializers/hyrax.rb is as follows:

        config.libreoffice_path = "/usr/bin/soffice"

Configure derivatives path for Hyrax

Create a derivatives folder on your application server, writeable to any user in the scholarspace group:

        sudo mkdir /opt/scholarspace/scholarspace-derivatives
        sudo chown -R scholarspace:scholarspace /opt/scholarspace/scholarspace-derivatives
        sudo chmod 775 -R /opt/scholarspace/scholarspace-derivatives

Add config.derivatives_path to config/initializers/hyrax.rb

        config.derivatives_path = "/opt/scholarspace/scholarspace-derivatives/"

Configure Contact form emailing

In config/initializers/hyrax.rb, set the email to which contact form submissions will be sent:

        config.contact_email =

Configure the account from which contact form emails will be sent, in config/environments/production.rb (or other desired environment), by configuring the user name and password in the config.action_mailer.smtp_settings block.
(Optional) Edit config/initializers/mailboxer.rb with email account from which to send messages and notifications:

        config.default_from =

Configure geonames username

In config/initializers/hyrax.rb, set the geonames username:

        config.geonames_username =

Configure Passenger and Apache2

Set up Passenger, and create Passenger config for Apache

        gem install passenger -v 6.0.9
        rvmsudo passenger-install-apache2-module

Select Ruby from the list of languages. The install script will direct you to copy several lines for the Apache configuration. They will look something similar to:

LoadModule passenger_module /usr/local/rvm/gems/ruby-2.7.3/gems/passenger-6.0.9/buildout/apache2/mod_passenger.so
<IfModule mod_passenger.c>
  PassengerRoot /usr/local/rvm/gems/ruby-2.7.3/gems/passenger-6.0.9
  PassengerDefaultRuby /usr/local/rvm/gems/ruby-2.7.3/wrappers/ruby
</IfModule>

Create /etc/apache2/conf-available/passenger.conf using the lines pasted from the Passenger install script.

Add the following line into passenger.conf:

  PassengerAllowEncodedSlashes on

Enable the Apache passenger.conf file and the rewrite Apache mod

        sudo a2enconf passenger
        sudo a2enmod rewrite
        sudo service apache2 restart

Create and enable an Apache2 virtual host

Retrieve scholarspace.conf from apache_conf/scholarspace.conf in the GitHub repo; copy to /etc/apache2/sites-available/scholarspace.conf

Retrieve scholarspace-ssl.conf from apache_conf/scholarspace-ssl.conf in the GitHub repo; copy to /etc/apache2/sites-available/scholarspace-ssl.conf. Adjust paths as needed.

Enable modssl:
```
        sudo a2enmod ssl
```
Generate certificates and place them in paths referenced in scholarspace-ssl.conf (modify the paths in scholarspace-ssl.conf if needed). Cert file names should also match. For a development instance, a self-signed cert is an option. Instructions provided here: https://github.com/gwu-libraries/ssl_howto
```
        sudo a2dissite 000-default.conf
        sudo a2ensite scholarspace.conf
        sudo a2ensite scholarspace-ssl.conf
```
If you are not implementing Shibboleth, you MUST remove the lines in scholarspace-ssl.conf referencing shibboleth.

Install mod_xsendfile

        cd /opt/install
        git clone https://github.com/nmaier/mod_xsendfile.git
        cd mod_xsendfile
        sudo apxs2 -cia mod_xsendfile.c
        sudo service apache2 restart

(Optional) Install etd-loader

Install the etd-loader application in /opt/etd-loader as per instructions at https://github.com/gwu-libraries/etd-loader

When configuring config.py, ensure that it contains the following values:

ingest_path = "/opt/scholarspace/scholarspace-hyrax"
ingest_command = "rake RAILS_ENV=production gwss:ingest_etd"

Set policy for file uploads

Edit /etc/ImageMagick-6/policy.xml to allow/disallow file types that may be processed for derivatives. For example, to allow PDF files, remove the line that blocks PDF files.

Final Deployment

Prepare databases and assets for production

        cd /opt/scholarspace/scholarspace-hyrax
        rake assets:precompile RAILS_ENV=production 
        sudo service apache2 restart

Create the user roles

Run the rake task that creates user roles called admin and content-admin:

        rake gwss:create_roles RAILS_ENV=production

At the rails console, add an initial user to the admin role. Make sure that your admin user has logged in at least once via the app's web UI (which should now be working).

        % RAILS_ENV=production rails c
        > r = Role.find_by_name('admin')
        > r.users << User.find_by_user_key('YOUR_ADMIN_USER_EMAIL@gwu.edu')
        > r.save

We can add the content-admin users as desired through the /roles UI.

Create `ETDs` admin set

First, create the default administrative set (Hyrax won't allow you to create any admin sets until you've created this first one):

        su - scholarspace
        cd /opt/scholarspace/scholarspace-hyrax
        RAILS_ENV=production rake hyrax:default_admin_set:create

Log in to the application as the admin user. Navigate to the Administrative page, and create an Administrative set called ETDs.

Populate the initial content blocks

Log in as the admin user. Edit the Above and Help pages; paste in the HTML from the repo TODO: Add initial HTML to repo

(Optional) Enable citation pages

Uncomment the following line in config/initializers/hyrax.rb and set to true
```
  config.citations = true
```

Post-deployment

Add content-admin users
Generate sitemap

bundle exec rake gwss:sitemap_queue_generate RAILS_ENV=production

Set up log rotation. production.log can grow quite large, quite quickly, without any sort of compression and/or rotation configured. A typical logrotate configuration would entail adding a configuration file into /etc/logrotate.d/. For example, create a file in /etc/logrotate.d/ called scholarspace-hyrax containing the following:

/opt/scholarspace/scholarspace-hyrax/log/production.log {
        daily
        missingok
        rotate 10
        compress
  delaycompress
        notifempty
        create 664 scholarspace scholarspace
}

Configure colors. As the admin user, go to the admin dashboard --> Settings --> Appearance. Set the Header background color to 004165 (You will need to select the color sliders, then RGB sliders, then enter the hex code in the Hex Color # box.)
Also under Settings, add back Pages and Content Blocks

(NEEDS REFRESH - see #83) (Optional) Add Google Analytics

Enable Google Analytics in config/initializers/hyrax.rb by editing the following lines:

   # Enable displaying usage statistics in the UI
   # Defaults to FALSE
   # Requires a Google Analytics id and OAuth2 keyfile.  See README for more info
   config.analytics = true
  
   # Specify a Google Analytics tracking ID to gather usage statistics
   config.google_analytics_id = 'UA-99999999-1'

   # Specify a date you wish to start collecting Google Analytic statistics for.
   config.analytic_start_date = DateTime.new(2015,11,10)

Copy the analytics.yml.template file in config

  cp config/analytics.yml.template config/analytics.yml

Populate the analyitcs.yml file with your Google Analyitcs credentials. See: https://github.com/samvera/hyrax/wiki/Hyrax-Management-Guide#analytics-and-usage-statistics for setup details. Note that Hyrax seems to expect the .p12 file version of the private key, rather than the json version.
Set up a cron job to import GA stats nightly

Test the script to make sure that it can run successfully. Make sure the script has execute permissions. Your analytics.yml file must also be set up correctly in order for the script to succeed.
```
    cd /opt/scholarspace/scholarspace-hyrax/script
    sudo chmod +x import_stats.sh
    ./import_stats.sh production
```
If it runs successfully, proceed with adding the cron job:
```
    crontab -e
```
Add a line similar to the following:
```
  0 5 * * * /opt/scholarspace/scholarspace-hyrax/script/import_stats.sh production
```

(NEEDS REFRESH - see #11) (Optional) Set up Shibboleth integration on the GW ScholarSpace server:

Please refer to https://github.com/gwu-libraries/shibboleth for the recommended steps for setting up the Shibboleth integration.

If Shibboleth has been setup on the GW ScholarSpace Server, enable Shibboleth in the appropriate environment file (ie: config/environments/production.rb):
```
   config.shibboleth = true
```

Note about ghostscript and ImageMagick

Hyrax can be ficky about versions of ghostscript and ImageMagick. A working combination seems to be: ghostscript 9.26 with ImageMagick 6.9.7-4

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

README.md

README.md

scholarspace-hyrax

Getting started

Repository server

Tomcat 7 setup

Solr setup

Fedora setup

Optional: Back up existing Fedora

Set up a PostgreSQL database for Fedora

Create a Fedora settings folder

Set up Fedora with audit support

Optional: Add SSL to Fedora Connections

Application server

Install dependencies

Configure the `minter-state` file path

Configure the tmp path

Configure path for libre-office

Configure derivatives path for Hyrax

Configure Contact form emailing

Configure geonames username

Configure Passenger and Apache2

(Optional) Install etd-loader

Set policy for file uploads

Final Deployment

Create the user roles

Create `ETDs` admin set

Populate the initial content blocks

(Optional) Enable citation pages

Post-deployment

(NEEDS REFRESH - see #83) (Optional) Add Google Analytics

(NEEDS REFRESH - see #11) (Optional) Set up Shibboleth integration on the GW ScholarSpace server:

Note about ghostscript and ImageMagick

Files

README.md

Latest commit

History

README.md

File metadata and controls

scholarspace-hyrax

Getting started

Repository server

Tomcat 7 setup

Solr setup

Fedora setup

Optional: Back up existing Fedora

Set up a PostgreSQL database for Fedora

Create a Fedora settings folder

Set up Fedora with audit support

Optional: Add SSL to Fedora Connections

Application server

Install dependencies

Configure the minter-state file path

Configure the tmp path

Configure path for libre-office

Configure derivatives path for Hyrax

Configure Contact form emailing

Configure geonames username

Configure Passenger and Apache2

(Optional) Install etd-loader

Set policy for file uploads

Final Deployment

Create the user roles

Create ETDs admin set

Populate the initial content blocks

(Optional) Enable citation pages

Post-deployment

(NEEDS REFRESH - see #83) (Optional) Add Google Analytics

(NEEDS REFRESH - see #11) (Optional) Set up Shibboleth integration on the GW ScholarSpace server:

Note about ghostscript and ImageMagick

Configure the `minter-state` file path

Create `ETDs` admin set