solidcommunity.net servers

Some history solid.community (deprecated) --> solidcommunity.net

Howto

• error 500 on https://<pod>/ : this may mean that https://<pod>/.meta is invalid.
  • Goto that link
  • login and delete the content
  • try url https://<pod>/. If it is OK. You MUST add the following content in url https://<pod>/.meta. Replace with your podName. Check twice.

# Root Meta resource for the user account
# Used to discover the account's WebID URI, given the account URI
<https://<your pod>/profile/card#me>
  <http://www.w3.org/ns/solid/terms#account>
  </>.

• fully remove a pod : reclaim a subdomain
• restore a pod folder (or folder in a pod) see backup
• blacklist pod names issue#6
• migrate old contacts group from NSS < v5.6.7 issue#50

Servers Management

Server Host

solidcommunity.net is hosted on a paid plan on DigitalOcean There are 2 volumes :

• /mnt/volume_lon1_01. It’s 100G, and is approx 50% utilized right now. (june 2021)
• /mnt/volume_lon1_03 is used for backup.

server host lock-out

If it appears that servers are not at all accessible.

• call for a restart by Server host Digital Ocean managers
• there is no clear identification of the issue that appeared with ubuntu 20 Clearing the buffer/cache regularly seems to to avoid the pb

# echo 3 | tee /proc/sys/vm/drop_caches

various

• Open a new port :

  First open the port on the server console. Then open new port in linux Ubuntu https://www.digitalocean.com/community/tutorials/opening-a-port-on-linux with ufw

Servers administration

let's encrypt certificates issue#18

• solidcommunity.net is delivered by gandi
• certificate for solidcommunity.net are created with the gandi plugin.
  • /etc/letsencrypt/renewal/solidcommunity.net.conf
  • /etc/letsencrypt/gandi.ini
• certificate update

MANUAL
   # certbot renew --dry-run # and after that run : certbot renew
   # pm2 restart all         # both servers (solidcommunity.net and solidcommunity.net:8443)

AUTO
 - systemd certs renewal every 2 months
   /lib/systemd/system/certbot.timer
 
 - cron
   # restart on monday every 3 weeks at 1am to eventually update the certs that are renewed with systemd every 2 months
   0 1 * * 6 [[ $(("( $(date +%s) - $(date +%s --date=20220606) ) / 86400 % 21")) -eq  0 ]] && pm2 restart all

nginx

• redirect http to https issue#8
• nginx defaults parameter are limiting the unit file size

pm2 : solid-server instances are managed using pm2

solidcommunity.net backup

• a root cron job every sunday at 1.00 am the server being CET - 2 hours it should be not heavily used worldwide.

  0 1 * * 0 rsync -a /mnt/volume_lon1_01/solid.community/ /mnt/volume_lon1_03/solidcommunity.net/data --stats > /var/log/solid/$(date +\%Y\%m\%d)-data.log
  0 2 * * 0 rsync -a /mnt/volume_lon1_01/.db/ /mnt/volume_lon1_03/solidcommunity.net/.db --stats > /var/log/solid/$(date +\%Y\%m\%d)-.db.log

• Summary logs available at /var/log/solid
• rsync backup do not delete files for the time being. It shall be implemented someday with a backup rotation.

solidcommunity.net NSS instance

• solid-config file :

  root@solidcommunity:/solid-config  
  root@solidcommunity:~/solid-config# cat config.json
  {
    "root": "/mnt/volume_lon1_01/solid.community",
    "port": "443",
    "serverUri": "https://solidcommunity.net",
    "webid": true,
    "mount": "/",
    "configPath": "./config",
    "configFile": "./config.json",
    "dbPath": "/mnt/volume_lon1_01/.db",
    "sslKey": "/etc/letsencrypt/live/solidcommunity.net/privkey.pem",
    "sslCert": "/etc/letsencrypt/live/solidcommunity.net/fullchain.pem",
    "multiuser": true,
    "corsProxy": "/proxy",
    "redirectHTTPFrom": 80,
    "disablePasswordChecks": false,
    "server": {
      "name": "solidcommunity.net",
      "description": "An experimental community solid server",
      "logo": ""
    },
    "email": {
      "host": "smtp.sendgrid.net",
      "port": "465",
      "sender": "no-reply@solidcommunity.net",
      "secure": true,
      "auth": {
        "user": "apikey",
        "pass": "XXXXXXXXXX"
      }
    }
  }

• config folder : /usr/local/bin/config
• solid app : /usr/local/bin --> ../lib/node_modules/solid-server/bin/solid

  # cd /usr/local/lib
  # npm install solid-server ## this is a local installation

• pm2 new instance

  # cd /usr/local/bin
  # pm2 start solid -- start --config-file ~/solid-config/config.json
  # pm2 save

• pm2 update

  # pm2 restart solid

• storage : due to a harsh migration from solid.community in automn 2020. The implementation is clumsy
  • data location : /mnt/volume_lon1_01/solid.community
  • .db location : /mnt/volume_lon1_01/.db

solidcommunity.net:8443 NSS test-server instance

• config file : /home/solid/test-solid/config.json must be created once

root@solidcommunity:/home/solid/test-solid# cat config.json
{
  "root": "/mnt/volume_lon1_01/test-solidcommunity.net/data",
  "port": "8443",
  "serverUri": "https://solidcommunity.net:8443",
  "webid": true,
  "mount": "/",
  "configPath": "./config",
  "configFile": "./config.json",
  "dbPath": "/mnt/volume_lon1_01/test-solidcommunity.net/.db",
  "sslKey": "/etc/letsencrypt/live/solidcommunity.net/privkey.pem",
  "sslCert": "/etc/letsencrypt/live/solidcommunity.net/fullchain.pem",
  "multiuser": true,
  "corsProxy": "/proxy",
  "redirectHTTPFrom": 80,
  "disablePasswordChecks": false,
  "server": {
    "name": "test solidcommunity.net",
    "description": "An experimental community solid server",
    "logo": ""
  },
  "email": {
    "host": "smtp.sendgrid.net",
    "port": "465",
    "sender": "no-reply@solidcommunity.net",
    "secure": true,
    "auth": {
      "user": "apikey",
      "pass": "XXXXXXXXXX"
    }
  }
}

• config folder : /home/solid/test-solid/config always uses default as test
• app

  # cd /home/solid/test-solid
  # git clone https://github.com/solid/node-solid-server
  # npm ci

• new instance

  # pm2 start --name test-solid /home/solid/test-solid/bin/solid -- start
  # pm2 save

• update

  # pm2 restart test-solid

• storage : data and .db

root@solidcommunity:/mnt/volume_lon1_01/test-solidcommunity.net# ls -a
.  ..  .db  data

CSS test server

CSS server test on solidcommunity.net

- git clone https://github.com/SolidOS/css-mashlib /home/solid/test-css
- mkdir /home/solid/css-config
- cp -r /home/solid/test-css/config /home/solid/css-config
- cd /home/solid/css-config

root@solidcommunity:/home/solid/css-config# ls -al
total 40
drwxr-xr-x 2 root root 4096 Mar  7 17:23 .
drwxr-xr-x 5 root root 4096 Feb  9 16:06 ..
-rw-r--r-- 1 root root 2862 Feb  9 16:06 config-mashlib.json
-rw-r--r-- 1 root root 3649 Feb  9 16:06 https-mashlib-subdomain-file.json
-rw-r--r-- 1 root root 3646 Feb  9 16:06 https-mashlib-suffix-file.json
-rw-r--r-- 1 root root  419 Feb  9 17:58 override-pod-templates-v5.json
-rw-r--r-- 1 root root  200 Feb  9 17:46 subdomain-3000.sh
-rw-r--r-- 1 root root 3725 Feb  9 17:30 subdomain-mashlib-v5.json
-rw-r--r-- 1 root root  194 Mar  7 17:23 suffix-3001.sh
-rw-r--r-- 1 root root 3722 Feb  9 16:06 suffix-mashlib-v5.json
root@solidcommunity:/home/solid/css-config#

CSS subdomain test server : port 3000

root@solidcommunity:/home/solid/css-config# cat subdomain-mashlib-v5.json
{
  "@context": "https://linkedsoftwaredependencies.org/bundles/npm/@solid/community-server/^5.0.0/components/context.jsonld",
  "import": [
    "css:config/app/main/default.json",
    "css:config/app/init/initialize-root.json",
    "css:config/app/setup/disabled.json",
    "css:config/app/variables/default.json",
    "css:config/http/handler/default.json",
    "css:config/http/middleware/websockets.json",

    "css:config/http/static/default.json",
    "css:config/identity/access/public.json",
    "css:config/identity/email/default.json",
    "css:config/identity/handler/default.json",
    "css:config/identity/ownership/token.json",
    "css:config/identity/pod/static.json",
    "css:config/identity/registration/enabled.json",
    "css:config/ldp/authentication/dpop-bearer.json",
    "css:config/ldp/authorization/webacl.json",
    "css:config/ldp/handler/default.json",
    "css:config/ldp/metadata-parser/default.json",
    "css:config/ldp/metadata-writer/default.json",
    "css:config/ldp/modes/default.json",
    "css:config/storage/backend/file.json",
    "css:config/storage/key-value/resource-store.json",
    "css:config/storage/middleware/default.json",
    "css:config/util/auxiliary/acl.json",
    "css:config/util/identifiers/subdomain.json",

    "css:config/util/logging/winston.json",
    "css:config/util/representation-conversion/default.json",
    "css:config/util/resource-locker/memory.json",
    "css:config/util/variables/default.json"
  ],
  "@graph": [
    {
      "comment": [
        "A filesystem-based server with Databrowser as UI.",
        "Derived from config/example-https-file.json"
      ]
    },

    {
      "comment": "The key/cert values should be replaces with paths to the correct files. The 'options' block can be removed if not needed.",
      "@id": "urn:solid-server:default:ServerFactory",
      "@type": "WebSocketServerFactory",
      "baseServerFactory": {
        "@id": "urn:solid-server:default:HttpServerFactory",
        "@type": "BaseHttpServerFactory",
        "handler": { "@id": "urn:solid-server:default:HttpHandler" },
        "options_showStackTrace": { "@id": "urn:solid-server:default:variable:showStackTrace" },
        "options_https": true,
        "options_key": "/etc/letsencrypt/live/solidcommunity.net/privkey.pem",
        "options_cert": "/etc/letsencrypt/live/solidcommunity.net/fullchain.pem"
      },
      "webSocketHandler": {
        "@type": "UnsecureWebSocketsProtocol",
        "source": { "@id": "urn:solid-server:default:ResourceStore" }
      }
    },

    {
      "comment": "Serve Databrowser as default representation",
      "@id": "urn:solid-server:default:DefaultUiConverter",
      "@type": "ConstantConverter",
      "contentType": "text/html",
      "filePath": "./node_modules/mashlib/dist/databrowser.html",
      "options_container": true,
      "options_document": true,
      "options_minQuality": 1,
      "options_disabledMediaRanges": [
        "image/*",
        "application/pdf"
      ]
    },

    {
      "comment": "Serve Mashlib static files.",
      "@id": "urn:solid-server:default:StaticAssetHandler",
      "@type": "StaticAssetHandler",
      "assets": [
        {
          "StaticAssetHandler:_assets_key": "/mash.css",
          "StaticAssetHandler:_assets_value": "./node_modules/mashlib/dist/mash.css"
        },
        {
          "StaticAssetHandler:_assets_key": "/mashlib.min.js",
          "StaticAssetHandler:_assets_value": "./node_modules/mashlib/dist/mashlib.min.js"
        },
        {
          "StaticAssetHandler:_assets_key": "/mashlib.min.js.map",
          "StaticAssetHandler:_assets_value": "./node_modules/mashlib/dist/mashlib.min.js.map"
        }
      ]
    }
  ]
}

root@solidcommunity:/home/solid/css-config# cat subdomain-3000.sh
npx community-solid-server -c ../css-config/subdomain-mashlib-v5.json ../css-config/override-pod-templates-v5.json -f /mnt/volume_lon1_01/test-css-subdomain -p 3000 -b https://solidcommunity.net:3000
root@solidcommunity:/home/solid/css-config#

- cd /home/solid/test-css
- npm ci
- pm2 del css-subdomain
- pm2 start /home/solid/css-config/subdomain-3000.sh --name css-subdomain

CSS suffix test server port 3001

root@solidcommunity:/home/solid/css-config# cat suffix-mashlib-v5.json
{
  "@context": "https://linkedsoftwaredependencies.org/bundles/npm/@solid/community-server/^5.0.0/components/context.jsonld",
  "import": [
    "css:config/app/main/default.json",
    "css:config/app/init/initialize-root.json",
    "css:config/app/setup/disabled.json",
    "css:config/app/variables/default.json",
    "css:config/http/handler/default.json",
    "css:config/http/middleware/websockets.json",

    "css:config/http/static/default.json",
    "css:config/identity/access/public.json",
    "css:config/identity/email/default.json",
    "css:config/identity/handler/default.json",
    "css:config/identity/ownership/token.json",
    "css:config/identity/pod/static.json",
    "css:config/identity/registration/enabled.json",
    "css:config/ldp/authentication/dpop-bearer.json",
    "css:config/ldp/authorization/webacl.json",
    "css:config/ldp/handler/default.json",
    "css:config/ldp/metadata-parser/default.json",
    "css:config/ldp/metadata-writer/default.json",
    "css:config/ldp/modes/default.json",
    "css:config/storage/backend/file.json",
    "css:config/storage/key-value/resource-store.json",
    "css:config/storage/middleware/default.json",
    "css:config/util/auxiliary/acl.json",
    "css:config/util/identifiers/suffix.json",

    "css:config/util/logging/winston.json",
    "css:config/util/representation-conversion/default.json",
    "css:config/util/resource-locker/memory.json",
    "css:config/util/variables/default.json"
  ],
  "@graph": [
    {
      "comment": [
        "A filesystem-based server with Databrowser as UI.",
        "Derived from config/example-https-file.json"
      ]
    },

    {
      "comment": "The key/cert values should be replaces with paths to the correct files. The 'options' block can be removed if not needed.",
      "@id": "urn:solid-server:default:ServerFactory",
      "@type": "WebSocketServerFactory",
      "baseServerFactory": {
        "@id": "urn:solid-server:default:HttpServerFactory",
        "@type": "BaseHttpServerFactory",
        "handler": { "@id": "urn:solid-server:default:HttpHandler" },
        "options_showStackTrace": { "@id": "urn:solid-server:default:variable:showStackTrace" },
        "options_https": true,
        "options_key": "/etc/letsencrypt/live/solidcommunity.net/privkey.pem",
        "options_cert": "/etc/letsencrypt/live/solidcommunity.net/fullchain.pem"
      },
      "webSocketHandler": {
        "@type": "UnsecureWebSocketsProtocol",
        "source": { "@id": "urn:solid-server:default:ResourceStore" }
      }
    },

    {
      "comment": "Serve Databrowser as default representation",
      "@id": "urn:solid-server:default:DefaultUiConverter",
      "@type": "ConstantConverter",
      "contentType": "text/html",
      "filePath": "./node_modules/mashlib/dist/databrowser.html",
      "options_container": true,
      "options_document": true,
      "options_minQuality": 1,
      "options_disabledMediaRanges": [
        "image/*",
        "application/pdf"
      ]
    },

    {
      "comment": "Serve Mashlib static files.",
      "@id": "urn:solid-server:default:StaticAssetHandler",
      "@type": "StaticAssetHandler",
      "assets": [
        {
          "StaticAssetHandler:_assets_key": "/mash.css",
          "StaticAssetHandler:_assets_value": "./node_modules/mashlib/dist/mash.css"
        },
        {
          "StaticAssetHandler:_assets_key": "/mashlib.min.js",
          "StaticAssetHandler:_assets_value": "./node_modules/mashlib/dist/mashlib.min.js"
        },
        {
          "StaticAssetHandler:_assets_key": "/mashlib.min.js.map",
          "StaticAssetHandler:_assets_value": "./node_modules/mashlib/dist/mashlib.min.js.map"
        }
      ]
    }
  ]
}

root@solidcommunity:/home/solid/css-config# cat suffix-3001.sh
npx community-solid-server -c ../css-config/suffix-mashlib-v5.json ../css-config/override-pod-templates-v5.json -f /mnt/volume_lon1_01/test-css-suffix -p 3001 -b https://solidcommunity.net:3001
root@solidcommunity:/home/solid/css-config#

- cd /home/solid/test-css
- npm ci
- pm2 del css-suffix
- pm2 start /home/solid/css-config/suffix-3001.sh --name css-suffix