Expose everything including the kitchen sink

[DonRichards]

For testing, demo, documentation, etc. it's helpful to have everything on and easy to find. This adds a optional make command to expose the frontend URLs for Drupal, Traefik, Fedora, Blazegraph, Activemq, Solr, Cantaloupe, Matomo, and Code Server.

The output of running this will give the user the URLs specific to their configuration.

$ make kitchen_sink
...
You can now access the following endpoints:
Drupal:                               https://islandora.traefik.me
Traefik:                              http://islandora.traefik.me:8080/dashboard/#/
Fedora:                               http://islandora.traefik.me:8081/fcrepo/rest
Blazegraph:                           http://islandora.traefik.me:8082/bigdata/#splash
Activemq:                             http://islandora.traefik.me:8161
Solr:                                 http://islandora.traefik.me:8983/solr/#/
Cantaloupe:                           https://islandora.traefik.me/cantaloupe
Matomo:                               https://islandora.traefik.me/matomo/
Code Server (Code Editor in Browser): https://islandora.traefik.me:8443

The help message returns the following text

Turn on/expose all endpoints for demo/testing; CANTALOUPE, MATOMO, DRUPAL, MYSQL, POSTGRES, TRAEFIK DASHBOARD, FEDORA, BLAZEGRAPH, ACTIVEMQ, SOLR, CODE SERVER. NOT FOR PRODUCTION!!!

To test

The assumption the user has already started isle-dc and doesn't have any or all of the items already exposed. This looks within the .env file and replaces false to true for each one of the above mentioned services to expose the service.

Run the following

$ make kitchen_sink

Navigate to each one of the links it provided and verify all of the URLs work. Please note that http & https are NOT interchangeable. Please use the exact URL indicated in the terminal.

Also review the added documentation for clarity on how to use this make command.

Additional house cleaning

1. The services URLs list had all services set to https and this is incorrect.
2. Link to /secrets/live/DRUPAL_DEFAULT_ACCOUNT_PASSWORD was incorrect
3. There was some white space at the end of a few lines

[ysuarez]

I could not get the make target "kitchen_sink" work for me, but for now I am assuming my setup is to blame.

It seems that this ifeq Makefile line

isle-dc/Makefile

Line 347 in 55e71db

ifeq ($(shell test -e docker-compose.yml && echo -n yes),yes)

is evaluating as false on my macOS machine, since I am getting this output...

Exposing everything including the kitchen sink...

Problem: Run this after you've run one of the make commands to build isle-dc (up, demo, local, etc.). Exiting...

from the related else clause...

isle-dc/Makefile

Lines 375 to 378 in 55e71db

	else
	echo ""
	echo "Problem: Run this after you've run one of the make commands to build isle-dc (up, demo, local, etc.). Exiting..."
	endif

For the record, I ran the kitchen_sink target after I had run make local successfully. I also made sure I had a valid .env file (with ENVIRONMENT=local) and there was a docker docker-compose.yml available.

I wondered if there could be a permission issue to find the docker-compose.yml file, but the permissions looked OK to me.

-rw-r--r--   1 ysuarez  staff   5363 Mar 23 09:34 .env
....
-rw-r--r--  1 ysuarez  staff  18145 Mar 23 09:35 docker-compose.yml

I will try to test this again tomorrow after once again deleting all images, volumes, etc. to try to eliminate potential issues. Though I would not be surprised if this PR worked fine for others if they also want to test it.

[ysuarez]

@DonRichards I tried testing this branch with the changes from today (2022-03-30) and I was not able to build make local, so I don't think I should even try to now test the make kitchen_sink

Here is the last few lines before make local stops running with an error...

Command migrate:import was not found. Drush was unable to query the databas  
                                                           e. As a result, many commands are unavailable. Re-run your command with --d  
                  ebug to see relevant log messages.                                           
                                                                                                                                                                              

                                                      make[1]: *** [Makefile:156: run-islandora-migrations] Error 1
make[1]: Leaving directory '/Users/ysuarez/Documents/isle_git_root/isle-dc_community_tests'
make: *** [Makefile:350: local] Error 2

Also, when I try to visit https://islandora.traefik.me/ I get redirected to https://islandora.traefik.me/core/install.php and see the first page of a Drupal 9 install page.

For the record, I tried running make local twice, and before both times I removed all image, containers, volumes, networks, and of ran make clean. Both times I got the same errors.

Any suggestions?

[DonRichards]

@ysuarez If you change TAG=1.0.0-alpha-15 & DRUPAL_INSTALL_PROFILE=standard in the .env file does it work?

[ysuarez]

@DonRichards I tried again making those I got a sed error on macOS when running make local or make demo, and I am not getting a chance to get to run make kitchen_sink

Here is the make demo error...

} vbo-exec media emit_media_event --configuration="queue=islandora-indexing-triplestore-index&event=Update"'
Action processing results: Emit a media event to a queue/topic (121).
                                                                     make[1]: Leaving directory '/Users/ysuarez/Documents/isle_git_root/isle-dc_community_tests'
sed: illegal option -- -
usage: sed script [-Ealnru] [-i extension] [file ...]
	sed [-Ealnu] [-i extension] [-e script] ... [-f script_file] ... [file ...]
make: *** [Makefile:334: demo] Error 1

After doing some research, Making sed work with “edit in place” in BOTH unix/linux and macOS had been an issue for a while, for example this 10 year old post https://stackoverflow.com/questions/5694228/sed-in-place-flag-that-works-both-on-mac-bsd-and-linux

but I got make local to work on Big Sur by replacing …
sed --in-place=''

with….
sed -I ''

[DonRichards]

@ysuarez It should work now on either Linux or Mac.

[ysuarez]

Testing this today. May not be able to test with make demo since it appears the /codebase ends up empty(?) so the make commands does not run.

[ysuarez]

I am still having issues with make local, the following ifeq is getting skipped, even though I have a cocker-compose.yml file.

isle-dc/Makefile

Line 381 in 4aa3b96

ifeq ($(shell test -e docker-compose.yml && echo -n yes),yes)

[ysuarez]

Today at the "tech call" it was discussed that I am probably running into an issue with macOS and using the bash shell. We will need to find a workaround for macOS bash, but I will try to test again with zsh which is the macOS Big Sur's default shell.

[ysuarez]

I tried testing again and looks like almost everything ran OK. The containers were stopped/removed. Changes were made in the .env file. The code slept for 10 seconds, and then building the Drupal container gave me an error.

My uneducated understanding is that I got errors related to line 183...

isle-dc/Makefile

Lines 182 to 183 in 10af581

	solr-cores:
	docker-compose exec -T drupal with-contenv bash -lc "for_all_sites create_solr_core_with_default_config"

These are the errors I got at the end.

                            // Import the listed configuration changes?: yes.                              

                                                                                                            [notice] Synchronized configuration: update key.key.islandora_rsa_key.
                                               [notice] Finalizing configuration synchronization.
                                                                                                  [success] The configuration was imported successfully.
                    islandora_defaults is not installed.  Skipping configuration
                                                                                 [success] Successfully enabled: search_api_solr
                                                                                                                                 [warning] Message: There have been some configuration changes. It is advisable to download and 
                                                                                            deploy an updated config.zip to your Solr server.

          [warning] Message: A new Solr field type has been installed due to configuration changes. It is 
                                                                                                          advisable to download and deploy an updated config.zip to your Solr server.

                                                  [success] Successfully uninstalled: search
                                                                                            matomo is not installed.  Skipping configuration
        openseadragon is not installed.  Skipping configuration
                                                               islandora_defaults is not installed.  Skipping configuration

                                                                                                                           In SolrCommandHelper.php line 41:
                                          
                                            Unknown server  
                                                                              

                                                                              search-api-solr:get-server-config [-h|--help] [-q|--quiet] [-v|vv|vvv|--verbose] [-V|--version] [--ansi] [--no-ansi] [-n|--no-interaction] [-d|--debug] [-y|--yes] [--no] [--remote-host REMOTE-HOST] [--remote-user REMOTE-USER] [-r|--root ROOT] [-l|--uri URI] [--simulate] [--pipe] [-D|--define DEFINE] [--xh-link XH-LINK] [--druplicon] [--notify [NOTIFY]] [--] <command> <server_id> [<file_name>] [<solr_version>]



                                                                                              ERROR: Could not generate SOLR config.zip!
    In Drupal, check Configuration -> Search API -> SOLR Server, and use the
                                                                            + Get config.zip option which should give you information into the actual error.


                        make[1]: *** [Makefile:183: solr-cores] Error 1
make[1]: Leaving directory '/Users/ysuarez/Documents/isle_git_root/isle-dc_community_tests'
make: *** [Makefile:396: kitchen_sink] Error 2

Hope this helps.