Swapout wodby image with Solr official image work

[g7morris]

• Removes container name values in docker-compose
• Updates solr image with official 8.4.1.
• Adds a few .env values as needed
• Attempts to bind mount config.zip for users to have a unique schema.xml

[noahwsmith]

Reference link: https://www.drupal.org/docs/8/modules/search-api-solr/search-api-solr-howtos/geospatial-search

[g7morris]

@nikathone & @dannylamb Just a heads up here:

Okay I've done some work on this and it appears that there is a Drupal config change to be made as a result of using the new Solr official image instead of wodby.

In http://idcp.localhost:8000/admin/config/search/search-api/server/default_solr_server/ or Home > Administration > Configuration > Search and metadata > Search API Solr Server

• the section Configure Standard Solr connector
• the Solr path should change from /solr to /

I'm not done testing with the schema yet but I had to make this change to get the Solr image / container working again. I'll make a ticket for the change once I issue the PR.

[Update]
I'm assuming this change needs to occur in the config/drupal/islandora-starter-config.tar.gz file

aka /codebase/config/sync either search_api_solr_settings.yml or search_api.server.default_solr_server.yml

Is there a reason why we're using a tar.gz to do this? Seems clunky and a blocker for edits of this nature. Is there a better way to do this perhaps? @nikathone & @dannylamb ? That way I could just issue a fuller PR with this change to the respective Drupal config. Apologies in advance if I'm being daft and not looking in the right spot?

[g7morris]

Running into challenges currently with the materials extracted from the Drupal config.zip (solr_0.x_config)

While I can create a Solr core called islandora, sadly it is missing the appropriate schema.xml. However in Solr 8.x+, schema.xml is considered classic and not the current method of using managed-schema. Additionally, despite multiple attempts to bindmount additional files and configs from the config/solr directory, the resulting container will crash and/or not create a core due to permission issues with bind-mounts. (user is 1000 not solr)

Or when the container doesn't crash I see in the Solr admin panel:

islandora: org.apache.solr.common.SolrException:org.apache.solr.common.SolrException: Could not load conf for core islandora: Error loading solr config from /var/solr/data/islandora/conf/solrconfig.xml

and that solrconfig.xml doesn't exist.

So far I've followed steps outlined here: (none work for bind-mounts that attempt to introduce a new schema or configuration)

• https://www.drupal.org/docs/8/modules/search-api-solr/search-api-solr-howtos/geospatial-search
• Using your own schema.xml docker-solr/docker-solr#10 (essentially what we want but how to steps fail)
• How to use this repo with Drupal 7 docker-solr/docker-solr#11
• How to create_core in Dockerfile or with docker-compose? docker-solr/docker-solr#14
• https://github.com/docker-solr/docker-solr/blob/master/README.md
• https://github.com/docker-solr/docker-solr/blob/master/Docker-FAQ.md
• https://github.com/docker-solr/docker-solr-examples/tree/master/custom-configset
• https://github.com/docker-solr/docker-solr-examples/tree/master/schema-api-startup
• https://www.drupal.org/project/search_api_solr/issues/3106876
• https://www.drupal.org/project/search_api_solr/issues/3015993

• https://stackoverflow.com/questions/39865829/what-is-the-difference-between-managed-schema-and-schema-xml

From the Solr 6.6 reference documentation:

managed-schema is the name for the schema file Solr uses by default to support making Schema changes at runtime via the Schema API, or Schemaless Mode features. You may explicitly configure the managed schema features to use an alternative filename if you choose, but the contents of the files are still updated automatically by Solr.

schema.xml is the traditional name for a schema file which can be edited manually by users who use the ClassicIndexSchemaFactory.

When I review here https://www.drupal.org/project/search_api_solr, I'm not sure what branch we're using per se but seems compatible.

I'm starting to think I don't quite understand the orders of operations for core creation and/or how to properly bind mount accordingly OR we need to investigating creating our own Solr image to avoid this issue with permissions and inability to create cores from bindmounted configsets.

@noahwsmith Including you here for our internal review tomorrow. We can discuss then on next steps.

[g7morris]

Repeated from #75

Unfortunately I'm a bit stuck with this error on the Drupal Status page:

After pulling this image settings apart, diffing and reviewing the following Solr settings (schema.xml, solrconfig.xml, stopwords etc) I can report the following:

I can get this image / container to boot, create a new islandora core from a bind-mounted islandora_solr_conf configset. However due to permission errors, I've had to incorporate a new command in the Makefile solr_init to essentially chmod the config/solr/islandora_solr_conf directory with the uid / guid (8983:8983) for the container to launch.

This also required me to git config fileMode false to stop all warnings.

Additional facts.

• Running previous wodby image solr configsets of search_api_solr_8.x-3.21 and a schema name of drupal-8.3.4-solr-8.x make the resulting container crash, Unclear why?

• Running Solr 7.1 with a configset of search_api_solr_8.x-3.2 and a schema / solrconfig name of drupal-8.3.8-solr-7.x` can get the image working but the same error appears in the Drupal Status page.

• Diffing codebase/web/modules/contrib/search_api_solr/solr-conf-templates/7.x to the results from running islandora-playbook and the current islandora_solr_conf results in a lot of, if not all files being the same.

• Oddly, when I attempt to apply the Drupal config.zip as normally directed, this config seems to introduce Solr config settings not used in the islandora-playbook or in the previous wodby image.
  In the solrcore.properties file, there is this line:
  solr.replication.confFiles=schema.xml,schema_extra_types.xml,schema_extra_fields.xml,elevate.xml,stopwords_en.txt,synonyms_en.txt,protwords_en.txt,accents_en.txt,stopwords_phonetic_und.txt,protwords_phonetic_und.txt,stopwords_phonetic_en.txt,protwords_phonetic_en.txt,stopwords_und.txt,synonyms_und.txt,protwords_und.txt,accents_und.txt solr.mlt.timeAllowed=2000

  • Specifically there are about 2-3 txt files with the word _phonetic that are only introduced and when attempted to be used i.e. the solrcore.properties file links to them, the container crashes. It is like this config introduces something Solr 8.4.1 chokes on.

Hoping someone on the team with more Solr insight can review and test here? I'd prefer not to have the permissions changes on the folder too, that seems sub-optimal but I'm out of ideas on how to get this to run.

@dannylamb @nikathone ?

[g7morris]

So having thought about this effort further, this doesn't feel like an improvement over the wodby image per se which if I recall wasn't having these issues and was green all the way. I'm going to put this down, make an agenda item to discuss for Thursday's call. Let's hold off on accepting this PR and testing it as there are real blockers that need fixing. If someone wants to pick it up later on and review / test, replace with an image we build with confd, or replace with bitnami for instance that might be fine. Leaving the PR as is.

[g7morris]

Ugh so it appears as a result of the merge, the previously working wodby image is now not working and has the same issues that the previously PR from the schema not being right (Drupal Report Status now indicating You are using an incompatible Solr schema. and the Solr path needing to be /. This feels like something changed in the Drupal settings / code level. Putting back on the TO DO list for us to test / fix.

[g7morris]

Filing a new ticket so these efforts don't cross #76

[g7morris]

@nikathone I think the question is if this can be fixed, should we move away from using wodby as the Solr image? (I've put you on this for both images to see what it reveals)