Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add docker instructions. #1635

Merged
merged 9 commits into from
Oct 19, 2020
Merged

Add docker instructions. #1635

merged 9 commits into from
Oct 19, 2020

Conversation

rosiel
Copy link
Member

@rosiel rosiel commented Oct 3, 2020

Issue: #1614

Purpose / why

To document how to use ISLE for 8!

What changes were made?

I added an 'installation' page called "Docker (ISLE)" beside "Ansible Playbook" (which was formerly "Automatic Provisioning"). It covers what @dannylamb walked me through during Thursday's user call of spinning up a simple Isle ... box?

Verification

Best verification is to get someone unfamiliar with using Docker to follow these instructions to successfully create a working Islandora.

Does this documentation work?
Does this documentation include lies that are apparent to someone who knows more about Docker, Docker compose, and ISLE?

I'd like to propose that the following terminology questions get discussed not as part of this pull request, because the PR format is a terrible place to do that, but please bring your thoughts to an upcoming tech call. To not get caught up in that, can this PR be focused on 'good enough to add and make things better'?

  • What is the thing that Docker creates? It's not a "vm" but is it a "box"? a "machine"?
  • What nouns does "Isle" go with? Is this an "ISLE site"? do you upgrade "your ISLE" when you do git pull?
  • What is the OS agnostic word for a Terminal window? I think I tried to use "shell" and/or "command line".

Interested Parties

@g7morris @dannylamb
and folks who were on that call: @manez @ajstanley @highermath

Checklist

Pull-request reviewer should ensure the following

Person merging should ensure the following

  • Does mkdocs still build successfully? (This is indicated by TravisCI passing. To test locally, and see warnings, see How To Build Documentation.)
  • If pages are renamed or removed, have all internal links to those pages been fixed?
  • If pages are added, have they been linked to or placed in the menu?
  • Did the PR receive at least one approval from a committer, and all issues raised have been addressed?

docs/installation/docker.md Outdated Show resolved Hide resolved
docs/installation/docker.md Outdated Show resolved Hide resolved
@rosiel
Copy link
Member Author

rosiel commented Oct 5, 2020

[EDIT: The make dev error has been fixed with isle-dc pulls https://github.com/Islandora-Devops/isle-dc/pull/114 and a related pull, https://github.com/Islandora-Devops/isle-dc/pull/116. The components failing was solved with increasing Docker's memory.]

Also, when doing make dev, the results ended with:

Creating network "isle-dc_default" with the default driver
Creating network "gateway" with driver "bridge"
Creating volume "isle-dc_activemq-data" with default driver
Creating volume "isle-dc_blazegraph-data" with default driver
Creating volume "isle-dc_cantaloupe-data" with default driver
Creating volume "isle-dc_drupal-sites-data" with default driver
Creating volume "isle-dc_fcrepo-data" with default driver
Creating volume "isle-dc_mariadb-data" with default driver
Creating volume "isle-dc_mariadb-files" with default driver
Creating volume "isle-dc_matomo-config-data" with default driver
Creating volume "isle-dc_solr-data" with default driver
Creating isle-dc_activemq_1   ... done
Creating isle-dc_alpaca_1     ... done
Creating isle-dc_watchtower_1 ... done
Creating traefik              ... done
Creating isle-dc_houdini_1    ... done
Creating isle-dc_gemini_1     ... done
Creating isle-dc_mariadb_1    ... done
Creating isle-dc_cantaloupe_1 ... done
Creating isle-dc_blazegraph_1 ... done
Creating isle-dc_homarus_1    ... done
Creating isle-dc_recast_1     ... done
Creating isle-dc_solr_1       ... done
Creating isle-dc_milliner_1   ... done
Creating isle-dc_hypercube_1  ... done
Creating isle-dc_fits_1       ... done
Creating isle-dc_crayfits_1   ... done
Creating isle-dc_matomo_1     ... done
Creating isle-dc_fcrepo_1     ... done
Creating isle-dc_drupal_1     ... done
Waiting for 80 on localhost to open.
make[1]: *** [create-codebase-from-demo] Error 143
make: *** [dev] Error 2

Why is error? I can access https://islandora-isle-dc.traefik.me/, but I have not yet tried processes that should result in derivatives. And in the little "whale in the toolbar" docker desktop dashboard, it shows isle-dc as yellow, and two containers - isle-dc_alpaca_1 and isle-dc_fits_1 to have 'EXITED (0)'. I have restarted alpaca at least once, and it "turned green for a while" but when i next went to look at my computer, it had exited again.

Screen Shot 2020-10-05 at 9 15 59 AM

@seth-shaw-unlv
Copy link
Contributor

Wish I had been there! I was just thinking about trying to figure this out this morning.

I see that the PR is still in Draft state; want me to be a Docker neophyte tester once it is ready for review, @rosiel?

@rosiel
Copy link
Member Author

rosiel commented Oct 5, 2020

Seth - it'd be great if you could give it a try! I'll start a slack discussion for faster/more open and iterative chatting, and use the results to improve this PR.

@rosiel rosiel marked this pull request as ready for review October 5, 2020 16:58

* Docker (version 19.x+)
* (Mac OS) XCode Command-Line Tools
* Perl (if using `make dev`)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems from the README that there are many more requirements. Shoot, I already had them so didn't run into this. Are they all required in all cases? Or can one do the simple make without all of these installed on the host?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes - these should only be required for 'make dev' and i'll update the docs.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just tried to run this on my Windows 10 machine and I don't have make installed. I tried "npm install make -g" and it runs but immediately gives an error so I'm not sure I got the correct thing. Output below

$ make dev make i info Invoking dev target '$' is not recognized as an internal or external command, operable program or batch file. make × ERR $(MAKE) download-default-certs $(MAKE) create-codebase-from-demo if grep -q DRUPAL_DEFAULT_CONFIGDIR docker-compose.env.yml; then \ perl -i -pe's/DRUPAL_DEFAULT_CONFIGDIR:.*/DRUPAL_DEFAULT_CONFIGDIR: \/var\/www\/drupal\/config\/sync/g' docker-compose.env.yml; \ else \ perl -i -pe's/DRUPAL_DEFAULT_SALT/DRUPAL_DEFAULT_CONFIGDIR: \/var\/www\/drupal\/config\/sync\n\ \ \ \ \ \ DRUPAL_DEFAULT_SALT/g' docker-compose.env.yml; \ fi if grep -q DRUPAL_DEFAULT_INSTALL_EXISTING_CONFIG docker-compose.env.yml; then \ perl -i -pe's/DRUPAL_DEFAULT_INSTALL_EXISTING_CONFIG:.*/DRUPAL_DEFAULT_INSTALL_EXISTING_CONFIG: "true"/g' docker-compose.env.yml; \ else \ perl -i -pe's/DRUPAL_DEFAULT_SALT/DRUPAL_DEFAULT_INSTALL_EXISTING_CONFIG: "true"\n\ \ \ \ \ \ DRUPAL_DEFAULT_SALT/g' docker-compose.env.yml; \ fi if grep -q DRUPAL_DEFAULT_PROFILE docker-compose.env.yml; then \ perl -i -pe's/DRUPAL_DEFAULT_PROFILE:.*/DRUPAL_DEFAULT_PROFILE: minimal/g' docker-compose.env.yml; \ else \ perl -i -pe's/DRUPAL_DEFAULT_SALT/DRUPAL_DEFAULT_PROFILE: minimal\n\ \ \ \ \ \ DRUPAL_DEFAULT_SALT/g' docker-compose.env.yml; \ fi $(MAKE) -B docker-compose.yml ENVIRONMENT=local docker-compose up -d $(MAKE) remove_standard_profile_references_from_config $(MAKE) install ENVIRONMENT=local $(MAKE) hydrate ENVIRONMENT=local make × ERR Recipe exited with code %d (node:20452) UnhandledPromiseRejectionWarning: Error: Recipe exited with code %d at ChildProcess.<anonymous> (C:\Users\catspaw\AppData\Roaming\npm\node_modules\make\src\cli.js:143:28) at ChildProcess.emit (events.js:198:13) at maybeClose (internal/child_process.js:982:16) at Process.ChildProcess._handle.onexit (internal/child_process.js:259:5) (node:20452) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1) (node:20452) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Running "make" by itself just results in a help page because it says it needs a target.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

all right! Thanks @dflitner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where are you running make @dflitner ? A Windows command terminal, power shell, git bash shell, wsl or wsl2 shell? Unfortunately, there are environments one can be in on Windows, and it is not clear which one you are using.

Copy link
Contributor

@eldonquijote eldonquijote Oct 15, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As I mentioned on Slack, I used the Windows Subsystem for Linux (WSL2) with Docker Desktop for Windows. On a freshly installed Debian, I had to install make and curl to spin up ISLE:
sudo apt-get install make
sudo apt-get install curl

There was also a small hiccup around user permissions and a config file for docker. I'll try to verify if that is an issue under Ubuntu-on-WSL too, so we can document it.

For this WSL-based setup I also had to limit the memory that the Linux distro has available, otherwise Docker can dramatically slow down the entire system: https://medium.com/@lewwybogus/how-to-stop-wsl2-from-hogging-all-your-ram-with-docker-d7846b9c5b37

docs/installation/docker.md Outdated Show resolved Hide resolved
@seth-shaw-unlv
Copy link
Contributor

@rosiel, it looks like 4 GB of memory is enough to get isle-dc enough space to test things out. So OS X users should up Docker Desktop's default 2 GB memory to 4 GB before starting things up.

@rosiel
Copy link
Member Author

rosiel commented Oct 8, 2020

I'm relieved it can run with less, but Nigel said 8 on slack... have you run it through any paces yet?

I alluded to composer later in the pr and I know that it's a memory hog. Maybe that's why Nigel suggested more.

I noticed that when I changed the memory and rebooted docker, my containers were down, but the state seemed to have persisted. Would it make sense to give Docker (Desktop) more memory when you are updating things with composer, and then set it back down to a lower amount for normal use?

@seth-shaw-unlv
Copy link
Contributor

@nigelgbanks had done a resource count that totalled up to ~7GB. It looks like fits, fcrepo, cantaloupe, and blazegraph were the heaviest users with alpaca and activemq following up behind.

My testing wasn't resource intensive... just manually adding a few items and poking the different features. So I would probably say 4 GB minimum for demo, 8 GB for dev, and then 16 GB for prod.

@rosiel
Copy link
Member Author

rosiel commented Oct 15, 2020

Thank you for your help and comments @seth-shaw-unlv @dflitner @noahwsmith @manez . Is anyone available to "do the review"?

@manez
Copy link
Member

manez commented Oct 19, 2020

Following through the instructions one last time for review 👍

Copy link
Member

@manez manez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Easy to follow and got me to a working Islandora demo site!

@manez manez merged commit 7e22da8 into Islandora:main Oct 19, 2020
@rosiel rosiel deleted the docker-instructions branch October 19, 2020 19:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants