Skip to content
This repository has been archived by the owner on Nov 3, 2024. It is now read-only.

Add support for Microsoft SQL Server #168

Merged
merged 30 commits into from
May 11, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
bf46cc9
Update README.md with link to SQL Server service
drupal-daffie Oct 16, 2021
2d88144
The new service for SQL Server
drupal-daffie Oct 16, 2021
0090956
Create docker-compose.sqlsrv.yaml
drupal-daffie Oct 16, 2021
ca54dcd
Create install-drupal-regex-function.sh
drupal-daffie Oct 16, 2021
5af58de
Create README.md
drupal-daffie Oct 16, 2021
953c6c8
Fixed typo in README.md
drupal-daffie Oct 16, 2021
51fe2e5
Delete README.md because it was in the wrong directory
drupal-daffie Oct 16, 2021
7d325d9
Delete docker-compose.sqlsrv.yaml because it was in the wrong directory
drupal-daffie Oct 16, 2021
a583d23
Delete install-drupal-regex-function.sh because it was in the wrong d…
drupal-daffie Oct 16, 2021
9dc60a4
Update README.md wrong link for installing the ODBC driver
drupal-daffie Oct 16, 2021
975b485
Updated the default setting for the SQL Server collation.
drupal-daffie Oct 19, 2021
822cce1
Updated the comment for why the settings are changed.
drupal-daffie Oct 19, 2021
b5bd7fd
Updated README.md for problems with uploadgrogress.
drupal-daffie Oct 25, 2021
ef44944
Updated README.md for problems with multiarch.
drupal-daffie Oct 25, 2021
1bc6f71
Update docker-compose-services/sqlsrv/README.md
drupal-daffie Oct 25, 2021
9cb6979
Update docker-compose-services/sqlsrv/README.md
drupal-daffie Oct 25, 2021
de6f708
Update docker-compose-services/sqlsrv/install-drupal-regex-function.sh
drupal-daffie Oct 25, 2021
00f4d9c
Update docker-compose-services/sqlsrv/README.md
drupal-daffie Oct 25, 2021
b380cf5
Update docker-compose-services/sqlsrv/README.md
drupal-daffie Oct 25, 2021
9878984
Updating install-drupal-regex-function.sh for removing the sqlsystem …
drupal-daffie Oct 26, 2021
9cc6792
Updating docker-compose.sqlsrv.yml for removing the volumes: sqlsyste…
drupal-daffie Oct 26, 2021
65f62e2
Updated the docker-compose.sqlsrv.yml file for the wrong volume.
drupal-daffie Oct 27, 2021
33cf0c1
Updated docker-compose.sqlsrv.yaml with comment about Apple M1.
drupal-daffie Jan 16, 2022
1f0f77a
Added the web-build Dockerfile
drupal-daffie Feb 26, 2022
d9877e6
Removed a number of post start command from README.md
drupal-daffie Feb 26, 2022
06cb200
Added the copying of the Dockerfile to .ddev/web-build
drupal-daffie Feb 27, 2022
4d12974
Commented out the last line of install-drupal-regex-function.sh
drupal-daffie Feb 27, 2022
824508f
Remove check for working sqlsrv and pdo_sqlsrv PHP extensions
drupal-daffie Feb 27, 2022
fb161d8
Added 2 links for MS driver for PHP
drupal-daffie Feb 28, 2022
d7e87eb
Updated Dockerfile for setting the PHP version.
drupal-daffie May 10, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ General information on how to do additional services and some additional example
* [Solr 7 Integration (Drupal-focused)](docker-compose-services/solr-7)
* [Solr Cloud (Drupal-focused)](docker-compose-services/solr-cloud)
* [Solr Integration (TYPO3-focused)](docker-compose-services/typo3-solr)
* [SQL Server (Microsoft)](docker-compose-services/sqlsrv)
* [Varnish](docker-compose-services/varnish)

## .ddev/web-build/Dockerfile examples to customize web container
Expand Down
23 changes: 23 additions & 0 deletions docker-compose-services/sqlsrv/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
ARG BASE_IMAGE
FROM $BASE_IMAGE

ENV PATH="${PATH}:/opt/mssql-tools/bin"
RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y -o Dpkg::Options::="--force-confold" --no-install-recommends --no-install-suggests apt-utils curl gnupg2 ca-certificates

RUN curl -sSL -O https://packages.microsoft.com/keys/microsoft.asc
RUN apt-key add <microsoft.asc
RUN curl -sSL -o /etc/apt/sources.list.d/mssql-release.list https://packages.microsoft.com/config/debian/11/prod.list

RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y -o Dpkg::Options::="--force-confold" --no-install-recommends --no-install-suggests build-essential dialog php-pear php-dev unixodbc-dev locales

RUN ACCEPT_EULA=Y DEBIAN_FRONTEND=noninteractive apt-get install -y msodbcsql18 mssql-tools

# Change the PHP version to what you want. It is currently set to version 8.0.
RUN pecl channel-update pecl.php.net
RUN pecl -d php_suffix=8.0 install sqlsrv
RUN pecl -d php_suffix=8.0 install pdo_sqlsrv

RUN echo 'extension=sqlsrv.so' >> "/etc/php/8.0/mods-available/sqlsrv.ini"
RUN echo 'extension=pdo_sqlsrv.so' >> "/etc/php/8.0/mods-available/pdo_sqlsrv.ini"

RUN phpenmod sqlsrv pdo_sqlsrv
106 changes: 106 additions & 0 deletions docker-compose-services/sqlsrv/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
# SQL Server (Microsoft)

Using SQL server from Microsoft.

## Installation

1. Copy `docker-compose.sqlsrv.yaml` to your project's `.ddev` directory.
2. Copy `Dockerfile` to your project's `.devv/web-build` directory.
3. Copy the full post start hook commands to your project's `config.yaml` or add them to `config.sqlsrv.yaml`.
4. *(optional)* For Drupal 9+ projects: copy `install-drupal-regex-function.sh` to your project's `.ddev` directory.

## Connection

Connect to `sqlsrv` host/db server from within the web container with:

```
Host: `ddev-projectname-sqlsrv`
User: sa
Password: password!
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the mysql and postgres db setups use db as username and password, could we use that here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We cannot use the password "db" as it would fail the restrictions set by SQL Server. See: https://docs.microsoft.com/en-us/sql/relational-databases/security/password-policy?view=sql-server-ver15

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, but could use the username 'db' true?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The default user on SQL Server is the user "sa". I think it stands for "system administrator", a.k.a. the root user on linux. An other user needs to be created with the SQL string "CREATE USER 'db' WITH PASSWORD 'db'". Is that what you would like to do? In this way we can also create the database "db".

Database: master
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it important for it to be named "master"? Again, the default mysql and postgres setups use "db" for the name of the primary database.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does not need to be master, it is only the default database for SQL Server. The "db" database needs to be created.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah if 'master' is default database then it makes sense to use it.

```

Connect to `sqlsrv` host/db server from project directory:

```bash
ddev exec -s sqlsrv "/opt/mssql-tools/bin/sqlcmd -P password! -S localhost -U sa -d master"
```

For external access, use the port used in your `docker-compose.sqlsrv.yaml` and `127.0.0.1` as host.

When using multiple databases in your project with SQL Server support, remember to update your `docker-compose.sqlsrv.yaml` to use different ports:

```yaml
ports:
- <EXTERNAL_PORT>:1433
```

## Installing the PHP extensions

The PHP extensions for SQL Server CANNOT be installed by adding them to the `webimage_extra_packages` setting. The problem is that they are not available as a Debian or any other distribution package. The 2 extensions (`sqlsrv` and `pdo_sqlsrv`) need to be compiled and this needs to be done after PHP is installed on the webimage. The following commands need to be copied to the end of the main `config.yaml` file:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please do this with a .ddev/web-build/Dockerfile instead - it's far more efficient (only has to be done once, instead of on every ddev start. It's also easier to understand and maintain. https://ddev.readthedocs.io/en/stable/users/extend/customizing-images/#adding-extra-dockerfiles-for-webimage-and-dbimage

Copy link
Contributor Author

@drupal-daffie drupal-daffie Oct 25, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have tried to do that, only when I did that there was no PHP installed yet, therefor it failed. Maybe I did something wrong or there is a way to do it after PHP is installed. For me, I was fantastic that I got it to work in the first place. It took me 2 weeks to get it to work. I just wanted to fix some bugs in the Drupal module sqlsrv.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will be happy to help with this of course. And appreciate you sharing the work that you did!

Copy link
Contributor Author

@drupal-daffie drupal-daffie Oct 26, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The main thing for me was to make it a lot easier for the people after me that would like to use DDEV with SQL Server. For some reason the PHP extension for SQL Server are not available as packages. Really annoying!
I appreciate your help with this very much!


```yaml
hooks:
post-start:
- exec: echo export PATH="$PATH:/opt/mssql-tools/bin" >> ~/.bashrc
- exec: source ~/.bashrc
```

The minimum required PHP version the these extensions is PHP 7.3. For more information about these extension, see: [MS SQL driver for PHP](https://github.com/microsoft/msphpsql).


## Drupal Notice

Drupal CMS needs the a database function installed that is mimicking the Regex function as Drupal requires. As a one-time setup for Drupal, install the database function by running the following command from your project's directory:

```bash
./install-drupal-regex-function.sh -u <username> -p <password> -d <database>
```

This script also changes the setting for the following database variables:
* `show advanced options` will be set to 1
* `clr strict security` will be set to 0
* `clr enable` will be set to 1

Drupal also the module `sqlsrv` to be installed as it is providing the database driver for SQL Server. The module can be installed with composer with the following command:

```bash
$ php composer require drupal/sqlsrv
```

## Disabling MySQL & MariaSQL

* If your project only uses a SQL Server database, you can disable the MySql & MariaDb services.
* Run the following command from your project root.

```bash
ddev config --omit-containers db
```

* Alternatively, you can update your project's `.ddev/config.yaml` directly by updating the following line:

```yaml
omit_containers: [db]
```

* See [.ddev/config.yaml Options](https://ddev.readthedocs.io/en/stable/users/extend/config_yaml/) for additional notes.

## TODO

Future enhancements (PR's welcome here) include:

* Provide custom commands.

## Links with useful information

* [SQL Server docker hub](https://hub.docker.com/_/microsoft-mssql-server)
* [Installing the ODBC driver for SQL Server](https://docs.microsoft.com/en-us/sql/connect/odbc/linux-mac/installing-the-microsoft-odbc-driver-for-sql-server)
* [Installing the ODBC driver for SQL Server Tutorial](https://docs.microsoft.com/en-us/sql/connect/php/installation-tutorial-linux-mac)
* [Installation tutorial for MS drivers for PHP](https://docs.microsoft.com/en-us/sql/connect/php/installation-tutorial-linux-mac)
* [The SQLCMD utility](https://docs.microsoft.com/en-us/sql/tools/sqlcmd-utility)
* [The SQL Server on Linux](https://docs.microsoft.com/en-us/sql/linux/sql-server-linux-overview)
* [The password policy](https://docs.microsoft.com/en-us/sql/relational-databases/security/password-policy)
* [The SQL Server environment variables](https://docs.microsoft.com/en-us/sql/linux/sql-server-linux-configure-environment-variables)
* [Beakerboy's Drupal Regex database function](https://github.com/Beakerboy/drupal-sqlsrv-regex)
* [Drupal's module for the SQL Server](https://www.drupal.org/project/sqlsrv)
* [Github MS drivers for PHP](https://github.com/microsoft/msphpsql)
52 changes: 52 additions & 0 deletions docker-compose-services/sqlsrv/docker-compose.sqlsrv.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
version: '3.6'

services:
sqlsrv:
container_name: ddev-${DDEV_SITENAME}-sqlsrv
hostname: ddev-${DDEV_SITENAME}-sqlsrv

# For possible options: https://hub.docker.com/_/microsoft-mssql-server.
# This does not yet work with Apple M1. See: https://github.com/microsoft/mssql-docker/issues/734
image: mcr.microsoft.com/mssql/server:2019-CU12-ubuntu-20.04

user: root
volumes:
- sqlsystem:/var/opt/mssql/
restart: "no"
ports:
- "1433:1433"
labels:
com.ddev.site-name: ${DDEV_SITENAME}
com.ddev.approot: $DDEV_APPROOT
environment:

# With the following setting you agree with Microsoft's end user license
# agreement.
- 'ACCEPT_EULA=Y'

# The SQL Server enforces password complexity. Trying to set a password
# that is not sufficiently complex will result in the password not be set.
# As a result you cannot login with the "SA" user account.
# The password for the SA account needs to follow the following policy:
# https://docs.microsoft.com/en-us/sql/relational-databases/security/password-policy?view=sql-server-ver15
- 'SA_PASSWORD=Password12!'

# The following setting is for selecting the SQL Server edition or product
# key. For possible options:
# https://docs.microsoft.com/en-us/sql/linux/sql-server-linux-configure-environment-variables?view=sql-server-ver15
- 'MSSQL_PID=Evaluation'

# Sets the default collation for SQL Server. This setting is optional. The
# current setting is best for Drupal projects. For more information, see:
# https://docs.microsoft.com/en-us/sql/relational-databases/collations/collation-and-unicode-support?view=sql-server-ver15
- 'MSSQL_COLLATION=LATIN1_GENERAL_100_CI_AS_SC_UTF8'

# For more possible environment variables that can be set, see:
# https://docs.microsoft.com/en-us/sql/relational-databases/security/password-policy?view=sql-server-ver15

web:
links:
- sqlsrv:sqlsrv

volumes:
sqlsystem:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ddev delete can only delete the one volume with the name of the service. Is there a way to get all these on a single volume? It would be better that way...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am more of a programmer than somebody who does production. Therefor if you think that is best then please do so.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't about programming, it's about where the mounts are. Since I know nothing about sqlsrv, I don't know why you are using volumes for all these things. Is is possible that only the database is actually required to be persistent, and that's sqldata, and it could be called sqlsrv? You only need the volumes for stuff that's persistent. On mysql and postgres, that's just the database itself.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated the SQL Server docker file. Removed the other volumes.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool, thanks!

55 changes: 55 additions & 0 deletions docker-compose-services/sqlsrv/install-drupal-regex-function.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
#!/bin/bash
# Installs the Regex database function for compatibility with Drupal version 9 or higher.
# This script only needs to be run once for a database.

usage() { echo "./install-drupal-regex-function.sh -u <username> -p <password> -d <database>" 1>&2; exit 1; }

# The password for the database defaults to ""
password=""

# The username for the database defaults to "SA"
username="SA"

# The database name defaults to "master"
database="master"

while getopts ":u:p:d:h:" arg; do
case $arg in
u)
username=${OPTARG}
;;
p)
password=${OPTARG}
;;
d)
database=${OPTARG}
;;
h)
usage
;;
*)
usage
;;
esac
done

if [ -z $password ]
then
echo "The parameter -p for the password is not set."
exit 1
fi

if ! ddev exec -s sqlsrv test -e "/RegEx.dll" &>/dev/null;
then
# Download the Regex.dll file and copy it to the right location.
ddev exec -s sqlsrv wget https://github.com/Beakerboy/drupal-sqlsrv-regex/releases/download/1.0/RegEx.dll
fi

# The following are changed to allow the installation and execution of the user provided database function.
ddev exec -s sqlsrv "/opt/mssql-tools/bin/sqlcmd -P $password -S localhost -U $username -d $database -Q 'EXEC sp_configure \"show advanced options\", 1; RECONFIGURE; EXEC sp_configure \"clr strict security\", 0; RECONFIGURE; EXEC sp_configure \"clr enable\", 1; RECONFIGURE;'"

# Create the assambly and the function for the Regex helper.
ddev exec -s sqlsrv "/opt/mssql-tools/bin/sqlcmd -P $password -S localhost -U $username -d $database -Q 'CREATE ASSEMBLY Regex from \"/RegEx.dll\" WITH PERMISSION_SET = SAFE'"
ddev exec -s sqlsrv "/opt/mssql-tools/bin/sqlcmd -P $password -S localhost -U $username -d $database -Q 'CREATE FUNCTION dbo.REGEXP(@pattern NVARCHAR(100), @matchString NVARCHAR(100)) RETURNS bit EXTERNAL NAME Regex.RegExCompiled.RegExCompiledMatch'"
drupal-daffie marked this conversation as resolved.
Show resolved Hide resolved

# **Contributed by [@drupal-daffie](https://github.com/drupal-daffie)**