Dockerfile to use with docker-compose.

[synchronizing]

Wondering if there is any available Dockerfile that containerizes the entire project without needing to run commands in the host machine, nor multiple docker run to finalize setup. Currently looking to run sftpgo with an external Postgres service with docker-compose, but having a hard time setting-up the project without needing to involve the host machine by modifying the provided Dockerfiles in the repo.

Any help would be appreciated.

[synchronizing]

Nevermind, I got it.

.
├── Dockerfile
└── config
   └── sftpgo.json

Dockerfile

# ------------------------------------------------------
# Storage solution.
# https://github.com/drakkan/sftpgo
#
#
# Notes:
#
#   This Dockerfile can be expanded further to add support
#   for SFTP backups and more. Reading the documentation
#   in the link above should help. This Dockerfile does not
#   include these services.
#
#   This Dockerfile is a multi-stage build. The first stage
#   (buieldenv) installs SFTPGO and sets it up to the latest 
#   stable release v1.0.0, and the second stage (runenv) 
#   setups the run environment.
# 
#  A custom settings .json file is required for this Dockerfile  
#  to work. Initialize it in the host machine as config/sftpgo.json
#  in the same folder as this Dockerfile. Read docs for how to 
#  setup this configuration file. 
#
# ------------------------------------------------------

# ------------------------------
#   Stage 1: Build Environment
# ------------------------------
FROM golang:latest as buildenv

# Sets the version to use, and the features to disable.
ARG TAG=v1.0.0
ARG FEATURES=nogcs,nos3,nobolt,nomysql,nopgsql,noportable,nometrics

# Installs SFTPGO.
RUN go get -d github.com/drakkan/sftpgo
WORKDIR /go/src/github.com/drakkan/sftpgo

# Installs and setsup the TAG and FEATURES above.
RUN git checkout $(if [ "${TAG}" = LATEST ]; then echo `git rev-list --tags --max-count=1`; elif [ -n "${TAG}" ]; then echo "${TAG}"; else echo HEAD; fi)
RUN go build $(if [ -n "${FEATURES}" ]; then echo "-tags ${FEATURES}"; fi) -ldflags "-s -w -X github.com/drakkan/sftpgo/version.commit=`git describe --always --dirty` -X github.com/drakkan/sftpgo/version.date=`date -u +%FT%TZ`" -o sftpgo

# ----------------------------
#   Stage 2: Run Environment
# ----------------------------
FROM debian:latest as runenv

# Directories variables.
ARG BASE_DIR=/app
ARG DATA_REL_DIR=data
ARG CONFIG_REL_DIR=config
ARG BACKUP_REL_DIR=backups
ARG WEB_REL_PATH=web

# Permission variables.
ARG USERNAME=sftpgo
ARG GROUPNAME=sftpgo
ARG UID=515
ARG GID=515

# Setups home directory.
# Setups data directory; where users files are stored.
# Setups up config directory; where private keys, configurations, etc. are stored.
# Setups up backup directory; where server backups are stored into. 
# Setups web directory; where web templates are found.
ENV HOME_DIR=${BASE_DIR}/${USERNAME}
ENV DATA_DIR=${BASE_DIR}/${DATA_REL_DIR}
ENV CONFIG_DIR=${BASE_DIR}/${CONFIG_REL_DIR}
ENV BACKUPS_DIR=${BASE_DIR}/${BACKUP_REL_DIR}
ENV WEB_DIR=${BASE_DIR}/${WEB_REL_PATH}

# Default configurations used during runtime.
ENV SFTPGO_CONFIG_DIR=${CONFIG_DIR}
ENV SFTPGO_LOG_FILE_PATH=""
ENV SFTPGO_HTTPD__BIND_ADDRESS=""
ENV SFTPGO_HTTPD__TEMPLATES_PATH=${WEB_DIR}/templates
ENV SFTPGO_HTTPD__STATIC_FILES_PATH=${WEB_DIR}/static
ENV SFTPGO_DATA_PROVIDER__USERS_BASE_DIR=${DATA_DIR}
ENV SFTPGO_HTTPD__BACKUPS_PATH=${BACKUPS_DIR}

# Installs depedencies.
RUN apt-get update && apt-get install -y ca-certificates && apt-get clean

# Creates folders and changes their permissions to user defined in ARGS above.
RUN mkdir -p ${DATA_DIR} ${CONFIG_DIR} ${WEB_DIR} ${BACKUPS_DIR}
RUN groupadd --system -g ${GID} ${GROUPNAME}
RUN useradd --system --create-home --no-log-init --home-dir ${HOME_DIR} --comment "SFTPGo user" --shell /bin/false --gid ${GID} --uid ${UID} ${USERNAME}

# Sets working directory to new users' home dir.
WORKDIR ${HOME_DIR}

# Creates config directories, and adds it to path.
RUN mkdir -p bin ${CONFIG_DIR}
ENV PATH ${HOME_DIR}/bin:$PATH

# Copies user configuration file.
COPY config/sftpgo.json ${CONFIG_DIR}

# Copies over binary, settings, and files from buildenv.
COPY --from=buildenv /go/src/github.com/drakkan/sftpgo/sftpgo bin/sftpgo
COPY --from=buildenv /go/src/github.com/drakkan/sftpgo/templates ${WEB_DIR}/templates
COPY --from=buildenv /go/src/github.com/drakkan/sftpgo/static ${WEB_DIR}/static
RUN chown -R ${UID}:${GID} ${DATA_DIR} ${BACKUPS_DIR} ${CONFIG_DIR}

# Switches user to non-root user.
USER ${USERNAME}

# Initializes provider(s).
RUN sftpgo initprovider -c ${CONFIG_DIR}

# Starts SFTPGO server.
ENTRYPOINT sftpgo serve

config/sftpgo.json

{
  "common": {
    "idle_timeout": 15,
    "upload_mode": 0,
    "actions": {
      "execute_on": [],
      "hook": ""
    },
    "setstat_mode": 0,
    "proxy_protocol": 0,
    "proxy_allowed": [],
    "post_connect_hook": ""
  },
  "sftpd": {
    "bind_port": 2022,
    "bind_address": "",
    "max_auth_tries": 0,
    "banner": "",
    "host_keys": [],
    "kex_algorithms": [],
    "ciphers": [],
    "macs": [],
    "trusted_user_ca_keys": [],
    "login_banner_file": "",
    "enabled_ssh_commands": [
      "md5sum",
      "sha1sum",
      "cd",
      "pwd",
      "scp"
    ],
    "keyboard_interactive_auth_hook": ""
  },
  "data_provider": {
    "driver": "sqlite",
    "name": "sftpgo.db",
    "host": "",
    "port": 5432,
    "username": "",
    "password": "",
    "sslmode": 0,
    "connection_string": "",
    "sql_tables_prefix": "",
    "manage_users": 1,
    "track_quota": 2,
    "pool_size": 0,
    "users_base_dir": "",
    "actions": {
      "execute_on": [],
      "hook": ""
    },
    "external_auth_hook": "",
    "external_auth_scope": 0,
    "credentials_path": "credentials",
    "pre_login_hook": "",
    "post_login_hook": "",
    "post_login_scope": 0,
    "check_password_hook": "",
    "check_password_scope": 0
  },
  "httpd": {
    "bind_port": 8080,
    "bind_address": "127.0.0.1",
    "templates_path": "templates",
    "static_files_path": "static",
    "backups_path": "backups",
    "auth_user_file": "",
    "certificate_file": "",
    "certificate_key_file": ""
  },
  "http": {
    "timeout": 20,
    "ca_certificates": [],
    "skip_tls_verify": false
  }
}

[synchronizing]

Will issue new pull request for addition of "simple installation" within the docker/sftpgo/debian/ folder in the coming days.

[drakkan]

This would be great, however I don't want to hardcode build args, thank you!

[synchronizing]

This would be great, however I don't want to hardcode build args, thank you!

I understand the sentiment. What do you constitute as hardcoded build args? Tag, features, directories, permissions, configurations? Would you be open to a "quick start-up" Dockerfile without any extended configurations, but well documented links?

[drakkan]

Hi,

first of all I have to say that I'm not a Docker expert so I could miss something, however I'm open to discuss and to learn.

SFTPGo provides sample dockerfiles that use multi stage build, inside the Dockerfile there are several ARG directives that should be set to customize your build, for example:

ARG TAG
ARG FEATURES

when you build the image you can set the ARG values, for example:

docker build -t="drakkan/sftpgo" --build-arg TAG=v1.0.0 .

the runtime configuration parameters could be set using ENV vars, this seems quite similar, for example to official mysql docker image. The only restriction is that a viper requires a config file in order to use ENV vars, so we cannot have an env vars only configuration but this could be fixed in future.

If these things can be improved I would be happy to merge your changes.

If you think that docs are not clear enough a pull request would be great (english is not my native language I'm aware that docs can be improved). Also providing some quick start-up Dockerfiles would be a nice additions, it could help new users. Thank you

[drakkan]

please take a look at the new README I hope it clarifies the things a bit. However pull requests are welcome, thank you!

[synchronizing]

Trust me when I say this: the documentation is outstanding, and the only reason I was able to solve my issue.

Regarding Docker, the issue in itself wasn't build-args, but rather the necessities to run commands on the host machine before execution (see docker/debian/README.md with mkdir, chown, curl; and docker/alpine/README.md with groupadd.) An example where this is not a net positive is if the user plans on using docker-compose (not the same as docker), Kubernetes, Swarn, etc. to run the project. In such case the user will need to also run some external commands on the host machines/cluster, which makes it painfully difficult to document and keep track of.

The Dockerfile I posted above is a simple solution to quickly executing the project without these external host dependencies. It removes some of the plus (backup, shared volumes, etc.), for quick deployment. To use your mysql example, with Docker one can do:

docker run -e MYSQL_ALLOW_EMPTY_PASSWORD=True mysql

To instantly have the mysql image downloaded from the internet and running without any modifications having to be done to the host system. The same cannot be said about sftpgo in its current status, as simply running either or of the current Dockerfiles presets will cause errors due to lack of host dependencies. Hopefully that clears it up.

I would be happy to modify the current Dockerfile above so all the user would have to do to get sftpgo up and running in a few seconds is do:

curl <dockerfile>
docker build --build-arg VERSION=v1.0.0 -t sftpgo .
docker run -p 8000:8000 -p 2022:2022 sftpgo 

Sometimes all a user wants to do is give the project a shot before using it as well. Adding extra steps and host-modifying commands aren't always pleasing. :)

[drakkan]

Ok, now I understand what you mean and it makes sense.

Please send a pull request with your proposed changes, so we can discuss about them. The review can take a while I don't know docker very well.

Reading the section "Using a custom MySQL configuration file" here seems that the way to ship configuration files in docker is inside the image itself. I'll try to find some time the next weekend to learn more about this. If we find a way to make anything configurable I could also submit a sftpgo image on docker hub. There are already several user contributed images there

[synchronizing]

Sounds good. Will likely need a few days myself -- project deadline for work is coming up, and I'm hard at work on that.

Regarding the user contributed images, the two I had found failed and didn't work. Perhaps a good way to go about this would be to automate the DockerHub push via Travis. I can help out in this regard, but it would be best for you to be the maintainer of both DockerHub and Travis. The automation would make things more efficient, and it would certainly be nice to just be able to do:

docker run -p 8000:8000 -p 2022:2022 sftpgo

[drakkan]

Sounds good. Will likely need a few days myself -- project deadline for work is coming up, and I'm hard at work on that.

Ok, no hurry

Regarding the user contributed images, the two I had found failed and didn't work. Perhaps a good way to go about this would be to automate the DockerHub push via Travis. I can help out in this regard, but it would be best for you to be the maintainer of both DockerHub and Travis. The automation would make things more efficient, and it would certainly be nice to just be able to do:

docker run -p 8000:8000 -p 2022:2022 sftpgo

I recently siwtched from Travis to GitHub Actions, anyway we will discuss about the official docker image and the automatic push once we found a way to build a general usable and full customizable way to make available the image itself, thank you!

[synchronizing]

Understood. Sounds like a plan. Expect a PR from me in the coming days!

[drakkan]

we probably could do something like this. This image has similar requirements, it needs to expose host keys, mount volumes etc.

Thank you!