Docker?

[roysbike]

Please add to docker

[mfreed]

Can you share more about what you are asking? Most of the time we see people run such tools (like pg_dump_) externally from the actual database service. So are you looking for us to add to our existing TimecaleDB docker image (why?), or are you looking for a thin Linux docker container that only includes this tool?

[J-F-Far]

It seems like the original submitter @roysbike is asking for the timescaledb-backup tools (ts_backup/ts_restore) to be added to the timescale Docker containers in the same manner that timescaledb-tune and timescaledb-parallel-copy are in https://github.com/timescale/timescaledb-docker

I was about to request this functionality, then I saw this issue.

[tim1670]

Hope to add this tool to docker.
https://hub.docker.com/r/timescale/timescaledb

I need to periodically back up the data in the timescaledb running in docker at 3 o'clock every day, but my docker host does not have pg_dump, pg_dumpall, and pg_restore installed.
I also tried timescaledb-wale, but I feel that wale configuration is very complicated, and wale has no documentation for docker-compose.
In addition, why can’t I find this tool in the official documentation? Or does it mean that the timescaledb-backup tool is still incomplete and unusable?

[mccarthysean]

Why not just create a standalone Docker image for the time being? That way people who use Docker can at least have it installed properly.

I've created one here on GitHub and here on Docker Hub, which helps people use ts-dump to save backups to AWS S3 buckets, but it would be nice to have an official Docker image as well.

Here's my "Dockerfile" for creating a container/image that has ts-dump and ts-restore installed:

ARG VERSION

# pg_dump and pg_restore come from this base image,
# based on the TimescaleDB version one wants
FROM postgres:${VERSION}-alpine

# Copy the Golang binaries from this official image,
# rather than installing manually
COPY --from=golang:rc-alpine /usr/local/go/ /usr/local/go/

# Configure Go
ENV GOROOT /usr/local/go
ENV GOPATH /go
ENV PATH /usr/local/go/bin:$PATH

RUN mkdir -p ${GOPATH}/src ${GOPATH}/bin

# Download and build the ts-dump and ts-restore Golang packages
RUN go get -u github.com/timescale/timescaledb-backup/ || true && \
    # Build ts-dump first
    cd /go/pkg/mod/github.com/timescale/timescaledb-backup@v0.0.0-20210311165201-c4343c888b98/cmd/ts-dump && \
    go mod tidy && \
    go build -o /usr/local/go/bin/ts-dump && \
    # Build ts-restore second
    cd ../ts-restore && \
    go mod tidy && \
    go build -o /usr/local/go/bin/ts-restore

[mccarthysean]

FYI, I've updated my Dockerfile so that instead of trying to go get... then go build... ts-dump, I'm now downloading the Linux x86-64 binaries and renaming them, and this seems to have updated the ts-dump version (although I have no way of verifying that). I've since tried to create backups (including my TimescaleDB materialized views and continuous aggregates) three times, and it's worked all three times! So this is the installation routine I'll run now.

ARG VERSION
# pg_dump and pg_restore come from this base image,
# based on the TimescaleDB version one wants
FROM postgres:${VERSION}-alpine

# Copy the Golang binaries from this official image,
# rather than installing manually
COPY --from=golang:rc-alpine /usr/local/go/ /usr/local/go/

# Configure Go
ENV GOROOT /usr/local/go
ENV GOPATH /go
ENV PATH /usr/local/go/bin:$PATH

RUN mkdir -p ${GOPATH}/src ${GOPATH}/bin

# Download the Linux binaries manually
RUN cd /usr/local/go/bin && \
    wget https://github.com/timescale/timescaledb-backup/releases/download/0.1.1/ts-dump_0.1.1_Linux_x86_64 && \
    wget https://github.com/timescale/timescaledb-backup/releases/download/0.1.1/ts-restore_0.1.1_Linux_x86_64 && \
    # Check the checksums for the downloaded binaries
    wget https://github.com/timescale/timescaledb-backup/releases/download/0.1.1/checksums.txt && \
    cat checksums.txt && \
    sha256sum ts-dump_0.1.1_Linux_x86_64 && \
    sha256sum ts-restore_0.1.1_Linux_x86_64 && \
    # Rename the downloaded binaries to be the default binaries with generic names
    mv ts-dump_0.1.1_Linux_x86_64 ts-dump && \
    mv ts-restore_0.1.1_Linux_x86_64 ts-restore && \
    # Make the downloaded binaries executable
    chmod +x ts-dump ts-restore