Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: add support for SIPI v3.0.0-rc.3 (DSP-433) #1663

Merged
merged 50 commits into from
Jun 29, 2020
Merged
Show file tree
Hide file tree
Changes from 49 commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
9da83f7
build: remove graphdb, add fuseki
subotic Jun 9, 2020
198af81
build: add fuseki db init (ongoing)
subotic Jun 9, 2020
97930ff
build: add fuseki db init (ongoing)
subotic Jun 10, 2020
2ca6947
Merge branch 'develop' into wip/DSP-30-dockerize-fuseki
subotic Jun 10, 2020
cc6e883
build: add fuseki db init (ongoing)
subotic Jun 10, 2020
ad60816
build: add fuseki db init (ongoing)
subotic Jun 11, 2020
ccfd9bb
build: add fuseki db init (ongoing)
subotic Jun 12, 2020
017e699
build: add fuseki db init
subotic Jun 12, 2020
0e16a71
Merge branch 'develop' into wip/DSP-30-dockerize-fuseki
subotic Jun 12, 2020
0c18f07
build: remove graphdb specific settings from github-ci
subotic Jun 12, 2020
c03ae5b
build: fix test upgrade
subotic Jun 12, 2020
3208b24
build: fix upgrade test
subotic Jun 12, 2020
f98adf7
build: fix upgrade test
subotic Jun 13, 2020
cdddafd
test: fix deprecation
subotic Jun 14, 2020
8c38dad
build: add testcontainers (ongoing)
subotic Jun 14, 2020
0e03f2e
build: add deleting of database volumes when init-db-xyz targets are run
subotic Jun 15, 2020
0822b20
build: add testcontainers (ongoing)
subotic Jun 15, 2020
2f0b141
build: add testcontainers (ongoing)
subotic Jun 15, 2020
994769e
build: add testcontainers (ongoing)
subotic Jun 16, 2020
326d3dd
build: add testcontainers (ongoing)
subotic Jun 16, 2020
2e7d550
build: add testcontainers (ongoing)
subotic Jun 16, 2020
ecb233a
docs: update readme
subotic Jun 16, 2020
447a818
build: bump sbt
subotic Jun 16, 2020
b557f3a
build: use build-all-scala on Github-CI
subotic Jun 16, 2020
51b7095
build: add parallel test execution config (ongoing)
subotic Jun 16, 2020
e96356e
build: run ci from inside Docker
subotic Jun 17, 2020
9deb568
build: run ci from inside Docker
subotic Jun 17, 2020
b536a3a
build: fix R2R tests with testcontainers (ongoing)
subotic Jun 17, 2020
e146b40
build: fix R2R tests with testcontainers
subotic Jun 18, 2020
14e3c84
Merge branch 'develop' into wip/DSP-30-dockerize-fuseki
Jun 18, 2020
eca606e
test(gravsearch): Don't use deprecated match() function with Fuseki.
Jun 18, 2020
e4e4e47
build: fix github-ci sbt error {ongoing}
subotic Jun 18, 2020
6b85d8c
Merge branch 'develop' into wip/DSP-30-dockerize-fuseki
subotic Jun 18, 2020
2e13af5
build: fix github-ci sbt error {ongoing}
subotic Jun 22, 2020
3568b06
build: change Fuseki config to TDB2
subotic Jun 22, 2020
30b4255
build: add using adoptopenjdk for github-ci
subotic Jun 22, 2020
54e3d39
doc: make Fuseki more prominent
subotic Jun 22, 2020
0f70b6c
build: fix auto init config
subotic Jun 23, 2020
ff70108
fix (sipi): use correct variable for existence checking
subotic Jun 23, 2020
2e28004
Merge branch 'develop' into wip/DSP-30-dockerize-fuseki
subotic Jun 23, 2020
daf4594
test: fix integration tests (ongoing)
subotic Jun 24, 2020
078931d
Merge branch 'wip/DSP-30-dockerize-fuseki' of github.com:dasch-swiss/…
subotic Jun 24, 2020
70fbc6b
tests: ignore tests requiring shared folder
subotic Jun 25, 2020
2d281d1
Merge branch 'develop' into wip/DSP-30-dockerize-fuseki
subotic Jun 25, 2020
1fbcdbd
build: bump sipi version
subotic Jun 26, 2020
96e529d
Merge branch 'develop' into wip/DSP-433-update-sipi
subotic Jun 26, 2020
38ee554
build: bump sipi version
subotic Jun 26, 2020
f2cdae7
feat: add support for changed IIIF 3.0 image API
subotic Jun 28, 2020
b9e9dac
build: add comment
subotic Jun 28, 2020
f33f125
Merge branch 'develop' into wip/DSP-433-update-sipi
Jun 29, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion KnoraBuild.sbt
Original file line number Diff line number Diff line change
Expand Up @@ -228,7 +228,7 @@ lazy val knoraSipi: Project = knoraModule("knora-sipi")
Docker / dockerExposedPorts ++= Seq(1024),
Docker / dockerCommands := Seq(
// FIXME: Someday find out how to reference here Dependencies.Versions.sipiImage
Cmd("FROM", "dhlabbasel/sipi:v2.0.1"),
Cmd("FROM", "daschswiss/sipi:v3.0.0-rc.3"),
Cmd("LABEL", s"""MAINTAINER="${maintainer.value}""""),
Cmd("COPY", "opt/docker/scripts", "/sipi/scripts"),
)
Expand Down
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ publish-knora-jena-fuseki-image: build-knora-jena-fuseki-image ## publish knora-
.PHONY: build-knora-sipi-image
build-knora-sipi-image: build-all-scala ## build and publish knora-sipi docker image locally
@mkdir -p .docker
@sed -e "s/@SIPI_VERSION@/$(SIPI_VERSION)/" docker/knora-sipi.template.dockerfile > .docker/knora-sipi.dockerfile
@sed -e "s|@SIPI_IMAGE@|$(SIPI_IMAGE)|g" docker/knora-sipi.template.dockerfile > .docker/knora-sipi.dockerfile
docker build -t $(KNORA_SIPI_IMAGE) -t $(REPO_PREFIX)/$(KNORA_SIPI_REPO):latest -f .docker/knora-sipi.dockerfile knora-sipi/target/universal

.PHONY: publish-knora-sipi-image
Expand Down
5 changes: 0 additions & 5 deletions docker/knora-assets.dockerfile

This file was deleted.

2 changes: 1 addition & 1 deletion docker/knora-sipi.template.dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM dhlabbasel/sipi:v@SIPI_VERSION@
FROM @SIPI_IMAGE@

COPY stage/scripts /sipi/scripts

Expand Down
11 changes: 0 additions & 11 deletions docker/knora-upgrade.dockerfile

This file was deleted.

2 changes: 1 addition & 1 deletion docs/src/paradox/07-sipi/sipi-and-knora.md
Original file line number Diff line number Diff line change
Expand Up @@ -262,7 +262,7 @@ quality levels:
{
"duration": ​0,
"nx": ​95,
"path": "http://sipiserver:port/knora/incunabula_0000000002.jpg/full/full/0/default.jpg",
"path": "http://sipiserver:port/knora/incunabula_0000000002.jpg/full/max/0/default.jpg",
"ny": ​128,
"fps": ​0,
"format_name": "JPEG",
Expand Down
2 changes: 1 addition & 1 deletion project/Dependencies.scala
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ object Dependencies {
akkaHttpVersion := "10.1.12",
jenaVersion := "3.14.0",
metricsVersion := "4.0.1",
sipiImage := "dhlabbasel/sipi:v2.0.1",
sipiImage := "daschswiss/sipi:v3.0.1-rc.3",
gdbSEImage := "daschswiss/graphdb:9.0.0-se",
gdbFreeImage := "daschswiss/graphdb:9.0.0-free"
)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,6 @@ let changeFileValueRequest: changeValueFormats.changeFileValueRequest = {
}
};

let changeFileValueResponse: changeValueFormats.changeFileValueResponse = {"locations":[{"duration":0,"nx":128,"path":"http://localhost:1024/knora/5XTEI1z10A2-D8ojQHrMiUz.jpg/full/full/0/default.jpg","ny":72,"fps":0,"format_name":"JPEG","origname":"2016-06-26+12.26.45.jpg","protocol":"file"},{"duration":0,"nx":3264,"path":"http://localhost:1024/knora/5XTEI1z10A2-D8ojQHrMiUz.jpx/full/3264,1836/0/default.jpg","ny":1836,"fps":0,"format_name":"JPEG2000","origname":"2016-06-26+12.26.45.jpg","protocol":"file"}],"status":0};
let changeFileValueResponse: changeValueFormats.changeFileValueResponse = {"locations":[{"duration":0,"nx":128,"path":"http://localhost:1024/knora/5XTEI1z10A2-D8ojQHrMiUz.jpg/full/max/0/default.jpg","ny":72,"fps":0,"format_name":"JPEG","origname":"2016-06-26+12.26.45.jpg","protocol":"file"},{"duration":0,"nx":3264,"path":"http://localhost:1024/knora/5XTEI1z10A2-D8ojQHrMiUz.jpx/full/3264,1836/0/default.jpg","ny":1836,"fps":0,"format_name":"JPEG2000","origname":"2016-06-26+12.26.45.jpg","protocol":"file"}],"status":0};

let changeValueComment: changeValueFormats.changeValueCommentRequest = {"project_id":"http://rdfh.ch/projects/77275339","comment":"dumm"};

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion sipi/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ print-env-file: env-file ## prints the env file used by sipi

.PHONY: env-file
env-file: ## write the env file used by sipi
@echo SIPI_VERSION=v$(SIPI_VERSION) > .env
@echo SIPI_IMAGE=$(SIPI_IMAGE) > .env
@echo DOCKERHOST=$(DOCKERHOST) >> .env

.PHONY: up
Expand Down
4 changes: 2 additions & 2 deletions sipi/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ version: '3'
services:
# sipi using default (production-like) configuration with additional routes for testing
sipi:
image: dhlabbasel/sipi:${SIPI_VERSION}
image: ${SIPI_IMAGE}
container_name: sipi
ports:
- "1024:1024"
Expand All @@ -26,7 +26,7 @@ services:

# sipi using configuration which disables authentication
sipi-no-auth:
image: dhlabbasel/sipi:${SIPI_VERSION}
image: ${SIPI_IMAGE}
container_name: sipi-no-auth
ports:
- "1024:1024"
Expand Down
2 changes: 2 additions & 0 deletions sipi/images/thumbs/.gitignore
Original file line number Diff line number Diff line change
@@ -1,2 +1,4 @@
# Ignore everything in this directory
*
# Except these files
!.gitignore
Binary file removed sipi/images/thumbs/2E8F4aDodnf-Br4rVn1d4FG.jpg
Binary file not shown.
Binary file removed sipi/images/thumbs/6JU4IuzMr7N-GA9F2syJysr.jpg
Binary file not shown.
Binary file removed sipi/images/thumbs/7vWhcdvCs5h-CCyR8SxDqDV.jpg
Binary file not shown.
Binary file removed sipi/images/thumbs/E9wuATEYjXt-BKQrZeu7goo.jpg
Binary file not shown.
Binary file removed sipi/images/thumbs/G1DWNcSWS59-BPLOBsPeafX.jpg
Binary file not shown.
2 changes: 1 addition & 1 deletion sipi/scripts/make_thumbnail.lua
Original file line number Diff line number Diff line change
Expand Up @@ -175,7 +175,7 @@ for imgindex, imgparam in pairs(server.uploads) do
nx_thumb = dims.nx,
ny_thumb = dims.ny,
mimetype_thumb = 'image/jpeg',
preview_path = get_external_protocol() .. "://" .. get_external_hostname() .. ":" .. get_external_port() .."/thumbs/" .. thumbName .. "/full/full/0/default.jpg",
preview_path = get_external_protocol() .. "://" .. get_external_hostname() .. ":" .. get_external_port() .."/thumbs/" .. thumbName .. "/full/max/0/default.jpg",
filename = tmpName, -- make this a IIIF URL
original_mimetype = submitted_mimetype.mimetype,
original_filename = filename,
Expand Down
2 changes: 1 addition & 1 deletion sipi/server/test.html
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
for (var i = 2; i <= 632; i++) {
var node = document.createElement("img");
var numstr = i.pad(10);
node.setAttribute("src", "http://localhost:1024/images/Incunabula_" + numstr + ".jpg/full/full/0/default.jpg");
node.setAttribute("src", "http://localhost:1024/images/Incunabula_" + numstr + ".jpg/full/max/0/default.jpg");
ele.appendChild(node);
}
}
Expand Down
4 changes: 3 additions & 1 deletion vars.mk
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
SIPI_VERSION := 2.0.1
SIPI_VERSION := v3.0.0-rc.3
SIPI_REPOSITORY := daschswiss/sipi
SIPI_IMAGE := $(SIPI_REPOSITORY):$(SIPI_VERSION)

FUSEKI_HEAP_SIZE := 3G

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ class KnoraSipiScriptsV1ITSpec extends ITKnoraFakeSpec(KnoraSipiScriptsV1ITSpec.
"mimetype_thumb":"image/jpeg",
"original_mimetype":"image/jpeg",
"nx_thumb":93,
"preview_path":"http://localhost:1024/thumbs/CjwDMhlrctI-BG7gms08BJ4.jpg/full/full/0/default.jpg",
"preview_path":"http://localhost:1024/thumbs/CjwDMhlrctI-BG7gms08BJ4.jpg/full/max/0/default.jpg",
"filename":"CjwDMhlrctI-BG7gms08BJ4",
"file_type":"IMAGE",
"original_filename":"Chlaus.jpg",
Expand Down Expand Up @@ -173,7 +173,7 @@ class KnoraSipiScriptsV1ITSpec extends ITKnoraFakeSpec(KnoraSipiScriptsV1ITSpec.
// Running with KnoraFakeService which always allows access to files.
// Send a GET request to Sipi, asking for full image
// not possible as authentication is required and file needs to be known by knora to be able to authenticate the request
val sipiGetImageRequest = Get(baseInternalSipiUrl + "/0001/" + filenameFull + "/full/full/0/default.jpg") ~> addCredentials(BasicHttpCredentials(username, password))
val sipiGetImageRequest = Get(baseInternalSipiUrl + "/0001/" + filenameFull + "/full/max/0/default.jpg") ~> addCredentials(BasicHttpCredentials(username, password))
checkResponseOK(sipiGetImageRequest)

// Send a GET request to Sipi, asking for the info.json of the image
Expand Down Expand Up @@ -219,7 +219,7 @@ class KnoraSipiScriptsV1ITSpec extends ITKnoraFakeSpec(KnoraSipiScriptsV1ITSpec.
//log.debug("sipiConvertFromPathPostResponseJson: {}", sipiConvertFromPathPostResponseJson)

// Running with KnoraFakeService which always allows access to files.
val sipiGetImageRequest = Get(baseInternalSipiUrl + "/0001/" + filenameFull + "/full/full/0/default.jpg") ~> addCredentials(BasicHttpCredentials(username, password))
val sipiGetImageRequest = Get(baseInternalSipiUrl + "/0001/" + filenameFull + "/full/max/0/default.jpg") ~> addCredentials(BasicHttpCredentials(username, password))
checkResponseOK(sipiGetImageRequest)

// Send a GET request to Sipi, asking for the info.json of the image
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -166,7 +166,7 @@ class KnoraSipiIntegrationV2ITSpec extends ITKnoraLiveSpec(KnoraSipiIntegrationV
// Request the temporary file from Sipi.
for (responseEntry <- sipiUploadResponse.uploadedFiles) {
val sipiGetTmpFileRequest: HttpRequest = if (responseEntry.fileType == "image") {
Get(responseEntry.temporaryUrl.replace("http://0.0.0.0:1024", baseExternalSipiUrl) + "/full/full/0/default.jpg")
Get(responseEntry.temporaryUrl.replace("http://0.0.0.0:1024", baseExternalSipiUrl) + "/full/max/0/default.jpg")
} else {
Get(responseEntry.temporaryUrl.replace("http://0.0.0.0:1024", baseExternalSipiUrl))
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ class ValueUtilV1(private val settings: KnoraSettingsImpl) {
}

def makeSipiImagePreviewGetUrlFromFilename(projectShortcode: String, filename: String): String = {
s"${settings.externalSipiIIIFGetUrl}/$projectShortcode/$filename/full/full/0/default.jpg"
s"${settings.externalSipiIIIFGetUrl}/$projectShortcode/$filename/full/max/0/default.jpg"
}

/**
Expand Down