Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

next round of refactoring #35

Merged
merged 17 commits into from
Oct 12, 2023
Merged

next round of refactoring #35

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
fc7fe1d
fix: namespaces and package info
line-o Oct 2, 2023
cbeaa09
feat: backup and restore existing configuration
line-o Oct 3, 2023
c94469b
fix: expath package dependencies, bump version
line-o Oct 3, 2023
86d0301
fix(api): error constructing status xml
line-o Oct 3, 2023
1229819
refactor: rename blacklist to ignore
line-o Oct 4, 2023
96287f9
refactor: optmise api handlers
line-o Oct 4, 2023
9d439dc
fix(api): check collection parameter value
line-o Oct 5, 2023
e1d2713
fix(api): several issues with deployment
line-o Oct 5, 2023
262567e
chore: remove unused files
line-o Oct 5, 2023
c10fbcd
style: min-height for main page
line-o Oct 5, 2023
84d8926
fix(controller): tighten security, remove cruft
line-o Oct 5, 2023
4d85d75
chore(app): rename one function
line-o Oct 5, 2023
8acbc8a
chore: fix xqdoc
line-o Oct 5, 2023
39a97ff
doc(api): clarify function purpose
line-o Oct 5, 2023
e4c209e
doc(api): clarify code checking for errors
line-o Oct 5, 2023
8ca00a7
fix(controller): tighten security for api.json
line-o Oct 6, 2023
4cd2919
fix(controller): cache-control for static files
line-o Oct 9, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions .existdb.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,13 +19,13 @@
]
},
"package": {
"author": "Dom&Jörn",
"author": "TEI-publisher project team",
"target": "tuttle",
"description": "tuttle - github integration.",
"namespace": "http://exist-db.org/apps/tuttle",
"website": "http://exist-db.org/apps/tuttle",
"description": "Synchronize data with git services",
"namespace": "http://e-editiones.org/tuttle",
"website": "https://github.com/eeditiones/tuttle",
"status": "dev",
"title": "tuttle - github integration",
"title": "tuttle - pull data from git APIs",
"license": "GPLv3"
}
}
20 changes: 20 additions & 0 deletions src/cleanup.xq
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
xquery version "3.1";

declare namespace sm="http://exist-db.org/xquery/securitymanager";

(: TODO: $target is not set in cleanup phase :)
declare variable $configuration-collection := "/db/apps/tuttle/data/";
declare variable $backup-collection := "/db/tuttle-backup/";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see my comment on mattermost, I d really prefer a collection name that is more flexible for other apps as well, and won't lead to one collection per app with similar demands. so far we only have system and app following the db root collection. Lets create a third location for this kind of backup files

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do favour a broader solution as well. Please understand that this is not the right moment to set de-facto standards and the collection in this case is very short-lived. The entire /db/tuttle-backup is removed after installation.
And I would also prefer to discuss a general solution within the community not just among us.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

happy to discuss a general solution with the community. I just don't see what we gain there, by adopting a single use solution now, instead of one that is open to extension

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we choose a collection name that is
a) the one other apps will be using then it needs extra protection from removal and will have to stay there even if unused
b) not the one we are choosing here, the code has to be changed anyway

declare variable $configuration-filename := "tuttle.xml";

(: backup tuttle configuration :)
if (not(xmldb:collection-available($backup-collection)))
then ((: move/copy to collection :)
util:log("info", "Creating configuration backup collection"),
xmldb:create-collection("/db", "tuttle-backup"),
sm:chmod(xs:anyURI($backup-collection), "rwxr-x---")
)
else ()
,
util:log("info", "Backing up configuration"),
xmldb:move($configuration-collection, $backup-collection, $configuration-filename)
69 changes: 16 additions & 53 deletions src/controller.xql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,16 +7,20 @@ declare variable $exist:controller external;
declare variable $exist:prefix external;
declare variable $exist:root external;

if ($exist:path eq "") then
declare variable $is-get := lower-case(request:get-method()) eq 'get';

if ($is-get and $exist:path eq "") then
<dispatch xmlns="http://exist.sourceforge.net/NS/exist">
<redirect url="{request:get-uri()}/"/>
</dispatch>

(: forward root path to index.xql :)
else if ($exist:path eq "/") then
else if ($is-get and $exist:path eq "/") then
<dispatch xmlns="http://exist.sourceforge.net/NS/exist">
<redirect url="index.html"/>
<forward url="{$exist:controller}/index.html">
<set-header name="Content-Type" value="text/html"/>
</forward>
</dispatch>

else if ($exist:resource eq 'login') then
let $loggedIn := login:set-user("org.exist.login", (), false())
let $user := request:get-attribute("org.exist.login.user")
Expand All @@ -40,56 +44,15 @@ else if ($exist:resource eq 'login') then
)

(: static HTML page for API documentation should be served directly to make sure it is always accessible :)
else if ($exist:path eq "/index.html") then
<dispatch xmlns="http://exist.sourceforge.net/NS/exist">
<set-header name="Content-Type" value="text/html"/>
</dispatch>
else if ($exist:path eq "/data/tuttle.xml") then
<dispatch xmlns="http://exist.sourceforge.net/NS/exist">
<forward url="{$exist:controller}/data/tuttle.xml">
<set-header name="Cache-Control" value="max-age=31536000"/>
</forward>
</dispatch>
else if ($exist:path eq "/api.html" or ends-with($exist:resource, "json")) then
<dispatch xmlns="http://exist.sourceforge.net/NS/exist">
</dispatch>

(: other images are resolved against the data collection and also returned directly :)
else if (matches($exist:resource, "\.(css)$", "s")) then
<dispatch xmlns="http://exist.sourceforge.net/NS/exist">
</dispatch>
else if ($is-get and $exist:path = ("/api.html", "/api.json")) then
<dispatch xmlns="http://exist.sourceforge.net/NS/exist" />

else if (matches($exist:resource, "\.(js)$", "s")) then
(: serve static resources :)
else if ($is-get and matches($exist:path, "^/(css|js|images)/[^/]+\.(css|js(\.map)?|svg|jpg|png)$")) then
<dispatch xmlns="http://exist.sourceforge.net/NS/exist">
</dispatch>

else if (matches($exist:resource, "\.(js.map)$", "s")) then
<dispatch xmlns="http://exist.sourceforge.net/NS/exist">
</dispatch>

else if (matches($exist:resource, "\.(png)$", "s")) then
<dispatch xmlns="http://exist.sourceforge.net/NS/exist">
</dispatch>
else if (matches($exist:resource, "\.(svg)$", "s")) then
<dispatch xmlns="http://exist.sourceforge.net/NS/exist">
</dispatch>

else if (matches($exist:resource, "\.(png|jpg|jpeg|gif|tif|tiff|txt|mei|js)$", "s")) then
<dispatch xmlns="http://exist.sourceforge.net/NS/exist">
<forward url="{$exist:controller}/data/{$exist:path}">
<set-header name="Cache-Control" value="max-age=31536000"/>
</forward>
</dispatch>

(: use a different Open API router, needs exist-jwt installed! :)
else if (starts-with($exist:path, '/jwt')) then
<dispatch xmlns="http://exist.sourceforge.net/NS/exist">
<forward url="{$exist:controller}/modules/custom-router.xq">
<set-header name="Access-Control-Allow-Origin" value="*"/>
<set-header name="Access-Control-Allow-Credentials" value="true"/>
<set-header name="Access-Control-Allow-Methods" value="GET, POST, DELETE, PUT, PATCH, OPTIONS"/>
<set-header name="Access-Control-Allow-Headers" value="Accept, Content-Type, Authorization, X-Auth-Token"/>
<set-header name="Cache-Control" value="no-cache"/>
<forward url="{$exist:controller}/resources/{$exist:path}">
<set-header name="Cache-Control" value="max-age=2419200, must-revalidate, stale-while-revalidate=86400"/>
<!-- <set-header name="Cache-Control" value="max-age=31536000"/> -->
</forward>
</dispatch>

Expand All @@ -103,4 +66,4 @@ else
<set-header name="Access-Control-Allow-Headers" value="Accept, Content-Type, Authorization, X-Start"/>
<set-header name="Cache-Control" value="no-cache"/>
</forward>
</dispatch>
</dispatch>
73 changes: 73 additions & 0 deletions src/data/tuttle-example-config.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
<tuttle>
<repos>
<collection name="tuttle-sample-data">
<!-- optional: make _one_ repository the default
for endpoints without {collection}
-->
<default>true</default>

<!-- which sercive to connect to -->
<type>github</type>
<baseurl>https://api.github.com/</baseurl>

<!-- necessary to provide for GitHub -->
<owner>eeditiones</owner>
<repo>tuttle-sample-data</repo>

<!-- branch to pull from -->
<ref>next</ref>

<!-- authenticate against remote service,
can also be read from environment variables -->
<token>XXX</token>

<!-- only necessary if you need the service to call tuttle -->
<hookuser>admin</hookuser>
<hookpasswd></hookpasswd>
</collection>

<collection name="tuttle-sample-gitlab">
<!-- which sercive to connect to -->
<type>gitlab</type>
<baseurl>https://gitlab.com/api/v4/</baseurl>

<!-- this is not needed for Gitlab but can be added for human consumption -->
<owner>line-o</owner>
<repo>tuttle-sample-data</repo>

<!-- necessary to provide for Gitlab -->
<project-id>50872175</project-id>

<!-- branch to pull from -->
<ref>main</ref>

<!-- authenticate against remote service,
can also be read from environment variables -->
<token>XXX</token>

<!-- only necessary if you need the service to call tuttle -->
<hookuser>admin</hookuser>
<hookpasswd></hookpasswd>
</collection>
</repos>

<!-- these files will not be extracted from archives and left untouched when cleaning up -->
<ignore>
<file>existdb.json</file>
<file>build.xml</file>
<file>README.md</file>
<file>.gitignore</file>
<file>expath-pkg.xml.tmpl</file>
<file>repo.xml.tmpl</file>
<file>build.properties.xml</file>
</ignore>

<!-- prefix, suffix, lock and apikeys can usually be left as-is -->
<config prefix="/db/apps/" suffix="-stage" lock="git-lock.xml" apikeys="/db/system/auth/tuttle-token.xml">
<!--
The permissions the deployed data gets assigned, if no expath-pkg.xml is found.
The initial setting is safe, but this can and should be adapted to your requirements.
-->
<sm user="admin" group="dba" mode="rw-r-----"/>
</config>
</tuttle>
42 changes: 0 additions & 42 deletions src/data/tuttle.xml
View file
Open in desktop

This file was deleted.

68 changes: 0 additions & 68 deletions src/demo.css
View file
Open in desktop

This file was deleted.

15 changes: 6 additions & 9 deletions src/expath-pkg.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,8 @@
<?xml version="1.0" encoding="UTF-8"?>
<package
xmlns="http://expath.org/ns/pkg"
name="http://e-editiones.org/tuttle"
abbrev="tuttle" version="1.2.0" spec="1.0">
<title>Tuttle - Git for eXist-db</title>
<dependency processor="http://exist-db.org" semver-min="5.5.1"/>
<dependency package="http://e-editiones.org/roaster" semver-min="1.0.0"/>
<dependency package="http://exist-db.org/apps/shared" semver-min="0.9.1"/>
<dependency package="http://expath.org/ns/crypto" semver-min="5.0.0"/>
<package xmlns="http://expath.org/ns/pkg" spec="1.0"
name="http://e-editiones.org/tuttle" abbrev="tuttle" version="2.0.0">
<title>Tuttle - Git for eXist-db</title>
<dependency processor="http://exist-db.org" semver-min="5.5.1" semver-max="6" />
<dependency package="http://e-editiones.org/roaster" semver-min="1.0.0"/>
<dependency package="http://expath.org/ns/crypto" semver-min="5.0.0"/>
</package>
32 changes: 32 additions & 0 deletions src/finish.xq
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
xquery version "3.1";

declare namespace sm="http://exist-db.org/xquery/securitymanager";

(: the target collection into which the app is deployed :)
declare variable $target external;

declare variable $configuration-collection := $target || "/data/";
declare variable $backup-collection := "/db/tuttle-backup/";
declare variable $configuration-filename := "tuttle.xml";

(: look for backed up tuttle configuration :)
if (doc-available($backup-collection || $configuration-filename))
then ((: move/copy to collection :)
util:log("info", "Restoring tuttle configuration from backup."),
xmldb:move($backup-collection, $configuration-collection, $configuration-filename),
xmldb:remove($backup-collection)
)
else ((: copy example configuration when no backup was found :)
util:log("info", "No previous tuttle configuration found."),
xmldb:copy-resource(
$configuration-collection, "tuttle-example-config.xml",
$configuration-collection, $configuration-filename
)
)
,
(: tighten security for configuration file :)
sm:chmod(xs:anyURI($configuration-collection || $configuration-filename), "rw-r-----")
,
(: set gid for API :)
sm:chmod(xs:anyURI($target || "/modules/api.xql"), "rwxr-sr-x")

Loading
Loading