…sts-20230209-edge

[tests-only][full-ci]Bump ocis commit id for tests in edge

* add unique id to events

Signed-off-by: jkoberg <jkoberg@owncloud.com>

* add ConsumeAll function

Signed-off-by: jkoberg <jkoberg@owncloud.com>

* add helper to create natsstream from config

Signed-off-by: jkoberg <jkoberg@owncloud.com>

---------

Signed-off-by: jkoberg <jkoberg@owncloud.com>

* Support FreeBSD xattr naming

There is a slight compatibilty issue due to FreeBSD disallowing
usage of the `user.` prefix when it comes to xattr names.
Just for FreeBSD this patch removes that `user.` prefix (leaving
`ocis.`) which allows it to run on FreeBSD.

* Cleanup the FreeBSD xattr naming

* Comment for default xattr namespace as per Codacy static analysis

* chore: changelog update for freebsd-xattr-support

---------

Co-authored-by: Jur van den Berg <jurl.berg@gmail.com>

* Bump ocis commit id to latest

* Fixed line numbers in expected failure

* Fixed ref-path of github from core to ocis

…cs3org#3649)

* Store node attributes in actual files instead of xattrs

* Use our xattrs instead of xattr everywhere. Fix tests.

* Cops the type attribute when copying attributes

* Use mtime of the childre dir when no tmtime is set

* Fix node.IsDir() logic

* Fix trash bin item types

* Fix part of the trashbin functionality

* Fix node id of recycle items

* Fix deleting references to subtrees that have been restored

* Fix cleaning up after restoring items from the trash bin

* Fix purging subdirectories of trash bin entries

* Hound fixes

* More linter fixes

* Use the node's GetTMTime() instead of reading it manually

* Do not change the mtime when setting attrs. xattrs don't do it either.

* Fix error message

* Store file metadata separately instead of moving the children elsewhere

Co-Authored-By: Jörn Friedrich Dreyer <jfd@butonic.de>

* Fix tests

* Filter meta files when listing revisions

* Fix tests

* Fix deep listing the recycle

* Fix reading items deeper in the trash

* Encode binary values

* Fall back to getting the node type by looking at the disk

* some xattr backend fixes

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* make metadata backend configurable

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* make linter happy

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* add changelog

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* add missing comments

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* more linter fixes

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* always initialize xattr backend to make tests pass

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* Fix typo

* Improve error messages

* Use lockedfile package to access metadata files

* Cache file metadata

* Tidy go modules

* Do not ignore error when writing the cache

* use global var for regex

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

---------

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Co-authored-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* add a config options for a max quota

* support setting quota when creating a home space

They are no longer used.

Signed-off-by: jkoberg <jkoberg@owncloud.com>

…23-edge

[tests-only] Bump APITESTS_COMMITID

* Fix ini metadata backend unit tests

* Adapt changelog

Add FileID Header on COPY

Signed-off-by: jkoberg <jkoberg@owncloud.com>

Return 425 on ocdav GET

)

When CreateStorageSpace is called without a quota but a max quota was set then the space can at most have max quota.

Signed-off-by: jkoberg <jkoberg@owncloud.com>

Signed-off-by: jkoberg <jkoberg@owncloud.com>

Signed-off-by: jkoberg <jkoberg@owncloud.com>

Fix caching

Signed-off-by: jkoberg <jkoberg@owncloud.com>

* Get and cache all attributes when reading a specific attribute

* Write tree size and tree time simultaneously

* Clean up lock handling when writing file metadata

Locks are either handled by the backend or given from external now.

* Add tests that prove that lockedfile does what we need

* Store metadata in a redis cache

* Truncate the meta file before writing the new data

* Fix setting empty attributes

* Lazily create metafiles

* Refactor xattrs backend interface to make life easier for consumers

* Use a reva store cache instead of redis directly

* Bump go-micro redis plugin to pull in sentinel support

* Allow configuring a redis sentinel cache

The sentinel configuration is given in the cache node in the following
form:

    <host>/<name of master>

e.g.

    10.10.0.207/mymaster

* Remove global xattrs backend variable

This allows for using different backends for different filesystems in
the same process.

* Refactoring: Cleanup xattrs/metadata package

* Switch the s3ng integration tests to the ini+redis metadata backend

* Properly create/cleanup locks when creating them in the xattrs backend

* Fix tests

* Fix data races in tests

* Ignore empty tmpetags

* Fix linter issues

* Do not encode already encoded data

* Add changelogs

* Use default reva cache for the ini metadata backend

Signed-off-by: jkoberg <jkoberg@owncloud.com>

Signed-off-by: jkoberg <jkoberg@owncloud.com>

Signed-off-by: jkoberg <jkoberg@owncloud.com>

Signed-off-by: jkoberg <jkoberg@owncloud.com>

Add More Info to Events

A stray parenthesis got in t he disabled filter code, this removes it.

Return a permission denied error when the user is not able to list the
to be delete space.

Fix error handling in DeleteStorage space

)

* Bump ocis commit id to latest

* Fixed line numbers in expected failure

When the space quota should be updated to '0' i.e. unlimited and a global max quota was configured we only give the space the max quota amount instead of unlimited storage.

respect the max quota when updating the space quota

properly enforce the password on writeable public links

* Use msgpack to serialize cache data instead of json

* Use storage-relative paths as the cache key

That prevents non-aligned cache keys when multiple providers mount the
storage at a different location.

* Create dedicated type for node attributes, based on map[string][]byte

* add messagepack backend

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* Switch msgpack library to a faster, leaner one

* Adapt changelog

* Cleanup

* Fix test

* Fix error message

* Use messagepack backend in the s3ng acceptance tests

* make hound happy

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* flip tests

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* Do not try to get the tmtime for files being uploaded

* Extend the metadata backend to read the attributes from a locked source

* Fix comment

---------

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Co-authored-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* Refactor tree migrations into a separate subpackage

* Add on-the-fly migration for switching from xattrs to metapack metadata

Also remember migration states to not run migrations over and over again.

* Hound fixes

* Linter fixes

* Add changelog

* More linter fixes

* Make hound happy

* Use existing logger

* Make sure to migrate storages only once at a time

* Be less verbose about old migrations during startup

* Allow for reusing existing spaceroots/parent information in ReadNode

* Do not resolve the parent just to transform the parent id

* Fetch child node information concurrently

* Transform child nodes concurrently

* Add changelog

Signed-off-by: jkoberg <jkoberg@owncloud.com>

add read_write_delete key

* Make metadata backend detection method public

* Add changelog

* Fix hound issue

…Permission

Check for SetProjectSpaceQuota permission

If renaming fails for whatever reason, it falls back to copying as before.
This comes for the cost of an interface change towards file name instead
of Reader.
Plus some fixed tests.

Try to rename the file after upload instead of copying.

…3org#3740)

* Limit the number of concurrent goroutines for listing directories

* Adapt tests

* Fix linter issues

* Add changelog

* prepare for antivirus

Signed-off-by: jkoberg <jkoberg@owncloud.com>

* add changelog

Signed-off-by: jkoberg <jkoberg@owncloud.com>

* set scan data after scan is finished

Signed-off-by: jkoberg <jkoberg@owncloud.com>

---------

Signed-off-by: jkoberg <jkoberg@owncloud.com>

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

…tiple-groups

Fix listing shares to multiple groups

[full-ci] [tests-only] Bump ocis commit id edge

Signed-off-by: jkoberg <jkoberg@owncloud.com>

[tests-only] Unmarshal PP events correctly

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

Co-authored-by: kobergj <juliankoberg@googlemail.com>

…_user_type_from_ldap

Add UserTypeProperty to get user type from ldap user entry

change language code validation

Signed-off-by: jkoberg <jkoberg@owncloud.com>

Signed-off-by: jkoberg <jkoberg@owncloud.com>

Signed-off-by: jkoberg <jkoberg@owncloud.com>

Signed-off-by: jkoberg <jkoberg@owncloud.com>

GDPR Export Preliminaries

check for password during update public link

…0230331-edge

[tests-only][full-ci] Bump ocis commit id 20230331 edge

Signed-off-by: jkoberg <jkoberg@owncloud.com>

Signed-off-by: jkoberg <jkoberg@owncloud.com>

Add capability for personal data export

…_listing_and_adding_sharees

Set correct share type when listing shares

add missing cors config in the ocdav service