Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Eliminate the environment variable OTOBO_SYNC_WITH_S3 #1912

Closed
bschmalhofer opened this issue Sep 5, 2022 · 5 comments
Closed

Eliminate the environment variable OTOBO_SYNC_WITH_S3 #1912

bschmalhofer opened this issue Sep 5, 2022 · 5 comments
Assignees
Labels
enhancement New feature or request
Milestone

Comments

@bschmalhofer
Copy link
Contributor

bschmalhofer commented Sep 5, 2022

As of OTOBO 10.1.5 the environment variable OTOBO_SYNC_WITH_S3 must be set when the S3 backend is used. This applied to:

  • web server GazelleOTOBO_SYNC_WITH_S3
  • web server Apache
  • Console commands
  • OTOBO Daemon

It is fairly easy to get this wrong. For example, one could do a SysConfig deployment via a webserver that does not have OTOBO_SYNC_WITH_S3 set. This results in a deployment of ZZZAAuto.pm directly into the file system. But the next action, where the variable is set, restores the old version on ZZZAAuto,pm from the S3 backend.

Let's list where and why OTOBO_SYNC_WITH_S3 is currently used:

Kernel/Config.pm and the template _Kernel/Config.pm.docker.dist

This is a guard that allows to active and deactivate the S3 backend. This feature is not really useful as usually the S3 configuration must be adapted manually anyways. In future the adaption could be handled by installer.pl, But that won't require the switch either.

See also #1427 .

Conditional loading of Kernel/System/Storage/S3

Conditional loading at compile time is useful and convenient. But the loading could be done on first use, implicitly depending on Storage::S3::Active from Kernel/Config.pm

Used in:

  • Kernel/System/ProcessManagement/DB/Process.pm
  • Kernel/System/Loader.pm
  • Kernel/System/UnitTest/Helper.pm
  • Kernel/System/SysConfig.pm
  • Kernel/System/ACL/DB/ACL.pm
  • Kernel/Config/Defaults.pm
  • bin/psgi-bin/otobo.psgi

Other uses that might be replaced with checking Storage::S3::Active

  • Kernel/System/ProcessManagement/DB/Process.pm
  • Kernel/Config/Defaults.pm

bin/psgi-bin/otobo.psgi

There should be an early decision whether the $SyscFromS3Middleware should be acitvated. This means that a minimal config object must be made available.

Gazelle Loader Module Kernel/cpan-lib/Plack/Loader/SyncWithS3.pm

This is bit of a special case, as the web server is started before installer.pl has run. This is no problem for the loader module itself, as Kernel/Config.pm must simply be adapted before Gazelle is started.

The decision whether the loader module is activated is currently done in bin/docker/entrypoint.sh . Maybe this usage can remain for now.

See https://github.com/RotherOSS/otobo/blob/rel-10_1/Kernel/cpan-lib/Plack/Loader/SyncWithS3.pm#L92

bin/docker/quick_setup.pl

This usage is unncessary. There already is a command line option for that.

Unittest scripts

All cases should be converted to checking Storage::S3::Active.

@bschmalhofer bschmalhofer added the enhancement New feature or request label Sep 5, 2022
@bschmalhofer bschmalhofer self-assigned this Sep 5, 2022
bschmalhofer added a commit that referenced this issue Sep 7, 2022
whether the S3 storage backend is active
@bschmalhofer
Copy link
Contributor Author

bschmalhofer commented Sep 7, 2022

Eliminated completely the usage of OTOBO_SYNC_WITH_S3. The information whether the S3 storage backend is active is now solely taken from Kernel/Config.pm . When running with Docker compost it is recommended to activate S3 before building the image.

TODO:

  • remove OTOBO_SYNC_WITH_S3 from otobo-docker.

@bschmalhofer bschmalhofer added this to the OTOBO 10.1.6 milestone Sep 7, 2022
@bschmalhofer
Copy link
Contributor Author

Merging the PR. Closing ticket only when things look stable.

@bschmalhofer
Copy link
Contributor Author

bschmalhofer commented Sep 8, 2022

TODO:

  • check whether the test suite still runs fine with the S3 backend enabled

bschmalhofer added a commit that referenced this issue Sep 8, 2022
Issue #1912: the documented way of reading Kernel/Config.pm
@bschmalhofer
Copy link
Contributor Author

The test suite shows two unexpected errors. These should be fixed before closing this issue.

Test Summary Report
-------------------
/opt/otobo/scripts/test/Selenium/Agent/Admin/ProcessManagement/AdminProcessManagementTransitionAction.t   (Wstat: 768 (exited 3) Tests: 87 Failed: 3)
  Failed tests:  81, 84, 87
  Non-zero exit status: 3
/opt/otobo/scripts/test/Selenium/Agent/AgentAppointmentCalendarOverview.t                                 (Wstat: 256 (exited 1) Tests: 16 Failed: 1)
  Failed test:  16
  Non-zero exit status: 1
/opt/otobo/scripts/test/Selenium/TestingMethods.t                                                         (Wstat: 0 Tests: 36 Failed: 0)
  TODO passed:   21, 24, 28
/opt/otobo/scripts/test/SysConfig/GlobalEffectiveValueGet.t                                               (Wstat: 256 (exited 1) Tests: 3 Failed: 1)
  Failed test:  2
  Non-zero exit status: 1
/opt/otobo/scripts/test/SysConfig/OTOBOCommunity/UserConfigurationDeploymentSync.t                        (Wstat: 256 (exited 1) Tests: 15 Failed: 1)
  Failed test:  11
  Non-zero exit status: 1
Files=964, Tests=80367, 6187 wallclock secs (30.25 usr  3.66 sys + 1879.63 cusr 231.47 csys = 2145.01 CPU)
Result: FAIL
ran tests for product OTOBO 10.1.x on host 3baa52e653b4 .

@bschmalhofer
Copy link
Contributor Author

See also #1909 and #1939. After fixing the deployment sync, the test suite looks mostly fine. Closing this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant