Docker image not using the /data volume for actions_log and packages

[w-flo]

Description

Hi, and thanks for gitea. I especially love the new gitea actions!

I've noticed I can't look at old gitea actions logs. The HTTP request fails with the following HTTP body (status 500):

storage open "w-flo/[..]/27/39.log": open /app/gitea/data/actions_log/w-flo/[..]/27/39.log: no such file or directory

And indeed that is expected, because my docker container only has a persistent /data/ volume like the gitea docs recommend, everything else will be dropped along with the container itself when I upgrade gitea.

I could probably set a PATH for [storage.actions_log], or preferably [storage], in app.ini, but I'd say the docker image should be configured to store actions logs in /data by default, along with everything else that should be stored permanently?

I have noticed previously that avatars go missing on a regular basis, so maybe avatars are not stored in /data either. Looking at the possible app.ini keys, maybe packages and attachments are affected as well? Edit: I have checked the container file system, and indeed the /app/gitea/data contains packages, avatars, attachments etc. (but I haven't used these features a lot so I hadn't noticed in the past years). So it's not only about actions_logs.

Repositories and issue trackers are not lost on docker upgrade though, as I have manually set [database] PATH and [repository] ROOT in app.ini. These should probably default to something inside /data as well if that is not the case already.

If the issue is that I should have changed something in app.ini to /data, I think it would be great to mention that in the "Installation with Docker" documentation.

Gitea Version

1.19.3

Can you reproduce the bug on the Gitea demo site?

No

Log Gist

No response

Screenshots

No response

Git Version

No response

Operating System

No response

How are you running Gitea?

Official docker image through docker-compose, with some modifications to app.ini (not through environment variables, directly in the file on the volume)

Database

SQLite

[wolfogre]

Updated the title because I thought avatars and attachments should work

gitea/docker/root/etc/templates/app.ini

Lines 39 to 50 in 5e5740a

	[picture]
	AVATAR_UPLOAD_PATH = /data/gitea/avatars
	REPOSITORY_AVATAR_UPLOAD_PATH = /data/gitea/repo-avatars
	[attachment]
	PATH = /data/gitea/attachments
	[log]
	MODE = console
	LEVEL = info
	ROUTER = console
	ROOT_PATH = /data/gitea/log

[w-flo]

Oh, I haven't seen that file. I need to look at how that template file is used. I don't use these settings in my own app.ini.

My installation is a couple of years old, migrated from gogs very early on. The /data volume actually contains some old avatars from 2018 in /data/gitea/data/avatars/, with file names "1", "2" etc. (probably user IDs), but the current avatars appear to be in /app/gitea/data and have different file names that look like a hash.

Are we supposed to add things to our app.ini files manually when the template file changes?

Ideally, I'd set only the things that I really want/need to change in app.ini (or through environment variables), and everything else would use default values appropriate for the docker environment, for example from that template file.

---

Edit: After looking at the differences between this template file and my own app.ini (that was created years ago, maybe even through a gogs docker container?), I applied anything that was missing from my file and now things seem to be correct. Even the packages and action logs are in /data now, probably because of the "APP_DATA_PATH" setting that was missing from my ini.

Still, I think it would be useful to be able to have a way to specify only a minimum amount of config (like mandatory INTERNAL_TOKEN and SECRET_KEY, maybe ROOT_URL, and then I'd add whatever I want to change vs. the "docker defaults") and let some script fill in the other "boilerplate docker defaults" to set all the paths to /data etc. It would make updating to newer gitea container versions with added settings easier. Maybe the easiest way to achieve this would be to use environment variables only (e.g. in docker-compose.yml) and have a script in the docker container entry point re-generate app.ini from the template file on every container start, instead of on first start only.

Basically add "OR if the GITEA_RECREATE_APP_INI environment variable is set" to the if [ ! -f ${GITEA_CUSTOM}/conf/app.ini ]; in the s6 gitea setup.

Edit 2: app.ini could be created somewhere outside of /data if the environment variable is set, so we could remove app.ini from the /data volume if we use that option.

[w-flo]

This issue has now basically turned into a feature request (to handle app.ini changes in a better way in the docker container), so the label & issue template are incorrect.

I'm closing the issue, let me know if I should open a new feature request issue about the "automatic docker app.ini upgrading".

[wolfogre]

automatic docker app.ini upgrading

I don't think so, it is always dangerous to touch an in-use config file.

[wxiaoguang]

What's the root problem of this issue?

If packages are stored in /app, the packages would be lost when upgrading the docker instance?

[wolfogre]

What's the root problem of this issue?

If packages are stored in /app, the packages would be lost when upgrading the docker instance?

In the document, it's recommended to mount /data for the Gitea container, and the default app.ini for docker has APP_DATA_PATH=/data.
But some old (maybe very old) app.ini doesn't have APP_DATA_PATH, so it fallback to APP_WORK_PATH/data(/app/gitea/data).

[wxiaoguang]

But in (new) docker, the WORK_PATH should also be "/data", maybe some old docker image do not have so?

[wolfogre]

But in (new) docker, the WORK_PATH should also be "/data", maybe some old docker image do not have so?

No, the WORK_PATH is /app/gitea

The logs of a fresh instance started by gitea/gitea:1.20.0-rc2:

[wxiaoguang]

That's really a mess and pretty dangerous ...... 😭

[w-flo]

automatic docker app.ini upgrading

I don't think so, it is always dangerous to touch an in-use config file.

I agree it would be dangerous to change the file if it was user-modified. However, if the only one who creates / changes that file is the docker container entrypoint script, it should be safe, right? The only issue might be gitea itself changing the app.ini sometimes, I'm not sure if that is possible? (Avatar settings in the admin view come to mind, I've never used these checkboxes, do they change the ini?)

So it would have to be "opt-in", so that existing users would not be surprised and have their app.ini files ignored on upgrade. Also, it should say "This file was generated automatically, don't change it, set config in your docker environment variables instead!" at top of the file. And it shouldn't be stored in /data when the user has opted into that way of managing the config, maybe simply in in /app, so that the user wouldn't see that file when browsing the docker volume. The app.ini file would become an "implementation detail" of gitea in the container, nothing the user needs to look at or change.

So the user basically adds something like this to the docker environment variables (docker-compose.yml in this case):

    environment:
      - USER_UID=1000
      - USER_GID=1000
      - GITEA_REWRITE_APP_INI=true
      - GITEA__security__SECRET_KEY=abcdefgh
      - GITEA__database__DB_TYPE=postgres
      - [....]

The docker entrypoint script would check if "GITEA_REWRITE_APP_INI" is set to "true", and in that case it would always create a new /app/gitea/app.ini based on the current template file from the new docker image and use that when starting gitea. I think that would be a pretty nice way to set up the config when using docker.

For setups that don't set "GITEA_REWRITE_APP_INI", everything stays the way it is and their app.ini files will eventually become out of date (which is quite honestly a problem caused by the gitea user, I should have checked changes to app.ini in the years since I have started using gitea!).

Another important setting that I was missing in my app.ini because it was created > 5 years ago is LOG_SQL = false, so my logs were quite huge with lots of SQL statements.