Search to accents not working with sqlite

[neszt]

Gitea Version

1.15.10

Git Version

git version 2.30.2

Operating System

Alpine Linux 3.13.7

How are you running Gitea?

Official docker image deployed with docker-compose:

$ cat docker-compose.yml
version: "3.3"
services:
  server:
    restart: always
    image: gitea/gitea:${GITEA_VERSION}
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /usr/share/zoneinfo:/usr/share/zoneinfo:ro
      - ${DATA}:/data
      - ${ENTRYPOINT}:/usr/bin/entrypoint
    ports:
      - "${HTTP_PORT}:3000"
      - "${SSH_PORT}:22"

$ cat .env
GITEA_VERSION=1.15.10
HTTP_PORT=3000
SSH_PORT=2222
DATA=./data
ENTRYPOINT=./custom/entrypoint

Database

SQLite

Can you reproduce the bug on the Gitea demo site?

No

Log Gist

No response

Description

I use a custom indexer setting to search for word fragments as well.

bash-5.1# tail -2 /data/gitea/conf/app.ini
[indexer]
ISSUE_INDEXER_TYPE = db

But unfortunately, accented search doesn't work without this option. (neither in the title nor in the content)

I have another gitea system installed with a postgesql database engine, the accented search works correctly there, so i think sqlite is the cause of the malfunction.

Is there any sqlite specific language option I should set?

Screenshots

Without search:

With accent search:

With accent free search:

[Gusted]

Hmm, SQLite is here wrong it seems. We are currently upper casing all characters of the keyword(to avoid case sensitive searches). Go is handling uppering accent characters: https://go.dev/play/p/MBiTQq-3mAl, however SQLite only supports uppering ASCII characters:

And thus these 2 will never match each other... I think the best option is to have a special ToUpper function for SQLite that only uppers ASCII strings and thus match SQLite's specs.

[zeripath]

Likely we should be letting the DB do the UPPER() and not expecting the go collation to be the same as the db.

(I would also note that we should probably be considering unicode normalizing the strings...

For example, ë and ë are not the same character.)

[Gusted]

@zeripath Unless I'm misunderstanding something, but it doesn't seem like we can use the UPPER function for provided values(as in, the values for ?):