Bibtex citekey has non-ASCII letters

[houshuang]

JabRef version 5.0, MacOS 10.14.16

• Mandatory: I have tested the latest development version from http://builds.jabref.org/master/ and the problem persists

I added a citation, and autogenerated a cite key, it contains a non-ASCII letter.

According to a 7-year old issue:

The key generator handles accented and other non-ASCII characters by replacement with ASCII characters deemed equivalent. If you have issues where this is not working properly, we'd be happy to look into it.
https://sourceforge.net/p/jabref/bugs/155/

@InProceedings{bakki2018modèle,
  author = {Aïcha Bakki},
  date   = {2018},
  title  = {Modèle et outil pour soutenir la scénarisation pédagogique de MOOC connectivistes},
  file   = {:/Users/stian/Downloads/Thèse_BAKKI_AICHA_Version_F.pdf:PDF},
  year   = {2018},
}

[AEgit]

Just a quick question: Have you actually tested the issue using the current dev version? You have ticked the field, but the version you report above (5.0) does not represent the current latest development version (see https://builds.jabref.org/master/).

If you have not used the latest development version, always remember to make a backup of your JabRef database, before installing the dev version.

If you have used the latest dev version, you can ignore this post.

[houshuang]

5.0 is my installed version but I did download and test the latest development version today.

On Thu, 4 Jun 2020 at 17:39, AEgit ***@***.***> wrote: Just a quick question: Have you actually tested the issue using the current dev version? You have ticked the field, but the version you report above (5.0) does not represent the current latest development version (see https://builds.jabref.org/master/). If you have not used the latest development version, always remember to make a backup of your JabRef database, before installing the dev version. If you have used the latest dev version, you can ignore this post. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#6583 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAPBB54TSJNC5FNOS5SDYTRU654HANCNFSM4NSNXRRQ> .

-- https://networkedthought.substack.com <http://networkedthought.substack.com> -- Networked Thoughts and Learning newsletter

[AEgit]

I reckon, the behaviour in the latest dev version is the same, isn't it?

[houshuang]

Indeed

On Thu, 4 Jun 2020 at 18:30, AEgit ***@***.***> wrote: I reckon, the behaviour in the latest dev version is the same, isn't it? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#6583 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAPBB3ES5KY4ONYLHM3GY3RU7D2FANCNFSM4NSNXRRQ> .

-- https://networkedthought.substack.com <http://networkedthought.substack.com> -- Networked Thoughts and Learning newsletter

[AEgit]

Cheers, ok, so this is something that needs to be fixed then.

[Siedlerchr]

Hmm. Maybe the Characters are not in the unwanted chars list in the preferences

[tobiasdiez]

Maybe the simplest solution would be to create new formatter that converts everything to ASCII (and maybe activate it by default).

jabref/src/main/java/org/jabref/logic/formatter/Formatters.java

Lines 43 to 50 in 7cc5747

	public static List<Formatter> getConverters() {
	return Arrays.asList(
	new HtmlToLatexFormatter(),
	new HtmlToUnicodeFormatter(),
	new LatexToUnicodeFormatter(),
	new UnicodeToLatexFormatter()
	);
	}

[Siedlerchr]

Didn't we recently had the change to the illegal and unwanted characters? @koppor

[tobiasdiez]

Yes we have, but I don't think unicode characters are illegal (at least not for biblatex). Moreover, illegal characters would be simply removed, but here we would like to replace them by their asci equivalent.

[k3KAW8Pnf7mkmdSMPHz27]

'è' and 'é' are legal characters but they should be replaced by 'e' (because of bibtex).

Normalizing with NFC fixes the issue for me.

I agree with @tobiasdiez that there should be a formatter, as it would allow users to remove Unicode from file/dir names if they want to (since it will be included by default among modifiers, so [title:UnicodeTo...] is valid). It can be done by wrapping StringUtils#replaceSpecialCharacters.

Should I take a look at it?

[Siedlerchr]

Thanks to @k3KAW8Pnf7mkmdSMPHz27 this should be fixed now in the latest development version.

The new version is currently building and approx ready in half an hour

We would like to ask you to use a development build from https://builds.jabref.org/master and report back if it works for you. Please remember to make a backup of your library before trying-out this version.