Skip to content

Commit 861a6ce

Browse files
hitalo-devkoppor
hitalo-dev
and
koppor
authored
Sanitize URLs in file fields to handle invalid pipe characters ('|') (#12156)
* Sanitize URLs in file fields to handle invalid pipe characters ('|') Closes #11876. - Introduced URLUtil.createUri() and URLUtil.create() to handle URL sanitization. - Replaced direct calls to URI.create() and URI.create().toURL() with the new utility methods. - URLs containing the pipe character ('|') are now properly encoded as '%7C' to prevent parsing errors. - Added test cases to URLUtilTest to verify correct sanitization and URL creation. * Sanitize URLs in file fields to handle invalid pipe characters ('|') Closes #11876. - Introduced URLUtil.createUri() and URLUtil.create() to handle URL sanitization. - Replaced direct calls to URI.create() and URI.create().toURL() with the new utility methods. - URLs containing the pipe character ('|') are now properly encoded as '%7C' to prevent parsing errors. - Added test cases to URLUtilTest to verify correct sanitization and URL creation. - Added @archtest to ensure that the URI.create() method is not directly called in the codebase. * Discard changes to src/main/java/module-info.java * Sanitize URLs in file fields to handle invalid pipe characters ('|') Closes #11876. - Introduced URLUtil.createUri() and URLUtil.create() to handle URL sanitization. - Replaced direct calls to URI.create() and URI.create().toURL() with the new utility methods. - URLs containing the pipe character ('|') are now properly encoded as '%7C' to prevent parsing errors. - Added test cases to URLUtilTest to verify correct sanitization and URL creation. - Added @archtest to ensure that the URI.create() method is not directly called in the codebase. * Sanitize URLs in file fields to handle invalid pipe characters ('|') Closes #11876. - Introduced URLUtil.createUri() and URLUtil.create() to handle URL sanitization. - Replaced direct calls to URI.create() and URI.create().toURL() with the new utility methods. - URLs containing the pipe character ('|') are now properly encoded as '%7C' to prevent parsing errors. - Added test cases to URLUtilTest to verify correct sanitization and URL creation. - Added @archtest to ensure that the URI.create() method is not directly called in the codebase. * Refine comment * Fix checkstyle * Fix import * Update Openrewrite * Move URLUtil to logic (where possible) * Fix architecture test * Fix FQN * Compilefix * Add Arch exception * Fix imports * Add CHANGELOG.md entry --------- Co-authored-by: Oliver Kopp <kopp.dev@gmail.com>
1 parent b453985 commit 861a6ce

File tree

61 files changed

+328
-246
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

61 files changed

+328
-246
lines changed

‎CHANGELOG.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -115,6 +115,7 @@ Note that this project **does not** adhere to [Semantic Versioning](https://semv
115115
- We fixed an issue where the "Do not ask again" checkbox was not working, when asking for permission to use Grobid [koppor#556](https://github.com/koppor/jabref/issues/566).
116116
- We fixed an issue where we display warning message for moving attached open files. [#10121](https://github.com/JabRef/jabref/issues/10121)
117117
- We fixed an issue where it was not possible to select selecting content of other user's comments.[#11106](https://github.com/JabRef/jabref/issues/11106)
118+
- We fixed an issue when handling URLs containing a pipe (`|`) character. [#11876](https://github.com/JabRef/jabref/issues/11876)
118119
- We fixed an issue where web search preferences "Custom API key" table modifications not discarded. [#11925](https://github.com/JabRef/jabref/issues/11925)
119120
- We fixed an issue when opening attached files in [extra file columns](https://docs.jabref.org/finding-sorting-and-cleaning-entries/filelinks#adding-additional-columns-to-entry-table-for-file-types). [#12005](https://github.com/JabRef/jabref/issues/12005)
120121
- We fixed an issue where trying to open a library from a failed mounted directory on Mac would cause an error. [#10548](https://github.com/JabRef/jabref/issues/10548)

‎build.gradle

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ plugins {
2929

3030
id 'idea'
3131

32-
id 'org.openrewrite.rewrite' version '6.27.0'
32+
id 'org.openrewrite.rewrite' version '6.27.2'
3333

3434
id "org.itsallcode.openfasttrace" version "3.0.1"
3535
}
@@ -400,7 +400,7 @@ dependencies {
400400
xjc group: 'org.glassfish.jaxb', name: 'jaxb-xjc', version: '3.0.2'
401401
xjc group: 'org.glassfish.jaxb', name: 'jaxb-runtime', version: '3.0.2'
402402

403-
rewrite(platform("org.openrewrite.recipe:rewrite-recipe-bom:2.19.0"))
403+
rewrite(platform("org.openrewrite.recipe:rewrite-recipe-bom:2.22.0"))
404404
rewrite("org.openrewrite.recipe:rewrite-static-analysis")
405405
rewrite("org.openrewrite.recipe:rewrite-logging-frameworks")
406406
rewrite("org.openrewrite.recipe:rewrite-testing-frameworks")

‎src/main/java/org/jabref/gui/entryeditor/citationrelationtab/semanticscholar/SemanticScholarFetcher.java

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,14 @@
11
package org.jabref.gui.entryeditor.citationrelationtab.semanticscholar;
22

33
import java.net.MalformedURLException;
4-
import java.net.URI;
54
import java.net.URL;
65
import java.util.List;
76

87
import org.jabref.logic.importer.FetcherException;
98
import org.jabref.logic.importer.ImporterPreferences;
109
import org.jabref.logic.importer.fetcher.CustomizableKeyFetcher;
1110
import org.jabref.logic.net.URLDownload;
11+
import org.jabref.logic.util.URLUtil;
1212
import org.jabref.model.entry.BibEntry;
1313

1414
import com.google.gson.Gson;
@@ -38,7 +38,7 @@ public List<BibEntry> searchCitedBy(BibEntry entry) throws FetcherException {
3838

3939
URL citationsUrl;
4040
try {
41-
citationsUrl = URI.create(getAPIUrl("citations", entry)).toURL();
41+
citationsUrl = URLUtil.create(getAPIUrl("citations", entry));
4242
} catch (MalformedURLException e) {
4343
throw new FetcherException("Malformed URL", e);
4444
}
@@ -62,7 +62,7 @@ public List<BibEntry> searchCiting(BibEntry entry) throws FetcherException {
6262

6363
URL referencesUrl;
6464
try {
65-
referencesUrl = URI.create(getAPIUrl("references", entry)).toURL();
65+
referencesUrl = URLUtil.create(getAPIUrl("references", entry));
6666
} catch (MalformedURLException e) {
6767
throw new FetcherException("Malformed URL", e);
6868
}

‎src/main/java/org/jabref/gui/fieldeditors/LinkedFilesEditorViewModel.java

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,6 @@
22

33
import java.io.IOException;
44
import java.net.MalformedURLException;
5-
import java.net.URI;
65
import java.net.URL;
76
import java.nio.file.Path;
87
import java.util.ArrayList;
@@ -38,6 +37,7 @@
3837
import org.jabref.logic.l10n.Localization;
3938
import org.jabref.logic.util.BackgroundTask;
4039
import org.jabref.logic.util.TaskExecutor;
40+
import org.jabref.logic.util.URLUtil;
4141
import org.jabref.logic.util.io.FileUtil;
4242
import org.jabref.model.database.BibDatabaseContext;
4343
import org.jabref.model.entry.BibEntry;
@@ -186,7 +186,7 @@ private List<LinkedFileViewModel> findAssociatedNotLinkedFiles(BibEntry entry) {
186186

187187
public boolean downloadFile(String urlText) {
188188
try {
189-
URL url = URI.create(urlText).toURL();
189+
URL url = URLUtil.create(urlText);
190190
addFromURLAndDownload(url);
191191
return true;
192192
} catch (MalformedURLException exception) {

‎src/main/java/org/jabref/gui/fieldeditors/URLUtil.java

Lines changed: 6 additions & 73 deletions
Original file line numberDiff line numberDiff line change
@@ -1,89 +1,22 @@
11
package org.jabref.gui.fieldeditors;
22

33
import java.net.MalformedURLException;
4-
import java.net.URI;
54
import java.net.URL;
6-
import java.net.URLDecoder;
7-
import java.nio.charset.StandardCharsets;
8-
import java.util.Objects;
95
import java.util.Optional;
106

117
import org.jabref.gui.externalfiletype.ExternalFileTypes;
128
import org.jabref.gui.frame.ExternalApplicationsPreferences;
139

10+
/**
11+
* URL utilities for URLs in the JabRef GUI.
12+
* <p>
13+
* For logic-oriented URL utilities see {@link org.jabref.logic.util.URLUtil}.
14+
*/
1415
public class URLUtil {
15-
private static final String URL_EXP = "^(https?|ftp)://.+";
16-
17-
// Detect Google search URL
18-
private static final String GOOGLE_SEARCH_EXP = "^https?://(?:www\\.)?google\\.[\\.a-z]+?/url.*";
1916

2017
private URLUtil() {
2118
}
2219

23-
/**
24-
* Cleans URLs returned by Google search.
25-
*
26-
* <example>
27-
* If you copy links from search results from Google, all links will be enriched with search meta data, e.g.
28-
* https://www.google.de/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&uact=8&&url=http%3A%2F%2Fwww.inrg.csie.ntu.edu.tw%2Falgorithm2014%2Fhomework%2FWagner-74.pdf&ei=DifeVYHkDYWqU5W0j6gD&usg=AFQjCNFl638rl5KVta1jIMWLyb4CPSZidg&sig2=0hSSMw9XZXL3HJWwEcJtOg
29-
* </example>
30-
*
31-
* @param url the Google search URL string
32-
* @return the cleaned Google URL or @code{url} if no search URL was detected
33-
*/
34-
public static String cleanGoogleSearchURL(String url) {
35-
Objects.requireNonNull(url);
36-
37-
if (!url.matches(GOOGLE_SEARCH_EXP)) {
38-
return url;
39-
}
40-
// Extract destination URL
41-
try {
42-
URL searchURL = URI.create(url).toURL();
43-
// URL parameters
44-
String query = searchURL.getQuery();
45-
// no parameters
46-
if (query == null) {
47-
return url;
48-
}
49-
// extract url parameter
50-
String[] pairs = query.split("&");
51-
52-
for (String pair : pairs) {
53-
// "clean" url is decoded value of "url" parameter
54-
if (pair.startsWith("url=")) {
55-
String value = pair.substring(pair.indexOf('=') + 1);
56-
57-
String decode = URLDecoder.decode(value, StandardCharsets.UTF_8);
58-
// url?
59-
if (decode.matches(URL_EXP)) {
60-
return decode;
61-
}
62-
}
63-
}
64-
return url;
65-
} catch (MalformedURLException e) {
66-
return url;
67-
}
68-
}
69-
70-
/**
71-
* Checks whether the given String is a URL.
72-
* <p>
73-
* Currently only checks for a protocol String.
74-
*
75-
* @param url the String to check for a URL
76-
* @return true if <c>url</c> contains a valid URL
77-
*/
78-
public static boolean isURL(String url) {
79-
try {
80-
URI.create(url).toURL();
81-
return true;
82-
} catch (MalformedURLException | IllegalArgumentException e) {
83-
return false;
84-
}
85-
}
86-
8720
/**
8821
* Look for the last '.' in the link, and return the following characters.
8922
* <p>
@@ -96,7 +29,7 @@ public static Optional<String> getSuffix(final String link, ExternalApplications
9629
String strippedLink = link;
9730
try {
9831
// Try to strip the query string, if any, to get the correct suffix:
99-
URL url = URI.create(link).toURL();
32+
URL url = org.jabref.logic.util.URLUtil.create(link);
10033
if ((url.getQuery() != null) && (url.getQuery().length() < (link.length() - 1))) {
10134
strippedLink = link.substring(0, link.length() - url.getQuery().length() - 1);
10235
}

‎src/main/java/org/jabref/gui/fieldeditors/UrlEditorViewModel.java

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@
1313
import org.jabref.gui.preferences.GuiPreferences;
1414
import org.jabref.logic.integrity.FieldCheckers;
1515
import org.jabref.logic.l10n.Localization;
16+
import org.jabref.logic.util.URLUtil;
1617
import org.jabref.model.entry.field.Field;
1718
import org.jabref.model.strings.StringUtil;
1819

‎src/main/java/org/jabref/gui/linkedfile/AttachFileFromURLAction.java

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,6 @@
11
package org.jabref.gui.linkedfile;
22

33
import java.net.MalformedURLException;
4-
import java.net.URI;
54
import java.net.URL;
65
import java.util.Optional;
76

@@ -14,6 +13,7 @@
1413
import org.jabref.gui.preferences.GuiPreferences;
1514
import org.jabref.logic.l10n.Localization;
1615
import org.jabref.logic.util.TaskExecutor;
16+
import org.jabref.logic.util.URLUtil;
1717
import org.jabref.model.database.BibDatabaseContext;
1818
import org.jabref.model.entry.BibEntry;
1919
import org.jabref.model.entry.LinkedFile;
@@ -61,7 +61,7 @@ public void execute() {
6161
}
6262

6363
try {
64-
URL url = URI.create(urlforDownload.get()).toURL();
64+
URL url = URLUtil.create(urlforDownload.get());
6565
LinkedFileViewModel onlineFile = new LinkedFileViewModel(
6666
new LinkedFile(url, ""),
6767
entry,

‎src/main/java/org/jabref/gui/linkedfile/LinkedFileEditDialogViewModel.java

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,6 @@
22

33
import java.io.IOException;
44
import java.net.MalformedURLException;
5-
import java.net.URI;
65
import java.nio.file.Files;
76
import java.nio.file.Path;
87
import java.util.Optional;
@@ -25,6 +24,7 @@
2524
import org.jabref.gui.util.FileDialogConfiguration;
2625
import org.jabref.logic.FilePreferences;
2726
import org.jabref.logic.l10n.Localization;
27+
import org.jabref.logic.util.URLUtil;
2828
import org.jabref.logic.util.io.FileNameCleaner;
2929
import org.jabref.logic.util.io.FileUtil;
3030
import org.jabref.model.database.BibDatabaseContext;
@@ -166,7 +166,7 @@ public LinkedFile getNewLinkedFile() {
166166

167167
if (LinkedFile.isOnlineLink(link.getValue())) {
168168
try {
169-
return new LinkedFile(description.getValue(), URI.create(link.getValue()).toURL(), fileType, sourceUrl.getValue());
169+
return new LinkedFile(description.getValue(), URLUtil.create(link.getValue()), fileType, sourceUrl.getValue());
170170
} catch (MalformedURLException e) {
171171
return new LinkedFile(description.getValue(), link.getValue(), fileType, sourceUrl.getValue());
172172
}

‎src/main/java/org/jabref/gui/mergeentries/newmergedialog/cell/FieldValueCell.java

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,10 +20,10 @@
2020
import javafx.scene.paint.Color;
2121

2222
import org.jabref.gui.actions.ActionFactory;
23-
import org.jabref.gui.fieldeditors.URLUtil;
2423
import org.jabref.gui.icon.IconTheme;
2524
import org.jabref.gui.preferences.GuiPreferences;
2625
import org.jabref.logic.l10n.Localization;
26+
import org.jabref.logic.util.URLUtil;
2727
import org.jabref.model.entry.identifier.DOI;
2828
import org.jabref.model.strings.StringUtil;
2929

‎src/main/java/org/jabref/gui/theme/StyleSheetFile.java

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,6 @@
22

33
import java.io.IOException;
44
import java.io.InputStream;
5-
import java.net.URI;
65
import java.net.URL;
76
import java.net.URLConnection;
87
import java.nio.file.Files;
@@ -11,6 +10,8 @@
1110
import java.util.Optional;
1211
import java.util.concurrent.atomic.AtomicReference;
1312

13+
import org.jabref.logic.util.URLUtil;
14+
1415
import com.google.common.base.Strings;
1516
import org.slf4j.Logger;
1617
import org.slf4j.LoggerFactory;
@@ -54,7 +55,7 @@ final class StyleSheetFile extends StyleSheet {
5455

5556
StyleSheetFile(URL url) {
5657
this.url = url;
57-
this.path = Path.of(URI.create(url.toExternalForm()));
58+
this.path = Path.of(URLUtil.createUri(url.toExternalForm()));
5859
reload();
5960
}
6061

0 commit comments

Comments
 (0)