Skip to content

Shared: Add library for filepath normalization #14500

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Oct 23, 2023
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
| | . |
| ./a/b/c/../d | a/b/d |
| / | / |
| /a/b/../c | /a/c |
| /a/b/c/../../d/ | /a/d |
| a/.. | . |
| a/b/../c | a/c |
| a/b//////c/./d/../e//d// | a/b/c/e/d |
| a/b/c | a/b/c |
| a/b/c/../../../../d/e/../f | ../d/f |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be good to add a similar test but with an absolute path (e.g. /a/b/c/../../../../d/e/../f should resolve to /d/f.

14 changes: 14 additions & 0 deletions csharp/ql/test/library-tests/utils/FilepathNormalizeTest.ql
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
import codeql.util.FilePath

class FilepathTest extends NormalizableFilepath {
FilepathTest() {
this =
[
"a/b/c", "a/b/../c", "a/..", "/a/b/../c", "a/b/c/../../../../d/e/../f", "", "/",
"./a/b/c/../d", "a/b//////c/./d/../e//d//", "/a/b/c/../../d/"
]
}
}

from FilepathTest s
select s, s.getNormalizedPath()
4 changes: 4 additions & 0 deletions shared/util/change-notes/2023-10-13-filepath-normalization.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
category: feature
---
* Added `FilePath` API for normalizing filepaths.
```
74 changes: 74 additions & 0 deletions shared/util/codeql/util/FilePath.qll
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
/** Provides a utility for normalizing filepaths. */

/**
* A filepath that should be normalized.
* Extend to provide additional strings that should be normalized as filepaths.
*/
abstract class NormalizableFilepath extends string {
bindingset[this]
NormalizableFilepath() { any() }

private string getComponent(int i) { result = this.splitAt("/", i) }

private int getNumComponents() { result = strictcount(int i | exists(this.getComponent(i))) }

/** count .. segments in prefix in normalization from index i */
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see you simply copied my algorithm suggestion verbatim, but these qldocs could be a bit nicer now that the code is moving from a slack thread and into actual main.

private int dotdotCountFrom(int i) {
result = 0 and i = this.getNumComponents()
or
exists(string c | c = this.getComponent(i) |
if c = ""
then result = this.dotdotCountFrom(i + 1)
else
if c = "."
then result = this.dotdotCountFrom(i + 1)
else
if c = ".."
then result = this.dotdotCountFrom(i + 1) + 1
else result = (this.dotdotCountFrom(i + 1) - 1).maximum(0)
)
}

/** count non-.. segments in postfix in normalization from index 0 to i-1 */
private int segmentCountUntil(int i) {
result = 0 and i = 0
or
exists(string c | c = this.getComponent(i - 1) |
if c = ""
then result = this.segmentCountUntil(i - 1)
else
if c = "."
then result = this.segmentCountUntil(i - 1)
else
if c = ".."
then result = (this.segmentCountUntil(i - 1) - 1).maximum(0)
else result = this.segmentCountUntil(i - 1) + 1
)
}

private string part(int i) {
result = this.getComponent(i) and
result != "." and
if result = ""
then i = 0
else (
result != ".." and
0 = this.dotdotCountFrom(i + 1)
or
result = ".." and
0 = this.segmentCountUntil(i)
)
}

/** Gets the normalized filepath for this string; traversing `/../` paths. */
string getNormalizedPath() {
exists(string norm | norm = concat(string s, int i | s = this.part(i) | s, "/" order by i) |
if norm != ""
then result = norm
else
if this.matches("/%")
then result = "/"
else result = "."
)
}
}