Lua filters: add make_relative_path to pandoc.system module

[cole-miller]

@tarleb suggested that I open a feature request here to add a make_relative_path function to the pandoc.system Lua submodule, after I implemented something similar for a filter. The prototype would be something like

function make_relative_path(to, from)
  -- `to', `from' are paths, passed as strings
  -- returns a path resolving to the same location as `to' and expressed
  -- relative to `from'
end

I used realpath from the GNU coreutils to do this, but it could be accomplished more portably in the parent module.

My use-case for this is that I had HTML output in mind for my filter, and I wanted filter-generated files to be linked in the output document with relative paths.

ETA: I'm happy to work on implementing this if it's judged a good idea.

ETA again: actually, since it looks like all the existing pandoc.system Lua functions are borrowed from hslua-module-system, perhaps it makes sense to do the same here?

[tarleb]

There is an interesting difficulty to this: The makeRelative function provided by the filepath package will never return .. paths, as that might break when used with symlinks. These kind of paths are frequently used in HTML. @cole-miller, would that still be helpful?

See the make-relative-path branch in https://github.com/hslua/hslua-module-system for an (unfinished) implementation using that function.

[cole-miller]

Pardon me for ignoring this for the last few weeks.

I think the function would be substantially less useful without the ability to produce .. paths. Maybe the best solution is to include a second function for producing relative paths in the absence of symlinks (Neil Mitchell's blog post linked in that documentation describes such a function in the Shake source code), and let the user choose between the two?

[gabyx]

I did not introduce makeRelative in #6565, because its not similar at all to e.g. os.path.relpath for example. Could we provide the function similar to this implementation, which does not use the filesystem, meaning it does not resolve symlinks. I think this behavior is good for pandoc filter code where paths need to be modified.

I think one should never resolve symlinks while processing paths in markdown. Symlinks are on filesystem level to help the operating system user.

Also, when we write markdown we have almost in all cases a base folder projectDir, and we should anyway never have paths which point to files/dirs before projectDir. I am questioning also why in lilypon.lua make_relative actually uses realpath. Is it really needed? @cole-miller Could you elaborate on this a little bit?

[cole-miller]

@gabyx Since there's some additional complexity that I didn't account for in the original issue I think it's reasonable to omit this from pandoc.system, so I'll close the issue.

As for the original use-case, I wanted pandoc to generate HTML with relativized links for a web site, so that I could relocate the site root in the filesystem without updating a bunch of pages with the changed absolute paths.

[tarleb]

I think you are describing a common usecase, so I'd like to keep this open.

[gabyx]

Is there any progress :). Would it be possible to integrate at least #6565
such that we have basic file path manipulation functionality?

[tarleb]

@jgm voiced the concern that too many Lua modules will prove unmaintainable, and I agree. From my point of view, this is what needs to be done:

1. Agree that a pandoc.paths module would be good and welcome.
2. Finish the hslua-module-paths package; it is missing proper tests and good documentation, among other things.
3. Improve documentation and tooling around self-documented Lua modules.
4. Develop tools and methods to keep module docs in pandoc's Lua documentation up-to-date.
5. Bring hslua-module-paths into pandoc.
6. Extend hslua-module-paths with the desired make_relative function.

The first point is up to @jgm. The second, third, and probably fourth points are all mine, but honestly not very high on my list. Realistically, this won't see progress anytime soon, unless the mentioned projects all suddenly experience a flood of high-quality pull requests.