Ability to determine the file a module is in without complete knowledge of the compilations module dependency graph

[leecannon]

Zig Version

0.14.0-dev.985+cf87a1a7c

Steps to Reproduce and Observed Behavior

Within a single compilation multiple modules can have the same name, the build system de-duplicates these by appending a number to the name.

But this means you cannot determine which module a file is actually in, as the @src().module string depends on both if that module has a name collision with another and also on the order the imports were added, which with the package manager and transitive dependencies is a challenge :).

// build.zig
pub fn build(b: *std.Build) void {
    const exe = b.addExecutable(.{
        .name = "fff",
        .root_source_file = b.path("src/main.zig"),
        .target = b.standardTargetOptions(.{}),
        .optimize = b.standardOptimizeOption(.{}),
    });

    const dep1 = b.createModule(.{
        .root_source_file = b.path("dep1/main.zig"),
    });
    exe.root_module.addImport("dep", dep1);

    const dep2 = b.createModule(.{
        .root_source_file = b.path("dep2/main.zig"),
    });
    dep1.addImport("dep", dep2);

    const run_cmd = b.addRunArtifact(exe);
    const run_step = b.step("run", "Run the app");
    run_step.dependOn(&run_cmd.step);
}

// src/main.zig
pub fn main() void {
    std.debug.print("{s} - {s}\n", .{ @src().module, @src().file });
    @import("dep").printSrcFile(); // this "dep" is dep1/main.zig
}

// dep1/main.zig
pub fn printSrcFile() void {
    std.debug.print("{s} - {s}\n", .{ @src().module, @src().file });
    @import("dep").printSrcFile(); // this "dep" is dep2/main.zig
}

// dep2/main.zig
pub fn printSrcFile() void {
    std.debug.print("{s} - {s}\n", .{ @src().module, @src().file });
}

$ zig build run
root - main.zig
dep - main.zig
dep0 - main.zig

Expected Behavior

Ability to determine the module a file is in without complete knowledge of the compilations module dependency graph.

[mlugg]

The issue title is a little inaccurate -- the module names are definitely unique, and your example shows that -- but I think your issue text is presenting the problem correctly.

One possible solution here would be to provide a @moduleName builtin which takes in the name of a dependency and gives back the name reported by @src, or something along those lines. Or perhaps, to avoid language-level significance being given to these typically-generated CLI names, we should make the module field a numeric ID (e.g. a hash of the name), and have a corresponding @moduleId builtin.

Before thinking too hard about solutions, though: is there a concrete use case here? What are you actually trying to use @src for?

[leecannon]

If I can't determine the file path on disk (relative to build.zig) from @src() then every usage of it I have no longer works.

So I was thinking about a module name to module base path lookup but then realised the module string can't be relied upon without knowledge of the full module graph.

[mnemnion]

I've tried building Zig a few times today (no luck), specifically to try the new @src().module and see if it addresses my use case. However, the output I'm seeing from @leecannon's test script would not do so.

Here's the line in ohsnap which uses the Zig 0.13 version of source_location.file to open the file in question. Because I didn't write that line, and I'm a stickler for that sort of thing, here's the original in TigerBeetle.

For my library to keep working, I need some reliable way to go from the output of @src(), to a path to the file in question. This seems like a reasonable, even basic, use for @src(): to find the source in question. There are a small handful of reasons someone might use that builtin, and this is one of them.

The simplest solution to this dilemma is the first one I proposed: save the full path-relative file name to .rodata, and have .file_path and .file, which both point to different parts of it. So in the example in the first post, for src.main.zig, @src().file would be "main.zig", and @src().file_path would be "src/main.zig". The slice @src.file_path[4..] would be identical, .ptr and .len, to the slice @src().file.

If it's also useful to the build system to have a .module field which provides some sort of unique name which is independent of the file path, that's certainly fine by me, I don't happen to need it but that's irrelevant. What I do need is a way to use @src() or something equivalent to it, to find the file itself, and the line where @src() was called, so that I can update snapshots on request.

[andrewrk]

I've tried building Zig a few times today (no luck)

The download page has the change now.

[mnemnion]

Thanks for the update. My problem was down to a silly typo in LDFLAGS, Zig is building just fine now.

So with a nice comptime conditional, I added a logging step to ohsnap, running it on itself I get this from master branch:

module: root
file: ohsnap.zig

The output from 0.13 is this:

module:
file: src/ohsnap.zig

So the remaining difficulty is that I have no way, so far as I know, of mapping the module name "root" to the directory "./src/", which is what I need to have to be able to open ./src/ohsnap.zig.

The difficulty is easy to illustrate: I made a copy of ohsnap.zig in a directory called /test, and changed a line in the build to read

        .root_source_file = b.path("test/ohsnap.zig"),

As expected, this new file builds and runs correctly, and with Zig master produces this output:

module: root
file: ohsnap.zig

On Zig 0.13:

module:
file: test/ohsnap.zig

So I can't simply guess that the directory of the module root is going to be /src, since it can be anywhere.

I'm sure that there are a few ways to handle this, my use case requires some way to go from the output of @src() to opening the file from which @src() is called. Ideally, that way is relatively simple, since I do anticipate that user code using @src() will frequently want to open the associated file, whether to read it or modify it. Even for the other use case, which would be some sort of logging or alert which uses the SourceLocation struct, I would hazard that the current use of .module would itself be somewhat confusing, since my build.zig doesn't use the word root at any point. It does have a .root_source_file field, but it strikes me that a user seeing an error with "root" in it will not necessarily or immediately make the connection.

Be that as it may, so long as there is some way to get from the output of @src() to opening the file in which it was called, my use case is satisfied.

[mnemnion]

I'd like to encourage some design work on this issue, it's important to me that it get solved before the 0.14 release.

Background: the ohsnap library, based on TigerBeetle's snapshot testing (which will have the same problem) uses @src() in 0.13 to open the test file and update the snapshots. Adding the module name to SourceLocation solves other problems, but not this one.

The goal is for user code to be able to call @src(), and for library code to, in some fashion, be able to use that data to open the source file. An ideal solution for this use case requires no changes to user code, I'm willing to do whatever is necessary in the library to make this work.

A less ideal, but acceptable solution, would involve user code providing some sort of data available to user code but not library code, perhaps something from builtin. If there's already a way to do this, I haven't figured it out, and I've tried.

What does a good solution to this look like? I admit ignorance in why the change was made in the first place, while assuming that the reasons are good (I could make some guesses but that doesn't seem important). I'm hoping, since this issue remains open, that the goal doesn't include making it impossible for user code to communicate its full source location to libraries, since that's a useful and powerful thing to be able to do, and, I would guess, the main use of @src() outside of the Zig compiler.

Again, if there's already a way forward and I missed it, great! Point me at it and we can close this issue. Thanks for your consideration in this matter.