Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v0.9.0 fails to build with package managers #236

Closed
Builditluc opened this issue Dec 4, 2024 · 46 comments
Closed

v0.9.0 fails to build with package managers #236

Builditluc opened this issue Dec 4, 2024 · 46 comments

Comments

@Builditluc
Copy link
Owner

This issue tracks the fixing of a bug where wiki-api won't compile with package managers (cargo install --locked works just fine).

@Builditluc
Copy link
Owner Author

@0323pin if I push a commit, can you test if the patch fixes the issue without creating a release?

@0323pin
Copy link
Contributor

0323pin commented Dec 4, 2024

My last build from git-HEAD pulling from #cf510cf33c2d0a8e68d6f9df4721aed86a1a2bfc worked just fine.

By default, all NetBSD builds are done in --locked mode as it's a requirement to build offline.

if I push a commit, can you test if the patch fixes the issue without creating a release?

Yes, of course.

@Builditluc
Copy link
Owner Author

My last build from git-HEAD pulling from #cf510cf33c2d0a8e68d6f9df4721aed86a1a2bfc worked just fine.

That's what I'm not understanding, all of my builds (local and with cargo install / cargo publish) work just fine, its just when I try to package the new version the build fails

@0323pin
Copy link
Contributor

0323pin commented Dec 4, 2024

Give a minute, I want to test something ...

@Builditluc
Copy link
Owner Author

I tested using the CARGO_MANIFEST_DIR env for finding the path to the languages.json file. Pushed this to fix-proc-macro and it seems to work fine when trying to compile it for nixpkgs. Does it work for you @0323pin?

@0323pin
Copy link
Contributor

0323pin commented Dec 4, 2024

Even when I'm building from specific git commits, I'm actually building in package mode but pulling from a specific commit tag.

If I do this using 48bb060 as tag, the same tag as the release, the build fails.

But, if I use the commit just prior to that, e882bbc the build finishes just fine

This might be what is causing it,
2024-12-04-214246_1366x768_scrot

@0323pin
Copy link
Contributor

0323pin commented Dec 4, 2024

Hope this helps!

@Builditluc
Copy link
Owner Author

Well, I needed to add that portion because cargo publish failed otherwise

@nunotexbsd
Copy link
Contributor

Experimenting error on FreeBSD.
Is this same issue?

Thanks

 Compiling wiki-api v0.1.0 (/wrkdirs/usr/ports/www/wiki-tui/work/wiki-tui-0.9.0/wiki-api)
     Running `CARGO=/usr/local/bin/cargo CARGO_CRATE_NAME=wiki_api CARGO_MANIFEST_DIR=/wrkdirs/usr/ports/www/wiki-tui/work/wiki-tui-0.9.0/wiki-api CARGO_MANIFEST_PATH=/wrkdirs/usr/ports/www/wiki-tui/work/wiki-tui
-0.9.0/wiki-api/Cargo.toml CARGO_PKG_AUTHORS='builditluc <37375448+Builditluc@users.noreply.github.com>' CARGO_PKG_DESCRIPTION='Backend for wiki-tui' CARGO_PKG_HOMEPAGE='https://wiki-tui.net' CARGO_PKG_LICENSE=MI
T CARGO_PKG_LICENSE_FILE='' CARGO_PKG_NAME=wiki-api CARGO_PKG_README='' CARGO_PKG_REPOSITORY='https://github.com/builditluc/wiki-tui' CARGO_PKG_RUST_VERSION='' CARGO_PKG_VERSION=0.1.0 CARGO_PKG_VERSION_MAJOR=0 CA
RGO_PKG_VERSION_MINOR=1 CARGO_PKG_VERSION_PATCH=0 CARGO_PKG_VERSION_PRE='' LD_LIBRARY_PATH=/wrkdirs/usr/ports/www/wiki-tui/work/target/release/deps /usr/local/bin/rustc --crate-name wiki_api --edition=2018 wiki-a
pi/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=2 -C panic=abort -C linker-plugin-lto -C codegen-units=1 --c
heck-cfg 'cfg(docsrs)' --check-cfg 'cfg(feature, values())' -C metadata=0331d7ff9f536fb5 -C extra-filename=-0331d7ff9f536fb5 --out-dir /wrkdirs/usr/ports/www/wiki-tui/work/target/release/deps -C strip=symbols -L
dependency=/wrkdirs/usr/ports/www/wiki-tui/work/target/release/deps --extern anyhow=/wrkdirs/usr/ports/www/wiki-tui/work/target/release/deps/libanyhow-33e3d6d6c15bbd8b.rmeta --extern bitflags=/wrkdirs/usr/ports/w
ww/wiki-tui/work/target/release/deps/libbitflags-6458776dbf57fced.rmeta --extern ego_tree=/wrkdirs/usr/ports/www/wiki-tui/work/target/release/deps/libego_tree-d271842ed2185580.rmeta --extern html5ever=/wrkdirs/us
r/ports/www/wiki-tui/work/target/release/deps/libhtml5ever-95160e05add8b9ee.rmeta --extern markup5ever_rcdom=/wrkdirs/usr/ports/www/wiki-tui/work/target/release/deps/libmarkup5ever_rcdom-7c942dc687d965ef.rmeta --
extern reqwest=/wrkdirs/usr/ports/www/wiki-tui/work/target/release/deps/libreqwest-5cc8e1e791f51574.rmeta --extern scraper=/wrkdirs/usr/ports/www/wiki-tui/work/target/release/deps/libscraper-d1276c11be8d064c.rmet
a --extern serde=/wrkdirs/usr/ports/www/wiki-tui/work/target/release/deps/libserde-ae961a0aea7ab481.rmeta --extern serde_json=/wrkdirs/usr/ports/www/wiki-tui/work/target/release/deps/libserde_json-eb49a61a7b85d85
b.rmeta --extern serde_repr=/wrkdirs/usr/ports/www/wiki-tui/work/target/release/deps/libserde_repr-314fcbef436734cf.so --extern snafu=/wrkdirs/usr/ports/www/wiki-tui/work/target/release/deps/libsnafu-3c33c163a426
dcac.rmeta --extern tracing=/wrkdirs/usr/ports/www/wiki-tui/work/target/release/deps/libtracing-61641ec67bb0f836.rmeta --extern url=/wrkdirs/usr/ports/www/wiki-tui/work/target/release/deps/liburl-bfd31c44ec95d654
.rmeta --extern urlencoding=/wrkdirs/usr/ports/www/wiki-tui/work/target/release/deps/liburlencoding-652015d40ec27615.rmeta --extern wiki_api_macros=/wrkdirs/usr/ports/www/wiki-tui/work/target/release/deps/libwiki
_api_macros-33b263dcbf590f30.so -C link-arg=-fstack-protector-strong -L native=/usr/lib`
error: proc macro panicked
 --> wiki-api/src/languages.rs:3:1
  |
3 | parse_languages!("./data/languages.json");
  | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  |
  = help: message: called `Result::unwrap()` on an `Err` value: Os { code: 2, kind: NotFound, message: "No such file or directory" }

error[E0432]: unresolved import `super::languages::Language`
  --> wiki-api/src/page.rs:15:5
   |
15 | use super::languages::Language;
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^ no `Language` in `languages`

error[E0432]: unresolved import `crate::languages::Language`
  --> wiki-api/src/page.rs:18:17
   |
18 |     use crate::{languages::Language, search::Namespace, Endpoint};
   |                 ^^^^^^^^^^^^^^^^^^^ no `Language` in `languages`
   |
   = help: consider importing this unresolved item through its public re-export instead:
           crate::page::Language

error[E0432]: unresolved import `crate::languages::Language`
 --> wiki-api/src/parser.rs:9:5
  |
9 |     languages::Language,
  |     ^^^^^^^^^^^^^^^^^^^ no `Language` in `languages`

error[E0432]: unresolved import `crate::languages::Language`
  --> wiki-api/src/search.rs:14:5
   |
14 | use crate::languages::Language;
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^ no `Language` in `languages`

For more information about this error, try `rustc --explain E0432`.

@Builditluc
Copy link
Owner Author

Yep, it is. Still unsure on how to fix it

@0323pin
Copy link
Contributor

0323pin commented Dec 4, 2024

Well, I needed to add that portion because cargo publish failed otherwise

I don't get it, let me try something.

@0323pin
Copy link
Contributor

0323pin commented Dec 4, 2024

@Builditluc I can cut a release without issues, https://github.com/0323pin/wiki-tui/releases/tag/0.9.0

@Builditluc
Copy link
Owner Author

Builditluc commented Dec 4, 2024

And cargo publish -p wiki-api --dry-run works?

EDIT: because applying the same patch as you did (reverting the path back to wiki-api/data/languages.json) causes the same error message I had when trying to publish in the first place

$ cargo publish -p wiki-api --dry-run --allow-dirty
....
   Compiling wiki-api v0.1.0 (/home/builditluc/dev/wiki-tui/target/package/wiki-api-0.1.0)
error: proc macro panicked
 --> src/languages.rs:3:1
  |
3 | parse_languages!("wiki-api/data/languages.json");
  | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  |
  = help: message: called `Result::unwrap()` on an `Err` value: Os { code: 2, kind: NotFound, message: "No such file or directory" }

@Builditluc
Copy link
Owner Author

Builditluc commented Dec 4, 2024

@nunotexbsd in your log, I see the CARGO_MANIFEST_DIR ist set to /wrkdirs/usr/ports/www/wiki-tui/work/wiki-tui-0.9.0/wiki-api.

If I modify the macro to use the CARGO_MANIFEST_DIR env and join the input (data/languages.json), the build should compile correctly.

I pushed a commit doing this to fix-proc-macro

@0323pin
Copy link
Contributor

0323pin commented Dec 4, 2024

And cargo publish -p wiki-api --dry-run works?

No, I get the same as you.

@Builditluc
Copy link
Owner Author

To me the only logical conclusion would be that the $PWD is set to something different when using cargo publish and building with package managers, causing the relative path to break

@0323pin
Copy link
Contributor

0323pin commented Dec 4, 2024

I could build from the previous commit but, then anyone running wiki-tui -V would get:

~> wiki-tui -V
wiki-tui 0.9.0-pre

which, would be confusing.

@Builditluc
Copy link
Owner Author

I'll publish wiki-api-macros v0.1.1 with the CARGO_MANIFEST_DIR patch so we can test the compilation of wiki-api (I have no problem of publishing multiple versions of wiki-api-macros since its a helper crate and not the main one)

@Builditluc
Copy link
Owner Author

wiki-tui 0.9.0-pre

I agree that is confusing, and that doesn't fix the issue going forward

@0323pin
Copy link
Contributor

0323pin commented Dec 4, 2024

I'll publish wiki-api-macros v0.1.1 with the CARGO_MANIFEST_DIR patch so we can test the compilation of wiki-api (I have no problem of publishing multiple versions of wiki-api-macros since its a helper crate and not the main one)

Bring it on 👍

@Builditluc
Copy link
Owner Author

Builditluc commented Dec 4, 2024

It's live, but I get the same error with cargo publish :/

EDIT: We need to pass in the CARGO_MANIFEST_DIR path of the wiki-api crate, not the macro's crate. But I'm unsure on how to achieve this (I didn't write the macro), since rust will only allow me to insert literal strings

@0323pin
Copy link
Contributor

0323pin commented Dec 4, 2024

A manual release still works

@Builditluc
Copy link
Owner Author

Well I really don't know hot to get it working. Maybe we should remove the proc-macro alltogether and write the code ourselves? But I'll have to look at it tomorrow

@0323pin
Copy link
Contributor

0323pin commented Dec 4, 2024

But I'll have to look at it tomorrow

Yeah, it's getting really late for me as well. Just let me know if you need any testing prior to release. Lets make sure things work before and, as I already said, I'm building in package mode, so it will get caught.

@0323pin
Copy link
Contributor

0323pin commented Dec 5, 2024

Sometimes is good with sleeping ...

I've added a patch at our end that allows the build to succeed and will merge the update soon.
For now, we will carry the following patch:

$NetBSD$

Allow finding languages.json
See, https://github.com/Builditluc/wiki-tui/issues/236

--- wiki-api/src/languages.rs.orig	2024-12-05 04:54:55.126975320 +0000
+++ wiki-api/src/languages.rs
@@ -1,3 +1,3 @@
 use wiki_api_macros::parse_languages;
 
-parse_languages!("./data/languages.json");
+parse_languages!("wiki-api/data/languages.json");

It would be nice to figure out what the actual issue is but, at least I can now merge the update.

@Builditluc
Copy link
Owner Author

Builditluc commented Dec 5, 2024

Yes, sometimes it does magic.
I'm currently testing how it would work if we removed the path completely and just passed in the literal json as a string to the macro

EDIT: Hey that works!

@Builditluc
Copy link
Owner Author

@0323pin I've pushed a commit to main that should fix the issue. I've already published the updated wiki-api-macros crate and am now going to publish wiki-api (these are both helper crates so I don't mind making a release there).

We should be then able to test the package compilation

@Builditluc
Copy link
Owner Author

Confirming this works using nixpkgs

   Compiling wiki-api-macros v0.1.2 (/build/source/wiki-api-macros)
   Compiling crossterm v0.27.0
   Compiling xml5ever v0.17.0
   Compiling tracing-error v0.2.0
   Compiling ratatui v0.26.3
   Compiling tui-input v0.9.0
   Compiling color-spantrace v0.2.1
   Compiling color-eyre v0.6.2
   Compiling markup5ever_rcdom v0.2.0
   Compiling scraper v0.17.1
   Compiling toml v0.8.19
   Compiling human-panic v1.2.2
   Compiling tui-logger v0.11.1
   Compiling tokio-util v0.7.8
   Compiling tokio-native-tls v0.3.1
   Compiling tokio-stream v0.1.14
   Compiling h2 v0.3.21
   Compiling hyper v0.14.27
   Compiling hyper-tls v0.5.0
   Compiling reqwest v0.11.20
   Compiling wiki-api v0.1.1 (/build/source/wiki-api)
   Compiling wiki-tui v0.9.1 (/build/source)
    Finished `release` profile [optimized] target(s) in 44.69s
Executing cargoInstallPostBuildHook
Finished cargoInstallPostBuildHook
Finished cargoBuildHook
buildPhase completed in 45 seconds
Running phase: checkPhase
Executing cargoCheckHook
cargoCheckHook flags: -j 20 --profile release --target x86_64-unknown-linux-gnu --offline -- --test-threads=20
   Compiling wiki-tui v0.9.1 (/build/source)
    Finished `release` profile [optimized] target(s) in 1.21s
     Running unittests src/lib.rs (target/x86_64-unknown-linux-gnu/release/deps/wiki_tui-44e2ed98e03b7459)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running unittests src/main.rs (target/x86_64-unknown-linux-gnu/release/deps/wiki_tui-43232a8220533480)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests wiki_tui

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

Finished cargoCheckHook
Running phase: installPhase
Executing cargoInstallHook
Finished cargoInstallHook
Running phase: fixupPhase
shrinking RPATHs of ELF executables and libraries in /nix/store/v3v0q1mvc1rs6bshfi6qdscr2w1ss6cq-wiki-tui-0.9.1
shrinking /nix/store/v3v0q1mvc1rs6bshfi6qdscr2w1ss6cq-wiki-tui-0.9.1/bin/wiki-tui
checking for references to /build/ in /nix/store/v3v0q1mvc1rs6bshfi6qdscr2w1ss6cq-wiki-tui-0.9.1...
patching script interpreter paths in /nix/store/v3v0q1mvc1rs6bshfi6qdscr2w1ss6cq-wiki-tui-0.9.1
stripping (with command strip and flags -S -p) in  /nix/store/v3v0q1mvc1rs6bshfi6qdscr2w1ss6cq-wiki-tui-0.9.1/bin
/nix/store/v3v0q1mvc1rs6bshfi6qdscr2w1ss6cq-wiki-tui-0.9.1

@Builditluc
Copy link
Owner Author

Builditluc commented Dec 5, 2024

@0323pin when you give me the go-ahead, I'll create a v0.9.1 release mentioning this issue and the commit message in main detailing the issue (gotta catch a train now!)

@0323pin
Copy link
Contributor

0323pin commented Dec 5, 2024

~> wiki-tui -V
wiki-tui 0.9.1

No patch needed ☺️

Hope you made it to the train.

@nunotexbsd
Copy link
Contributor

@Builditluc

Hello,
I can confirm that it builds fine with rust-1.83.0 on FreeBSD. Also run test is OK.

v0.9.0 + patch 536deee

Thanks

@nunotexbsd
Copy link
Contributor

nunotexbsd commented Dec 5, 2024

Lets wait for 0.9.1 version to be released.

One thing that I've noticed is that my ~/.config/wiki-tui/config.toml isn't being read as up and down cursor keys not working as I have in config:

[api]
base_url = 'https://en.wikipedia.org/'

[theme]
text = 'white'
title = 'red'
highlight = 'red'
background = 'black'
search_match = 'red'
highlight_text = 'white'
highlight_inactive = 'black'

[logging]
enabled = false
log_dir = 'wiki_tui.log'
log_level = 'INFO'

[features]
links = true
toc = true
[keybindings.down]
key = 'down'
mode = 'normal'

[keybindings.up]
key = 'up'
mode = 'normal'

[keybindings.left]
key = 'left'
mode = 'normal'

[keybindings.right]
key = 'right'
mode = 'normal'

[keybindings.focus_next]
key = 'tab'
mode = 'normal'

[keybindings.focus_prev]
key = 'tab'
mode = 'shift'
[settings.toc]
position = 'Right'
title = 'Default'
min_width = 20
max_width = 60
scroll_x = true
scroll_y = true
item_format = '{NUMBER} {TEXT}'

Reverting to 0.8.2, cursor keys and colors theme works fine.

Should I wait for next release and open a PR about it?

@0323pin
Copy link
Contributor

0323pin commented Dec 5, 2024

@nunotexbsd That's because it was a complete re-write and these things have changed.
I have a very bare bones config if you want something to start with.

My plan was to have fixed it by now but, other things came in the way. Anyway, hope it helps for a start and from there you could follow the docs.

@nunotexbsd
Copy link
Contributor

nunotexbsd commented Dec 5, 2024

@0323pin

Thanks for clarification.
I will test new configs then.

Cheers

EDIT: I followed your link and now I have working configs.

@Builditluc
Copy link
Owner Author

Builditluc commented Dec 5, 2024

~> wiki-tui -V
wiki-tui 0.9.1

No patch needed ☺️

Hope you made it to the train.

Yes I did, the train was late so I was able to wait in the cold (yay).

Should I change the patch to say v0.9.0, so it doesn't get confusing as to why there is no 0.9.1 published? (EDIT: Didn't see the latest comments before posting, I can publish 0.9.1in a few minutes, have to wait until my lecture is over)

@Builditluc
Copy link
Owner Author

@nunotexbsd yes, the configuration schema changed almost completely. I tried reusing the old schema where I could.

Since we already need to push another version to fix the build, should we include a more visible note about the config changes in the release notes (@nunotexbsd @0323pin, what do you think?)

@0323pin
Copy link
Contributor

0323pin commented Dec 5, 2024

@Builditluc You don't need to rush to push 0.9.1, there's plenty of time.

I was thinking of having a default set of config files covering all options but, I haven't managed to cover that, yet.
I guess you could just point to the configuration section of the web page.

@nunotexbsd
Copy link
Contributor

nunotexbsd commented Dec 5, 2024

@Builditluc

I was thinking the same about adding a pkg message for users updating from 0.8.2 to 0.9.x to adapt configs and add some helpfull links.

Yes, adding config changes to release notes is good.

EDIT: What about adding config samples to tarball? This way, packagers could install those files as samples so users try out.

@0323pin
Copy link
Contributor

0323pin commented Dec 5, 2024

Added a few more settings to my config files, https://codeberg.org/pin/netbsd/src/branch/main/.config/wiki-tui

Still not a 100% coverage of all options but, hopefully an example.

@Builditluc
Copy link
Owner Author

We already have a how to upgrade page, maybe one could expand this. In the release notes of 0.9.1 I'll point to this page and remind users again of the schema change.

@Builditluc
Copy link
Owner Author

Added a few more settings to my config files, https://codeberg.org/pin/netbsd/src/branch/main/.config/wiki-tui

Still not a 100% coverage of all options but, hopefully an example.

Looks good! We can also add some example configs with pictures to the website, but that would be a thing for the next version

@Builditluc
Copy link
Owner Author

Since I already bumped the version in the commit to v0.9.1, I'll create the git tag and publish the fixed version in a few minutes

@Builditluc
Copy link
Owner Author

@nunotexbsd @0323pin the release is up, you should now be able to update your packages

@0323pin
Copy link
Contributor

0323pin commented Dec 5, 2024

Cheers!
I've actually pushed 0.9.0 this morning but, will bump that sometime during this afternoon.

@Builditluc
Copy link
Owner Author

Closing the issue as v0.9.1 fixes it

@github-project-automation github-project-automation bot moved this from Todo to Done in wiki-tui: Roadmap Dec 5, 2024
@0323pin
Copy link
Contributor

0323pin commented Dec 5, 2024

Earlier, 0.9.0
Just a few moments ago, 0.9.1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

No branches or pull requests

3 participants