Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

v24.08rc2 - bitcoin-cli not a valid config option #7595

Closed
nakoshi-satamoto opened this issue Aug 21, 2024 · 28 comments · Fixed by #7618
Closed

v24.08rc2 - bitcoin-cli not a valid config option #7595

nakoshi-satamoto opened this issue Aug 21, 2024 · 28 comments · Fixed by #7618
Assignees
@rustyrussell
Milestone
v24.08

Comments

@nakoshi-satamoto
Copy link

nakoshi-satamoto commented Aug 21, 2024 •
edited
Loading

I'm trying to test v24.08rc2
When I attempt to start lightningd I get an error message saying that "bitcoin-cli" is an unknown configuration option. This is a valid config option in v24.05 though, my bitcoin-cli is in a specific filepath that needs declared. Was this config changed in 24.08rc2 to something else?
@nakoshi-satamoto
Copy link
Author

Example of my current config

bitcoin-cli=/home/satoshi/bitcoin-27.3/bin/bitcoin-cli

@nakoshi-satamoto
Copy link
Author

lightningd: Config file /home/user/.lightning/config line 8: bitcoin-cli=/home/satoshi/bitcoin-27.0/bin/bitcoin-cli: unknown option

@vincenzopalazzo
Copy link
Collaborator

Can you post your logs? it is possible that some plugin like bcli.c is dying?

@nakoshi-satamoto
Copy link
Author

When I go back to version 24.05 it works as normally. In this case the log shows nothing for 24.08rc2, I can re-attempt later in debug mode and post logs here.

@nakoshi-satamoto
Copy link
Author

nakoshi-satamoto commented Aug 21, 2024 •
edited
Loading

No other data in log file when attempted in debug

`2024-08-21T16:12:35.311Z INFO lightningd: v24.08rc2

2024-08-21T16:12:35.311Z DEBUG lightningd: Opened log file /home/satoshi/.lightning/ln-log.txt

2024-08-21T16:12:35.326Z DEBUG plugin-manager: started(1446405) /home/satoshi/apps/peerswap/peerswap

2024-08-21T16:12:35.341Z DEBUG lightningd: io_break: check_plugins_manifests

2024-08-21T16:12:35.341Z DEBUG lightningd: io_loop_with_timers: plugins_init
`

standard output shows
lightningd: Config file /home/satoshi/.lightning/config line 8: bitcoin-cli=/home/satoshi/apps/bitcoin-core/bitcoin-27.0/bin/bitcoin-cli: unknown option

@nakoshi-satamoto nakoshi-satamoto mentioned this issue Aug 27, 2024
Open
@nakoshi-satamoto
Copy link
Author

v24.08rc2 is not accepting required config settings necessary to work with Bitcoin Core.

@Lagrang3
Copy link
Collaborator

Lagrang3 commented Aug 27, 2024 •
edited
Loading

When lightningd starts you should see a lot of plugin-maganer lines indicating the PID of the default plugins.
bitcoin-cli is an option defined within bcli plugin. In your logs there is only peerswap being started. Usually you should see:

2024-08-27T15:31:30.965Z INFO    lightningd: v24.08rc3-1-g48fa438-modded
2024-08-27T15:31:30.994Z INFO    lightningd: Creating configuration directory /tmp/l1/regtest
2024-08-27T15:31:30.994Z DEBUG   lightningd: Opened log file /tmp/l1/log
2024-08-27T15:31:30.996Z DEBUG   plugin-manager: started(508405) /home/lagrange/lightning/plugins/autoclean
2024-08-27T15:31:30.997Z DEBUG   plugin-manager: started(508408) /home/lagrange/lightning/plugins/chanbackup
2024-08-27T15:31:30.999Z DEBUG   plugin-manager: started(508413) /home/lagrange/lightning/plugins/bcli
2024-08-27T15:31:31.000Z DEBUG   plugin-manager: started(508416) /home/lagrange/lightning/plugins/commando
2024-08-27T15:31:31.002Z DEBUG   plugin-manager: started(508420) /home/lagrange/lightning/plugins/funder
2024-08-27T15:31:31.003Z DEBUG   plugin-manager: started(508424) /home/lagrange/lightning/plugins/topology
2024-08-27T15:31:31.004Z DEBUG   plugin-manager: started(508427) /home/lagrange/lightning/plugins/keysend
2024-08-27T15:31:31.005Z DEBUG   plugin-manager: started(508430) /home/lagrange/lightning/plugins/offers
2024-08-27T15:31:31.006Z DEBUG   plugin-manager: started(508433) /home/lagrange/lightning/plugins/pay
2024-08-27T15:31:31.008Z DEBUG   plugin-manager: started(508436) /home/lagrange/lightning/plugins/recklessrpc
2024-08-27T15:31:31.009Z DEBUG   plugin-manager: started(508441) /home/lagrange/lightning/plugins/recover
2024-08-27T15:31:31.010Z DEBUG   plugin-manager: started(508444) /home/lagrange/lightning/plugins/txprepare
2024-08-27T15:31:31.011Z DEBUG   plugin-manager: started(508447) /home/lagrange/lightning/plugins/cln-renepay
2024-08-27T15:31:31.012Z DEBUG   plugin-manager: started(508449) /home/lagrange/lightning/plugins/spenderp
2024-08-27T15:31:31.013Z DEBUG   plugin-manager: started(508453) /home/lagrange/lightning/plugins/cln-askrene
2024-08-27T15:31:31.014Z DEBUG   plugin-manager: started(508456) /home/lagrange/lightning/plugins/sql
2024-08-27T15:31:31.015Z DEBUG   plugin-manager: started(508459) /home/lagrange/lightning/plugins/bookkeeper
2024-08-27T15:31:31.017Z DEBUG   plugin-manager: started(508460) /home/lagrange/lightning/plugins/clnrest/clnrest
2024-08-27T15:31:31.018Z DEBUG   plugin-manager: started(508461) /home/lagrange/lightning/plugins/wss-proxy/wss-proxy
2024-08-27T15:31:31.226Z UNUSUAL plugin-bookkeeper: topic 'utxo_deposit' is not a known notification topic
2024-08-27T15:31:31.226Z UNUSUAL plugin-bookkeeper: topic 'utxo_spend' is not a known notification topic
2024-08-27T15:31:31.226Z DEBUG   lightningd: io_break: check_plugins_manifests
2024-08-27T15:31:31.226Z DEBUG   lightningd: io_loop_with_timers: plugins_init
...

To me it seems lightningd is not finding the path to the builtin plugins in your setup.

@nakoshi-satamoto
Copy link
Author

so what is the path to bcli then @Lagrang3 ? I never had to declare the path to that in the earlier versions.

@Lagrang3
Copy link
Collaborator

Lagrang3 commented Aug 27, 2024 •
edited
Loading

I am looking into the code, it seems that lightningd searches for the builtin plugins relative to his own path ../plugins.
Do you use make install?

@nakoshi-satamoto
Copy link
Author

nakoshi-satamoto commented Aug 27, 2024 •
edited
Loading

I found it

corelightning-24.05/usr/libexec/c-lightning/plugins/bcli

I'll test today when I get a chance with this path declared as a plugin in the config

@nakoshi-satamoto
Copy link
Author

I am looking into the code, it seems that lightningd searches for the builtin plugins relative to his own path ../plugins. Do you use make install?

I am just using the signed binaries. I did not compile.

@grubles
Copy link
Contributor

grubles commented Aug 27, 2024

Couldn't you symlink the binary to somewhere in your PATH?

@nakoshi-satamoto
Copy link
Author

Declaring the path of the plugin does not work either, I get this
error starting plugin '/home/satoshi/clightning/corelightning-v24.08rc3/usr/libexec/c-lightning/plugins': opening pipe: Permission denied

@nakoshi-satamoto
Copy link
Author

nakoshi-satamoto commented Aug 27, 2024 •
edited
Loading

Couldn't you symlink the binary to somewhere in your PATH?

And what would be the proper way to do that? This seems like a hack work around. Would users of core lightning be expected to do the same?

@nakoshi-satamoto
Copy link
Author

Declaring the path of the plugin does not work either, I get this error starting plugin '/home/satoshi/clightning/corelightning-v24.08rc3/usr/libexec/c-lightning/plugins': opening pipe: Permission denied

I see I made an error by not including /bcli at the end of that path. I did so and now I get this

lightningd: Could not run /usr/libexec/c-lightning/lightning_channeld: No such file or directory

@nakoshi-satamoto
Copy link
Author

nakoshi-satamoto commented Aug 27, 2024 •
edited
Loading

So Core Lightning now only works if it is installed system-wide rather than from a local filepath, and only works if it is the same for Bitcoin Core?

@nakoshi-satamoto
Copy link
Author

I have been running this under a filepath of /home/satoshi/clightning/corelightning-v24.08rc3/ (an example)

@Lagrang3
Copy link
Collaborator

So Core Lightning now only works if it is installed system-wide rather than from a local filepath, and only works if it is the same for Bitcoin Core?

No. It's not that. My node's binaries are in a custom path. And you shouldn't have to hack your way around.

@grubles
Copy link
Contributor

grubles commented Aug 27, 2024

Yes I just meant symlinking as a temporary workaround to get your node back up.

@nakoshi-satamoto
Copy link
Author

Thanks everyone. If I need to do symlinking I can, although not ideal. @Lagrang3 I'm curious what you are doing differently than me. Are you using bitcoin core in a custom path also? I'm using both Bitcoin Core and Core Lightning in a custom path.

I'm hopeful that this pre-release might also fix the payment issues I was having, why I was wanting to test this.

@Lagrang3
Copy link
Collaborator

Lagrang3 commented Aug 27, 2024 •
edited
Loading

yes. I have core lightning and bitcoin binaries in custom paths.
To upgrade I go through the steps ./configure --prefix=MYCUSTOMPATH && make && make install that produce a directory structure like this:

$ tree /opt/lightning/24.08/
/opt/lightning/24.08/
├── bin
│   ├── lightning-cli
│   ├── lightningd
│   ├── lightning-hsmtool
│   └── reckless
├── libexec
│   └── c-lightning
│       ├── lightning_channeld
│       ├── lightning_closingd
│       ├── lightning_connectd
│       ├── lightning_dualopend
│       ├── lightning_gossipd
│       ├── lightning_hsmd
│       ├── lightning_onchaind
│       ├── lightning_openingd
│       ├── lightning_websocketd
│       └── plugins
│           ├── autoclean
│           ├── bcli
│           ├── bookkeeper
│           ├── chanbackup
│           ├── cln-askrene
│           ├── cln-renepay
...

which is exactly what you get when you extract the release binaries.

Then I have a script in a cron job that calls lightningd, eg

/opt/lightning/24.08/bin/lightningd --daemon ...

lightningd uses the path of the executable to figure out the path to the other binaries.

@Lagrang3
Copy link
Collaborator

Lagrang3 commented Aug 27, 2024 •
edited
Loading

I think we should add log messages to inform that builtin plugins are not loaded and also inform where are we trying to look for them. We don't have those as far as I can tell from testing now.

@nakoshi-satamoto
Copy link
Author

I think we should add log messages to inform that builtin plugins are not loaded and also inform where are we trying to look for them. We don't have those as far as I can tell from testing now.

Great idea

@Lagrang3
Copy link
Collaborator

I hit it locally with the release binaries:

$ /home/lagrange/tmp/cln-24.08-release/bin/lightningd "--network=$network" "--lightning-dir=$LIGHTNING_DIR/l1" "--bitcoin-datadir=$BITCOIN_DIR" "--database-upgrade=true"
lightningd: Config file /tmp/lightning/l1/config line 1: bitcoin-cli=/opt/bitcoin/bin/bitcoin-cli: unknown option

Instead I had no issues with binaries compiled from source.

@Lagrang3 Lagrang3 mentioned this issue Aug 27, 2024
Merged
7 tasks
@ShahanaFarooqui ShahanaFarooqui added this to the v24.08 milestone Aug 27, 2024
@rustyrussell
Copy link
Contributor

I have been running this under a filepath of /home/satoshi/clightning/corelightning-v24.08rc3/ (an example)

Right. CLN has two modes, one is that it expects to be in the build directory (good for developers), the other in the --prefix directory it was configured with.

I'm surprised that this used to work! But thanks for the report, I'll make it work!

rustyrussell added a commit to rustyrussell/lightning that referenced this issue Aug 27, 2024
@rustyrussell
lightningd: fix up installs in subdirectories.
b062ba2 
Commit a1fdeee "Makefile: clean up install path handling."
broke the ability to configure with one path and then run in a
different path.  Turns out people actually do this!  So, we have
to use relative paths, compared to our existing binary.

Fixes: ElementsProject#7595
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
@rustyrussell rustyrussell mentioned this issue Aug 27, 2024
Merged
7 tasks
@nakoshi-satamoto
Copy link
Author

I compiled and it returns the same error. Can we bring back the bitcoin-cli= config option to declare a custom path for bitcoin-cli?
Right now the new version of CLN only supports bitcoin core if it is installed system wide at a specific filepath.

@nakoshi-satamoto
Copy link
Author

I made a softlink of /usr/local/bin/bitcoin-cli to the bitcoin-cli in my custom path. Yeah this still does not work
when I run lightningd I get error
lightningd: Could not run /usr/local/libexec/c-lightning/lightning_channeld: Permission denied

I am trying to run v24.08rc3 that I compiled.

@nakoshi-satamoto
Copy link
Author

The last issue "permission denied" is a different issue. I think it resulted from me installing with "sudo make install" while my umask was set to 027 instead of 022. I fixed the permissions with chmod.

I am now running v24.08rc3.

How easy would it be to bring back the bitcoin-cli= option in the lightning config file? This would eliminate the need of people to have to make symbolic links or compile with a custom path.

rustyrussell added a commit to rustyrussell/lightning that referenced this issue Aug 28, 2024
@rustyrussell
lightningd: fix up installs in subdirectories.
895052d 
Commit a1fdeee "Makefile: clean up install path handling."
broke the ability to configure with one path and then run in a
different path.  Turns out people actually do this!  So, we have
to use relative paths, compared to our existing binary.

And we can't use path_rel, because that requires that the path
exist (thanks @Lagrang3!).

Fixes: ElementsProject#7595
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
rustyrussell added a commit to rustyrussell/lightning that referenced this issue Aug 28, 2024
@rustyrussell
lightningd: fix up installs in subdirectories.
d7eb717 
Commit a1fdeee "Makefile: clean up install path handling."
broke the ability to configure with one path and then run in a
different path.  Turns out people actually do this!  So, we have
to use relative paths, compared to our existing binary.

And we can't use path_rel, because that requires that the path
exist (thanks @Lagrang3!).

Fixes: ElementsProject#7595
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
ShahanaFarooqui pushed a commit to rustyrussell/lightning that referenced this issue Aug 28, 2024
@rustyrussell @ShahanaFarooqui
lightningd: fix up installs in subdirectories.
e59d980 
Commit a1fdeee "Makefile: clean up install path handling."
broke the ability to configure with one path and then run in a
different path.  Turns out people actually do this!  So, we have
to use relative paths, compared to our existing binary.

And we can't use path_rel, because that requires that the path
exist (thanks @Lagrang3!).

Fixes: ElementsProject#7595
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
ShahanaFarooqui pushed a commit to rustyrussell/lightning that referenced this issue Aug 28, 2024
@rustyrussell @ShahanaFarooqui
lightningd: fix up installs in subdirectories.
a0f7225 
Commit a1fdeee "Makefile: clean up install path handling."
broke the ability to configure with one path and then run in a
different path.  Turns out people actually do this!  So, we have
to use relative paths, compared to our existing binary.

And we can't use path_rel, because that requires that the path
exist (thanks @Lagrang3!).

Fixes: ElementsProject#7595
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@rustyrussell rustyrussell

Labels
None yet
Projects
None yet
Milestone
v24.08
Development

Successfully merging a pull request may close this issue.

lightningd: fix up installs in subdirectories. rustyrussell/lightning
6 participants
@rustyrussell @grubles @vincenzopalazzo @ShahanaFarooqui @Lagrang3 @nakoshi-satamoto