Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

inps/go-secp256k1 dep causing duplicate link issues with go-ethereum in the same go module as lotus #9811

Open
7 of 18 tasks
Schwartz10 opened this issue Dec 7, 2022 · 4 comments
Open
7 of 18 tasks

inps/go-secp256k1 dep causing duplicate link issues with go-ethereum in the same go module as lotus #9811

Schwartz10 opened this issue Dec 7, 2022 · 4 comments
Labels
kind/bug Kind: Bug kind/discussion Kind: Discussion

Comments

@Schwartz10
Copy link
Contributor

Schwartz10 commented Dec 7, 2022 •
edited
Loading

Checklist

  • This is not a security-related bug/issue. If it is, please follow please follow the security policy.
  • This is not a question or a support request. If you have any lotus related questions, please ask in the lotus forum.
  • This is not a new feature request. If it is, please file a feature request instead.
  • This is not an enhancement request. If it is, please file a improvement suggestion instead.
  • I have searched on the issue tracker and the lotus forum, and there is no existing related issue or discussion.
  • I am running the Latest release, or the most recent RC(release canadiate) for the upcoming release or the dev branch(master), or have an issue updating to any of these.
  • I did not make any code changes to lotus.

Lotus component

  • lotus daemon - chain sync
  • lotus miner - mining and block production
  • lotus miner/worker - sealing
  • lotus miner - proving(WindowPoSt)
  • lotus miner/market - storage deal
  • lotus miner/market - retrieval deal
  • lotus miner/market - data transfer
  • lotus client
  • lotus JSON-RPC API
  • lotus message management (mpool)
  • Other

Lotus Version

lotus `v1.18.1-0.20221201215704-9bf6052069de`

Describe the Bug

Thanks @Stebalien for the help on articulating this issue:

ipsn/go-secp256k1 is a fork of go-ethereum/crypto, which is used in filecoin-project/go-crypto (filecoin-project/go-crypto go.mod)

Because ipsn/go-secp256k1 is a fork of go-ethereum/crypto, it reuses the same C libraries, with the same symbols.

The result is that importing both go-ethereum/ethereum and filecoin-project/lotus in the same go module causes build errors when running clang. I expect relying on these two modules will be a not-so-uncommon use case among early FEVM builders, since there's helpful APIs in both.

Steb had two ideas:

  1. "We might be able to just switch to go-ethereum/crypto/secp256k1.. We didn't do that before because go couldn't do "partial" dependencies. But now it can."
  2. "Perform a mass symbol rename in go-secp256k1 as to avoid duplicate link conflicts"

Logging Information

/usr/local/Cellar/go/1.19.3/libexec/pkg/tool/darwin_amd64/link: running clang failed: exit status 1
duplicate symbol '_secp256k1GoPanicError' in:
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000005.o
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000018.o
duplicate symbol '_secp256k1GoPanicIllegal' in:
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000005.o
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000018.o
duplicate symbol '_secp256k1_context_destroy' in:
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000008.o
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000021.o
duplicate symbol '_secp256k1_ec_seckey_verify' in:
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000008.o
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000021.o
duplicate symbol '_secp256k1_ecdsa_verify' in:
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000008.o
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000021.o
duplicate symbol '_secp256k1_ecdsa_recoverable_signature_convert' in:
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000008.o
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000021.o
duplicate symbol '_secp256k1_nonce_function_default' in:
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000008.o
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000021.o
duplicate symbol '_secp256k1_ecdsa_recoverable_signature_serialize_compact' in:
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000008.o
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000021.o
duplicate symbol '_secp256k1_ecdsa_signature_serialize_compact' in:
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000008.o
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000021.o
duplicate symbol '_secp256k1_ecdsa_recoverable_signature_parse_compact' in:
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000008.o
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000021.o
duplicate symbol '_secp256k1_ecdsa_signature_parse_compact' in:
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000008.o
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000021.o
duplicate symbol '_secp256k1_ecdsa_recover' in:
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000008.o
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000021.o
duplicate symbol '_secp256k1_ecdsa_signature_serialize_der' in:
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000008.o
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000021.o
duplicate symbol '_secp256k1_ecdsa_signature_parse_der' in:
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000008.o
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000021.o
duplicate symbol '_secp256k1_ecdsa_sign' in:
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000008.o
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000021.o
duplicate symbol '_secp256k1_ext_scalar_mul' in:
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000008.o
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000021.o
duplicate symbol '_secp256k1_ec_privkey_tweak_mul' in:
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000008.o
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000021.o
duplicate symbol '_secp256k1_ec_pubkey_tweak_mul' in:
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000008.o
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000021.o
duplicate symbol '_secp256k1_context_set_error_callback' in:
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000008.o
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000021.o
duplicate symbol '_secp256k1_context_set_illegal_callback' in:
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000008.o
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000021.o
duplicate symbol '_secp256k1_context_randomize' in:
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000008.o
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000021.o
duplicate symbol '_secp256k1_ecdsa_signature_normalize' in:
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000008.o
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000021.o
duplicate symbol '_secp256k1_ec_pubkey_serialize' in:
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000008.o
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000021.o
duplicate symbol '_secp256k1_ec_pubkey_create' in:
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000008.o
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000021.o
duplicate symbol '_secp256k1_context_create' in:
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000008.o
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000021.o
duplicate symbol '_secp256k1_ec_pubkey_parse' in:
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000008.o
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000021.o
duplicate symbol '_secp256k1_context_clone' in:
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000008.o
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000021.o
duplicate symbol '_secp256k1_ec_pubkey_combine' in:
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000008.o
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000021.o
duplicate symbol '_secp256k1_ecdsa_sign_recoverable' in:
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000008.o
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000021.o
duplicate symbol '_secp256k1_ec_privkey_tweak_add' in:
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000008.o
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000021.o
duplicate symbol '_secp256k1_ec_pubkey_tweak_add' in:
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000008.o
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000021.o
duplicate symbol '_CURVE_B' in:
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000008.o
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000021.o
duplicate symbol '_secp256k1_nonce_function_rfc6979' in:
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000008.o
    /var/folders/lf/szs6pm8n7xn5wx2s5_k52w2r0000gn/T/go-link-4286204756/000021.o
ld: 33 duplicate symbols for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)



### Repo Steps

_No response_
@Schwartz10
Copy link
Contributor Author

Hey @arajasek - i went ahead and just did the name remappings and things are working again on my end. A simple replace of https://github.com/ipsn/go-secp256k1 => https://github.com/glifio/go-secp256k1 v0.0.1 does the trick

@jennijuju jennijuju added kind/discussion Kind: Discussion and removed need/triage labels Dec 8, 2022
@jennijuju jennijuju added this to the Network v18 milestone Dec 8, 2022
@jennijuju
Copy link
Member

@Kubuxu @magik6k thoughts?

@Kubuxu
Copy link
Contributor

Kubuxu commented Dec 8, 2022

Let’s try go-ethereum , keeping it to release versions not git commit.

@scotthconner
Copy link

It kind of looks like here that @Schwartz10 fixed this with the symbol renaming. Arugably #1 may be a little cleaner as it reduces the dependencies requirements by 1, but is this fixed? Is there a PR for Jon's fix?

@jennijuju jennijuju removed this from the Network v18 milestone Dec 16, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
kind/bug Kind: Bug kind/discussion Kind: Discussion
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@scotthconner @Kubuxu @Schwartz10 @jennijuju