Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Left sig != right sig. Signature check failure. message occurs very frequently - follow up to 6288 #6545

Closed
LaurenWags opened this issue Oct 18, 2019 · 14 comments · Fixed by brave/brave-core#3819

Comments

@LaurenWags
Copy link
Member

LaurenWags commented Oct 18, 2019

Description

While testing #6288 I noticed that I very frequently got the Left sig != right sig. Signature check failure. message in the terminal. This results in not all ballots being sent out, so not all BAT is properly distributed.

Steps to Reproduce

  1. enable rewards with short contribution interval on staging. I also launched with --enable-logging=stderr --vmodule=*rewards*=6 to see log messages.
  2. claim grant (30 BAT)
  3. set AC amount to Up to 20 BAT
  4. add verified and non verified publishers to AC table
  5. add verified recurring tips in this order (5, 10, 10, 10, 1)
  6. wait for monthly contribution to trigger
  7. you should see 26 BAT goes through for monthly and 4 BAT from AC
  8. Watch terminal.
  • for simplicity, all verified publishers used belonged to the same publisher email

Actual result:

See several Left sig != right sig. Signature check failure. messages. Watch publisher account. Not all BAT is distributed as expected.
Screen Shot 2019-10-18 at 12 52 47 PM

Expected result:

No Left sig != right sig. Signature check failure. messages

Reproduces how often:

easily

Brave version (brave://version info)

Brave 0.73.8 Chromium: 78.0.3904.50 (Official Build) nightly (64-bit)
Revision 2accdc52c79976e264cd2694df6db31d1fccd8e8-refs/branch-heads/3904@{#658}
OS macOS Version 10.13.6 (Build 17G5019)

Version/Channel Information:

  • Can you reproduce this issue with the current release? unsure
  • Can you reproduce this issue with the beta channel? unsure
  • Can you reproduce this issue with the dev channel? unsure
  • Can you reproduce this issue with the nightly channel? yes

Other Additional Information:

  • Does the issue resolve itself when disabling Brave Shields? n/a
  • Does the issue resolve itself when disabling Brave Rewards? n/a
  • Is the issue reproducible on the latest version of Chrome? n/a

Miscellaneous Information:

cc @NejcZdovc @rebron @brave/legacy_qa

@LaurenWags
Copy link
Member Author

LaurenWags commented Oct 18, 2019

Ran some tests with just Auto Contribute and no recurring tips - did not see this message in the terminal. See brave/brave-core#3658 (comment) - but without recurring tips, all BAT expected was transferred to the publisher account.

@NejcZdovc NejcZdovc added the priority/P1 A very extremely bad problem. We might push a hotfix for it. label Oct 21, 2019
@mandar-brave
Copy link

mandar-brave commented Oct 21, 2019

This is an issue observed before #6288 and has been on nightly since 0.68.
Likely impacting everyone.

cc @rebron

@NejcZdovc
Copy link
Contributor

some more info about it

Left sig != right sig. Signature check failure.
02:59:32   
02:59:32  left 3i5HecEMhBDPUkIl1q6/4bEmmO17/FuawODhmx6Q2CR 8IB6ei3jUrm/gWWcquSy1B6AJ2UFPQJVQ5BQ7KY2E/m AdiauYiRCW81ABitpQzQGy+ZyjxtuvxVkwL1vWplOWU 6eFcOKcGZMcSn9KTTMqrhEFUHfCW6SVpnIUcUb7KvXQ AcE/5eiVGrXj5spee1f8StGDgHclgSlPZmbNb+xwUt2 2IbtraiAIbsMB449gt5RZpkNlRSKZk5tb91XIl0NI0G 5fmCzZ62FkiQlgdvbSa9OCiM2nptliESec3psfh7xE9 3woC807nH39Ybu8A/VM+389DvX1MZn0wSnDirZtZz5O 9cHfIXPEnWfE1pz31Hm4+knBJxdLn7OStCyC0owbN9k L1ZYM2Cs7w6mQWS4BPj+4bcaTJC43YJSPXrx9D+dNB 8IYsVVSbBdLsKIcCEjET8OC0J56QZGjUXxQOZdc0IrD 1VJTAzl9egs6EPI7w/Ps0G9U9qAb6Na8z/iTK8fkKtv
02:59:32  
02:59:32  right 5Wn11fx060n1v+s547jf3da1NZV6Ul5+kEMm2PTaU/2 8xxZxcoUnmp5hnaSSGKCJgU0vT+74rqn1kWhL8EDAPW 3nXuDanSqsyz/lXnhSTdzvdAxTiFnj1HYVRIOmCWI6H A4MdUpHLkLFccDykdbx40E/CPGqkNaTmPHVzbmA5XRI 6IjGKo7+Gn5sfpxTzITQfIgS1reVDET3Ht3smwTFRwV 4MWWTtuzpVhkWyr24ua3SORlEZc5HfQVQX/kj6dZkph 4QDuM0HvDRj4nwbbSwtFYIALLzgwcLIR5JMuQSE379D 7GJa/DxSIU/8P4cK356DWiUSzR0I1ELb5wQqwiRSHBh 9m0JqSaY6/rSVCyLr/BYC/1Aos2oj9XQpp0GA2dg+Ov BNjY4/f1JZ7L26wdw6SQl0ofB8hfn++t8DIl91Vzx3k AA/YaE8bjghz+LFevCr8pv00BrEOyts5yHaSdC4hlsX 2lWc4rHWXnVEUwlNtzK8rkCX9m0Ioh19cBxM0jvb3xM
02:59:32  Invalid signature in registerUserFinal 0c25b253059a42a828898e5d4013555
02:59:32  Cred.read: failed
02:59:32  Could not read ravk or cred. 
02:59:32  Cred.read: failed
02:59:32  Could not read ravk or cred. 
02:59:32  Cred.read: failed
02:59:32  Could not read ravk or cred. 
02:59:32  Cred.read: failed
02:59:32  Could not read ravk or cred. 

from https://staging.ci.brave.com/job/brave-browser-build-pr/job/native-button-styles/15/execution/node/658/log/

@kjozwiak
Copy link
Member

@GeetaSarvadnya
Copy link

Reproduced in Windows 10 x64 - 0.70.120

image

@LaurenWags
Copy link
Member Author

LaurenWags commented Oct 23, 2019

Worked with @NejcZdovc to confirm retries are working, so even though this error is fairly easily reproduced, the retry logic does kick in and the votes which failed are retried. Used the following STR to show this:

  1. enable rewards with short contribution interval on staging. Launched with --enable-logging=stderr --vmodule=*rewards*=6 --rewards=staging=true,reconcile-interval=10,short-retries=true to see log messages, use staging env, set short contribution interval, and short retry interval.
  2. Recovered wallet with 30 BAT
  3. set AC amount to Up to 20 BAT
  4. add verified and non verified publishers to AC table
  5. add verified recurring tips in this order (5, 10, 10, 10, 1)
  6. wait for monthly contribution to trigger
  7. you should see 26 BAT goes through for monthly and 4 BAT from AC
  8. Watch terminal and see Left sig != right sig. error messages
  9. Continue to watch terminal to see retries occur.
    10 Also, check ledger_state to confirm. As retries are successful, the file gets smaller. Eventually, the batch section is empty, indicating that all retries were successful.

Confirmed above scenario using

Brave 0.70.121 Chromium: 78.0.3904.70 (Official Build) (64-bit)
Revision edb9c9f3de0247fd912a77b7f6cae7447f6d3ad5-refs/branch-heads/3904@{#800}
OS macOS Version 10.13.6 (Build 17G5019)

@jhoneycutt
Copy link

This appears to be caused by braveledger_contribution::PhaseTwo::PrepareBatchCallback overwriting an existing prepareBallot object with an object for a different viewingId. I believe that the check there for ballots[i].proofBallot_.empty() is a typo and should be ballots[i].prepareBallot_.empty()

@jhoneycutt
Copy link

A similar but separate issue exists in braveledger_contribution::PhaseTwo::ProofBatchCallback, filed as #6607.

@mandar-brave
Copy link

@jhoneycutt is there a fix being made available for this issue and #6607?

@mrose17
Copy link
Member

mrose17 commented Oct 29, 2019

@jhoneycutt - i'm trying to figure which line in phase_two.cc you are referring to in #6545 (comment) ... can you point me to the line in particular? thanks!

@jhoneycutt
Copy link

@mrose17 https://github.com/brave/brave-core/blob/master/vendor/bat-native-ledger/src/bat/ledger/internal/contribution/phase_two.cc#L292. However, fixing this typo isn't sufficient to fix the issue. We need to check that we are only assigning prepareBallot objects to ballots for the current viewingId.

@bsclifton
Copy link
Member

Added to 0.74.x milestone, since this is merged 😄👍

@LaurenWags
Copy link
Member Author

LaurenWags commented Nov 8, 2019

Verified on Nightly with

Brave 0.74.9 Chromium: 78.0.3904.87 (Official Build) nightly (64-bit)
Revision 20c21f4010010f32462ea8e1d6af30cef66d48c8-refs/branch-heads/3904@{#840}
OS macOS Version 10.13.6 (Build 17G5019)

Verified test plan from brave/brave-core#3819 on Nightly 3x, however since amount of staging grants has changed (17.5 BAT instead of 30 BAT) the expected contributed amounts were different. I was only able to check that my own testing publisher accts got the expected BAT. Not all accounts used for testing were owned by me. However, during each test no left sig != right sig error was encountered.

@LaurenWags
Copy link
Member Author

LaurenWags commented Nov 8, 2019

Verified passed with

Brave 0.71.113 Chromium: 78.0.3904.87 (Official Build) (64-bit)
Revision 20c21f4010010f32462ea8e1d6af30cef66d48c8-refs/branch-heads/3904@{#840}
OS macOS Version 10.13.6 (Build 17G5019)

Verification passed on

Brave 0.71.114 Chromium: 78.0.3904.97 (Official Build) (64-bit)
Revision 021b9028c246d820be17a10e5b393ee90f41375e-refs/branch-heads/3904@{#859}
OS Windows 10 OS Version 1803 (Build 17134.1006)

Verification passed on

Brave 0.71.114 Chromium: 78.0.3904.97 (Official Build) (64-bit)
Revision 021b9028c246d820be17a10e5b393ee90f41375e-refs/branch-heads/3904@{#859}
OS Ubuntu 18.04 LTS

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

10 participants