GetSignedTokens goes into retry loop when server responds with Captcha Required Exception

[btlechowski]

Follow up to #15600

Race condition when GetSignedTokens goes into retry loop when initially server responds with no captcha for wallet, So captcha is not shown when GetSignedTokens throws a Captcha Required Exception.

Steps to Reproduce

1. Overwrite https://grant.rewards.bravesoftware.com/v3/captcha/challenge/* to respond with 404 (means no captcha)
2. Clean profile
3. Run Brave
4. Enable Rewards and Ads
5. Schedule a captcha for your payment ID
6. Trigger unblinded tokens refill
7. Observe the logs

Actual result:

GetSignedTokens goes into retry loop when server responds with Captcha Required Exception

[24294:24294:1014/233540.620240:VERBOSE1:refill_unblinded_tokens.cc(116)] Refill unblinded tokens
[24294:24294:1014/233540.620402:VERBOSE1:refill_unblinded_tokens.cc(134)] GetScheduledCaptcha
[24294:24294:1014/233541.386908:VERBOSE1:refill_unblinded_tokens.cc(143)] OnGetScheduledCaptcha
[24294:24294:1014/233541.386957:VERBOSE1:refill_unblinded_tokens.cc(157)] RequestSignedTokens
[24294:24294:1014/233541.386995:VERBOSE2:refill_unblinded_tokens.cc(158)] POST /v1/confirmation/token/{payment_id}
[24294:24294:1014/233541.394233:VERBOSE6:refill_unblinded_tokens.cc(168)] URL Request:
  URL: https://ads-serve.bravesoftware.com/v1/confirmation/token/0312abca-fca7-424c-a8bd-82cbd816df23
  Content: {"blindedTokens":["MrmnQhkKWuos1A6B9rcJzqX82iFWrB26r3/RueQ+GF0=","WkVv28UVXL/7jtfcCiUtfNSlYWtVopm8Ro6nVm0Elhc=","dsXDKZWcgzBp5ahKQxid/jJdqWnEkmYR0+HU+Y0erTQ=","0F77bgA6MIt18fTi2eh0iGGUTlpQGeflsYBJAF2FJSM=","mJL++MuTUIGyq8OctaugY2aOXBEvkCQkG8fVjNtrB1g=","oj8MA4EEzr1MzWFxNSQRZMI0iINYxnFIReo7QO+c3wY=","yAfJB1KyodpA8OR6xM0zJvB9lyoFc9O67CmuW2jcLgM=","LB3iL4bf5DgjC9j64ZfnKtiuUF55o9bemd98c+vTQR0=","/IKTOwDb2Mq9pgyGGXtbXC1Q7uQrYeYW0pTk7A9DW0c=","PAXip0rZ89GHxsm9P/0wO20Brm4WWOo14fkjUblQRWc=","ALPf/97gon+lxf6+wbTLHSioZ73LQk5T9dDLBnFi6Ww=","3iIp4Ku4qNnMJgyZeZsB21M2F+orDKLLvjXzY+Z+VSs=","bvVbQhhkq7K63QROG070GtRCySdQmEcF+hvH/mHYuSE=","2P3zo10uHZq3TVCrzbf5V3oSZFoxkvS5jjiPjh8Rjks=","AIQ4RMHLkrQsD+lXobY4QFJgkbTWG9pX0KIneecLokI=","Uqs2g+4VevGiCH+s7iGZ5y41FjpA0zB4+NpYtwwg/lw=","DuoPvGcrIiNOXrJE0h7jMRYx8agnSAGRXoRt4SxPSn8=","IMVRvvv2MaEFztXowXkIcGAT88Lb2EVkssOJ0WbM/WQ=","KIHw8ya4kMvLoLpJ6TUhK1u8dmqrPhrCht0yDdV9tQU=","iHAnpW0uI+T9dIZluRY/a25n/52Z4ngBwXD4xuS+A2M=","Ri4acwuyRd7KdDerzSDZv6PUPejAY2C28UfGA/23qUg=","etAVhyUCwZXxb57C/wc3we4GPTyNO/MQXYu3n3QsSTA=","ZEkI4OO6bLb+yO8bdjd5MbDM2xoSiY6X+4Yw5QUg2xw=","hmM93yr3CD9OzDbVkrwuEhd5+3Anzw9QY4mfjSvDikk=","IFb01pLxjPVfdLcpO1La4EjHO3XoVv7V7LiO1rkHMiU=","SKqAEwjmE5FxjMf7bu0J2dgrv63lCu3kW1HSaRmyC3U=","KOnhd+TDtrVPHzNSX7uwQ3dZEnOl3IImMH35cpfHlGA=","8u8q70H/UG7WAvs0f1LUErob4mOrKGZ3WmMuT8vlKVs=","8kCsjancCxMqc1a0u7l31bFymYklwp0olVXYQpNjEjI=","ugVrfuquyCVlNoAmku/Ajl4Q0M392KvAmCDSwS1c5jk=","trX9It5I4NxB0cC1MxIRQCBSLjHO0MACWvjYE5xd4Q0="]}
  Content Type: application/json
  Method: kPost
[24294:24294:1014/233541.659060:VERBOSE1:refill_unblinded_tokens.cc(178)] OnRequestSignedTokens
[24294:24294:1014/233541.659360:VERBOSE6:refill_unblinded_tokens.cc(180)] URL Response:
  URL: https://ads-serve.bravesoftware.com/v1/confirmation/token/0312abca-fca7-424c-a8bd-82cbd816df23
  Response Status Code: 201
  Response: {"nonce":"a29c7ca1-bf5c-4e90-9f00-f5200046668f"}
[24294:24294:1014/233541.659895:VERBOSE1:refill_unblinded_tokens.cc(211)] GetSignedTokens
[24294:24294:1014/233541.660122:VERBOSE2:refill_unblinded_tokens.cc(212)] GET /v1/confirmation/token/{payment_id}?nonce={nonce}
[24294:24294:1014/233541.660365:VERBOSE6:refill_unblinded_tokens.cc(216)] URL Request:
  URL: https://ads-serve.bravesoftware.com/v1/confirmation/token/0312abca-fca7-424c-a8bd-82cbd816df23?nonce=a29c7ca1-bf5c-4e90-9f00-f5200046668f
  Method: kGet
[24294:24294:1014/233541.877253:VERBOSE1:refill_unblinded_tokens.cc(226)] OnGetSignedTokens
[24294:24294:1014/233541.877375:VERBOSE6:refill_unblinded_tokens.cc(228)] URL Response:
  URL: https://ads-serve.bravesoftware.com/v1/confirmation/token/0312abca-fca7-424c-a8bd-82cbd816df23?nonce=a29c7ca1-bf5c-4e90-9f00-f5200046668f
  Response Status Code: 429
  Response: "Captcha Required Exception"
[24294:24294:1014/233541.879639:INFO:refill_unblinded_tokens.cc(232)] Failed to get signed tokens
[24294:24294:1014/233541.879760:VERBOSE1:refill_unblinded_tokens.cc(385)] Retry refilling unblinded tokens in 0 godzin, 0 minut i 20 sekund at 23:36
[24294:24294:1014/233544.079137:VERBOSE1:browser_manager.cc(65)] Browser window is inactive
[24294:24294:1014/233544.080012:VERBOSE6:user_activity.cc(44)] Triggered event: 17 (7.5:2:3600 s)
[24294:24294:1014/233544.080459:VERBOSE1:browser_manager.cc(99)] Browser did enter background
[24294:24294:1014/233544.080595:VERBOSE6:user_activity.cc(44)] Triggered event: 02 (8:2:3600 s)
[24294:24294:1014/233556.546712:VERBOSE1:confirmations.cc(217)] No failed confirmations to retry
[24294:24294:1014/233601.877892:VERBOSE1:refill_unblinded_tokens.cc(211)] GetSignedTokens
[24294:24294:1014/233601.878010:VERBOSE2:refill_unblinded_tokens.cc(212)] GET /v1/confirmation/token/{payment_id}?nonce={nonce}
[24294:24294:1014/233601.878108:VERBOSE6:refill_unblinded_tokens.cc(216)] URL Request:
  URL: https://ads-serve.bravesoftware.com/v1/confirmation/token/0312abca-fca7-424c-a8bd-82cbd816df23?nonce=a29c7ca1-bf5c-4e90-9f00-f5200046668f
  Method: kGet
[24294:24294:1014/233602.103522:VERBOSE1:refill_unblinded_tokens.cc(226)] OnGetSignedTokens
[24294:24294:1014/233602.103609:VERBOSE6:refill_unblinded_tokens.cc(228)] URL Response:
  URL: https://ads-serve.bravesoftware.com/v1/confirmation/token/0312abca-fca7-424c-a8bd-82cbd816df23?nonce=a29c7ca1-bf5c-4e90-9f00-f5200046668f
  Response Status Code: 429
  Response: "Captcha Required Exception"
[24294:24294:1014/233602.104630:INFO:refill_unblinded_tokens.cc(232)] Failed to get signed tokens

GetScheduledCaptcha is not re-scheduled after requests fails
Captcha is not shown

Expected result:

The captcha is shown.

Reproduces how often:

Easily reproduced

Brave version (brave://version info)

Brave	1.31.85 Chromium: 95.0.4638.50 (Official Build) (64-bit)
Revision	3e9db37693d0cf8e31a89fbcf50401ae91125cef-refs/branch-heads/4638@{#816}
OS	Ubuntu 18.04 LTS

cc @emerick @tmancey @jsecretan @brave/legacy_qa @rebron

[emerick]

@gpestana @Sneagan Is there anything else we can do to track down where this 429 HTTP response comes from or what leads to it? I copied the relevant part of the log file below, which shows us making a call to get the signed tokens and receiving a 429 response. Before this call, I guess @btlechowski had configured https://grant.rewards.bravesoftware.com/v3/captcha/challenge/* to respond with a 404 for testing purposes.

[24294:24294:1014/233541.660122:VERBOSE2:refill_unblinded_tokens.cc(212)] GET /v1/confirmation/token/{payment_id}?nonce={nonce}
[24294:24294:1014/233541.660365:VERBOSE6:refill_unblinded_tokens.cc(216)] URL Request:
  URL: https://ads-serve.bravesoftware.com/v1/confirmation/token/0312abca-fca7-424c-a8bd-82cbd816df23?nonce=a29c7ca1-bf5c-4e90-9f00-f5200046668f
  Method: kGet
[24294:24294:1014/233541.877253:VERBOSE1:refill_unblinded_tokens.cc(226)] OnGetSignedTokens
[24294:24294:1014/233541.877375:VERBOSE6:refill_unblinded_tokens.cc(228)] URL Response:
  URL: https://ads-serve.bravesoftware.com/v1/confirmation/token/0312abca-fca7-424c-a8bd-82cbd816df23?nonce=a29c7ca1-bf5c-4e90-9f00-f5200046668f
  Response Status Code: 429
  Response: "Captcha Required Exception"

[Sneagan]

@emerick Yes, I'm trying to run this down on the ads-serve side. I've put the other details you shared in this issue as well.

[emerick]

@Sneagan fixed this via https://github.com/brave/ads-serve/issues/1743, but let's keep this issue open until we address #18278.

[emerick]

Closing this as #18278 is now merged.

[btlechowski]

Verification passed on

Brave	1.34.63 Chromium: 96.0.4664.110 (Official Build) beta (64-bit)
Revision	d5ef0e8214bc14c9b5bbf69a1515e431394c62a6-refs/branch-heads/4664@{#1283}
OS	Ubuntu 18.04 LTS

Verified test plan from the description

Verified OnGetSignedTokens no longer goes into retry loop and the captcha is immediately scheduled.

[10069:10069:1216/170117.034242:VERBOSE1:refill_unblinded_tokens.cc(188)] OnGetSignedTokens
[10069:10069:1216/170117.034283:VERBOSE6:refill_unblinded_tokens.cc(190)] URL Response:
  URL: https://ads-serve.bravesoftware.com/v1/confirmation/token/7f92cfcd-32d8-493a-a2a1-9d3c82288984?nonce=ec272e19-ecd9-46a6-9f55-86ddfcd923c0
  Response Status Code: 401
  Response: {"message":"Captcha Required","captcha_id":"1b3e210d-9f55-455a-b520-35c0f933c399"}
[10069:10069:1216/170117.034323:VERBOSE1:refill_unblinded_tokens.cc(211)] Captcha required
[10069:10069:1216/170117.034359:VERBOSE1:account.cc(217)] Captcha required to refill unblinded tokens

Logged #20131, because there is no retry logic on OnGetSignedTokens so the unblinded tokens refill is not triggered (user need to restart browser to get unblinded tokens)