Look at OFVChallenger

[kongzii]

After a few days:

• resolve some markets manually to check his accuracy
• verify it's claiming bonds back
• check his losses
• ???

[kongzii]

resolve some markets manually to check his accuracy

I manually annotated 40 questions in Langfuse:

• 2 are resolved incorrectly, so we have 95% accuracy
• 7 are resolved differently from Olas' resolver, from these, 5 are correct
  • so Olas' resolver has 87.5% accuracy

Unfortunately, no human challenged these 2 wrong answers. I caught the second one just in time and corrected it by myself, but the first one is now finalized wrongly.

I will check another batch of questions again next week.

[kongzii]

verify it's claiming bonds back
check his losses

The agent is getting its xDai back, for example, this transaction https://gnosisscan.io/tx/0x0a8f6a00388be5479600cb2503fb6dabef77c78eb1483c9c3bdda9855a2cb67a.

Just that it loses 0.001 xDai every time it posts the same answer as already posted.

[kongzii]

I'm regularly checking the ofvresolver accuracy and it's around 90%. i think this can be closed now