GTP tournament game expansion

Proposed GTP tournament game expansion ver 0.1

Intent

To see the readings and evaluations of both sides in Go AI tournaments and CGOS.

Proposed Extension

Extended command switching by list_commands is used in KGS/GoGui/Lizzie and other extended support, so it should be followed.
GTP multi-line send/receive is also used in list_commands final_status_list showboard, so it is expected to be less problematic for both server and client.
Add cgos-genmove_analyze based on lz-genmove_analyze specification.
cgos clients and GTP-NNGS bridges will use list_commands to determine if they support extensions.
For software that supports extended commands, use cgos-genmove_analyze instead of genmove to request them to start.
cgos-genmove_analyze <color>
- Return analysis in a single line json format and play <coordinate>.
- ownership is encoded to single character per point. -1.0 = 'A' ... 0.0 = 'f' ... 1.0 = '+'

def encodeOwnership(ownership: List[float]) -> str:
    CHARS = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+"

    def encode(r: float) -> str:
        i = max(0, min((r + 1) / 2, 1))
        # i = r
        return CHARS[round(i * 62)]

    return "".join([encode(f) for f in ownership])

JSON example (Response doesn't actually include line breaks)

{
  "winrate": 0.13,
  "score": -11.5,
  "visits": 258,
  "moves": [
    {
      "move": "O15",
      "winrate": 0.1355,
      "score": -11.2142,
      "prior": 0.831028,
      "pv": "P17 O18 N17 O8 P6 N7 Q7 O6 P5",
      "visits": 189
    },
    {
      "move": "M16",
      "winrate": 0.111084,
      "score": -12.2514,
      "prior": 0.123766,
      "pv": "N17 O15 L16 O18",
      "visits": 69
    }
  ],
  "ownership": "y35678961tmifdgkpwxr6575+9+zrkgglgkk1xsB78ge+8xndbPkNeb1yNBBBBBC++pfULKV99rxEBGLSXXU+paO++8+rusDDFGljrU+iNn+88+5kmDDDED99ppVep99++99gDDDFEB908fIk9+E++SdDDDDEB972ocGF+E9HSWCCBCABB9988rHEE9HHYCCCCCDBB9rvcDRMHI5sCCCDDFHIIwhdTkiI756CCCDEGKQdecYVKxl775DBCDFIMQVYXTEZQ3243CCCDEJPRDTPMHCWoy4d+AAAAAQffjBBBB222CX+++++RDgh++U++BCCCE+9++97++5898+87CDDD+9998888778889kQDCC",
  "comment": "\\u3053\\u306e\\u7881\\u3092\\u6295\\u4e86\\u3059\\u308b\\u306e\\u3082\\u7acb\\u3066\\u76f4\\u3059\\u306e\\u3082\\u3053\\u3053\\u306b\\u3044\\u308b\\u30aa\\u30ec\\u3057\\u304b\\u3044\\u306a\\u3044\\uff01"
}

GTP example

cgos-genmove_analyze w
=
{"moves": [{"move": "O15", "winrate": 0.1355, "score": -11.2142, "prior": 0.831028, "pv": "P17 O18 N17 O8 P6 N7 Q7 O6 P5", "visits": 189}, {"move": "M16", "winrate": 0.111084, "score": -12.2514, "prior": 0.123766, "pv": "N17 O15 L16 O18"}, {"move": "E14", "winrate": 0.0925099, "score": -11.2273, "prior": 0.0296406, "pv": "D14 O15", "visits": 69}], "ownership": "y35678961tmifdgkpwxr6575+9+zrkgglgkk1xsB78ge+8xndbPkNeb1yNBBBBBC++pfULKV99rxEBGLSXXU+paO++8+rusDDFGljrU+iNn+88+5kmDDDED99ppVep99++99gDDDFEB908fIk9+E++SdDDDDEB972ocGF+E9HSWCCBCABB9988rHEE9HHYCCCCCDBB9rvcDRMHI5sCCCDDFHIIwhdTkiI756CCCDEGKQdecYVKxl775DBCDFIMQVYXTEZQ3243CCCDEJPRDTPMHCWoy4d+AAAAAQffjBBBB222CX+++++RDgh++U++BCCCE+9++97++5898+87CDDD+9998888778889kQDCC", "comment": "\\u3053\\u306e\\u7881\\u3092\\u6295\\u4e86\\u3059\\u308b\\u306e\\u3082\\u7acb\\u3066\\u76f4\\u3059\\u306e\\u3082\\u3053\\u3053\\u306b\\u3044\\u308b\\u30aa\\u30ec\\u3057\\u304b\\u3044\\u306a\\u3044\\uff01"}
play O15

JSON Key
- winrate : Win-rate from the engine's point of view. +1 is win, 0 is loss.
- score : Sum of territory points including komi from the engine's point of view. If black is +11 on the board with 7.5 komi, +3.5 for black program, -3.5 for white program.
- pv : Best line. Excluding the first move. If "O15 P17 O18" is principal variation, then "P17 O18".
- ownership: Territory. Represented by a string of 361(19x19) characters (consist of 63 different characters). If a point is 100% your territory, it is +1. If 100% opponent, it is -1. Sample code converts this a character. From top-left(A19) to top-right(T19), and last is bottom-right(T1).
- visits : Number of times that a move has been explored.
- prior : A move probability if using Policy Network.
- comment : UTF-8 comment. It will be displayed in the comments section of WGo.

Considerations.

Support lz-genmove_analyze interval and lz-analyze for real-time read notifications?
- The initial version does not support interval, only cgos-genmove_analyze <color>.
Should we follow LeelaZero and KataGo in sending attributes or minimize them?
1. move, pv, winrate
2. score, ownership, lcb, ucb, visits
3. ...
Generalize so that arbitrary attributes can be sent?
- Especially PV is currently space-delimited, so it is not possible to send arbitrary attributes with the lz-genmove_analyze specification.

Existing implementation

Leela Zero

Added as GTP extension to Leela Zero for displaying readings in Lizzie. The lz-genmove_analyze command returns =[ID] immediately as a response and notifies readings and moves after the search is finished.

The lz-analyze command is in analysis mode, and the readings are periodically notified. The analysis mode is terminated when another GTP command is received.

The response is based on UCI protocol.

Implementation by bittsitt Extend GTP to add support for displaying real time winrates and variations

GTP command format lz-genmove-analyze <color> <interval in centis>

The actual response format has been changed from the pull request documentation and is info <attribute name1> <attribute value1> <attribute name2> <attribute value2> ...

move: coordinate (GTP formant)
visits: number
winrate: [0, 10000]
prior: prior policy [0, 10000]
lcb: [0, 10000]
pv: Arary of coordinates (GTP format)

Leela Zero output no info tag without interval.

Example

lz-genmove_analyze b 1000
=
info move D16 visits 6 winrate 4197 prior 1760 lcb 3838 order 0 pv D16 Q16 D4 Q4 info move D4 visits 4 winrate 4198 prior 1730 lcb 2346 order 1 pv D4 Q16 Q4 D16 info move Q5 visits 4 winrate 4212 prior 35 lcb 2273 order 2 pv Q5 Q16 D4 R3 info move D3 visits 3 winrate 4223 prior 248 lcb 0 order 3 pv D3 Q16 R4 info move R16 visits 3 winrate 4221 prior 247 lcb 0 order 4 pv R16 D4 Q3 info move D17 visits 3 winrate 4229 prior 249 lcb 0 order 5 pv D17 Q4 R16 info move C4 visits 3 winrate 4215 prior 244 lcb 0 order 6 pv C4 Q16 D17 info move Q3 visits 3 winrate 4225 prior 249 lcb 0 order 7 pv Q3 D16 C4 info move Q17 visits 3 winrate 4242 prior 246 lcb 0 order 8 pv Q17 D4 C16 info move R4 visits 3 winrate 4220 prior 253 lcb 0 order 9 pv R4 D16 Q17 info move R17 visits 3 winrate 4162 prior 24 lcb 0 order 10 pv R17 D4 D16 info move Q4 visits 3 winrate 4206 prior 1714 lcb 0 order 11 pv Q4 D16 D4 info move C5 visits 3 winrate 4158 prior 12 lcb 0 order 12 pv C5 D16 Q4 info move Q16 visits 3 winrate 4219 prior 1745 lcb 0 order 13 pv Q16 D4 D16 info move D15 visits 3 winrate 4232 prior 33 lcb 0 order 14 pv D15 D4 Q16 info move C16 visits 3 winrate 4224 prior 249 lcb 0 order 15 pv C16 Q4 D3 info move C15 visits 3 winrate 4177 prior 11 lcb 0 order 16 pv C15 Q17 D17 info move P4 visits 3 winrate 4239 prior 35 lcb 0 order 17 pv P4 D4 Q16 info move C3 visits 3 winrate 4196 prior 23 lcb 0 order 18 pv C3 Q16 Q4 info move E16 visits 3 winrate 4238 prior 33 lcb 0 order 19 pv E16 Q16 D4 info move R5 visits 3 winrate 4189 prior 11 lcb 0 order 20 pv R5 D3 Q3 info move Q15 visits 3 winrate 4220 prior 36 lcb 0 order 21 pv Q15 Q4 D16 info move E17 visits 3 winrate 4199 prior 12 lcb 0 order 22 pv E17 C4 C16 info move E3 visits 3 winrate 4206 prior 11 lcb 0 order 23 pv E3 C16 C4 info move P16 visits 3 winrate 4233 prior 35 lcb 0 order 24 pv P16 D16 Q4 info move E4 visits 3 winrate 4225 prior 34 lcb 0 order 25 pv E4 Q4 D16 info move P3 visits 3 winrate 4135 prior 11 lcb 0 order 26 pv P3 R16 D4 info move R3 visits 3 winrate 4176 prior 23 lcb 0 order 27 pv R3 D16 D4 info move R15 visits 3 winrate 4101 prior 12 lcb 0 order 28 pv R15 Q4 D16 info move C17 visits 3 winrate 4155 prior 25 lcb 0 order 29 pv C17 Q4 Q16 info move D5 visits 3 winrate 4234 prior 33 lcb 0 order 30 pv D5 D16 Q4 info move P17 visits 3 winrate 4083 prior 12 lcb 0 order 31 pv P17 D16 Q4
play D16
info move Q4 visits 14 winrate 5783 prior 3408 lcb 5713 order 0 pv Q4 Q16 D4 R6 C17 D17 info move D4 visits 9 winrate 5774 prior 2593 lcb 5611 order 1 pv D4 Q16 Q4 R6 C17 D17 C16 info move Q16 visits 10 winrate 5774 prior 2616 lcb 5601 order 2 pv Q16 D4 Q4 O17 C3 C4 D3 info move Q5 visits 4 winrate 5502 prior 8 lcb 2954 order 3 pv Q5 R3 P3 info move C4 visits 3 winrate 5693 prior 276 lcb 0 order 4 pv C4 E3 Q16 info move R16 visits 3 winrate 5683 prior 33 lcb 0 order 5 pv R16 P17 D4 info move C17 visits 3 winrate 5596 prior 8 lcb 0 order 6 pv C17 C16 D17 info move D5 visits 3 winrate 5481 prior 9 lcb 0 order 7 pv D5 Q4 Q16 info move R4 visits 3 winrate 5655 prior 117 lcb 0 order 8 pv R4 D4 Q16 info move Q3 visits 3 winrate 5688 prior 110 lcb 0 order 9 pv Q3 Q16 D4 info move F17 visits 3 winrate 5611 prior 8 lcb 0 order 10 pv F17 Q16 D4 info move Q15 visits 3 winrate 5483 prior 8 lcb 0 order 11 pv Q15 Q4 D4 info move E4 visits 3 winrate 5438 prior 9 lcb 0 order 12 pv E4 Q4 Q16 info move C14 visits 3 winrate 5573 prior 8 lcb 0 order 13 pv C14 D4 Q16 info move R15 visits 3 winrate 5223 prior 3 lcb 0 order 14 pv R15 Q4 D4 info move P3 visits 3 winrate 5449 prior 4 lcb 0 order 15 pv P3 R3 Q5 info move P16 visits 3 winrate 5459 prior 9 lcb 0 order 16 pv P16 Q4 D4 info move C5 visits 3 winrate 5147 prior 4 lcb 0 order 17 pv C5 Q4 Q16 info move Q17 visits 2 winrate 5689 prior 283 lcb 0 order 18 pv Q17 R15 info move D3 visits 2 winrate 5664 prior 34 lcb 0 order 19 pv D3 C5 info move C3 visits 2 winrate 5619 prior 13 lcb 0 order 20 pv C3 Q16 info move R17 visits 2 winrate 5614 prior 13 lcb 0 order 21 pv R17 D4 info move R3 visits 2 winrate 5574 prior 12 lcb 0 order 22 pv R3 Q16 info move P4 visits 2 winrate 5479 prior 9 lcb 0 order 23 pv P4 R3 info move R5 visits 2 winrate 5430 prior 4 lcb 0 order 24 pv R5 R3 info move E3 visits 2 winrate 5378 prior 3 lcb 0 order 25 pv E3 Q4 info move P17 visits 2 winrate 5307 prior 4 lcb 0 order 26 pv P17 Q4 info move C16 visits 2 winrate 4959 prior 2 lcb 0 order 27 pv C16 C17

KataGo

KataGo implements kata-genmove_analyze and kata-analyze as GTP extensions.

The basic behavior follows lz-genmove_analyze and lz-analyze. KataGo-specific information such as score and ground information, which Leela Zero did not support, are added. Win rate is float value. KataGo output analyze info even if no interval specified.

winrate: [0.0, 1.0]
prior: prior policy [0.0, 1.0]
lcb: [0.0, 1.0]
utility
scoreMean
scoreStdev
scoreLead
scoreSelfplay
utilityLcb
weight
isSymmetryOf
order
pvEdgeVisits
movesOwnership
movesOwnershipStdev
ownership
ownershipStdev

KataGo GTP Extensions

Example

kata-genmove_analyze
=
info move D4 visits 695 utility 0.111301 winrate 0.555005 scoreMean 1.34153 scoreStdev 24.1778 scoreLead 1.34153 scoreSelfplay 1.34153 prior 0.383687 lcb 0.55354 utilityLcb 0.107201 weight 695 order 0 pv D4 D16 Q4 R3 R4 Q3 O3 P3 P4 O2 C17 C16 D17 F17 info move D16 visits 292 utility 0.105322 winrate 0.552243 scoreMean 1.15246 scoreStdev 23.6713 scoreLead 1.15246 scoreSelfplay 1.15246 prior 0.275052 lcb 0.550414 utilityLcb 0.100203 weight 291.642 order 1 pv D16 D4 Q4 R3 Q3 R4 R6 R5 Q5 S6 R17 Q17 R16 R14 info move Q4 visits 292 utility 0.105322 winrate 0.552243 scoreMean 1.15246 scoreStdev 23.6713 scoreLead 1.15246 scoreSelfplay 1.15246 prior 0.275052 lcb 0.550414 utilityLcb 0.100203 weight 291.642 isSymmetryOf D16 order 2 pv Q4 D4 D16 C17 C16 D17 F17 E17 E16 F18 R17 R16 Q17 O17 info move C4 visits 189 utility 0.113599 winrate 0.554989 scoreMean 1.53496 scoreStdev 24.4114 scoreLead 1.53496 scoreSelfplay 1.53496 prior 0.0047951 lcb 0.551615 utilityLcb 0.104152 weight 189 order 3 pv C4 Q4 D16 E3 D5 C17 D17 C16 C14 C15 D15 B14 R3 R4 info move D3 visits 189 utility 0.113599 winrate 0.554989 scoreMean 1.53496 scoreStdev 24.4114 scoreLead 1.53496 scoreSelfplay 1.53496 prior 0.0047951 lcb 0.551615 utilityLcb 0.104152 weight 189 isSymmetryOf C4 order 4 pv D3 D16 Q4 C5 E4 R3 R4 Q3 O3 P3 P4 O2 C17 D17 info move C16 visits 42 utility 0.0997575 winrate 0.548788 scoreMean 1.15864 scoreStdev 24.2863 scoreLead 1.15864 scoreSelfplay 1.15864 prior 0.00384986 lcb 0.543774 utilityLcb 0.0857202 weight 42 order 5 pv C16 D4 Q4 E17 D14 R3 Q3 R4 R5 info move Q3 visits 42 utility 0.0997575 winrate 0.548788 scoreMean 1.15864 scoreStdev 24.2863 scoreLead 1.15864 scoreSelfplay 1.15864 prior 0.00384986 lcb 0.543774 utilityLcb 0.0857202 weight 42 isSymmetryOf C16 order 6 pv Q3 D4 D16 R5 O4 C17 C16 D17 E17 info move D17 visits 44 utility 0.0999837 winrate 0.548683 scoreMean 1.18234 scoreStdev 24.3054 scoreLead 1.18234 scoreSelfplay 1.18234 prior 0.00187856 lcb 0.543961 utilityLcb 0.0867636 weight 44 order 7 pv D17 D4 Q4 C15 F16 R3 Q3 R4 R5 S5 info move R4 visits 44 utility 0.0999837 winrate 0.548683 scoreMean 1.18234 scoreStdev 24.3054 scoreLead 1.18234 scoreSelfplay 1.18234 prior 0.00187856 lcb 0.543961 utilityLcb 0.0867636 weight 44 isSymmetryOf D17 order 8 pv R4 D4 D16 P3 Q6 C17 C16 D17 E17 E18
play D4

shogi-server extension

This extension is used in computer Shogi to show evaluation value graphs and PV in Floodgate and denryu-sen (世界将棋AI 電竜戦) games.

Extension to record evaluation values and yomisuji

This is implemented in the form of a comment sent by the client to the server when the client sends a move in the CSA protocol.

+7776FU,'jouseki.

In this case, a comment in a specific format starting with ** is interpreted as a space-separated record of the evaluation value and the readings. +7776FU,'** 30 -3334FU +2726FU

TODO

Investigate compatibility with CGOS implementation.
- CGOS server
  - Minimal Python server implementation completed.
  - No Tcl/Tk viewer yet.
- Async realtime notification seems to require many design changes.
Investigate compatibility with NNGS implementation

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GTP tournament game expansion

Proposed GTP tournament game expansion ver 0.1

Intent

Proposed Extension

Considerations.

Existing implementation

Leela Zero

Example

KataGo

Example

shogi-server extension

TODO

Clone this wiki locally