Created task fragments RPC #4120

kmazurek · 2019-04-15T12:42:55Z

Resolves: #4041

Adds an RPC endpoint which returns fragments for a given rendering task.
A fragment is a collection of subtasks which all refer to a common part
of the whole task (currently it is the start_task index from subtask
extra_data).

Adds an RPC endpoint which returns fragments for a given rendering task. A fragment is a collection of subtasks which all refer to a common part of the whole task (currently it is the start_task index from subtask extra_data).

kmazurek · 2019-04-15T12:56:50Z

Sample JSON response from the RPC in its current form (subtask_count is 5, only one subtask has been assigned):

{
    "1": [
        {
            "node_id": "da0853e0ec7477edbbf5fdd036093066616c5763f5aa6a670742617dc11f0e325846c53dbccba9f3aada8c6dceecf2acbbe0c44f0bfe2ec87db7cf53553a19a8",
            "node_name": "",
            "progress": 1.0,
            "results": [
                "/home/.../ComputerRes/31d06954-5c6f-11e9-8455-03db94344a05/tmp/small_shark100_10001.png",
                "/home/.../ComputerRes/31d06954-5c6f-11e9-8455-03db94344a05/tmp/small_shark100_10002.png"
            ],
            "status": "Finished",
            "stderr": "/home/.../ComputerRes/31d06954-5c6f-11e9-8455-03db94344a05/tmp/47cb889c-5c6f-11e9-8d2d-03db94344a05/stderr.log",
            "stdout": "/home/.../ComputerRes/31d06954-5c6f-11e9-8455-03db94344a05/tmp/47cb889c-5c6f-11e9-8d2d-03db94344a05/stdout.log",
            "subtask_id": "47cb889c-5c6f-11e9-8d2d-03db94344a05",
            "time_remaining": 0.0,
            "time_started": 1554996863.0686917
        }
    ],
    "2": [],
    "3": [],
    "4": [],
    "5": []
}

golem/client.py

shadeofblue

$abstract-paper-fracture-detail-lasercut-paper-by-robin-hanhart$

shadeofblue · 2019-04-16T13:06:06Z

golem/task/rpc.py

@@ -5,7 +5,7 @@
 import logging
 import os.path
 import re
-import typing
+from typing import Any, Dict, Generator, Iterable, List, Optional, Tuple


why?... maybe it's just me but I find typing.X immediately more readable than just X ...

I find prefixing overly verbose, especially for collection type signatures (when the prefix gets repeated multiple times). I suppose it's mostly subjective, but from typing seems to be more widely used in our codebase (113 vs 29 files).

I second @shadeofblue in that. IMO this is a case of namespace pollution.

golem/task/rpc.py

codecov · 2019-04-16T13:26:43Z

Codecov Report

Merging #4120 into develop will increase coverage by 0.03%.
The diff coverage is 100%.

@@             Coverage Diff             @@
##           develop    #4120      +/-   ##
===========================================
+ Coverage    88.64%   88.67%   +0.03%     
===========================================
  Files          214      214              
  Lines        18682    18697      +15     
===========================================
+ Hits         16561    16580      +19     
+ Misses        2121     2117       -4

jiivan

Added remote procedure looks good 👍

I don't like changing clean namespace into this polluted... thing. :/

>>> import this
The Zen of Python, by Tim Peters
[...]
Namespaces are one honking great idea -- let's do more of those!

jiivan · 2019-04-17T09:20:44Z

golem/task/rpc.py

@@ -5,7 +5,7 @@
 import logging
 import os.path
 import re
-import typing
+from typing import Any, Dict, Generator, Iterable, List, Optional, Tuple


I second @shadeofblue in that. IMO this is a case of namespace pollution.

jiivan

Created task fragments RPC

1284f2d

Adds an RPC endpoint which returns fragments for a given rendering task. A fragment is a collection of subtasks which all refer to a common part of the whole task (currently it is the start_task index from subtask extra_data).

kmazurek requested review from jiivan and shadeofblue April 15, 2019 12:42

kmazurek self-assigned this Apr 15, 2019

ghost added the in progress label Apr 15, 2019

jiivan reviewed Apr 15, 2019

View reviewed changes

golem/client.py Outdated Show resolved Hide resolved

Moved fragments endpoint to client RPC provider

4e17ffe

shadeofblue approved these changes Apr 16, 2019

View reviewed changes

Added error messages and documentation

3faa41f

jiivan reviewed Apr 17, 2019

View reviewed changes

Brought back typing namespace

3c9b5b6

jiivan approved these changes Apr 17, 2019

View reviewed changes

kmazurek added 2 commits April 17, 2019 12:05

Merge branch 'develop' into task-fragments-rpc

01474a6

lint

a6bdef7

kmazurek merged commit f54145c into develop Apr 17, 2019

ghost removed the in progress label Apr 17, 2019

kmazurek deleted the task-fragments-rpc branch April 17, 2019 10:41

maaktweluit mentioned this pull request Apr 18, 2019

Cleaned up typing for linter errors in taskkeeper #4122

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Created task fragments RPC #4120

Created task fragments RPC #4120

kmazurek commented Apr 15, 2019

kmazurek commented Apr 15, 2019

shadeofblue left a comment

shadeofblue Apr 16, 2019

kmazurek Apr 16, 2019

jiivan Apr 17, 2019

kmazurek Apr 17, 2019

codecov bot commented Apr 16, 2019 •

edited

Loading

jiivan left a comment

jiivan Apr 17, 2019

jiivan left a comment

Created task fragments RPC #4120

Created task fragments RPC #4120

Conversation

kmazurek commented Apr 15, 2019

kmazurek commented Apr 15, 2019

shadeofblue left a comment

Choose a reason for hiding this comment

shadeofblue Apr 16, 2019

Choose a reason for hiding this comment

kmazurek Apr 16, 2019

Choose a reason for hiding this comment

jiivan Apr 17, 2019

Choose a reason for hiding this comment

kmazurek Apr 17, 2019

Choose a reason for hiding this comment

codecov bot commented Apr 16, 2019 • edited Loading

Codecov Report

jiivan left a comment

Choose a reason for hiding this comment

jiivan Apr 17, 2019

Choose a reason for hiding this comment

jiivan left a comment

Choose a reason for hiding this comment

codecov bot commented Apr 16, 2019 •

edited

Loading