JRFC 37 - Dependencies on Github #37

jbenet · 2016-08-04T02:56:34Z

Adding Dependencies to Github Issues

This is an old idea, many people have had it. I'm just laying out here as a point for discussion, so that we can build it.

Desires

I would like to be able to link github issues to each other with a "depends on" relationship.
I want graphs like this:
It wont happen on github core: Support issue dependencies isaacs/github#210
I want gantt charts

Proposal

1. Parse Dependencies from Comments

We can parse comments in an issue and detect text like:

Depends on https://github.com/ipfs/go-ipfs/issues/100

To make the dependency graph explicit. Then we can create a tool (or a site like waffle.io) that crawls a repo's issues.

2. Put dependencies on ipfs

Take the crawled dependencies and put them on IPFS (dag base)

3. Generate visual graphs

Use tools like graphviz to visualize dependency graphs.

4. Webapp showing deps

We can make a webapp that shows dependency graphs for:

a single gtihub issue
a selcted set of github issues (github search)
a github label
a github milestone
a github repo
a github org

(Note that if we implement support based on running github searches, we get most of these for free.)

Other Considerations

Tool/site should be able to crawl a set of orgs (all repos), or a set of repos.
Tool should show what's done (green), ready to be worked on (yellow), what's blocked by a dependency (red), etc. (See how other tools figure out the state / info to show)
Security: option to only look at the main issue comment/post (first, top). This way people can't troll an issue adding bogus dependencies. Requires admin powers to edit the main issue post and add the dependencies there.

The text was updated successfully, but these errors were encountered:

lidel · 2016-08-04T08:15:46Z

Sounds really useful, especially with a webapp: dependency graph could be embedded via ![](https://webapp/ghuser/project/deps.png) (just like CI badges are).

jbenet · 2016-08-05T02:20:55Z

Oh yeah, great idea!
On Thu, Aug 4, 2016 at 04:15 Marcin Rataj notifications@github.com wrote:

Sounds really useful, especially with public webapp: dependency graph
could be embedded via (just
like CI badges are).

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#37 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAIcocJNd8ZWowBiq5XV3_rcdAERSkWoks5qcZ-zgaJpZM4JcSnd
.

jbenet · 2016-08-13T22:16:01Z

Relationships:

Depends on denotes a "dependency", an issue that MUST be finished before this one can finish.
Contains denotes a "sub-issue", something that's not just a dependency, it's "part of" this issue.

Could use checkboxes to show when they're done. Would prob involve manual overhead though, unless we wanted to do a lot of scripting...

# example with checkboxes:
- [ ] Depends on: <link-to-issue>
- [ ] Contains: <link-to-issue>

jbenet · 2016-08-13T22:16:19Z

I'm actively looking for someone to make this for me, could pay you for it -- I would really benefit from this and dont have the bw to make it atm.

hackergrrl · 2016-08-15T02:36:47Z

Hey @jbenet 👋

I really like the idea, and would also like to see this happen. I spent some time today writing https://github.com/noffle/github-dependency-crawl. It crawls a github repo recursively for issue dependencies, and spits out the dependency graph. Its pretty pluggable too (you don't even need to use github as the data source, so you could e.g. pull issue data from IPFS). I'm actually using it to manage its own dependencies 😀 hackergrrl/github-dependency-crawl#7

jbenet · 2016-08-15T07:40:37Z

@noffle 👋 very nice! thanks for making that, it's is a great start. 👍 🎉

you don't even need to use github as the data source, so you could e.g. pull issue data from IPFS

may want to keep the full URL of the issue then? looks like right now it removes github.com/ and issues/ which may not be desired if issues are pulled from elsewhere? (not sure)

jbenet · 2016-08-15T07:57:28Z

graphviz on the web:

https://github.com/mdaines/viz.js/ and https://mdaines.github.io/viz.js/ <--- this looks to be the best bet for graphviz
http://www.webgraphviz.com/ <-- not sure if it's the same. no links to code.
https://github.com/mstefaniuk/graph-viz-d3-js
http://graphviz.it/#/gallery/longflat.gv

(3 years ago i really wanted these and they didnt exist. very happy they do now)

other libs:

not related, but this is pretty cool:

https://www.websequencediagrams.com/

jbenet · 2016-08-15T08:01:04Z

This seems to be the trac plugin's code:

https://trac-hacks.org/browser/tracticketdepgraphplugin/0.11/depgraph#templates
this produces the graphivz: https://trac-hacks.org/browser/tracticketdepgraphplugin/0.11/depgraph/depgraph.py
should be able to port that to using @noffle's module

hackergrrl · 2016-08-15T17:32:44Z

may want to keep the full URL of the issue then? looks like right now it removes github.com/ and issues/ which may not be desired if issues are pulled from elsewhere? (not sure)

Without getting into detail here, the canonical string should be sufficient to let you deduce the source URL from the data source of choice (owner/repo/issue has a well-documented mapping to an API endpoint, and if it's on IPFS that mapping should also be well-documented, e.g. /ipfs/HASH/gh/owner/repo/issue).

jbenet · 2016-08-17T09:55:48Z

I took a stab at the dot generation. this is a super basic version: https://github.com/jbenet/issue-dep-graph

> issue-dep-dot https://github.com/noffle/github-dependency-crawl
digraph deps {
    noffle_github_dependency_crawl_12 [label="noffle/github-dependency-crawl/12"];
    noffle_github_dependency_crawl_11 [label="noffle/github-dependency-crawl/11"];
    noffle_github_dependency_crawl_10 [label="noffle/github-dependency-crawl/10"];
    noffle_github_dependency_crawl_9 [label="noffle/github-dependency-crawl/9"];
    noffle_github_dependency_crawl_8 [label="noffle/github-dependency-crawl/8"];
    noffle_github_dependency_crawl_7 [label="noffle/github-dependency-crawl/7"];
    noffle_github_dependency_crawl_4 [label="noffle/github-dependency-crawl/4"];
    noffle_github_dependency_crawl_5 [label="noffle/github-dependency-crawl/5"];
    noffle_github_dependency_crawl_6 [label="noffle/github-dependency-crawl/6"];
    noffle_github_dependency_crawl_3 [label="noffle/github-dependency-crawl/3"];
    noffle_ipget_18 [label="noffle/ipget/18"];
    noffle_github_dependency_crawl_2 [label="noffle/github-dependency-crawl/2"];
    noffle_github_dependency_crawl_1 [label="noffle/github-dependency-crawl/1"];
    ipfs_ipget_24 [label="ipfs/ipget/24"];
    ipfs_ipget_26 [label="ipfs/ipget/26"];
    ipfs_ipget_20 [label="ipfs/ipget/20"];
    ipfs_ipget_21 [label="ipfs/ipget/21"];

    noffle_github_dependency_crawl_7 -> noffle_github_dependency_crawl_4;
    noffle_github_dependency_crawl_7 -> noffle_github_dependency_crawl_5;
    noffle_github_dependency_crawl_7 -> noffle_github_dependency_crawl_6;
    noffle_github_dependency_crawl_3 -> noffle_ipget_18;
    noffle_github_dependency_crawl_2 -> noffle_github_dependency_crawl_3;
    noffle_github_dependency_crawl_1 -> noffle_github_dependency_crawl_2;
    noffle_github_dependency_crawl_1 -> noffle_github_dependency_crawl_3;
    noffle_ipget_18 -> ipfs_ipget_24;
    noffle_ipget_18 -> ipfs_ipget_26;
    noffle_ipget_18 -> ipfs_ipget_20;
    noffle_ipget_18 -> ipfs_ipget_21;
}

zjiekai · 2016-08-18T12:19:06Z

Is gantt charts something like this(http://dhtmlx.com/docs/products/dhtmlxGantt/01_basic.html)?

jbenet · 2016-08-19T00:08:47Z

@zjiekai yeah! I'd like to generate them from github issues that have some dependency (and "containing") structure.

It's hard to figure out the time it will take, but even charting out the dependenceis, some basic assumptions (like N hrs per issue) may generate a really good perspective, and logging the past accurately would be super helpful.

"containing issues" means one issue "contains" another (more than a simple dependency, it's treating an issue like a bundle of other issues (milestones or epics). GH Milestones don't fit all needs because GH Milestones are per repo and not across repos. (some prefer milestones backed by an issue)

jbenet · 2016-08-21T07:43:04Z

depviz.mocks.pdf

preview:

jbenet · 2016-11-19T07:11:26Z

I have created https://github.com/jbenet/depviz to track work on this webapp.

burdges · 2016-11-22T18:01:16Z

I'd worry less about the charts per se than simply about allowing users to edit the information sanely.

In particular, if you're discussing github issues, then an arrow might just be someone's opinion that resolving an issue in necessary for some other issue. If however either issue got resolved somewhat differently, then that arrow might not make sense.

Arrow existence and direction still remains more relevant than just an internal github backwards link though because it'll help people understand the discussion faster. And help newbies pick issues to work on.

jbenet · 2016-12-05T05:06:13Z

@burdges yeah makes sense. i think the flexibility we have with marking depends on X in an issue is enough here-- people will be able to decide on that. Toggling modes so that only markings in the initial issue comment counts (i.e. must have edit perms) can strike the right balance for what a project's thoughts are on what counts as a dependency (whether to be strict, or loose)

jbenet · 2016-12-05T10:14:06Z

Quick update here: the excellent @wking and I have been making some headway and progress in the implementaiton of depviz (impl all by wking). Follow the progress: https://github.com/jbenet/depviz or try it out: https://jbenet.github.io/depviz

examples:

SparksFyz · 2016-12-27T13:31:38Z

HI, I have a question. When you use viz.js, the node in viz is svg, but deps node here is very complex(like embeded html) and interactive(embeded buttons...), how your guys solve this issue?

wking · 2016-12-27T20:36:16Z

On Tue, Dec 27, 2016 at 05:31:39AM -0800, Sparks 范宜臻 wrote: … the node in viz is svg, but deps node here is very complex(like embeded html) and interactive(embeded buttons...), how your guys solve this issue?

The nodes are rectangular, so all depviz uses viz.js for positioning rectangular nodes (and eventually for routing edges between them, jbenet/depviz#24). The graph-positioning engine doesn't have to care about who's drawing the node content. depviz currently uses SVG node content as well (which supports embedded buttons, etc.), but if you wanted you could use foreignObject and HTML inside the nodes [1]. [1]: https://developer.mozilla.org/en-US/docs/Web/SVG/Element/foreignObject

SparksFyz · 2016-12-28T06:27:16Z

thanks~ I understand what you mean. @wking

moul · 2017-01-05T16:36:48Z

Something new from GH: https://github.com/blog/2300-visualize-your-project-s-community

wking · 2017-01-05T18:00:39Z

On Thu, Jan 05, 2017 at 08:36:50AM -0800, Manfred Touron wrote: Something new from GH: https://github.com/blog/2300-visualize-your-project-s-community

That is for dependencies between projects. This issue and depviz are for dependencies between issues. The latter are likely to be a more-specific version of the former (e.g. “graphql-parallel depends on graphql-ruby” is a project-level dependency and “graphql-parallel#123 depends on graphql-ruby#456” is an issue-level dependency).

hackergrrl mentioned this issue Aug 15, 2016

Sprint: Aug 8 → Aug 14 ipfs/team-mgmt#133

Closed

jbenet mentioned this issue Aug 29, 2016

Define options for the Mechanics of Tracking Project Milestones ipfs/team-mgmt#154

Closed

jbenet mentioned this issue Nov 19, 2016

depviz v0: single page rendering jbenet/depviz#1

Closed

2 tasks

machawk1 mentioned this issue Jun 29, 2017

Use docker for testing communication between ipwb instances oduwsdl/ipwb#209

Open

jbenet mentioned this issue Jul 13, 2019

Project dependency tracking ipfs/notes#385

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

JRFC 37 - Dependencies on Github #37

JRFC 37 - Dependencies on Github #37

jbenet commented Aug 4, 2016

lidel commented Aug 4, 2016 •

edited

Loading

jbenet commented Aug 5, 2016

jbenet commented Aug 13, 2016

jbenet commented Aug 13, 2016 •

edited

Loading

hackergrrl commented Aug 15, 2016

jbenet commented Aug 15, 2016

jbenet commented Aug 15, 2016 •

edited

Loading

jbenet commented Aug 15, 2016

hackergrrl commented Aug 15, 2016

jbenet commented Aug 17, 2016 •

edited

Loading

zjiekai commented Aug 18, 2016

jbenet commented Aug 19, 2016

jbenet commented Aug 21, 2016

zjiekai commented Aug 21, 2016 •

edited

Loading

hackergrrl commented Aug 21, 2016 via email

jbenet commented Aug 29, 2016

zjiekai commented Aug 30, 2016 •

edited

Loading

serapath commented Aug 30, 2016

jbenet commented Aug 30, 2016 •

edited

Loading

jbenet commented Nov 15, 2016

jbenet commented Nov 19, 2016

jbenet commented Nov 19, 2016 •

edited

Loading

burdges commented Nov 22, 2016

jbenet commented Dec 5, 2016

jbenet commented Dec 5, 2016 •

edited

Loading

SparksFyz commented Dec 27, 2016

wking commented Dec 27, 2016 via email

SparksFyz commented Dec 28, 2016 •

edited

Loading

moul commented Jan 5, 2017

wking commented Jan 5, 2017 via email

JRFC 37 - Dependencies on Github #37

JRFC 37 - Dependencies on Github #37

Comments

jbenet commented Aug 4, 2016

Adding Dependencies to Github Issues

Desires

Proposal

1. Parse Dependencies from Comments

2. Put dependencies on ipfs

3. Generate visual graphs

4. Webapp showing deps

Other Considerations

lidel commented Aug 4, 2016 • edited Loading

jbenet commented Aug 5, 2016

jbenet commented Aug 13, 2016

jbenet commented Aug 13, 2016 • edited Loading

hackergrrl commented Aug 15, 2016

jbenet commented Aug 15, 2016

jbenet commented Aug 15, 2016 • edited Loading

jbenet commented Aug 15, 2016

hackergrrl commented Aug 15, 2016

jbenet commented Aug 17, 2016 • edited Loading

zjiekai commented Aug 18, 2016

jbenet commented Aug 19, 2016

jbenet commented Aug 21, 2016

zjiekai commented Aug 21, 2016 • edited Loading

hackergrrl commented Aug 21, 2016 via email

jbenet commented Aug 29, 2016

zjiekai commented Aug 30, 2016 • edited Loading

serapath commented Aug 30, 2016

jbenet commented Aug 30, 2016 • edited Loading

jbenet commented Nov 15, 2016

jbenet commented Nov 19, 2016

depviz.mocks.pdf

jbenet commented Nov 19, 2016 • edited Loading

I have created https://github.com/jbenet/depviz to track work on this webapp.

burdges commented Nov 22, 2016

jbenet commented Dec 5, 2016

jbenet commented Dec 5, 2016 • edited Loading

SparksFyz commented Dec 27, 2016

wking commented Dec 27, 2016 via email

SparksFyz commented Dec 28, 2016 • edited Loading

moul commented Jan 5, 2017

wking commented Jan 5, 2017 via email

lidel commented Aug 4, 2016 •

edited

Loading

jbenet commented Aug 13, 2016 •

edited

Loading

jbenet commented Aug 15, 2016 •

edited

Loading

jbenet commented Aug 17, 2016 •

edited

Loading

zjiekai commented Aug 21, 2016 •

edited

Loading

zjiekai commented Aug 30, 2016 •

edited

Loading

jbenet commented Aug 30, 2016 •

edited

Loading

jbenet commented Nov 19, 2016 •

edited

Loading

jbenet commented Dec 5, 2016 •

edited

Loading

SparksFyz commented Dec 28, 2016 •

edited

Loading