[feature] More control over wire parameters

[Tyler-Ward]

Based on discussions in #11 it might be worth adding the ability alter more properties of individual wires in a cable or bundle e.g. guage (an example would be a usb cable with two thinner wires and two power wires). This will obviously add some additional complexity to both the definition format and to rendering cables/bundles with multiple parameters.

This feature could also allow for sub wires to be templated which will make reusing wires in multiple bundles or several times in the same bundle easier. A modified format concept from #11 is shown below for reference.

templates:
  - &blueWire
    part_number: 12345
    color: BU

  W1:
    category: bundle
    length: 0.3
    wireinfo:
      1:
        part_number: 12344
        color: BK
      2: <<: &blueWire
      3: <<: &blueWire
      4:
        part_number: 12343
        color: WH

[formatc1702]

A simple solution might be to allow every parameter of the cable/bundle to be either a string / int (applying it to all wires) or a list (length = wirecount),similar to how e.g. part numbers for bundles are handled now. It would be slightly more concise, and, when formatted cleverly, easier to read?

W1:
  category: bundle
  length: 0.3 # maybe even this could become a list? does it make sense?
  colors:                   [BK,    BU,    BU,    RD   ]
  gauge:                    [0.25,  0.25,  0.5,   0.5  ]
  manufacturer_part_number: [12344, 12345, 12346, 12343]

Templating would not be as easy, though. Just a thought.

[tim292stro]

Parameterizing is one of the major convenience factors of tools found in the commercial marketplace. For example being able to say "This part" then have that part be defined by a separate drawing, which is then pulled in at compile time - allows for that to be a separate file even that can be managed separately.

Similarly, being able to do math in line on a parameter, allows for cases like "this branch needs to be 2/3 of the way along this trunk", where the trunk length is a variable and the Branch(Length) = Trunk(Length) * (2/3). This way if the trunk length is changed, the branch length will be changed maintaining the ratio.

I think since this can become multi-dimensional, it's a good ideal to break this out earlier in a file as a general practice - much like doing "global defines". When it gets down to a bundle, this would be an array or arrays, with each conductor definition being its own array of defined attributes inherited from the conductor definition.

[tim292stro]

I think this is also a suitable feature request to piggyback with another suggestion - that wire parameters not be limited to simply DC electrical characteristics. Consider this type of cable, where there is 2x Cat-6, 2x RG-6 RF, and 2x multimode fiber.:

Or this example in robotics where it's not just Data+Power+light, but also air or hydraulic fluid being transmitted (or paint, or glue, etc...) - and note that the "bundle" may not be is something like a split loom or braid - but a drag chain (as shown directly below), or rigid conduit:

Some robotics harnesses can be a mess of multiple types of medium:

[Tyler-Ward]

For including data/templates from a separate file it looks to be possible to add support for this in pyyaml https://stackoverflow.com/questions/528281/how-can-i-include-a-yaml-file-inside-another

[formatc1702]

It was added very discreetly and I haven't tested it, but at least running WireViz from CLI allows to include a 'prepend file' in addition to the source YAML (see code).

I wouldn't want to mess too much with the pyyaml internals TBH, maybe this is a more reasonable approach?

[formatc1702]

Just a bit of brainstorming here.

We should look at this issue from two perspectives:

1. The internal data representation

It might make sense to shuffle around the representation of connectors and cables in the code, towards a more "object-oriented" approach, for lack of a better term.

Example (not intended to be complete, just to show my general idea):

cable:
  name: ...
  category: ...
  type: ...
  show_equiv: ...
  ... # other global parameters
  wires:
    1: # unique internal auto-incremented number
      id: Wire1 # unique user-assigned ID, with case-sensitivity issues as per #160
      label: VCC # arbitrary label
      length: 123
      gauge: 0.25 mm2
      color: 0xFF0000 # internal unambiguous color for rendering
      color_name: RD # user-visible color label
      ...
    2:
      ... # repeat
    ... # repeat
  additional_bom_items:

This nested hierarchy would allow a good degree of granularity for all kinds of parameters, and cleanly separate between user-assigned info (id,label,color_name) and internal info (the wire's dict key, color)...
This would get rid of having to rely on potentially ambiguous user-provided info, getting rid of #160 and maybe others.

2. The user input

For many cases, some parameters nested within each individual wire in the example above, will be identical for all wires in a cable/bundle. Therefore, it would be nice to keep the current syntax for these simple cases to avoid repetition:

W1:
  gauge: 0.25mm2 # applies to all wires
  length: 1 m # applies to all wires
  colors: [RD, BK, ...] # gets split among all the wires
  ..

The question is how to expand this for cases where not all wires are identical.

1. Use a column-based approach like the example in the Jul 3 comment above?
2. Use the same syntax as the internal representation?
   This would allow setting parameters both at the cable and the wire level, allowing the children (wires) to inherit the value, as well as overriding a parameter for individual wires where necessary.

Since the proposed internal representation would have all info split across all children (so as not to have to back-track to find inherited parameter values), it would be necessary to check at rendering time, whether a parameter (e.g. length) is equal for all children, and can thus be shown in the surrounding cable node, or if it differs from wire to wire, which would require showing the individual values next to the corresponding wires.

I am afraid that this would require a major refactoring, and perhaps it would be wise to issue a feature-freeze at some point to tackle this. I welcome any thoughts on this topic.

[tim292stro]

When I read this, I get the impression that you are work down to the wire from the major cable - rather than defining a wire and then building a cable out of it parametrically.

Think of it from the perspective of the cable designer. They would determine solid or stranded, and if stranded how many strands of what gauge wire. Then they would pick an insulation for that conductor or conductor bundle. From there they would group wire types (insulated, non insulated, non conductive, etc...), then that group might have a foil shield around it and that shield may be conductive to a drain wire on either face. There might be a a braided shield on outside that foil shield, then a jacket.

This may be produced on a spool "put up" of 1000Meters or so (from my experience quoting custom cables).

From there, the cable may be cut down to a length, then the jacket stripped back to reveal a conductor or sub group of conductors, and those individual conductors may then be stripped back and terminated to one of possibly many variety of terminations...

If you built a wire, then a cable, then bundled cables - it might be more organizationally rational to manage the data that way, but be ready to override data both up and down....

For example, building a CAT-5 cable:

Conductor:
    Name: TypeA
        Material: Copper
        Type: Solid
        Gauge: 23

Wire:
    Name: TPHalf
        Insulator: Fluorinated Ethylene Propylene
        Conductor: TypeA

Bundle:
    Name: TP
        Wire1:
            WireType: TPHalf
            ColorSolid:  {DEFAULT}  # <-- need some mechanism to populate this variable with a later call by a reference name
        Wire2:
            WireType: TPHalf
            ColorSolid: WH
            ColorStripe: {DEFAULT}
        TwistLength:  10mm

    Name: CAT-5
        Bundle1:
            Type: TP
            Wire1>ColorSolid: BU  # <-- this is an example of overriding a child's variable with a later call by a reference name
            Wire2>ColorStripe: BU
        Bundle2:
            Type: TP
            Wire1>ColorSolid: OR
            Wire2>ColorStripe: OR
        Bundle3:
            Type: TP
            Wire1>ColorSolid: GR
            Wire2>ColorStripe: GR
        Bundle4:
            Type: TP
            Wire1>ColorSolid: BR
            Wire2>ColorStripe: BR
        Jacket:
            Insulator: Polyvinyl Chloride
            ColorSolid: {DEFAULT}

Cable:
    Name: Network
        Type: CAT-5
        CAT-5>Jacket>ColorSolid:BU

This would define a Blue Cat5 cable in a PVC jacket, with 4x twisted pairs of FPE insulated 23 gauge copper, in color groups of:
Blue/Blue-White
Orange/Orange-White
Green/Green-White
Brown/Brown-White

Where I think you need a bit of work is the splits. Right now you seem to work on the individual conductor only presuming it has a start and end. I think you need to refactor that to presume cable bundling... If you define a cable bundle, your working presumption is that all of those internal wires are going to the same place (because... they are bundled). Say I wanted to split of just the Blue and Orange pairs from this bundle. Until I cut open the bundle and pull those pairs out, they are the same length as the other two pairs.

I also can't have any more wires that I have inside a defined bundle - you'll see reference to this in my cable work-around in issue #174 . A bundle will stay a bundle until you split it. If I did split, it should only be able to fan-out what is inside it, the longest length of the fan-out plus the length of the unseparated bundle defines how long the total original bundle must be before cutting it (useful to know for BOM purposes, i.e. "start with a 25-foot length of CAT-5).

I envision being able to connect a bundle only to what I'll call a "fan-out" on either end. Fan-out being either connected:

• Bundle-to-Bundle
• Bundle-to-Connector
• Bundle-to-Fan-out
• Bundle-to-NoConnect

In this way I could, take an entire group of twisted pair conductors and (respectively):

• Continue the original bundle as a sub-bundle of another defined bundle (rather than wiring each conductor inside one bundle to another). For example, a 50-foot CAT-5 may join 26x other CAT-5 heading into a network closet - I'd just want to say "CAT-5-A" joined bundle "Closet" Which contains: "CAT-5-A, CAT-5-B, CAT-5-C, ..., CAT-5-X, CAT-5-Y, CAT-5-Z" rather than defining each conductor in the major bundle.
• Fan them out to a connector (say and RJ-45 plug) - if a conductor serves a named purpose, calling them out at the connector would be best.
• Strip off just a fraction of the original bundle to another group. For example, I may have a CAT-5 that terminates to a device on one pair only, with the other 3x pairs joining another bundle, in an end terminated daisy chain (think like into a device out of the device to the next device. Perhaps that stub is a bit longer than just a fan-out, and now you have a different bundle of two twisted pairs...
• Have a conductor in a bundle that you don't terminate on "this" end. Because a fan-out exists at the end of a bundle (cable, group of cables), it's a known end of the bundle - this is open ended in the current implementation.

So the take-away here is that if one defines the cable bundle from its base components, and spends a bit of effort defining what happens at each end of the bundle, there is no room for interpretation.

Probably clear as mud...

[formatc1702]

Thanks for your perspective! Really interesting insights.

• I took the liberty to edit your post, enclosing the code in

  ```yaml
    <code>
  ```
  

  to make it appear as a block, and to enable highlighting.

• I really like the idea of parametrizing a cable like that.
  In the end, by using YAML anchors, what you describe can already be accomplished in a way.
  The main difference is that, so far, I haven't been thinking from the perspective of an actual cable designer like you, but of someone using existing cables or wires, and bundling those. It's a subtle difference, but of course a cable designer will want much more freedom in how they define parameters, in a very granular way.

• Where I think you need a bit of work is the splits. Right now you seem to work on the individual conductor only presuming it has a start and end. I think you need to refactor that to presume cable bundling...

  I think you've hit the nail on the head here.

Time to think about this a bit more, since it's clearly a major potential change...

Linking #31 + #127 + #155 for reference.

[formatc1702]

Copying the contents of #127 (closed for being closely related to the discussion here) for reference.

Currently, the cable dataclass stores the information on how many wires the cable/bundle has inside the wirecount field, but iterating over all the wires happens using the colors List, which is not too elegant, or necessarily intuitive.

For v0.3, perhaps it makes sense to revamp DataClasses.py to more closely and accurately represent the physical makeup of the components. This could [at least partly] address #29, #31 and #56.

Here are some ideas, in no particular order, on how the code could be cleaned up.

• A new wire class that lists the properties (gauge, color, length, etc.) of a single wire within the cable.
• Instead of wire, a name like conductor or similar might allow more future flexibility, to define pneumatic or hydraulic lines in a similar fashion to cables.
• Maybe have separate classes (but related by inheritance) for cables and bundles?
  • A cable could then contain a list of wires/conductors.
  • A bundle could work the same way, but additionally, allow other bundle and cable elements as children. This grouping would allow things like sheathing/heatshrink around multiple elements of the different classes.
  • A cable could also contain a kind of sub-bundles for shielded groups, and maybe twisted pairs.
• A parent cable could have gauge and length properties that all children inherit by default. But child elements (e.g. individual wires) could override the default, thus allowing different gauges within a cable or bundle, for example.
• It would be nice to preserve the current YAML syntax for simple cases when all parameters of the child elements are shared, or assigned in a defined way (e.g. color codes). Only users who need fine grained control would need to modify their source YAML files to take advantage of nesting and custom properties for each child.

A similar thing could be undertaken for the connector type. It might contain a collection of pin elements, each with its own properties and manufacturer info. However, I don't see the same level of benefit here, compared to cables.

This issue is intended as a starting point for discussion, there is no sense in starting to code and submit PRs before a new standard is settled... also, it is not top priority at the moment, just something to keep in mind and to think about a little bit :)

Currently, I believe @tim292stro's perspective is quite valuable, and might lead to a major refactoring in a later WireViz release... definitely not v0.3; perhaps a release of its own to prevent conflicts with other PRs running in parallel.

[tim292stro]

I'm glad my comments helped rather than the alternative (sorry about the code block thing- my son needed my attention and I did not think to return to fixing that afterwards). I'll help where I can as WireViz's utility genuinely interests me.

Looking at the above comment from #127 , I wanted to comment an agreement regarding leaving the pin types open to being sub-classified - when I read that comment I recalled MIL-DTL-38999 circular connectors with varying types of pins that can be used.

Examples:

• Single conductor pin
• Shielded twisted pair
• Fiber optic
• Coax
• etc...

I see this as a common requirement, even if you refer up to my comment above (here) - a custom manufactured header for a robot to interchange end-effectors has essentially the same needs, where a drilled hole would be an equivalent "pin" location and could be stuffed with really anything (compressed air, vacuum, hydraulic pressure and return, locating boss, etc...).

For the sake of conversation, what are the current data structures and limitations on inheritance (and which direction do those flow)?

Above in this thread (here), @formatc1702 mentions a parameter being either a STRING or INT - I wanted to plant the seed of inline variable, and movement of data up and down the parent/child relationship. I did an example (which I'll edit in a moment to improve) of this above in the CAT-5 cable definition by surrounding something that had to be a reference in curly braces {}. Why I think this would be important, some values may need to be defined by default to exists, but you may wish to push down from a parent an attribute to a child, and being able to reference it and set its value (like a key/value pair). How I conceive of this being implemented from a high level, is that you define a structure that will eventually become the child of another object later. Then when creating a parent, all of the attributes of the parent and pre-processed, then if a replacement/override set exists - as the parent pulls in the child structure it does a replace as it come across the various matching parent definitions.

Also consider the case of templates and workflows. Once I had created a part for CAT-5, I would not want to recreate that in future designs if none of the characteristics have changed - I'd want to pull that defined part from an external template. This would be especially important in the bundle level abstraction... Why I think this would be important, in engineering and manufacturing - cost sensitivity and functional performance is often related but secondary to what it already in a parts bin. from the perspective of a hobbyist or hardware hacker - one makes do with what they have laying around. If cost or performance genuinely creates a need for obtaining a new cable or wire, then that is the point of time where that effort is expended. This is similar to PCB design - one selects a part to do a job. If they already have it and "it'll do" then they probably already have the schematic symbol and the hole/pad layout. In pretty much every tool I've ever used in either the professionally or as a hobbyist, there has been some sort of component library. I consider WireViz as a back-end tool for a CAD-like workflow - we aren't necessarily designing the cable in YAML. If I said I wanted a cable to run from one termination point to another, and drew a complex 3D path - I might already need that path to know what the diameter of the cable is in order to bundle it or drill a hole for the bundle to pass through at the design stage to reduce rework needed when it's actually built. In a CAD program I might select a wire type as a template - and passing that template reference out to WireViz to build the harness simplifies/eliminates hand coding steps, and reduces the total amount of data that has to be managed.

[kvid]

One idea that is developing in my mind, is to create a generic Component dataclass that contains all attributes that are common to all components, and then make the subclasses Connector, Conductor (or some other generic term that can be cable, wire, bundle, tube, pipe, drag chain, etc.), and Device (or some other generic term that can be a switch, a LED, a resistor, etc. - see #142) that all inherit from Component and add or override what is needed for their purposes. Maybe add an intermediate dataclass in the inheritance structure as well if needed - thinking object oriented. However, such a change in the main data structure doesn't require any change of the input syntax, unless perhaps unifying a few attributes to reduce the differences between the different main dataclasses. The improved BOM generation merged from PR #115 makes it easier to implement this idea.

Originally posted by @kvid in #127 (comment), and slightly modified above

A basic draft structure might look something like this:

@dataclass
class BaseComponent:
    # Maybe use this class as an attribute instead of a superclass below?
    type: str
    manufacturer: Optional[str] = None
    mpn: Optional[str] = None
    pn: Optional[str] = None

@dataclass
class AdditionalComponent(BaseComponent):
    subtype: Optional[str] = None  # Maybe move this to BaseComponent?
    qty: float = 1
    unit: Optional[str] = None
    qty_multiplier: Union[DeviceMultiplier, ConductorMultiplier, None] = None

@dataclass
class Component(BaseComponent):
    name: str
    category: Optional[str] = None
    color: Optional[Color] = None
    image: Optional[Image] = None
    notes: Optional[str] = None
    show_name: bool = True
    ignore_in_bom: bool = False
    additional_components: List[AdditionalComponent] = field(default_factory=list)

@dataclass
class Device(Component):
    style: Optional[str] = None
    subtype: Optional[str] = None
    pincount: Optional[int] = None
    pinlabels: List[Pin] = field(default_factory=list)
    pins: List[Pin] = field(default_factory=list)
    show_pincount: Optional[bool] = None
    hide_disconnected_pins: bool = False
    autogenerate: bool = False
    loops: List[List[Pin]] = field(default_factory=list)

@dataclass
class Connector(Device):
    # Overriding BOM description and maybe add an optional reference to a mating connector

@dataclass
class Conductor(Component):
    # Split this class into several subclasses: Cable, Bundle, Wire, OpticFibre, Tube, etc.
    # while collecting common attributes in a common superclass
    manufacturer: Union[str, List[str], None] = None
    mpn: Union[str, List[str], None] = None
    pn: Union[str, List[str], None] = None
    gauge: Optional[float] = None
    gauge_unit: Optional[str] = None
    show_equiv: bool = False
    length: float = 0
    wirecount: Optional[int] = None
    shield: Union[bool, Color] = False
    colors: List[Colors] = field(default_factory=list)
    color_code: Optional[ColorScheme] = None
    show_wirecount: bool = True

[tim292stro]

I've stewed on this for a bit and I think a way to express the relationship of a child object's structure is probably the least painful approach. An example revision to my above CAT-5 example:

Conductor:
    Name: TypeA
        Material: Copper
        Type: Solid
        Gauge: 23

Wire:
    Name: TPHalf
        Insulator: Fluorinated Ethylene Propylene
        Conductor: TypeA

Bundle:
    Name: TP
        Wire1:
            Child: TPHalf
            ColorSolid:  {DEFAULT}  # <-- need some mechanism to populate this variable with a later call by a reference name
        Wire2:
            Child: TPHalf
            ColorSolid: WH
            ColorStripe: {DEFAULT}
        TwistLength:  10mm

    Name: CAT-5
        Bundle1:
            Child: TP
                Wire1>ColorSolid: BU  # <-- this is an example of overriding a child's variable with a later call by a reference name
                Wire2>ColorStripe: BU
        Bundle2:
            Child: TP
                Wire1>ColorSolid: OR
                Wire2>ColorStripe: OR
        Bundle3:
            Child: TP
                Wire1>ColorSolid: GR
                Wire2>ColorStripe: GR
        Bundle4:
            Child: TP
                Wire1>ColorSolid: BR
                Wire2>ColorStripe: BR
        Jacket:
            Insulator: Polyvinyl Chloride
            ColorSolid: {DEFAULT}

    Name: SecureNetwork
        Child: CAT-5
            CAT-5>Jacket>ColorSolid: RE

    Name: NonSecureNetwork
        Child: CAT-5
            CAT-5>Jacket>ColorSolid: BK

    Name: GeneralNetwork
        Child: CAT-5
            CAT-5>Jacket>ColorSolid: BU

In this way some simple rules can be defined which are probably easier to code:

1. A "child" will be instantiated when its "name" is called. (Name must exist in this file or an included file)
2. Overrides to the child's attributes are taken from an indented sub group, with the full path to the overridden attribute given to the left of the colon (:)
3. Forcing the user to define a {DEFAULT} will indicate that this attribute is eligible for overwriting/modification

[formatc1702]

I've decided to start a separate repo, dubbed WireViz-OO (for object-oriented), to discuss the issues of accurate data representation for a future refactoring of WireViz, by building a skeleton of dataclasses loosely based on the discussion here.

Feel free to have a look, check the Readme, discuss in the issues and submit PRs! I expect this to be a parallel process to the regular WireViz development, and at some point in the future, I'd like to refactor WireViz using the results from WireViz-OO as a template.