Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#4900: New model for cable paths #5212

Merged
merged 67 commits into from
Oct 8, 2020
Merged
Show file tree
Hide file tree
Changes from 28 commits
Commits
Show all changes
67 commits
Select commit Hold shift + click to select a range
587e6fc
Initial work on cable paths (WIP)
jeremystretch Sep 30, 2020
9851977
Add initial tests
jeremystretch Sep 30, 2020
319329e
Extend cable path tests
jeremystretch Sep 30, 2020
cd71799
Ignore the position stack when traversing single-position rear ports
jeremystretch Sep 30, 2020
e53ae1d
Extend cable path tests
jeremystretch Sep 30, 2020
46df5a9
Remove extraneous test objects
jeremystretch Sep 30, 2020
19a3a4d
Add GenericRelation to originating cable paths on PathEndpoint
jeremystretch Oct 1, 2020
105c0fd
Introduce retrace_paths management command
jeremystretch Oct 1, 2020
8abc055
CircuitTermination and PowerFeed are path endpoints
jeremystretch Oct 1, 2020
cd398b1
retrace_paths should ignore case in model names
jeremystretch Oct 1, 2020
610420c
Drop support for split paths
jeremystretch Oct 1, 2020
c974c56
Capture path end-to-end status in CablePath
jeremystretch Oct 1, 2020
0d07b03
Add test for connecting cables out of order
jeremystretch Oct 1, 2020
3b0a75e
Add test for updated paths on cable status change
jeremystretch Oct 1, 2020
d50a0d9
Add tests for multiple pass-through breakouts
jeremystretch Oct 2, 2020
9d10c57
Remove legacy CablePathTestCase
jeremystretch Oct 2, 2020
9f24221
Rename test elements to be more readable
jeremystretch Oct 2, 2020
4fd1219
Add tests for all PathEndpoint classes
jeremystretch Oct 2, 2020
e0abd7e
Remove dcim.tests.test_api.ConnectionTest
jeremystretch Oct 2, 2020
66355da
CablePath.origin should be unique
jeremystretch Oct 2, 2020
aa0d4c4
Replace connection_status filter with is_connected
jeremystretch Oct 2, 2020
e9da84f
Replace legacy trace() method
jeremystretch Oct 2, 2020
7ff247c
Add trace view for PowerFeed
jeremystretch Oct 2, 2020
8cb636b
Update console/power/interface connection tables
jeremystretch Oct 2, 2020
5737f6f
Cache each CablePath on its originating endpoint
jeremystretch Oct 2, 2020
f8800b8
Optimize console/power/interface connection lists
jeremystretch Oct 3, 2020
079c422
Remove legacy connected endpoint fields
jeremystretch Oct 5, 2020
df73737
Remove legacy connection_status fields
jeremystretch Oct 5, 2020
13db22d
Initial changelog notes for #4900
jeremystretch Oct 5, 2020
3d34f1c
Rename CablePath.is_connected to is_active
jeremystretch Oct 5, 2020
b846f63
Rename connection_status to connected_endpoint_reachable
jeremystretch Oct 5, 2020
32aa2da
PowerFeedSerializer should subclass ConnectedEndpointSerializer
jeremystretch Oct 5, 2020
b2066bc
Merge schema migrations
jeremystretch Oct 5, 2020
50aecd0
Fix up connection lists (pending additional work)
jeremystretch Oct 5, 2020
32b8148
Standardize path endpoint templates
jeremystretch Oct 5, 2020
d5d6b0e
Optimize path prefetching
jeremystretch Oct 5, 2020
19430dd
Extend cable trace view to show related paths
jeremystretch Oct 5, 2020
56ee425
Introduce PathContains lookup to allow filtering against objects in p…
jeremystretch Oct 6, 2020
ffdf551
Tweak component templates
jeremystretch Oct 6, 2020
6275c8c
Prefetch path & destination for API views
jeremystretch Oct 6, 2020
d59f089
Cache peer termination on CableTerminations
jeremystretch Oct 6, 2020
d984dbd
Extend device view to show local cable termination for all components
jeremystretch Oct 6, 2020
c813ae4
Clean up power connection tables
jeremystretch Oct 6, 2020
23cde6d
Include cable_peer on CableTermination serializers
jeremystretch Oct 6, 2020
3870f5d
Remove unused CablePathManager
jeremystretch Oct 6, 2020
52ec35b
Correct serializer field lists
jeremystretch Oct 6, 2020
534364a
Improve model docstrings
jeremystretch Oct 6, 2020
6b3a199
Add test_is_connected to CircuitTerminationTestCase
jeremystretch Oct 6, 2020
a6e0ef8
Clean up console/power/interface connections views
jeremystretch Oct 6, 2020
a072d40
Update v2.10 changelog
jeremystretch Oct 6, 2020
2c9ae60
Optimize path node representations
jeremystretch Oct 6, 2020
44b8425
Restore total length count on trace view
jeremystretch Oct 6, 2020
c7c6662
Standardize 'cabled' and 'connected' filters; complete tests
jeremystretch Oct 6, 2020
6db3c65
Swap order of cabling migrations
jeremystretch Oct 7, 2020
f560693
Rewrite trace_paths management command and call in upgrade.sh
jeremystretch Oct 7, 2020
eaf8d95
Clean up power utilization logic
jeremystretch Oct 7, 2020
85439fd
Fix PowerFeed display in cable traces
jeremystretch Oct 7, 2020
35759fd
Redo the cable trace UI (WIP)
jeremystretch Oct 7, 2020
aa0ee27
Add cable paths API detail view for pass-through ports
jeremystretch Oct 8, 2020
55268c9
Replace connection_status with connected_endpoint_reachable on Interf…
jeremystretch Oct 8, 2020
ae1ceb2
Standardize cable/connection field ordering
jeremystretch Oct 8, 2020
29eebf9
Update REST API changes
jeremystretch Oct 8, 2020
0c5efa2
Handle traces which split at a RearPort
jeremystretch Oct 8, 2020
0e41bc4
Add /trace API endpoints for CircuitTermination and PowerFeed
jeremystretch Oct 8, 2020
75ddc63
Handle split paths
jeremystretch Oct 8, 2020
a716ca7
Rewrite cablepath tests to create components within each test
jeremystretch Oct 8, 2020
44caa40
Delete obsolete LoopDetected exception
jeremystretch Oct 8, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 2 additions & 4 deletions netbox/circuits/api/views.py
Original file line number Diff line number Diff line change
Expand Up @@ -46,9 +46,7 @@ class CircuitTypeViewSet(ModelViewSet):

class CircuitViewSet(CustomFieldModelViewSet):
queryset = Circuit.objects.prefetch_related(
Prefetch('terminations', queryset=CircuitTermination.objects.prefetch_related(
'site', 'connected_endpoint__device'
)),
Prefetch('terminations', queryset=CircuitTermination.objects.prefetch_related('site')),
'type', 'tenant', 'provider',
).prefetch_related('tags')
serializer_class = serializers.CircuitSerializer
Expand All @@ -61,7 +59,7 @@ class CircuitViewSet(CustomFieldModelViewSet):

class CircuitTerminationViewSet(ModelViewSet):
queryset = CircuitTermination.objects.prefetch_related(
'circuit', 'site', 'connected_endpoint__device', 'cable'
'circuit', 'site', 'cable'
)
serializer_class = serializers.CircuitTerminationSerializer
filterset_class = filters.CircuitTerminationFilterSet
3 changes: 2 additions & 1 deletion netbox/circuits/filters.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import django_filters
from django.db.models import Q

from dcim.filters import PathEndpointFilterSet
from dcim.models import Region, Site
from extras.filters import CustomFieldFilterSet, CreatedUpdatedFilterSet
from tenancy.filters import TenancyFilterSet
Expand Down Expand Up @@ -144,7 +145,7 @@ def search(self, queryset, name, value):
).distinct()


class CircuitTerminationFilterSet(BaseFilterSet):
class CircuitTerminationFilterSet(BaseFilterSet, PathEndpointFilterSet):
q = django_filters.CharFilter(
method='search',
label='Search',
Expand Down
20 changes: 20 additions & 0 deletions netbox/circuits/migrations/0021_cablepath.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Generated by Django 3.1 on 2020-10-02 19:43

from django.db import migrations, models
import django.db.models.deletion


class Migration(migrations.Migration):

dependencies = [
('dcim', '0120_cablepath'),
('circuits', '0020_custom_field_data'),
]

operations = [
migrations.AddField(
model_name='circuittermination',
name='_path',
field=models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.SET_NULL, to='dcim.cablepath'),
),
]
21 changes: 21 additions & 0 deletions netbox/circuits/migrations/0022_drop_connected_endpoint.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Generated by Django 3.1 on 2020-10-05 14:07

from django.db import migrations


class Migration(migrations.Migration):

dependencies = [
('circuits', '0021_cablepath'),
]

operations = [
migrations.RemoveField(
model_name='circuittermination',
name='connected_endpoint',
),
migrations.RemoveField(
model_name='circuittermination',
name='connection_status',
),
]
17 changes: 2 additions & 15 deletions netbox/circuits/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,8 @@
from django.urls import reverse
from taggit.managers import TaggableManager

from dcim.constants import CONNECTION_STATUS_CHOICES
from dcim.fields import ASNField
from dcim.models import CableTermination
from dcim.models import CableTermination, PathEndpoint
from extras.models import ChangeLoggedModel, CustomFieldModel, ObjectChange, TaggedItem
from extras.utils import extras_features
from utilities.querysets import RestrictedQuerySet
Expand Down Expand Up @@ -232,7 +231,7 @@ def termination_z(self):
return self._get_termination('Z')


class CircuitTermination(CableTermination):
class CircuitTermination(PathEndpoint, CableTermination):
circuit = models.ForeignKey(
to='circuits.Circuit',
on_delete=models.CASCADE,
Expand All @@ -248,18 +247,6 @@ class CircuitTermination(CableTermination):
on_delete=models.PROTECT,
related_name='circuit_terminations'
)
connected_endpoint = models.OneToOneField(
to='dcim.Interface',
on_delete=models.SET_NULL,
related_name='+',
blank=True,
null=True
)
connection_status = models.BooleanField(
choices=CONNECTION_STATUS_CHOICES,
blank=True,
null=True
)
port_speed = models.PositiveIntegerField(
verbose_name='Port speed (Kbps)'
)
Expand Down
4 changes: 2 additions & 2 deletions netbox/circuits/urls.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
from django.urls import path

from dcim.views import CableCreateView, CableTraceView
from dcim.views import CableCreateView, PathTraceView
from extras.views import ObjectChangeLogView
from . import views
from .models import Circuit, CircuitTermination, CircuitType, Provider
Expand Down Expand Up @@ -45,6 +45,6 @@
path('circuit-terminations/<int:pk>/edit/', views.CircuitTerminationEditView.as_view(), name='circuittermination_edit'),
path('circuit-terminations/<int:pk>/delete/', views.CircuitTerminationDeleteView.as_view(), name='circuittermination_delete'),
path('circuit-terminations/<int:termination_a_id>/connect/<str:termination_b_type>/', CableCreateView.as_view(), name='circuittermination_connect', kwargs={'termination_a_type': CircuitTermination}),
path('circuit-terminations/<int:pk>/trace/', CableTraceView.as_view(), name='circuittermination_trace', kwargs={'model': CircuitTermination}),
path('circuit-terminations/<int:pk>/trace/', PathTraceView.as_view(), name='circuittermination_trace', kwargs={'model': CircuitTermination}),

]
4 changes: 2 additions & 2 deletions netbox/circuits/views.py
Original file line number Diff line number Diff line change
Expand Up @@ -131,15 +131,15 @@ def get(self, request, pk):
circuit = get_object_or_404(self.queryset, pk=pk)

termination_a = CircuitTermination.objects.restrict(request.user, 'view').prefetch_related(
'site__region', 'connected_endpoint__device'
'site__region'
).filter(
circuit=circuit, term_side=CircuitTerminationSideChoices.SIDE_A
).first()
if termination_a and termination_a.connected_endpoint:
termination_a.ip_addresses = termination_a.connected_endpoint.ip_addresses.restrict(request.user, 'view')

termination_z = CircuitTermination.objects.restrict(request.user, 'view').prefetch_related(
'site__region', 'connected_endpoint__device'
'site__region'
).filter(
circuit=circuit, term_side=CircuitTerminationSideChoices.SIDE_Z
).first()
Expand Down
18 changes: 6 additions & 12 deletions netbox/dcim/api/nested_serializers.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
from rest_framework import serializers

from dcim.constants import CONNECTION_STATUS_CHOICES
from dcim import models
from utilities.api import ChoiceField, WritableNestedSerializer
from utilities.api import WritableNestedSerializer

__all__ = [
'NestedCableSerializer',
Expand Down Expand Up @@ -228,51 +227,46 @@ class Meta:
class NestedConsoleServerPortSerializer(WritableNestedSerializer):
url = serializers.HyperlinkedIdentityField(view_name='dcim-api:consoleserverport-detail')
device = NestedDeviceSerializer(read_only=True)
connection_status = ChoiceField(choices=CONNECTION_STATUS_CHOICES, read_only=True)

class Meta:
model = models.ConsoleServerPort
fields = ['id', 'url', 'device', 'name', 'cable', 'connection_status']
fields = ['id', 'url', 'device', 'name', 'cable']


class NestedConsolePortSerializer(WritableNestedSerializer):
url = serializers.HyperlinkedIdentityField(view_name='dcim-api:consoleport-detail')
device = NestedDeviceSerializer(read_only=True)
connection_status = ChoiceField(choices=CONNECTION_STATUS_CHOICES, read_only=True)

class Meta:
model = models.ConsolePort
fields = ['id', 'url', 'device', 'name', 'cable', 'connection_status']
fields = ['id', 'url', 'device', 'name', 'cable']


class NestedPowerOutletSerializer(WritableNestedSerializer):
url = serializers.HyperlinkedIdentityField(view_name='dcim-api:poweroutlet-detail')
device = NestedDeviceSerializer(read_only=True)
connection_status = ChoiceField(choices=CONNECTION_STATUS_CHOICES, read_only=True)

class Meta:
model = models.PowerOutlet
fields = ['id', 'url', 'device', 'name', 'cable', 'connection_status']
fields = ['id', 'url', 'device', 'name', 'cable']


class NestedPowerPortSerializer(WritableNestedSerializer):
url = serializers.HyperlinkedIdentityField(view_name='dcim-api:powerport-detail')
device = NestedDeviceSerializer(read_only=True)
connection_status = ChoiceField(choices=CONNECTION_STATUS_CHOICES, read_only=True)

class Meta:
model = models.PowerPort
fields = ['id', 'url', 'device', 'name', 'cable', 'connection_status']
fields = ['id', 'url', 'device', 'name', 'cable']


class NestedInterfaceSerializer(WritableNestedSerializer):
device = NestedDeviceSerializer(read_only=True)
url = serializers.HyperlinkedIdentityField(view_name='dcim-api:interface-detail')
connection_status = ChoiceField(choices=CONNECTION_STATUS_CHOICES, read_only=True)

class Meta:
model = models.Interface
fields = ['id', 'url', 'device', 'name', 'cable', 'connection_status']
fields = ['id', 'url', 'device', 'name', 'cable']


class NestedRearPortSerializer(WritableNestedSerializer):
Expand Down
29 changes: 15 additions & 14 deletions netbox/dcim/api/serializers.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,29 +30,30 @@
class ConnectedEndpointSerializer(ValidatedModelSerializer):
connected_endpoint_type = serializers.SerializerMethodField(read_only=True)
connected_endpoint = serializers.SerializerMethodField(read_only=True)
connection_status = ChoiceField(choices=CONNECTION_STATUS_CHOICES, read_only=True)
connection_status = serializers.SerializerMethodField(read_only=True)

def get_connected_endpoint_type(self, obj):
if hasattr(obj, 'connected_endpoint') and obj.connected_endpoint is not None:
return '{}.{}'.format(
obj.connected_endpoint._meta.app_label,
obj.connected_endpoint._meta.model_name
)
if obj._path is not None and obj._path.destination is not None:
return f'{obj._path.destination._meta.app_label}.{obj._path.destination._meta.model_name}'
return None

@swagger_serializer_method(serializer_or_field=serializers.DictField)
def get_connected_endpoint(self, obj):
"""
Return the appropriate serializer for the type of connected object.
"""
if getattr(obj, 'connected_endpoint', None) is None:
return None

serializer = get_serializer_for_model(obj.connected_endpoint, prefix='Nested')
context = {'request': self.context['request']}
data = serializer(obj.connected_endpoint, context=context).data
if obj._path is not None and obj._path.destination is not None:
serializer = get_serializer_for_model(obj._path.destination, prefix='Nested')
context = {'request': self.context['request']}
return serializer(obj._path.destination, context=context).data
return None

return data
# TODO: Tweak the representation for this field
@swagger_serializer_method(serializer_or_field=serializers.BooleanField)
def get_connection_status(self, obj):
if obj._path is not None:
return obj._path.is_connected
return None


#
Expand Down Expand Up @@ -715,7 +716,7 @@ class Meta:
class InterfaceConnectionSerializer(ValidatedModelSerializer):
interface_a = serializers.SerializerMethodField()
interface_b = NestedInterfaceSerializer(source='connected_endpoint')
connection_status = ChoiceField(choices=CONNECTION_STATUS_CHOICES, required=False)
# connection_status = ChoiceField(choices=CONNECTION_STATUS_CHOICES, required=False)

class Meta:
model = Interface
Expand Down
Loading