Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

AttributeError: 'CollectingNode' object has no attribute 'parent_cluster' error calling get_grafana_snapshot function #7323

Closed
2 tasks
vponomaryov opened this issue Apr 5, 2024 · 3 comments · Fixed by #7356
Closed
2 tasks

AttributeError: 'CollectingNode' object has no attribute 'parent_cluster' error calling get_grafana_snapshot function #7323

vponomaryov opened this issue Apr 5, 2024 · 3 comments · Fixed by #7356
Assignees
@vponomaryov

Comments

@vponomaryov
Copy link
Contributor

Packages

Base Scylla version: 5.4.5-20240328.fd7d57b9fa26 with build-id 0bb6a5f3f87d481576ab0c3c6247b2beba69c17a
Target Scylla version (or git commit hash): 5.5.0~dev-20240403.baae811142e2 with build-id 030312786e4ec5fc082f65330ae641c075e2ddb5

Kernel Version: 5.15.0-1053-gcp

Issue description

  • This issue is a regression.
  • It is unknown if this issue is a regression.

In the log collection stage we started having following error:

21:59:36  scylla-monitoring-src, branch-4.6, 5.4
21:59:44  Error taking monitor screenshot: 'CollectingNode' object has no attribute 'parent_cluster', traceback: Traceback (most recent call last):
21:59:44    File "/home/ubuntu/scylla-cluster-tests/sdcm/logcollector.py", line 493, in get_grafana_screenshot
21:59:44      self.remote_browser = RemoteBrowser(node)
21:59:44    File "/home/ubuntu/scylla-cluster-tests/sdcm/utils/remotewebbrowser.py", line 67, in __init__
21:59:44      backend = self.node.parent_cluster.params.get("cluster_backend")
21:59:44  AttributeError: 'CollectingNode' object has no attribute 'parent_cluster'
21:59:44  
21:59:45  Error taking monitor snapshot: 'CollectingNode' object has no attribute 'parent_cluster', traceback: Traceback (most recent call last):
21:59:45    File "/home/ubuntu/scylla-cluster-tests/sdcm/logcollector.py", line 555, in get_grafana_snapshot
21:59:45      self.remote_browser = RemoteBrowser(node)
21:59:45    File "/home/ubuntu/scylla-cluster-tests/sdcm/utils/remotewebbrowser.py", line 67, in __init__
21:59:45      backend = self.node.parent_cluster.params.get("cluster_backend")
21:59:45  AttributeError: 'CollectingNode' object has no attribute 'parent_cluster'

The root cause is here: 5014b13

Installation details

Cluster size: 4 nodes (n2-highmem-64)

Scylla Nodes used in this run:

  • rolling-upgrade--ubuntu-focal-db-node-151d470d-0-4 (34.148.186.115 | 10.142.0.77) (shards: 60)
  • rolling-upgrade--ubuntu-focal-db-node-151d470d-0-3 (34.148.169.76 | 10.142.0.72) (shards: 60)
  • rolling-upgrade--ubuntu-focal-db-node-151d470d-0-2 (34.148.94.141 | 10.142.0.63) (shards: 60)
  • rolling-upgrade--ubuntu-focal-db-node-151d470d-0-1 (34.74.61.253 | 10.142.0.31) (shards: 60)

OS / Image: https://www.googleapis.com/compute/v1/projects/ubuntu-os-cloud/global/images/family/ubuntu-2004-lts (gce: undefined_region)

Test: vp-rolling-upgrade-ubuntu20.04-test
Test id: 151d470d-0dd5-4f07-b3f7-6a6329d839b5
Test name: scylla-staging/valerii/vp-rolling-upgrade-ubuntu20.04-test
Test config file(s):

Logs and commands
  • Restore Monitor Stack command: $ hydra investigate show-monitor 151d470d-0dd5-4f07-b3f7-6a6329d839b5
  • Restore monitor on AWS instance using Jenkins job
  • Show all stored logs command: $ hydra investigate show-logs 151d470d-0dd5-4f07-b3f7-6a6329d839b5

Logs:

Jenkins job URL
Argus
@vponomaryov vponomaryov mentioned this issue Apr 5, 2024
Merged
3 tasks
@fruch
Copy link
Contributor

fruch commented Apr 5, 2024

The job you reference here has much more concerning failure

 Fatal Python error: Segmentation fault

@vponomaryov
Copy link
Contributor Author

The job you reference here has much more concerning failure

 Fatal Python error: Segmentation fault

This one is not repeatable. And looks unrelated to our codebase.

@fruch fruch removed their assignment Apr 14, 2024
@fruch
Copy link
Contributor

fruch commented Apr 14, 2024

@vponomaryov

can you please look into fixing this one ? (I won't be able to get to it)

fruch added a commit to fruch/scylla-cluster-tests that referenced this issue Apr 21, 2024
@fruch
fix(RemoteBrowser): stop assuming parent_cluster exists
74b666b 
cause of supporting docker backend, we introduce a check of the backend
to decide if tunnelling is needed.

this broke the log collection phase, since `CollectingNode` doesn't have `parent_cluster`
or access to SCT configuration.

for now we can assume if we don't have `parent_cluster` we should enable tunneling
even that tunneling is only needed when working from diffrent regions (or from local env)

Fix scylladb#7323
@fruch fruch mentioned this issue Apr 21, 2024
Merged
2 tasks
soyacz pushed a commit that referenced this issue Apr 22, 2024
@fruch @soyacz
fix(RemoteBrowser): stop assuming parent_cluster exists
5f530e2 
cause of supporting docker backend, we introduce a check of the backend
to decide if tunnelling is needed.

this broke the log collection phase, since `CollectingNode` doesn't have `parent_cluster`
or access to SCT configuration.

for now we can assume if we don't have `parent_cluster` we should enable tunneling
even that tunneling is only needed when working from diffrent regions (or from local env)

Fix #7323
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@vponomaryov vponomaryov

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix(RemoteBrowser): stop assuming parent_cluster exists fruch/scylla-cluster-tests
2 participants
@fruch @vponomaryov