04/27/2025 18:11:42.225 7f3199497978 [contrail-device-manager] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 04/27/2025 18:11:42.225 7f3199497978 [contrail-device-manager] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 04/27/2025 18:11:42.226 7f3199497978 [contrail-device-manager] [INFO]: SANDESH: Flow Logging: [None] -> [False] 04/27/2025 18:11:42.228 7f3199497978 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@localhost:5672/'] 04/27/2025 18:11:42.229 7f3199497978 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 04/27/2025 18:11:42.229 7f3199497978 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED <cfgm_common.tests.test_utils.FakeKombu.Connection object at 0x7f31994a2f60> 04/27/2025 18:11:42.229 7f3199497978 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 04/27/2025 18:11:42.229 7f3199497978 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED <cfgm_common.tests.test_utils.FakeKombu.Connection object at 0x7f31994a2d30> 04/27/2025 18:11:42.230 7f3199497978 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 04/27/2025 18:11:43.242 7f3199497978 [contrail-device-manager] [WARNING]: Process 12065 prepared to run in Full mode... 04/27/2025 18:11:43.242 7f3199497978 [contrail-device-manager] [WARNING]: Elected master Device Manager node. Initializing... 04/27/2025 18:11:43.242 7f3199497978 [contrail-device-manager] [INFO]: SANDESH: INTROSPECT IS ON: 0.0.0.0:41389 04/27/2025 18:11:43.336 7f3199438588 [contrail-device-manager] [ERROR]: Cannot write http_port 41389 to /tmp/contrail-device-manager.12058.http_port 04/27/2025 18:11:43.336 7f3199438588 [contrail-device-manager] [ERROR]: Starting Introspect on HTTP Port 41389 04/27/2025 18:11:43.345 7f3199497978 [contrail-device-manager] [ERROR]: predef payloads file does not exist: /opt/contrail/fabric_ansible_playbooks/conf/predef_payloads.json 04/27/2025 18:11:43.346 7f3199497978 [contrail-device-manager] [ERROR]: Unable to load init data 04/27/2025 18:11:43.346 7f3199497978 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 04/27/2025 18:11:43.351 7f3199497978 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@localhost:5672/'] 04/27/2025 18:11:43.352 7f3199497978 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 04/27/2025 18:11:43.352 7f3199497978 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED <cfgm_common.tests.test_utils.FakeKombu.Connection object at 0x7f31994a24e0> 04/27/2025 18:11:43.352 7f3199497978 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 04/27/2025 18:11:43.352 7f3199497978 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED <cfgm_common.tests.test_utils.FakeKombu.Connection object at 0x7f3199427518> 04/27/2025 18:11:43.352 7f319f1400f0 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 04/27/2025 18:11:43.353 7f319f1400f0 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED <cfgm_common.tests.test_utils.FakeKombu.Connection object at 0x7f3199427518> 04/27/2025 18:11:47.203 7f31993ddf60 [contrail-device-manager] [INFO]: Found feature plugin for pr=routertest.test_dm_ansible_dhcp_relay.TestAnsibleDhcpRelayDM.test_dhcp_relay_config_push_one_vn_in_network(225516b9-2542-4979-ab45-facc8a3a23fd), feature=vn-interconnect 04/27/2025 18:11:47.203 7f31993ddf60 [contrail-device-manager] [INFO]: Found feature plugin for pr=routertest.test_dm_ansible_dhcp_relay.TestAnsibleDhcpRelayDM.test_dhcp_relay_config_push_one_vn_in_network(225516b9-2542-4979-ab45-facc8a3a23fd), feature=l3-gateway 04/27/2025 18:11:47.204 7f31993ddf60 [contrail-device-manager] [INFO]: Found ansible plugin pr=225516b9-2542-4979-ab45-facc8a3a23fd, role/vendor/product=spine:juniper:qfx10008 04/27/2025 18:11:47.248 7f31993ddf60 [contrail-device-manager] [INFO]: Found feature plugin for pr=routertest.test_dm_ansible_dhcp_relay.TestAnsibleDhcpRelayDM.test_dhcp_relay_config_push_one_vn_in_network(225516b9-2542-4979-ab45-facc8a3a23fd), feature=vn-interconnect 04/27/2025 18:11:47.248 7f31993ddf60 [contrail-device-manager] [INFO]: Found feature plugin for pr=routertest.test_dm_ansible_dhcp_relay.TestAnsibleDhcpRelayDM.test_dhcp_relay_config_push_one_vn_in_network(225516b9-2542-4979-ab45-facc8a3a23fd), feature=l3-gateway 04/27/2025 18:11:47.248 7f31993ddf60 [contrail-device-manager] [INFO]: Found ansible plugin pr=225516b9-2542-4979-ab45-facc8a3a23fd, role/vendor/product=spine:juniper:qfx10008 04/27/2025 18:11:47.568 7f31993ddf60 [contrail-device-manager] [ERROR]: AttributeError Python 3.6.8: /root/work/build/debug/config/device-manager/.tox/py36/bin/python Sun Apr 27 18:11:47 2025 A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred. /root/work/build/debug/config/device-manager/.tox/py36/lib/python3.6/site-packages/cfgm_common/vnc_amqp.py in _vnc_subscribe_callback(self=<device_manager.dm_amqp.DMAmqpHandle object>, oper_info={'fq_name': ['default-domain', 'default-project', '__contrail_lr_internal_vn_a9cecf50-5e21-4a9a-8362-1d1dad5d8b22__', '__contrail_lr_internal_vn_a9cecf50-5e21-4a9a-8362-1d1dad5d8b22__'], 'obj_dict': {'display_name': '__contrail_lr_internal_vn_a9cecf50-5e21-4a9a-8362-1d1dad5d8b22__', 'fq_name': ['default-domain', 'default-project', '__contrail_lr_internal_vn_a9cecf50-5e21-4a9a-8362-1d1dad5d8b22__', '__contrail_lr_internal_vn_a9cecf50-5e21-4a9a-8362-1d1dad5d8b22__'], 'id_perms': {'created': '2025-04-28T01:11:47.537994', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-04-28T01:11:47.537994', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'contrail-api', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9912506270482837420, 'uuid_mslong': 3108509927159384825}}, 'parent_type': 'virtual-network', 'parent_uuid': 'cb91a0ad-d078-49e3-914b-12ce294069e4', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'routing_instance_fabric_snat': False, 'routing_instance_has_pnf': False, 'routing_instance_is_default': True, 'uuid': '2b23a54f-251c-4af9-8990-4bef2df62fac'}, 'oper': 'CREATE', 'request-id': 'req-01cee2f9-731a-49fe-9d42-a012c1114567', 'type': 'routing_instance', 'uuid': '2b23a54f-251c-4af9-8990-4bef2df62fac'}) 82 try: 83 self.oper_info = oper_info 84 self.vnc_subscribe_actions() 85 86 except ConnectionError: self = <device_manager.dm_amqp.DMAmqpHandle object> self.vnc_subscribe_actions = <bound method VncAmqpHandle.vnc_subscribe_actions of <device_manager.dm_amqp.DMAmqpHandle object>> /root/work/build/debug/config/device-manager/.tox/py36/lib/python3.6/site-packages/cfgm_common/vnc_amqp.py in vnc_subscribe_actions(self=<device_manager.dm_amqp.DMAmqpHandle object>) 146 (self. obj_type, obj_id, oper)) 147 return 148 self.evaluate_dependency() 149 150 def _get_key_from_oper_info(self): self = <device_manager.dm_amqp.DMAmqpHandle object> self.evaluate_dependency = <bound method DMAmqpHandle.evaluate_dependency of <device_manager.dm_amqp.DMAmqpHandle object>> /root/work/build/debug/config/device-manager/device_manager/dm_amqp.py in evaluate_dependency(self=<device_manager.dm_amqp.DMAmqpHandle object>) 49 []): 50 pr = PhysicalRouterDM.get(pr_id) 51 if pr is not None: 52 pr.set_config_state() 53 pr.uve_send() pr = <device_manager.db.PhysicalRouterDM object> pr.set_config_state = <bound method PhysicalRouterDM.set_config_state of <device_manager.db.PhysicalRouterDM object>> /root/work/build/debug/config/device-manager/device_manager/db.py in set_config_state(self=<device_manager.db.PhysicalRouterDM object>, reset_retry_count=True) 1044 self.nc_q.put_nowait(1) 1045 except queue.Full: 1046 self._logger.warn(f"queue is full") 1047 1048 # end self = <device_manager.db.PhysicalRouterDM object> self._logger = <device_manager.logger.DeviceManagerLogger object> self._logger.warn undefined AttributeError: 'DeviceManagerLogger' object has no attribute 'warn' __cause__ = None __class__ = <class 'AttributeError'> __context__ = Full() __delattr__ = <method-wrapper '__delattr__' of AttributeError object> __dict__ = {} __dir__ = <built-in method __dir__ of AttributeError object> __doc__ = 'Attribute not found.' __eq__ = <method-wrapper '__eq__' of AttributeError object> __format__ = <built-in method __format__ of AttributeError object> __ge__ = <method-wrapper '__ge__' of AttributeError object> __getattribute__ = <method-wrapper '__getattribute__' of AttributeError object> __gt__ = <method-wrapper '__gt__' of AttributeError object> __hash__ = <method-wrapper '__hash__' of AttributeError object> __init__ = <method-wrapper '__init__' of AttributeError object> __init_subclass__ = <built-in method __init_subclass__ of type object> __le__ = <method-wrapper '__le__' of AttributeError object> __lt__ = <method-wrapper '__lt__' of AttributeError object> __ne__ = <method-wrapper '__ne__' of AttributeError object> __new__ = <built-in method __new__ of type object> __reduce__ = <built-in method __reduce__ of AttributeError object> __reduce_ex__ = <built-in method __reduce_ex__ of AttributeError object> __repr__ = <method-wrapper '__repr__' of AttributeError object> __setattr__ = <method-wrapper '__setattr__' of AttributeError object> __setstate__ = <built-in method __setstate__ of AttributeError object> __sizeof__ = <built-in method __sizeof__ of AttributeError object> __str__ = <method-wrapper '__str__' of AttributeError object> __subclasshook__ = <built-in method __subclasshook__ of type object> __suppress_context__ = False __traceback__ = <traceback object> args = ("'DeviceManagerLogger' object has no attribute 'warn'",) with_traceback = <built-in method with_traceback of AttributeError object> The above is a description of an error in a Python program. Here is the original traceback: Traceback (most recent call last): File "/root/work/build/debug/config/device-manager/device_manager/db.py", line 1044, in set_config_state self.nc_q.put_nowait(1) File "src/gevent/queue.py", line 290, in gevent._gevent_cqueue.Queue.put_nowait File "src/gevent/queue.py", line 296, in gevent._gevent_cqueue.Queue.put_nowait File "src/gevent/queue.py", line 288, in gevent._gevent_cqueue.Queue.put queue.Full During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/root/work/build/debug/config/device-manager/.tox/py36/lib/python3.6/site-packages/cfgm_common/vnc_amqp.py", line 84, in _vnc_subscribe_callback self.vnc_subscribe_actions() File "/root/work/build/debug/config/device-manager/.tox/py36/lib/python3.6/site-packages/cfgm_common/vnc_amqp.py", line 148, in vnc_subscribe_actions self.evaluate_dependency() File "/root/work/build/debug/config/device-manager/device_manager/dm_amqp.py", line 52, in evaluate_dependency pr.set_config_state() File "/root/work/build/debug/config/device-manager/device_manager/db.py", line 1046, in set_config_state self._logger.warn(f"queue is full") AttributeError: 'DeviceManagerLogger' object has no attribute 'warn' 04/27/2025 18:11:47.653 7f31993ddf60 [contrail-device-manager] [WARNING]: Inconsistency discovered. wanted=1, received=0, missing={'a9cecf50-5e21-4a9a-8362-1d1dad5d8b22'}. This may indicate that the cluster needs a 'nodetool repair'. 04/27/2025 18:11:47.653 7f31993ddf60 [contrail-device-manager] [WARNING]: logical_router uuid a9cecf50-5e21-4a9a-8362-1d1dad5d8b22 update caused NoIdError 04/27/2025 18:11:49.907 7f31993ddf60 [contrail-device-manager] [INFO]: Found feature plugin for pr=routertest.test_dm_ansible_dhcp_relay.TestAnsibleDhcpRelayDM.test_dhcp_relay_config_push_one_vn_inet0(fb0ef392-e5bf-4ca2-9233-e4471b1fc7b8), feature=vn-interconnect 04/27/2025 18:11:49.908 7f31993ddf60 [contrail-device-manager] [INFO]: Found feature plugin for pr=routertest.test_dm_ansible_dhcp_relay.TestAnsibleDhcpRelayDM.test_dhcp_relay_config_push_one_vn_inet0(fb0ef392-e5bf-4ca2-9233-e4471b1fc7b8), feature=l3-gateway 04/27/2025 18:11:49.908 7f31993ddf60 [contrail-device-manager] [INFO]: Found ansible plugin pr=fb0ef392-e5bf-4ca2-9233-e4471b1fc7b8, role/vendor/product=spine:juniper:qfx10008 04/27/2025 18:11:49.949 7f31993ddf60 [contrail-device-manager] [INFO]: Found feature plugin for pr=routertest.test_dm_ansible_dhcp_relay.TestAnsibleDhcpRelayDM.test_dhcp_relay_config_push_one_vn_inet0(fb0ef392-e5bf-4ca2-9233-e4471b1fc7b8), feature=vn-interconnect 04/27/2025 18:11:49.949 7f31993ddf60 [contrail-device-manager] [INFO]: Found feature plugin for pr=routertest.test_dm_ansible_dhcp_relay.TestAnsibleDhcpRelayDM.test_dhcp_relay_config_push_one_vn_inet0(fb0ef392-e5bf-4ca2-9233-e4471b1fc7b8), feature=l3-gateway 04/27/2025 18:11:49.950 7f31993ddf60 [contrail-device-manager] [INFO]: Found ansible plugin pr=fb0ef392-e5bf-4ca2-9233-e4471b1fc7b8, role/vendor/product=spine:juniper:qfx10008 04/27/2025 18:11:50.292 7f31993ddf60 [contrail-device-manager] [ERROR]: AttributeError Python 3.6.8: /root/work/build/debug/config/device-manager/.tox/py36/bin/python Sun Apr 27 18:11:50 2025 A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred. /root/work/build/debug/config/device-manager/.tox/py36/lib/python3.6/site-packages/cfgm_common/vnc_amqp.py in _vnc_subscribe_callback(self=<device_manager.dm_amqp.DMAmqpHandle object>, oper_info={'fq_name': ['default-domain', 'default-project', '__contrail_lr_internal_vn_df80d7d8-7a25-4d20-a2cb-629eb45ccaa0__', '__contrail_lr_internal_vn_df80d7d8-7a25-4d20-a2cb-629eb45ccaa0__'], 'obj_dict': {'display_name': '__contrail_lr_internal_vn_df80d7d8-7a25-4d20-a2cb-629eb45ccaa0__', 'fq_name': ['default-domain', 'default-project', '__contrail_lr_internal_vn_df80d7d8-7a25-4d20-a2cb-629eb45ccaa0__', '__contrail_lr_internal_vn_df80d7d8-7a25-4d20-a2cb-629eb45ccaa0__'], 'id_perms': {'created': '2025-04-28T01:11:50.257595', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-04-28T01:11:50.257595', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'contrail-api', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10145311971740659186, 'uuid_mslong': 2382793508795337614}}, 'parent_type': 'virtual-network', 'parent_uuid': 'bcb5d178-3d08-4ec1-9242-f19967d3cf24', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'routing_instance_fabric_snat': False, 'routing_instance_has_pnf': False, 'routing_instance_is_default': True, 'uuid': '21116212-58da-4b8e-8ccb-637c731eadf2'}, 'oper': 'CREATE', 'request-id': 'req-d7d728e4-d3f3-44be-bbcd-68777611f293', 'type': 'routing_instance', 'uuid': '21116212-58da-4b8e-8ccb-637c731eadf2'}) 82 try: 83 self.oper_info = oper_info 84 self.vnc_subscribe_actions() 85 86 except ConnectionError: self = <device_manager.dm_amqp.DMAmqpHandle object> self.vnc_subscribe_actions = <bound method VncAmqpHandle.vnc_subscribe_actions of <device_manager.dm_amqp.DMAmqpHandle object>> /root/work/build/debug/config/device-manager/.tox/py36/lib/python3.6/site-packages/cfgm_common/vnc_amqp.py in vnc_subscribe_actions(self=<device_manager.dm_amqp.DMAmqpHandle object>) 146 (self. obj_type, obj_id, oper)) 147 return 148 self.evaluate_dependency() 149 150 def _get_key_from_oper_info(self): self = <device_manager.dm_amqp.DMAmqpHandle object> self.evaluate_dependency = <bound method DMAmqpHandle.evaluate_dependency of <device_manager.dm_amqp.DMAmqpHandle object>> /root/work/build/debug/config/device-manager/device_manager/dm_amqp.py in evaluate_dependency(self=<device_manager.dm_amqp.DMAmqpHandle object>) 49 []): 50 pr = PhysicalRouterDM.get(pr_id) 51 if pr is not None: 52 pr.set_config_state() 53 pr.uve_send() pr = <device_manager.db.PhysicalRouterDM object> pr.set_config_state = <bound method PhysicalRouterDM.set_config_state of <device_manager.db.PhysicalRouterDM object>> /root/work/build/debug/config/device-manager/device_manager/db.py in set_config_state(self=<device_manager.db.PhysicalRouterDM object>, reset_retry_count=True) 1044 self.nc_q.put_nowait(1) 1045 except queue.Full: 1046 self._logger.warn(f"queue is full") 1047 1048 # end self = <device_manager.db.PhysicalRouterDM object> self._logger = <device_manager.logger.DeviceManagerLogger object> self._logger.warn undefined AttributeError: 'DeviceManagerLogger' object has no attribute 'warn' __cause__ = None __class__ = <class 'AttributeError'> __context__ = Full() __delattr__ = <method-wrapper '__delattr__' of AttributeError object> __dict__ = {} __dir__ = <built-in method __dir__ of AttributeError object> __doc__ = 'Attribute not found.' __eq__ = <method-wrapper '__eq__' of AttributeError object> __format__ = <built-in method __format__ of AttributeError object> __ge__ = <method-wrapper '__ge__' of AttributeError object> __getattribute__ = <method-wrapper '__getattribute__' of AttributeError object> __gt__ = <method-wrapper '__gt__' of AttributeError object> __hash__ = <method-wrapper '__hash__' of AttributeError object> __init__ = <method-wrapper '__init__' of AttributeError object> __init_subclass__ = <built-in method __init_subclass__ of type object> __le__ = <method-wrapper '__le__' of AttributeError object> __lt__ = <method-wrapper '__lt__' of AttributeError object> __ne__ = <method-wrapper '__ne__' of AttributeError object> __new__ = <built-in method __new__ of type object> __reduce__ = <built-in method __reduce__ of AttributeError object> __reduce_ex__ = <built-in method __reduce_ex__ of AttributeError object> __repr__ = <method-wrapper '__repr__' of AttributeError object> __setattr__ = <method-wrapper '__setattr__' of AttributeError object> __setstate__ = <built-in method __setstate__ of AttributeError object> __sizeof__ = <built-in method __sizeof__ of AttributeError object> __str__ = <method-wrapper '__str__' of AttributeError object> __subclasshook__ = <built-in method __subclasshook__ of type object> __suppress_context__ = False __traceback__ = <traceback object> args = ("'DeviceManagerLogger' object has no attribute 'warn'",) with_traceback = <built-in method with_traceback of AttributeError object> The above is a description of an error in a Python program. Here is the original traceback: Traceback (most recent call last): File "/root/work/build/debug/config/device-manager/device_manager/db.py", line 1044, in set_config_state self.nc_q.put_nowait(1) File "src/gevent/queue.py", line 290, in gevent._gevent_cqueue.Queue.put_nowait File "src/gevent/queue.py", line 296, in gevent._gevent_cqueue.Queue.put_nowait File "src/gevent/queue.py", line 288, in gevent._gevent_cqueue.Queue.put queue.Full During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/root/work/build/debug/config/device-manager/.tox/py36/lib/python3.6/site-packages/cfgm_common/vnc_amqp.py", line 84, in _vnc_subscribe_callback self.vnc_subscribe_actions() File "/root/work/build/debug/config/device-manager/.tox/py36/lib/python3.6/site-packages/cfgm_common/vnc_amqp.py", line 148, in vnc_subscribe_actions self.evaluate_dependency() File "/root/work/build/debug/config/device-manager/device_manager/dm_amqp.py", line 52, in evaluate_dependency pr.set_config_state() File "/root/work/build/debug/config/device-manager/device_manager/db.py", line 1046, in set_config_state self._logger.warn(f"queue is full") AttributeError: 'DeviceManagerLogger' object has no attribute 'warn' 04/27/2025 18:11:50.299 7f31993ddf60 [contrail-device-manager] [ERROR]: AttributeError Python 3.6.8: /root/work/build/debug/config/device-manager/.tox/py36/bin/python Sun Apr 27 18:11:50 2025 A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred. /root/work/build/debug/config/device-manager/.tox/py36/lib/python3.6/site-packages/cfgm_common/vnc_amqp.py in _vnc_subscribe_callback(self=<device_manager.dm_amqp.DMAmqpHandle object>, oper_info={'fq_name': ['default-domain', 'default-project', 'lr-test.test_dm_ansible_dhcp_relay.TestAnsibleDhcpRelayDM.test_dhcp_relay_config_push_one_vn_inet0'], 'oper': 'UPDATE', 'request-id': 'req-d7d728e4-d3f3-44be-bbcd-68777611f293', 'type': 'logical_router', 'uuid': 'df80d7d8-7a25-4d20-a2cb-629eb45ccaa0'}) 82 try: 83 self.oper_info = oper_info 84 self.vnc_subscribe_actions() 85 86 except ConnectionError: self = <device_manager.dm_amqp.DMAmqpHandle object> self.vnc_subscribe_actions = <bound method VncAmqpHandle.vnc_subscribe_actions of <device_manager.dm_amqp.DMAmqpHandle object>> /root/work/build/debug/config/device-manager/.tox/py36/lib/python3.6/site-packages/cfgm_common/vnc_amqp.py in vnc_subscribe_actions(self=<device_manager.dm_amqp.DMAmqpHandle object>) 146 (self. obj_type, obj_id, oper)) 147 return 148 self.evaluate_dependency() 149 150 def _get_key_from_oper_info(self): self = <device_manager.dm_amqp.DMAmqpHandle object> self.evaluate_dependency = <bound method DMAmqpHandle.evaluate_dependency of <device_manager.dm_amqp.DMAmqpHandle object>> /root/work/build/debug/config/device-manager/device_manager/dm_amqp.py in evaluate_dependency(self=<device_manager.dm_amqp.DMAmqpHandle object>) 49 []): 50 pr = PhysicalRouterDM.get(pr_id) 51 if pr is not None: 52 pr.set_config_state() 53 pr.uve_send() pr = <device_manager.db.PhysicalRouterDM object> pr.set_config_state = <bound method PhysicalRouterDM.set_config_state of <device_manager.db.PhysicalRouterDM object>> /root/work/build/debug/config/device-manager/device_manager/db.py in set_config_state(self=<device_manager.db.PhysicalRouterDM object>, reset_retry_count=True) 1044 self.nc_q.put_nowait(1) 1045 except queue.Full: 1046 self._logger.warn(f"queue is full") 1047 1048 # end self = <device_manager.db.PhysicalRouterDM object> self._logger = <device_manager.logger.DeviceManagerLogger object> self._logger.warn undefined AttributeError: 'DeviceManagerLogger' object has no attribute 'warn' __cause__ = None __class__ = <class 'AttributeError'> __context__ = Full() __delattr__ = <method-wrapper '__delattr__' of AttributeError object> __dict__ = {} __dir__ = <built-in method __dir__ of AttributeError object> __doc__ = 'Attribute not found.' __eq__ = <method-wrapper '__eq__' of AttributeError object> __format__ = <built-in method __format__ of AttributeError object> __ge__ = <method-wrapper '__ge__' of AttributeError object> __getattribute__ = <method-wrapper '__getattribute__' of AttributeError object> __gt__ = <method-wrapper '__gt__' of AttributeError object> __hash__ = <method-wrapper '__hash__' of AttributeError object> __init__ = <method-wrapper '__init__' of AttributeError object> __init_subclass__ = <built-in method __init_subclass__ of type object> __le__ = <method-wrapper '__le__' of AttributeError object> __lt__ = <method-wrapper '__lt__' of AttributeError object> __ne__ = <method-wrapper '__ne__' of AttributeError object> __new__ = <built-in method __new__ of type object> __reduce__ = <built-in method __reduce__ of AttributeError object> __reduce_ex__ = <built-in method __reduce_ex__ of AttributeError object> __repr__ = <method-wrapper '__repr__' of AttributeError object> __setattr__ = <method-wrapper '__setattr__' of AttributeError object> __setstate__ = <built-in method __setstate__ of AttributeError object> __sizeof__ = <built-in method __sizeof__ of AttributeError object> __str__ = <method-wrapper '__str__' of AttributeError object> __subclasshook__ = <built-in method __subclasshook__ of type object> __suppress_context__ = False __traceback__ = <traceback object> args = ("'DeviceManagerLogger' object has no attribute 'warn'",) with_traceback = <built-in method with_traceback of AttributeError object> The above is a description of an error in a Python program. Here is the original traceback: Traceback (most recent call last): File "/root/work/build/debug/config/device-manager/device_manager/db.py", line 1044, in set_config_state self.nc_q.put_nowait(1) File "src/gevent/queue.py", line 290, in gevent._gevent_cqueue.Queue.put_nowait File "src/gevent/queue.py", line 296, in gevent._gevent_cqueue.Queue.put_nowait File "src/gevent/queue.py", line 288, in gevent._gevent_cqueue.Queue.put queue.Full During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/root/work/build/debug/config/device-manager/.tox/py36/lib/python3.6/site-packages/cfgm_common/vnc_amqp.py", line 84, in _vnc_subscribe_callback self.vnc_subscribe_actions() File "/root/work/build/debug/config/device-manager/.tox/py36/lib/python3.6/site-packages/cfgm_common/vnc_amqp.py", line 148, in vnc_subscribe_actions self.evaluate_dependency() File "/root/work/build/debug/config/device-manager/device_manager/dm_amqp.py", line 52, in evaluate_dependency pr.set_config_state() File "/root/work/build/debug/config/device-manager/device_manager/db.py", line 1046, in set_config_state self._logger.warn(f"queue is full") AttributeError: 'DeviceManagerLogger' object has no attribute 'warn' 04/27/2025 18:11:50.386 7f31993ddf60 [contrail-device-manager] [WARNING]: Inconsistency discovered. wanted=1, received=0, missing={'df80d7d8-7a25-4d20-a2cb-629eb45ccaa0'}. This may indicate that the cluster needs a 'nodetool repair'. 04/27/2025 18:11:50.386 7f31993ddf60 [contrail-device-manager] [WARNING]: logical_router uuid df80d7d8-7a25-4d20-a2cb-629eb45ccaa0 update caused NoIdError 04/27/2025 18:11:52.718 7f31993ddf60 [contrail-device-manager] [INFO]: Found feature plugin for pr=routertest.test_dm_ansible_dhcp_relay.TestAnsibleDhcpRelayDM.test_dhcp_relay_config_push_one_vn_two_dhcp_servers(b6a367de-6981-40f0-b7e8-a1ad781e4729), feature=l3-gateway 04/27/2025 18:11:52.718 7f31993ddf60 [contrail-device-manager] [INFO]: Found feature plugin for pr=routertest.test_dm_ansible_dhcp_relay.TestAnsibleDhcpRelayDM.test_dhcp_relay_config_push_one_vn_two_dhcp_servers(b6a367de-6981-40f0-b7e8-a1ad781e4729), feature=vn-interconnect 04/27/2025 18:11:52.718 7f31993ddf60 [contrail-device-manager] [INFO]: Found ansible plugin pr=b6a367de-6981-40f0-b7e8-a1ad781e4729, role/vendor/product=spine:juniper:qfx10008 04/27/2025 18:11:52.762 7f31993ddf60 [contrail-device-manager] [INFO]: Found feature plugin for pr=routertest.test_dm_ansible_dhcp_relay.TestAnsibleDhcpRelayDM.test_dhcp_relay_config_push_one_vn_two_dhcp_servers(b6a367de-6981-40f0-b7e8-a1ad781e4729), feature=l3-gateway 04/27/2025 18:11:52.762 7f31993ddf60 [contrail-device-manager] [INFO]: Found feature plugin for pr=routertest.test_dm_ansible_dhcp_relay.TestAnsibleDhcpRelayDM.test_dhcp_relay_config_push_one_vn_two_dhcp_servers(b6a367de-6981-40f0-b7e8-a1ad781e4729), feature=vn-interconnect 04/27/2025 18:11:52.762 7f31993ddf60 [contrail-device-manager] [INFO]: Found ansible plugin pr=b6a367de-6981-40f0-b7e8-a1ad781e4729, role/vendor/product=spine:juniper:qfx10008 04/27/2025 18:11:53.080 7f31993ddf60 [contrail-device-manager] [ERROR]: AttributeError Python 3.6.8: /root/work/build/debug/config/device-manager/.tox/py36/bin/python Sun Apr 27 18:11:53 2025 A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred. /root/work/build/debug/config/device-manager/.tox/py36/lib/python3.6/site-packages/cfgm_common/vnc_amqp.py in _vnc_subscribe_callback(self=<device_manager.dm_amqp.DMAmqpHandle object>, oper_info={'fq_name': ['default-domain', 'default-project', '__contrail_lr_internal_vn_cc86e93e-f794-42d1-ad28-4fc68b353a57__', '__contrail_lr_internal_vn_cc86e93e-f794-42d1-ad28-4fc68b353a57__'], 'obj_dict': {'display_name': '__contrail_lr_internal_vn_cc86e93e-f794-42d1-ad28-4fc68b353a57__', 'fq_name': ['default-domain', 'default-project', '__contrail_lr_internal_vn_cc86e93e-f794-42d1-ad28-4fc68b353a57__', '__contrail_lr_internal_vn_cc86e93e-f794-42d1-ad28-4fc68b353a57__'], 'id_perms': {'created': '2025-04-28T01:11:53.050108', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-04-28T01:11:53.050108', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'contrail-api', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10793861003772453931, 'uuid_mslong': 8816482420314554416}}, 'parent_type': 'virtual-network', 'parent_uuid': 'dc7a4b99-83ef-469a-b836-0f4d5a041047', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'routing_instance_fabric_snat': False, 'routing_instance_has_pnf': False, 'routing_instance_is_default': True, 'uuid': '7a5a700c-af53-4030-95cb-7f6504cc642b'}, 'oper': 'CREATE', 'request-id': 'req-d10efdf1-1725-4021-ac6d-5bbc6fd1822c', 'type': 'routing_instance', 'uuid': '7a5a700c-af53-4030-95cb-7f6504cc642b'}) 82 try: 83 self.oper_info = oper_info 84 self.vnc_subscribe_actions() 85 86 except ConnectionError: self = <device_manager.dm_amqp.DMAmqpHandle object> self.vnc_subscribe_actions = <bound method VncAmqpHandle.vnc_subscribe_actions of <device_manager.dm_amqp.DMAmqpHandle object>> /root/work/build/debug/config/device-manager/.tox/py36/lib/python3.6/site-packages/cfgm_common/vnc_amqp.py in vnc_subscribe_actions(self=<device_manager.dm_amqp.DMAmqpHandle object>) 146 (self. obj_type, obj_id, oper)) 147 return 148 self.evaluate_dependency() 149 150 def _get_key_from_oper_info(self): self = <device_manager.dm_amqp.DMAmqpHandle object> self.evaluate_dependency = <bound method DMAmqpHandle.evaluate_dependency of <device_manager.dm_amqp.DMAmqpHandle object>> /root/work/build/debug/config/device-manager/device_manager/dm_amqp.py in evaluate_dependency(self=<device_manager.dm_amqp.DMAmqpHandle object>) 49 []): 50 pr = PhysicalRouterDM.get(pr_id) 51 if pr is not None: 52 pr.set_config_state() 53 pr.uve_send() pr = <device_manager.db.PhysicalRouterDM object> pr.set_config_state = <bound method PhysicalRouterDM.set_config_state of <device_manager.db.PhysicalRouterDM object>> /root/work/build/debug/config/device-manager/device_manager/db.py in set_config_state(self=<device_manager.db.PhysicalRouterDM object>, reset_retry_count=True) 1044 self.nc_q.put_nowait(1) 1045 except queue.Full: 1046 self._logger.warn(f"queue is full") 1047 1048 # end self = <device_manager.db.PhysicalRouterDM object> self._logger = <device_manager.logger.DeviceManagerLogger object> self._logger.warn undefined AttributeError: 'DeviceManagerLogger' object has no attribute 'warn' __cause__ = None __class__ = <class 'AttributeError'> __context__ = Full() __delattr__ = <method-wrapper '__delattr__' of AttributeError object> __dict__ = {} __dir__ = <built-in method __dir__ of AttributeError object> __doc__ = 'Attribute not found.' __eq__ = <method-wrapper '__eq__' of AttributeError object> __format__ = <built-in method __format__ of AttributeError object> __ge__ = <method-wrapper '__ge__' of AttributeError object> __getattribute__ = <method-wrapper '__getattribute__' of AttributeError object> __gt__ = <method-wrapper '__gt__' of AttributeError object> __hash__ = <method-wrapper '__hash__' of AttributeError object> __init__ = <method-wrapper '__init__' of AttributeError object> __init_subclass__ = <built-in method __init_subclass__ of type object> __le__ = <method-wrapper '__le__' of AttributeError object> __lt__ = <method-wrapper '__lt__' of AttributeError object> __ne__ = <method-wrapper '__ne__' of AttributeError object> __new__ = <built-in method __new__ of type object> __reduce__ = <built-in method __reduce__ of AttributeError object> __reduce_ex__ = <built-in method __reduce_ex__ of AttributeError object> __repr__ = <method-wrapper '__repr__' of AttributeError object> __setattr__ = <method-wrapper '__setattr__' of AttributeError object> __setstate__ = <built-in method __setstate__ of AttributeError object> __sizeof__ = <built-in method __sizeof__ of AttributeError object> __str__ = <method-wrapper '__str__' of AttributeError object> __subclasshook__ = <built-in method __subclasshook__ of type object> __suppress_context__ = False __traceback__ = <traceback object> args = ("'DeviceManagerLogger' object has no attribute 'warn'",) with_traceback = <built-in method with_traceback of AttributeError object> The above is a description of an error in a Python program. Here is the original traceback: Traceback (most recent call last): File "/root/work/build/debug/config/device-manager/device_manager/db.py", line 1044, in set_config_state self.nc_q.put_nowait(1) File "src/gevent/queue.py", line 290, in gevent._gevent_cqueue.Queue.put_nowait File "src/gevent/queue.py", line 296, in gevent._gevent_cqueue.Queue.put_nowait File "src/gevent/queue.py", line 288, in gevent._gevent_cqueue.Queue.put queue.Full During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/root/work/build/debug/config/device-manager/.tox/py36/lib/python3.6/site-packages/cfgm_common/vnc_amqp.py", line 84, in _vnc_subscribe_callback self.vnc_subscribe_actions() File "/root/work/build/debug/config/device-manager/.tox/py36/lib/python3.6/site-packages/cfgm_common/vnc_amqp.py", line 148, in vnc_subscribe_actions self.evaluate_dependency() File "/root/work/build/debug/config/device-manager/device_manager/dm_amqp.py", line 52, in evaluate_dependency pr.set_config_state() File "/root/work/build/debug/config/device-manager/device_manager/db.py", line 1046, in set_config_state self._logger.warn(f"queue is full") AttributeError: 'DeviceManagerLogger' object has no attribute 'warn' 04/27/2025 18:11:53.161 7f31993ddf60 [contrail-device-manager] [WARNING]: Inconsistency discovered. wanted=1, received=0, missing={'cc86e93e-f794-42d1-ad28-4fc68b353a57'}. This may indicate that the cluster needs a 'nodetool repair'. 04/27/2025 18:11:53.161 7f31993ddf60 [contrail-device-manager] [WARNING]: logical_router uuid cc86e93e-f794-42d1-ad28-4fc68b353a57 update caused NoIdError 04/27/2025 18:11:55.392 7f31993ddf60 [contrail-device-manager] [INFO]: Found feature plugin for pr=routertest.test_dm_ansible_dhcp_relay.TestAnsibleDhcpRelayDM.test_dhcp_relay_config_push_two_lrs_with_dhcp_in_one(1c209e20-fcde-4f7f-9a1d-500d500f5d06), feature=l3-gateway 04/27/2025 18:11:55.392 7f31993ddf60 [contrail-device-manager] [INFO]: Found feature plugin for pr=routertest.test_dm_ansible_dhcp_relay.TestAnsibleDhcpRelayDM.test_dhcp_relay_config_push_two_lrs_with_dhcp_in_one(1c209e20-fcde-4f7f-9a1d-500d500f5d06), feature=vn-interconnect 04/27/2025 18:11:55.392 7f31993ddf60 [contrail-device-manager] [INFO]: Found ansible plugin pr=1c209e20-fcde-4f7f-9a1d-500d500f5d06, role/vendor/product=spine:juniper:qfx10008 04/27/2025 18:11:55.434 7f31993ddf60 [contrail-device-manager] [INFO]: Found feature plugin for pr=routertest.test_dm_ansible_dhcp_relay.TestAnsibleDhcpRelayDM.test_dhcp_relay_config_push_two_lrs_with_dhcp_in_one(1c209e20-fcde-4f7f-9a1d-500d500f5d06), feature=l3-gateway 04/27/2025 18:11:55.434 7f31993ddf60 [contrail-device-manager] [INFO]: Found feature plugin for pr=routertest.test_dm_ansible_dhcp_relay.TestAnsibleDhcpRelayDM.test_dhcp_relay_config_push_two_lrs_with_dhcp_in_one(1c209e20-fcde-4f7f-9a1d-500d500f5d06), feature=vn-interconnect 04/27/2025 18:11:55.435 7f31993ddf60 [contrail-device-manager] [INFO]: Found ansible plugin pr=1c209e20-fcde-4f7f-9a1d-500d500f5d06, role/vendor/product=spine:juniper:qfx10008 04/27/2025 18:11:55.750 7f31993ddf60 [contrail-device-manager] [ERROR]: AttributeError Python 3.6.8: /root/work/build/debug/config/device-manager/.tox/py36/bin/python Sun Apr 27 18:11:55 2025 A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred. /root/work/build/debug/config/device-manager/.tox/py36/lib/python3.6/site-packages/cfgm_common/vnc_amqp.py in _vnc_subscribe_callback(self=<device_manager.dm_amqp.DMAmqpHandle object>, oper_info={'fq_name': ['default-domain', 'default-project', '__contrail_lr_internal_vn_8c5ee1dd-8cef-409a-b4dd-8e9371596b9c__', '__contrail_lr_internal_vn_8c5ee1dd-8cef-409a-b4dd-8e9371596b9c__'], 'obj_dict': {'display_name': '__contrail_lr_internal_vn_8c5ee1dd-8cef-409a-b4dd-8e9371596b9c__', 'fq_name': ['default-domain', 'default-project', '__contrail_lr_internal_vn_8c5ee1dd-8cef-409a-b4dd-8e9371596b9c__', '__contrail_lr_internal_vn_8c5ee1dd-8cef-409a-b4dd-8e9371596b9c__'], 'id_perms': {'created': '2025-04-28T01:11:55.724392', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-04-28T01:11:55.724392', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'contrail-api', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12081507054996460405, 'uuid_mslong': 10090470703157035463}}, 'parent_type': 'virtual-network', 'parent_uuid': '05e52074-7f3e-4169-87f2-ea7ffa53eb05', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'routing_instance_fabric_snat': False, 'routing_instance_has_pnf': False, 'routing_instance_is_default': True, 'uuid': '8c088da4-9629-41c7-a7aa-22a7ec709375'}, 'oper': 'CREATE', 'request-id': 'req-3c9dbd32-f352-4c51-ab35-e88423a0cae0', 'type': 'routing_instance', 'uuid': '8c088da4-9629-41c7-a7aa-22a7ec709375'}) 82 try: 83 self.oper_info = oper_info 84 self.vnc_subscribe_actions() 85 86 except ConnectionError: self = <device_manager.dm_amqp.DMAmqpHandle object> self.vnc_subscribe_actions = <bound method VncAmqpHandle.vnc_subscribe_actions of <device_manager.dm_amqp.DMAmqpHandle object>> /root/work/build/debug/config/device-manager/.tox/py36/lib/python3.6/site-packages/cfgm_common/vnc_amqp.py in vnc_subscribe_actions(self=<device_manager.dm_amqp.DMAmqpHandle object>) 146 (self. obj_type, obj_id, oper)) 147 return 148 self.evaluate_dependency() 149 150 def _get_key_from_oper_info(self): self = <device_manager.dm_amqp.DMAmqpHandle object> self.evaluate_dependency = <bound method DMAmqpHandle.evaluate_dependency of <device_manager.dm_amqp.DMAmqpHandle object>> /root/work/build/debug/config/device-manager/device_manager/dm_amqp.py in evaluate_dependency(self=<device_manager.dm_amqp.DMAmqpHandle object>) 49 []): 50 pr = PhysicalRouterDM.get(pr_id) 51 if pr is not None: 52 pr.set_config_state() 53 pr.uve_send() pr = <device_manager.db.PhysicalRouterDM object> pr.set_config_state = <bound method PhysicalRouterDM.set_config_state of <device_manager.db.PhysicalRouterDM object>> /root/work/build/debug/config/device-manager/device_manager/db.py in set_config_state(self=<device_manager.db.PhysicalRouterDM object>, reset_retry_count=True) 1044 self.nc_q.put_nowait(1) 1045 except queue.Full: 1046 self._logger.warn(f"queue is full") 1047 1048 # end self = <device_manager.db.PhysicalRouterDM object> self._logger = <device_manager.logger.DeviceManagerLogger object> self._logger.warn undefined AttributeError: 'DeviceManagerLogger' object has no attribute 'warn' __cause__ = None __class__ = <class 'AttributeError'> __context__ = Full() __delattr__ = <method-wrapper '__delattr__' of AttributeError object> __dict__ = {} __dir__ = <built-in method __dir__ of AttributeError object> __doc__ = 'Attribute not found.' __eq__ = <method-wrapper '__eq__' of AttributeError object> __format__ = <built-in method __format__ of AttributeError object> __ge__ = <method-wrapper '__ge__' of AttributeError object> __getattribute__ = <method-wrapper '__getattribute__' of AttributeError object> __gt__ = <method-wrapper '__gt__' of AttributeError object> __hash__ = <method-wrapper '__hash__' of AttributeError object> __init__ = <method-wrapper '__init__' of AttributeError object> __init_subclass__ = <built-in method __init_subclass__ of type object> __le__ = <method-wrapper '__le__' of AttributeError object> __lt__ = <method-wrapper '__lt__' of AttributeError object> __ne__ = <method-wrapper '__ne__' of AttributeError object> __new__ = <built-in method __new__ of type object> __reduce__ = <built-in method __reduce__ of AttributeError object> __reduce_ex__ = <built-in method __reduce_ex__ of AttributeError object> __repr__ = <method-wrapper '__repr__' of AttributeError object> __setattr__ = <method-wrapper '__setattr__' of AttributeError object> __setstate__ = <built-in method __setstate__ of AttributeError object> __sizeof__ = <built-in method __sizeof__ of AttributeError object> __str__ = <method-wrapper '__str__' of AttributeError object> __subclasshook__ = <built-in method __subclasshook__ of type object> __suppress_context__ = False __traceback__ = <traceback object> args = ("'DeviceManagerLogger' object has no attribute 'warn'",) with_traceback = <built-in method with_traceback of AttributeError object> The above is a description of an error in a Python program. Here is the original traceback: Traceback (most recent call last): File "/root/work/build/debug/config/device-manager/device_manager/db.py", line 1044, in set_config_state self.nc_q.put_nowait(1) File "src/gevent/queue.py", line 290, in gevent._gevent_cqueue.Queue.put_nowait File "src/gevent/queue.py", line 296, in gevent._gevent_cqueue.Queue.put_nowait File "src/gevent/queue.py", line 288, in gevent._gevent_cqueue.Queue.put queue.Full During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/root/work/build/debug/config/device-manager/.tox/py36/lib/python3.6/site-packages/cfgm_common/vnc_amqp.py", line 84, in _vnc_subscribe_callback self.vnc_subscribe_actions() File "/root/work/build/debug/config/device-manager/.tox/py36/lib/python3.6/site-packages/cfgm_common/vnc_amqp.py", line 148, in vnc_subscribe_actions self.evaluate_dependency() File "/root/work/build/debug/config/device-manager/device_manager/dm_amqp.py", line 52, in evaluate_dependency pr.set_config_state() File "/root/work/build/debug/config/device-manager/device_manager/db.py", line 1046, in set_config_state self._logger.warn(f"queue is full") AttributeError: 'DeviceManagerLogger' object has no attribute 'warn' 04/27/2025 18:11:56.066 7f31993ddf60 [contrail-device-manager] [ERROR]: AttributeError Python 3.6.8: /root/work/build/debug/config/device-manager/.tox/py36/bin/python Sun Apr 27 18:11:56 2025 A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred. /root/work/build/debug/config/device-manager/.tox/py36/lib/python3.6/site-packages/cfgm_common/vnc_amqp.py in _vnc_subscribe_callback(self=<device_manager.dm_amqp.DMAmqpHandle object>, oper_info={'fq_name': ['default-domain', 'default-project', '__contrail_lr_internal_vn_e24815d6-0232-453e-b724-1225b94c2c1b__', '__contrail_lr_internal_vn_e24815d6-0232-453e-b724-1225b94c2c1b__'], 'obj_dict': {'display_name': '__contrail_lr_internal_vn_e24815d6-0232-453e-b724-1225b94c2c1b__', 'fq_name': ['default-domain', 'default-project', '__contrail_lr_internal_vn_e24815d6-0232-453e-b724-1225b94c2c1b__', '__contrail_lr_internal_vn_e24815d6-0232-453e-b724-1225b94c2c1b__'], 'id_perms': {'created': '2025-04-28T01:11:56.040463', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-04-28T01:11:56.040463', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'contrail-api', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13139224137155010291, 'uuid_mslong': 338007979147479958}}, 'parent_type': 'virtual-network', 'parent_uuid': '2c8d4ee5-55cd-46bf-b616-6855e6f3c2f0', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'routing_instance_fabric_snat': False, 'routing_instance_has_pnf': False, 'routing_instance_is_default': True, 'uuid': '04b0d877-5855-4b96-b657-e6bcf75cdef3'}, 'oper': 'CREATE', 'request-id': 'req-3d7f43d4-a2e2-4f46-92d2-d3bb4253493c', 'type': 'routing_instance', 'uuid': '04b0d877-5855-4b96-b657-e6bcf75cdef3'}) 82 try: 83 self.oper_info = oper_info 84 self.vnc_subscribe_actions() 85 86 except ConnectionError: self = <device_manager.dm_amqp.DMAmqpHandle object> self.vnc_subscribe_actions = <bound method VncAmqpHandle.vnc_subscribe_actions of <device_manager.dm_amqp.DMAmqpHandle object>> /root/work/build/debug/config/device-manager/.tox/py36/lib/python3.6/site-packages/cfgm_common/vnc_amqp.py in vnc_subscribe_actions(self=<device_manager.dm_amqp.DMAmqpHandle object>) 146 (self. obj_type, obj_id, oper)) 147 return 148 self.evaluate_dependency() 149 150 def _get_key_from_oper_info(self): self = <device_manager.dm_amqp.DMAmqpHandle object> self.evaluate_dependency = <bound method DMAmqpHandle.evaluate_dependency of <device_manager.dm_amqp.DMAmqpHandle object>> /root/work/build/debug/config/device-manager/device_manager/dm_amqp.py in evaluate_dependency(self=<device_manager.dm_amqp.DMAmqpHandle object>) 49 []): 50 pr = PhysicalRouterDM.get(pr_id) 51 if pr is not None: 52 pr.set_config_state() 53 pr.uve_send() pr = <device_manager.db.PhysicalRouterDM object> pr.set_config_state = <bound method PhysicalRouterDM.set_config_state of <device_manager.db.PhysicalRouterDM object>> /root/work/build/debug/config/device-manager/device_manager/db.py in set_config_state(self=<device_manager.db.PhysicalRouterDM object>, reset_retry_count=True) 1044 self.nc_q.put_nowait(1) 1045 except queue.Full: 1046 self._logger.warn(f"queue is full") 1047 1048 # end self = <device_manager.db.PhysicalRouterDM object> self._logger = <device_manager.logger.DeviceManagerLogger object> self._logger.warn undefined AttributeError: 'DeviceManagerLogger' object has no attribute 'warn' __cause__ = None __class__ = <class 'AttributeError'> __context__ = Full() __delattr__ = <method-wrapper '__delattr__' of AttributeError object> __dict__ = {} __dir__ = <built-in method __dir__ of AttributeError object> __doc__ = 'Attribute not found.' __eq__ = <method-wrapper '__eq__' of AttributeError object> __format__ = <built-in method __format__ of AttributeError object> __ge__ = <method-wrapper '__ge__' of AttributeError object> __getattribute__ = <method-wrapper '__getattribute__' of AttributeError object> __gt__ = <method-wrapper '__gt__' of AttributeError object> __hash__ = <method-wrapper '__hash__' of AttributeError object> __init__ = <method-wrapper '__init__' of AttributeError object> __init_subclass__ = <built-in method __init_subclass__ of type object> __le__ = <method-wrapper '__le__' of AttributeError object> __lt__ = <method-wrapper '__lt__' of AttributeError object> __ne__ = <method-wrapper '__ne__' of AttributeError object> __new__ = <built-in method __new__ of type object> __reduce__ = <built-in method __reduce__ of AttributeError object> __reduce_ex__ = <built-in method __reduce_ex__ of AttributeError object> __repr__ = <method-wrapper '__repr__' of AttributeError object> __setattr__ = <method-wrapper '__setattr__' of AttributeError object> __setstate__ = <built-in method __setstate__ of AttributeError object> __sizeof__ = <built-in method __sizeof__ of AttributeError object> __str__ = <method-wrapper '__str__' of AttributeError object> __subclasshook__ = <built-in method __subclasshook__ of type object> __suppress_context__ = False __traceback__ = <traceback object> args = ("'DeviceManagerLogger' object has no attribute 'warn'",) with_traceback = <built-in method with_traceback of AttributeError object> The above is a description of an error in a Python program. Here is the original traceback: Traceback (most recent call last): File "/root/work/build/debug/config/device-manager/device_manager/db.py", line 1044, in set_config_state self.nc_q.put_nowait(1) File "src/gevent/queue.py", line 290, in gevent._gevent_cqueue.Queue.put_nowait File "src/gevent/queue.py", line 296, in gevent._gevent_cqueue.Queue.put_nowait File "src/gevent/queue.py", line 288, in gevent._gevent_cqueue.Queue.put queue.Full During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/root/work/build/debug/config/device-manager/.tox/py36/lib/python3.6/site-packages/cfgm_common/vnc_amqp.py", line 84, in _vnc_subscribe_callback self.vnc_subscribe_actions() File "/root/work/build/debug/config/device-manager/.tox/py36/lib/python3.6/site-packages/cfgm_common/vnc_amqp.py", line 148, in vnc_subscribe_actions self.evaluate_dependency() File "/root/work/build/debug/config/device-manager/device_manager/dm_amqp.py", line 52, in evaluate_dependency pr.set_config_state() File "/root/work/build/debug/config/device-manager/device_manager/db.py", line 1046, in set_config_state self._logger.warn(f"queue is full") AttributeError: 'DeviceManagerLogger' object has no attribute 'warn' 04/27/2025 18:11:56.073 7f31993ddf60 [contrail-device-manager] [ERROR]: AttributeError Python 3.6.8: /root/work/build/debug/config/device-manager/.tox/py36/bin/python Sun Apr 27 18:11:56 2025 A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred. /root/work/build/debug/config/device-manager/.tox/py36/lib/python3.6/site-packages/cfgm_common/vnc_amqp.py in _vnc_subscribe_callback(self=<device_manager.dm_amqp.DMAmqpHandle object>, oper_info={'fq_name': ['default-domain', 'default-project', 'lr2-test.test_dm_ansible_dhcp_relay.TestAnsibleD...t_dhcp_relay_config_push_two_lrs_with_dhcp_in_one'], 'oper': 'UPDATE', 'request-id': 'req-3d7f43d4-a2e2-4f46-92d2-d3bb4253493c', 'type': 'logical_router', 'uuid': 'e24815d6-0232-453e-b724-1225b94c2c1b'}) 82 try: 83 self.oper_info = oper_info 84 self.vnc_subscribe_actions() 85 86 except ConnectionError: self = <device_manager.dm_amqp.DMAmqpHandle object> self.vnc_subscribe_actions = <bound method VncAmqpHandle.vnc_subscribe_actions of <device_manager.dm_amqp.DMAmqpHandle object>> /root/work/build/debug/config/device-manager/.tox/py36/lib/python3.6/site-packages/cfgm_common/vnc_amqp.py in vnc_subscribe_actions(self=<device_manager.dm_amqp.DMAmqpHandle object>) 146 (self. obj_type, obj_id, oper)) 147 return 148 self.evaluate_dependency() 149 150 def _get_key_from_oper_info(self): self = <device_manager.dm_amqp.DMAmqpHandle object> self.evaluate_dependency = <bound method DMAmqpHandle.evaluate_dependency of <device_manager.dm_amqp.DMAmqpHandle object>> /root/work/build/debug/config/device-manager/device_manager/dm_amqp.py in evaluate_dependency(self=<device_manager.dm_amqp.DMAmqpHandle object>) 49 []): 50 pr = PhysicalRouterDM.get(pr_id) 51 if pr is not None: 52 pr.set_config_state() 53 pr.uve_send() pr = <device_manager.db.PhysicalRouterDM object> pr.set_config_state = <bound method PhysicalRouterDM.set_config_state of <device_manager.db.PhysicalRouterDM object>> /root/work/build/debug/config/device-manager/device_manager/db.py in set_config_state(self=<device_manager.db.PhysicalRouterDM object>, reset_retry_count=True) 1044 self.nc_q.put_nowait(1) 1045 except queue.Full: 1046 self._logger.warn(f"queue is full") 1047 1048 # end self = <device_manager.db.PhysicalRouterDM object> self._logger = <device_manager.logger.DeviceManagerLogger object> self._logger.warn undefined AttributeError: 'DeviceManagerLogger' object has no attribute 'warn' __cause__ = None __class__ = <class 'AttributeError'> __context__ = Full() __delattr__ = <method-wrapper '__delattr__' of AttributeError object> __dict__ = {} __dir__ = <built-in method __dir__ of AttributeError object> __doc__ = 'Attribute not found.' __eq__ = <method-wrapper '__eq__' of AttributeError object> __format__ = <built-in method __format__ of AttributeError object> __ge__ = <method-wrapper '__ge__' of AttributeError object> __getattribute__ = <method-wrapper '__getattribute__' of AttributeError object> __gt__ = <method-wrapper '__gt__' of AttributeError object> __hash__ = <method-wrapper '__hash__' of AttributeError object> __init__ = <method-wrapper '__init__' of AttributeError object> __init_subclass__ = <built-in method __init_subclass__ of type object> __le__ = <method-wrapper '__le__' of AttributeError object> __lt__ = <method-wrapper '__lt__' of AttributeError object> __ne__ = <method-wrapper '__ne__' of AttributeError object> __new__ = <built-in method __new__ of type object> __reduce__ = <built-in method __reduce__ of AttributeError object> __reduce_ex__ = <built-in method __reduce_ex__ of AttributeError object> __repr__ = <method-wrapper '__repr__' of AttributeError object> __setattr__ = <method-wrapper '__setattr__' of AttributeError object> __setstate__ = <built-in method __setstate__ of AttributeError object> __sizeof__ = <built-in method __sizeof__ of AttributeError object> __str__ = <method-wrapper '__str__' of AttributeError object> __subclasshook__ = <built-in method __subclasshook__ of type object> __suppress_context__ = False __traceback__ = <traceback object> args = ("'DeviceManagerLogger' object has no attribute 'warn'",) with_traceback = <built-in method with_traceback of AttributeError object> The above is a description of an error in a Python program. Here is the original traceback: Traceback (most recent call last): File "/root/work/build/debug/config/device-manager/device_manager/db.py", line 1044, in set_config_state self.nc_q.put_nowait(1) File "src/gevent/queue.py", line 290, in gevent._gevent_cqueue.Queue.put_nowait File "src/gevent/queue.py", line 296, in gevent._gevent_cqueue.Queue.put_nowait File "src/gevent/queue.py", line 288, in gevent._gevent_cqueue.Queue.put queue.Full During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/root/work/build/debug/config/device-manager/.tox/py36/lib/python3.6/site-packages/cfgm_common/vnc_amqp.py", line 84, in _vnc_subscribe_callback self.vnc_subscribe_actions() File "/root/work/build/debug/config/device-manager/.tox/py36/lib/python3.6/site-packages/cfgm_common/vnc_amqp.py", line 148, in vnc_subscribe_actions self.evaluate_dependency() File "/root/work/build/debug/config/device-manager/device_manager/dm_amqp.py", line 52, in evaluate_dependency pr.set_config_state() File "/root/work/build/debug/config/device-manager/device_manager/db.py", line 1046, in set_config_state self._logger.warn(f"queue is full") AttributeError: 'DeviceManagerLogger' object has no attribute 'warn' 04/27/2025 18:11:56.151 7f31993ddf60 [contrail-device-manager] [WARNING]: Inconsistency discovered. wanted=1, received=0, missing={'8c5ee1dd-8cef-409a-b4dd-8e9371596b9c'}. This may indicate that the cluster needs a 'nodetool repair'. 04/27/2025 18:11:56.151 7f31993ddf60 [contrail-device-manager] [WARNING]: logical_router uuid 8c5ee1dd-8cef-409a-b4dd-8e9371596b9c update caused NoIdError 04/27/2025 18:11:56.210 7f31993ddf60 [contrail-device-manager] [WARNING]: Inconsistency discovered. wanted=1, received=0, missing={'e24815d6-0232-453e-b724-1225b94c2c1b'}. This may indicate that the cluster needs a 'nodetool repair'. 04/27/2025 18:11:56.210 7f31993ddf60 [contrail-device-manager] [WARNING]: logical_router uuid e24815d6-0232-453e-b724-1225b94c2c1b update caused NoIdError 04/27/2025 18:11:58.560 7f31993ddf60 [contrail-device-manager] [INFO]: Found feature plugin for pr=routertest.test_dm_ansible_dhcp_relay.TestAnsibleDhcpRelayDM.test_dhcp_relay_config_push_two_vns_in_network(698edd22-331e-4a9c-97a9-815d3d5c49ab), feature=l3-gateway 04/27/2025 18:11:58.561 7f31993ddf60 [contrail-device-manager] [INFO]: Found feature plugin for pr=routertest.test_dm_ansible_dhcp_relay.TestAnsibleDhcpRelayDM.test_dhcp_relay_config_push_two_vns_in_network(698edd22-331e-4a9c-97a9-815d3d5c49ab), feature=vn-interconnect 04/27/2025 18:11:58.561 7f31993ddf60 [contrail-device-manager] [INFO]: Found ansible plugin pr=698edd22-331e-4a9c-97a9-815d3d5c49ab, role/vendor/product=spine:juniper:qfx10008 04/27/2025 18:11:58.603 7f31993ddf60 [contrail-device-manager] [INFO]: Found feature plugin for pr=routertest.test_dm_ansible_dhcp_relay.TestAnsibleDhcpRelayDM.test_dhcp_relay_config_push_two_vns_in_network(698edd22-331e-4a9c-97a9-815d3d5c49ab), feature=l3-gateway 04/27/2025 18:11:58.603 7f31993ddf60 [contrail-device-manager] [INFO]: Found feature plugin for pr=routertest.test_dm_ansible_dhcp_relay.TestAnsibleDhcpRelayDM.test_dhcp_relay_config_push_two_vns_in_network(698edd22-331e-4a9c-97a9-815d3d5c49ab), feature=vn-interconnect 04/27/2025 18:11:58.603 7f31993ddf60 [contrail-device-manager] [INFO]: Found ansible plugin pr=698edd22-331e-4a9c-97a9-815d3d5c49ab, role/vendor/product=spine:juniper:qfx10008 04/27/2025 18:11:59.170 7f31993ddf60 [contrail-device-manager] [ERROR]: AttributeError Python 3.6.8: /root/work/build/debug/config/device-manager/.tox/py36/bin/python Sun Apr 27 18:11:59 2025 A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred. /root/work/build/debug/config/device-manager/.tox/py36/lib/python3.6/site-packages/cfgm_common/vnc_amqp.py in _vnc_subscribe_callback(self=<device_manager.dm_amqp.DMAmqpHandle object>, oper_info={'fq_name': ['default-domain', 'default-project', '__contrail_lr_internal_vn_89895d23-c213-4838-ab11-8c7d66ddaaf7__', '__contrail_lr_internal_vn_89895d23-c213-4838-ab11-8c7d66ddaaf7__'], 'obj_dict': {'display_name': '__contrail_lr_internal_vn_89895d23-c213-4838-ab11-8c7d66ddaaf7__', 'fq_name': ['default-domain', 'default-project', '__contrail_lr_internal_vn_89895d23-c213-4838-ab11-8c7d66ddaaf7__', '__contrail_lr_internal_vn_89895d23-c213-4838-ab11-8c7d66ddaaf7__'], 'id_perms': {'created': '2025-04-28T01:11:59.143647', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-04-28T01:11:59.143647', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'contrail-api', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10088682537671317647, 'uuid_mslong': 5232419504340289757}}, 'parent_type': 'virtual-network', 'parent_uuid': 'd8719a71-86be-455e-958a-3c871591d94b', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'routing_instance_fabric_snat': False, 'routing_instance_has_pnf': False, 'routing_instance_is_default': True, 'uuid': '489d49d1-1007-48dd-8c02-3350dd95d88f'}, 'oper': 'CREATE', 'request-id': 'req-0fa906e8-19a0-498c-9953-b8bb6622dd27', 'type': 'routing_instance', 'uuid': '489d49d1-1007-48dd-8c02-3350dd95d88f'}) 82 try: 83 self.oper_info = oper_info 84 self.vnc_subscribe_actions() 85 86 except ConnectionError: self = <device_manager.dm_amqp.DMAmqpHandle object> self.vnc_subscribe_actions = <bound method VncAmqpHandle.vnc_subscribe_actions of <device_manager.dm_amqp.DMAmqpHandle object>> /root/work/build/debug/config/device-manager/.tox/py36/lib/python3.6/site-packages/cfgm_common/vnc_amqp.py in vnc_subscribe_actions(self=<device_manager.dm_amqp.DMAmqpHandle object>) 146 (self. obj_type, obj_id, oper)) 147 return 148 self.evaluate_dependency() 149 150 def _get_key_from_oper_info(self): self = <device_manager.dm_amqp.DMAmqpHandle object> self.evaluate_dependency = <bound method DMAmqpHandle.evaluate_dependency of <device_manager.dm_amqp.DMAmqpHandle object>> /root/work/build/debug/config/device-manager/device_manager/dm_amqp.py in evaluate_dependency(self=<device_manager.dm_amqp.DMAmqpHandle object>) 49 []): 50 pr = PhysicalRouterDM.get(pr_id) 51 if pr is not None: 52 pr.set_config_state() 53 pr.uve_send() pr = <device_manager.db.PhysicalRouterDM object> pr.set_config_state = <bound method PhysicalRouterDM.set_config_state of <device_manager.db.PhysicalRouterDM object>> /root/work/build/debug/config/device-manager/device_manager/db.py in set_config_state(self=<device_manager.db.PhysicalRouterDM object>, reset_retry_count=True) 1044 self.nc_q.put_nowait(1) 1045 except queue.Full: 1046 self._logger.warn(f"queue is full") 1047 1048 # end self = <device_manager.db.PhysicalRouterDM object> self._logger = <device_manager.logger.DeviceManagerLogger object> self._logger.warn undefined AttributeError: 'DeviceManagerLogger' object has no attribute 'warn' __cause__ = None __class__ = <class 'AttributeError'> __context__ = Full() __delattr__ = <method-wrapper '__delattr__' of AttributeError object> __dict__ = {} __dir__ = <built-in method __dir__ of AttributeError object> __doc__ = 'Attribute not found.' __eq__ = <method-wrapper '__eq__' of AttributeError object> __format__ = <built-in method __format__ of AttributeError object> __ge__ = <method-wrapper '__ge__' of AttributeError object> __getattribute__ = <method-wrapper '__getattribute__' of AttributeError object> __gt__ = <method-wrapper '__gt__' of AttributeError object> __hash__ = <method-wrapper '__hash__' of AttributeError object> __init__ = <method-wrapper '__init__' of AttributeError object> __init_subclass__ = <built-in method __init_subclass__ of type object> __le__ = <method-wrapper '__le__' of AttributeError object> __lt__ = <method-wrapper '__lt__' of AttributeError object> __ne__ = <method-wrapper '__ne__' of AttributeError object> __new__ = <built-in method __new__ of type object> __reduce__ = <built-in method __reduce__ of AttributeError object> __reduce_ex__ = <built-in method __reduce_ex__ of AttributeError object> __repr__ = <method-wrapper '__repr__' of AttributeError object> __setattr__ = <method-wrapper '__setattr__' of AttributeError object> __setstate__ = <built-in method __setstate__ of AttributeError object> __sizeof__ = <built-in method __sizeof__ of AttributeError object> __str__ = <method-wrapper '__str__' of AttributeError object> __subclasshook__ = <built-in method __subclasshook__ of type object> __suppress_context__ = False __traceback__ = <traceback object> args = ("'DeviceManagerLogger' object has no attribute 'warn'",) with_traceback = <built-in method with_traceback of AttributeError object> The above is a description of an error in a Python program. Here is the original traceback: Traceback (most recent call last): File "/root/work/build/debug/config/device-manager/device_manager/db.py", line 1044, in set_config_state self.nc_q.put_nowait(1) File "src/gevent/queue.py", line 290, in gevent._gevent_cqueue.Queue.put_nowait File "src/gevent/queue.py", line 296, in gevent._gevent_cqueue.Queue.put_nowait File "src/gevent/queue.py", line 288, in gevent._gevent_cqueue.Queue.put queue.Full During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/root/work/build/debug/config/device-manager/.tox/py36/lib/python3.6/site-packages/cfgm_common/vnc_amqp.py", line 84, in _vnc_subscribe_callback self.vnc_subscribe_actions() File "/root/work/build/debug/config/device-manager/.tox/py36/lib/python3.6/site-packages/cfgm_common/vnc_amqp.py", line 148, in vnc_subscribe_actions self.evaluate_dependency() File "/root/work/build/debug/config/device-manager/device_manager/dm_amqp.py", line 52, in evaluate_dependency pr.set_config_state() File "/root/work/build/debug/config/device-manager/device_manager/db.py", line 1046, in set_config_state self._logger.warn(f"queue is full") AttributeError: 'DeviceManagerLogger' object has no attribute 'warn' 04/27/2025 18:11:59.177 7f31993ddf60 [contrail-device-manager] [ERROR]: AttributeError Python 3.6.8: /root/work/build/debug/config/device-manager/.tox/py36/bin/python Sun Apr 27 18:11:59 2025 A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred. /root/work/build/debug/config/device-manager/.tox/py36/lib/python3.6/site-packages/cfgm_common/vnc_amqp.py in _vnc_subscribe_callback(self=<device_manager.dm_amqp.DMAmqpHandle object>, oper_info={'fq_name': ['default-domain', 'default-project', 'lr-test.test_dm_ansible_dhcp_relay.TestAnsibleDh...DM.test_dhcp_relay_config_push_two_vns_in_network'], 'oper': 'UPDATE', 'request-id': 'req-0fa906e8-19a0-498c-9953-b8bb6622dd27', 'type': 'logical_router', 'uuid': '89895d23-c213-4838-ab11-8c7d66ddaaf7'}) 82 try: 83 self.oper_info = oper_info 84 self.vnc_subscribe_actions() 85 86 except ConnectionError: self = <device_manager.dm_amqp.DMAmqpHandle object> self.vnc_subscribe_actions = <bound method VncAmqpHandle.vnc_subscribe_actions of <device_manager.dm_amqp.DMAmqpHandle object>> /root/work/build/debug/config/device-manager/.tox/py36/lib/python3.6/site-packages/cfgm_common/vnc_amqp.py in vnc_subscribe_actions(self=<device_manager.dm_amqp.DMAmqpHandle object>) 146 (self. obj_type, obj_id, oper)) 147 return 148 self.evaluate_dependency() 149 150 def _get_key_from_oper_info(self): self = <device_manager.dm_amqp.DMAmqpHandle object> self.evaluate_dependency = <bound method DMAmqpHandle.evaluate_dependency of <device_manager.dm_amqp.DMAmqpHandle object>> /root/work/build/debug/config/device-manager/device_manager/dm_amqp.py in evaluate_dependency(self=<device_manager.dm_amqp.DMAmqpHandle object>) 49 []): 50 pr = PhysicalRouterDM.get(pr_id) 51 if pr is not None: 52 pr.set_config_state() 53 pr.uve_send() pr = <device_manager.db.PhysicalRouterDM object> pr.set_config_state = <bound method PhysicalRouterDM.set_config_state of <device_manager.db.PhysicalRouterDM object>> /root/work/build/debug/config/device-manager/device_manager/db.py in set_config_state(self=<device_manager.db.PhysicalRouterDM object>, reset_retry_count=True) 1044 self.nc_q.put_nowait(1) 1045 except queue.Full: 1046 self._logger.warn(f"queue is full") 1047 1048 # end self = <device_manager.db.PhysicalRouterDM object> self._logger = <device_manager.logger.DeviceManagerLogger object> self._logger.warn undefined AttributeError: 'DeviceManagerLogger' object has no attribute 'warn' __cause__ = None __class__ = <class 'AttributeError'> __context__ = Full() __delattr__ = <method-wrapper '__delattr__' of AttributeError object> __dict__ = {} __dir__ = <built-in method __dir__ of AttributeError object> __doc__ = 'Attribute not found.' __eq__ = <method-wrapper '__eq__' of AttributeError object> __format__ = <built-in method __format__ of AttributeError object> __ge__ = <method-wrapper '__ge__' of AttributeError object> __getattribute__ = <method-wrapper '__getattribute__' of AttributeError object> __gt__ = <method-wrapper '__gt__' of AttributeError object> __hash__ = <method-wrapper '__hash__' of AttributeError object> __init__ = <method-wrapper '__init__' of AttributeError object> __init_subclass__ = <built-in method __init_subclass__ of type object> __le__ = <method-wrapper '__le__' of AttributeError object> __lt__ = <method-wrapper '__lt__' of AttributeError object> __ne__ = <method-wrapper '__ne__' of AttributeError object> __new__ = <built-in method __new__ of type object> __reduce__ = <built-in method __reduce__ of AttributeError object> __reduce_ex__ = <built-in method __reduce_ex__ of AttributeError object> __repr__ = <method-wrapper '__repr__' of AttributeError object> __setattr__ = <method-wrapper '__setattr__' of AttributeError object> __setstate__ = <built-in method __setstate__ of AttributeError object> __sizeof__ = <built-in method __sizeof__ of AttributeError object> __str__ = <method-wrapper '__str__' of AttributeError object> __subclasshook__ = <built-in method __subclasshook__ of type object> __suppress_context__ = False __traceback__ = <traceback object> args = ("'DeviceManagerLogger' object has no attribute 'warn'",) with_traceback = <built-in method with_traceback of AttributeError object> The above is a description of an error in a Python program. Here is the original traceback: Traceback (most recent call last): File "/root/work/build/debug/config/device-manager/device_manager/db.py", line 1044, in set_config_state self.nc_q.put_nowait(1) File "src/gevent/queue.py", line 290, in gevent._gevent_cqueue.Queue.put_nowait File "src/gevent/queue.py", line 296, in gevent._gevent_cqueue.Queue.put_nowait File "src/gevent/queue.py", line 288, in gevent._gevent_cqueue.Queue.put queue.Full During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/root/work/build/debug/config/device-manager/.tox/py36/lib/python3.6/site-packages/cfgm_common/vnc_amqp.py", line 84, in _vnc_subscribe_callback self.vnc_subscribe_actions() File "/root/work/build/debug/config/device-manager/.tox/py36/lib/python3.6/site-packages/cfgm_common/vnc_amqp.py", line 148, in vnc_subscribe_actions self.evaluate_dependency() File "/root/work/build/debug/config/device-manager/device_manager/dm_amqp.py", line 52, in evaluate_dependency pr.set_config_state() File "/root/work/build/debug/config/device-manager/device_manager/db.py", line 1046, in set_config_state self._logger.warn(f"queue is full") AttributeError: 'DeviceManagerLogger' object has no attribute 'warn' 04/27/2025 18:11:59.188 7f31993ddf60 [contrail-device-manager] [ERROR]: AttributeError Python 3.6.8: /root/work/build/debug/config/device-manager/.tox/py36/bin/python Sun Apr 27 18:11:59 2025 A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred. /root/work/build/debug/config/device-manager/.tox/py36/lib/python3.6/site-packages/cfgm_common/vnc_amqp.py in _vnc_subscribe_callback(self=<device_manager.dm_amqp.DMAmqpHandle object>, oper_info={'fq_name': ['default-domain', 'default-project', 'lr-test.test_dm_ansible_dhcp_relay.TestAnsibleDh...DM.test_dhcp_relay_config_push_two_vns_in_network'], 'oper': 'UPDATE', 'request-id': 'req-65e2ebe9-fcff-480f-ab31-e5a09bf9cb4a', 'type': 'logical_router', 'uuid': '89895d23-c213-4838-ab11-8c7d66ddaaf7'}) 82 try: 83 self.oper_info = oper_info 84 self.vnc_subscribe_actions() 85 86 except ConnectionError: self = <device_manager.dm_amqp.DMAmqpHandle object> self.vnc_subscribe_actions = <bound method VncAmqpHandle.vnc_subscribe_actions of <device_manager.dm_amqp.DMAmqpHandle object>> /root/work/build/debug/config/device-manager/.tox/py36/lib/python3.6/site-packages/cfgm_common/vnc_amqp.py in vnc_subscribe_actions(self=<device_manager.dm_amqp.DMAmqpHandle object>) 146 (self. obj_type, obj_id, oper)) 147 return 148 self.evaluate_dependency() 149 150 def _get_key_from_oper_info(self): self = <device_manager.dm_amqp.DMAmqpHandle object> self.evaluate_dependency = <bound method DMAmqpHandle.evaluate_dependency of <device_manager.dm_amqp.DMAmqpHandle object>> /root/work/build/debug/config/device-manager/device_manager/dm_amqp.py in evaluate_dependency(self=<device_manager.dm_amqp.DMAmqpHandle object>) 49 []): 50 pr = PhysicalRouterDM.get(pr_id) 51 if pr is not None: 52 pr.set_config_state() 53 pr.uve_send() pr = <device_manager.db.PhysicalRouterDM object> pr.set_config_state = <bound method PhysicalRouterDM.set_config_state of <device_manager.db.PhysicalRouterDM object>> /root/work/build/debug/config/device-manager/device_manager/db.py in set_config_state(self=<device_manager.db.PhysicalRouterDM object>, reset_retry_count=True) 1044 self.nc_q.put_nowait(1) 1045 except queue.Full: 1046 self._logger.warn(f"queue is full") 1047 1048 # end self = <device_manager.db.PhysicalRouterDM object> self._logger = <device_manager.logger.DeviceManagerLogger object> self._logger.warn undefined AttributeError: 'DeviceManagerLogger' object has no attribute 'warn' __cause__ = None __class__ = <class 'AttributeError'> __context__ = Full() __delattr__ = <method-wrapper '__delattr__' of AttributeError object> __dict__ = {} __dir__ = <built-in method __dir__ of AttributeError object> __doc__ = 'Attribute not found.' __eq__ = <method-wrapper '__eq__' of AttributeError object> __format__ = <built-in method __format__ of AttributeError object> __ge__ = <method-wrapper '__ge__' of AttributeError object> __getattribute__ = <method-wrapper '__getattribute__' of AttributeError object> __gt__ = <method-wrapper '__gt__' of AttributeError object> __hash__ = <method-wrapper '__hash__' of AttributeError object> __init__ = <method-wrapper '__init__' of AttributeError object> __init_subclass__ = <built-in method __init_subclass__ of type object> __le__ = <method-wrapper '__le__' of AttributeError object> __lt__ = <method-wrapper '__lt__' of AttributeError object> __ne__ = <method-wrapper '__ne__' of AttributeError object> __new__ = <built-in method __new__ of type object> __reduce__ = <built-in method __reduce__ of AttributeError object> __reduce_ex__ = <built-in method __reduce_ex__ of AttributeError object> __repr__ = <method-wrapper '__repr__' of AttributeError object> __setattr__ = <method-wrapper '__setattr__' of AttributeError object> __setstate__ = <built-in method __setstate__ of AttributeError object> __sizeof__ = <built-in method __sizeof__ of AttributeError object> __str__ = <method-wrapper '__str__' of AttributeError object> __subclasshook__ = <built-in method __subclasshook__ of type object> __suppress_context__ = False __traceback__ = <traceback object> args = ("'DeviceManagerLogger' object has no attribute 'warn'",) with_traceback = <built-in method with_traceback of AttributeError object> The above is a description of an error in a Python program. Here is the original traceback: Traceback (most recent call last): File "/root/work/build/debug/config/device-manager/device_manager/db.py", line 1044, in set_config_state self.nc_q.put_nowait(1) File "src/gevent/queue.py", line 290, in gevent._gevent_cqueue.Queue.put_nowait File "src/gevent/queue.py", line 296, in gevent._gevent_cqueue.Queue.put_nowait File "src/gevent/queue.py", line 288, in gevent._gevent_cqueue.Queue.put queue.Full During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/root/work/build/debug/config/device-manager/.tox/py36/lib/python3.6/site-packages/cfgm_common/vnc_amqp.py", line 84, in _vnc_subscribe_callback self.vnc_subscribe_actions() File "/root/work/build/debug/config/device-manager/.tox/py36/lib/python3.6/site-packages/cfgm_common/vnc_amqp.py", line 148, in vnc_subscribe_actions self.evaluate_dependency() File "/root/work/build/debug/config/device-manager/device_manager/dm_amqp.py", line 52, in evaluate_dependency pr.set_config_state() File "/root/work/build/debug/config/device-manager/device_manager/db.py", line 1046, in set_config_state self._logger.warn(f"queue is full") AttributeError: 'DeviceManagerLogger' object has no attribute 'warn' 04/27/2025 18:12:00.208 7f31993ddf60 [contrail-device-manager] [WARNING]: Inconsistency discovered. wanted=1, received=0, missing={'89895d23-c213-4838-ab11-8c7d66ddaaf7'}. This may indicate that the cluster needs a 'nodetool repair'. 04/27/2025 18:12:00.208 7f31993ddf60 [contrail-device-manager] [WARNING]: logical_router uuid 89895d23-c213-4838-ab11-8c7d66ddaaf7 update caused NoIdError 04/27/2025 18:12:04.688 7f3198cf5240 [contrail-device-manager] [INFO]: SANDESH: CONNECT TO COLLECTOR: False 04/27/2025 18:12:04.698 7f3198cf5240 [contrail-device-manager] [INFO]: SANDESH: Logging: LEVEL: [SYS_INFO] -> [SYS_INFO] 04/27/2025 18:12:04.699 7f3198cf5240 [contrail-device-manager] [INFO]: SANDESH: Logging: FILE: [None] -> [device_manager_TestNetworkDM.log]