On Tuesday afternoon our redundant core routers and core switches dropped the connection between each other. Our network has a mix of Cisco and Arista routers and switches within. In the last few years we have brought in Arista switches to support our new 10 GB network for our new cloud services. We eventually found out that our Cisco switches and our Arista switches were running two different Spanning Tree Protocol modes. As a result, the switches were flooding each other with different versions of the same network routes. This overflow built up over time, resulting in the network failure. Once this error was found and corrected, the network returned to normal operation.

The correct changes were made to fix the problem, and the network is functioning properly. For six days we have not experienced any further network related issues. We are monitoring the network very closely and believe the immediate major network problem has been solved."this is txt file