Month: May 2015

LON01 – 26-05-2015 – 20:00 – 20:15 – Network Maintenance (BGP Filters) *COMPLETE*

We will be installing 2 new internal outbound BGP route maps on CORE01.LON01 & CORE02.LON01 tonight between 20:00 – 20:15. This work is to further improve BGP security internally between cores. The new rules should not cause any BGP session resets or any loss of existing known routes from upstream peers, however we may reset … Continue reading “LON01 – 26-05-2015 – 20:00 – 20:15 – Network Maintenance (BGP Filters) *COMPLETE*”

We will be installing 2 new internal outbound BGP route maps on CORE01.LON01 & CORE02.LON01 tonight between 20:00 – 20:15.

This work is to further improve BGP security internally between cores.

The new rules should not cause any BGP session resets or any loss of existing known routes from upstream peers, however we may reset the sessions to ensure they are working correctly. This will cause traffic to re-route for a few moments before returning to normal.

UPDATE 01 – 20:20
This window has been extended until 21:00 due to a late start by engineers.

UPDATE 02 – 20:27
CORE01.LON01 has been updated and the filters tested without the need for a full session reset.

UPDATE 03 – 20:35
CORE02.LON01 has been updated and the filters tested without the need for a full session reset.

Network monitoring shows all routing tables have returned to normal however engineers will monitor for the next few hours to ensure all routing tables remain consistent between Cores.

UPDATE 04 – 21:30
The routing tables on both CORE01.LON01 and CORE02.LON01 have remained constant for the past hour and we are happy to fully close this maintenance window.

LON01 – 06-05-2015 – Adapt / Level3 Transit Outage *Resolved*

At 10:01 our network monitoring detected a loss of IP transit to our primary carrier Adapt / Level3. Engineers have shutdown this link and traffic is routing via alternative carriers. We will continue to monitor and update as more information becomes available. We apologize for any inconvenience caused. UPDATE 01 – 11:01 We are continuing … Continue reading “LON01 – 06-05-2015 – Adapt / Level3 Transit Outage *Resolved*”

At 10:01 our network monitoring detected a loss of IP transit to our primary carrier Adapt / Level3. Engineers have shutdown this link and traffic is routing via alternative carriers. We will continue to monitor and update as more information becomes available.

We apologize for any inconvenience caused.

UPDATE 01 – 11:01
We are continuing to try and contact Adapt for an update as there are no issues with the Level3 side of the link. We believe from external testing there is a routing failure within the Adapt network.

Traffic is continuing to route via other links, however due to the volume of traffic being observed latency may be higher than normal. Should this continue then we will look to re-route traffic via our 3rd interconnect on CORE01 that has a higher capacity link.

UPDATE 02 – 11:18
We have been advised that the issue has been resolved, however we are seeking further confirmation before we bring the peer back in to service.

UPDATE 03 – 11:18
We have spoken to the senior network architect at Adapt who has confirmed service has been restored and we have re-enabled the session. We are observing normal traffic flow, however will continue to monitor the link. Once an RFO has been published by Adapt we will provide this.