Monthly Archives: June 2016

LON01 – Level3 – 30/06/2016 – 15:30

We are aware of a problem with one of our primary transit interconnects with Level3. This is affecting services across the network that transverse over that link.

We are currently looking in to this as a priority.

UPDATE01 – 15:40
After reviewing our network we can confirm this is a problem with our Adapt transit link and not the Level3 side and are taking steps to shutdown this session.

UPDATE02 – 15:45
We have received notice from Adapt that the network issue there side is a result of a high level DDos attack on part of there network.

UPDATE03 – 15:49
Adapt have confirmed the traffic has been mitigated and we have re-enabled the link.

UPDATE04 – 15:50
CORE02 unexpectedly reloaded when this link was re-enabled due to a full BGP table being received and not filtered correctly which would have affected DSL services terminating on this core. DSL services re-routed to CORE01 as expected. We apologise for any inconvenience this may have caused.

UPDATE04 – 15:55
CORE02 has recovered and is accepting connections again.

LON01 – Access Switches – 27/06/2016 – 21:12 *COMPLETE*

We have identified a security bug within the core firmware installed and running on the following access switches within our network:

primary-sw.r02
backup-sw.r02
primary-sw.r03
backup-sw.r03
primary-sw.r04
backup-sw.r04

Due to the nature of this security bug we have had little option but to immediately update these devices. This update required the above switches to be reloaded so the new IOS could be loaded. Services with a redundant network links to other parts of our network would have seen no disruption.

Other switches are unaffected and this work is now complete.

We apologise for any inconvenience caused.

LON01 – EasyXDSL – 21/06/2016 – 15:32

We are aware lns01.dsl.structuredcommunications.co.uk crashed as a result of a kernel panic within the OS. circuits connected to this LNS would have suffered a drop while they where re-routed to a backup LNS within our network.

We are currently engaged with our hardware vendor as to the root cause.

lns01.dsl.structuredcommunications.co.uk has recovered to an operation state and is accepting connections again.

We apologise for any inconvenience caused.

UPDATE 01 – 11:00 – 22/06/2016
We have followed up with of hardware vendor as to the progress of this crash report.