Emergency Works – Broadband – 12/05/2021 – 22:00

We are observing odd behaviour on LNS04. The device is operational, however we are observing latency on some network operations. We have spoken to our vendor who has advised we apply a software patch. This will result in the device reloading. Circuits connected via this device will experience a PPP drop but will re-route to other gateways.

UPDATE01 – 13:15

We have taken this gateway out of service for new / reconnections connections. If you are having issues then power cycling your router will result in a connection to another non affected gateway.

UPDATE02 – FINAL

The gateway is now back in operation.

Emergency Works 14/04/2021 – 02:00 (Broadband)

We have been made aware by our suppler that they will be making changes to our 10GB interconnects at Telehouse North and East at 02:00 on the 14/04/2021 to resolve the low level packet loss being seen on some broadband services and non critical traffic types. This work should not be service affecting however one of the access switches we are connected to may reload as a result due to an unrelated issue there side. Should this occur, services will fail over to redundant paths but users will experience a PPP reconnect.

An update will be posted tomorrow morning to confirm closure.

Emergency Works 30/03/2021

An engineer is on route to Telehouse to complete a reboot of our 2 London core devices. Works are due to start from 07:30am today. This will be done in succession pending a successful reboot of the first device and to allow services where possible to re-route across the network. Leased line circuits without backup will see disruption when the associated core reloads.

We do apologise for the short notice.

UPDATE01 – 07:35

This work is about to start

UPDATE02 – 08:30

This work is complete.

14/09/2020 22:00 – Broadband Network

Due to a unexpected reload of a broadband gateway on our network this afternoon, we are seeing a traffic imbalance on part of our broadband network.

We will be taking steps to disconnect sessions gratefully and re balance the affected gateways.

End users will see a PPP reconnect taking approximately 5-20 seconds. In the rare event your connection does not restore. You will need to power OFF and ON your router.

UPDATE01 – 22:10

This work is now complete.

16/04/2020 23:00 – Broadband Maintenance

We will be making some changes to our broadband network tonight in order to isolate 2 upstream gateways we suspect of causing additional latency to circuits routed via them.

This will cause existing connection via these gateways to drop and reconnect. Due to the nature of the change this can take up-to 20 minutes.

UPDATE 01 – 23:03

This work is about to start.

UPDATE 02 – 23:06

Tunnels have been terminated and traffic is starting to move across to other gateways.

UPDATE 03 – 23:28

We have seen an issue with the L2TP control messages not being accepted by the upstream gateways and releasing circuits to other gateways. We have therefore had to revert part of the configuration. Further works will be required at a later date.

Emergency Work – 23/11/2018 – 21:30 – LNS01 Broadband Gateway

Our network monitoring has flagged high CPU usage on LNS01 that is starting to affect its operation. We are undertaking an emergency reboot to prevent it crashing completely. This will drop active sessions and force them on to other gateways. UPDATE01 – 21:35 The reboot is complete. Services transferred to redundant gateways as expected. LNS01 … Continue reading “Emergency Work – 23/11/2018 – 21:30 – LNS01 Broadband Gateway”

Our network monitoring has flagged high CPU usage on LNS01 that is starting to affect its operation. We are undertaking an emergency reboot to prevent it crashing completely. This will drop active sessions and force them on to other gateways.

UPDATE01 – 21:35
The reboot is complete. Services transferred to redundant gateways as expected. LNS01 is now back in operation. If you do not have service please power down your router for 20 minutes.

We apologise for any inconvenience caused.

LON01 – VoIP – 21/11/2017

We are aware 1 of our media gateways is not releasing channels once a call has cleared down. This is causing BUSY tones or limit exceeded messages We are currently working to resolve this ASAP. UPDATE 01 – 19:00 Emergency Works have started. Any active calls on SIP02 have been dropped. We are sorry for … Continue reading “LON01 – VoIP – 21/11/2017”

We are aware 1 of our media gateways is not releasing channels once a call has cleared down. This is causing BUSY tones or limit exceeded messages

We are currently working to resolve this ASAP.

UPDATE 01 – 19:00
Emergency Works have started. Any active calls on SIP02 have been dropped. We are sorry for any inconvenience caused.

UPDATE 02 – 19:03
SIP02 has reloaded and all services have restored. We will now look at SIP01

UPDATE 03 – 19:05
Emergency Works have started. Any active calls on SIP01 have been dropped. We are sorry for any inconvenience caused.

UPDATE 04 – 19:03
SIP01 has reloaded and all services have restored.

LON01 – EasyXDSL – 21/02/2017 – 20:30 *Firmware Upgrade*

At the request of our vendor to address a bug which has been resulting in delayed RADIUS ACCOUNTING data we will be upgrading to a firmware that should address this problem. Each firmware upgrade will take no longer than 10 seconds, however a reload of each LNS is required. They will be done 1 at … Continue reading “LON01 – EasyXDSL – 21/02/2017 – 20:30 *Firmware Upgrade*”

At the request of our vendor to address a bug which has been resulting in delayed RADIUS ACCOUNTING data we will be upgrading to a firmware that should address this problem. Each firmware upgrade will take no longer than 10 seconds, however a reload of each LNS is required. They will be done 1 at a time and will result in a PPP drop for each DSL circuit. This should automatically re-establish within 60 seconds.

UPDATE01
This work is complete. Any user without a working internet connection are advised to power off there hardware for 20 minutes.

LON01 – EasyXDSL Fibre Hostlink– 06/09/2016 – 23:00

We have been advised by one of our fibre wave providers that they will be conducting some emergency maintenance on some fibre interconnects which includes a circuit used by ourselves. To avoid any unexpected issues we will manually re-route traffic and flag that link as unavailable. Some PPP sessions may drop and re-establish for DSL … Continue reading “LON01 – EasyXDSL Fibre Hostlink– 06/09/2016 – 23:00”

We have been advised by one of our fibre wave providers that they will be conducting some emergency maintenance on some fibre interconnects which includes a circuit used by ourselves.

To avoid any unexpected issues we will manually re-route traffic and flag that link as unavailable. Some PPP sessions may drop and re-establish for DSL circuits using this link.

UPDATE 01 – 22:23

Traffic has been re-routed. We where able to maintain reachability to our L2TP endpoints, as such no PPP session drops where seen. The link will be bought back in to service once we have received a clear from that wave provider.

LON01 – CORE03 – 24/08/2016 – 20:00 Till 23:59 *Emergency Work *

Following on from recent repeated hardware failures on core03.structuredcommunications.co.uk as detailed HERE the decision has been taken to fully replace the device. We will also be taking the opportunity to upgrade the IOS image to bring this device in line with the current images across the rest of our network. This work will involve powering … Continue reading “LON01 – CORE03 – 24/08/2016 – 20:00 Till 23:59 *Emergency Work *”

Following on from recent repeated hardware failures on core03.structuredcommunications.co.uk as detailed HERE the decision has been taken to fully replace the device. We will also be taking the opportunity to upgrade the IOS image to bring this device in line with the current images across the rest of our network.

This work will involve powering down and physically moving the current device along with all installed line cards. Due to this all directly connected services (listed below) will be unavailable for the duration of the works.

> Bonded DSL on AG1 & AG2

> un-managed SIP trunking provided via sipwise.easyipt.co.uk

> Managed VoIP services provided via primary-sw.r03.core03 & primary-sw.r04.core03

> Webhosting via server01.easyhttp.co.uk

> VPS sessions on esxi10.r02.structuredcommunications.co.uk

Other services will remain unaffected. Redundant services provided via other parts of the network (Such as DNS & SMTP) will take over. Please ensure you configuration is up to date.

UPDATE01 – 20:10 – 24/08/2016 Engineers are on site and these works have started.

UPDATE02 – 22:06 – 24/08/2016 Engineers have completed the above works ahead of schedule and we can confirm all services have returned to normal. We apologize for the inconvenience caused.

We will continue to monitor the new device to ensure continued operation.