Category: Emergency Work

Broadband – 27/10/2022 – IPv6

We are aware of an IPv6 issue on the network following on from a firmware upgrade within our core last night.

We have been working with the hardware vendor to resolve the issue but while this is ongoing , we have been reverting back to a previous version of firmware and moving connections between gateways.

Some users will experience a graceful PPP drop of around 5 seconds while there connection re-authenticates.

We do apologise for any inconvenicance

UPDATE01

LNS02, LNS03, LNS04 have been reverted and IPv6 connectivity has been restored.

LNS01 has been isolated for testing. Anyone experiencing slow DNS lookups or applications now loading are advised to reboot there router which will be routed to one of the other gateways with the fix applied.

UPDATE02

Further issues where identified with IPv6 within our core network (Broadband facing) Work to resolve this has now completed and IPv6 should now be fully operational again across all gateways.

Broadband – Emergency Maintenance (31/08/2021 – 22:00)

We have been advised by one of our wholesale providers that they will be undertaking emergency maintiance on the following exchanges that may result in an outage of up to 30 minutes for DSL circuits.

If you have still have no service after the work, Please power down your hardware for 20 minutes and power back on.

We apologise for any inconvenience caused.

Exchange areas affected (including ADSL2+, FTTC, SoGEA, FTTP, G.Fast)

Upper Holloway

Stratford On Avon

Great Yarmouth Town

Norwich City

Arkwright

Leicester Central

Nottingham Longbow

Newark

Rushden

Cupar Crossgates

Penrith

Todmorden

Wigan

Poplar

Deptford

Ewell

Greenwich

Orpington

Redhill

Skelmersdale

Ealing

Golders Green

Hayes North

Hendon

Twickenham

Didsbury

Stockport

Gravesend

Crawley

Hove

Letchworth

Bedminster

Downend

Frome Keyford

Gloucester

North

South

West

Bridgend

Merthyr Tydfil

Whitchurch

Godalming

Haslemere

Henley-on-thames

Newbury

Reading South

Lords

Maida Vale

Bromsgrove

St Peters

Chester/south

Ross On Wye

South Kensington

Western

Barnstaple

Exeter

Falmouth

Newquay

Redruth

St Austell

Taunton

Truro

Emergency Works – Broadband – 12/05/2021 – 22:00

We are observing odd behaviour on LNS04. The device is operational, however we are observing latency on some network operations. We have spoken to our vendor who has advised we apply a software patch. This will result in the device reloading. Circuits connected via this device will experience a PPP drop but will re-route to other gateways.

UPDATE01 – 13:15

We have taken this gateway out of service for new / reconnections connections. If you are having issues then power cycling your router will result in a connection to another non affected gateway.

UPDATE02 – FINAL

The gateway is now back in operation.

Emergency Works 14/04/2021 – 02:00 (Broadband)

We have been made aware by our suppler that they will be making changes to our 10GB interconnects at Telehouse North and East at 02:00 on the 14/04/2021 to resolve the low level packet loss being seen on some broadband services and non critical traffic types. This work should not be service affecting however one of the access switches we are connected to may reload as a result due to an unrelated issue there side. Should this occur, services will fail over to redundant paths but users will experience a PPP reconnect.

An update will be posted tomorrow morning to confirm closure.

Emergency Works 30/03/2021

An engineer is on route to Telehouse to complete a reboot of our 2 London core devices. Works are due to start from 07:30am today. This will be done in succession pending a successful reboot of the first device and to allow services where possible to re-route across the network. Leased line circuits without backup will see disruption when the associated core reloads.

We do apologise for the short notice.

UPDATE01 – 07:35

This work is about to start

UPDATE02 – 08:30

This work is complete.

14/09/2020 22:00 – Broadband Network

Due to a unexpected reload of a broadband gateway on our network this afternoon, we are seeing a traffic imbalance on part of our broadband network.

We will be taking steps to disconnect sessions gratefully and re balance the affected gateways.

End users will see a PPP reconnect taking approximately 5-20 seconds. In the rare event your connection does not restore. You will need to power OFF and ON your router.

UPDATE01 – 22:10

This work is now complete.

16/04/2020 23:00 – Broadband Maintenance

We will be making some changes to our broadband network tonight in order to isolate 2 upstream gateways we suspect of causing additional latency to circuits routed via them.

This will cause existing connection via these gateways to drop and reconnect. Due to the nature of the change this can take up-to 20 minutes.

UPDATE 01 – 23:03

This work is about to start.

UPDATE 02 – 23:06

Tunnels have been terminated and traffic is starting to move across to other gateways.

UPDATE 03 – 23:28

We have seen an issue with the L2TP control messages not being accepted by the upstream gateways and releasing circuits to other gateways. We have therefore had to revert part of the configuration. Further works will be required at a later date.

Emergency Work – 23/11/2018 – 21:30 – LNS01 Broadband Gateway

Our network monitoring has flagged high CPU usage on LNS01 that is starting to affect its operation. We are undertaking an emergency reboot to prevent it crashing completely. This will drop active sessions and force them on to other gateways. UPDATE01 – 21:35 The reboot is complete. Services transferred to redundant gateways as expected. LNS01 … Continue reading “Emergency Work – 23/11/2018 – 21:30 – LNS01 Broadband Gateway”

Our network monitoring has flagged high CPU usage on LNS01 that is starting to affect its operation. We are undertaking an emergency reboot to prevent it crashing completely. This will drop active sessions and force them on to other gateways.

UPDATE01 – 21:35
The reboot is complete. Services transferred to redundant gateways as expected. LNS01 is now back in operation. If you do not have service please power down your router for 20 minutes.

We apologise for any inconvenience caused.

LON01 – VoIP – 21/11/2017

We are aware 1 of our media gateways is not releasing channels once a call has cleared down. This is causing BUSY tones or limit exceeded messages We are currently working to resolve this ASAP. UPDATE 01 – 19:00 Emergency Works have started. Any active calls on SIP02 have been dropped. We are sorry for … Continue reading “LON01 – VoIP – 21/11/2017”

We are aware 1 of our media gateways is not releasing channels once a call has cleared down. This is causing BUSY tones or limit exceeded messages

We are currently working to resolve this ASAP.

UPDATE 01 – 19:00
Emergency Works have started. Any active calls on SIP02 have been dropped. We are sorry for any inconvenience caused.

UPDATE 02 – 19:03
SIP02 has reloaded and all services have restored. We will now look at SIP01

UPDATE 03 – 19:05
Emergency Works have started. Any active calls on SIP01 have been dropped. We are sorry for any inconvenience caused.

UPDATE 04 – 19:03
SIP01 has reloaded and all services have restored.

LON01 – EasyXDSL – 21/02/2017 – 20:30 *Firmware Upgrade*

At the request of our vendor to address a bug which has been resulting in delayed RADIUS ACCOUNTING data we will be upgrading to a firmware that should address this problem. Each firmware upgrade will take no longer than 10 seconds, however a reload of each LNS is required. They will be done 1 at … Continue reading “LON01 – EasyXDSL – 21/02/2017 – 20:30 *Firmware Upgrade*”

At the request of our vendor to address a bug which has been resulting in delayed RADIUS ACCOUNTING data we will be upgrading to a firmware that should address this problem. Each firmware upgrade will take no longer than 10 seconds, however a reload of each LNS is required. They will be done 1 at a time and will result in a PPP drop for each DSL circuit. This should automatically re-establish within 60 seconds.

UPDATE01
This work is complete. Any user without a working internet connection are advised to power off there hardware for 20 minutes.