Category: Incidents

Our network monitoring has alerted us to an issue at the London LINX exchange. This will be having an impact on any customer who uses a 3rd party connection to connect to our services.

UPDATE 01 – 11:32
Service has been restored and we are seeing traffic route via LINX again. As we have no direct relationship or take services from LINX, there wont be a RFO.

We are aware of a problem with an upstream provider that is preventing the termination of OFFNET (Mobile, Local, National and international) calls. We are currently working to resolve the fault and will provide updates shortly.

UPDATE01 – 11:36
Local and national calls should now be terminating correctly, however mobile calls are still resulting in a BUSY response from our carrier.

UPDATE02 – 12:00
Mobile and international calls are now terminating correctly.

We have logged a case with the carrier for details and will provide this once made available.

We apologise for any inconvenience caused

UPDATE03 – 12:24
We have been advised the root cause of the failure was “a major outage on the BT IP exchange interconnect which caused a high proportion of calls to fail.”

In light of this outage, Structured Communications will be implementing carrier diverse outbound routing. This will require large network changes and further details to follow in the coming weeks.

During an internal training exercise a “reload” command was issued to CORE01-LON instead of CORE01-HOR which resulted in a reboot of both the primary and secondary supervisor modules. This reload caused a 5 minute loss of service to LON01 for all directly connected services and customers.

Safe guards will be put in place on LON01 to prevent this happening again in such a scenario.

We apologise for any inconvenience caused.

We are currently aware of fibre break in the London area which is affecting outbound calls for some of our users. We have been informed by suppliers that this is due do fibre damage to which is currently being repaired. We are expecting an update by midday.

Calls from customers outside of London to customers in London may fail. Broadband speeds should NOT be affected. This incident will be impacting BT customers and other operators.

In the mean time if you are experiencing issues please prefix the dialled number with 1280 for example 128002033014000. If you use a telephone system and normally prefix calls with 9 an example would be 9128002033014000.

We apologise for any inconvenience caused.

We are currently experiencing an issue that is preventing some DSL users connecting to our service. Our initial investigations indicate that this is a BT issue and we are working with them to resolve as soon as possible. The issue spans multiple nodes and is not geographically restrained to one area. We are aiming to have the service restored as a matter of urgency.

-Update- 11:40
Customer sessions are now being restored and our monitoring indicates that the majority of services are now working again. We’ve yet to receive details from BT as to the cause but will update when we have further information.

We are aware of a wholesale issue currently affecting our xDSL customer base, manifesting itself as poor performance.

Our Systems team are currently investigating. Further information will be provided as and when it becomes available.

UPDATE01 – 17:38 – 17/01/2014

We have been advised that the cause of the issue has been a “flapping” port between BT wholesale and our wholesale provider causing a loss in bandwidth. This has now been resolved however will have caused stale PPP sessions to occur. If you don’t have service can you please reboot all your hardware.

We apologise for any inconvenience caused.

UPDATE02 – 12:09 – 21/01/2014

An incident report has been made available for this issue. Major Incident Report – Enta Outage 17-01-2014

We are aware of a network wide issue affecting DSL sync and PPP sessions on some 02 wholesale LLU ADSL2+ circuits.

Engineers are working to locate the fault.

We will provide further updates once they are provided to us.

We apologise for any inconvenience.

We have been advised of a fibre failure within the 02 Wholesale network. A case has been logged with our wholesale provider.

We apologise for any inconvenience caused

At 20:17 we were made aware by our networking monitoring software that both our /21 ranges where being announced by our previous transit provider (AS24867) and not ourselves (AS199283)

We are current working to get this resolved.

UPDATE01 – 21:22

We have discovered the announcement was detected in the Czech Republic learnt by 6881 15685 6939 24867. Engineers are still working on the root cause of the rouge announcement, however this incident is now closed. A report will be made available as and when a full investigation has taken place.

We are aware of a network wide issue affecting DSL sync and PPP sessions on some 02 wholesale LLU ADSL2+ circuits.
Engineers are working to locate the fault.
We will provide further updates once they are provided to us.

UPDATE 01 – 14:23
Engineers are still working within wholesale to find the root cause of the fault. Connectivity has been intermittently restored.

UPDATE 01 – 17:10
We’ve now been advised that the supplier has identified a fibre fault at Telehouse and are now attempting to get it resolved as soon as possible. We’ll continue to update as the updates are available.