LON01 – VoIP – 21/11/2017

We are aware 1 of our media gateways is not releasing channels once a call has cleared down. This is causing BUSY tones or limit exceeded messages

We are currently working to resolve this ASAP.

UPDATE 01 – 19:00
Emergency Works have started. Any active calls on SIP02 have been dropped. We are sorry for any inconvenience caused.

UPDATE 02 – 19:03
SIP02 has reloaded and all services have restored. We will now look at SIP01

UPDATE 03 – 19:05
Emergency Works have started. Any active calls on SIP01 have been dropped. We are sorry for any inconvenience caused.

UPDATE 04 – 19:03
SIP01 has reloaded and all services have restored.

LON01 – C002.017 – 06/11/2017 – 11.32 – *At Risk*

Our network monitoring had alerted us to a power issues within one of our Cross-Connect / POP cabinets on the 2nd floor within Goswell Road. Our 4th floor suite is unaffected, however all services routing via C002.017 should be considered at risk until this is resolved. We are current engaged with Level3 and will provide updates shortly.

UPDATE01 – 11.57
Level3 have responded to advise tier 2 Technicians are to investigate the incident and provide feedback on their findings within 1 hour.

UPDATE02 – 12:11
Level3 have responded to advise they have sent a subcase to thier techs in Goswell Road to continue to investigate.

UPDATE03 – 13:20
Level3 have issued the following statement:

A non-service affecting partial power rack failure at the Goswell Gateway in London, England is ongoing. The European NOC has advised that an electrical contractor has been engaged and is onsite working to resolve the power issue. There is no estimate as to when the power issue will be resolved; however, the European NOC has confirmed that this issue is not impacting services. Please be advised that updates for this event will be relayed at a minimum of hourly unless otherwise noted.

We do have a number of servers and core network equipment operating in redundancy mode due to the power loss and remain at risk.

UPDATE04 – 14:25
Level3 have issued the following update:

Event The European NOC has advised that the electrical contractor has replaced the faulty power cabling onsite clearing all of the alarms related to the partial power failure. At this time, efforts to contact the impacted clients to confirm service restoral are now underway.

We are still seeing network alarms in relation to power and have asked Level3 to tech to step in as we suspect one of our circuit breakers may have tripped. We will advise once further updates are given.

UPDATE05 – 18:30
We have chased Level3 for an update as no response has been given to our request. A level 1 escalation has also been raised.

UPDATE06 – 20:30
We have chased Level3 for an update as no response has been given to our request. A level 2 escalation has also been raised.

UPDATE07 – 21:58
Power has been restored and services are no longer “at risk” initial investigations advise it was more than a circuit breaker at fault. A complaint has been raised.