Category: RESOLVED

Notice: Draytek Vulnerability

We have seen an increasing number of attacks targeting Draytek routers, following reports that approximately 700,000+ devices globally are vulnerable due to recently discovered security flaws. You can read more about this issue here: The Hacker News – Draytek Router Vulnerability.

These attacks are causing significant connection instability for affected users. To mitigate potential risks, we strongly advise all users with Draytek devices to ensure their routers are updated with the latest firmware versions as soon as possible.

If you require assistance with updating your device or have any concerns, please contact your IT provider.

UPDATE01 – 06/10/2024

We have seen first hand a number of compromised routers since Friday. These include VPN users being created, both HTTP and HTTPS ports being changed as well as IP objects being created and enabled.

We would highly recommend updating your Draytek router and talking with your IT company. We have a number of providers we can recommend if you don’t have one.

SIP / Voice – OVH – 09:15

We are aware of a network issue affecting OVH and a small number of VPS images we still have hosted with them. This is having an impact on calls for these clients. Engineers are investigating.

UPDATE01 – 09:30

We have attempted to re-route our traffic around the issue, however the issue is within the core OVH network.

UPDATE02- 09:40

We are starting to see traffic to OVH return to normal. We are monitoring and can only apologise for the issues.

Incident: sip03.easyipt.co.uk – 29/07/2024 – 12:20

We are aware SIP03 is suffering from issues following an emergency reboot. We are currently working on this as a matter of urgency.

UPDATE01 – 12:42

We are currently awaiting the database to recover. As a precaution, we are also restoring a backup of the platform to another host.

UPDATE02 – 13:10

A restore of the platform has resolved the issue. We are sorry for the disruption.

UPDATE03 – 13:15

After review we will be migrating this server to a new version of the software.

UPDATE04 – 17:37

A new software build has been deployed and accounts migrated over. A further window will be raised for the migration works.

UPDATE05 – 02/08/2024 – 21:00

SIP03 accounts have been migrated to the new platform. Test calls have been completed successfully.

Horsham DC – Power Maintenance – 21/06/2024 – 14:00 – 18:00

UK Power Networks have advised us they will be undertaking work on our main incoming power feed for our Horsham Data Center between 14:00 and 18:00 on Friday the 21st June. We will be running on backup power for the duration of the works.

Both UKPN and Structured engineers will be on site for the duration. No impact is expected but the site will be classed as “at risk”

Load tests will be completed prior to the works to ensure a smooth transfer.

UPDATE 01 – 14:29

The site is now on backup power.

UPDATE 02 – 16:00

This work is now complete and the facility is now back on grid power with both A and B feeds being back in service. No service disruption was experienced.

Scheduled Network Maintenance – THN-B (26/07/2024) 22:00 *Amended*

As part of our Scheduled Core Network Enhancements outlined for Q2 of 2024, as communicated in February, we are continuing the replacement of our core devices at Telehouse North. This work will is being done in multiple phases. This work will cover THN-A

Broadband connections routed via THN-A will undergo automatic re-routing before the commencement of the maintenance activities.

However, leased lines terminating on this specific device will experience a service interruption during the maintenance window. Leased lines equipped with backup connections will be seamlessly re-routed.

All customers will see a brief interruption to routing as our transit links converge to THN-B. This should be no longer than 30 seconds.

Should you require further clarification or have any concerns, please do not hesitate to reach out.

UPDATE 01 –

This work has been rescheduled.

UPDATE 02 – 16/07/2024

LNS’s on THN-A have been re-routed to other gateways to lower the number of broadband services we will need to manually drop on Friday.

UPDATE 03 – 18/07/2024

Due to engineering sickness, this work has been proponed until the 26/07/2024

UPDATE 04 – 26/07/2024 – 23:30

This work has started.

UPDATE 05 – 27/07/2024 – 02:15

This work is complete. Engineers are monitoring.

Incident: partner01.easyipt.co.uk – 20/05/2024 – 09:01

We are aware partner01 is suffering a repeat of the issues experienced Friday and are currently investigating.

UPDATE 01 – 09:15

This has been resolved and calls are now processing. We are aware of the root cause of the issue.

UPDATE 02 – 14:00

We have made changes to the server to prevent a service being reloaded that we believe to be the root cause of the issue.

UPDATE 03 – 17:00

In light of the recent issues, we have decided to replace this host and deploy a new version of the software. This will be done manually to ensure a clean build. No changes are expected to be required from users.

UPDATE 04 – 22/05/2024 – 16:00

The new platform has been built, audited and deployed to the relevant ESXi host. We expect to take the current host out of service shortly after 20:00 and bring into service the new one at around 20:05.

This should trigger a re-registration of end points to the new host.

We will update this once tested.

UPDATE 05 – 22/05/2024 – 20:00

This work has started.

UPDATE 06 – 22/05/2024 – 20:05

We have not seen subscribers reconnect as expected. This has been tracked to a service not operating as expected due to being loaded with VPS resources disconnected and a reload is required.

UPDATE 07 – 22/05/2024 – 20:09

We have encountered a problem with incoming calls due to a firewall rule. This is being corrected.

UPDATE 08 – 22/05/2024 – 20:15

We are seeing trunks re-register and testing is now under way.

UPDATE 09 – 22/05/2024 – 20:20

We have tested a number of online trunks without issue. We have also tested on our test platform again without issue. We will monitor for another 30 minutes

UPDATE 10 – 22/05/2024 – 21:15

We have continued to make test calls which have all completed successfully. We are seeing calls progress as expected.

UPDATE 11 – 23/05/2024 – 14:45

We have seen the platform operate without issue so are now closing this.

Scheduled Network Maintenance – THN-B (31/05/2024) 22:00

As part of our Scheduled Core Network Enhancements outlined for Q2 of 2024, as communicated in February, we are now ready for the replacement of our core devices at Telehouse North. This work will be done in multiple phases starting with THN-B to mitigate disruptions as much as possible.

Broadband connections routed via THN-B will undergo automatic re-routing before the commencement of the maintenance activities.

However, leased lines terminating on this specific device will experience a service interruption during the maintenance window. Leased lines equipped with backup connections will be seamlessly re-routed.

Should you require further clarification or have any concerns, please do not hesitate to reach out.

UPDATE 01 – 22:15

This work has started.

UPDATE 02 – 00:50

This work is complete.

Incident: partner01.easyipt.co.uk – 17/05/2024 – 09:01

We are aware the media gateway partner01 has stopped processing inbound and outbound calls. We are currently investigating as a matter of urgency.

We apologise for the disruption this is causing.

UPDATE01 – 09:10

The issue has been located with a backend service and database We are working to restore.

UPDATE02 – 09:45

Service has been restored and we are looking in to why this occurred. Once again we apologise for the disruption caused.

Scheduled Network Maintenance – Broadband (15/04/2024)

We are delighted to inform you that all LNS upgrades have been successfully implemented. As part of this upgrade, we have reconfigured the RADIUS steering away from the legacy devices that remain, facilitating a gradual cessation of sessions on these devices until they are decommissioned during Phase 2 of our network upgrades.

Furthermore, we have removed restrictions on all limited 950Mbps FTTP packages following the completion of the LNS works, thereby enabling the full 950Mbps download throughput for our valued customers.

Scheduled Network Maintenance – Broadband (12/04/2024)

Following the installation of our new broadband gateways, they are now ready to be deployed into active service. Our engineers will be decommissioning the existing gateways, ensuring a smooth migration of sessions. Once a gateway is cleared of sessions, it will be taken offline and replaced with the new device.

LNS01 will be removed from active service for new sessions as of today (12/04/2024). PPP sessions will be gracefully terminated, prompting your router to establish a connection with an alternate gateway. This process should be completed within a few seconds.

UPDATE01 – 12/04/2024

LNS01 has been swapped out with the new device and is now accepting new connections. We will monitor the device and then make the remaining gateway changes.