NOC: SSL Certificate – LON01 (EasyHTTP Service) *Complete*

Following on from our planned maintenance works on EasyHTTP we removed the SSL certificate securing the control panel.

We have been unable to reinstall this. We have a backup for both IIS and Plesk. We have logged an issue with our software vendor to see if these can be re-installed. If not then we will simply install a new one.

UPDATE01 – 15:15 – 24/12/2013
A new SSL certificate has been ordered and will be installed shortly.

UPDATE02 – 16:00 – 24/12/2013
The new SSL certificate has just been issed by AlphaSSL and installed on the system. This Notice is closed

NOC: Planned Maintenance – 20/12/2013 20:00 till 23/12/2013 23:59 – LON01 (EasyHTTP Service) *COMPLETE*

In line with our current service level agreements and previously advised (where applicable) we are pleased to announce we will be upgrading our current webhosting service (EasyHTTP) to our new hardware platform (HP DL580) very shortly. Our planned migration date for this is the 20th December 2013, however this will likely change depending on the migration method used and should be considered as TBC (to be confirmed)

There are no planned changes to IP addressing at this moment in time, and is the preferred migration method. However this will result in some downtime in order to ensure no new data is upload to the old server during the migration window.

Inbound and outbound SMTP will be refused at times during this window, unless you are using our backup MX service. In which case inbound mail will be held and delivered once the new server is online.

The new platform consists of:

Quad power supplies 6000 Watts (one per CPU)
4 x Intel quad core 2.93 GHz processers
32GB of RAM
8 x 500GB SAS drives in a RAID 5 configuration (3.2TB of useable space)
Dual 1Gbps uplinks

The upgrade is a very welcome project and more details will follow as the migration plan is finalised.

UPDATE01 – 17:04 – 09/12/2013
During an audit of existing scripting languages we have discovered PHP 4.4.9 is still installed. As this release has gone EOL (End of Life) it will not be installed on to the new platform. All existing sites using 4.4.9 will be re-graded to 5.2.17

We will also be upgrading MS SQL to 2012 which will result in a Instance Name change from “default” to “MSSQLSERVER2012” (excluding the “”) PLease ensure you update any connection strings.

UPDATE02 – 21:30 – 19/12/2013
We have started a FULL manual backup of the platform which will be used as the base / current image on the new array. Should the restore go as planned and once we are happy then we will manually copy over any new email received on the old system between now and midnight tomorrow. Once this is complete we will be taking the old system offline. Any new website data will not be migrated due to the size and time it would take resulting is a further loss of data. We would advise keeping any new uploads / changes to existing website data to a minimum.

We will update this once the backup and restore has completed advising of the next steps.

UPDATE03 – 23:00 – 20/12/2013
Migration is running behind due to an issue that has occurred with the use of the same IP addressing and the way Plesk maps this to the physical network card when not physically enabled. A new VLAN was installed to avoid this however we have had to process the backup / restore another way due to the extreme size of the backup and processing limitations on various protocol.

We are migrating data to a temporary IP range that we will revert once complete.

UPDATE04 – 01:15 – 21/12/2013
Migration work is still on-going and back on track, However due to the additional changes required we have extended this maintenance window for a further 24 hours.

This window will now close Sunday 6am.

UPDATE05 – 15:15 – 21/12/2013
The migration work completed last night however several warnings where encountered as MySQL stopped accepting connections from “localhost” and prevented a small number of domains having their mail boxes created. This issue has been resolved and we are now re-restoring the effected domains.

UPDATE06 – 18:15 – 21/12/2013
We have finished restoring the effected domains and the primary migration is now complete. We have now started the test migration of any email received since the initial copy. We have again extended this maintenance window for a further 24 hours.

This window will now close Monday 6am.

UPDATE07 – 19:00 – 22/12/2013
We have now started the final migration of any email received since the initial copy. Concerns over the size means we will be unable to copy any further email received on the old system from this time (19:00) We advise everyone to make a local copy of email received from this point to their local mail platform which can be copied back after

UPDATE08 – 23:30 – 22/12/2013

We have encountered an error that is preventing us from changing the primary default ip address. The error we are getting is in relation to an IIS module, however we suspect this is a corruption in an MS sql database.
We have been forced to log this with the software vendor as an escalation. As we have no influence on timing we are unlikely to have this resolved within our maintenance window and have therefore suspended this change. We are hopeful however this should be resolved by midday tomorrow so have extended this maintenance window until 23:00 tomorrow (Monday 23rd) This will allow us to migrate email data (again) however we are unable to restore any new website content due to time implications.

UPDATE09 – 12:15 – 23/12/2013

We have been advised by our software vendor that they have resolved the issue effecting the reconfiguration of IP addressing and the issue was related to corruption in the Master Plesk database and missing SSL certificate data following on from the migration. As the migration was completed via the built in migration tool we have raised a BUG with the vendor and ask them to investigate, however this may be related to the way we were forced to complete the migration.

The changes have now been made and the server is ready to come online and take over following the migration of any new email. As the new server now has the same IP addressing as the old system we are unable to complete a direct copy and will be required to copy the data to 3rd party storage device within our network. This additional step will require more time to complete the transfer

We therefore have confirmed the following times:

Copy email: 21:00
Down old server: 22:00
Up new server: 23:30

This will result in a planned 1 hour outage.

If any problems are experanced with the server then we will revert.

UPDATE10 – 20:15 – 23/12/2013

The work above has started. Times have been amended due to delays with weather and engineering.

UPDATE11 – 22:50 – 23/12/2013

We have completed the copy of any new email delivered to the old platform since 21:00 and are currently downloading this to the new system. The additional migration to an external storage source has added delay as expected. We have added an extra hour to the maintenance window.

UPDATE12 – 23:59 – 23/12/2013

We have completed the migration of email and the VLANS have been swapped on our core. This upgrade is now complete