Current network status and maintenance

Our ongoing efforts over the weekend have resulted in improvements to email services.
Many affected customers should now experience improved performance. However, some customers may continue to experience intermittent issues.

If you are completely unable to access your email, please report this to us using the following form so that our team can investigate:

https://email-report.xneelo.co.za

Our engineers continue to monitor the environment closely and are working to restore normal service for all affected customers.

We will provide our next update at 14h00 SAST today.

We are on day six of the email service disruption affecting some of our Web Hosting customers on shared servers in Johannesburg.

We know how serious and disruptive this has been for our customers. This is an unprecedented and complex incident in our environment and continues to have the full focus of our engineering and leadership teams.

Below is a clearer breakdown of how we got here, what we have done so far, and where things stand today.

How we got here:
Earlier this year, we started the migration of our shared web hosting platform from bare-metal servers to our virtualised cloud environment.

Our new virtualised environment consists of two different storage pools optimised for high-intensity web workloads and standard email workloads. We prioritised the web workloads on premium storage.

This was designed to improve performance and scalability and has been operating successfully.

In August, we rolled out a major operating-system (Debian Linux) upgrade across our entire managed hosting platform (approximately 4,000 servers).

This upgrade followed our tried and tested process:
- Extensive development and testing on test servers.
- Production rollout to a small number of customer hosting servers.
- Monitoring the environment and customer feedback for any observed issues.
- Proceeding with increasing batch sizes, while continuing to monitor for issues between batches.

At the time of initiating the upgrade on our cloud-based shared hosting servers, we had completed the upgrades of approximately 3,400 dedicated and shared servers.
- No issues were observed or reported during these upgrades.
- During the weekend beginning 28 August, we continued with upgrades on our cloud-based shared hosting servers, with no observed or reported issues on Monday 31 August.

On Tuesday,1 September, it became clear that something was wrong.

The underlining issue:
Our storage infrastructure includes redundancy and failover for hardware failures. This incident is different: it is caused by software-level behaviour introduced by the operating-system upgrade and was present across the affected environment. Moving workloads between redundant hardware therefore could not bypass the underlying bottleneck.

An ongoing investigation has identified that the recent operating system upgrade introduced a fundamental change in how servers save data to and read data from our Cloud storage drives.

Why we didn’t enact a roll-back:
We did not roll back the operating system because the security risks and potential impact of running on end-of-life software were too high.

What we have done:
As soon as the scale and impact of the degradation became apparent, we initiated our incident response process.

Because there was no single, obvious failure, such as a hardware fault, and the issue only emerged under full production load, identifying the cause was complex.

Our engineers had to work systematically through a number of interrelated possibilities, ruling them out one by one while protecting the integrity of customer email and data.

This troubleshooting phase took significant time. Several interventions addressed individual symptoms or reduced pressure on the environment, but did not fully resolve the underlying storage bottleneck.

Once we had isolated storage performance as the core constraint and established that moving servers could safely relieve pressure on the affected environment, we began migrating affected servers to a premium storage tier.

We also introduced new controls to restrict resource-intensive background processes and moved some logging and mailbox-management workloads onto the premium storage tier.

Moving a server requires very large volumes of live email data to be transferred safely. These migrations themselves place additional demand on the storage infrastructure, so they have had to be completed carefully and in a controlled sequence to ensure data integrity and avoid any risk of email loss. This careful migration process has accounted for much of the recovery time.

We have now successfully migrated 50% of the affected servers to more premium storage. Because the impact was widespread across the affected environment, we did not prioritise specific servers for migration.

Our premium storage tier has finite capacity. We are therefore not able to migrate any more servers to this premium tier with the capacity currently available.

These migrations have removed a significant amount of demand from the affected storage environment. We expected the reduction in load to improve email performance for customers whose servers remain on the standard storage tier as well.

What remains
Customers hosted on servers that have been migrated to premium storage have had their email services restored.

Approximately 50% of affected servers remain on the standard storage tier and cannot currently be migrated to premium storage.

We are continuing to tune the operating system and storage environment while monitoring email delivery, connection performance and the processing of queued mail.

Some customers may therefore continue to experience delayed email delivery, connection timeouts or unreliable mailbox access while this work continues.

We have no evidence that customer email data has been lost. Delayed email remains securely queued and is processed as storage performance allows.

When will email service be fully restored?
We know this is the question our affected customers most need us to answer.

We recognise that after six days without reliable email, another update without a restoration time is extremely difficult. At this stage, we cannot provide an ETA that we can confidently stand behind.

The work completed so far has reduced pressure on the affected storage infrastructure. We now need to confirm that this translates into sustained, reliable email performance for customers whose servers remain on the standard storage tier as email activity increases.

Testing continues today using simulating busy workloads to assess whether our recovery measures can support the demands of a busy business day.

Our teams remain fully focused on restoring reliable email service. We are deeply sorry for the duration of this incident and the impact it has had on you and your business.

We will provide our next update by 09h00 SAST on Monday, 7 September 2026.

Our engineering team is making progress while continuing to troubleshoot and closely monitor performance of those servers not yet migrated. We have processed a significant volume of queued email, although delivery backlogs remain.

Temporary suspensions of email access may still occur on specific servers while this work continues. Incoming email will remain securely queued for delivery, with no loss of email, and website functionality will not be affected.

Our next update will be Sunday, 15h00 SAST.