Connection Suite outage

Incident Report for Dualog

Postmortem

Connection Suite outage - 21. Oct 2018

After a thorough investigation of the incident, we have identified the root cause to be a malfunction within the storage system to one of our providers.

A disk-failure caused the rebuilding process of the storage system to misbehave and degraded the performance to such a point that our services were affected.

Email transfer, file transfer, and the dualog.net portal were down between 15:29 - 16:52 UTC

We are working together with our provider to look into how to prevent this from happening in the future.

System reliability is a top priority at Dualog, and we are continually improving to make our services better. We apologize for the inconvenience caused by the outage.

Posted Oct 24, 2018 - 13:29 UTC

Resolved

The applied fix proves to have resolved the issue, and the system has been running stable.
All services are fully operational.

Posted Oct 21, 2018 - 17:58 UTC

Monitoring

A fix has been implemented, and all systems are back online. We are actively monitoring the results to verify that everything is functioning as it should.
The system is currently processing the backlog. Some email delay is to be expected.

Posted Oct 21, 2018 - 16:52 UTC

Update

Resolving the issue is taking longer than expected.

Services currently down:
- Dualog.net
- Email transfer to and from ship
- File transfer to and from ship

Our Operations team is working tirelessly on restoring operations.

Posted Oct 21, 2018 - 16:03 UTC

Update

We are still experiencing issues. We are continuing to work on a fix.
Dualog.net, email- and file-transfers are currently not working.

Posted Oct 21, 2018 - 15:43 UTC

Update

The database in question is being rebooted. The mentioned services will be down for some minutes throughout the reboot.

Posted Oct 21, 2018 - 15:29 UTC

Identified

We're experiencing degraded performance on one of our databases. You may experience slowness in accessing dualog.net, as well as file- and email transfer to/from ship.

The issue has been identified and our Operations team is working on resolving it.

Posted Oct 21, 2018 - 15:14 UTC

This incident affected: Dualog Portal (apps.dualog.com) and Email and Automatic File Transfer.