This incident is now fully resolved and we've published a comprehensive retrospective analysis and report on our site, here: https://ghost.org/faq/salt-incident/
Posted May 07, 2020 - 12:28 UTC
Update
All traces of the crypto-mining virus were successfully eliminated yesterday, all systems remain stable, and we have not discovered any further concerns or issues on our network. The team is now working hard on remediation to clean and rebuild our entire network. We will keep this incident open and continue to share updates until it is fully resolved. We will also be contacting all customers directly to notify them of the incident, and publishing a public post-mortem later this week.
Posted May 04, 2020 - 08:29 UTC
Monitoring
A fix has been implemented and we are monitoring the results.
Posted May 03, 2020 - 13:06 UTC
Update
Our additional firewall configurations are now running and working as expected. All connectivity issues have been resolved and customer sites are loading as normal again.
We're continuing to monitor all systems closely, whilst also working carefully to cycle all sessions, passwords and keys on every affected service as a precaution.
We will share more updates now when we have further information.
Posted May 03, 2020 - 13:06 UTC
Update
We are continuing work to address the network instability caused by additional firewalls introduced on Ghost(Pro) today. Connections are beginning to be restored now, and we are working toward getting back to full capacity.
Further information on the security issue from earlier today: Our investigation indicates that a critical vulnerability in our server management infrastructure (Saltstack, CVE-2020-11651 CVE-2020-11652) was used in an attempt to mine cryptocurrency on our servers. The mining attempt spiked CPUs and quickly overloaded most of our systems, which alerted us to the issue immediately.
At this time there is no evidence of any attempts to access any of our systems or data. Nevertheless, all sessions, passwords and keys are being cycled and all servers are being re-provisioned.
Posted May 03, 2020 - 12:46 UTC
Update
We’ve introduced multiple new firewalls and security precautions today which are unfortunately causing instability on our network and affecting some customer sites. Our team is hard at work restoring all sites as quickly as possible, whilst going to extra lengths to ensure that all customer data is secured.
We’ll continue to share updates as things progress, and we’ll share a detailed postmortem after everything is resolved.
Posted May 03, 2020 - 11:44 UTC
Update
We are continuing to work on a fix for this issue.
This affects both Ghost(Pro) sites and Ghost.org billing services.
We are able to verify that: - No credit card information is affected - No credentials are stored in plaintext
There is no direct evidence that private customer data, passwords or other information has been compromised. All sessions, passwords and keys are being cycled and all servers are being re-provisioned.
Posted May 03, 2020 - 09:15 UTC
Identified
The issue has been identified and a fix is being implemented.
Posted May 03, 2020 - 07:52 UTC
Update
We have restored all services and everything should be functioning as normal. We are still investigating the root cause of the issue with our upstream providers.
Posted May 03, 2020 - 06:42 UTC
Update
We are continuing to investigate the cause of the outage, whilst also working to restore as much functionality as possible.
We'll post more detail as soon as we have it.
Posted May 03, 2020 - 03:53 UTC
Update
We are continuing to investigate this issue.
Posted May 03, 2020 - 03:52 UTC
Investigating
We are investigating the cause of an outage. We will update as soon as we have more info