Her real comment was that there are only 3 major cloud providers they can consider: AWS, GCP, and Azure. They chose AWS and AWS only. So there are a few options for them going forward — 1) keep doing what they’re doing and hope a single cloud provider can improve reliability, 2) modify their architecture to a multi-cloud architecture given the odds of more than one major provider going down simultaneously is much rarer, or 3) build their own datacenters/use colos which have a learning curve yet are still viable alternatives. Those that are serious about software own their own hardware, after all.
Each choice has its strengths and drawbacks. The economics are tough with any choice. Comes down to priorities, ability to differentiate, and value in differentiation :)
Meredith mentioned in a reply to her posts that they do leverage multi-cloud and were able to fall back onto GCP (Google Cloud Platform), which enabled Signal to recover quicker than just waiting on AWS. I’d link to source but on phone, it’s somewhere in this thread: https://mastodon.world/@Mer__edith/115445701583902092
The big 3 also offer disgustingly fast interconnection. Google, Amazon and Microsoft lay their own undersea fiber for better performance.
If willing to sacrifice a bit of everything, OVH has North-American and European locations, as well as one in India, one in Singapore and one in Australia. They’re building a few more in India, one in Dubai, two in Africa, one in NZ and 3 in South America. Once they add a few more on top of those, that’s damn near worldwide coverage too. And OVH is a French company, so the US government has less leverage over it than Amazon.
Those are the only 3 that matter at the top tier/enterprise class of infrastructure. Oracle could be considered as well for nuanced/specialized deployments that are (largely) Oracle DB heavy; but AWS is so far ahead of Azure and GCP from a tooling standpoint it’s not even worth considering the other two if AWS is on the table.
It’s so bad with other cloud providers that ones like Azure offers insane discounts on their MSSQL DB (basically “free”) licensing just to use them over AWS. Sometimes the cost savings are worth it, but you take a usability and infrastructure hit by using anything other than AWS.
I honestly, legitimately, wish there was some other cloud provider out there that could do what AWS can do, but they don’t exist. Anyone else is a pale imitation from a devops perspective. It sucks. There should be other real competitors, especially to the US based cloud companies as the US cannot be trusted anymore, but they just don’t exist without taking a huge hit in terms of tools, APIs, and reliability options, to AWS.
I always wondered why people don’t implement a multi-cloud infrastructure if they want/need extra HA. And I know Oracle offers a solution with Azure and GCP, with AWS on the horizon. Not to advertise for Oracle because they’re terrible otherwise, but I can’t imagine wanting a multi-cloud option and not consider them.
Multi region is already hard enough with transactional management not being easy to split between the regions, and multi-cloud is another order of magnitude more difficult than multi region.
With that said, use2 and others were still up, so if they were just multi region and failed over to east2 they would have been fine.
Her real comment was that there are only 3 major cloud providers they can consider: AWS, GCP, and Azure. They chose AWS and AWS only. So there are a few options for them going forward — 1) keep doing what they’re doing and hope a single cloud provider can improve reliability, 2) modify their architecture to a multi-cloud architecture given the odds of more than one major provider going down simultaneously is much rarer, or 3) build their own datacenters/use colos which have a learning curve yet are still viable alternatives. Those that are serious about software own their own hardware, after all.
Each choice has its strengths and drawbacks. The economics are tough with any choice. Comes down to priorities, ability to differentiate, and value in differentiation :)
Meredith mentioned in a reply to her posts that they do leverage multi-cloud and were able to fall back onto GCP (Google Cloud Platform), which enabled Signal to recover quicker than just waiting on AWS. I’d link to source but on phone, it’s somewhere in this thread: https://mastodon.world/@Mer__edith/115445701583902092
What reason do they give for only wanting to use those three cloud providers? There are many others.
Scale, they need worldwide coverage.
https://mastodon.world/@Mer__edith/115445705126997025
The big 3 also offer disgustingly fast interconnection. Google, Amazon and Microsoft lay their own undersea fiber for better performance.
If willing to sacrifice a bit of everything, OVH has North-American and European locations, as well as one in India, one in Singapore and one in Australia. They’re building a few more in India, one in Dubai, two in Africa, one in NZ and 3 in South America. Once they add a few more on top of those, that’s damn near worldwide coverage too. And OVH is a French company, so the US government has less leverage over it than Amazon.
And yet a single availability zone in AWS going down caused an outage?
Yes, because scale is not the same as redundancy.
Those are the only 3 that matter at the top tier/enterprise class of infrastructure. Oracle could be considered as well for nuanced/specialized deployments that are (largely) Oracle DB heavy; but AWS is so far ahead of Azure and GCP from a tooling standpoint it’s not even worth considering the other two if AWS is on the table.
It’s so bad with other cloud providers that ones like Azure offers insane discounts on their MSSQL DB (basically “free”) licensing just to use them over AWS. Sometimes the cost savings are worth it, but you take a usability and infrastructure hit by using anything other than AWS.
I honestly, legitimately, wish there was some other cloud provider out there that could do what AWS can do, but they don’t exist. Anyone else is a pale imitation from a devops perspective. It sucks. There should be other real competitors, especially to the US based cloud companies as the US cannot be trusted anymore, but they just don’t exist without taking a huge hit in terms of tools, APIs, and reliability options, to AWS.
I always wondered why people don’t implement a multi-cloud infrastructure if they want/need extra HA. And I know Oracle offers a solution with Azure and GCP, with AWS on the horizon. Not to advertise for Oracle because they’re terrible otherwise, but I can’t imagine wanting a multi-cloud option and not consider them.
Multi cloud is very difficult to do well.
Multi region is already hard enough with transactional management not being easy to split between the regions, and multi-cloud is another order of magnitude more difficult than multi region.
With that said, use2 and others were still up, so if they were just multi region and failed over to east2 they would have been fine.