Choosing between Amazon ECS and EKS¶
We frequently get asked by users that are currently on AWS whether they should be using Amazon ECS or EKS to deploy and operate their containerized applications. Since this is such a common question and the answers are somewhat nuanced, we wanted to share our thoughts and recommendations for the benefit of all users.
Synopsis¶
Here is a table summarizing all the important criteria you need to factor in as you evaluate between Amazon ECS and EKS.
| # | Amazon ECS | Amazon EKS |
|---|---|---|
| 1 | You have a simple app with not many microservices | Your app is a large, complex and distributed with many microservices |
| 2 | Your app has minimal distributed dependencies | Your app has complex inter-app dependencies. You may need to operate on-demand, parallel workloads |
| 3 | App team's part time job is Infra/Ops on AWS | You have an existing IT/Ops team that can support you |
| 4 | You have selected AWS as your ONLY infra provider for the long term | You need to design for consistency and portability to support multi-cloud or hybrid |
| 5 | Budget is not a major concern | Cost controls are critical |
Why Amazon ECS?¶
If you are new to containers and cloud. And on top of this, if the app team is being forced to make infrastructure decisions for their containerized application, they will find it extremely easy to get started with Amazon Elastic Container Service (ECS). They will typically experience the following benefits right away:
- Do not have to deal with a moving target with Kubernetes versions
- Zero cost for the managed ECS control plane
- A pay-as-you-go model for the Fargate based data plane with full support for Spot etc
Note
ECS offers two launch types: Fargate and EC2. Fargate is the serverless option and users do not have to provision or manage the underlying servers. If you cannot use ECS with Fargate, users will not see most of the benefits of ECS.
We always caution customers to consider and evaluate carefully avoid going down a "one-way door". This is particularly critical for mid-to-large size organizations to consider carefully so that they can avoid painful and expensive migrations to Amazon EKS down the line.
Why Amazon EKS?¶
To help understand this better, let's also look at this from the lens of Kubernetes, Amazon EKS and why it became so popular.
Low Control Plane Cost¶
The monthly cost for the SLA backed, managed control plane for Amazon EKS is only $70 per month. Amazon EKS integrates natively with AWS services esp. such as Spot, IAM and Fargate etc. With the use of market leading add-ons such as Karpenter, operational effort associated with EKS cluster sizing and scaling can be completely automated as well.
The cost of the EKS managed control plane is a rounding error for any budget.
Forced Update Cadence¶
A forced update cadence due to enhancements via new upstream Kubernetes releases is actually a good thing! There is enormous momentum behind Kubernetes that organizations are benefiting from. By standardizing on Kubernetes, organizations ensure they do not run the risk of stagnation.
The cost/burden of periodic updates can be extremely low if organizations leverage platforms with "fleet", "API deprecation checks" and "Add-On Update" type capabilities to keep their clusters standardized, consistent and on the latest versions. This should be a critical requirement in your evaluation criteria for Kubernetes Management.
Access to IT/Ops Talent¶
Organizations know that personnel costs can get prohibitive and not having the qualified personnel can significantly impact their corporate initiatives. With CNCF backing Kubernetes with a comprehensive certification program, there is an extremely large and fast growing pool of talent that organizations are able to tap into for Operations and DevOps talent.
It is significantly easier and more cost effective to find Kubernetes expertise in the market relative to Amazon ECS.
Standardized API¶
One of the biggest misconceptions in the market is that the value of Kubernetes is about the distribution and the managed/hosted control plane. This is grossly incorrect!
The biggest benefit of Kubernetes is that your applications get to use a standardized, portable, version controlled API (i.e. the Kubernetes API). Because of this, we have seen our customers deploy the exact same application on multiple clouds with zero changes.
An application using the Kubernetes API can be deployed on ANY Kubernetes distribution on ANY infrastructure provider as long as the distribution itself is compliant with upstream Kubernetes.
It would be terrible for an organization to squander this superpower. Just think of the leverage this provides you in your engagement with your provider and how it empowers you to support your business requirements.
Vendor Lock-in¶
When we refer to "vendor lock-in", we are not just referring to the infrastructure or the managed service. We are also referring to the tooling supporting the infrastructure and applications. In fact, this is typically the long pole in the tent for most organizations.
Being a closed, proprietary offering, Amazon ECS does not have the kind of support for deploying and configuring plugins to manage security, monitoring, and more importantly automation. For example, instead of being limited to
- AWS CodePipeline, With EKS, you can use ArgoCD or FluxCD or Rafay or a long list of CD tools to deploy and operate your applications.
- AWS IAM, you have Kubernetes RBAC that can be seamlessly integrated with your corporate Identity Provider (IdP) such as Okta, Microsoft EntraID, Ping etc.
- AWS based autoscaling configs, you can use Kubernetes HPA/VPA/KEDA etc.
- AWS CloudWatch, you have Prometheus, Grafana and a plethora or commercial offerings
- AWS Step Functions, you have a number of 3rd party workflow tools
Since EKS is essentially standard upstream Kubernetes, there is an endless range of 3rd party tooling available for this eco-system
Identical Developer Environments¶
With Amazon EKS, you can have almost identical setups so what you test locally is the same and you can use the same network plugins, security pod restrictions, etc.
What you actively develop and test on can mirror the production environment
Since you cannot run ECS locally, you have to resort to a "dual stack" approach i.e. running production apps in a black box while leaving your developers to fend for themselves with things like docker compose etc.
Best of Breed Patterns¶
With Amazon EKS, you can leverage and use battle tested patterns such as GitOps from a range of providers like ArgoCD, FluxCD and Rafay. This reduces business risk significantly.
Power of Community¶
There is an enormous and active community for Kubernetes. For example, it is trivial to find a solution for a problem packaged as Helm charts vs hoping someone has written/published an ECS task for your problem.
Customization¶
Doing anything custom in Amazon ECS can be extremely painful or impossible. With Kubernetes, you have access to well documented and established patterns such as CRDs and Operator patterns. As always, our recommendation would be to pursue the customization route only if the use case warrants it.
Kubernetes is the unchallenged king of customization.
Segregation of Access¶
With ECS, task separation vs host access grants "too much" or "too little" wrt access. Trying to manage that granularity through AWS IAM is extremely brittle or downright impossible in most cases.
With EKS (Kubernetes), organizations can segregate access by service, deployment, and enable developers to do more without risk of breaking things or accessing things they should not. They can combine it with things like AWS Secrets Manager or HashiCorp Vault for secrets management. In a nutshell, you have elegant and simple ways of enabling secure secrets in your services without making it available for developers who can touch the ECS host itself.
Summary¶
If you are a mid-to-large organization, even if your application is the first containerized application, you need to skate to where the puck is going. The IT/Ops team will eventually have to support other containerized applications and the business will likely be required to support other infrastructure providers as well. As a result, it may make sense for organizations currently on AWS to stick with the logical, practical and sensible option i.e. Amazon EKS.
Whether you are an organization starting out with your first containerized application or have a mature practice, you can leverage the Rafay platform to streamline and optimize your operations. Read how Rafay's customers have benefited.
Blog Ideas¶
Sincere thanks to readers of our blog who spend time reading our product blogs and suggest interesting topics to use. Please Contact the Rafay Product Team if you would like us to write about other topics.