Using External Deployment Controller in ECS

Recently, AWS added support for External Deployment Controllers for ECS Services. What does it mean for ECS users? We have received long-awaited possibility to control the deployment process from scratch, leveraging techniques like canary deployment.

In regards to ECS, there are three main concepts essential to manage containers scheduling:

1. Task Definitions
2. Task Sets
3. Services

Task Definition and Service are old abstractions, and Task Set was introduced with External Deployment Controllers support. Just to remind, Task Definition describes container configuration (image, port mappings, environment variables, etc.) and Service takes care of maintaining the desired number of container replicas (most often but not only) in cluster.
Task Set is a new abstraction. There is no direct mapping from Service to Task Definition anymore, and Task Set is an entity that glues everything together (in the example below, only subset of options are shown – for complete output, take a look at aws ecs create-task-set --generate-cli-skeleton command):

{
   "service": "service-name",
   "cluster": "cluster-name",
   "taskDefinition": "task-definition",
   "launchType": "EC2",
   "scale": {
       "unit": "PERCENT",
       "value": 1
   }
}

AWS provides three API calls to manage Task Sets:

func (c *ECS) CreateTaskSet(input *CreateTaskSetInput (*CreateTaskSetOutput, error)
func (c *ECS) UpdateTaskSet(input *UpdateTaskSetInput (*UpdateTaskSetOutput, error)
func (c *ECS) DeleteTaskSet(input *DeleteTaskSetInput (*DeleteTaskSetOutput, error)

func (c *ECS) UpdateServicePrimaryTaskSet(input *UpdateServicePrimaryTaskSetInput) (*UpdateServicePrimaryTaskSetOutput, error)

Having that in mind, deployment process could be coded as follows:

n := CreateTaskSet(...)

// check if service is stable

UpdateServicePrimaryTaskSet(n)
DeleteTaskSet(old)

Because everything is wrapped in API calls, any check can be made between a call to CreateTaskSet and DeleteTaskSet. A couple of things need an explanation:

1. There can be multiple Task Sets within a single Service. Each Task Set can use different Task Definition and be replicated with a different number of containers.
2. scale parameter defines the percentage number of Service desired count, that will be instantiated with that Task Definition.
3. Only one Task Set can be set to Primary. The key characteristic of Primary Task Set is that it can not be deleted with DeleteTaskSet call (it can be deleted with DeleteService though).
4. Task Sets are identified by ARNs and IDs (i.e. ecs-svc/9223370472108164444). Enabled Task Sets can be seen in Deployments tab of ECS Service.

Once you get familiar with (simple) rules of TaskSets, you will be able to build any custom CI/CD pipeline, tailored to your needs. Thanks to flexibility that External Deployment Controller offers, any advanced deployment technique can be used with applications running on ECS platform.

Sources:

https://aws.amazon.com/about-aws/whats-new/2019/03/aws-fargate-and-amazon-ecs-support-external-deployment-controlle/

https://docs.aws.amazon.com/sdk-for-go/api/service/ecs/#pkg-overview