How to Quickly Fix AWS CodeDeploy “ApplicationStop” Event Failures
By Bobby Gill on July 27, 2021
At Blue Label Labs, we leverage the triumverate of Amazon Web Services’ (AWS) CodePipeline/CodeDeploy/CodeBuild services to automate the CD/CI deployment pipelines for most of our app development projects. While on the whole, the AWS code integration and deployment services work well, they do have their warts. One very common issue developers run into is that a CodeDeploy deployment fails due to an “ApplicationStop” failure. Essentially this error is thrown when the script executing in the ApplicationStop lifecycle event at the start of a new deployment fails to complete. The frustrating part is that this actually happens quite a bit and for reasons that usually aren’t related to the new deployment we are attempting. In this post I will show you how to quickly resolve this issue using the CodeDeploy UI.
We generally like to use the pm2 process management tool for running and hosting our backend services, so within our ApplicationStop script we generally stop the pm2 process and remove it. However, there are a lot of times when the pm2 process is either already stopped or not even present, usually due to a previous legitimate CD/CI failure. So right after we manage to fix the original error in your deployment, we make our commit and expect the world to return to normal, except it doesn’t as CodeDeploy promptly runs into an ApplicationStop error, throws up its hands, gives up and goes out for a long coffee break.
Before going on this article, if you are looking for help in setting up CodeDeploy/CodePipeline to deploy Node-based backends onto EC2, then check out this 3-part series outlining every step in getting an automated pipeline in place. If you are a .NET developer and looking to do the same with an ASP.NET Core web api project, then check out this tutorial walking you through the steps to setup CodeDeploy/CodeBuild/CodePipeline to deploy an ASP.NET Core project.
Fortunately, overcoming these frustrating ApplicationStop issues can be done rather easily through the AWS CodeDeploy UI. Here’s how to do it:
How to Fix CodeDeploy ApplicationStop Errors in the AWS UI
1.) Copy the Artifact Revision Location from the failed Deployment in the AWS CodeDeploy UI
In the CodeDeploy console, locate the CodeDeploy application in question.
Click open the CodeDeploy Application and then identify the Deployment Group which threw the error.
In the resulting page, scroll down to the “Deployment group deployment history” table, locate the most recent deployment which failed and click open the Deployment.
Note down the s3 path that is displayed as the value for the Revision location field. Generally you really only need to remember the last 4 digits of the revision.
2.) Create a new Deployment using the same Artifact Revision Location
In the CodeDeploy console, locate once again the CodeDeploy application, click it open.
Click open the same Deployment Group as identified in the step 1.)
In the upper right hand corner, click the “Create Deployment” button.
In the resulting screen, check the Revision type option of “My application is stored in Amazon S3”.
Now under Revision location, click the text box and you should see a list of s3 paths for all the previous artifacts used within this Deployment group. Identify the revision location you noted down at the end of Step 1.) and select it.
Under Revision file type, select “.zip” file.
Now finally, the most important step, under “ApplicationStop lifecycle event failure“, check the box that says “Don’t fail the deployment to an instance if this lifecycle event on the instance fails”.
Then click “Create deployment”
And that’s it! Now you should have a deployment running that will still hit the same ApplicationStop failure as before, however, it will no longer fail and stop the entire deployment. CodeDeploy will swallow the error silently and proceed through the rest of the steps and should ultimately succeed.