Docker is an efficient way to use command-line tools without needing to install them or their required libraries. Recently, I had a use case where I needed to take an SQL dump of a schema from a PostgreSQL database. The problem was that the PostgreSQL DB was version 16.3, while my local DB was only version 15. Using my local pg_dump to access the remote database resulted in a version incompatibility error. To fix this, I would either need to upgrade pg_dump or have both versions installed simultaneously on my machine. I didn’t want to do either. This is where Docker came to the rescue.

Amazon Elastic Container Registry (ECR) is a service provided by Amazon for hosting our Docker/container images. The cost of downloading these images to a server or ECS (Elastic Container Service) within the same zone is free. However, downloading or uploading from the public internet incurs costs, as does storage space usage.

During the development and deployment of your project, you may find yourself building images as frequently as every 30 minutes. This frequency reflects my own experience. Whenever new code is merged with the master branch, it triggers a code build pipeline. This pipeline ultimately pushes the latest image to ECR, typically tagged as “latest.” Consequently, the previous image remains in the repository but becomes untagged. While retaining a few versions of older deployments facilitates easy rollbacks, after a day or two, they tend to become redundant. At this juncture, they serve no purpose other than incurring storage costs.

As someone who has been working with servers for over a decade and a half, I feel embarrassed to admit that I only recently stumbled upon this setting. Throughout my years of connecting to remote servers over the internet, I’ve encountered instances where connections freeze or break after a period of inactivity. Sometimes it happens more frequently, while other times it never occurs at all. When I was in the US connecting to Amazon servers, there were occasions when the connection remained stable for days on end. This led me to believe that the issue might be related to the internet or the complexity of the network routing required to reach the server.

Recently I been trying to convince my team to write more tests, or some test. One of the arguments I heard during this was - automated test doesn’t remove manual test. Thats true, you still have to manually test it as well. But it avoids the repeated testing, and forgetting to test old features when new modifications are being done. I was thinking more about what to reply to people that says this, when it dawn to me that.

Recently I have been working with deploying self hosted application in AWS lightsail for the company that I work for. My strategy for deploying application has been as follows:

• Launch a new ubuntu instance (since my team and I use it a dev machine we are familiar with it).
• Install Docker
• Install Docker Compose
• Run the self hosted app and its requires services using docker.
• Run nginx reverse proxy to serve the app over https

I am pretty much managing the whole app through files. Its just the initial setup that I am doing manually. I thought of creating a base linux image for the new servers, which would have docker and docker-compose pre installed. But lightsail didn’t seem to have the option to use custom images, like EC2 servers.

Thus I decided to use ansible.

Ansible is an open source automation tool that is used for configuration management, deployment and task automation. It automates the steps that I do manually. It can run this command on multiple servers without having us do anything manually. We can also automate weekly maintenance check to login to all the servers, ensures all the security updates are installed, cache files are cleared, etc.

The thing I found most interesting about ansible compared to chef (another automation tool) is that it is agentless. That is it doesn’t expect anything to be running on the new server other open ssh. Chef on the other hand requires an agent to be running on the server. There are its own advantages of having an agent running on the server, it can do updates much faster and keep everything in sync. But since that is not a requirement for me at the moment, I decide to use ansible. I will write another article on how to do this using chef.