Overview#
In this tutorial, we'll understand the requirements of metrics for our server instances. We'll set up an EC2 instance and configure cloudwatch to track metrics for the instance and set up alerts when certain criteria are met
Prerequisites#
You'll need an AWS account. If you do not have one, sign up on aws.amazon.com.
Metrics, and why we need them#
Every server instance we use has a finite amount of load it can hold, the CPU power, the number of reads/writes, and so on. In case of excessive load, the server might crash, leading to service disruption for users. While it doesn't sound like a big deal when working on personal projects, it can have serious business consequences when working with actual users. Remember the time Google went down for just 45 mins? The world practically came to a standstill. To avoid this, we use metrics that track server activity - how many requests it's handling, the CPU being used, and so on. If we see traffic hitting the server's limits, we can configure additional instances and balance load across these
Introduction to AWS Cloudwatch#
AWS Cloudwatch is a service that tracks various metrics for your AWS resources, including EC2 instances, S3 buckets, lambda functions, EBS and more.
You can create dashboards to track the metrics over time and can take intelligent decisions regarding scaling up your server capacity
Most importantly, it also allows to setup alarms when certain critical thresholds are hit, so that you can take action immediately without any service disruption
Getting hands on with Cloudwatch#
In this tutorial, we'll be setting up an EC2 instance and run a simple React app on it. We'll then track the metrics of the instance as we make hits to the server and set up an alarm when the requests cross a certain threshold
The following section describes the steps to set up and run a React app on an EC2 instance. If you already have one running, skip this section and go to the next one.
Setting up a react app on an EC2 instance#
Next, let’s set up a remote EC2 server instance. As said before, you’ll need an AWS account for the same. If you don’t already have one, you’d need to create it. Remember, it’ll ask you for debit/credit card credentials, but as long as you follow the steps in this tutorial, you will not get charged for it.
To set up an AWS account, go to https://aws.amazon.com and follow the steps to set up an account. You’ll get a confirmatory mail once your account is set up and ready.
Once you login to the account, you should see a screen similar to this
Click on the blue ‘Launch a virtual machine’ line, and you’ll be taken to the EC2 setup screen, wherein you’d have to select an AMI, an Amazon Machine Image.
An AMI describes the configuration of the server you’d be using to host your application, including the OS configuration - Linux, Ubuntu, Windows etc. If you have been following tech news, a Mac version was also released for the first time in early 2021.
We’ll be going with Ubuntu server 20.04. You may choose another, but the rest of the steps might vary slightly. Also, do NOT choose an option that doesn’t have the ‘Free tier eligible’ tag, otherwise, you’ll be having to sell off some jewellery to pay the AWS bill.
The next step is choosing an instance type. This describes the server configuration, including CPU, memory, storage, and so on.
Here, we’ll pick the t2.micro instance types, which is the only one available in the free tier. You’ll need larger ones as your application size and requirements in RAM or processing speed increase. In case you’re not clear with any of the column fields, click the information icon next to the headings to get a description of what it means.
Once this is done, click on Next: Configure Instance Details
Here, you’re asked the number of server instances you wish to create and some properties regarding them. We only need one server instance. The rest of the properties are auto filled based on the configuration we selected in earlier steps and/or default values, and thus, should be kept as it is.
Next, click on Add storage
As the name suggests, storage refers to the amount of storage in our server. Note that this isn’t the storage you’d consider for storing databases. This is temporary storage that will last only as long as the instance lasts, and thus, can be used for things like caching. A size of 8GB, that’s part of the free tier, and is the default, suffices our purpose.
Next, we’d be adding a tag for our instance. It is a key:value pair that describes an instance. Since we only have a single instance right now, it is not very useful, but when you are working with multiple instances and instance volumes, as will be the case when the application scales, it is used to group, sort and manage these instances.
Next, we’ll be adding a security group to our instance. A SG is practically a firewall for your instance, restricting the traffic that can come in, what ports it can access, called inbound, and the traffic that can go out, called outbound. There’s further options to restrict the traffic based on IP. For instance, your application will run on port 3000, and thus, that’s a port you’d want all your users to be able to access. Compare that to a Postgres database service running on port 5432. You don’t want anyone else but you meddling with that, so you’ll restrict the IP of that port to only you.
Create a new security group. Next, we have to add the rules for the group, describing what ports are accessible to the outside world, and who they are accessible to. Note that outbound traffic has no restrictions by default, meaning that your application can send a request to anywhere without any restriction from the SG unless you choose to restrict it. As for inbound, we’ll first add HTTP on port 80 and HTTPS on port 443. Next, we’ll add an SSH rule for port 22. SSH stands for Secure Socket Shell and will allow you to connect to your instance, as we’ll soon see in the coming section. Finally, we’ll add a custom TCP rule for the port our application is going to expose - port 3000.
For simplicity, we’ll keep the sources of all of those at ‘anywhere’. Ideally, SSH should be limited only to those you want to allow to connect to your instance, but for the sake of the tutorial, we’ll keep it at anywhere.
Once the rules are set, click on Review and Launch. You’ll be shown the configurations you’ve selected to ensure you didn’t make a mistake anywhere. Once you hit launch, you’ll be asked to create/select a key pair. As the name suggests, it’s a pair of keys - one held by AWS, and the other by you, that acts as a sort of password for you to connect to your instance. Anyone wishing to SSH into this instance must have access to this key file or they won’t be able to.
The content of the file is RSA encrypted, which uniquely determines your access to the instance. Click on create new, give it a name(that you must remember), and download it.
It’s recommended that you download the .pem key file to C:/Users/Home directory on Windows( /home/usr or similar for Linux and Mac), to avoid any access issues.
Once the file is downloaded, you’ll get a prompt that your instance is starting, and after a few minutes, your instance will be started. Your EC2 home page should look like this. Note the Name : Main(tag), the Instance type t2.micro that we selected when we were setting up the instance.
Next, select the instance, and click on Connect on the top bar. It’ll open this page :
This lists a few ways in which you can connect to the instance. Go to the SSH client tab. Now, we’ll be using the terminal to connect to your instance(remote server). For that, open a new terminal as administrator(superuser or sudo for linux), and navigate to the directory where you stored the .pem key file.
First, we’ll run the chmod 400 keyfilename.pem command to allow read permission on that file, and remove all other permissions. Note that if the key file gets overwritten, you’ll lose SSH access to that instance forever, and you’ll have to recreate the instance, since you won’t get the .pem file to download again.
And once you’re done with that, it’s time for the high jump - connecting via a simple command to a remote computer thousands of miles away. The command to run will be on the AWS page as shown above - the ssh -i … one
It means that we’re ssh-ing into the instance defined by the DNS(the .amazonaws.com thing), and proof that we’re authorized to do it, is in the pem file.
It’ll ask a confirmation prompt that you have to type yes to, and if all works well, you should see a welcome to Ubuntu text as shown above, which means that you’re now logged into the instance.
Great going.
Now, our next step is to bring the code into our instance and run it. To do that, we’ll do a git clone exactly the same way we cloned the repo on our local system, using the git clone command.
Once you’re done cloning the repo, the next step is to install the dependencies and start the application. Navigate to the repo directory and try running
npm install
Did you get an error? Ofcourse you did. You need to install NodeJS on the instance. How do you do that? The answer’s in the error itself :
sudo apt install nodejs
This will take a few minutes to complete. Once it’s done, try running npm install again, and you’ll see that this time, you’re able to.
Finally, the moment of truth - run
npm run start
Once you see the application live on localhost:5000 written on the terminal, you’ll have to navigate to the server IP to check if it works.
This IP can be found from the AWS instance details - Public IPV4 address. Copy that, and paste it onto a browser tab, and add :3000 after it.
If the application did work correctly - you should be able to see the same screen that you were able to see locally on your machine.
Setting up cloudwatch#
Now that you have a working application, we'll setup Cloudwatch.
Go to the search bar and type Cloudwatch. You'll get the service option there like so
Click on it, and you'll be taken to the Cloudwatch home page. Look at the navigation tab carefully - it has options for Logs, events, metrics, dashboards and so on.
Click on the Create dashboard button, and give it a name of your choice
Next, you'll be prompted for the widget type you want to add, line graph/cumulative/alarm, etc
We'll pick the line graph. We can always add more widgets later
Next, you'll be asked where should this graph's data come from - the metrics, or the logs? We'll pick metrics since that's what we want to track
Next, you'll get a screen like this, and a list of services which you can track. Click on EC2, and use the select all button to have all EC2 metrics showing up on the widget.
Finally click Create widget, and you'll be able to see the widget on the dashboard
Similarly, you can add another widget for numeric data like so :
Finally, we'll be trying to set up an alarm.
Click on add new widget and select alarm
You'll be redirected to the alarms dashboard
Click on Create alarm
We'll be asked to select the metric on which we want to set an alarm. Search, and select CPUUtilization
You'll then be asked to specify the conditions for the alarm - we'll set the alarm when the CPUUtilization is Greater than 0.6. (It's a pretty low number, but since we wish to see the alarm triggered while not having that amount of utilization, we're keeping it this way)
You'll then be prompted to configure notifications - we choose to get notified 'in alarm', that is, when the threshold has been breached
Next, we are asked to select an SNS topic. SNS stands for Simple notification service, an AWS service used to send alerts to users. We'll click on creating a new topic, and add our email ID in the email endpoint
Click on create topic
Finally, you'll be asked to enter the name of the alarm. And then, the alarm will be created
You'll get a notification on the top stating that you'd need to verify your subscription to the SNS. Go to the email ID you'd entered in the alarm page, and you'll see a mail from AWS with a confirmation link, like this :
If you do not see it, check spam.
Once you hit the confirm link, you're then confirmed to start receiving the notification messages.
Now, go to your SSH terminal,and run the following command, to trigger the CPU usage.
sudo npm i -g pm2
Within a few seconds, you'll see the state of the alarm change to 'In alarm', and you'll have received an email from AWS with the alert
Conclusion#
Thus, in this tutorial, you understood why metrics are important, and how we can use AWS's cloudwatch service to setup and track metrics for your instances. We set up an EC2 instance, and configured cloudwatch to track metrics on it.
You can further expand on this knowledge and track metrics across your projects to drive improvements