Server cost is usually not a concern for most funded startups, but for a boot strapped SaaS product like ours, it was important to have an AWS bill that is easy on the pocket and a little more proportional to the MRR.
To that end when we started superlemon.xyz one of the first things I did was to find ways to consume the least amount of resources on the cloud. We currently serve a traffic of ~ 250 requests per second with our AWS setup.
Most applications require certain common cloud resources and these include
- Compute instances
- Database instances
- Caching instances
- CDN
- A web server that acts as a reverse proxy and load balancer
I will now go through each of these resources and talk about both the expensive way and the cheap way to implement them
Compute instances
When it comes to compute instances, most people go with AWS EC2 instances. EC2 instances are the safest choice to make for running server applications as they are highly configurable, scalable and you can change the configuration on demand according to your needs. However, sometimes you do not really need this level of control on your compute instances and that brings us to AWS Lightsail.
Lightsail as the name suggests is a lightweight version of EC2. Under the hood Lightsail instances are actually EC2 instances, but this is not apparent to the user and unlike the highly configurable EC2, Lightsail comes with a fixed configuration that once you provision cannot be changed. The billing is also a fixed amount per month as opposed to EC2 which is billed by the hour.
To give you a sense of how much less complicated Lightsail is, please take a look at these screenshots of the EC2 dashboard and Lightsail dashboard.
But we are not here to debate the complexity of EC2 vs Lightsail, so let’s let us talk about the cost.
An EC2 instance with 2 virtual cores, 4GB RAM and a storage of 80GB costs roughly 37$ a month and a Lightsail instance with the exact same configuration costs 20$ a month which is almost half the cost!
The only drawback here as previously mentioned is that the instance is fixed and neither the storage nor the compute power can be tweaked later according to spikes in traffic and usage.
In our case we do not have a need for too much storage on the compute instance, and as for the computing power it was easy for us to simply provision another Lightsail instance when there is an increase in traffic and set it up behind a load balancer. This way our system is still scalable.
Database instances
Choosing a database provider for your application can be a tricky decision, but on a high level there is just one thing that is absolutely required for any database provider - the ability to take regular backups of the DB automatically and the ability to restore the database from one of the backups.
All of this functionality is provided by AWS RDS along with an array of other capabilities like autoscaling of storage, multiple availability zones, etc. However we did not not really need this level of control over the DB for our simple SaaS product, not to mention the fact that RDS would cost us a minimum of 200$ a month with the lowest acceptable configuration.
Once again our saviour was Lightsail which provides managed Databases with a fixed storage at very cheap prices. Only MySQL and PostgreSQL are available though which was fine with us since I am quite comfortable with MySQL. We have currently provisioned one MySQL DB with 2 vCPUS, 4GB RAM and 120GB SSD and it costs us 60$ a month.
And as mentioned before Lightsail provides the basic capabilities of backups and restoration.
The drawback here is that the DB will not scale automatically so your will have to make sure that your application does not use storage beyond what is available in the instance you select. We do this by regularly purging our Database of data that is older than X days and data that belongs to users who churned from our app more than X days ago and haven’t come back since.
Caching instances
For our application we needed a Caching layer as well as the ability to queue up jobs that can be executed asynchronously. The seasoned folks here might have realised that the best tool to use for this is Redis and AWS has a service called ElastiCache which is Redis under the hood.
Once again this would be a very safe choice to make because it is completely managed and scalable. Our need was a Redis instance with at least 6GB of memory and 2vCPUs and if we went with ElastiCache our cost would roughly come out to be 112$ a month, not to mention the additional cost of running our async workers somewhere else.
I might sound like a broken record at this point but what we ended up doing was to provision a Lightsail instance with 2vCPUs and 8GB of memory for the cost of 40$ a month and installed an open source version of Redis on this instance. We also use the same instance to run our asynchronous workers which read from the Redis queue and execute jobs.
So what is the drawback? Well since it is not a managed service, you would have to monitor the Redis server yourself. There are many tools out there that help you do this and the one I would recommend is prometheus.io.
This means that you have to do a little bit of extra work to setup metrics collection from your Redis instance and use a grafana dashboard where you can view these metrics, but this extra work saves a lot of money in the long run and you get to look at a super cool dashboard like this one
CDN
Our application requires a javascript file to be loaded into the websites of our customers. This JS file gets a ton of traffic because this traffic scales according to how many visitors our customers get. Now this can be scary because it means we really had no idea how many requests the JS file might actually end up receiving so we hesitated to go with CloudFront which is the goto solution for a CDN on AWS.
So we ended up taking an entirely different approach for this. Our application is a Shopify app and during the process of building the application we created a Shopify store. Every Shopify store gets its own personal CDN where you can manually upload anything and it will be served over the Shopify CDN. So we minified and uploaded our JS file to the CDN of our Shopify store and now we serve 20000 Shopify stores using this method at zero cost.
There is a glaring drawback to this approach. The Shopify store CDN does not provide an API that you can use to programmatically upload files and it has to be done manually. Also unlike CloudFront, there is no option of invalidating a file once it is uploaded. So If you have to make updates to your file you would have to upload a new file and migrate all of your existing users to this new file.
This might sound like a big drawback, but it actually took me an hour to whip up a script that updates the JS file for all our existing 20000 stores to the updated JS file that I provide. So whenever I make a new release to this JS file, I simply upload a new file manually to the CDN, then take that file as the input and run this script and we re live! This approach works for us because we do not make too many releases to the JS file to begin with.
Web server + load balancer
Every application requires the ability to route requests coming to their domain or subdomain to various applications running under the hood. The best way to do this is to use AWS ELB which can be used to automatically distribute incoming application traffic across multiple targets, such as Amazon EC2 instances, containers, IP addresses, and Lambda functions.
This sounds quite fancy, but all we needed for our product was 4 things
- Serving static files for our application dashboard
- Distribute traffic among different Lightsail instances based on the path of the incoming URL
- Load balance traffic that comes from our Merchants Shopify stores amongst N identical Lightsail instances.
- Rate limiting of requests to prevent DDOS attacks.
All of this can be done using NGINX which is a great piece of software and is quite robust even with the default settings that it comes with.
So we simply installed a stable version of NGINX on one of our existing Lightsail instances which was already being used to host one of our server applications. We also use amplify.nginx.com for monitoring it. This setup is pretty much equivalent to using a managed service at zero cost.
Summary
- Use lightsail instances (20$ per instance) instead of EC2 instances (37$ per instance)
- Use a lightsail database (60$ per DB) instead of RDS (200$ per DB)
- Use a self hosted redis server on a compute instance (40$) instead of ElastiCache (112$)
- If feasible, use a free CDN (cost savings depends on traffic size)
- Use a self hosted NGINX server (20$ fixed cost) instead of ELB (cost depends on traffic and usage)
Closing notes
I would like to put emphasis on the fact that we are a micro-SaaS product that solves a small and specific use case and therefore this kind of AWS setup worked for us. This may not work for big organisations or products where the traffic is erratic.
This setup will also not work for folks who have a ton of stuff to do already and would prefer to use managed services and not take the additional headache of monitoring, maintaining and provisioning hardware resources on a regular basis because this has a time cost to it.
We are a team of 2 people with a product that is not computation heavy and has cloud requirements that are quite straightforward. We have been running this product for a little over 1 year with this AWS setup and so far we have not encountered any problems.
from Hacker News https://ift.tt/3g9Fjgn
No comments:
Post a Comment
Note: Only a member of this blog may post a comment.