When running your databases on an Amazon server instances, the disks on which your databases run are what Amazon calls “Elastic Block Storage” (EBS) volumes. EBS volumes are networked attached (NAS) disks. This means that data moved to and from the disks goes through the network interface card along with all the network traffic. This is currently the main bottleneck to be aware of when running SQL Server on AWS instances.
Most AWS instances, except for the higher end ones, have a maximum of 1 Gigabit connection per host. Note that I said host, this means that if you are sharing the host with other virtual instances, which is very likely, then the performance of your databases could be seriously hampered because all the data and network traffic of the guest instances running on the host are going through that single Gigabit network pipe, which you are sharing with other people.
To get an idea of what type of disk performance could be achieved on a fairly beefy AWS instance (High-Memory Quadruple Extra Large) I ran some tests using SQLIO. The purpose of the test was to get some numbers that I could compare to production servers running on a SAN and to find out what type of disk configuration is optimal for SQL Server running on AWS.
The tests were run on a server with the following specifications:
Generally, I found that, read performance was fairly close between mirrored, striped, and basic disk configurations. Generally, mirrored disks were faster but there was not much of a difference. Write performance was much faster on basic disk configuration.
- OS: Windows 2003
- Disk partition offset: 1024
- EC2 instance type: High-Memory Quadruple Extra Large Instance
- Size of test file: 60GB
I tested three different disk configurations. Note that all RAID was set as software RAID within windows:
- Disk U: Basic partition
- Disk S: Dynamic striped disk
- Disk X: Dynamic mirrored disk
Here are the numbers that were achieved for basic disk configuration. For comparison purposes, I have included numbers from a recent test on a virtual instance running on the SAN of a Hardware as a Service (HaaS) provider:
A typical OLTP workload consists of small writes and reads of data. With one IO outstanding, we can see that the HaaS’s instance outperforms the AWS by double the IOs per second for reads and gets a random write speed of about 1/3rd more IOs per second.
Now let’s have a look at a typical workload that you may see in a data warehouse or OLAP application. A typical write that may happen on a data warehouse is a bulk insert of data:
We can see that up to a certain point, the AWS instance keeps up fairly well, but once we start putting more stress on the EBS volumes, starting at about 4 IOs outstanding, the HaaS instance outperforms the AWS instance by about double.
Finally, here is a typical read operation that would happen in a data warehouse, reading a large amount of data:
We can see that the HaaS instance scales quite well and would probably have continued doing so as we increased the load beyond 16 IOs outstanding. The AWS instance seems to max out at just around 100MB per second however.
To summarize, it is clear that AWS instances do not come close to the disk performance that can be achieved on a SQL Server running on traditional corporate level infrastructure. This is quite understandable if we consider the underlying disk infrastructure of an AWS instance which consists of networked attached storage that is shared by all guests on the host server. In a next blog post I will go over some of the steps that you can take to get the most of your AWS instance disks.