What are these ephemeral drives on my VM?
Ephemeral drives are a bit of a hot topic when it comes to computing in the cloud. Many larger companies which maintain their own VM infrastructure have even come around to the wonders of ephemeral storage – VMWare for instance even has a dedicated local ephemeral swap feature.
We’ll start at the top by first explaining what ephemeral storage is (bear with me here, it’s not a long story). In a virtual or cloud computing, it almost always makes sense to use shared (SAN) storage for any and all data storage. It’s relatively cheap, centrally managed and generally performs as needed. This shared storage is usually accessed via a network such as iSCSI or over Fibre-Channel – but either way it’s considered non-local storage and is just not good at satisfying chatty IO.
Furthermore, many types of IO are not even necessary or desirable to persist long-term, so why write them to permanent storage over your already busy storage network channels?
These facts beg the question: why not just use a local storage for these temporary IOs? Bingo! That’s exactly what ephemeral drives are all about! They are very simply disks that are local to a VM host and exposed directly to the virtual machines for your use.
Two thigs should be well understood about ephemeral drives. (1) Any data written to these disks will be lost once a virtual machine has been power-cycled. It is therefore imperative that you do not use these disk for anything which you want to keep! (2) The main steam cloud providers offer “free SSD ephemeral disks” but they DO throttle the IO to these disk based on your selected instance size. This is not a big deal really, just benchmark the disk using one of the many disk benchmarking tools available on the internet (notably Crystal Disk Mark) to be sure it will perform as needed.
When properly utilized, these disks will not only allow you to perform better – they will also save you recurring revenue by allowing you to use less permanent storage, less metered IO bandwidth and select smaller VM instance sizes since you will not need as many disks (that one is Azure specific).
My experience has led me to four distinct use cases for these handy high-performance disks which I’ll touch on in later postings.
Windows OS swap-files.
SQL Server tempdb files.
Temporary ISO or zip file extractions for installations (be careful of reboots)
Webservers which perform file manipulation.