The term 'web scale' has become very popular lately. It is one that is now thrown about willy-nilly, and refers to a site's ability to scale well. We're not talking my blog scale; we are talking Twitter and Facebook scale. If you want your architecture to scale well you are going to have to rely on caching in several places. Microsoft has just released Windows Server AppFabric which has a distributed caching component that can fill this need quite well, and it works well with Microsoft .NET Framework 3.5 and 4.
Windows Server AppFabric is a new server tool that provides two services. The first is workflow and service hosting, in addition to, and on top of IIS and WAS. The other major service is the distributed caching component. The caching component is designed to be your caching tier (which usually resides just on top of your data tier).
Using Cache in your Application
Even if you are not building the next Twitter, or feel that you have to have 'web scale' to feel up to date and modern, most likely you are already using caching in your application. Most ASP.NET applications use session state, which is a form of caching. It caches data that represents the state of the user's use of your application in the server's memory for use the next time (within a few minutes) the user returns. As your system scales, and you add web servers, this session state is often moved to another storage location, usually a SQL Server database.
There are many other ways to use cache in your application. You will likely also want to reduce the load on a backend system by reducing the number of repeated queries for read only data. For example, why should you always requery the database for a list of the states to put in a drop down? Why should you always requery the list of products from the database? Neither of the two lists tend to change too often (we haven't added a new state to the US in a while, and most companies have a fairly static list of products) and can be cached for periods of time with safety. This will reduce the load on the backend system, as well as improve the performance of the application and the responsiveness to the user.
I have seen other systems that use cache as a way to bypass or reduce the impact from a slower system. Imagine a doctor's office that needs to download the full patient record from the main office (the hospital) to have it handy when the patient is seen during their appointment. If they waited to download that data it would lead to delays in seeing and helping the patient. It would be much faster if the system could fetch these the night before based on the patient schedule for the day, and have them ready in the local cache before they are needed. This would speed up record retrieval for sure.
Installing Windows Server AppFabric
The cache server for Windows Server AppFabric needs to be installed on each server in your cache tier. You will also need to install the 'client' on each web server calling that cache tier. The easiest way to install Windows Server AppFabric is to use the Web Platform Installer. It will automatically download and install all the parts and any dependencies you may need. It is important to note that AppFabric requires .NET Framework 4 installed to run on the server, but clients can use .NET 3.5 or greater.
All of your web servers will be configured to speak to the cache servers (we discuss exactly how in a few moments). We will write our code using a very standard pattern, and one that you are probably using right now in your application. This pattern is called '
cache aside.' When we need data we will check the cache first. If it has the data, the cache will return it. If it doesn't, we will go to the data source to get the data, and then place it in the cache. This is the primary pattern that
AppFabric Cache was designed to support. You will see this relationship in the following diagram. The web servers talk to both the cache servers and the SQL servers. The cache servers do not talk to the data source themselves. They are ignorant of the data storage system.
When you install the cache server on a server be aware that you don't want to be running anything else on that server. You don't want to install this on your data server, or your web server because AppFabric Cache will grab as much memory as it can, and starve out other applications on the server. Cache Server is not meant to be installed side by side with other applications, it is meant to be installed on its own server. You can install it into a virtual server if you want.
When a client is accessing the cache server it needs several DLLs. There are two ways to install the cache client DLLs. The first is to use the Web Platform Installer on the web servers, and choose to install just the client. The second is to copy the two needed DLLs into your solution with
copy local = true. This will deploy the DLLs with your code, and make for a much easier deployment.
In the diagram above I used two cache servers as an example.
AppFabric Cache can be deployed to just one server. While this is less expensive (uses less hardware) it will not meet the usual high availability requirements many environments require. You can put as many servers into your caching tier as you think you need. They will auto-configure with each other, working out which data needs to be replicated to each server. When they do this they are using several self-healing algorithms Microsoft developed for Microsoft Azure. This means that not every piece of data in the cache will be copied to every server. Instead the Cache engine determines what the optimal layout will be, and then implements it.
The servers use three different ports on the network. The first is the port that will be used to communicate with the clients (the web servers). The second port is used by the cache servers to communicate with each other, to replicate their data. The third port is used as a heartbeat, so that the cache servers can monitor their health and status.
Using the Cache
Now that we have our servers setup we can start writing code. As we discussed earlier, the pattern you will use will be to check the cache for the data. If it is not there, then you will fetch the data yourself, and then add it to the cache.
We first need to use PowerShell to create a cache on the servers. Each cache has a name, and is its own container for data. If you look in your start menu you are likely to find a pre-configured PowerShell for Cache called "Caching Administration Windows PowerShell". You can open this up, or open up a plain PowerShell prompt and import the Cache management tools with the command "
Import-Module DistributedCacheAdministration". Now to create the cache we use the command "
New-Cache <nameofcache>". In my sample I will use "
New-Cache ProductData". This will create a cache in the cluster called
ProductData. This cache will be empty at first. But now we can use it in our code. It is important to know that the Windows Server AppFabric does not ship with an admin user interface, it is all managed through PowerShell.
Open up a web project and add these two references,
Microsoft.ApplicationServer.Caching.Core. These are the client DLLs that we mentioned earlier.
We aren't going to cover HOW to get the data from the original data source. AppFabric Cache doesn't care about where the data comes from, or what it is. The first step is to create a
DataCacheFactory object. This object will help us create the other cache objects we need to use in our code. We will then use this factory to create our myCache object. This object represents the cache we create in PowerShell above. When you call the
GetCache method on the factory you can pass in the name of the cache you want to use. If you don't provide a cache name you will receive a reference to the default cache on the server.
private DataCache myCache = null;
var myCache = CacheFactory.GetCache("ProductData");
Now we can start reading and writing to the cache. We will fetch the data from the cache. If it is not in the cache, then we will receive null, and we know we will need to fetch the data from the source, and store it in the cache for future use.
var productList = myCache["productList"] as List<Product>;
if (productList == null)
// get the data from your data source
var productsConnection = new ProductsEntities();
productList = productsConnection.Products.ToList();
//add it to the cache for next time
myCache["productList"] = productList;
You could put this right in your code behind, but I would strongly encourage you to integrate this into your data tier classes so that the use of the cache is transparent to your UI code, and is a factor of your data code. While you do this make sure you don't sprinkle references to the cache all over the place. Make sure you refactor those references out to one cache manager class that can handle those references. This will help you manage all of the related code in one place.
It is easiest to work with read only data in the cache, such as our products list. When we do add a new product we would expire and refresh the cache. Perhaps we do this once per day on the web site. AppFabric Cache Server does support locking so that you can write to the cache. Perhaps you might keep a customer's shopping cart in the cache instead of in session state. In this case you can use optimistic or pessimistic locking on the data in the cache.
What is Happening on the Server
The client and server use Windows Communication Foundation (WCF) to communicate amongst each other. By default all of these connections use both message and transport encryption for security purposes, and the NetTCP binding. This results in very secure communication between your servers, but does introduce some overhead with all of the encryption and decryption of messages. You can use the configuration files to turn this off, sending the data in the clear. Doing this will drastically increase the performance of the cache server, but I would only do this if you are 100% sure the network in use is secure.
The server will manage what data is stored on which server in a peer-to-peer type arrangement. While the client can be configured to speak to any one server (but under the covers will communicate with all of them) the server is responsible for replicating data across the servers, and updating the cache map on the client.
What is happening on the client
One step we skipped above is telling the cache client the address of the cache cluster. When we create a cluster with two or more servers we treat it as a single unit, not as individual servers. We then only have to give one of the cache servers names to the cache client to connect to. When the client first starts up it will connect to this one server and download all of the data about the cache cluster, including the other cache server names, ports, and a map of what data is located on which server. You can provide this configuration in the web.config or in code. You will usually want to provide this in the web.config, where it is more easily maintained at runtime. To do this we will have to add a config section to your web.config.
After that we can provide the cache cluster configuration. You can provide a host entry for some or all of the cache servers in your cluster, but you don't have to. The server name can be a Windows server name, or a fully qualified host name or IP address.
If you leave your configuration like this then the client will be able to access the correct server based on the cache item it is trying to find. In this mode the client does not store or cache any of your data, but does keep track of where the cluster has stored which piece of data.
The client can be configured to run with a local cache. When configured this way your local client will look in its local cache first, and then in the cluster. You cannot configure which pieces of data are stored locally and which aren't, the client works this out on its own.
You can configure the amount of data that is stored in the local cache (the local cache won't consume all of the local RAM like the server will). To activate the local cache, and configure it, add the following to the
dataCacheClient section of your web.config.
In this sample we have configured the local cache to be limited to 5,000 items. Each item will expire after 600 seconds (10 minutes). You can of course increase or decrease either of these settings to suit your needs.
Moving ASP.NET Session State to AppFabric Cache
Many people use ASP.NET Session State in their applications. I try to avoid it as much as I can because of the brittleness it adds to your load balancing and server reliability and because the API doesn't give you a lot of power over when something is removed from state.
Because of the provider model you can easily wire in a new place for the session state cache to be kept, instead of in-process RAM. Traditionally you have the SQL Server provider, and others have been released recently as well for other storage mechanisms. AppFabric Cache ships with a provider that makes it easy for you to move your session state from memory on the web server (where it is not highly available) and to the memory of the cache server.
To make this configuration change, enter the following lines to your web.config. This will require absolutely NO code change on your part, just a configuration change.
<sessionState mode="Custom" customProvider="AppFabricCacheSessionStoreProvider">
All you need to do is add this to your web.config and your site will start storing session state data in the AppFabric Cache instead of in memory. The web application will restart when you do this, and you should test it in QA first.
Caching is an important element for any application serious about performance and scaling. Windows Server AppFabric provides an easy and clean way to cache your data in a highly available and reliable way.
Don't just think of your static data for caching. Even caching data that changes often, for even a few seconds, can really improve the performance of your application. Remember that of the hundreds or thousands of simultaneous users on your application they will all be hitting the cache, so don't look at the cache from a single user's perspective.