aiCache Web Application Acceleration Blog

In the last 12 months CNBC.COM deployed aiCache for their main financial and mobile websites.  In December  they moved the to our Dynamic Cloud Cache (DCC’s), a solution based on aiCache instances with  geo-routing by Dyn.com.

Details

On the 6th of May the financial market moved over 1000 points and drove the highest financial news traffic day in history.  Details of the event as reported by the New York times are here:

TechCrunch said that at the peak of the sell-off, Web sites were “failing left and right.” The TechCrunch blogger MG Siegler wrote, “Google Finance keeps bringing up an error message to ‘please try again in 30 seconds.’ Yahoo Finance, meanwhile is completely down. Trying to look for the news via Twitter, meanwhile, yields mixed results. At one point when the Dow was down about 1,000, plenty of people were still tweeting that it was down 400. Others were saying it was down 600, etc. The problem is that the ‘realtime’ web wasn’t even fast enough for how fast things were crashing.”

During this unprecedented peak period CNBC.COM and its Mobile site had the highest traffic day ever:  While the rest of the industry was unable to keep up with the demand of the market for information, CNBC performed flawlessly without missing a single page view using the aiCache and Dyn.com solution.

Press Release CNBC

aiCache is proud of the hard work done by the engineering team at CNBC and superb quality of its partner Dyn.com. We look forward to continuing support as their important mission of serving the financial community continues.

The secret of caching is controlling the freshness of content.  This is managed using (TTL’s) Time To Live settings.  These settings balance the speed of delivery with the freshness of content by controlling how frequently content is generated.

Website admins struggle to balance content freshness and speed of delivery.  The challenge is that  settings that work well for normal traffic can be ineffective during peak periods.

aiCache solves this issue by allowing the TTL’s to make dynamic adjustments during peak periods, returning to normal setting when the load diminishes.

Under normal load aiCache obeys the caching TTL’s. When load increases in response to viral content, being featured in a web consolidator such as Digg, Drudgereport etc., aiCache allows you to bend the TTL’s.

aiCache implements this “bending” by allowing you to factor an increase in the TTL for a specified period.

You can tell aiCache that under peak load it should bend the TTL’s by a certain factor, for example 5, for a particular length of time.  Content that would have ordinarily refreshed in 30 seconds will now be served from aiCache for 150 seconds, when load reaches a defined threshold.

With aiCache capable of producing content at thousands of times the speed of the normal environment, this simple factoring of the TTL will allow the site to manage all additional traffic quickly, for as long as it remains under load.

aiCache analyzes response times every few seconds and reports on both the scale factor and the normalization.

The end result? With a simple one line configuration you receive the best of both worlds,  optimally fresh content for users and high speed reliable serving under peak loads.

aiCache’s pattern and origin server tagging (os_tag directive) is ideally suited for easing the typical chores of website transitions.

Many an IT professional shiver at the mere mentioning of this two words: website transition. It doesn’t have to be painful. Let’s imagine that Acme Inc has its current website of news.acme.com hosted with a legacy provider, FlyByNite WebHosting Inc.

The site was developed a while back and is running a custom written CMS that is, quite frankly, growing long in the tooth. It is rather slow, lacking in features, hard to maintain and especially hard to modify. It can go down without much of a warning and doesn’t have much of monitoring in place – which results in fair bit of frustration at Acme, as helpless Acme web site crew observes, in horror, another meltdown, unable to do anything to help the situation.

Not content with status quo, Acme leadership team decides to embark onto a difficult journey – a complete redo of the site’s CMS, this time going not with a home-grown solution, but adopting one of more popular Open Source CMS.

Such endeavors are typically very labor intensive and error prone all-or-nothing approaches. In other words, after spending much time and money, you’d throw the proverbial switch and send traffic from old environment to the new one and hope it all works out.

With aiCache in the mix it doesn’t have to be such a monolithic effort. Instead, you can transition your site section by section, without any impact on your users, having complete control and confidence every step of the way.

Acme starts by placing aiCache in front of news.acme.com domain. They can now cache some content, reducing the load on the frail legacy environment, observe traffic and it’s vital statistics in real time, alert when things go sour, still keep the site up in case of complete hosting meltdowns, that are growing more frequent at FlyByNite WebHosting Inc’s datacenter.

The situation is instantly improves and there’s now some breathing room for the transition. Next, Acme sets up their own CMS instance (with most CMS, 2-3 web servers and a DB, ideally in a highly available setup). Now Acme chooses a section that they want to improve first – typically one that gets most traffic. Let’s say it is US News and it lives under news.acme.com/us_news .

Having setup the section on the new CMS, now we can tell aiCache to fill the requests for news.acme.com/us_news not from legacy, but from the new CMS – by tagging the relevant pattern and adding the new CMS web servers to the configuration.

With a simple, instantaneous aiCache configuration change, Acme can introduce the new CMS into the traffic path in an incremental, controlled, measurable and fail-safe way.

As more and more sections are developed in the new CMS, they too, are configured to be served from the new CMS. Incrementally, section by section, Acme takes over the control and hosting of news.acme.com and after a while the FlyByNite WebHosting Inc is out of the picture.

The picture below depicts the setup, with legacy CMS on the left and the new CMS on the right. You can see basic flows of traffic coming to Acme’s hosting environment and aiCache deciding where to fill the content from. A very straightforward setup indeed. We also attach a small snippet of aiCache configuration file that shows basics of using os_tag settings to accomplish the desired effect.

……
website
hostname news.acme.com

pattern /us_news simple 1m # Cache for 60 seconds
os_tag 2 # Fill from the new CMS by specifying os_tag of 2

pattern ….. # Patterns for content to be served from old CMS
pattern …..

aiCache’s pattern and origin server tagging (os_tag directive) is ideally suited for addressing the well known limitation of AJAX cross-domain scripting.

As you may well be aware, the XMLHttpRequest object (aka XMLHTTP object in Internet Explorer) is the centerpiece of today’s AJAX-driven web applications. Many a popular site owes its rich functionality and responsive interface to the use of AJAX.

AJAX has one significant security constraint; one cannot create an AJAX request to a domain different from the one from which the Javascript itself was downloaded. For example, if one has AJAX functionality embedded in news.acme.com/ajax.js, the AJAX calls from the said JS file can only address URLs at news.acme.com and not any other domain, or even a subdomain of acme.com. You shall see that this limitation can be easily addressed by using aiCache os_tag setting.

Let’s imagine that news.acme.com has “newsmaker” page – a mash-up of data about celebrity of the day. The page displays the editorial content about the person, along with user-submitted comments and may be latest RSS-driven news about the celebrity.
We also display the search results so that users can see prior content that was published by Acme. The comments, the search results and the latest RSS data are embedded into the page client-side, using AJAX. So as new comments are posted and RSS feeds are updates, the page refreshes itself without user having to reload it.

To obtain this functionality, there’s some JS code, downloaded from news.acme.com that needs to access URLs at comments.acme.com, search.acme.com and rss.acme.com. The three subdomains we mentioned could be hosted in different environments and/or datacenters. Some could be .Net-based, some PHP-based and some developed in Java.
Some environments could be white-labeled from an external provider. No matter the technology, you can see that accessing three different subdomains presents a problem. It won’t work unless one of the two different technologies is deployed to address it:
a client side Flash proxy
(see http://blog.monstuff.com/archives/000294.html)
a server-side proxy solution
Flash proxy solution requires Flash support in the user browser.

While almost ubiquitous these days, such Flash support is not present in quite a number of mobile devices, including Android-based ones and the latest product from Apple, the iPad. From the looks of it, we can take as a given that this Flash-deficiency is not likely to get rectified any time soon.

Server-side proxy solution doesn’t suffer from such limitations and is completely browser and client-agnostic (although, clearly, for any of AJAX to work, client browser must support AJAX). Instead of trying to access the required functionality directly on the three different sub-domains, we instead request the functionality from news.acme.com and have aiCache servers to direct (relay) the requests to proper sub-domains.

So let’s provide a configuration file snippet showing how one can configure aiCache servers that accelerate news.acme.com, to transparently forward requests, based on the request URLs, to different origin servers.
We will assume that requests sent to news.acme.com/search.php?anything should be sent to search origin servers.
Likewise requests to news.acme.com/comments.do?anything should be sent to comments origin servers. Lastly, requests sent to news.acme.com/rss.apsx?anything should be sent to rss origin servers. In all three cases, we will rewrite the Host header to match the origin farm we choose.

….
website
hostname news.acme.com

pattern /search.php simple 1m # Cache for 1 minute
os_tag 2 # Send it to search servers
sub_hostname search.acme.com

pattern /comments.do simple 1m # Cache for 1 minute
os_tag 3 # Send it to comments servers
sub_hostname comments.acme.com

pattern /rss.aspx simple 1m # Cache for 1 minute
os_tag 4 # Send it to RSS servers
sub_hostname rss.acme.com

….
origin 1.1.1.1 80 # Regular origin servers for news.acme.com, no os_tag !
origin 1.1.1.2 80 # Regular origin servers for news.acme.com, no os_tag !

origin 2.2.2.2 80 2 # origin servers for search.acme.com, note os_tag of 2 !

origin 3.3.3.3 80 3 # origin servers for comments.acme.com, note os_tag of 3 !

origin 4.4.4.4 80 4 # origin servers for rss.acme.com, note os_tag of 4 !

As you can see, with a very straightforward and easy-to-manage configuration setup, we can now joggle, on the fly and completely transparently to the end users, requests amongst 4 different subdomains. All while caching responses, having real-time view of the traffic statistics, along with ability to shield end-users from any issues at the said sub-sites.
Please note that such request relaying happens completely transparently to end users, in a sense that it is aiCache that obtains the actual response, no HTTP redirects are sent to the requesting browsers (and even if they were sent, such redirect still won’t work).

When any of the sub-sites are hosted by external providers, we don’t require any involvement from them, to accomplish such setups. Also, with such hosted-API arrangements, where charges are based on volume of requests, we can save significant amount of money by caching responses.

aiCache makes it possible to control whether or not a web page is cacheable, based on page’s (response) content. For example, www.acmenews.com/breakingnews.aspx web page is normally cached for 10 seconds, unless editorial team decides to publish a survey (poll) on that page, in which case you cannot cache the page for as long as the poll is active. As soon as poll is removed from the page, you can restore the caching back to 10 second.

aiCache offers support for this kind of scenarios via so called “Content Driven Caching” or CDC for short. Here’s how it works.

First, you define a cacheable pattern, just as usual. Then you add one or more of cdc_pattern pattern-level settings under the matching pattern, specifying regular expression match strings. With these specified, aiCache will analyze response bodies, looking to match the response body to a defined cdc_pattern.

Should a match be found, aiCache temporarily overrides TTL for the matching response and sets it to 0. Effectively, the matching web page is declared non-cacheable. Periodically, aiCache will attempt to match the page’s content again and should no match be found, the page will have its TTL restored back to the one specified by the pattern. You can control how frequently such-rechecking is done by setting cdc_interval pattern level setting, it defaults to 5 seconds.

website www.acmenews.com
…
pattern breakingnews simple 10
cdc_pattern acme\spoll
cdc_pattern acme\ssurvey

In the example above, we match response’s body (content) to see if contains “acme poll” or “acme survey” in it. Should a match be found, aiCache will declare the page non-cacheable and no longer serve it out of cache. Sometimes, you might find it easier to match for auxiliary content URLs instead. For example, you might know that every time a poll is published on a page, the poll’s Javascript is included into the page so you can look for that Javascript URL instead. For example:

cdc_pattern userpoll.js
cdc_pattern usercomment.js
Matching for such JS “includes” might be less resource intensive, as they are often located at the very beginning of the page’s HTML, so aiCache can find the match faster.

Let’s consider another situation. Acmenews’s editorial team might decide to publish polls or enable comments, in any of the following pages: usnews.jsp, worlnews.jsp, marketnews.asp and cenews.asp. As you can see, all of these URLs have a common “news.jsp” component to them , so you can create a single pattern to cover all of them:

website www.acmenews.com
…
pattern news.jsp simple 10
cdc_pattern userpoll.js
cdc_pattern usercomment.js

Now all of the mentioned pages will have their content analyzed for CDC-overrides, independently of each other. In other words, usnews.jsp might end up in CDC-override state, while worldnews.jsp is still served from cache.

The handling of page with the CDC-overridden TTL is no different from the way 0TTL requests are normally handled by aicache. Specifically, all of the cookies are sent both to OS and back to the requesting client.

When page is in “regular”, cacheable state, aiCache only attempts to match the response body against the CDC patterns when the page is refreshed, so we recommend you keep TTL for such pages low enough, so aiCache can detect the change in the page’s content quickly enough. As mentioned earlier, when in “CDC-override” state, the matches are performed every cdc_interval seconds – every 5 seconds by default.

When aiCache needs to perform content matching for CDC patterns, it requests the response in plain, non-compressed form. This way no CPU cycles need to be spent by OS to compress the response and by aiCache to uncompress it, before matching could be performed. The response will be compressed by aiCache, in accordance with on-the-fly compression settings and client browser indicating support for compression