Details

On the 6th of May the financial market moved over 1000 points and drove the highest financial news traffic day in history.  Details of the event as reported by the New York times are here:

TechCrunch said that at the peak of the sell-off, Web sites were “failing left and right.” The TechCrunch blogger MG Siegler wrote, “Google Finance keeps bringing up an error message to ‘please try again in 30 seconds.’ Yahoo Finance, meanwhile is completely down. Trying to look for the news via Twitter, meanwhile, yields mixed results. At one point when the Dow was down about 1,000, plenty of people were still tweeting that it was down 400. Others were saying it was down 600, etc. The problem is that the ‘realtime’ web wasn’t even fast enough for how fast things were crashing.”

During this unprecedented peak period CNBC.COM and its Mobile site had the highest traffic day ever:  While the rest of the industry was unable to keep up with the demand of the market for information, CNBC performed flawlessly without missing a single page view using the aiCache and Dyn.com solution.

Press Release CNBC

aiCache is proud of the hard work done by the engineering team at CNBC and superb quality of its partner Dyn.com. We look forward to continuing support as their important mission of serving the financial community continues.

The secret of caching is controlling the freshness of content.  This is managed using (TTL’s) Time To Live settings.  These settings balance the speed of delivery with the freshness of content by controlling how frequently content is generated.

Website admins struggle to balance content freshness and speed of delivery.  The challenge is that  settings that work well for normal traffic can be ineffective during peak periods.

aiCache solves this issue by allowing the TTL’s to make dynamic adjustments during peak periods, returning to normal setting when the load diminishes.

Under normal load aiCache obeys the caching TTL’s. When load increases in response to viral content, being featured in a web consolidator such as Digg, Drudgereport etc., aiCache allows you to bend the TTL’s.

aiCache implements this “bending” by allowing you to factor an increase in the TTL for a specified period.

You can tell aiCache that under peak load it should bend the TTL’s by a certain factor, for example 5, for a particular length of time.  Content that would have ordinarily refreshed in 30 seconds will now be served from aiCache for 150 seconds, when load reaches a defined threshold.

With aiCache capable of producing content at thousands of times the speed of the normal environment, this simple factoring of the TTL will allow the site to manage all additional traffic quickly, for as long as it remains under load.

aiCache analyzes response times every few seconds and reports on both the scale factor and the normalization.

The end result? With a simple one line configuration you receive the best of both worlds,  optimally fresh content for users and high speed reliable serving under peak loads.

Many an IT professional shiver at the mere mentioning of this two words: website transition. It doesn’t have to be painful. Let’s imagine that Acme Inc has its current website of news.acme.com hosted with a legacy provider, FlyByNite WebHosting Inc.

The site was developed a while back and is running a custom written CMS that is, quite frankly, growing long in the tooth. It is rather slow, lacking in features, hard to maintain and especially hard to modify. It can go down without much of a warning and doesn’t have much of monitoring in place – which results in fair bit of frustration at Acme, as helpless Acme web site crew observes, in horror, another meltdown, unable to do anything to help the situation.

Not content with status quo, Acme leadership team decides to embark onto a difficult journey – a complete redo of the site’s CMS, this time going not with a home-grown solution, but adopting one of more popular Open Source CMS.

Such endeavors are typically very labor intensive and error prone all-or-nothing approaches. In other words, after spending much time and money, you’d throw the proverbial switch and send traffic from old environment to the new one and hope it all works out.

With aiCache in the mix it doesn’t have to be such a monolithic effort. Instead, you can transition your site section by section, without any impact on your users, having complete control and confidence every step of the way.

Acme starts by placing aiCache in front of news.acme.com domain. They can now cache some content, reducing the load on the frail legacy environment, observe traffic and it’s vital statistics in real time, alert when things go sour, still keep the site up in case of complete hosting meltdowns, that are growing more frequent at FlyByNite WebHosting Inc’s datacenter.

The situation is instantly improves and there’s now some breathing room for the transition. Next, Acme sets up their own CMS instance (with most CMS, 2-3 web servers and a DB, ideally in a highly available setup). Now Acme chooses a section that they want to improve first – typically one that gets most traffic. Let’s say it is US News and it lives under news.acme.com/us_news .

Having setup the section on the new CMS, now we can tell aiCache to fill the requests for news.acme.com/us_news not from legacy, but from the new CMS – by tagging the relevant pattern and adding the new CMS web servers to the configuration.

With a simple, instantaneous aiCache configuration change, Acme can introduce the new CMS into the traffic path in an incremental, controlled, measurable and fail-safe way.

As more and more sections are developed in the new CMS, they too, are configured to be served from the new CMS. Incrementally, section by section, Acme takes over the control and hosting of news.acme.com and after a while the FlyByNite WebHosting Inc is out of the picture.

The picture below depicts the setup, with legacy CMS on the left and the new CMS on the right. You can see basic flows of traffic coming to Acme’s hosting environment and aiCache deciding where to fill the content from. A very straightforward setup indeed. We also attach a small snippet of aiCache configuration file that shows basics of using os_tag settings to accomplish the desired effect.

……
website
hostname news.acme.com

pattern /us_news simple 1m # Cache for 60 seconds
os_tag 2 # Fill from the new CMS by specifying os_tag of 2

pattern ….. # Patterns for content to be served from old CMS
pattern …..

As you may well be aware, the XMLHttpRequest object (aka XMLHTTP object in Internet Explorer) is the centerpiece of today’s AJAX-driven web applications. Many a popular site owes its rich functionality and responsive interface to the use of AJAX.

AJAX has one significant security constraint; one cannot create an AJAX request to a domain different from the one from which the Javascript itself was downloaded. For example, if one has AJAX functionality embedded in news.acme.com/ajax.js, the AJAX calls from the said JS file can only address URLs at news.acme.com and not any other domain, or even a subdomain of acme.com. You shall see that this limitation can be easily addressed by using aiCache os_tag setting.

Let’s imagine that news.acme.com has “newsmaker” page – a mash-up of data about celebrity of the day. The page displays the editorial content about the person, along with user-submitted comments and may be latest RSS-driven news about the celebrity.
We also display the search results so that users can see prior content that was published by Acme. The comments, the search results and the latest RSS data are embedded into the page client-side, using AJAX. So as new comments are posted and RSS feeds are updates, the page refreshes itself without user having to reload it.

To obtain this functionality, there’s some JS code, downloaded from news.acme.com that needs to access URLs at comments.acme.com, search.acme.com and rss.acme.com. The three subdomains we mentioned could be hosted in different environments and/or datacenters. Some could be .Net-based, some PHP-based and some developed in Java.
Some environments could be white-labeled from an external provider. No matter the technology, you can see that accessing three different subdomains presents a problem. It won’t work unless one of the two different technologies is deployed to address it:
a client side Flash proxy
(see http://blog.monstuff.com/archives/000294.html)
a server-side proxy solution
Flash proxy solution requires Flash support in the user browser.

While almost ubiquitous these days, such Flash support is not present in quite a number of mobile devices, including Android-based ones and the latest product from Apple, the iPad. From the looks of it, we can take as a given that this Flash-deficiency is not likely to get rectified any time soon.

Server-side proxy solution doesn’t suffer from such limitations and is completely browser and client-agnostic (although, clearly, for any of AJAX to work, client browser must support AJAX). Instead of trying to access the required functionality directly on the three different sub-domains, we instead request the functionality from news.acme.com and have aiCache servers to direct (relay) the requests to proper sub-domains.

So let’s provide a configuration file snippet showing how one can configure aiCache servers that accelerate news.acme.com, to transparently forward requests, based on the request URLs, to different origin servers.
We will assume that requests sent to news.acme.com/search.php?anything should be sent to search origin servers.
Likewise requests to news.acme.com/comments.do?anything should be sent to comments origin servers. Lastly, requests sent to news.acme.com/rss.apsx?anything should be sent to rss origin servers. In all three cases, we will rewrite the Host header to match the origin farm we choose.

….
website
hostname news.acme.com

pattern /search.php simple 1m # Cache for 1 minute
os_tag 2 # Send it to search servers
sub_hostname search.acme.com

pattern /comments.do simple 1m # Cache for 1 minute
os_tag 3 # Send it to comments servers
sub_hostname comments.acme.com

pattern /rss.aspx simple 1m # Cache for 1 minute
os_tag 4 # Send it to RSS servers
sub_hostname rss.acme.com

….
origin 1.1.1.1 80 # Regular origin servers for news.acme.com, no os_tag !
origin 1.1.1.2 80 # Regular origin servers for news.acme.com, no os_tag !

origin 2.2.2.2 80 2 # origin servers for search.acme.com, note os_tag of 2 !

origin 3.3.3.3 80 3 # origin servers for comments.acme.com, note os_tag of 3 !

origin 4.4.4.4 80 4 # origin servers for rss.acme.com, note os_tag of 4 !

As you can see, with a very straightforward and easy-to-manage configuration setup, we can now joggle, on the fly and completely transparently to the end users, requests amongst 4 different subdomains. All while caching responses, having real-time view of the traffic statistics, along with ability to shield end-users from any issues at the said sub-sites.
Please note that such request relaying happens completely transparently to end users, in a sense that it is aiCache that obtains the actual response, no HTTP redirects are sent to the requesting browsers (and even if they were sent, such redirect still won’t work).

When any of the sub-sites are hosted by external providers, we don’t require any involvement from them, to accomplish such setups. Also, with such hosted-API arrangements, where charges are based on volume of requests, we can save significant amount of money by caching responses.

aiCache offers support for this kind of scenarios via so called “Content Driven Caching” or CDC for short. Here’s how it works.

First, you define a cacheable pattern, just as usual. Then you add one or more of cdc_pattern pattern-level settings under the matching pattern, specifying regular expression match strings. With these specified, aiCache will analyze response bodies, looking to match the response body to a defined cdc_pattern.

Should a match be found, aiCache temporarily overrides TTL for the matching response and sets it to 0. Effectively, the matching web page is declared non-cacheable. Periodically, aiCache will attempt to match the page’s content again and should no match be found, the page will have its TTL restored back to the one specified by the pattern. You can control how frequently such-rechecking is done by setting cdc_interval pattern level setting, it defaults to 5 seconds.

website www.acmenews.com
…
pattern breakingnews simple 10
cdc_pattern acme\spoll
cdc_pattern acme\ssurvey

In the example above, we match response’s body (content) to see if contains “acme poll” or “acme survey” in it. Should a match be found, aiCache will declare the page non-cacheable and no longer serve it out of cache. Sometimes, you might find it easier to match for auxiliary content URLs instead. For example, you might know that every time a poll is published on a page, the poll’s Javascript is included into the page so you can look for that Javascript URL instead. For example:

cdc_pattern userpoll.js
cdc_pattern usercomment.js
Matching for such JS “includes” might be less resource intensive, as they are often located at the very beginning of the page’s HTML, so aiCache can find the match faster.

Let’s consider another situation. Acmenews’s editorial team might decide to publish polls or enable comments, in any of the following pages: usnews.jsp, worlnews.jsp, marketnews.asp and cenews.asp. As you can see, all of these URLs have a common “news.jsp” component to them , so you can create a single pattern to cover all of them:

website www.acmenews.com
…
pattern news.jsp simple 10
cdc_pattern userpoll.js
cdc_pattern usercomment.js

Now all of the mentioned pages will have their content analyzed for CDC-overrides, independently of each other. In other words, usnews.jsp might end up in CDC-override state, while worldnews.jsp is still served from cache.

The handling of page with the CDC-overridden TTL is no different from the way 0TTL requests are normally handled by aicache. Specifically, all of the cookies are sent both to OS and back to the requesting client.

When page is in “regular”, cacheable state, aiCache only attempts to match the response body against the CDC patterns when the page is refreshed, so we recommend you keep TTL for such pages low enough, so aiCache can detect the change in the page’s content quickly enough. As mentioned earlier, when in “CDC-override” state, the matches are performed every cdc_interval seconds – every 5 seconds by default.

When aiCache needs to perform content matching for CDC patterns, it requests the response in plain, non-compressed form. This way no CPU cycles need to be spent by OS to compress the response and by aiCache to uncompress it, before matching could be performed. The response will be compressed by aiCache, in accordance with on-the-fly compression settings and client browser indicating support for compression

aiCache solves the industry issue of slow SSL speeds and the need for expensive dedicated hardware, by bringing high speed SSL termination to the Cloud and Data Centers, on common hardware.

This is a special benefit for users of cloud and virtualization technology. Virtualization solutions such as Amazon Web Services do not allow dedicated hardware. Cloud customers have been without options for handling large scale SSL implementations until now. aiCache removes the SSL barrier by running in virtualized instances, providing dedicated hardware level performance.

Version Six supports over 250,000 transactions per second for non-encrypted content and over 30,000 RSA1024 SSL sessions per second, all running on a single server.

aiCache manages HTTPS and HTTP traffic, relieving web servers from the overhead of encryption,, while providing a superior customer experience and delivering full complement of industry-leading aiCache Web Application Acceleration features.

The new release fully supports earlier configuration file syntax, allowing existing customers to simply drop-in a replacement binary.

Customers in the Cloud, with Amazon Web Services and Rightscale, will have the Version Six capabilities available at no additional cost.

Normally, accomplishing this requires writing custom server-side code that would receive the error notification and execute some form of reporting, logging and alerting function. However, outside of having to write such code, you might face another danger when following down this path: when in error state, your site will be receiving large number of error message and that in itself, can cause further deterioration of service which would result in even more of error conditions being reported. In just few quick seconds you whole site just might melt down.

aiCache offers an easy way to accomplish the required functionality, without requiring any custom code to be written or any code to be executed by your web, application or database servers. It is aiCache that receives, logs and alerts on error condition, without ever forwarding any of the traffic to the rest of your infrastructure.

Reporting using a dedicated error reporting website.

To accomplish this, setup a dummy website in aiCache, for example error.acmenews.com . Identify the site as dummy website via dummy_website website-level configuration directive. You don’t need to specify any origin server for websites defined as dummy, as no requests ever are sent to origin servers for such websites.

Setup the site to alert on when number of requests exceed the desired threshold – for example set alert_req_sec_max to alert when this website receives more than 2 requests per second.

Setup the client side code to execute a simple HTTP GET against this website, passing arbitrary URL and parameters, as part of the request. For example, to report a problem with your News API, you might request http://error.acme.com/app=newsapi&errorcode=123&client=premier . It is up to you just what parameters you want to pass via such a request.

Now, when an error condition is detected, a request is made to the specified URL. aiCache receives the request and logs it. When number of such requests exceed the configurable threshold, an alert is sent out. So there you have it: without writing any of the server side logic and without placing any extra load on your infrastructure, you now can receive, log and alert on error conditions – all courtesy of aiCache.

It doesn’t have to be only the client-side logic that sends such error-reporting requests. You can use method of your choosing to execute such requests – including server-side code, custom script etc. You can also redirect clients to the error-reporting URL in case of a problem that you discover when executing some logic. Again, you’re only limited by your imagination in how this functionality can be deployed.

You can train your personnel to look at the aiCache log file for such error reporting web sites, when the error condition is reported and alerted by aiCache, to see the additional information that is embedded inside of the error reporting URLs (such as application, error code and client information in the example above).

To assist in rapidly detecting the error condition, aiCache lights up the respective web site name in red, in its self-refreshing web monitor screen, when it detect that RPS for a dummy website exceeds the set limit. This is in addition to regular email alerting on the error condition – which should be configured as per instructions in Automated Alerting and Monitoring section of this manual. Specifically, you must provide the email address to send the alert to, at either or both global and website levels.

Reporting using pattern attribute.

You can set a bad_response pattern flag. When aiCache matches a request to such pattern, the response is reported as bad response (same as say 5xx response code) – so you can monitor for such bad responses, log and alert on them.

Here’s an example explaining how you can use this feature. Some sites have dedicated pages setup for common errors – such as PNF (page-not-found) and similar. So let’s imagine there is such a dedicated error page called 404error.html . The page is requested/redirected to in response to assorted error conditions, as detected by assorted server and client side code.

By setting bad_response flag for matching pattern, you will configure aiCache to report and optionally, alert on, whenever this page is requested, providing for 0-cost monitoring and alerting of this error condition.

Please note that setting of bad_response flag doesn’t affect anything else in terms of request/response handling logic, its sole purpose is for reporting and optional alerting of matching traffic.

aiCache provides an elegant solution to this with pre-fetch technology.

aiCache allows you to configure a set of slower URLs to pre-load. It then pre-fetches and actively maintain a queue of fresh responses When these are requested, aiCache instantly serves the pre-fetched response.

By tailoring pre-fetch settings you can optimize the site so all responses are pre-fetched. aiCache collects and reports on pre-fetch statistics simplifying this task.

Each second, aiCache analyzes the queues of the advertising URL’s. When a queue contains too few responses, aiCache requests additional ads to maintain the optimum number.  When a request comes for any advertising URL, aiCache removes a response from appropriate queue and sends it to the browser. The result is the website never waits for an advertising response.

aiCache provides a few seconds of queuing, ensuring an accurate set of statistics for served ads allowing accurate fast serving of content that scales independent of the performance of the advertising API.

A site serving mobile clients might serve responses whose formatting depends on exact mobile device/browser. To accommodate for such clients, while allowing for caching of responses, we must use User Agent information as part of cached response signature.

When a URL named “news.html” is accessed by 3 different mobile devices, we shall have 3 different responses cached – each containing mobile device’s User Agent string.

Using this feature has the potential to significantly increase the size of response cache (as multiple versions of same URLs are now cached). To solve this problem aiCache provides:

Using reduced/rewritten User Agent string in signature of cacheable responses.
This feature is only available in mobile-enabled edition of aiCache.

Related to the previous feature, this allows you to rewrite/reduce User-Agent strings to a smaller subset and use the rewritten/reduced values as part of cacheable response signature.

The problem this feature addresses has to do with a great variety of mobile devices currently available. Every firmware revision, different mobile providers/carriers/market all result in a different User-Agent strings sent by mobile device, making it a challenge to accommodate for all of these devices.

The great majority of mobile devices on the market can be reduced to about a dozen distinctly different devices (device families). So Blackberries can be grouped into may be 2 different sets, Iphones’s into its own set and so on, based on capabilities, support for Javascript and available screen sizes.

aiCache allows you to accomplish just that. You specify rewrite rules for User-Agent strings. The rewritten/reduced User-Agent strings are then used as part of signature of cached responses.

The same “compressed” User-Agent string is also forwarded as X-UA-Rewrite header in requests sent to origin servers. The origin servers (server-side code) can then programmatically access and act on this header value, with a goal of modifying responses to accommodate for mobile device differences.

-For example resorting to Javascript-free versions of content for devices that don’t support Javascript, resizing the images to accommodate for different screen sizes.

The server-side code can also tailor the responses based on the actual value of User-Agent string (aiCache never modifies it, it is forwarded verbatim from requesting device to the origin servers) – but then the server-side code has to accommodate for a much larger variety of devices.

Reducing the large number of different User-Agent strings to a much smaller subset also has a positive impact on caching of responses – allowing to achieve much higher cache hit ratios and to proportionally reduce the traffic and demands on origin server infrastructure (Web Servers, DB servers etc).

It also allows to greatly simplify the logic required to handle the variety of mobile devices available on the market today.

It is most desirable to accommodate for different mobile devices in a fashion that doesn’t require changing the URLs – so that no matter what mobile device is being used, News page is always accessed as /news.html, Sports section is always /sports.html.

The normal solution where URLs change, say by prefixing every URL with a device type, to accommodate for the device type is confusing to the customer.

aiCache provides a manageable elegant solution to these issues.