The inclusion of these smaller instance sizes substantially lowers the cost for smaller sites to use aiCache technology. We have already deployed the instances through the Devpay program and they will be accesible on March 24th, 2012.

These sizes will open the doors for sites needing a smaller CPU and memory footprint to access aiCache on Amazon EC2. Since deploying on AWS we have repeatedly heard from smaller customer that the Large instance size was a barrier to utilizing the Dynamic Caching. This should resolve that issue.

aiCache continues to maintain consistency in pricing across the AWS offering. Our pricing will remain 2x the cost of the base AWS image.

aiCache is a single ascii text configuration file. This means that moving between instance sizes as your requirements change is as easy as transferring a single file to a new instance and moving the Elastic IP. This can be done on the fly without interruption to ongoing operations. We are also providing a new configuration, test, deployment and monitoring tool online.

We are very pleased to grow our Amazon offering and super psyched to see smaller customer enabled with the new instance types. We will continue to offer our two free hours of setup and deployment at no cost regardless of instance size.

We think it would be nice if your DoS protection could understand when an attack starts and then implement protection immediately.

So Introducing auto-activation and notification of our IP throttling technology.

This allows you to set thresholds such as Request-Per-Second that would indicate a DoS attack is in progress. When these thresholds are exceeded aiCache will auto-implement IP throttling and send a notification indicating you are under attack.

No worries even if your thresholds are exceeded by normal traffic spikes, aiCache will not effect your performance. In fact we like to think of ourselves as the only DoS protection that radically increases the speed of your site.

So if you are unfamiliar with our four level dos Protection please see our overview:

Or for the more detailed tech view grab DoS in the index of our Admin guide.

Below you can see a more techy view of how the auto-implementation is configured.

You can tell aiCache to auto-activate the IP throttling by configuring the following server-level
settings:

auto_throttle_cps and
auto_throttle_cps_interval.

Effectively, you’re telling aiCache: activate throttling when number of client connections per second exceeds the auto_throttle_cps , activate it for auto_throttle_cps_interval seconds (defaults to 600 seconds).

After this interval passed, aiCache will re-test the CPS against the provided limit and should the CPS drop below the threshold, the throttling will be disabled.

Both enabling and disabling of the throttling will be logged in the aiCache error log file.

Both the total and per/sec numbers of requests and/or connections that were throttled due to intelligent request throttling, is displayed/reported via CLI, SNMP and Web interfaces.

____________________________________________________________

You can implement aiCache in your data-center or from a cloud instance without changing your architecture and we give 2 hours of free install support.

We generally specialize in sitting between the client browsers and the Web tier.  With a little digging into the AWS docs we began to look at S3 as a great big web server.

We wanted to set up an environment that would simulate a busy web site grabbing files off S3 and see how the aiCache would effect the results.  The details of the test are linked at the bottom of this post on our wiki.

With cached object we saw the time to load a 50 Kilobyte file was on average about 60 Milliseconds from s3.  It was about 4 milliseconds if accessed from aiCache, providing it had been cached.  So in short the first user would still endure the 60 Milliseconds, but the second user and all subsequent for the TTL specified would get it in about 4ms.  This factoring in network latency.

We did find an interesting fact.  Files that where not cached loaded about 5% faster with aiCache.  We believe this has to do with more efficient handling of the session management.   We also found that load times for files off s3 without aicache ranges widely from 35 Milliseconds to over 200.  The same files coming from aiCache where a uniform 4 to 6 milliseconds.

The rule of thumb in dynamic caching is that you need to have a good percentage of your content being accessed multiple times within the Time to live to make it make sense.

In a specific environment you would need to look at your log files and see if there is a certain percentage of files that are repeatedly coming from S3 within short periods of time, i.e. shorter than their being replaced by another file.  While maybe this is only 1% of your files, if that are being accessed 10′s, 100′s, or thousands of times within a reasonable TTL, (settable by pattern) the aiCache can have an enormous impact on your site speed.

We found that micro instances where basically worthless as they have no throughput.  Anything running aiCache on large instance and up significantly outperformed S3 both in the time to generate a specific file and the total throughput.

There are scenarios where this makes sense as described above. There are also scenarios where it makes no sense.  aiCache as a RAM based cache must consume the entire file before serving it.  Trying this with large files, say 100MB will actually slow down the initial request.  Also you need to be mindful of the memory footprint.  Additionally not a lot can be done for streaming protocols, with the exception of HTTP based streaming.

The synopsis is that aiCache can make a big impact as a caching tier between S3 and your web app, or directly serving to your end users client.

Our entire test environment is show in detail here:

AICACHE S3 ACCELERATION

You can configure aiCache to apply flexible request decimation logic in a number of ways.
For example, should your site be bombarded by certain requests that you don’t mind replying to under normal, low load conditions, yet would like to decimate when traffic starts to spike.

In its most basic form, you simply tell aiCache to allow a portion of the matching traffic to be allowed through via decimate_req, no matter the load:

Example:
____________________________________
pattern notalwaysthere.asp simple 0
decimate_req 10 # Only let through 1/10 of the traffic
____________________________________

In more evolved setup, you can configure aiCache to apply decimation when load (pattern or website-level RPS) exceed certain threshold:

____________________________________
pattern notalwaysthere.asp simple 0
decimate_req 10 # Only let through 1/10 of the traffic
decimate_req_pat_rps 100 # Only decimate when this pattern sees more than 100 RP
____________________________________

Or, testing against website-level RPS

____________________________________
pattern notalwaysthere.asp simple 0
decimate_req 10 # Only let through 1/10 of the traffic
decimate_req_ws_rps 400 # Only decimate when this website sees more than 400 RP
____________________________________

When activating pattern-level decimation in this load-driven fashion, aiCache will turn the decimation on for 60 seconds, before re-testing for the load condition and possibly turning decimation off. You can change this interval by setting decimate_req_interval pattern-level setting. You can think of this interval as a hysteresis feature.

____________________________________
pattern notalwaysthere.asp simple 0
decimate_req 10 # Only let through 1/10 of the traffic
decimate_req_ws_rps 400 # Only decimate when this website sees more than 400 RPS
decimate_req_interval 600 # decimate for 10 mins before retesting the load condition
____________________________________

Note that when requests are decimated in such fashion, they are silently and instantly dropped, to reduce the load on aiCache. No grace responses and/or redirection of any sort is sent back to the requesting clients.

Also note that aiCache automation activates pattern-level stat gathering when you enable request
decimation.

The number of decimated requests is reported via usual means – the Web, CLI and SNMP stats.

If your in this boat (And you now who you are) you probably want to look at this as well.
Bending the TTL’s. Dynamic traffic adaptation by aiCache

The issue is that this leads to a lot of cached content the is the same eating up memory.
In a Dynamic cache like aiCache, where all objects reside in Memory, this can require an unnecessary amount of additional RAM.

The solution:
We have enabled aiCache to identify common cached elements using the our regular expression technology. All HTTP requests that match these patterns become a single memory object shared across the sites.

You can still get as granular as you need with inclusion and exclusions on a domain basis.
The bottom line is you can now maximize the efficiency of common shared objects in a dynamic environment with some straight forward easy to use configurations.

WARNING IT GETS GEEKIER BELOW THIS LINE

You might have a setup when a number of different sites cached via aiCache, for example a.com,
b.com and c.com. Yet some of the content on these sites is identical and you’d like to cache(store) only a single copy of it, effectively sharing it across different sites.

For example, let’s assume that you want to share URLs that contain /css and /images prefix. To accomplish this, we simply tell aiCache to use a different signature prefix – instead of matching website’s hostname, we configure a different prefix via pattern-level sig_hostname setting.

For example:

website a.com

pattern /css simple 1d
sig_hostname css_content

website b.com
pattern /css simple 1d
sig_hostname css_content

website c.com
pattern /css simple 1d
sig_hostname css_content

Now, when a request is made for a.com/css/main.css, b.com/css/main.css or c.com/css/main.css, same cached response will be returned, saving the overhead of having to maintain 3 different copies of the same content.

As usual, make sure that the URLs are cacheable and are not website-specific in any way.

When asked what we think about Google website optimizer and Amazon ElastiCache our response is…its about time!!

So in short order what this means for aiCache? A bit more industy awareness of the importance of scalable architecture.

It helps to understand that ElasticCache is memcache. Its the same facility for providing an in-memory cache for developers to optimize databases for application servers. If your not doing this currently, you should.

Efficient use of memory based caching can dramatically increase the speed of content generation. It is also an excellent resource for static object retrieval, similar to a CDN or traditional static cache like Squid.

aiCache sits at a different tier, in front of the web server at the HTTP layer. We provide a massive increase in the distibution of dynamic content by caching requests before they hit the web, app, and database.

In short the two technologies should be used to optimize the creation and distibution of dynamic content. We applaud AWS for providing necessary component of modern app development. We look forward to accelerating these sites.

Google’s entry into optimization is great for sites without the resources or sophistication to write good code. This puts companies like Aptimize and Strangeloop squarely in their sites, (pun intended). In our opinion these where always temporary technologies. Managing the number of round trips in http transactions and more efficient handling of small auxillary content is handy, but is largely making up for bad code and inefficient http implementations. One can be solved by good code and the other by incremental industry improvement. In general, at scale, I don’t want anyone screwing with my code or content.

The thinly veiled entrance of aiCache into the CDN market is a problem for CDN providers like Akamai & Limelight. We think the CDN business is a dying beast (see DCC). We believe more sophisticated network delivery in inevitable. We hope google brings something great to the table.

Meanwhile back in aiCache land we will continue providing our unique value proposition of allowing you to massively increase the speed and scale of your dynamic website, without architecture change.

Google, Amazon welcome to the party, drinks are on us.

ESI allows you to slot in content at the Edge device, by including tags in the html code, that the edge server interprets, requests, and “includes” in the content.  You can take a web page that is mostly common content and cache all the common components.  Then at the point users make the request you can have unique or user specific data assembled at the edge and delivered to the browser.

At first glance it seems like a clever solution.  We think it makes no sense for two reasons.  First your simply moving the problem of page assembly from the origin environment to the edge.  Any edge device is going to have finite processing power and the overhead of assembling pages kills performance.  The  whole point of edge devices is to be lightning fast.  The second is that its a generally proprietary technology whose origin is in the architecture of a single network.

A fair bit of what ESI offers, is in the ESI language because customers of a large un-named CDN “with litigious tendencies” cannot configure the edge proxies any other way than “in-band”. While this is perfectly sensible for the a specific business model, it is of little relevance in a aiCache context, where the content-provider is in control of the servers.

The industry has a better smarter solution and that is replacing ESI –  with Ajax; client side page assembly. We feel ESI has a short shelf life and at present is a strategic dead end.

We continue to watch the specification as it evolves but have not committed to the coding until we see the more open market accepted solution, such as Ajax; client side assimilation, which we do support.

With Dynamic Caching you can provide all ESI functionality, using Ajax; client side page assembly. The key benefit is that using Ajax calls to assemble content at the browser moves the page assembly to the client.  Clients have two huge advantages.  First, dedicated bandwidth.  There is no contention at the client with a million other requests as you have at an edge server.  The second is that the processing power is local.  There are a few exceptions with devices that do not support Ajax in the mobile world.  We address these specifically in the mobile aiCache product.

Developers who build their applications with Ajax based client side assembly get the best of both worlds.  They get the capability for nearly unlimited scale using dynamic caching technology and they get client specific customization that does not inhibit performance.

As browser technology gets better we believe the future of web technology will be smarter and smarter clients capable of dynamic communication with the server side and edge environments.

We are focusing aiCache to support developers in deploying this technology quickly and easily to provide fast, stable, rich web applications customized to the user.

To address this requirement we  implemented a special thread dedicated to out-of-band resolution of DNS-defined origin servers. The thread runs in the background and periodically resolves OS DNS names to numeric IPs via regular DNS resolution. It is done so that aiCache is not burned with overhead of DNS resolution when it needs to access origin servers – such resolution happens out-of-band.

You configure how frequently aiCache runs such DNS resolution, at per-website basis, via dns_interval setting. Specified in seconds, the default value is 120 seconds (2 minutes).

Consider the following example:

origin 127.0.0.1 8888 0 10

origin 127.0.0.1 8889 0 20

origin 127.0.0.1 8890 0 30

website

hostname www.acme.com

dns_interval 600

..origin origin.acme.com 80

www.aiCache.com

Upon startup, aiCache resolves all and any DNS-defined origin servers. Assuming origin.acme.com has 5 DNS “A” records defined, 5 OS will be defined, used and reported.

Every 600 seconds (10 minutes) aiCache re-runs DNS resolution for this website, attempting to resolve origin.acme.com to a list of DNS “A” records. As long as the same 5 “A” records are returned, there are no changes to OS configuration that was established during the startup.

If an additional record is returned, it is added as an additional OS for www.acme.com. It is also reflected in the aiCache error log file.

When a previously-defined “A” record disappears, it is marked as “DNS-disabled” and is not used in OS capacity (no traffic is sent to it). However, the record of that OS is kept by aiCache – so you can see its statistics etc. It is also reflected in the aiCache error log file.

When a previously-disabled “A” record re-appears, it is marked as “DNS-enabled” and is set to again receive its share of traffic. It is also reflected in the aiCache error log file.

aiCache also reports when a website has any DNS-defined OS. Time of last DNS resolution is also logged and reported. All DNS-defined OS are reported likewise.

To assist in initial DNS setup, aiCache offers global debug_dns flag setting. When set, copious amounts of DNS debug information are written out to aiCache error log file.

We expect our customer to continue to come up with creative scenarios and we enjoy adding feature functionality like this to accomodate.