Web Cache: Why isn't this app recommended for most networks?

Web Cache can be very useful in certain situations; in others it can slow down browsing and drain the resources of your NG Firewall without much benefit. We've written up an article to give you some background on when (and when not) to use Web Cache. 

Thinking it through

Here's an analogy I use a lot. Pick a task you do almost every day; say, commuting to work. Ask yourself, "Self, would I drive an extra mile every day if it gave me a 1% chance to skip my commute entirely on any given day?"

Think that through for a minute.

For most people the answer should probably be no. Let's assume most people commute 10 miles. If they said yes, they would now be driving 11 miles every day with the 1% chance to save 10 miles on any given day. For every extra 100 miles driven, they would save 10. That math doesn't really work out.

You would answer "yes" if you commute over 100 miles a day, or you were going to drive an extra mile out of your way anyway (perhaps for coffee) because in this situation it will save you from driving. 

When to use Web Cache

You should run Web Cache if your internet is very slow (i.e. a 100+ mile commute) or you have unspent server resources (a sunk cost) that you want to recover (you're driving extra mile to get your coffee anyway). In most cases, you should not run Web Cache because you'll end up spending $100 in server resources to save $10 in internet traffic and saved browsing time.

The problem when people say that Web Cache is great unilaterally and that it saves time is that they're not accounting for the server resources spent on caching, or the sunk cost they've already spent on server resources that are sitting idly doing nothing. If you have a sunk cost in idle server resources, Web Cache might indeed be able to help you recover some of this cost, but this is not money earned, it's losses recovered.

In reality, most often we see large sites (like schools) with decent internet connections (i.e. a <10 mile commute) that have extremely busy NG Firewall servers (i.e. no extra unspent resources) because they are literally running every app NG Firewall offers with the data retention time in Reports cranked up, introducing some extra load on the disk for good measure. The CPU is busy, disk I/O is saturated, and memory is being used. In this case you'll save maybe $5 of bandwidth a month, however you will lose much, much more in added latency to run the cache and possible downtime (depending on how busy your server is).

It's important to take into account a few key trends:

• Bandwidth is now huge and growing (on the scale of cachable web content)
• More and more web content is dynamic (uncachable)
• Web content is growing dramatically (making user dupe cache hits less likely)
• Modern browser speed wars have made browser caches extremely efficient.

These four trends have made stand-alone web caching much less effective. The internet is very different from 1996 when squid was launched and businesses were running on ISDN lines, most web content was static, and browsers were terrible. 

Conclusion

You should run Web Cache if meet a few of the following conditions:

• Your ISP is really, really slow
• You have multiple users browsing duplicate content (otherwise the browser cache will do this work for you)
• You have plenty of unspent resources on the NG Firewall server (mostly disk I/O and CPU; Web Cache doesn't require much memory)
• You have really expensive pay-per-usage bandwidth