Random Musings of an Insane Mind

#!/bin/sh

FETCHES=1000
PARALLEL=10

/usr/sbin/apache2ctl stop
/etc/init.d/mysql restart
apache2ctl start
echo Sleeping
sleep 30
time ( \
echo First Run; \
ab -n $FETCHES -c $PARALLEL http://example.com/; \
echo Second Run; \
ab -n $FETCHES -c $PARALLEL http://example.com/; \
\
echo First Run; \
./http_load -parallel $PARALLEL -fetches $FETCHES wordpresstest; \
echo Second Run; \
./http_load -parallel $PARALLEL -fetches $FETCHES wordpresstest; \
)

http://example.com/
http://example.com/2010/03/hello-world/
http://example.com/2008/09/layout-test/
http://example.com/2008/04/simple-gallery-test/
http://example.com/2007/12/category-name-clash/
http://example.com/2007/12/test-with-enclosures/
http://example.com/2007/11/block-quotes/
http://example.com/2007/11/many-categories/
http://example.com/2007/11/many-tags/
http://example.com/2007/11/tags-a-and-c/
http://example.com/2007/11/tags-b-and-c/
http://example.com/2007/11/tags-a-and-b/
http://example.com/2007/11/tag-c/
http://example.com/2007/11/tag-b/
http://example.com/2007/11/tag-a/
http://example.com/2007/09/tags-a-b-c/
http://example.com/2007/09/raw-html-code/
http://example.com/2007/09/simple-markup-test/
http://example.com/2007/09/embedded-video/
http://example.com/2007/09/contributor-post-approved/
http://example.com/2007/09/one-comment/
http://example.com/2007/09/no-comments/
http://example.com/2007/09/many-trackbacks/
http://example.com/2007/09/one-trackback/
http://example.com/2007/09/comment-test/
http://example.com/2007/09/a-post-with-multiple-pages/
http://example.com/2007/09/lorem-ipsum/
http://example.com/2007/09/cat-c/
http://example.com/2007/09/cat-b/
http://example.com/2007/09/cat-a/
http://example.com/2007/09/cats-a-and-c/

25 Responses to “WordPress Cache Plugin Benchmarks”

1. Visit my site and it tries to download a GZ file - Page 2 - WordPress Tavern Forum Says:
   April 15th, 2010 at 2:31 pm

   […] up to answer my own question: http://cd34.com/blog/scalability/wor…in-benchmarks/ Basically what I (still) don't like about W3 Total is that there's so MANY options, and I'm not […]

2. Rootz.de > Wordpress: Benchmarking von Caching-Plugins Says:
   April 30th, 2010 at 8:01 am

   […] http://cd34.com/blog/scalability/wordpress-cache-plugin-benchmarks […]

3. Maximize WordPress and BuddyPress Performance With W3 Total Cache - WordPress MU and BuddyPress plugins, themes, support, tips and how to's Says:
   May 10th, 2010 at 10:05 am

   […] and yoast.com. If you’d like to compare it to some other caching plugins, have a look at some WordPress cache plugin benchmarks for a thorough analysis of current WordPress scalability […]

4. WordPressハッカーズ Says:
   June 12th, 2010 at 8:41 am

   WordPress Cache Plugin Benchmarks…

   この記事は以下サイトに掲載されています。 This article has been featured on WordPressハッカーズ…

5. Information Technology Education | Blog | Maximize WordPress and BuddyPress Performance With W3 Total Cache Says:
   August 19th, 2010 at 10:02 am

   […] and yoast.com. If you’d like to compare it to some other caching plugins, have a look at some WordPress cache plugin benchmarks for a thorough analysis of current WordPress scalability […]

6. High Performance WordPress — Konstruktors Says:
   November 4th, 2010 at 7:16 am

   […] WordPress Cache Plugin Benchmarks […]

7. Peter Says:
   November 20th, 2010 at 10:23 am

   Very informative. It would have been nice to see how W3 with memcache performs as well, but you can’t have it all.

8. cd34 Says:
   November 20th, 2010 at 12:46 pm

   @peter, the results are sorted in order by performance. W3/memcache test results are within the results, results 7, 8 and 9.

9. Peter Says:
   November 22nd, 2010 at 2:14 pm

   Sorry, I see it now.

10. blaenk Says:
    December 30th, 2010 at 7:53 pm

    Excellent post. It motivated me to try out varnish and it is as blazing fast as you prove it to be in your post. I’m still pretty confused on the whole removing cookies part though, and I was wondering if you would ever be interested in writing a post which goes over your configuration of it for working with WordPress. There are scattered posts around the web but they all seem pretty inconsistent, particularly the part about unsetting cookies.

    Also, another question: If you’re using Varnish, is there a point to also use W3 Total Cache (for opcode caching)? Or should one just use Varnish and leave it at that?

    Thanks again for the very informative post.

11. cd34 Says:
    December 31st, 2010 at 3:37 pm

    Varnish doesn’t cache content if a cookie is sent within the request, so, to cache static assets, you need to remove the cookie in vcl_recv so that varnish will cache it.

    Ideally, any caching solution needs fragment caching, something that Frederick Townes (the author of W3TC) and I talked about at length a while back and is the reason I wrote the WordPress Varnish ESI plugin. Varnish doesn’t compress ESI after assembling it, which results in a convoluted installation with nginx -> varnish -> apache/nginx -> wordpress. The reason you need fragment caching is the sidebar is the same on almost every page, but, generating that sidebar has two of the most expensive queries in WordPress (Categories and Tag Cloud).

    Caching itself is difficult. Cache too much and your site runs quickly, but loses interactivity. Cache too little and your site is sluggish. While anyone can write caching, purging is the difficult part. If you were to run Varnish -> W3TC -> WordPress, using the WP-Varnish plugin, modified posts would have to purge the entry from Varnish and W3TC, and when Varnish gets a request for that new page, W3TC would have to cache it, then hand it to Varnish – resulting in double caching and probably not a lot of net benefit. You really want to run one or the other. If you can run Varnish with wp-varnish, I think that provides the best solution for now. If you aren’t able to run Varnish because you’re on a shared host, then, W3TC or WP-Super-Cache are about the best you can get. And Frederick Townes doesn’t sit still for long – since these benchmarks were run, he’s made the plugin much more intelligent and enhanced it quite a bit.

    I’ll see if I can make a more generic WordPress .vcl and post it.

12. Wordpress optimieren: Varnish und mehr Says:
    January 16th, 2011 at 9:57 am

    […] Einen sehr guten Überblick über die verschiedenen Caching-Lösungen gibt dieser Benchmark-Artikel.Die Installation ist recht einfach und kann mittels ein paar Befehlen unter Debian ausgeführt […]

13. matt Says:
    February 17th, 2011 at 12:22 pm

    Just came across this post. You mention in regards to W3TC that “If you can’t use APC caching or Memcached for caching Database queries or Minification, turn both off. ” – I’m using eaccelerator for those. I didn’t see that option in the benchmark..I’m wondering if I should assume that eaccelerator isn’t as good as just leaving the db caching and minification off?

    Also, I’m using the development version of W3TC and I notice it has a purge Varnish option – I’m wondering if that would eliminate the need for WP-Varnish.

14. cd34 Says:
    February 17th, 2011 at 2:32 pm

    As this post was written about a year ago, I don’t recall whether eaccelerator was an option. If it was, I’m sure I would have tested it. W3TC has gone through a number of changes since I tested it, so, it is also possible some of the areas where it didn’t benchmark as well as one might have expected, may have been fixed. The other issue I ran into after this test is that both W3TC and WP-Super-Cache installations were very dependent on the machine it was installed on, so, you really need to have a test methodology to see if it is improving things with all of the different options available.

    When I did the test, Frederick Townes was talking about adding a Varnish component to it, but, since he’s doing page caching, if his purge does what I think it should, it would work similar to wp-varnish with the added bonus of minification, css combining, etc. Basically, with the other features, I think W3TC with Varnish would probably trump WP-Varnish overall if you consider the total benefit of those other features.

15. WordPress Caching Plugins and Observations with W3 Total Cache | Techno Thoughts! Says:
    March 3rd, 2011 at 6:19 pm

    […] WordPress Cache Plugin Benchmarks […]

16. Short-term Todo List/Plans Says:
    May 2nd, 2011 at 2:50 pm

    […] WP-Varnish (Can handle a lot more requests than W3 Total Cache, and I feel like getting my geek on and trying it […]

17. Ivete Tecedor Creative » Wordpress Caching Benchmarks Says:
    June 22nd, 2011 at 7:11 am

    […] just came across an impressive post benchmarking the performance of various WordPress caching plugins and had to share. The post was written over a year ago but it’s so thorough and useful that […]

18. Плагин для кэширования Wordpress или адронный коллайдер для блога | seozip Says:
    June 28th, 2011 at 6:26 am

    […] – Для выбора рекомендую ознакомится с бэнчмаркоми плагинов кэширования […]

19. Usando Varnish como proxy [WordPress + Apache] Says:
    July 26th, 2011 at 8:28 pm

    […] algunos posts que me fueron útiles, ya que contienen instrucciones sencillas:Blog de Daniel MesserArtículo en CD34.comHow-to ForgeEuperia blogA posteriori: Comparando el rendimientoEs muy importante medir el efecto que […]

20. todsul Says:
    August 6th, 2011 at 4:49 pm

    Hey Chris,

    I’ve just tested the following on a default WordPress 3 install:
    – bare bones Nginx
    – Nginx + W3TC (page cache and minify using disk enhanced)
    – Nginx + Varnish (removed all cookies) + WP Varnish plugin

    All of these have APC opcode cache enabled.

    The bottom 2 configs are producing very similar results, but W3TC is more stable. My ab test for Varnish hangs on the first try and also slows down after I repeat the test many times. I’ve checked memory and there’s plenty free.

    Also, in terms of user experience, there’s a big banner image on the default WordPress theme that rotates randomly. W3TC caches that page and the same banner image shows. Varnish is letting the home page image reload even though I’ve removed all cookies in the VCL. I wonder if it’s to do with browser cache settings (will investigate).

    I saw a comment from Frederick Townes somewhere that he’s lost interest in Varnish. I wonder if that’s based on his own tests.

    As for my results:
    – bare bones Nginx = 57 req/s
    – Nginx + W3TC = 5180 req/s
    – Nginx + Varnish = 5124 req/s

    I wanted to have a nice clean Nginx + Varnish solution (incl Plugin), maybe with a minify plugin, but that’s it. But the Varnish results aren’t especially impressive for the additionally complexity it adds.

    Any thoughts on anything I’m missing regarding Varnish? My configs are really bare bones. No funny business.

21. todsul Says:
    August 6th, 2011 at 5:34 pm

    Just tested – Nginx + WP Super Cache

    Doubled the performance of W3TC and Varnish with 10,602 requests/s.
    The load for non cached pages feels noticeably quicker.

    This is all on the smallest Linode VPS.

22. cd34 Says:
    August 6th, 2011 at 10:10 pm

    As the benchmarks here are over a year and a half old, I would not be surprised that you couldn’t get numbers like that. In brief testing even with Varnish 3.0, we’ve got machines pushing 40k rps and Kristian Lyngstol has benchmarked Varnish at 275k rps (http://kristianlyng.wordpress.com/2010/10/23/275k-req/). There are also considerable performance scheduling changes in 2.6.39 and network improvements in 2.6.37 that would have considerable impact. Also, I don’t believe the shm file was in ram on these tests but, I’m sure I used malloc. libc6 has also received considerable improvement with malloc.

    W3TC is a page cache which uses obstart() and grabs the entire page, writes it to disk or memcached and then outputs it. The header rotator outputs html which is inserted in the page. When you’re logged in, Varnish wouldn’t actually cache the index page, allowing the img src which rotates the header to be generated on each pageload.

    W3TC’s config options are very confusing and without testing and retesting, it is very difficult to find the best configuration. WP Super Cache’s out of the box setup is really much better than W3TC’s. With enough time, you could find the magic combination of settings to be faster than WP Super Cache. There are other benefits to W3TC that can be duplicated with other plugins, but, if you wanted everything in one place, W3TC contains a number of useful plugins all in one place.

    The other thing you need to consider with your benchmark is that Linode uses virtio on the interface and benchmarking on the machine without actually using the network is going to allow you to have considerably higher numbers than if you’re actually moving traffic across the network. You can tell W3TC to not cache for logged in users which more closely replicates how Varnish is handling things. Also, I would suspect that on non-logged in users, the images wouldn’t be rotating until the cache time expired or the page was purged from the cache on both W3TC and Varnish – and, as your latest comment suggests, WP Super Cache.

23. W3 Total Cache or WP Super Cache? - Admins Goodies Says:
    August 14th, 2011 at 2:33 pm

    […] http://cd34.com/blog/scalability/wordpress-cache-plugin-benchmarks/ […]

24. WordPress 缓存插件性能对比 » 域名宝盒 – DNCase.com Says:
    September 8th, 2011 at 4:40 am

    […] 原文：《WordPress Cache Plugin Benchmarks》 […]

25. 性能优化插件–坑爹的W3 Total Cache | 风水的英文网赚和黑帽SEO博客 Says:
    September 11th, 2011 at 9:22 pm

    […] not W3 Total Cache (评论闪闪发光,很多技术细节) WordPress load test part 2 – amendment WordPress Cache Plugin Benchmarks W3 Total Cache versus WP Super […]

Leave a Reply

You must be logged in to post a comment.