How I built “Have Baby. Need Stuff!”

Have Baby. Need Stuff! is a baby gear site that my wife and I just launched. I thought I’d share how I built it.

WordPress Core

WordPress is a Git submodule, with the content directory moved to the /content/ directory. This makes my Git repo smaller, as WordPress isn’t actually in it.

Underscores

For a theme base, I started with the Underscores starter theme by the theme team at Automattic. Underscores is not a theme itself… it’s a starting point for building your own theme.

Bootstrap

Next, I integrated Bootstrap, by Twitter, to handle the CSS base and the grid system. Bootstrap is a really powerful framework, and version 2.0 has great responsive design support, which allowed me to create a single design that scales up to big screens or down to tablet or phone screen sizes. Try resizing it in your browser to see the responsiveness in action!

The CSS for the site is authored using LESS, which plays well with Bootstrap. I’m compiling/minifying/concatenating the CSS and JS using CodeKit, an amazing Mac OS X app that makes development a breeze.

Typekit

For web fonts, it’s hard to beat Typekit.

Subtle Patterns

I needed some patterns to use on the site, but I was frustrated with the licensing terms on many pattern sites I was finding. And then I found Subtle Patterns. Gorgeous, subtle patterns, liberally licensed. And hey, their site is WordPress powered too!

Posts 2 Posts

The site has the concepts of Departments, Needs, and Products. Each Department has multiple Needs. Each Need has multiple Products. I used Scribu’s phenomenal Posts 2 Posts plugin to handle these relationships.

Here’s the basic Posts 2 Posts connection code:

<?php

function hbns_register_p2p_relationships() {
	if ( !function_exists( 'p2p_register_connection_type' ) )
		return;

	// Connect Departments to Needs
	p2p_register_connection_type( array(
		'name' => 'departments_to_needs',
		'from' => 'department',
		'to' => 'need',
		'sortable' => 'to',
		'admin_box' => 'to',
		'admin_column' => 'any',
		'cardinality' => 'one-to-many',
	) );

	// Connect Needs to Products
	p2p_register_connection_type( array(
		'name' => 'needs_to_products',
		'from' => 'need',
		'to' => 'product',
		'sortable' => 'from',
		'admin_column' => 'any',
		'admin_box' => array(
			'show' => 'any',
			'context' => 'advanced',
		),
		'cardinality' => 'many-to-many',
		'fields' => array(
			'description' => 'Description',
		),
	) );
}

add_action( 'wp_loaded', 'hbns_register_p2p_relationships' );

I created a Custom Post Type for each of Departments, Needs, and Products, and connected them all using Posts 2 Posts. The connection between a Need and a Product also contains description metadata, as seen here:

Since Posts 2 Posts was a required plugin for the site to function, I didn’t want there to be any possibility of accidental deactivation. So I wrote a quick mu-plugins drop-in to “lock” certain plugins on.

<?php
class HBNS_Always_Active_Plugins {
	static $instance;
	private $always_active_plugins;

	function __construct() {
		$this->always_active_plugins = array(
			'batcache/batcache.php',
			'posts-to-posts/posts-to-posts.php',
			'login-logo/login-logo.php',
			'manual-control/manual-control.php',
		);
		foreach ( $this->always_active_plugins as $p ) {
			add_filter( 'plugin_action_links_' . plugin_basename( $p ), array( $this, 'remove_deactivation_link' ) );
		}
		add_filter( 'option_active_plugins', array( $this, 'active_plugins' ) );
	}

	function remove_deactivation_link( $actions ) {
		unset( $actions['deactivate'] );
		return $actions;
	}

	function active_plugins( $plugins ) {
		foreach ( $this->always_active_plugins as $p ) {
			if ( !array_search( $p, $plugins ) )
				$plugins[] = $p;
		}
		return $plugins;
	}
}

new HBNS_Always_Active_Plugins;

Custom Post Types

I’m using the Products post type in a slightly odd way. You don’t ever go to a product URL. You instead go the URL for the Need that the Product fulfills, and that page lists all of the connected Products. As such, I wanted to make it so that URLs for products pointed to their Need, and I wanted to add an admin bar Edit link for the primary product on its Need page.


<?php
/*
Plugin Name: Post Links
Version: 0.1
Author: Mark Jaquith
Author URI: http://coveredwebservices.com/
*/

// Convenience methods
if(!class_exists('CWS_Plugin_v2')){class CWS_Plugin_v2{function hook($h){$p=10;$m=$this->sanitize_method($h);$b=func_get_args();unset($b[0]);foreach((array)$b as $a){if(is_int($a))$p=$a;else $m=$a;}return add_action($h,array($this,$m),$p,999);}private function sanitize_method($m){return str_replace(array('.','-'),array('_DOT_','_DASH_'),$m);}}}

// The plugin
class CWS_HBNS_Post_Links_Plugin extends CWS_Plugin_v2 {
	public static $instance;

	public function __construct() {
		self::$instance = $this;
		$this->hook( 'plugins_loaded' );
	}

	public function plugins_loaded() {
		$this->hook( 'post_type_link' );
		$this->hook( 'add_admin_bar_menus' );
	}

	public function add_admin_bar_menus() {
		$this->hook( 'admin_bar_menu', 81 );
	}

	public function admin_bar_menu( $bar ) {
		if ( is_single() && 'need' == get_queried_object()->post_type ) {
			$primary_product = new WP_Query( array(
				'connected_type' => 'needs_to_products',
				'connected_items' => get_queried_object(),
			) );
			if ( $primary_product->have_posts() ) {
				$bar->add_menu( array(
					'id' => 'edit-primary-product',
					'title' => 'Edit Primary Product',
					'href' => get_edit_post_link( $primary_product->posts[0] ),
				) );
			}
		}
	}

	public function post_type_link( $link, $post ) {
		switch ( $post->post_type ) {
			case 'product' :
				$need = new WP_Query( array(
					'connected_type' => 'needs_to_products',
					'connected_items' => $post,
				) );
				if ( $need->have_posts() )
					return get_permalink( $need->posts[0] );
				break;
		}
		return $link;
	}
}

new CWS_HBNS_Post_Links_Plugin;

For entering data about Products, I made a custom Meta Box that provided a simple interface for entering the Amazon.com link, the approximate price, and then a freeform textarea for key/value pairs and miscellaneous bullet points.

Misc

Because I’m using a Git-backed and Capistrano-deployed repo, I don’t want any local file editing. So I dropped this code in:


<?php

define( 'DISALLOW_FILE_EDIT', true );

function hbns_disable_plugin_deletion( $actions ) {
	unset( $actions['delete'] );
	return $actions;
}

add_action( 'plugin_action_links', 'hbns_disable_plugin_deletion' );

I was playing a lot with different Product thumbnail sizes, so Viper007Bond’s Regenerate Thumbnails plugin was invaluable, for going back and reprocessing the images I’d uploaded.

And of course, no WordPress developer should make a site without Debug Bar and Debug Bar Console.

Nginx and PHP-FPM

My server runs Nginx and PHP-FPM, in lieu of Apache and mod_php. My normal setup is to use Batcache with an APC backend to do HTML output caching, but I also have an Nginx “microcache” that caches anonymous page views for a short amount of time (5 seconds). But with this site, I wanted to cache more aggressively. Because there are no comments, the site’s content remains static unless we change it. So I cranked my microcache up to 10 minutes (I guess it’s not a microcache anymore!). But I wanted a way to purge the cache if a Product or Post was updated, without having to wait up to 10 minutes. So I modified the Nginx config to recognize a special header that would force a dynamic page load, effectively updating the cache.

Here’s the relevant part of the Nginx config:

	location ~ \.php$ {
		# Set some proxy cache stuff
		fastcgi_cache microcache_fpm;
		fastcgi_cache_key $scheme$host$request_method$request_uri;
		fastcgi_cache_valid 200 304 10m;
		fastcgi_cache_use_stale updating;
		fastcgi_max_temp_file_size 1M;

		set $no_cache_set  0;
		set $no_cache_get  0;

		if ( $http_cookie ~* "comment_author_|wordpress_(?!test_cookie)|wp-postpass_" ) {
			set $no_cache_set 1;
			set $no_cache_get 1;
		}

		# If a request comes in with a X-Nginx-Cache-Purge: 1 header, do not grab from cache
		# But note that we will still store to cache
		# We use this to proactively update items in the cache!
		if ( $http_x_nginx_cache_purge ) {
			set $no_cache_get 1;
		}

		# For cached requests, tell client to hang on to them for 5 minutes
		if ( $no_cache_set = 0 ) {
		        expires 5m;
		}

		# fastcgi_no_cache means "Do not store this proxy response in the cache"
		fastcgi_no_cache $no_cache_set;
		# fastcgi_cache_bypass means "Do not look in the cache for this request"
		fastcgi_cache_bypass $no_cache_get;

		include        /etc/nginx/fastcgi_params;
		fastcgi_index  index.php;
		try_files      $uri =404;

		fastcgi_pass phpfpm;
	}

Now I just needed to have WordPress ping those URLs with that header to refresh them when something changed. Here’s the “meat” of that code:

<?php
	public function transition_post_status( $new, $old, $post ) {
		if ( 'publish' !== $old && 'publish' !== $new )
			return;
		$post = get_post( $post );
		$url = get_permalink( $post );

		// Purge this URL
		$this->purge( $url );

		// Purge the front page
		$this->purge( home_url( '/' ) );

		// If a Product changes, flush its Need and that Need's Department
		if ( 'product' === $post->post_type ) {
			// Flush the connected need
			$need = new WP_Query( array(
				'connected_type' => 'needs_to_products',
				'connected_items' => $post,
			) );
			if ( $need->have_posts() ) {
				$this->purge( get_permalink( $need->posts[0] ) );
				// Now this need's connected Department
				$department = new WP_Query( array(
					'connected_type' => 'departments_to_needs',
					'connected_items' => $need->posts[0],
				) );
				if ( $department->have_posts() )
					$this->purge( get_permalink( $department->posts[0] ) );
			}
		// If a Post changes, flush the main Blog page
		} elseif ( 'post' === $post->post_type ) {
			$this->purge( home_url( '/blog/' ) );
		}
	}

	private function purge( $url ) {
		wp_remote_get( $url, array( 'timeout' => 0.01, 'blocking' => false, 'headers' => array( 'X-Nginx-Cache-Purge' => '1' ) ) );
	}

Boom. Now I get the benefit of long cache times, but with the ability to have updates go live quickly when I need to. The upshot here is that while I have Batcache installed, it’s not really going to get a lot of use, as the outer Nginx caching layer should handle everything. This doesn’t just mean that the site scales (Apache Bench has it handling many thousands of requests a second with ease), but that the site is really, really fast to browse. Your experience will vary according to network and geography, of course. But for me, I’m getting 34ms HTML delivery times for pages in the Nginx cache.

Questions?

So that’s how I did it. Let me know if you have any questions!

123 thoughts on “How I built “Have Baby. Need Stuff!”

  1. This whole thing is a combination of every cool new trick thing I’ve been wanting to try from submodules to _s to LESS to P2P… Awesome stuff, thanks!

  2. seriously awesome post Mark – you’ve brought lots of cool tech together here – I’m gonna have to schedule a few hours to conduct some experiments based on this! By the way – is your comment form something custom you put together? that’s also uber cool 🙂

    1. ahh yes – forgot I was on a wordpress.com site for a minute 🙂 Looks great!

  3. Out of curiosity, are you using any tricks to add the custom meta data (especially the “Other data” in the “Product data” meta box) to revision history? I could see that being really helpful if you make a mistake or need to roll back any of the data there.

    1. I’m not, but I certainly could be. That’s a really good idea. WordPress doesn’t support that out of the box but you can get it if you use a slightly lower level function for saving the metadata, and handle the restoration yourself.

    1. A few reasons.

      One, I didn’t want to complicate things. Nginx modules need to be compiled in. As of right now I’m using stock Nginx installed with Aptitude. Compiling is a headache I didn’t want to put on myself.

      Two, I didn’t just want to purge, I wanted to update. If i just purged, then the next view to that URL would hit PHP-FPM/WordPress, and that view would be slower for that person. By doing proactive refreshing, I can effectively make sure no one ever gets a live WordPress view. (I’ve since added a cron job that refreshes every page before it expires from the cache — that’s something you can do on a mostly static site with a limited amount of content).

    2. Mark, in attempting to make sense of all the WordPress caching plugins, and more general caching techniques, could you please explain the following…

      If you’re using Nginx to microcache pages for 10 minutes, and you’re also running a cron job to purge and refresh the cache preemptively before the 10 minutes is up, do you actually need Batcache in the mix at all? What purpose does it serve in this case?

  4. Good stuff. Echo the idea that this is pretty much all the new stuff I’m looking to do with next WP project. One thing i didn’t see, about how many hours do you think the entire build took you? So I can double that for my own expectations…

    1. I didn’t really track my time, but considering that I had to lean Bootstrap, learn Less, learn the Nginx caching stuff, learn Posts 2 Posts, come up with a design, etc, it was probably at least 40 hours. I certainly could have done it faster by using a stock theme and doing less “fancy” stuff, but part of the appeal was for me to use it as a learning experience.

    2. Thanks. That’s exactly what I was looking for. I think there are a lot of us who will be doing the same soon – putting some of these things like bootstrap in to practice for the first time.

  5. It’s interesting seeing the timing of this for me personally, because I just had my baby girl not too long ago and can still remember reading lots of reviews on baby bjorns, co-sleepers, swaddles, etc.. In fact, I got just as motivated as you (ok, maybe not on such a grand scale) to log into WordPress and publish this post http://bit.ly/calmingbaby. Out of curiosity… are you planning on integrating BP for site member profiles, etc? Anyway, thanks for sharing the code. I might have to play around with Posts 2 Posts down the road =)

    1. No plans for anything like that right now. My wife does plan to write blog posts, however. She’s currently working on a blog post about the baby gear you shouldn’t buy.

  6. We typically use W3 Total Cache with Nginx and PHP-FPM, combined with memcached and APC, as well as a Nginx configuration file that W3TC creates. I also create a submodule from the WordPress github repo in a folder named /wp/ with a local-config.php file that is ignored by git which contains local configuration variables. I also built a base theme with styles powered by SASS (similar to LESS) and Twitter Bootstrap with full bbPress and BuddyPress support:

    https://github.com/LiftUX/SASS-Bootstrap-Theme

    It’s not yet ready for production sites but we’re trying to get it finished as time allows. I love seeing how other folks like to do things and it helps verify that I’m not completely crazy when I see someone thinking along the same lines as I am.

    1. If you’re using a single server, then you shouldn’t use Memcached. It’s not as fast as using APC. Of course if you have multiple servers, then Memcached is your winner!

    2. You mention in the article that you’re using Batcache – doesn’t that use Memcached for caching? If, as you say “using a single server, then you shouldn’t use Memcached”, does this mean “Have Baby. Need Stuff!” is running on multiple servers? I’m just trying to get my head around this so no offence intended, but multiple servers sounds a little excessive for such a site?

      Fantastic article, it’s great to see everything brought together like this, especially by a core dev.

    3. @emyr — This is not a well-known fact, but Batcache can use either the Memcached or the APC object cache backend. It just requires persistence and access to an incrementor, both of which those backends provide. I’m using the APC backend, because this is just one server (in fact, this server is serving about a dozen sites as well as acting as being a Minecraft server. 🙂 ).

    4. @mark That’s good to know. Found more info on Batcache & APC on this very blog in fact! 😉

      in fact, this server is serving about a dozen sites as well as acting as being a Minecraft server

      In terms of server resources, how beefy is that server you mentioned? I have quite a few small low-traffic WordPress sites which I’m planning on hosting on a small Linode VPS running Debian 6, and wondering what I can reasonably get away with.

    5. @emyr — It’s a Linode 2048. Ubuntu 10.04.4 LTS. Most of the CPU and half the RAM goes towards running Minecraft, which is a hog. I could have put it on a much smaller one. With a well crafted server setup (i.e. nginx and PHP-FPM instead of Apache and mod_php), you can get away with a much smaller VPS than that.

    6. @emyr I’ve built a (mostly static) WordPress site serving 10k+ page visits a day on Linode 512 (nginx +php-fpm). Though with VPS you can always re-size if you mis-calculate initial requirements 🙂

      Websites each have their quirks and requirements, but having used Linode + WP + Nginx for a few years now, I do find that people tend to overestimate their needs. I can heartily recommend it.

  7. I’ve been using a similar git repo setup as this, taken from one of your WordCamp presentations, and love it. It’s very streamlined, and having separate repos for plugins, themes and core is ideal.

    Have you tried using Forge for building themes with pre-processing? It suppors Compass, Sass, Less, Coffeescript and .erb. I used it to build my own theme framework, and found that it really streamlines development, and provides nice structure. I made the framework project public at http://github.com/jeffsebring/render.forge

    I’m looking forward to having a reason to use Post 2 Posts. That would have been useful for some projects I’ve worked on in the past.

    1. Forge was built by TheThemeFoundry, and can be found at http://forge.thethemefoundry.com. There are some good Docs there for it. Forge is what TwentyTwelve is being built with.

      It’s basically a Ruby Gem, that works like Compass. It can generate a type of scaffolding to start with, watch files for changes, build the theme, and package it in a zip.

      Here’s a quick start for those who have Ruby installed:

      • gem install forge
      • forge create mytheme
      • cd mytheme
      • forge link /THEME_DIRECTORY/mythemename
      • forge watch

      This will generate a new forge project, create symlinks to the theme directory, and build the theme files in it. Forge watch works just like Compass watch, but updates all theme files.

  8. Thanks for the great post! Would you mind also posting your standard Nginx “microcache” config for WP, haven’t successfully been able to get it to work.

    1. Here you go:


      server {
      listen 80;
      server_name example.com;
      access_log /var/log/nginx/access.log main;
      root /srv/www/example.com;
      index index.html index.htm index.php;
      location ~ \.(jpe?g|gif|png|css|bmp|js|ico)$ {
      expires 30d;
      }
      location / {
      try_files $uri $uri/ /index.php?$args;
      }
      # redirect server error pages to the static page /50x.html
      #
      error_page 500 502 503 504 /50x.html;
      location = /50x.html {
      root /var/www/nginx-default;
      }
      location ~ /\.ht[a-z]+$ {
      deny all;
      }
      location ~ \.php$ {
      # Set some proxy cache stuff
      fastcgi_cache microcache_fpm;
      fastcgi_cache_key $scheme$host$request_method$request_uri;
      fastcgi_cache_valid 200 5s;
      fastcgi_cache_use_stale updating;
      fastcgi_max_temp_file_size 1M;
      fastcgi_cache_min_uses 3; # Hit a URL 3 times before caching it
      set $no_cache_set 0;
      set $no_cache_get 0;
      set $temp_caching_exemption 0;
      if ($request_method !~ ^(GET|HEAD)$) {
      set $temp_caching_exemption 1;
      }
      if ( $temp_caching_exemption = 1 ) {
      add_header Set-Cookie "_mcnc=1; Max-Age=10; Path=/";
      }
      # Bypass cache if no-cache cookie is set
      if ( $http_cookie ~* "_mcnc" ) {
      set $no_cache_set 1;
      set $no_cache_get 1;
      }
      if ( $http_cookie ~* "comment_author_|wordpress_(?!test_cookie)|wp-postpass_" ) {
      set $no_cache_set 1;
      set $no_cache_get 1;
      }
      # fastcgi_no_cache means "Do not store this proxy response in the cache"
      fastcgi_no_cache $no_cache_set;
      # fastcgi_cache_bypass means "Do not look in the cache for this request"
      fastcgi_cache_bypass $no_cache_get;
      include /etc/nginx/fastcgi_params;
      fastcgi_index index.php;
      try_files $uri =404;
      fastcgi_pass phpfpm;
      }
      }


      # Obviously not the whole thing, but have this somewhere in the http{} block
      upstream phpfpm {
      server unix:/var/run/php5-fpm.sock;
      }
      fastcgi_cache_path /var/cache/nginx levels=1:2 keys_zone=microcache_fpm:30m max_size=1000m;

      view raw

      nginx.conf

      hosted with ❤ by GitHub

  9. Hey Mark,

    Glad you’re using microcaching (thanks for your comments on Fenn’s post back in the day: http://fennb.com/microcaching-speed-your-app-up-250x-with-no-n). I’m using it as well in a high-load production system, and it works great.

    Thanks for putting your thoughts down – these are valuable even for developers who have been doing WP development for a while. The main part isn’t even all the components – I’m aware of them separately – it’s the combination of what you think worked best and why that’s valuable.

  10. I think I’d rather tear my own eyeballs out of their sockets than touch a CSS framework, but thanks for the tutorial on everything else 🙂

    I’m planning to set up my own NGINX setup soon and was googling around for information about it only last night. Great to see I have a reliable place to look back for configuration tips now 🙂

  11. Curious how you decided between relationships+CPT and taxonomy structure? Your higher level organizational levels seem to be more like taxonomy than like posts.

    1. Well, the first two levels would have been relating a taxonomy to a taxonomy, which WordPress core doesn’t really do. I wanted three levels of “things”, all directly related to each other, which is exactly the problem Posts 2 Posts was intended to solve.

    1. I’m using Nginx 1.2.0. Last year, Nacin was trying to get this type of configuration to work, and he ran into problems, like you did. But sometime between then and now, the bugs seem to have been fixed!

  12. If you leave Batcache running on the backend then Nginx fastcgi_cache honors the backend’s Cache-Control, Expires, and etc. since version 0.7.48. How do you know the Nginx cache is being hit and not Batcache? Have you tried removing Batcache to see how the Nginx cache performs? I found the Nginx cache (disk cache) has a higher CPU load than does Batcache using APC or Memcached. It also appeared that the Nginx cache was ignored and content was served instead from Batcache when Batcache was installed behind the Nginx cache.

    Could you share your experience if you had both installed at the same time?

    The configuration above left out one crucial part. In the nginx.conf file, inside the http container, fastcgi_cache_path and fastcgi_temp_path need to be defined.

    Thank you for sharing your work.

    1. How do you know the Nginx cache is being hit and not Batcache? Have you tried removing Batcache to see how the Nginx cache performs?

      Well, for one Batcache isn’t outputting its debug info into the HTML body. And secondly, I tried stopping php5-fpm. The site still works (if I’m not logged in to WordPress).

      I found the Nginx cache (disk cache) has a higher CPU load than does Batcache using APC or Memcached.

      I’ve not had any problems. But if you’re looking at the kind of concurrency that’d have Nginx seriously stress the CPU, you might look into using a Varnish layer!

      It also appeared that the Nginx cache was ignored and content was served instead from Batcache when Batcache was installed behind the Nginx cache.

      That’s odd. I’d suspect something is wrong in your Nginx config. You might look at the HTTP request/response headers being generated to see why that might be. Either it’s not being cached by Nginx, or for some reason requests are skipping the cache.

      And yes, I left a few things out of the Nginx configs… here’s a slightly more complete (albeit less aggressive) microcaching setup:


      server {
      listen 80;
      server_name example.com;
      access_log /var/log/nginx/access.log main;
      root /srv/www/example.com;
      index index.html index.htm index.php;
      location ~ \.(jpe?g|gif|png|css|bmp|js|ico)$ {
      expires 30d;
      }
      location / {
      try_files $uri $uri/ /index.php?$args;
      }
      # redirect server error pages to the static page /50x.html
      #
      error_page 500 502 503 504 /50x.html;
      location = /50x.html {
      root /var/www/nginx-default;
      }
      location ~ /\.ht[a-z]+$ {
      deny all;
      }
      location ~ \.php$ {
      # Set some proxy cache stuff
      fastcgi_cache microcache_fpm;
      fastcgi_cache_key $scheme$host$request_method$request_uri;
      fastcgi_cache_valid 200 5s;
      fastcgi_cache_use_stale updating;
      fastcgi_max_temp_file_size 1M;
      fastcgi_cache_min_uses 3; # Hit a URL 3 times before caching it
      set $no_cache_set 0;
      set $no_cache_get 0;
      set $temp_caching_exemption 0;
      if ($request_method !~ ^(GET|HEAD)$) {
      set $temp_caching_exemption 1;
      }
      if ( $temp_caching_exemption = 1 ) {
      add_header Set-Cookie "_mcnc=1; Max-Age=10; Path=/";
      }
      # Bypass cache if no-cache cookie is set
      if ( $http_cookie ~* "_mcnc" ) {
      set $no_cache_set 1;
      set $no_cache_get 1;
      }
      if ( $http_cookie ~* "comment_author_|wordpress_(?!test_cookie)|wp-postpass_" ) {
      set $no_cache_set 1;
      set $no_cache_get 1;
      }
      # fastcgi_no_cache means "Do not store this proxy response in the cache"
      fastcgi_no_cache $no_cache_set;
      # fastcgi_cache_bypass means "Do not look in the cache for this request"
      fastcgi_cache_bypass $no_cache_get;
      include /etc/nginx/fastcgi_params;
      fastcgi_index index.php;
      try_files $uri =404;
      fastcgi_pass phpfpm;
      }
      }


      # Obviously not the whole thing, but have this somewhere in the http{} block
      upstream phpfpm {
      server unix:/var/run/php5-fpm.sock;
      }
      fastcgi_cache_path /var/cache/nginx levels=1:2 keys_zone=microcache_fpm:30m max_size=1000m;

      view raw

      nginx.conf

      hosted with ❤ by GitHub

    2. On my setup I use Batcache (advanced-cache.php) and object-cache.php (Memcached plugin) both using Memcached, and Nginx caching. If I remove only Batcache, and leave object-cache.php my CPU jumps a lot. When I turn off PHP-FPM I get the stale Nginx cache when I’m not logged in same as you.

      With just Batcache and the Memcached plugin both using Memcached the CPU load is extremely low even with a ton of traffic.When I then add Nginx caching I still have no increased load until I remove just Batcache, then the load jumps by about 20 times.

      My Nginx config matches yours almost exactly, so the issue does is not related to Nginx as far as I can see. I’ve been using Nginx for about 5 years now. This is puzzling behavior. I wonder if having WP_CACHE defined in wp-config.php is causing WordPress to expect the advanced-cache.php if it also finds the object-cache.php file in wp-content, and then causes the CPU load. WordPress keeps sending visitors to PHP-FPM causing the load to go up with WP-Cache defined, advanced-cache.php removed, and object-cache.php still in wp-content, and that’s when the CPU load is high. After turning off PHP-FPM Nginx cache serves stale content with an extremely low CPU load.

    3. You were right Mark. I did have a problem with my Nginx configuration. It turns out Nginx made some changes with how they handle cookies. Once that was fixed I removed Batcache, and Nginx fastcgi_cache is handling everything with sometimes lower CPU load than Batcache + Memcached.

      I had to make some modifications to your Nginx config, and to the fastcgi.conf file, plus write a plugin to properly clear the appropriate parts of the cache when publishing and updating a post/page, to make it all work the same as Batcache did.

      Thanks for pointing me in the right direction.

    4. @ toddlahman, could you post the plugin to properly clear the appropriate parts of the cache to GitHub? Would love to see how this could be ported for general usage.

    5. @TJ Stein

      Currently testing it on a production server to make sure the relevant parts of the cache are cleared automatically before it gets released.

  13. Very cool. Some of these things were on my radar to learn and it is awesome to hear your experience using them.

    1. It’s a Linode 2048. So 2GB of RAM, and some portion of the CPU cycles. None of my sites are highly trafficked. If I ditched Minecraft, I could probably go down to a 1GB server.

  14. Forgot to mention the whole point of giving those stats – if you optimize away and set up your infrastructure right, one VPS server will go a long away. I’m using nginx+apache, microcaching, wp-supercache (I tried W3TC with memcached but keep going back every time for performance reasons), APC, and a careful selection of plugins to make sure they don’t have bugs that severely affect server load.

    1. That’s very true. It’s honestly pretty rare that a WordPress site needs multiple web servers (though it might choose to have them for redundancy).

    1. Yep, I’m using Git submodules now for WordPress (well, half of my WP installs are — some of them haven’t been converted). Not using submodules for plugins.

  15. My setup is Nginx (using your fastcgi_cache configuration), APC (to cache PHP), Batcache (to cache pages and MySQL in) Memcached, and PHP-FPM.

    Fastcgi_cache is set to cache a page after 1 hit (it’s a heavily trafficked server). Batcache is set to cache a post if it is hit 2 times in 120 seconds, and the cache_control variable is disabled so it doesn’t send those headers.

    With Memcached I can view the contents of the memory slabs, where I can see Batcache saving some cached posts, although not as many as it would without Nginx fastcgi_cache.

    I had tried serving pages using only the Nginx fastcgi_cache, but the CPU load went up 20 times higher than just using Batcache. When I use Nginx fastcgi_cache + Batcache the CPU load is about what it is using only Batcache.

    Can you post the contents of your fastcgi_params file?

    What I’d like to see is traffic hit PHP-FPM once, then have Nginx fastcgi_cache cache the post so it doesn’t hit Batcache.

    Is there a way to get Nginx fastcgi_cache to aggressively cache posts so traffic rarely, or never, hits Batcache?

    1. fastcgi_params is Nginx standard. Haven’t touched it. To get fastcgi_cache to cache more aggressively, just have it hold on to things longer. I’d skeptical that your setup is working correctly if you see load go up like that.

    2. What I have found is there can be some bleed through of traffic that doesn’t get cached right away by Nginx, so it will start to drive up the CPU load when it hits PHP-FPM. To mitigate this it is a good idea to use Batcache as a backend 2nd layer cache using either APC or Memcached (which can also be used on a single server, and has better tools to see how efficient it is than APC offers for object caching).

      Using Nginx Cache + Batcache may or may not provide an advantage over just using Batcache. I’d like to see a benchmark comparison. The Nginx Cache probably serves pages slightly faster since it never hits the PHP processor to serve pages from the cache, and that might be desirable since every ms counts in user experience.

      In many forums I have seen it mentioned to set these two settings:

      fastcgi_hide_header Set-Cookie;
      fastcgi_ignore_headers Cache-Control Expires Set-Cookie;

      If cookies are blocked there is no way to login to WordPress, since authentication requires both a username and password, and a cookie.

      To know if the Nginx cache is being hit for sure add the following setting:

      add_header X-Cache-Status $upstream_cache_status;

      Then copy and paste the url for a post into something like this site:

      http://www.gidnetwork.com/tools/gzip-test.php

      When the the response header says “x-cache-status HIT,” the post was served from the Nginx cache, and a “x-cache-status MISS” means the post was served from PHP-FPM, or Batcache depending on your settings. It’s a good idea to set Batcache to cache pages at a number higher than the Nginx fastcgi_cache_min_uses number (defaults to 1).

      The $cache_control Batcache variable should also be set to false so it doesn’t send headers. The Nginx cache will respect backend cache control headers.

      Overall the CPU usage is a few percentage points higher using Nginx Cache + Batcache, as opposed to just Batcache, but it may be serving pages slightly faster, and might be able to handle a larger concurrent traffic load, so this setup seems to be a good option. Hopefully someone will take some time to run benchmark a benchmark comparison.

      With the setup above there is no need for either WP Super Cache, or W3 Total Cache. If you need a Content Delivery Network just add the CDN Linker plugin:

      https://github.com/wmark/CDN-Linker

  16. Hi Mark, nice work and a great explanation on how you did it as well! Can I ask (or dare I ask) that this is in a way an affiliate site saw the tag in the URL code.

    I think and I really do think that you have kind of in a weird way started a new trend in the creation of Amazon affiliate sites! But worthy ones with decent product descriptions that have not just been ripped off of the Amazon. I smell an e-book coming chap!

    Would be great to get a follow up post on how this site is doing in terms of hits, revenue etc etc!

    Great work kudos to you sir, kudos indeed.

    1. The Amazon links are indeed affiliate links. I honestly don’t expect the site to make a lot of money. I’d love it if it did, but I don’t have any great hopes. If it does, I’ll be sure to say! I wanted to make a site that would be useful, with original content, not just be a run-of-the-mill affiliate link farm.

  17. Thanks for this really informative article! 🙂 I’m wanting to branch into more responsive websites and start using LESS and at least I know what to use as a solid template and CSS grid: as used by MJ himself!

    I did know of the existence of Posts 2 Posts – scribu also did an excellent taxonomy selector plugin back in the day – but had never considered how I could use it since CPTs were introduced.

    Now I think: do I really need custom post types for what I had in mind if I can just relate them using P2P?

    1. Thanks Mark for the correction 🙂 I’ve realised that scribu’s latest version of the plugin actually incorporates filters/shortcodes that helps with displaying content on the frontend of the site as well now.

      p/s: Is so chuffed to get a reply – even better that its a correction 😀

  18. Why did you use that Posts 2 Posts plugin?
    Why couldn’t just use categories and subcategories or tags?

    For example: d1 => (n1=>p11,p12,p13,…),(n2=>p21,p22,p23,…),n3

    1. I considered it. But I wanted the ability to attach metadata to the connections and to the categories (Departments, Needs) themselves. Plus, I find that the UI for Posts 2 Posts gives me more control.

  19. Hey Mark,

    Thanks for showing the more in-depth process of putting to use the topics you described in WorldCamp SF (most of it new to me!). I was sadly a cowboy coder until recently when I started using git with a simple local development setup. I hope you keep posting articles about this subject as I find resources on this topic to be rare.

    Keep up the good work!

    1. Content marketing works well if you add unique posts on a regular basis. Its amazing how some new sites can grab a lot of the search engine traffic and new subscribers by publishing useful content related to the overall topic of the site.

      Based on my own experience, 1,000 subscribers will return you $1,000 a month and 1,000 page views a day will give you another $1,000 a month.

      You’ve got a good profile that you can leverage off and there’s plenty of big sites that get around 50k views a day that would accept a guest post from you.

      Might not take long in your case to get a few thousand subscribers

  20. Was thinking about this…and trying to figure out the best way to make the cache different page types different. Thought someone following here might have some idea.

    Single Post
    Home Page
    Category/Archive Pages

    So far, I thought of moving the cache valid parameter to a series of specific location parameters outside of the php location block.

    1. My understanding is that ?args serves no purpose at all. I think it was @konstruktors who told me that. He is somewhat of a guru at this sort of thing and I think would happily fill you in on the details if you asked him.

    1. This blog is featured on the WordPress Planet feed. The WordPress Planet feed is shown in the WordPress Dashboard by default. And that feed is cached to the options table. So when you dump the table, you’ll see references to recent posts from the WordPress Planet.

  21. I’m attempting to setup a similar type of setup at the moment.

    I’m confused as to why you would bother with Batcache when you are already using NGINX for page caching. Won’t the NGINX caching simply eliminate the purpose of Batcache?

    1. Yes — in theory it should. I keep it in place just in case I ever need to disable the nginx caching (as that is less-well-tread territory). But yeah, with a good HTML caching proxy, you shouldn’t need a WordPress HTML cache (though doing fragment and object caching is still a good idea).

  22. “Next, I integrated Bootstrap, by Twitter, to handle the CSS base and the grid system.”

    I tried this, and didnt get it working. I downloaded a WP Bootstrap theme, but I would like to give it another try. What is the best way to integrate Bootstrap with _s?

  23. I found this post looking for Varnish, Nginx APC Php-Fpc, but the one time you mentioned Varnish on this page, is making me think twice about using it. You said Memcache is not good for single server WordPress, and Apc/Batcache is; how about a WordPress network? If you were to change your baby things online store to allow shoppers to create subdomain blogs, how would that change your configuration regarding caches?

  24. Great tut mark – I\’m interested in setting up something similar. There\’s a couple things that caught my eye for clarification (if you please) … (1) \”cron job that refreshes every page before it expires\” — can you expand on this? (2) the wp transition_post_status and purge functions – where did you place those and how do you enable them?. Much thanks

    1. I think I found the answer to (2) as add_action(‘transition_post_status’, ‘my_function’, 10, 3);. Still would like to what the cron job (1) would look like?

  25. Pingback: Buy Sell Swap
  26. If there is a change in the search engine algorithms, it is also up to the service provider to keep a watch on such
    activities. This means you tend to be copying it through file sharing websites like rapidshare, megaupload, hotfile, etcetera.

    You can then take a hard look at the companies that come up for that particular keyword.

    Naturally, they will flock to your competitor who does.
    If you simply copy the original manufacturer’s description, you don’t add value to your product and you certainly don’t add your personal touch that can make you stand out from your competitors. Search engines are getting smarter at detecting sites that are spamming keywords or made solely for the purpose of advertising. They make their strategy according to the audience their client is targeting. Having realized that it is practically not possible to combat with SEO Next in terms of services and offerings at this point of time, rivals have started thinking of an easier alternative of being successful in their mission. Isn’t that something
    you and your business should be a part of. What are the advantages of article submission.
    organic search, frequency of blog posts, frequency of on-page optimization,
    the relative importance of links, the use of social media, the best
    way to measure results, etc. ), watching movie trailers or other
    people’s funny animal videos. Let us assume that at this point you have 6 keywords. Think wine, and keep the URL intact. And, as more and more companies move larger portions of their advertising budgets to internet marketing, there is an even greater demand for their skills. SEO defined. Law firms face various challenges and have issues in remaining at the top in the market. When a person enters a keyword into the search box of a particular search engine such as Google, there will be scores of results showing out which are a result of a search engine algorithm devised by that particular search engine. What is the magic formula. Image Optimization is also a part of SEO services in Pune.

  27. Hey there.
    I was contemplating adding a backlink back to your website since both of our sites are based around the same topic.
    Would you prefer I link to you using your site address: http://markjaquith.
    wordpress.com/2012/05/15/how-i-built-have-baby-need-stuff/
    or blog title: How I built Have Baby. Need Stuff! | Mark on WordPress.
    Please let me know! Kudos

    1. Financially? Not really. But I get people periodically tell me that it’s helped them out or that they got pregnant and knowing they had such a resource made them feel less stressed. So on that count: success!

  28. I would like to thank you for the efforts you’ve put in penning this website. I am hoping to check out the same high-grade content from you later on as well. In truth, your creative writing abilities has motivated me to get my very own blog now 😉

  29. I’m really impressed with your writing skills as well as with the layout on your blog. Is this a paid theme or did you customize it yourself? Either way keep up the nice quality writing, it’s rare
    to see a nice blog like this one these days.

  30. It plays an important role in the production of
    Adenosine Triphosphate (ATP). Furthermore eating
    very often holds the nitrogen balance of the body high so the body doesn’t eat away on its own muscle reserves. A great way to always maintain the proper form is to leave your feet firmly planted on the ground, and always grip the bar no more than shoulder width apart.

  31. You can buy a relatively cheap one or you buy one that
    cost a couple hundred dollars. The latest Electric Tooth Brushes have
    options for brush heads and they allow different types
    and shapes of Brush heads to be used as per the user’s preferences. Every time my parents will ask me if I brushed my teeth before going to bed I would always say yes even if I didn’t.

  32. Hey excellent blog! Does running a blog like this take a lot of
    work? I have very little understanding of computer programming however I had been hoping to start my own blog in the near future.
    Anyway, should you have any ideas or tips
    for new blog owners please share. I know this is off subject nevertheless
    I simply needed to ask. Appreciate it!

  33. Howdy! I just wish to offer you a big thumbs up for your great information you’ve got right here on this post. I am coming back to your site for more soon.

Leave a comment