It’s the second week of running WordPress, so I’ve been trialing a few plugins… Suddenly, about half the time I do anything I get “Apologies, but the page you requested could not be found.” The page loads fine the second time with a simple page refresh. The site is brand new, so there shouldn’t be any traffic. So what gives?Getting on-chat with DreamHost support (which itself is a great service)… five minutes later they had tracked down that my WordPress was exceeding the user RAM limit on their shared-hosting. The system process watcher ProcWatch was killing off the connection. Here is the log message supplied by DreamHost:
2011-03-10 03:55:15 procwatch2 INFO: Process(pid=1755, name='php5.cgi', uid=xxxxx(10537396), tty=None, cpu=2.0, rss=42516.0, vsize=148964.0): kill for total RAM
And one of many messages from file: logs/http/error.log:
[Thu Mar 10 04:01:59 2011] [error] [client 18.104.22.168] Premature end of script headers: symposium_bar_functions.php, referer: http://blog.openinworld.com/2011/03/morphic-performance-testing
Keeping the system balanced for all tusers is fair – but how does that affect me directly? Am I able to work with this? WordPress appears to put a relatively heavy strain on server memory. I did not anticipate it the memory limit could be exceeded so easily, but I guess that is the reality of shared hosting.
DreamHost refrains from specifying what the RAM limit is. Perhaps its dynamic, or a marketing issue, or it changes with time. I’ve seen speculation elsewhere of it being 100MB, 120MB & 200MB in posts from 2006, 2010 & 2007. From a shell with the `top` command I monitored my live processes. How to Profile Memory in Linux describes the columns below. Specifically we are interested in the resident memory. Three WordPress php5.cgi listeners are shown, each idling comfortably around 45MB. That might peg the RAM limit somewhere between 150MB to 200MB.
Interestingly, comparing this to the `top` snapshots at DreamHost WordPress Optimization I note the RES memory there is around 7.5MB versus my 35MB. While part of this may be be due to general software bloat between versions, I suspect it may mostly be from RAM usage doubling between 32-bit and 64-bit operating systems. Hopefully the RAM Limit was doubled accordingly to compensate.
So what to do? The simple suggestion from DreamHost was to “disable all your WordPress plugins.“ That seemed a bit extreme, but will at least help to understand the issue. Before anything else, I went looking for a WordPress plugin to show plug-in memory usage. I installed TPC! Memory Usage (TPCMU), which immediately started firing the following email at me several times a second. Note that the 32MB is a user defined soft limit.
WordPress memory usage exceeded 32 MB WordPress peak memory usage: 34.85 MB Number of database queries: 36
I then removed every plugin except TPCMU , and `top` then reported RAM usage 10MB better per listener – at around 35MB each. TPCMU reported WordPress peak memory usage as 22.4MB.
To observe the effect of reinstating each plugin, I tracked the Memory Usage displayed in the footer of the Admin > Plugins > Active Plugins page. The results are tabulated below. Marked in red are the plugins I ultimately removed, judging that functionality gain not worth the pain of required RAM. I believe that remaining plugins are a reasonable balance of functionality versus memory usage – particularly since two thirds of that is aimed at performance and security, with an overall benefit to other users of the server.
|TPC! Memory Usage|
|WP Super Cache||2.3|
|SI Captacha Anti-Spam||0.5|
|After the Deadline||0.2|
|Subscribe To Comments Reloaded||0.3|
|WP Super Mailer||0.5|
|WP Forum Server||1.6|
|Open Web Analytics||0.06|
|WordPress Database Backup||0.6|
|Broken Link Checker||3.8|
|Extended Comment Options||0.3|
|WP Stats Dashboard||1.8|
TPCMU now reported 28.5MB and `top` showed php.cgi at 41MB.
I saw several tips suggesting that rather than just deactivating plugins, deleting them saves more space. I had 42 Inactive plug-ins, including those just deactivated and others I had tried and not liked. Deleting all of them only reduced the TPCMU report down 0.1MB to 28.4MB – not much really.
DreamHost has some community information available. DreamHost WordPress_Performance discusses page load speed. It recommends a few caching plugins and the use of FastCGI. Faster processing reduces overall memory usage somewhat due to there being less concurrent php.cgi processes.
DreamHost WordPress Optimization discusses four site configurations in relation to memory usage. It also recommends caching. However it recommends NoFastCGI saying:
[Cache + NoFastCGI] has great performance while having the lowest overall resource hit on the server. It should always be using the least amount of memory and CPU time that it possibly can [...] whereas in a FastCGI configuration the number configured to spawn would remain open whether or not any requests were coming in.
When idle, FastCGI processes sit around continuing to consume memory. Hence the cumulative memory usage over time would be greater than NoFastCGI. That could be considered a greater load on the server. However FastCGI provides a more steady and predictable load, which is more desirable for tuning that one that fluctuates. FastCGI should handle peak loads more gracefully. NoFastCGI has a fairly important proviso…
The only time [Cache + NoFastCGI] might be less efficient is if a large number of un-cached pages are accessed simultaneously as a number of php5.cgi processes would spawn to handle those.
…in which case ProcWatch would assumedly start killing off random connections for exceeding the user RAM limit. For a high traffic site (wishful thinking!) FastCGI should handle each cache-miss faster, thus requiring less overall RAM with less chance of connections being killed. Ideally, the number of spawned FastCGI processes could be limited to avoid a “Kill For RAM” scenario. However I haven’t been able to determine how to limit the number of FastCGI processes spawned.
I might have to keep an eye on FastCGI though, since this WordPress: PHP CGI vs PHP FCGI post points to some issues with FastCGI – that I have not experienced that so far.
In conclusion, Dreamhost offer shared hosting that is unlimited in many of its (advertised) parameters. However overall service capability is limited by the tightest constraint – which in this case is RAM – the one they don’t advertise (and neither does any other shaed host I’ve seen.) To work within this limit and resolve the “kill for total RAM” issue, I have:
- investigated the memory usage of WordPress plugins
- reduced the number WordPress plug-ins
- activated an application-aware cache “Supercache”
- continued to use FastCGI (which is the DreamHost default)
In the end, these are only simple things – but I’ve learnt a lot along the way. I was considering further action with some of the methods from the WordPress Optimization Bible, but my error.log file has been clear for a few days, so I’ll leave those for later.