Tweaking fastcgi-buffers

Following can help you tweak fastcgi_buffers size & numbers.

Maximum Response Size

awk '($9 ~ /200/)' access.log  | awk '{print $10}' | sort -nr | head -n 1

Please note we taking HTTP 200 OK response only into consideration.

Average Response Size

echo $(( `awk '($9 ~ /200/)' access.log | awk '{print $10}' | awk '{s+=$1} END {print s}'` / `awk '($9 ~ /200/)' access.log  | wc -l` ))

Based on above result, for rtCamp.com we found following values:

Avg. 24807
Max. 629622

So we are using:

fastcgi_buffers 32 32k;
fastcgi_buffer_size 32k;

Related: Nginx Log Parsing Examples

6 responses to “Tweaking fastcgi-buffers”

  1. Ok so you’re setting the buffer to be slightly bigger than the average response size but I currently have the same settings show above here in the example and my values are avg. 5286 max. 635 so I could get a way with a buffer of 1kb but I doubt my current values are correct since phpfpm has been restarted recently….

    So if I keep my values for now, is a bugger that is too big bad or is it just a waste of resources?

    • As far as I know, Nginx won’t allocate 1mb in one go. fastcgi_buffers 32 32k; will allocate buffers in pages of size 32k. First 32 will limit number of buffers/pages allowed to use in memory.

      We used 32k because average response size was around 24k as shown from calculation in article above.
      We limited max to 1mb because top most response was over 600k.

      Idea is to set default single buffer size for average response size so that multiple buffer allocation won’t be needed for average size. It is controlled using fastcgi_buffer_size.
      Then use fastcgi_buffers to provision additional buffers, in chunks, to hold max response in memory.

      If I missed anything, feel free to correct me.

  3. Is there a major downside to using too many buffers or buffers that are too large?

    Are these buffers duplicated per connection? Process?

    I run a server with several wordpress installs, with 2 GB of ram with nginx and php-fpm with like over a gig of ram free

    • Answers:
      1. No. Tweaking buffers will not make noticeable difference to even sites with million daily hits (unless fastcgi output way larger than default size output)
      2. I think they are allocated per request.