Category Archives: performance tuning

nginx proxy_pass and cache regexp location.

Posted by on July 10, 2015 No comments

nginx cannot proxy_pass at regexp location. I made this workaround.
Works great! Now I can cache any static data provided by backend. 🙂 from any location!

location ~* \.(gif|jpg|png|ico)$ {
      rewrite ^.(gif|jpg|png|ico) /$1 break;
      proxy_pass         http://127.0.0.1:8080;
      proxy_redirect     off;
      proxy_set_header    Host             $host;
      proxy_set_header    X-Real-IP        $remote_addr;

      proxy_set_header   X-Forwarded-For  $proxy_add_x_forwarded_for;
      client_max_body_size       150m;
      client_body_buffer_size    128k;
      proxy_connect_timeout      90;
      proxy_send_timeout         90;
      proxy_read_timeout         90;
      proxy_buffer_size          4k;
      proxy_buffers              4 32k;
      proxy_busy_buffers_size    64k;
      proxy_temp_file_write_size 64k;

      proxy_cache cache_common;
      proxy_cache_key "$host|$request_uri";
      proxy_cache_valid 200 302 301 15m;
      proxy_cache_valid 404         10s;
      proxy_cache_valid any          1m;
        }

dns requests statistics

Posted by on December 20, 2014 No comments

Today I made huge investigation, why our DNS servers serve huge ammount of traffic 5Mb\s per server. (we have only 200000000 advertising requests).
Googling gives me ultimate utility!

dnstop

In ubuntu just

aptitude update; aptitude install dnstop

dnstop works via libpcap, so no matter what DNS Server you use. (bind, dnsmasq or powerdns).
I found, that some our mail relays were misconfigured and use master DNS servers instead of local bind cachers\resolvers. And solve huge DNS traffic issue.

So one more useful utility in addition to top,htop,iftop,mytop.
🙂

iptables SNAT vs MASQUERADE

Posted by on November 25, 2014 No comments

What is a difference and why should we use SNAT instead of MASQUERADE.

According to official documentation:

There is a specialized case of Source NAT called masquerading: it should only be used for dynamically-assigned IP addresses, such as standard dialups (for static IP addresses, use SNAT above).

With SNAT, the kernel’s connection tracking keeps track of all the connections when the interface is taken down and brought back up. For the MASQUERADE target connection will be lost.

With MASQUERADE some issues can occur if your have more than one ip on outgoing interface.

With MASQUERADE kernel determine nat outgoing ip address for every connection (it looks for interface IP) it`s rather expensive operation.

But in 99.99% cases MASQUERADE is o.k.

I Use following iptables construction to nat rear outgoing SMTP connections. (postfix started at physical server, and lxc containers relay mail to base system via ssmtp or nullmailer).

/sbin/iptables -t nat -A POSTROUTING -s 10.2.1.254/32  -o eth0 -j MASQUERADE

It`s universal chain, works great at number of servers, and you should not determine outgoing interface address. (as for -j SNAT –to-source X.X.X.X)

bind GENERATE and CDN

Posted by on November 10, 2014 No comments

In our advertising network we use landing page with great number of images.

And previous team use domain static.OURDOMAIN.net for CDN. But modern browsers open from 4 to 6 simultaneous connections to one domain name. So it takes huge ammount of time to load all page.

i.e. 4 images start loading, browser waits for 1 image load complete, next image loading.
Damn not good and extremely slow 🙂
so use force Luke:

bind zone:

$GENERATE 1-64 static$      IN  A   1.1.1.1     ;s7
$GENERATE 1-64 static$      IN  A   2.2.2.2     ;sgr1
$GENERATE 1-64 static$      IN  A   3.3.3.3     ;sf6
$GENERATE 1-64 static$      IN  A   4.4.4.4     ;sf31

And Use something like http://static.’random(1-64)’.OURDOMAIN.net/IMAGE.PNG in application code.

We significantly speed up page loading. (up to 3 times).

Linux How to increase maximum open files for running process.

Posted by on October 24, 2014 No comments

We use postgres and pgbouncer as kernel DB for one huge project. Some secure requirements:

/etc/security/limits.conf
soft nofile 65535
hard nofile 65535

/etc/sysctl.conf
fs.file-max = 1000000
fs.inotify.max_user_watches = 1000000

Don`t forget to remove # at /etc/pam.d/login (remarked by default). And how to change nolimit on the fly.
Read more »

ext4 perfomance tuning

Posted by on May 17, 2014 No comments

I use following mount options.
In some projects it gives significant performance boost.

errors=remount-ro – need for hardware problem case. Because if disc remains mounted, further writing attempts can deadly damage file system. And one more case – easy monitoring. Just check via zabbix or nagios that you have no ro file system.

noatime, nodiratime – not fix access time. Double check that your applications doesn`t need this.

discard – use trim for SSD drive. In case SATA or SAS this option ignored by system.
commit, nobarrier – dangerous in case of power outage. But for my infrastructure o.k.

ext4 errors=remount-ro,noatime,nodiratime,commit=100,discard,nobarrier

And SED for fstab fixing (I use puppet, chef, fabric).

sed -r -i 's/ext4\s+defaults/ext4 errors=remount-ro,noatime,nodiratime,commit=100,discard,nobarrier/' /etc/fstab