Category Archives: Linux - Page 2

dns requests statistics

Posted by on December 20, 2014 No comments

Today I made huge investigation, why our DNS servers serve huge ammount of traffic 5Mb\s per server. (we have only 200000000 advertising requests).
Googling gives me ultimate utility!

dnstop

In ubuntu just

aptitude update; aptitude install dnstop

dnstop works via libpcap, so no matter what DNS Server you use. (bind, dnsmasq or powerdns).
I found, that some our mail relays were misconfigured and use master DNS servers instead of local bind cachers\resolvers. And solve huge DNS traffic issue.

So one more useful utility in addition to top,htop,iftop,mytop.
🙂

iptables SNAT vs MASQUERADE

Posted by on November 25, 2014 No comments

What is a difference and why should we use SNAT instead of MASQUERADE.

According to official documentation:

There is a specialized case of Source NAT called masquerading: it should only be used for dynamically-assigned IP addresses, such as standard dialups (for static IP addresses, use SNAT above).

With SNAT, the kernel’s connection tracking keeps track of all the connections when the interface is taken down and brought back up. For the MASQUERADE target connection will be lost.

With MASQUERADE some issues can occur if your have more than one ip on outgoing interface.

With MASQUERADE kernel determine nat outgoing ip address for every connection (it looks for interface IP) it`s rather expensive operation.

But in 99.99% cases MASQUERADE is o.k.

I Use following iptables construction to nat rear outgoing SMTP connections. (postfix started at physical server, and lxc containers relay mail to base system via ssmtp or nullmailer).

/sbin/iptables -t nat -A POSTROUTING -s 10.2.1.254/32  -o eth0 -j MASQUERADE

It`s universal chain, works great at number of servers, and you should not determine outgoing interface address. (as for -j SNAT –to-source X.X.X.X)

my OsX bash .profile

Posted by on November 3, 2014 No comments

Just to remember. I add some locale settings to avoid ????? instead of letters in console Unicode programms (such as MC)

# MacPorts Installer addition on 2013-01-22_at_14:43:51: adding an appropriate PATH variable for use with MacPorts.
export PATH=/opt/local/bin:/opt/local/sbin:$PATH
# Finished adapting your PATH environment variable for use with MacPorts.

HISTCONTROL=ignoredups:ignorespace

# for setting history length see HISTSIZE and HISTFILESIZE in bash
HISTSIZE=2500
HISTFILESIZE=5000

export LANG=en_US.UTF-8
export LANGUAGE=en_US.UTF-8
export LC_MESSAGES=POSIX
export LC_ALL=en_US.UTF-8

export editor=/usr/bin/vim

export PS1='\[\e]0;\u@\h: \w\a\]\[\033[01;31m\]\u\[\033[01;33m\]@\[\033[01;36m\]\h \[\033[01;33m\]\w \[\033[01;35m\]\$ \[\033[00m\]'

export force_color_prompt=yes
export HISTTIMEFORMAT="%h/%d - %H:%M:%S "

    #color ls
    alias ls='ls -G'
    alias grep='grep --color=auto'
    alias fgrep='fgrep --color=auto'
    alias egrep='egrep --color=auto'
 
    alias ll='ls -alF'
    alias la='ls -A'
    alias l='ls -CF'

Linux How to increase maximum open files for running process.

Posted by on October 24, 2014 No comments

We use postgres and pgbouncer as kernel DB for one huge project. Some secure requirements:

/etc/security/limits.conf
soft nofile 65535
hard nofile 65535

/etc/sysctl.conf
fs.file-max = 1000000
fs.inotify.max_user_watches = 1000000

Don`t forget to remove # at /etc/pam.d/login (remarked by default). And how to change nolimit on the fly.
Read more »

linux how to create a service

Posted by on September 24, 2014 No comments

First of all read http://upstart.ubuntu.com/getting-started.html upstart is beautiful 🙂 If your linux supports it.
But in some systems you should use old init style scripts 🙁

cp /etc/init.d/sceleton /etc/init.d/rec-runner
and edit it.

/etc/init.d/rec-runner
Read more »

Nginx purge (invalidate) cache

Posted by on June 19, 2014 No comments

How to purge cache record in ngix cache via http request.
Today we install awesome nginx plugin in production https://github.com/FRiCKLE/ngx_cache_purge/
How to use it – real life example:

1. Our cache location and proxy settings:

proxy_cache_path /var/cache/nginx/proxy_cache_quick levels=1:2 keys_zone=quick_cache:300m max_size=2m inactive=7d;

Read more »

How to test cdn delivery speed via curl

Posted by on June 8, 2014 No comments

Our company use our own CDN based on nginx caching. 7 high loaded (40 000 RPS per server) servers in 2 datacenters.
And periodically I observer some deviations in delivery time. from 0.15 to 7.5 or even 30 seconds.
We have nginx SLA module + Graphics and monitoring. But I need to test all servers for anomaly delivery time.

#!/bin/bash

for l in ip1.x.x.x \
         ip2.x.x.x \
         ....
         ipN.x.x.x; do

echo $l;

    for i in {1..1024}; do
    curl -s -w "%{time_total} -- %{time_connect}\n" -o /dev/null --resolve it.randomthemes.com:443:$l https://it.randomthemes.com/favicon.ico >> ./$l.txt
    done

done

Then analyse ipN.x.x.x.txt any way you like.

cat | sort -n | tail -n 25
etc.

Have a nice day. I really like curl and hope this will help someone.

ext4 perfomance tuning

Posted by on May 17, 2014 No comments

I use following mount options.
In some projects it gives significant performance boost.

errors=remount-ro – need for hardware problem case. Because if disc remains mounted, further writing attempts can deadly damage file system. And one more case – easy monitoring. Just check via zabbix or nagios that you have no ro file system.

noatime, nodiratime – not fix access time. Double check that your applications doesn`t need this.

discard – use trim for SSD drive. In case SATA or SAS this option ignored by system.
commit, nobarrier – dangerous in case of power outage. But for my infrastructure o.k.

ext4 errors=remount-ro,noatime,nodiratime,commit=100,discard,nobarrier

And SED for fstab fixing (I use puppet, chef, fabric).

sed -r -i 's/ext4\s+defaults/ext4 errors=remount-ro,noatime,nodiratime,commit=100,discard,nobarrier/' /etc/fstab

nginx error page depends on user browser language

Posted by on March 12, 2014 No comments

Task – return different pages depends on user browser language.
i.e. different html if backend return error. And for domain it.randomthemes.com always return english error page on backend error.

nginx.conf

map $http_accept_language $lang {
    default en;
    ~ru ru;
          }

Server context:

set $ep /50x.html; #default error page

if ( $host ~* it.randomthemes.com ) {
set $ep /50x.en.html;
}


if ( $lang ~* en ) {
set $ep /50x.en.html;
}

error_page  503          /dinner.html;
error_page  500 502 504  $ep;
error_page  400          /400.html;

nginx 301 redirect entire domain

Posted by on March 6, 2014 No comments

Task – redirect all requests from old-domain.com to new-domain.com
use nginx, luke! It`s simple.

server {
        server_name old-domain.com www.old-domain.com;
        rewrite ^/(.*)$ http://new-domain.com/$1 permanent;
}