Category Archives: FreeBSD

How to delete files without big iowait

Posted by on July 23, 2015 No comments

I know 2 ways, tested in high loaded production.

if scheduler support ionice (on some systems makes LA)

 # ionice -c 3 nice -n 20 find  /DIRECTORY -type f -delete

Just ajust sleep time, according to your system LA

while true; do find /DIRECTORY/ -type f -print  -delete -quit; sleep 0.01; done

linux how to create a service

Posted by on September 24, 2014 No comments

First of all read http://upstart.ubuntu.com/getting-started.html upstart is beautiful 🙂 If your linux supports it.
But in some systems you should use old init style scripts 🙁

cp /etc/init.d/sceleton /etc/init.d/rec-runner
and edit it.

/etc/init.d/rec-runner
Read more »

How to test cdn delivery speed via curl

Posted by on June 8, 2014 No comments

Our company use our own CDN based on nginx caching. 7 high loaded (40 000 RPS per server) servers in 2 datacenters.
And periodically I observer some deviations in delivery time. from 0.15 to 7.5 or even 30 seconds.
We have nginx SLA module + Graphics and monitoring. But I need to test all servers for anomaly delivery time.

#!/bin/bash

for l in ip1.x.x.x \
         ip2.x.x.x \
         ....
         ipN.x.x.x; do

echo $l;

    for i in {1..1024}; do
    curl -s -w "%{time_total} -- %{time_connect}\n" -o /dev/null --resolve it.randomthemes.com:443:$l https://it.randomthemes.com/favicon.ico >> ./$l.txt
    done

done

Then analyse ipN.x.x.x.txt any way you like.

cat | sort -n | tail -n 25
etc.

Have a nice day. I really like curl and hope this will help someone.

nginx error page depends on user browser language

Posted by on March 12, 2014 No comments

Task – return different pages depends on user browser language.
i.e. different html if backend return error. And for domain it.randomthemes.com always return english error page on backend error.

nginx.conf

map $http_accept_language $lang {
    default en;
    ~ru ru;
          }

Server context:

set $ep /50x.html; #default error page

if ( $host ~* it.randomthemes.com ) {
set $ep /50x.en.html;
}


if ( $lang ~* en ) {
set $ep /50x.en.html;
}

error_page  503          /dinner.html;
error_page  500 502 504  $ep;
error_page  400          /400.html;

nginx 301 redirect entire domain

Posted by on March 6, 2014 No comments

Task – redirect all requests from old-domain.com to new-domain.com
use nginx, luke! It`s simple.

server {
        server_name old-domain.com www.old-domain.com;
        rewrite ^/(.*)$ http://new-domain.com/$1 permanent;
}

Mysql can do partitioning on the fly :)

Posted by on July 23, 2013 No comments

Mysql tuning in action…
Yesterday me, and our development team made some tuning of one old project we have couple of tables with 1-10 million records.
It`s NOT BIGDATA, but application makes huge writes to this table.
Only current date records, but you understand partitioning affects index size.
Table can be partitioned by date field. So how to do it:

  1. make sure that dt not null.
  2. recreate primary key. Field for partition should be part of primary key.
  3. create partition for old years and partition for every month.

Read more »

Mysql slave lag monitoring

Posted by on May 16, 2013 2 comments

Everybody know

SHOW SLAVE STATUS;

Also everybody know that ‘Seconds_Behind_Master’ shows difference in seconds between the slave SQL thread and the slave I/O thread.

Sometime it shows nonsense, and if you build monitoring It is not good practice to use ‘Seconds_Behind_Master’
Example from real life:
If replication becomes stalled due to connectivity problems, Seconds_Behind_Master shows 0 while replica is far away from master, changing timeout values not help. 🙁 i mean:

slave_net_timeout=300

So we implement following monitoring – every 3 second write current timestamp at master.
and check replica delay.
Or You can use percona heartbeat, It do almost the same.
Read more »

How to capture bad email addresses in mass mail.

Posted by on April 16, 2013 No comments

We have huge project with a great number of registered users. They receive notification via email, when actions occur. (new comment, gift, some other activity). Project targeting – Russia and former USSR In Russia some free email hosting providers like mail.ru delete user account after 2-3 years of inactivity.
So we have now 2-3000000 users with bad email address.
How to find them and remove from mailing list:

According to RFC 5321 (smtp rfc) in case of wrong email in rcpt to, server should return 550 no such user.
Read more »

Squid with simple authentication

Posted by on November 15, 2012 No comments

Typical task:
We order server at India, for development needs. Need to setup web proxy with authentication. Use squid, Luke!

#htpasswd -c  /etc/squid/passwd username
#htpasswd  /etc/squid/passwd

squid.conf

forwarded_for off #don`t show user real IP at HTTP headers
auth_param basic program /usr/lib64/squid/ncsa_auth /etc/squid/passwd
<span class="zbench-more-link"> <a href="https://it.randomthemes.com/2012/11/15/squid-with-simple-authentication/#more-193" class="more-link">Read more &raquo;</a></span>

qmail maillists and Jira messages delivery issues.

Posted by on October 29, 2012 No comments

We had interesting issue with jira and qmail.
No mail comes to mail list group.

Not trivial email routing probles, message for mail list comes to ezmlm, but ezmlm rejects messages with tag 

Precedence: bulk

Read more »