Is heaper threading enabled linux?

How to check is heaper threading enabled at your server.

use /proc/cpuinfo
Some OPS recommend to check HT cpu flag, but according to manuals HT – for some processors mean HyperTransport for others mean Heaper Threading

Better way to count siblings

The kernel data Linux exposes in /proc/cpuinfo will show each logical cpu with a unique processor number. A logical cpu can be a hyperthreading sibling, a shared core in a dual or quad core, or a separate physical cpu. We must look at the siblings, cpu cores and core id to tell the difference.

If the number of cores = the number of siblings for a given physical processor, then hyperthreading is OFF.

1
cat /proc/cpuinfo | grep siblings | wc -l; cat /proc/cpuinfo | grep cores | wc -l

iptables SNAT vs MASQUERADE

What is a difference and why should we use SNAT instead of MASQUERADE.

According to official documentation:

There is a specialized case of Source NAT called masquerading: it should only be used for dynamically-assigned IP addresses, such as standard dialups (for static IP addresses, use SNAT above).

With SNAT, the kernel’s connection tracking keeps track of all the connections when the interface is taken down and brought back up. For the MASQUERADE target connection will be lost.

With MASQUERADE some issues can occur if your have more than one ip on outgoing interface.

With MASQUERADE kernel determine nat outgoing ip address for every connection (it looks for interface IP) it`s rather expensive operation.

But in 99.99% cases MASQUERADE is o.k.

I Use following iptables construction to nat rear outgoing SMTP connections. (postfix started at physical server, and lxc containers relay mail to base system via ssmtp or nullmailer).

/sbin/iptables -t nat -A POSTROUTING -s 10.2.1.254/32  -o eth0 -j MASQUERADE

It`s universal chain, works great at number of servers, and you should not determine outgoing interface address. (as for -j SNAT –to-source X.X.X.X)

bind GENERATE and CDN

In our advertising network we use landing page with great number of images.

And previous team use domain static.OURDOMAIN.net for CDN. But modern browsers open from 4 to 6 simultaneous connections to one domain name. So it takes huge ammount of time to load all page.

i.e. 4 images start loading, browser waits for 1 image load complete, next image loading.
Damn not good and extremely slow :)
so use force Luke:

bind zone:

1
2
3
4
$GENERATE 1-64 static$      IN  A   1.1.1.1     ;s7
$GENERATE 1-64 static$      IN  A   2.2.2.2     ;sgr1
$GENERATE 1-64 static$      IN  A   3.3.3.3     ;sf6
$GENERATE 1-64 static$      IN  A   4.4.4.4     ;sf31

And Use something like http://static.’random(1-64)’.OURDOMAIN.net/IMAGE.PNG in application code.

We significantly speed up page loading. (up to 3 times).

my OsX bash .profile

Just to remember. I add some locale settings to avoid ????? instead of letters in console Unicode programms (such as MC)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
# MacPorts Installer addition on 2013-01-22_at_14:43:51: adding an appropriate PATH variable for use with MacPorts.
export PATH=/opt/local/bin:/opt/local/sbin:$PATH
# Finished adapting your PATH environment variable for use with MacPorts.

HISTCONTROL=ignoredups:ignorespace

# for setting history length see HISTSIZE and HISTFILESIZE in bash
HISTSIZE=2500
HISTFILESIZE=5000

export LANG=en_US.UTF-8
export LANGUAGE=en_US.UTF-8
export LC_MESSAGES=POSIX
export LC_ALL=en_US.UTF-8

export editor=/usr/bin/vim

export PS1='\[\e]0;\u@\h: \w\a\]\[\033[01;31m\]\u\[\033[01;33m\]@\[\033[01;36m\]\h \[\033[01;33m\]\w \[\033[01;35m\]\$ \[\033[00m\]'

export force_color_prompt=yes
export HISTTIMEFORMAT="%h/%d - %H:%M:%S "

    #color ls
    alias ls='ls -G'
    alias grep='grep --color=auto'
    alias fgrep='fgrep --color=auto'
    alias egrep='egrep --color=auto'
 
    alias ll='ls -alF'
    alias la='ls -A'
    alias l='ls -CF'

Linux How to increase maximum open files for running process.

We use postgres and pgbouncer as kernel DB for one huge project. Some secure requirements:

1
2
3
4
5
6
7
/etc/security/limits.conf
soft nofile 65535
hard nofile 65535

/etc/sysctl.conf
fs.file-max = 1000000
fs.inotify.max_user_watches = 1000000

Don`t forget to remove # at /etc/pam.d/login (remarked by default). And how to change nolimit on the fly.
Read more »

linux how to create a service

First of all read http://upstart.ubuntu.com/getting-started.html upstart is beautiful :) If your linux supports it.
But in some systems you should use old init style scripts :(

cp /etc/init.d/sceleton /etc/init.d/rec-runner
and edit it.

/etc/init.d/rec-runner
Read more »

rpaf for apache

Today I scan it.randomthemes.com with acunetix.
And suddenly found than
/server-status enabled with public access… WTF

1
2
3
4
5
6
7
8
/etc/apache2/mods-enabled # cat ./status.conf

<Location /server-status>
    SetHandler server-status
    Order deny,allow
    Deny from all
    Allow from 127.0.0.1 ::1
</Location>

And part of nginx config

1
2
3
  proxy_set_header   Host             $host;
  proxy_set_header   X-Real-IP        $remote_addr;
  proxy_set_header   X-Forwarded-For  $proxy_add_x_forwarded_for;

Headers looks o.k…

DAMN! I JUST FORGET TO INSTALL RPAF FOR APACHE:(

About apache rpaf
It changes the remote address of the client visible to other Apache modules when two conditions are satisfied. First condition is that the remote client is actually a proxy that is defined in httpd.conf. Secondly if there is an incoming X-Forwarded-For header and the proxy is in it’s list of known proxies it takes the last IP from the incoming X-Forwarded-For header and changes the remote address of the client in the request structure. It also takes the incoming X-Host header and updates the virtualhost settings accordingly. For Apache2 mod_proxy it takes the X-Forwared-Host header and updates the virtualhosts

Nginx purge (invalidate) cache

How to purge cache record in ngix cache via http request.
Today we install awesome nginx plugin in production https://github.com/FRiCKLE/ngx_cache_purge/
How to use it – real life example:

1. Our cache location and proxy settings:

1
proxy_cache_path /var/cache/nginx/proxy_cache_quick levels=1:2 keys_zone=quick_cache:300m max_size=2m inactive=7d;

Read more »

How to test cdn delivery speed via curl

Our company use our own CDN based on nginx caching. 7 high loaded (40 000 RPS per server) servers in 2 datacenters.
And periodically I observer some deviations in delivery time. from 0.15 to 7.5 or even 30 seconds.
We have nginx SLA module + Graphics and monitoring. But I need to test all servers for anomaly delivery time.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
#!/bin/bash

for l in ip1.x.x.x \
         ip2.x.x.x \
         ....
         ipN.x.x.x; do

echo $l;

    for i in {1..1024}; do
    curl -s -w "%{time_total} -- %{time_connect}\n" -o /dev/null --resolve it.randomthemes.com:$l http://it.randomthemes.com/favicon.ico >> ./$l.txt
    done

done

Then analyse ipN.x.x.x.txt any way you like.

cat | sort -n | tail -n 25
etc.

Have a nice day. I really like curl and hope this will help someone.

How to secure wipe file system

Before you cancel rented dedicated server, it`s good practice to secure wipe disc drives. Reboot to recovery console, and:
Use shred, Luke!

1
shred -n 0 -f -v -z /dev/sda