Monthly Archives: September 2015

How to tar.gz yesterday logs (some etl magic)

Task: need to tar yesteday logs in one file and gzip it.
Little bash code, just to save my time in future.

#!/bin/bash

src='/var/spool/etl/archive'

dt=`date --date="1 day ago" +"%Y-%m-%d"`
#create empty tar archive
tar cvf $src/$dt.tar --files-from /dev/null

for i in `ls -1 $src/*$dt* | grep -v gz | grep -v tar`; do
  tar -rf $src/$dt.tar $i
  rm -f $i
done
gzip $src/$dt.tar

video sound track merger

Few “years” ago I made part of small promo project for Nestle Russia as subcontractor.
It was promo action, website with some videos. Kids make sound track and my task was to merge user sound track and original video soundtrack.
It`s really easy to do with ffmpeg or mencoder.

BTW: ffmeg much better it works o.k. with aac codec and mp4 container.

Code is VERY VERY dirty, we had absolutely no time, but it can be useful to someone. And I save it “just to remember”.

#!/bin/bash

( #start subprocess
  # Wait for lock on /var/lock/.merger-1.lock (fd 200) for 10 seconds
  flock -x -w 3 200
  if [ "$?" != "0" ]; then echo Cannot lock!; exit 1; fi
  echo $$>>/var/lock/.merger-1.lock #for backward lockdir compatibility, notice this command is executed AFTER command bottom  ) 200>/var/lock/.myscript-1.exclusivelock.


sourcevideo="/var/www/kinder_prod/sourcevideo"
sourceaudio="/var/www/kinder_prod/audioupload"
targetdir="/var/www/kinder_prod/processedvideo"
processedaudio="/var/www/kinder_prod/processedaudio"

while true; do

if [ "$(ls -A $sourceaudio)" ]; then

  for i in `ls -1 $sourceaudio/*.wav | xargs -n1 basename`; do
  videoid=`echo $i | awk -F"--" '{print $1}'`
  audioid=`echo $i | awk -F"--" '{print $2}' | awk -F"." '{print $1}'`

  sox $sourceaudio/$i /tmp1/$i rate 44100; mv /tmp1/$i $sourceaudio/$i; chown milkslice:milkslice $sourceaudio/$i || exit 1

  sox -m $sourcevideo/$videoid.mp3 $sourceaudio/$i /tmp1/$videoid--$audioid.mp3 && \
  ffmpeg -y -i /tmp1/$videoid--$audioid.mp3 -strict experimental -acodec aac -bsf:a aac_adtstoasc /tmp1/$videoid--$audioid.aac && \
    ffmpeg -y -i /tmp1/$videoid--$audioid.aac -i $sourcevideo/$videoid.mp4 -bsf:a aac_adtstoasc -preset ultrafast -c copy $targetdir/$videoid--$audioid.mp4 || exit 1
#   mencoder -of lavf -lavfopts format=mp4 -oac copy  -fafmttag 0x706D  \
#-audiofile /tmp1/$videoid--$audioid.aac  -ovc copy $sourcevideo/$videoid.mp4 -o $targetdir/$videoid--$audioid.mp4 || exit 1
    chown milkslice:milkslice $targetdir/$videoid--$audioid.mp4
    mv -f $sourceaudio/$i $processedaudio
    rm /tmp1/$videoid--$audioid.mp3
        rm /tmp1/$videoid--$audioid.aac

    done

fi

sleep 1;
done

) 200>/var/lock/.merger-1.lock   #exit subprocess

FLOCKEXIT=$?  #save exitcode status

exit $FLOCKEXIT

And run screen with script. (alternative to upstart)

/usr/bin/screen -dm bash -c 'cd /root/merger-prod; /root/merger-prod/merger-prod.sh'

How to build dpkg from pecl

We need new mongo driver.
pecl install lastest
is not good solution, leads to chaos in system.

get the desired mongo extension tgz
http://pecl.php.net/package/mongo

aptitude install  dh-make-php php5-dev build-essential debhelper

wget http://pecl.php.net/get/mongo-1.6.11.tgz
OR pecl download mongo
dh-make-pecl --phpversion 5 --prefix php5- mongo-1.6.11.tgz
./debian/rules binary

Great Thanks to author: https://www.dotdeb.org/2008/09/25/how-to-package-php-extensions-by-yourself/
He saved my day.