Search This Blog

Thursday, January 6, 2011

Haproxy PostgreSQL health check patch

I have written a small patch to enable a correct PostgreSQL health check
It works similar to mysql-check with the very same parameters.

E.g.:
listen pgsql 127.0.0.1:5432
mode tcp
option pgsql-check user pgsql
server masterdb pgsql.server.com:5432 check inter 10000

The patch has been commited to 1.5 tree.
You can use it against 1.4 tree without any problems.
Moreover, I have submitted the FreeBSD PR to 1.4 port in ports tree as well.


Audit ports in jail on FreeBSD

The modified metaportaudit.sh script to perform a ports security audit in jails on FreeBSD from here .
This script audit only base jails not the cloned ones. You can grab it here.

Do not forget to edit /usr/local/etc/periodic/security/*.portaudit on about line 55
you'll want to change:

echo
echo /usr/local/sbin/portaudit -a |
su -fm "${daily_status_security_portaudit_user:-nobody}" || rc=$?

to

echo
echo /root/bin/metaportaudit.sh -a |
su -fm "${daily_status_security_portaudit_user:-nobody}" || rc=$?

Tuesday, December 28, 2010

Notes about creating FreeBSD jails

Something like that:

/bin/sh
mkdir /var/jails && cd /var/jails

for D in `ls`; do
mkdir -p $D/root $D/bin $D/dev $D/etc $D/lib $D/libexec $D/tmp $D/usr $D/var $D/home/$D
chmod o+rwxt $D/tmp
mtree -deU -f /etc/mtree/BSD.var.dist -p $D/var
done

cd /usr/src
for D in `ls /var/jails/`; do
make distribution DESTDIR=/var/jails/$D
done


cat >> /etc/rc.conf
# Jails
jail_enable="YES"
jail_mount_enable="YES"
jail_sysvipc_allow="YES"
jail_list="jail1 jail2 jail3"

## jail1
jail_jail1_name="jail1"
jail_jail1_hostname="jail1.com"
jail_jail1_ip="127.0.0.10/8"
jail_jail1_interface="lo0"
jail_jail1_rootdir="/var/jails/jail1"
jail_jail1_devfs_enable="YES"
jail_jail1_devfs_ruleset="devfsrules_jail"

## jail2
jail_jail2_name="jail2"
jail_jail2_hostname="jail2.com"
jail_jail2_ip="127.0.0.20/8"
jail_jail2_interface="lo0"
jail_jail2_rootdir="/var/jails/jail2"
jail_jail2_devfs_enable="YES"
jail_jail2_devfs_ruleset="devfsrules_jail"


## jail3
jail_jail3_name="jail3"
jail_jail3_hostname="jail3.com"
jail_jail3_ip="127.0.0.30/8"
jail_jail3_interface="lo0"
jail_jail3_rootdir="/var/jails/jail3"
jail_jail3_devfs_enable="YES"
jail_jail3_devfs_ruleset="devfsrules_jail"

^D


cat /etc/fstab.jail1
/var/jails/ruby/bin /var/jails/jail1/bin nullfs ro 0 0
/var/jails/ruby/lib /var/jails/jail1/lib nullfs ro 0 0
/var/jails/ruby/libexec /var/jails/jail1/libexec nullfs ro 0 0
/var/jails/ruby/usr /var/jails/jail1/usr nullfs ro 0 0
tmpfs /var/jails/jail1/tmp tmpfs rw,size=1G 0 0

Sunday, November 7, 2010

Simple script to monitor replication in PostgreSQL 9

Here is the simple script to monitor PostgreSQL replication on FreeBSD:

========= Cut here =========
#!/bin/sh

# Timeout in seconds
readonly timeout=600

last_chkp=`pg_controldata /var/db/pgsql/ | grep "Time of latest checkpoint" | sed "s/.*checkpoint: *//"`
last_chkp=`date -j -f "%a %b %d %T %Z %Y" "$last_chkp" "+%s"`

now=`date`
now=`date -j -f "%a %b %d %T %Z %Y" "$now" "+%s"`

if [ `expr $now - $last_chkp` -gt $timeout ]; then
echo Replication kaput!
fi

Monday, October 4, 2010

Tarsnap cleanup on FreeBSD

The simple script to preform tarsnap cleanup on FreeBSD:


#!/bin/sh

keyfile="/path/to/rw_keyfile"

# man(1) date will help (keep files for last 7 days in this example)
# we prefer to keep files in this format 2009-10-18+06:15-database as mentioned in earlier post
rd=`date -v-7d -v0H -v0M -v0S`
pd=`date -j -f "%a %b %d %T %Z %Y" "$rd" "+%s"`

for file in `tarsnap --keyfile $keyfile --list-archives | sort`; do
fd=`echo -n $file | cut -d'+' -f 1`
fd=`date -j -f "%Y-%m-%d %T" "$fd 00:00:00" "+%s"`
if [ $fd -lt $pd ]; then
echo Deleting the $file archive
tarsnap --keyfile $keyfile -d -f $file
fi
done

Script to perform tarsnapping of data on FreeBSD

Here is the script to perform tarsnapping on FreeBSD:


#!/bin/sh

# Directories to backup
DIRS=/usr/local/etc/tarsnap-backup.conf

#EXTRA_FLAGS=--dry-run

BACKUP=`date "+%Y-%m-%d+%H:%M"`
TARSNAP="/usr/local/bin/tarsnap"

# Do backups
IFS='
'
for dirs in `grep -v -e '^#.' $DIRS`; do
# Needs to set IFS to space to allow passing arguments to tarsnap
IFS=' '
echo ${dirs}
$TARSNAP $EXTRA_FLAGS -c -f $BACKUP-${dirs}
done


The conf file should look like:
archive_name1 /list /of /directories1
archive_name2 /list /of /directories2
misc /root /etc /boot/loader.conf /usr/local/etc /usr/local/opt /var/cron/tabs /var/services /var/mail /var/qmail/control /var/qmail/alias

Thursday, November 26, 2009

Get NetFlow entries from a flowd via logsock to put to PostgreSQL in Perl

Here is a small piece of code in Perl to get NetFlow entries from a flowd via logging socket (logsock) and put to a PostgreSQL database.
Beware, no sanity checking at all!!!

=== cut here ===

#!/usr/bin/perl
use IO::Socket;
use Socket;
use Flowd;
use DBI;

# Database settings
my $DBI_DRIVER = "Pg"; # or one of "Pg" "mysql" "mysqlPP"
my $DB = "netflow";
my $HOST = "localhost";
my $TABLE = "flow";
my $USER = "netflow";
my $PASS = "password";

$sock_addr="/var/run/flowd/flowd.sock";
unlink($sock_addr);

$sock = IO::Socket::UNIX->new( Local => $sock_addr, Type => SOCK_DGRAM)
or die "Can't bind to Unix Socket: $!\n";
$sock->setsockopt(SOL_SOCKET, SO_RCVBUF, 65440);

my $db = DBI->connect("dbi:$DBI_DRIVER:host=$HOST;database=$DB", $USER, $PASS)
or die "DBI->connect error: " . $DBI::errstr;

print "Started.\n";
while ($bytes = $sock->recv($input,1024)) {
$flowfields = Flowd::deserialise($input);

$recv_time = sprintf "%s.%03d",$flowfields->{recv_sec}, $flowfields->{recv_usec};
$flow_start = $recv_time + ($flowfields->{flow_start} - $flowfields->{sys_uptime_ms})/1000;
$flow_finish = $recv_time + ($flowfields->{flow_finish} - $flowfields->{sys_uptime_ms})/1000;

$sql = sprintf("INSERT INTO flows (recv_time, agent_addr, protocol_id, src_addr, src_port, dst_addr, dst_port, packets, octets, flow_start, flow_finish) VALUES (to_timestamp('%s'), '%s', '%u', '%s', '%u', '%s', '%u', '%s', '%s', to_timestamp('%s'), to_timestamp('%s'))",
$recv_time,
$flowfields->{agent_addr},
$flowfields->{protocol},
$flowfields->{src_addr},
$flowfields->{src_port},
$flowfields->{dst_addr},
$flowfields->{dst_port},
$flowfields->{flow_packets},
$flowfields->{flow_octets},
$flow_start,
$flow_finish
);
$db->do($sql) or die "db->do failed: " . $DBI::errstr;
}
1;

=== cut here ===

The SQL schema:
CREATE TABLE flows (
id serial NOT NULL,
recv_time timestamp with time zone DEFAULT now() NOT NULL,
agent_addr inet NOT NULL,
protocol_id integer NOT NULL,
src_addr inet NOT NULL,
src_port integer NOT NULL,
dst_addr inet NOT NULL,
dst_port integer NOT NULL,
packets bigint DEFAULT 0 NOT NULL,
octets bigint DEFAULT 0 NOT NULL,
flow_start timestamp with time zone NOT NULL,
flow_finish timestamp with time zone NOT NULL
);