Thursday, January 31, 2013

rsync server daemon on Mac OS X with launchctl

(Update: Added the --no-detach option to the rsync command. Newer MacOS versions wouldn't start the daemon without it. With the added argument, it now works again in Sierra.)

There are many web pages describing how to enable the rsync daemon on Mac OS X using launchd/launchctl mechanism. But I had to use a different (and simpler) plist file in LaunchDaemons to make it work across reboots on Lion (10.7.4).

(I started by following this guide , and this very similar one. And I also read this and this. In the end, what helped me getting the plist file right was this thread. Particularly this post: "For one you have both a Program and a ProgramArguments key, when you should have only one or the other (you use Program if there is just one element to the command, or ProgramArguments if there are multiple." And this one.)

This is the .plist file I used in /Library/LaunchDaemons/org.samba.rsync.plist : 

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "">
<plist version="1.0">

This is an example /etc/rsyncd.conf file:

secrets file = /etc/rsyncd.secrets
hosts allow = *.cust.isp.tld

uid = nobody
gid = nobody
list = yes
read only = yes

path = /Users/Shared
comment = Users-Shared
uid = someuser
gid = admin
auth users = user_in_secrets

The file /etc/rsyncd.secrets looks like:


To install it:

sudo -s
chown root:wheel /etc/rsyncd.*
chmod 644 /etc/rsyncd.conf
chmod 600 /etc/rsyncd.secrets
launchctl load /Library/LaunchDaemons/org.samba.rsync.plist
launchctl start org.samba.rsync ## (this last command is probably unneeded)

To check if it is installed and running:

launchctl list | grep rsync
808  -    0x7fddb4806c10.anonymous.rsync
-    0    org.samba.rsync

ps ax | grep [r]sync
  808   ??  Ss     0:00.00 /usr/bin/rsync --daemon

rsync --stats someuser@localhost::

To remove it:

sudo launchctl unload /Library/LaunchDaemons/org.samba.rsync.plist
sudo killall rsync

For logging transfers, add

log file = /var/log/rsyncd.log
transfer logging = yes

to /etc/rsyncd.conf. And to have the log rotated, create a file like /etc/newsyslog.d/rsyncd.conf and add

# logfilename          [owner:group]    mode count size when  flags [/pid_file] [sig_num]
/var/log/rsyncd.log   644  5    5000 *     J


Labels: , , ,

Sunday, January 06, 2013

scripting disk partitionning in Linux - take 2

It is possible to use parted to script/automate disk partitioning in Linux, as described in "Command-line partitioning and formatting".

Another way is to use sgdisk from the GPT fdisk programs.

In Debian and derivatives, it can be installed with sudo apt-get install gdisk.

The current version 0.8.1 from the Ubuntu 12.04 repository would partition only the first 2TB of a 4 TB. disk. So you may need to get a more recent version from the downloads page. I got version 0.8.5 for x64, and that worked very well.

The following will create and format a single NTFS partition on an entire drive:

disk=/dev/sdb            # Make sure you got this right !!
echo "disk $disk will be completely erased."

sudo sgdisk -Z $disk
sudo sgdisk --new=0:0:-8M -t 1:0700 $disk
sudo sgdisk -p $disk
sudo mkntfs --verbose --fast --label "$label" --no-indexing --with-uuid ${disk}1

-Z removes any left-over partitions

--new=0:0:-8M creates a single partition from the start of the disk to 8MB before the end (just in case it's useful to not end on the very last sector)

-t 1:0700 sets the first partition we just created to type "Microsoft Basic Partition", which is the type we want for a simple NTFS partition. Linux would be -t 1:8300. Use sgdisk -L to get a list of partition types.

Note that for comfortable (and safer) manual partitioning, there is also cgdisk. It is like the old cfdisk, but works with new disks over 2TB.

Labels: , , , , , , , ,

Wednesday, January 02, 2013

Set up your own Dynamic DNS

The problem with external dynamic DNS services like,, etc. is that you constantly have to look after them. Either they are free, but they expire after 1 month and you have to go to their web site to re-activate your account. Or you pay for them, but then you need to take care of the payments, update the credit card info, etc. This is all much too cumbersome for something that should be entirely automated.

If you manage your own DNS anyway, it may be simpler in the long run to set-up your own dynamic DNS system.

Bind has everything needed. There is a lot of info on the Internet on how to do it, but what I found tended to be more complicated than becessary or insecure or both. So here is how I did it on a Debian 6 ("squeeze") server.

The steps described below are:

Initialize variables

To make it easier to copy/paste commands, we initialize a few variables


(In Debian, you can use grep directory /etc/bind/named.conf.options to find the correct binddir value)

For dynamic hosts, we will use a subdomain of our main zone:


Create key

Most example use the dnssec-keygen command. That would create 2 files (with ugly names): one .private and one .key (public) file. This is useless since the secret key is the same in both files, and the nsupdate method doesn't use a public/private key mechanism anyway.

There is a less-known and more appropriate command in recent distributions : ddns-confgen. By default, it will just print sample entries with instructions to STDOUT. You can try it out with:

ddns-confgen -r /dev/urandom -s $host.$zone.

The options we use here are to use an "hmac-md5" algorithm instead of the default "hmac-sha256". It simplifies things with nsupdate later. And we also specify the key name to be the same as the host's name. That way, we can use a wildcard in the "update-policy" in named.conf.local and don't need to update it every time we add a host.

ddns-confgen -r /dev/urandom -q -a hmac-md5 -k $host.$zone -s $host.$zone. | tee -a $etcdir/$zone.keys

chown root:bind   $etcdir/$zone.keys
chmod u=rw,g=r,o= $etcdir/$zone.keys

Depending on how you intend to use nsupdate, you may want to also have a separate key file for every host key. nsupdate cannot use the $zone.keys file if it contains multiple keys. So you might prefer to directly create these individual keyfiles by adding something like > $etcdir/key.$host.$zone :

ddns-confgen -r /dev/urandom -q -a hmac-md5 -k $host.$zone -s $host.$zone. | tee -a $etcdir/$zone.keys > $etcdir/key.$host.$zone

chown root:bind   $etcdir/$zone.keys $etcdir/key.*
chmod u=rw,g=r,o= $etcdir/$zone.keys $etcdir/key.*

Configure bind

Create zone file

Edit $binddir/$zone :

$TTL  3600 ; 1 hour IN SOA (
         1 ; serial (start at 1 for a dynamic zone instead of the usual date-based serial)
      3600 ; refresh by secondaries (but they get NOTIFY-ed anyway)
       600 ; retry (every 10 minutes if refresh fails)
    604800 ; expire (slaves remove the record after 1 week if they could not refresh it)
       300 ; minimum ttl for negative answers (5 minutes)


Edit /etc/bind/named.conf.local

Edit /etc/bind/named.conf.local to add :

// DDNS keys
include "/etc/bind/";

// Dynamic zone
zone "" {
    type master;
    file "/var/cache/bind/";
    update-policy {
        // allow host to update themselves with a key having their own name
        grant * self;

Reload server config

rndc reload && sleep 3 && grep named /var/log/daemon.log | tail -20

(adjust the sleep and tail values depending on the number of zones your DNS server handles, so that it has time to report any problems)


If you created individual key files, or your $zone.keys file contains only a single key, you can test like this:

host=myhost; ip=;;; keyfile=$etcdir/key.$host.$zone
echo -e "server $server\n zone $zone.\n update delete $host.$zone.\n update add $host.$zone. 600 A $ip\n send" | nsupdate -k "$keyfile"

Or, more readable and with an extra TXT record:

cat <<EOF | nsupdate -k $keyfile
server $server
zone $zone.
update delete $host.$zone.
update add $host.$zone. 600 A $ip
update add $host.$zone. 600 TXT "Updated on $(date)"

(If you get a could not read key from $keyfile: file not found error, and the file actually exists and is owned by the bind process user, you may be using an older version of nsupdate (like the version in Debian Etch). In that case, replace nsupdate -k $keyfile with nsupdate -y "$key_name:$secret" using the key name and secret found in your key file.)

Check the result:

host -t ANY $host.$zone

It should output something like descriptive text "Update on Tue Jan  1 17:16:03 CET 2013" has address
If you try to use a file with multiple keys in the -k option to nsupdate, you will get an error like this:

... 'key' redefined near 'key'
could not read key from FILENAME.keys.{private,key}: already exists


In a /etc/network/if-up.d/ddnsupdate script.

If you have setup an update CGI page on your server, you could use something like this, letting the web server use the IP address it received anyway with your request.

secret="xBa2pz6ZCGQJ5obmvmp26w==" # copy the right key from $etcdir/$zone.keys

wget -O /dev/null --no-check-certificate "https://$server/ddns/update.cgi?host=$host;secret=$secret"
Otherwise, you can use nsupdate, but you need to determine your external IP first :

secret="xBa2pz6ZCGQJ5obmvmp26w==" # copy the right key from $etcdir/$zone.keys

ip=$(wget -q -O -

cat <<EOF | nsupdate
server $server
zone $zone.
key $host.$zone $secret
update delete $host.$zone.
update add $host.$zone. 600 A $ip
update add $host.$zone. 600 TXT "Updated on $(date)"

I used a very simple myip.cgi script on the web server, to avoid having to parse the output of the various existing services which show your IP in the browser:

echo "Content-type: text/plain"
echo ""

This alternative script example uses SNMP to get the WAN IP from the cable router. It only does the update if the address has changed, and logs to syslog.


server=$(dig +short -t SOA $zone | awk '{print $1}')

ip=$( snmpwalk -v1 -m RFC1213-MIB -c public $router ipAdEntAddr | awk '!'"/$router/ {print \$4}" )

if [ -z "$ip" ]; then
 echo "Error getting wan ip from $router" 1>&2
 exit 1

oldip=$(dig +short $host.$zone)

if [ "$ip" == "$oldip" ]; then
 logger -t `basename $0` "No IP change for $host.$zone ($ip)"

cat <<EOF | nsupdate
server $server
zone $zone.
key $host.$zone $secret
update delete $host.$zone.
update add $host.$zone. 600 A $ip
update add $host.$zone. 600 TXT "Updated on $(date)"

logger -t `basename $0` "IP for $host.$zone changed from $oldip to $ip"

Web server update.cgi

An example update.cgi :


## Use nsupdate to update a DDNS zone.

## (This could be done with the Net::DNS module. It
##  would be more portable (Windows, etc.), but also
##  more complicated. So I chose the nsupdate utility
##  that comes with Bind instead.)

# "mi\", 2013

use strict;

my $VERSION = 0.2;
my $debug = 1;

my $title = "DDNS update";

my $zone     = "";
my $server   = "localhost";
my $nsupdate = "/usr/bin/nsupdate";

use CGI qw(:standard);

my $q = new CGI;

my $CR = "\r\n";

print $q->header(),
      $q->start_html(-title => $title),

if (param("debug")) {
    $debug = 1;

my $host   = param("host");
my $secret = param("secret");
my $ip     = param("ip") || $ENV{"REMOTE_ADDR"};
my $time   = localtime(time);

foreach ($host, $secret, $ip) {
    s/[^A-Za-z0-9\.\/\+=]//g; # sanitize, just in case...
    unless (length($_)) {
        die "Missing or bad parameters. host='$host', secret='$secret', ip='$ip'\n";

my $commands = qq{
server $server
zone $zone.
key $host.$zone $secret
update delete $host.$zone.
update add $host.$zone. 600 A $ip
update add $host.$zone. 600 TXT "Updated by $0 v. $VERSION, $time"

print $q->p("sending update commands to $nsupdate:"), $CR,
      $q->pre($commands), $CR;

open( NSUPDATE, "| $nsupdate" ) or die "Cannot open pipe to $nsupdate : $!\n";
print NSUPDATE $commands        or die "Error writing to $nsupdate : $!\n";
close NSUPDATE                  or die "Error closing $nsupdate : $!\n";

print $q->p("Done:"), $CR;

my @result = `host -t ANY $host.$zone`;

foreach (@result) {
    print $q->pre($_), $CR;

if ($debug) {
# also log received parameters
    my @lines;
    for my $key (param) {
        my @values = param($key);
        push @lines, "$key=" . join(", ", @values);
    warn join("; ", @lines), "\n";

print $q->end_html, $CR;


Labels: , , , , , , , ,