Importing root certificates into Firefox and Thunderbird

This is ridiculously complicated and makes me wonder whether I should just drop Firefox in Windows and go back to IE.

The problem:

How to automatically pre-import your self-signed certification authority into all user profiles for Firefox and Thunderbird.

The solution:

You need the Mozilla certutil utility (not the Microsoft certutil.exe).

In Windows, you would need to compile nss tools or use some ancient hard to find Windows binary to get it. But all my user profiles are on a Samba server, so it was much easier to do it on the server, with the added benefit of having Bash and not needing to struggle with the horrible cmd.exe.

First install the tools. In Debian, it would be:

apt-get install libnss3-tools

Then adapt this long command to your paths:

find /path/to/users-profiles -name cert8.db -printf "%h\n" | \
while read dir; do \
  certutil -A -n "My Own CA" -t "C,C,C" -d "$dir" -i "/path/to/my_own_cacert.cer"; \
done

(-printf "%h\n" prints just the directory, without the file name, one per line. That is fed to the $dir variable needed in the certutil command. The -n option is a required nickname for the certificate. -t "C,C,C" is what will make you accept any certificate signed by this CA you are importing).

See also: the certutil documentation, and a better explanation of the trust arguments (-t option).

Postgresql 8.4 LDAP unknown authentication

The syntax to specify LDAP authentication in pg_hba.conf seems to have changed between version 8.3 and 8.4. At least in Debian Squeeze, where I have both installed.

For 8.3, I had a line like this in pg_hba.conf:

host  all  all  192.168.44.0/24  ldap "ldap://localhost/dc=mydomain,dc=lan;uid=;,ou=People,dc=mydomain,dc=lan"

When trying to start 8.4 with the same line, I was getting

# /etc/init.d/postgresql restart 8.4
Restarting PostgreSQL 8.4 database server: main
The PostgreSQL server failed to start. Please check the log output: [...]
invalid authentication method ":ldap" [...]
FATAL: could not load pg_hba.conf ... failed!

The new syntax which works in 8.4 is

host  all  all  192.168.44.0/24  ldap ldapserver=localhost ldapprefix="uid=" ldapsuffix=", ou=People, dc=mydomain, dc=lan"

Etch to Lenny trouble with libxml2

While upgrading a few Debian Etch systems to Lenny, I had a lot of trouble which showed up like this:

symbol lookup error: /usr/lib/libxml2.so.2: undefined symbol: gzopen64

The real cause seems to have been that I had 2 libz libraries installed:

 # /sbin/ldconfig -pNX | grep libz
 libz.so.1 (libc6) => /lib/libz.so.1
 libz.so.1 (libc6) => /usr/lib/libz.so.1

So the solution was quite simple:

 # rm /lib/libz.so.1*

That's all that was needed to get rid of the mountain of dpkg errors, and continue the upgrades following the Debian guide. The next upgrade to Squeeze went smoothly.

For the benefit of Google searchers, here is a full error listing:

 Unpacking replacement shared-mime-info ...
update-mime-database: symbol lookup error: /usr/lib/libxml2.so.2: undefined symbol: gzopen64
dpkg: warning - old post-removal script returned error exit status 127
dpkg - trying script from the new package instead ...
update-mime-database: symbol lookup error: /usr/lib/libxml2.so.2: undefined symbol: gzopen64
dpkg: error processing /var/cache/apt/archives/shared-mime-info_0.30-2_i386.deb (--unpack):
 subprocess new post-removal script returned error exit status 127
update-mime-database: symbol lookup error: /usr/lib/libxml2.so.2: undefined symbol: gzopen64
dpkg: error while cleaning up:
 subprocess post-removal script returned error exit status 127
Preparing to replace libgnomevfs2-common 1:2.14.2-7 (using .../libgnomevfs2-common_1%3a2.22.0-5_all.deb) ...
Unpacking replacement libgnomevfs2-common ...
gconftool-2: symbol lookup error: /usr/lib/libxml2.so.2: undefined symbol: gzopen64
dpkg: warning - old post-removal script returned error exit status 127
dpkg - trying script from the new package instead ...
gconftool-2: symbol lookup error: /usr/lib/libxml2.so.2: undefined symbol: gzopen64
dpkg: error processing /var/cache/apt/archives/libgnomevfs2-common_1%3a2.22.0-5_all.deb (--unpack):
 subprocess new post-removal script returned error exit status 127
gconftool-2: symbol lookup error: /usr/lib/libxml2.so.2: undefined symbol: gzopen64
dpkg: error while cleaning up:
 subprocess post-removal script returned error exit status 127
Preparing to replace libgnome2-common 2.16.0-2 (using .../libgnome2-common_2.20.1.1-1_all.deb) ...
Unpacking replacement libgnome2-common ...
gconftool-2: symbol lookup error: /usr/lib/libxml2.so.2: undefined symbol: gzopen64
dpkg: warning - old post-removal script returned error exit status 127
dpkg - trying script from the new package instead ...
gconftool-2: symbol lookup error: /usr/lib/libxml2.so.2: undefined symbol: gzopen64
dpkg: error processing /var/cache/apt/archives/libgnome2-common_2.20.1.1-1_all.deb (--unpack):
 subprocess new post-removal script returned error exit status 127
gconftool-2: symbol lookup error: /usr/lib/libxml2.so.2: undefined symbol: gzopen64
dpkg: error while cleaning up:
 subprocess post-removal script returned error exit status 127
Errors were encountered while processing:
 /var/cache/apt/archives/shared-mime-info_0.30-2_i386.deb
 /var/cache/apt/archives/libgnomevfs2-common_1%3a2.22.0-5_all.deb
 /var/cache/apt/archives/libgnome2-common_2.20.1.1-1_all.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)
A package failed to install.  Trying to recover:
dpkg: dependency problems prevent configuration of libbonoboui2-0:
 libbonoboui2-0 depends on libglade2-0 (>= 1:2.6.1); however:
  Version of libglade2-0 on system is 1:2.6.0-4.
 libbonoboui2-0 depends on libgtk2.0-0 (>= 2.12.0); however:
  Version of libgtk2.0-0 on system is 2.8.20-7.
dpkg: error processing libbonoboui2-0 (--configure):
 dependency problems - leaving unconfigured
dpkg: dependency problems prevent configuration of libgnomecanvas2-0:
 libgnomecanvas2-0 depends on libglade2-0 (>= 1:2.6.1); however:
  Version of libglade2-0 on system is 1:2.6.0-4.
 libgnomecanvas2-0 depends on libgtk2.0-0 (>= 2.12.0); however:
  Version of libgtk2.0-0 on system is 2.8.20-7.
dpkg: error processing libgnomecanvas2-0 (--configure):
 dependency problems - leaving unconfigured
dpkg: dependency problems prevent configuration of libgail18:
 libgail18 depends on libgtk2.0-0 (>= 2.12.0); however:
  Version of libgtk2.0-0 on system is 2.8.20-7.
dpkg: error processing libgail18 (--configure):
 dependency problems - leaving unconfigured
dpkg: dependency problems prevent configuration of libgail-common:
 libgail-common depends on libgail18 (>= 1.9.1); however:
  Package libgail18 is not configured yet.
 libgail-common depends on libgtk2.0-0 (>= 2.12.0); however:
  Version of libgtk2.0-0 on system is 2.8.20-7.
dpkg: error processing libgail-common (--configure):
 dependency problems - leaving unconfigured
dpkg: dependency problems prevent configuration of libgnomevfs2-extra:
 libgnomevfs2-extra depends on libgnomevfs2-common (>= 1:2.22); however:
  Package libgnomevfs2-common is not installed.
 libgnomevfs2-extra depends on libgnomevfs2-common (<< 1:2.23); however:
  Package libgnomevfs2-common is not installed.
dpkg: error processing libgnomevfs2-extra (--configure):
 dependency problems - leaving unconfigured
dpkg: dependency problems prevent configuration of libgnomevfs2-0:
 libgnomevfs2-0 depends on libgnomevfs2-common (>= 1:2.22); however:
  Package libgnomevfs2-common is not installed.
 libgnomevfs2-0 depends on libgnomevfs2-common (<< 1:2.23); however:
  Package libgnomevfs2-common is not installed.
dpkg: error processing libgnomevfs2-0 (--configure):
 dependency problems - leaving unconfigured
Setting up libgnome-keyring0 (2.22.3-2) ...
dpkg: dependency problems prevent configuration of libgnome2-0:
 libgnome2-0 depends on libgnomevfs2-0 (>= 1:2.17.90); however:
  Package libgnomevfs2-0 is not configured yet.
 libgnome2-0 depends on libgnome2-common (>= 2.20); however:
  Package libgnome2-common is not installed.
 libgnome2-0 depends on libgnome2-common (<< 2.21); however:
  Package libgnome2-common is not installed.
dpkg: error processing libgnome2-0 (--configure):
 dependency problems - leaving unconfigured
dpkg: dependency problems prevent configuration of xserver-xorg-input-mouse:
 xserver-xorg-input-mouse depends on xserver-xorg-core (>= 2:1.4); however:
  Version of xserver-xorg-core on system is 2:1.1.1-21etch5.
dpkg: error processing xserver-xorg-input-mouse (--configure):
 dependency problems - leaving unconfigured
dpkg: dependency problems prevent configuration of xserver-xorg-input-kbd:
 xserver-xorg-input-kbd depends on xserver-xorg-core (>= 2:1.4); however:
  Version of xserver-xorg-core on system is 2:1.1.1-21etch5.
dpkg: error processing xserver-xorg-input-kbd (--configure):
 dependency problems - leaving unconfigured
Errors were encountered while processing:
 libbonoboui2-0
 libgnomecanvas2-0
 libgail18
 libgail-common
 libgnomevfs2-extra
 libgnomevfs2-0
 libgnome2-0
 xserver-xorg-input-mouse
 xserver-xorg-input-kbd

Command-line partitioning and formatting

Automatic non-interactive formatting in Linux is possible with parted.

The following creates 1 single ext3 partition on an entire disk. Of course, if you assign the wrong disk to the $disk variable, it will be a bad day...

# Select the disk device and chose a label for the partition
disk=/dev/sdx; label=my_part_label

I have added sleep commands, so I can just copy/paste the whole thing and still have a chance to Ctrl-C if I change my mind at the last second.

# print the current partition(s) state
parted $disk print ; sleep 10

# create a standard ("msdos") partition table
parted -a optimal $disk mklabel msdos ; sleep 5

# create the partition, starting at 2048 sectors which may be better
# with newer disks
parted -a optimal -- $disk unit s mkpart primary ext3 "2048" "-1" ; sleep 5

# format it
mke2fs -j -v -L "$label" ${disk}1 && echo "OK. That's it"

Windows installers options for silent installs

Different installers use different command-line options for silent or unattended installs. Since I had started these notes, I have found a good overview on unattended.sourceforge.net.

Inno Setup

can be identified with the "Inno Setup" string appearing in various places in the installer's .exe. The options are described here. The most useful ones are:

• /SAVEINF="filename"
  Save installation settings to the specified file.
• /LOADINF="filename"
  Load the settings from the specified file after having checked the command line.
• /SP-
  Disables the This will install... Do you wish to continue? prompt at the beginning of Setup.
• /SILENT, /VERYSILENT
  When Setup is silent the wizard and the background window are not displayed but the installation progress window is. When a setup is very silent this installation progress window is not displayed. Everything else is normal so for example error messages during installation are displayed.
• /DIR="x:\dirname"
• /LANG=language
  Specifies the language to use. language specifies the internal name of the language as specified in a [Languages] section entry.

Nullsoft's NSIS

can be identified with the "NSIS" string appearing in various places in the installer's .exe. The options are described here, but there seem to be only 2 useful ones:

• /S
  Silent installation
• /D=C:\Bla
  Set output folder

Mac and OpenLDAP: Local homes for network users

I wanted a Mac to authenticate users against our Debian OpenLDAP server, but to create a local home directory on the Mac (see here for more details). The usual configuration for network users on the Mac is to mount their homes from the server over NFS. There are many excellent instructions on the net on how to do that. But finding help on how to have them use a local home instead was much more difficult.

It turns out it can be done very simply, by disabling one line in /etc/auto_master on the Mac. By default, it contains +auto_master, which tells the Mac's automounter to look for an automount map in LDAP. If this line is disabled, the Mac will create a local home for network users the first time they log in. Since our userHomes in LDAP are defined as /home/username, the Mac home is created under /home instead of /Users, which is fine.

So for such a setup, you do NOT need to import an Apple schema into your LDAP directory. (That was quite a hassle because you need to tweak the original schema which is not quite kosher; but it was unnecessary).

All you need to do is comment out this single line in /etc/auto_master to make it

#+auto_master  # Use directory service

Or copy/paste this:

sudo perl -i.orig -pe 's/^(\+auto_master.*)/## $1/' /etc/auto_master

Kill the Final Cut registration screen

I came across this nicely detailed post explaining how to get rid of the forced registration screen of Final Cut Pro/Studio, which always pops up when you really don't want to be bothered with this idiocy.

But I felt the solution was worse than the problem. It involved far too much clicking around for my taste. And you need the Property List Editor. You only have that once you have installed over 1 GB (!!) of developer tools. If you can remember where you put your OS X disk, that is.

Surely, there must be a better way to do it, by just copying a command from some web page and pasting it into Terminal?

It turned out to be 3 commands. And getting them right was much worse than the solution I din't like. You need your machine ID, which is in an XML file that defaults read doesn't want to read. And in that file it is encoded in Base64. You need to put this ID into a property list file as data. That can be done with defaults write, but the data needs to be in hex. I should just have registered, I guess...

Anyway, the detailed explanations are in the link of the first sentence, and the 3 ridiculous commands to paste into Terminal are here:

id=$(perl -MMIME::Base64 -ne '/^\s+(\S{64})\s*$/ && print unpack("H*",decode_base64($1));' "/Library/Application Support/ProApps/Final Cut Studio System ID"|tail -1)
sudo defaults write /Library/Preferences/com.apple.RegFinalCutStudio "{ AECoreTechRegister=1; AECoreTechRegSent=1; }"
sudo defaults write /Library/Preferences/com.apple.RegFinalCutStudio AECoreTechRegInfo -data "$id"

Moving IMAP Maildir to another user

A little recipe to move a user's IMAP mails to another user. (Tested on the Courier IMAP server on Debian).

Useful in situations like John leaving the company and Bob needing to have access to John's old emails.

olduser=john; newuser=bob
maildirmake -f $olduser /home/$newuser/Maildir/
cd /home/$olduser/Maildir/
for d in * ; do \
    cp -pr "$d" "/home/$newuser/Maildir/.$olduser/"; \
done
echo "INBOX.$olduser"  >>/home/$newuser/Maildir/courierimapsubscribed
for d in .??*; do \
    cp -pr "$d" "/home/$newuser/Maildir/.$olduser$d"; \
    echo "INBOX.$olduser$d" >>/home/$newuser/Maildir/courierimapsubscribed; \
done
chown -R $newuser /home/$newuser/Maildir

(Beware that if John had a folder with a one-letter name, that one will not be copied. It's because "for d in .*" would do a mess trying to copy "." and "..". So line 6 uses "for d in .??*" instead.)

Quicktime Error 2041 an invalid sample description was found in the movie

After converting an MPEG-TS file to mp4, I had trouble with some players. Particularly, the latest Quicktime (7.6.5) wouldn't play it, saying "Error -2041: an invalid sample description was found in the movie".

Searching Google on this error turned up thousands of results, but nothing I read was directly useful. The main "solution" proposed was "rename the .mp4 file to .3gp". That seems to work in some cases for .3gp phone files which the phone happens to name .mp4.

The only solution which worked for me was to use mp4creator with the " -aac-profile=4" option.

In case anybody wants to know all the details, they are below. If you happen to know the real source of the problem and/or a version of MP4Box or ffmpeg which fixes the problem, please leave a comment.

The source file was an MPEG-TS containing h264 video and aac audio. ffmpeg reported this:

$ ffmpeg -i INPUT_FILE.mpg
FFmpeg version 0.5-svn17737+3:0.svn20090303-1ubuntu6, Copyright (c) 2000-2009 Fabrice Bellard, et al.
  configuration: --enable-gpl --enable-postproc --enable-swscale --enable-x11grab
 --extra-version=svn17737+3:0.svn20090303-1ubuntu6 --prefix=/usr --enable-avfilter
 --enable-avfilter-lavf --enable-libgsm --enable-libschroedinger --enable-libspeex
 --enable-libtheora --enable-libvorbis --enable-pthreads --disable-stripping --disable-vhook
 --enable-libdc1394 --disable-armv5te --disable-armv6 --disable-armv6t2 --disable-armvfp
 --disable-neon --disable-altivec --disable-vis --enable-shared --disable-static
  libavutil     49.15. 0 / 49.15. 0
  libavcodec    52.20. 0 / 52.20. 0
  libavformat   52.31. 0 / 52.31. 0
  libavdevice   52. 1. 0 / 52. 1. 0
  libavfilter    0. 4. 0 /  0. 4. 0
  libswscale     0. 7. 1 /  0. 7. 1
  libpostproc   51. 2. 0 / 51. 2. 0
  built on Apr 10 2009 23:18:41, gcc: 4.3.3

Seems stream 0 codec frame rate differs from container frame rate: 50.00 (50/1) -> 25.00 (25/1)
Input #0, mpegts, from 'INPUT_FILE.mpg':
  Duration: 00:00:36.05, start: 2.050022, bitrate: 9188 kb/s
  Program 1
    Stream #0.0[0x810]: Video: h264, yuv420p, 1280x688 [PAR 1:1 DAR 80:43], 25 tbr, 90k tbn, 50 tbc
    Stream #0.1[0x814](eng): Audio: aac, 48000 Hz, stereo, s16, 184 kb/s

The simple thing I did first and which gave a bad file was

$ ffmpeg -i INPUT_FILE.mpg -vcodec copy -acodec copy -f mp4 ffmpeg.mp4

 Looking at that file with various tools gives this:

$ mp4info ffmpeg.mp4
mp4info version 1.6
ffmpeg.mp4:
Track   Type    Info
1       video   H264 High@4.1, 36.000 secs, 7889 kbps, 1280x688 @ 25.000000 fps
2       audio   MPEG-4 Unknown Profile(0), 36.032 secs, 0 kbps, 48000 Hz
 Tool: Lavf52.31.0

$ MP4Box -info ffmpeg.mp4
* Movie Info *
        Timescale 1000 - Duration 00:00:36.032
        Fragmented File no - 2 track(s)
        File Brand isom - version 512
        Created: GMT Fri Jan  2 00:00:00 1970

File has no MPEG4 IOD/OD

iTunes Info:
        Encoder Software: Lavf52.31.0

Track # 1 Info - TrackID 1 - TimeScale 50 - Duration 00:00:36.000
Media Info: Language "Undetermined" - Type "vide:avc1" - 900 samples
MPEG-4 Config: Visual Stream - ObjectTypeIndication 0x21
AVC/H264 Video - Visual Size 1280 x 688 - Profile High @ Level 4.1
NAL Unit length bits: 32
Pixel Aspect Ratio 1:1 - Indicated track size 1280 x 688
Self-synchronized

Track # 2 Info - TrackID 2 - TimeScale 48000 - Duration 00:00:36.032
Media Info: Language "English" - Type "soun:mp4a" - 1689 samples
MPEG-4 Config: Audio Stream - ObjectTypeIndication 0x40
Segmentation fault

Next, I tried to extract them separately:

ffmpeg -i INPUT_FILE.mpg -an -vcodec copy -f h264 video.h264
ffmpeg -i INPUT_FILE.mpg -vn -acodec copy -f aac audio.aac

and re-mux them using MP4Box (version 0.4.4):

$ MP4Box -add video.h264#video -add audio.aac#audio mp4box.mp4

That didn't work any better, even though mp4box didn't report a segmentation fault anymore:

$ MP4Box -info mp4box.mp4
* Movie Info *
 ⁞       Timescale 600 - Duration 00:00:36.031
        Fragmented File no - 2 track(s)
        File Brand isom - version 1
        Created: GMT Sat Feb 27 15:33:40 2010

File has root IOD
Scene PL 0xff - Graphics PL 0xff - OD PL 0xff
Visual PL: AVC/H264 Profile (0x15)
Audio PL: AAC Profile @ Level 2 (0x29)
No streams included in root OD

Track # 1 Info - TrackID 1 - TimeScale 25000 - Duration 00:00:36.000
Media Info: Language "Undetermined" - Type "vide:avc1" - 900 samples
MPEG-4 Config: Visual Stream - ObjectTypeIndication 0x21
AVC/H264 Video - Visual Size 1280 x 688 - Profile High @ Level 4.1
NAL Unit length bits: 32
Pixel Aspect Ratio 1:1 - Indicated track size 1280 x 688
Self-synchronized

Track # 2 Info - TrackID 2 - TimeScale 48000 - Duration 00:00:36.032
Media Info: Language "Undetermined" - Type "soun:mp4a" - 1689 samples
MPEG-4 Config: Audio Stream - ObjectTypeIndication 0x67
MPEG-2 Audio AAC LC - 2 Channel(s) - SampleRate 48000
Synchronized on stream 1

and mp4info also seemed to like the file better:

$ mp4info mp4box.mp4
mp4info version 1.6
mp4box.mp4:
Track   Type    Info
1       video   H264 High@4.1, 36.000 secs, 7717 kbps, 1280x688 @ 25.000000 fps
2       audio   MPEG-2 AAC LC, 36.032 secs, 150 kbps, 48000 Hz

Note that re-wrapping this into a Quicktime .mov file ($ ffmpeg -i mp4box.mp4 -acodec copy -vcodec copy -f mov mp4box.mov) worked and gave a file Quicktime played without trouble.

But to get a working .mp4, I had to use yet another tool for the audio: mp4creator. With the -aac-profile=4 option:

$ mp4creator -aac-profile=4 audio.aac mp4creator-audio.mp4
Warning - you have changed the AAC profile level. ⁞ This is not recommended
If you have problems with the resultant file, it is your own fault
Do not contact project creators

Despite the warning, this worked after re-muxing with mp4box:

$ MP4Box -add video.h264#video -add mp4creator-audio.mp4#audio mp4box.mp4

$ mp4info mp4box-2.mp4
mp4info version 1.6
mp4box-2.mp4:
Track ⁞⁞⁞  Type    Info
1       video   H264 High@4.1, 36.000 secs, 7717 kbps, 1280x688 @ 25.000000 fps
2       audio   MPEG-4 AAC LC, 36.032 secs, 150 kbps, 48000 Hz

Examining the difference, it seems that both ffmpeg and MP4Box were making an MPEG-2 AAC track, and Quicktime wanted an MPEG-4 track instead.

PDF to Word conversion notes

Had a complex PDF to convert to something editable like .doc, so I had another look at what was available.

This comparative test from 2008 was very helpful, as were some readers' comments. It concluded by recommending the koolwire.com service, which was indeed quite good, and also very convenient because it can be used through email. It produced an RTF with mostly actual tables. Visually, however, the tables in this particular case would have needed quite some re-formatting to look like the original ones.

Several readers suggested the PDF-to-Word service at pdftoword.com. For me, this gave me the best looking results. It converted the complex tables into columnized sections instead, but that was fine. (As an aside, it is not very clear which engine this service is using. It is related to Nitro PDF, a commercial Windows application which is promoted from the pdtftoword.com page. Also, the Nitro PDF pages link to the free pdftoword.com service as their free version. However, the produced Word document mentions Solid Converter PDF, another commercial Windows application, in it's properties. Weird...)

I also tried the convertpdftoword.net service which others suggested. It also gave a good looking Word document, but built it with tons of independent text boxes which was quite unconvenient in my case. A closer look, showed that this service was actually using VeryPDF's PDF2Word, which produced an RTF file (but with a .doc extension). PDF2Word turns out to actually be a re-packaging of xpdf, and is free (GPL) software. The source is available, but VeryPDF sells the Windows executable.

The funny thing from theses tests: the only completely useless conversions happened to be the one from Adobe itself.

Conclusion: I had the best results with pdftoword.com. But it all depends on your source document and what you want to do with it.