Saturday, May 23, 2009

Hard drive partitions and file system essentials

What most normal users need to know about hard disk partitions and filesystems to be able to move hard disks between various operating systems like Mac or Windows.

Hard disks contain 1 or more partitions. To the user, each partition appears as if it were a separate hard disk.

(In Windows, each partition receives a separate drive letter like C:, D:, etc.; on a Mac, you see a separate icon on the desktop for each partition; in Linux, each is a device like /dev/sdb1, /dev/sdb2, etc.)

Every partition needs to be formatted with a file system to let the operating system store and retrieve files. (On Mac, this formatting process is called "erasing")

There are many different types of file systems. Your system needs to understand these file systems to be able to use them. Unfortunately, various operating systems use different file systems. The problem is to find which one will be understood by all the systems you intend to connect your drive to. Also, some systems only support reading some file systems, not writing to them.

Summary

Below is a table trying to summarize the compatibility between the 3 main operating systems and the 4 main file system types. There are many others, but if you know about them, you probably don't need this page.


Windows Mac OS X Linux
FAT32 or DOS

Native support

Max. 4GB. file size

Read/Write

Max. 4GB. file size

Read/Write

Max. 4GB. file size

NTFS Native support

Read only.

Write support through external drivers. 1

Read/Write on recent distributions. 2
HFS+ or "Mac OS extended" Requires third party programs for reading and writing. 3 Native support

Read only.

Write if the journaling feature has been turned off in Mac OS X. 4

Ext2 or Ext3 Free drivers allow Read/Write access.5 Requires commercial driver. 6 Native support
FAT or FAT32 (named "MS-DOS" in Macs)

This the oldest of the file systems commonly used today. As such, it has the greatest compatibility and the least functionality. It is a sort of lowest common denominator.

All operating systems can read and write to it. It is the file system generally used on USB flash drives, memory cards for photo cameras, etc.

It cannot store files greater than 4 Gigabytes. It is also the least reliable of the current file systems, and has many other drawbacks (fragmentation, no support for permission, time stamps in local time and with only 2 seconds resolution, etc.)

The Windows disk manager refuses to format a FAT32 partition greater than 32 GB. But it can be formatted in Windows with the free fat32format.exe utility, or can be formatted to the wanted size on Mac or Linux.

NTFS

Is the native file system of Windows.

Macs can read it, but cannot write to it.

However, there is a Mac version of the open source NTFS-3G driver which can write to NTFS. 1

Recent Linux versions can both read it and write to it (thes usually have this NTFS-3G driver installed by default). 2

HFS aka. "Mac OS X" HFS+ aka. "Mac OS X Extended (journaled)"

Is the native file system on Macs.The Mac default is the HFS+ journaled variant.

Windows needs special programs installed to be able to read or write it. 3

Linux can read it when it has the hfsutils package installed. It can also write to it if journaling has been disabled. 4

Ext2 or Ext3

is the most common file system on Linux.

(If you wonder why you would need to know anything about Linux: while it is not very common as a desktop operating system, it is the system used in almost all your non-computer devices which contain a hard disk, like your NAS backup disk, your media player, etc. If that device breaks, you may be able to recover the files from it's hard disk by connecting it to your main computer and installing the driver for the ext2 file system)

Windows can read and write to it using free drivers. 5

There is a Mac driver, but it may be problematic. 6

Footnotes:

1. Mac -> NTFS : The free and open source ntfs-3g driver for Mac is available on http://sourceforge.net/projects/catacombae/files/. The commercial version is based on the same code, but improves speed. You may also want to have a look at the user guide and the macntfs-3g blog.

2. Linux -> NTFS : In case you have an older distribution which doesn't have it pre-installed, you can normally install "ntfs-3g" using your distribution's package manager. Or have a look at their availability page.

3. Windows -> HFS : If you only need to copy files from a Mac disk to your Windows machine, you can use the free HFSExplorer, which will open your drive in a Windows Explorer-like window and let you copy files from there.

For full support, you may need commercial software like MacDrive or similar.

4. Linux -> HFS : If it isn't already on your system, you will need to install the "hfsutils" package.

If you need to write to the HFS disk, journaling must be disabled. You need to do this on a Mac. Afterwards, you can re-enable journaling (again on a Mac). To disable journaling on a Mac, open Disk Utility, select the volume, hold the Option (or Alt) key while opening the File menu, which will make the "Disable Journal" menu entry appear in the menu. Alternatively, you can enter diskutil disableJournal "/Volumes/YOUR_VOLUME_NAME" in Terminal

5. Windows -> ext2/3 : There are 2 free drivers. The open source one is at http://www.ext2fsd.com/ and the closed source one is at http://www.fs-driver.org/.

6. Mac -> Ext2/3 : You can try the commercial ExtFS for Mac OS X. Or the open source fuse-ext2 which I have never tried. (There is also another free open source driver (http://sourceforge.net/projects/ext2fsx), but that project doesn't seem to be actively maintained. It may have worked well on older Mac OS versions, but when I tried a simple folder move with the current version 1.4d4 on a Mac OS X 10.5 system, it made the system crash hard, and left a badly corrupted drive, which I had to repair using e2fsck on Linux.)

Labels: , , , , ,