Sunday, September 17, 2017

Hard drive partitions and file system essentials v2

What most normal users need to know about hard disk partitions and filesystems to be able to move hard disks between various operating systems like Mac or Windows.

Partitions

Hard disks contain 1 or more partitions. To the user, each partition appears as if it were a separate hard disk.
(In Windows, each partition receives a separate drive letter like C:, D:, etc.; on a Mac and most Linux, you see a separate icon on the desktop for each partition, and the contents is accessible in a folder like /Volumes/YourDiskName.)
The disk contains a partition table which describes the size and placement of the partitions on the disk. There are 2 main types of partition tables:

  • MBR or DOS : supported everywhere, but only for disks up to 2 TB.
  • GPT or GUID : for disks over 2 TB and for Mac OS X boot disks.

Filesystems

Every partition needs to be formatted with a file system to let the operating system store and retrieve files. (On Mac, this formatting process is called "erasing")
There are many different types of file systems. Your system needs to understand these file systems to be able to use them. Unfortunately, various operating systems use different file systems. The problem is to find which one will be understood by all the systems you intend to connect your drive to. Also, some systems only support reading some file systems, not writing to them.

Summary

Below is a table trying to summarize the compatibility between the 3 main operating systems and the 5 main file system types. There are many others, but if you know about them, you probably don't need this page.
WindowsMac OS XLinux
FAT32 or DOSNative support
Max. 4GB. file size!
Read/Write
Max. 4GB. file size!
Read/Write
Max. 4GB. file size!
NTFSNative supportRead only. Write support through additional software 1Read/Write on recent distributions.
HFS+ or "Mac OS extended"Requires third party programs for reading and writing. 2Native supportRead only. Write if forced or  journaling feature disabled. 3
ExfatNative support since Windows Vista/7Native support since 10.6.5Needs driver install
Ext2 or Ext3Requires driver 4Requires driver. 4Native support
FAT or FAT32 (named "MS-DOS" in Macs)
This the oldest of the file systems commonly used today. As such, it has the greatest compatibility and the least functionality. It is a sort of lowest common denominator. All operating systems can read and write to it. It is the file system generally used on USB flash drives, memory cards for photo cameras, etc. It cannot store files greater than 4 Gigabytes. It is also the least reliable of the current file systems, and has many other drawbacks (fragmentation, no support for permission, time stamps in local time, etc.)
The Windows disk manager refuses to format a FAT32 partition greater than 32 GB. But it can be formatted to the wanted size on Mac or Linux, or with the free fat32format utilityin Windows.
NTFS
Is the native file system of Windows. Macs can read it, but cannot write to it. However, there is a Mac version of the open source NTFS-3G driver which can write to NTFS. 1Recent Linux versions can both read it and write to it (thes usually have this NTFS-3G driver installed by default). 2
HFS aka. "Mac OS X" HFS+ aka. "Mac OS X Extended (journaled)"
Is the native file system on Macs.The Mac default is the HFS+ journaled variant. Windows needs special programs installed to be able to read or write it. 3Linux can read it when it has the hfsutils package installed. It can also write to it if journaling has been disabled. 4
Exfat
Meant to replace FAT32 on digital cameras etc. Supports files greater than 4GB, but not as feature-rich and reliable as the others.
Ext2 or Ext3
Is the official standard for DCP disks and the most common file system on Linux. You could try some Windows or Mac driver, but it's probably much easier to install Linux on some old machine and access it through the network.
UDF
And what about UDF, the "Universal Disk Format" which is even a true ISO standard? It is used on professional camera cards and on Blu-ray disks, and can in theory be read and written by all 3 current systems. But in practice, this is only true if it is correctly formatted. And since the normal formatting tools in Mac and Windows don't offer it as an option, I would only recommend it to geeks willing to use this command-line formatting script.
Footnotes:
1. Mac -> NTFS: To enable writing of NTFS on a Mac, you need a commercial program like Paragon or Tuxera.
2. Windows -> HFS: If you only need to copy files from a Mac disk to your Windows machine, you can use the free HFSExplorer, which will open your drive in a Windows Explorer-like window and let you copy files from there. For full support, you may need commercial software like MacDrive or Paragon.
3. Linux -> HFS: If you need to write to the HFS disk, journaling must be disabled on a Mac first (through Disk Utility or diskutil disableJournal "/Volumes/YOUR_VOLUME_NAME"in Terminal). Alternatively, you can force the mount point to be writable.
4. Windows/Mac -> ext2/3/4: There are various free drivers for Windows and Mac, but when I tried them a few years ago, they were probematic. There is also a commercial driver from Paragon which I haven't tried. But really, a Linux machine on the network is so much easier.