Phenquestions
For the sake of clarity, when we say ZFS we mean OpenZFS and not the commercial version supported by Oracle Inc.
The two filesystems are immensely mature and have a nice community built around them. We will be focusing on the major factors that would determine you to chose the file system best suited for you. It is not going to be about an endless cycle of pros and cons with a diplomatic conclusion. No! we will get to the bottom of this.
1. Community and Supported Platforms
XFS is largely supported on Linux along with IRIX. Read Experimental write support on FreeBSD is also available. OpenZFS on the other hand has an extremely broad user base. FreeBSD community strives to keep up with the upstream OpenZFS branch. Entire corporations like Delphix, iXsystems and many more use OpenZFS or recommend it specifically because they want the most reliable platform to store and manage their and their customer's data.
Along with that OpenZFS is also supported on Linux, OS X (that's right you can have it running natively on your MacBook! ) and now they are even working towards bringing it on Windows although the idea is still in its infancy. More esoteric operating systems like Illumos and SmartOS also fully support OpenZFS.
The License used by OpenZFS is the CDDL license which, as it turns out, was incompatible with Linux' GPL license which is why you need to install OpenZFS as a separate binary and a loadable Kernel module instead of having it shipped with the distro. This adds an extra step in terms of installation and upgrading the software.
XFS, on the other hand, comes shipped with most Linux Distros and RedHat Inc. actually recommends its users to use the file system and offers commercial support for it.
2. Architecture
While ZFS is designed to be reliable, XFS' main selling point is its support for fast parallel I/O. OpenZFS is designed as a copy-on-write file system which means even when data is being modified, it is done by writing a new data block first and then getting rid of the old data block. This ensures data integrity in the face of power-failures or kernel panic. ZFS also supports checksums which means silent data corruptions when your disks start to fail is reported, if not automatically fixed (which would happen if you have RAIDZ or mirroring enabled).
As mentioned earlier, XFS offers higher performance it terms of I/O. Being a Journaling File System it still keeps track of changes that are to be made to the data on the disk. This offers some security against power-outages and system crashes. No security is offered against 'bit rot' or silent disk failures.
For the more tech savvy readers, OpenZFS is a 128-bit file system while XFS is a 64-bit file system. This means that the while the latter can store upto 264 files, with each file upto 8 exbibytes in size. The maximum size of a volume is also 8 exbibytes (1 exbibyte is 260 bytes).
This number is dwarfed when compared to the 16 exbibytes of max file size, 256 trillion yobibytes (1 yobibyte is 280 bytes) of maximum volume size offered by OpenZFS. While both the file systems offer absurd flexibility in this regard, they both have one flaw due to their architecture. They both support only 255 character long file names. OpenZFS, at the time of this writing, is working towards increasing this limit further.
One last point of architectural difference is this - XFS has B+ tree implementation for searching files and allocating space. This makes searching and fetching the data much faster. OpenZFS has no such feature.
3. Features
Robust architecture is much more important than the set of features. Sadly, features are much much more visible to the user than the architecture. And even here, OpenZFS wins by a huge margin. It has its own RAID implementation (RAIDZ and mirroring) so you can have redundancy on a software level which is far superior to any other software or hardware RAID solution. On top of that it supports a wide range of built-in compression algorithms like lz4 and gzip so you can store your files compressed. You can have periodic snapshots of your data without a tremendous penalty on the space usage.
You can also have deduplication on OpenZFS and zfs send can be a decent alternative to rsync. However, for XFS most of these features are still on the drawing board waiting to be implemented.
Using OpenZFS you can also have support for the most bleeding edge hardware such as Intel Optane or other NVMe devices and use them as read or write caches. Both the file systems have decent TRIM support for SSDs which improves the performance and longevity of solid state devices.
4.Resource utilization and performance
Resource utilization is one of the major concerns that drives people away from using OpenZFS. The notion being that such an advanced file system must have a huge penalty on the memory and CPU usage. This is not the complete truth, however.
If you are using ZFS with all the default values set, then it will use more resources than XFS and perform slightly worse in terms of I/O but, with modern hardware and faster CPUs, this difference is negligible. The resource hogging features like data deduplication are turned off by default causing no noticeable difference.
XFS is optimized for huge files and for parallel I/O this makes it the easier choice for use cases like NASA Advanced Supercomputing Division. Where as, ZFS needs tuning to fit the use case. If you are running a database then set the record size of your database as a multiple of your ZFS block size. To get this done right requires time for testing and benchmarking. Where as, if you are serving large files, then the blocksize can be maxed out to improve performance.
Conclusion
To cut a long story short, OpenZFS is the most future-proof and robust solution for your precious data. It offers reliability along with a tonne of features. It is actively being developed to meet newer industry standards and has a head start which puts file systems like XFS into a huge disadvantage.
XFS merits consideration only for very specific use cases where large files are involved and your application can make use of parallel I/O and even then data integrity and error detection is absent. OpenZFS on the other hand can be recommended even when all the future use cases are not immediately apparent. With the surety that if it doesn't corrupt your data or silently gives you back the wrong information, OpenZFS is the obvious choice for your valuable data.
