Thursday, August 13, 2009

Whither Btrfs?

Is the best technology the pathway to success? Nope. In this post, I'll take a strategic look at the future of the Btrfs file system.

Using B-trees (or modified B-trees) for space allocation has been the rage among file system designers in the past few years. Some of the more notable efforts are ZFS, Btrfs, Reiser4, and NILFS. The availability of open source operating systems, especially BSD and Linux, has enabled explorations of interesting new ways to manage storage and implement file systems. This is a good thing. But being technologically cool and successful does not foretell commercial success. For the purpose of evaluating file systems, I'll define commercially successful as having a large installed base for decades. The list of commercially successful file systems is fairly small: FAT, NTFS, HFS+, UFS, and ext2/3 are perhaps the most commercially successful general-purpose file systems today. The key to commercial success is to provide good value and have a good delivery channel.

Btrfs was announced by Oracle in June 2007 and is being integrated into the Linux kernel. It offers some of the more interesting features of other file systems built on B-tree notions: snapshots, efficient backups, copy-on-write, multiple file systems in a single logical volume (called subvolumes), dynamic inode allocation, multiple device support, internal mirroring, etc. These are all cool features and represent a viable technology direction. But technological feats often run into barriers to adoption which prevent them from becoming commercially successful.

The most important barrier to adoption is the delivery channel. Clearly, Microsoft dominates the industry as it carefully controls the deliver channel of software onto approximately 90% of the computer systems volume. Microsoft owns (is the proprietor of) NTFS and FAT which dominate the market. The next major vendor by volume is Apple, who owns HFS+ that is used as the default file system for OSX. The largest Linux channels, Red Hat and SuSE, use ext2/3 and seem to be planning to use ext4 in the future. Changing the default file system for a popular OS is a very expensive, time-consuming, and disruptive event, which is why OS vendors will spend a lot of time and money to fix and incrementally improve the default file system when possible. The life cycle for a default file system is measured in decades. The development of a new file system takes time, too -- on the order of 5-6 years seems to be typical as measured by having enough stable new features that the value of migration is greater than the inertia of the legacy. The barrier here is time to maturity and time to become the default in the channel. Introduced in 2007, we can expect to see Btrfs be mature in the 2012 timeframe. But what about the prospects of becoming the default in the channel?

Oracle has been trying to reduce their costs by eliminating the OS vendor for many years. Until recently, their efforts were to completely eliminate the OS vendor (raw iron) or to take away Linux from Red Hat (so-called Oracle Enterprise Linux, aka Larry Linux). Neither has been very successful. But Oracle's acquisition of Sun Microsystems changes the industry structure in many ways. Now, Oracle will have an entire solution stack: software, hardware, and services. The solution stack represents a channel for Oracle to deliver innovations, such as a spiffy new file system. Herein lies the problem for Btrfs: Oracle will now own ZFS. This means:
  • Btrfs is not mature enough to become the default file system for OEL. ZFS is more than 5 years old and stable enough to become the default file system for Solaris 10 and OpenSolaris.
  • It makes little sense for Oracle to continue funding two, competing file system projects -- one trying to match features with the other. ZFS has approximately 45 associated patents, and patents have real value ($) in the US.
  • Tossing Btrfs to the open-source winds is not likely to improve its schedule or channel prospects.
There are a couple of scenarios that could still play out -- Oracle could break the GPLv2 barrier that prevents Linux from accepting ZFS in the kernel or Oracle could take a more competitive stance against Red Hat and Novell by leveraging [Open]Solaris. Either way I don't see a good business case for Oracle to continue to invest in Btrfs. What do you think?

3 comments:

  1. BTRFS is interesting to Oracle because it's primary focus is on helping database applications work great. ZFS is interesting to Oracle because it's primary focus is on making commodity hardware provide stable, enterprise features. Databases are only one part of what enterprise systems need. There is a lot of overlap between ZFS and BTRFS.

    I'm still betting that ZFS will win, but I'm also wondering why Apple is silent on this "feature" removal.

    I'm guessing it's because they want it to be supported at MacOSXForge for a bit longer so that something really cool can fall out of having a community involved.

    But, we haven't see the community committers happen yet...

    ReplyDelete
  2. It is not clear to me that BTRFS would be a good file system for a database, for the same reasons that impact ZFS. Rather, I think that Oracle's push for ASM will continue.

    ReplyDelete
  3. I would think that if both file systems are based on similar data structures, they could both be optimized in about the same ways. I.e. if I were Oracle I'd start from ZFS and make database-optimization changes to it.

    My colleague and I have posted some thoughts on the Oracle / Sun merger at http://ctistrategy.com.

    ReplyDelete