[PATCH v3 0/5] Implement reflink support

[Erich E. Hoover]

On Mon, Feb 7, 2022 at 12:44 PM Alex Xu (Hello71) <alex_y_xu at yahoo.ca> wrote:
>
> Excerpts from Alexandre Julliard's message of February 6, 2022 9:51 am:
> > "Alex Xu (Hello71)" <alex_y_xu at yahoo.ca> writes:
> >> Full reflink functionality with arbitrary offsets and overwrite is
> >> non-trivial to implement in the filesystem. However, we really only need
> >> the limited functionality of a whole-file reflink. This could be
> >> simulated using a hard or symbolic link which gets broken by Wine when
> >> any write occurs. However, as with reparse points, this requires
> >> modifying every file access that Wine makes, which would be a
> >> significant undertaking.
> >
> > I'm not convinced that it would be that hard, and if it can work it
> > would clearly be a better approach. Prefix size is a real problem, but I
> > don't think reflinks are the answer.
> >
> > --
> > Alexandre Julliard
> > julliard at winehq.org
> >
>
> I disfavor the "magic symlink" approach for three reasons:

I'm coming late to the party here (been on a work trip for about a
month), but are you talking about the proposal to have the core Wine
DLLs use special symlinks inside the prefix that are copy-on-write?

> 1. Implementing copy_file_range for CopyFile improves performance not
> only for wineboot, but also for every program using CopyFile. It allows
> reflink but also remote-side copy for cifs on Linux. I doubt anybody has
> Wine prefix on cifs, but maybe some people put files or programs on a
> network drive.

We could conceivably use the symlink approach for this (as long as
you're working with whole files).

> 2. Who will review the resulting patch? This submission is now the third
> revision of this patchset with no non-trivial review.

This mailing list and, ultimately, AJ.  My interpretation of the
response you have is that he's not convinced that implementing full
reflink support is as hard as you think.  You might ask him for
clarification, as it seems to me that depending on what you're talking
about that that would be very difficult.

> While we may
> disagree on whether "magic symlink" will be 2x or 10x the current
> patchset, it will certainly be larger. In particular, the setupapi part,
> which is the most complex part of this patchset, will certainly be
> larger and even more complex; passing file names down to write_fake_dll
> is harder than passing file handles, not easier.

This comment is why I'm wondering if you're talking about something
else.  My view of the symlink approach is that (after some comments
from an RFC) there's about three steps:
1) implement generic reparse point storage (convert reparse points
into symlinks with contents that are not usable outside Wine) and
retrieval
2) implement specialized reparse point storage (convert common reparse
points into unix-usable paths) and retrieval
3) implement a Wine-specific reparse point with copy-on-write behavior

Steps 1&2 of this process are useful in general (since we don't
currently have reparse point support), which makes step #3 a much
smaller lift once the rest of the infrastructure (1&2) are in place.
I will grant you that the full implementation is a lot of patches (I
have not separated 1&2 yet, and currently have 39 patches just for
that), but a prototype I put together to test the feasibility of #3 is
only about 3 small patches.

> 3. Who will fix the resulting breakage in winetricks, protontricks, and
> who knows what other programs?

The people that maintain these applications will be forced to update
if things break outside of Wine.

> As far as I can tell, any option for
> representing the magic link on the Unix side will result in issues.
> Either it uses symlinks, and "cp override_mydll.dll .../mydll.dll" will
> be broken, or it doesn't create the Unix side files, so ls will be
> broken, or it uses empty files, which is possibly the least worst
> solution but will break either "cp mydll.dll mydll2.dll" or "cp -a
> $WINEPREFIX $WINEPREFIX.bak", depending on the Wine implementation.

That really depends upon what you stick into the symlink and whether
you expect the symlink to work outside Wine.  Part of the feedback I
got from my last symlink/reparse point RFC was "does this really need
to work outside of Wine?"  I think it's important to have standard
Unix tools work with any Wine symlinks, but if the contents of the
symlink look something like "${WINEPREFIX}/path/to/target" then that
makes it possible to backup (cp -a) a prefix where a "proper"
implementation that has fully working links on the Unix side would not
allow that.  For what you're doing here I would expect that you would
want to use absolute paths or prefix-relative paths, which would not
break if you use "cp -a override_mydll.dll .../mydll.dll" (though you
would need to use "-a").

> That being said, I am perfectly happy for someone else to write this
> "magic symlink" implementation.

I've been putting something together in that vein since 2014, though I
am taking a very different approach from your proposal.  People might
like your approach better.

> I agree that the full reflink
> functionality is not required, and it would be more beneficial to users
> to reduce prefix sizes on ext4.

Symlinks work cross-volume, so if you have /usr/lib/wine on a
different volume (very common in a lot of environments) then reflinks
won't save you.

> Even though most other Linux filesystems
> support reflink (even XFS, which is not CoW), ext4 is still the most
> popular option. However, I will not implement this, as I have already
> spent significant time working on this with no serious response from
> Wine project, only some side comments. "can it work on mac" is a fine
> question to ask, but is not actionable.

Wine doesn't particularly like to add features that are not
generalizable, as these features have a tendency to add a large
maintenance burden and don't benefit the vast majority of users.  An
approach that sometimes works here is to create a general solution
(possibly symlinks) that works for everyone that with a simple
extension (reflinks) then works _better_ if that feature is available.
This approach also means that when there's a bug that you can ask
people "have you tried turning off feature X?" to narrow in on what's
wrong.

Best,
Erich