RFC: reducing wasted disk space from addon files

[Kai Krakow]

Hello!

Am Mi., 6. März 2019 um 14:41 Uhr schrieb Gabriel Ivăncescu
<gabrielopcode at gmail.com>:
>
> On 3/6/19 3:35 PM, André Hentschel wrote:
> > Hi,
> >
> > just my two cents on it.
> >
> > Can't we keep the shared stuff in the home directory? It would be
> > user-wide instead of system-wide, but already a reduction and we
> > wouldn't need to mess with access restrictions in /usr. Possible naming:
> > $HOME/.wine-addons-gecko $HOME/.wine-addons-mono
> >
> > Another idea would be to outsource this topic to steam or even better
> > winetricks. Most filesystems in use support hardlinks, a winetricks verb
> > with wine prefixes as arguments that hardlinks all gecko/mono stuff in
> > them (if their shaXXX hash match) looks very easy and suitable to me and
> > steam could use it too. That way the prefix remains transferable.
> >
> >
>
> This gives me an idea. Why not have it in the wine directory? This way
> it can be decided whether to have it system-wide or $HOME or anywhere
> else you place Wine to.
>
> I do agree with the gist of your idea though, and hard linking sounds
> more reasonable if that can be pulled off.

There's one caveat with hard linking: If one of the installation
modifies the files (for whatever reason), it will also change in the
other prefixes although they should be distinct (unless you unlink the
file before replacing it). Also, hard links do not work accross file
system boundaries, and I think it's quite common to have such a
boundary between /home and /usr, some people may even have the wine
prefixes (or the Steam lib aka Proton games) on different partitions.

A better way would be to redirect installation to a shared directory
and then symlink the files to the intended destination.

Another solution (which also has the filesystem boundary problem)
would be to use the new reflink capability that comes with btrfs and
xfs. Those filesystems can share the file contents without hard
linking the inode. This leaves an installation with shared data but
distinct inode properties (fname, mtime, ctime, aclc, etc...). Proton
could try using a similar copy mode (cp --reflink=auto) while filling
the per-game prefix from the shared prefix installation. It is safe if
one prefix modifies the files because the change wouldn't reflect back
into the other copies. Such a mechanism could fall back to symlinks
when reflink is not available (tho, it wouldn't prevent accidental
changes to the files).

I'm currently solving the problem here by using wine on btrfs and
running the bees daemon to deduplicate file data from the different
wine prefixes. This works very well, fast, and has a well-defined
overhead (bees uses constant memory and low IO/CPU). The result is a
perfect reflinked copy of each prefix.

Wine should maybe start with using a shared download cache at least,
similar to how winetricks does it. Data deduplication should maybe be
left to external processes that know how to handle the situation, wine
could provide example script or helper programs that could be run by
cron. Such a helper could sha1sum all files and do the hard linking,
or reflinking if available. Such an approach would also probably work
better for Proton, as the prefixes are already managed by the Proton
launcher script.

Regards,
Kai