30 Mar
2020
30 Mar
'20
12:24 p.m.
Hi all!
I have been playing with fuse-overlayfs [1] lately to reduce prefix size
and speed up wine startup time, and I thought maybe someone else could
be interested, and I would definitely be interested in some feedback,
especially if anyone already tried.
So far it has been working good for me, but to be fair I haven't used it
extensively yet. To enable the overlays, define the WINEPREFIX_OVERLAYS
environment variable to point to one or more drive_c to stack under
the prefix's drive_c, from top to bottom. For instance, creating a new
layered prefix with an existing default prefix and an additional
intermediate layer:
WINEPREFIX_OVERLAYS=$PWD/pfx_layer1/drive_c:$PWD/default_pfx/drive_c \
WINEPREFIX=$PWD/.wine \
wine winecfg
Another possible advantage is that it makes it immediately clear which
files were modified by an application, as only these files will reside
in the upper layer.
Then there's a few drawbacks with this approach, notably prefix updates
becomes a bit complicated. The file updates aren't a real issue, and
it's actually very fitting for that, as the lower-most dir would
probably be a default wine prefix with all the builtin libraries, and
the upper dir contain only custom installed dlls. But the registry
updates on the other hand are more tricky as the registry backing files
quickly gets modified and copied to the upper dir.
For now I just disabled prefix updates when overlays are used, for
simplicity. I have a few ideas on how to do that properly but I didn't
implement anything yet.
Also, for simplicity only drive_c is currently overlayed, and I moved
wine-specific files under it. Having the whole prefix mounted creates
additional trouble as we use the prefix directory inode and dev numbers
for server identification, and having a mount here changes the values
after the initial server startup.
Cheers,
[1] https://github.com/containers/fuse-overlayfs
* OverlayFS is a file system concept with a set of "lower" read-only
directories layered under an "upper" read-write directory. The lower
dirs files are copied-on-write to the "upper" directory.
* fuse-overlayfs is a FUSE implementation of overlayfs so that it's
possible for a unpriviledged user to mount such a filesystem. It's
available on most linux distros as far as I know.
Rémi Bernon (5):
server: Move registry files to drive_c/.wine.
wineboot: Move .update-timestamp to drive_c/.wine.
wineboot: Open update-timestamp read-only first.
wineboot: Disable prefix update if overlays are used.
server: Support mounting a FUSE overlayfs on drive_c.
programs/wineboot/wineboot.c | 39 ++++++++++++--------
server/main.c | 71 ++++++++++++++++++++++++++++++++++++
server/registry.c | 13 +++++--
3 files changed, 104 insertions(+), 19 deletions(-)
--
2.26.0.rc2
30 Mar
30 Mar
12:24 p.m.
New subject: [RFC PATCH 1/5] server: Move registry files to drive_c/.wine.
When mounting an overlayfs to drive_c in the next patch, we need to
expose the lower layer registry, as wineboot is not going to initialize
the upper layer.
---
server/registry.c | 13 ++++++++++---
1 file changed, 10 insertions(+), 3 deletions(-)
diff --git a/server/registry.c b/server/registry.c
index dcbb3f791e10..c1730ae24bb8 100644
--- a/server/registry.c
+++ b/server/registry.c
@@ -1805,12 +1805,19 @@ void init_registry(void)
assert( root_key );
make_object_static( &root_key->obj );
+ mkdir( "drive_c/.wine", 0777 );
+
+ /* create symlinks for backward compatibility */
+ symlink( "drive_c/.wine/system.reg", "system.reg" );
+ symlink( "drive_c/.wine/userdef.reg", "userdef.reg" );
+ symlink( "drive_c/.wine/user.reg", "user.reg" );
+
/* load system.reg into Registry\Machine */
if (!(hklm = create_key_recursive( root_key, &HKLM_name, current_time )))
fatal_error( "could not create Machine registry key\n" );
- if (!load_init_registry_from_file( "system.reg", hklm ))
+ if (!load_init_registry_from_file( "drive_c/.wine/system.reg", hklm ))
{
if ((p = getenv( "WINEARCH" )) && !strcmp( p, "win32"
))
prefix_type = PREFIX_32BIT;
@@ -1825,7 +1832,7 @@ void init_registry(void)
if (!(key = create_key_recursive( root_key, &HKU_name, current_time )))
fatal_error( "could not create User\\.Default registry key\n" );
- load_init_registry_from_file( "userdef.reg", key );
+ load_init_registry_from_file( "drive_c/.wine/userdef.reg", key );
release_object( key );
/* load user.reg into HKEY_CURRENT_USER */
@@ -1836,7 +1843,7 @@ void init_registry(void)
!(hkcu = create_key_recursive( root_key, ¤t_user_str, current_time )))
fatal_error( "could not create HKEY_CURRENT_USER registry key\n" );
free( current_user_path );
- load_init_registry_from_file( "user.reg", hkcu );
+ load_init_registry_from_file( "drive_c/.wine/user.reg", hkcu );
/* set the shared flag on Software\Classes\Wow6432Node */
if (prefix_type == PREFIX_64BIT)
--
2.26.0.rc2
12:24 p.m.
New subject: [RFC PATCH 2/5] wineboot: Move .update-timestamp to drive_c/.wine.
When mounting an overlayfs to drive_c in the next patch, we need to
expose the lower layer timestamp to wineboot, so it doesn't update the
upper layer every time.
---
programs/wineboot/wineboot.c | 14 ++++----------
1 file changed, 4 insertions(+), 10 deletions(-)
diff --git a/programs/wineboot/wineboot.c b/programs/wineboot/wineboot.c
index e1dbe6630221..913c967d4d4d 100644
--- a/programs/wineboot/wineboot.c
+++ b/programs/wineboot/wineboot.c
@@ -118,17 +118,12 @@ static WCHAR *get_wine_inf_path(void)
/* update the timestamp if different from the reference time */
static BOOL update_timestamp( const WCHAR *config_dir, unsigned long timestamp )
{
- static const WCHAR timestampW[] =
{'\\','.','u','p','d','a','t','e','-','t','i','m','e','s','t','a','m','p',0};
+ static const WCHAR timestampW[] =
{'C',':','\\','.','w','i','n','e','\\','u','p','d','a','t','e','-','t','i','m','e','s','t','a','m','p',0};
BOOL ret = FALSE;
int fd, count;
char buffer[100];
- WCHAR *file = HeapAlloc( GetProcessHeap(), 0, lstrlenW(config_dir) * sizeof(WCHAR) +
sizeof(timestampW) );
- if (!file) return FALSE;
- lstrcpyW( file, config_dir );
- lstrcatW( file, timestampW );
-
- if ((fd = _wopen( file, O_RDWR )) != -1)
+ if ((fd = _wopen( timestampW, O_RDWR )) != -1)
{
if ((count = read( fd, buffer, sizeof(buffer) - 1 )) >= 0)
{
@@ -142,20 +137,19 @@ static BOOL update_timestamp( const WCHAR *config_dir, unsigned long
timestamp )
else
{
if (errno != ENOENT) goto done;
- if ((fd = _wopen( file, O_WRONLY | O_CREAT | O_TRUNC, 0666 )) == -1) goto done;
+ if ((fd = _wopen( timestampW, O_WRONLY | O_CREAT | O_TRUNC, 0666 )) == -1) goto
done;
}
count = sprintf( buffer, "%lu\n", timestamp );
if (write( fd, buffer, count ) != count)
{
- WINE_WARN( "failed to update timestamp in %s\n", debugstr_w(file) );
+ WINE_WARN( "failed to update timestamp in %s\n", debugstr_w(timestampW)
);
chsize( fd, 0 );
}
else ret = TRUE;
done:
if (fd != -1) close( fd );
- HeapFree( GetProcessHeap(), 0, file );
return ret;
}
--
2.26.0.rc2
12:24 p.m.
New subject: [RFC PATCH 3/5] wineboot: Open update-timestamp read-only first.
Opening it with O_RDWR makes the overlay copy it to the upper layer,
although we are only reading it first.
---
programs/wineboot/wineboot.c | 13 +++++--------
1 file changed, 5 insertions(+), 8 deletions(-)
diff --git a/programs/wineboot/wineboot.c b/programs/wineboot/wineboot.c
index 913c967d4d4d..2463b7a0e050 100644
--- a/programs/wineboot/wineboot.c
+++ b/programs/wineboot/wineboot.c
@@ -123,7 +123,7 @@ static BOOL update_timestamp( const WCHAR *config_dir, unsigned long
timestamp )
int fd, count;
char buffer[100];
- if ((fd = _wopen( timestampW, O_RDWR )) != -1)
+ if ((fd = _wopen( timestampW, O_RDONLY )) != -1)
{
if ((count = read( fd, buffer, sizeof(buffer) - 1 )) >= 0)
{
@@ -131,14 +131,11 @@ static BOOL update_timestamp( const WCHAR *config_dir, unsigned long
timestamp )
if (!strncmp( buffer, "disable", sizeof("disable")-1 ))
goto done;
if (timestamp == strtoul( buffer, NULL, 10 )) goto done;
}
- lseek( fd, 0, SEEK_SET );
- chsize( fd, 0 );
- }
- else
- {
- if (errno != ENOENT) goto done;
- if ((fd = _wopen( timestampW, O_WRONLY | O_CREAT | O_TRUNC, 0666 )) == -1) goto
done;
+ close( fd );
}
+ else if (errno != ENOENT) goto done;
+
+ if ((fd = _wopen( timestampW, O_WRONLY | O_CREAT | O_TRUNC, 0666 )) == -1) goto
done;
count = sprintf( buffer, "%lu\n", timestamp );
if (write( fd, buffer, count ) != count)
--
2.26.0.rc2
12:24 p.m.
New subject: [RFC PATCH 4/5] wineboot: Disable prefix update if overlays are used.
Updating the prefix with overlays enabled creates all sort of yet
unsolved problems, and will update the upper dir, defeating the purpose
of using overlays.
---
programs/wineboot/wineboot.c | 18 +++++++++++++++++-
1 file changed, 17 insertions(+), 1 deletion(-)
diff --git a/programs/wineboot/wineboot.c b/programs/wineboot/wineboot.c
index 2463b7a0e050..53a634a70614 100644
--- a/programs/wineboot/wineboot.c
+++ b/programs/wineboot/wineboot.c
@@ -86,6 +86,7 @@ static const BOOL is_64bit = sizeof(void *) > sizeof(int);
static const WCHAR winebuilddirW[] =
{'W','I','N','E','B','U','I','L','D','D','I','R',0};
static const WCHAR winedatadirW[] =
{'W','I','N','E','D','A','T','A','D','I','R',0};
static const WCHAR wineconfigdirW[] =
{'W','I','N','E','C','O','N','F','I','G','D','I','R',0};
+static const WCHAR wineprefix_overlaysW[] =
{'W','I','N','E','P','R','E','F','I','X','_','O','V','E','R','L','A','Y','S',0};
/* retrieve the path to the wine.inf file */
static WCHAR *get_wine_inf_path(void)
@@ -138,12 +139,21 @@ static BOOL update_timestamp( const WCHAR *config_dir, unsigned long
timestamp )
if ((fd = _wopen( timestampW, O_WRONLY | O_CREAT | O_TRUNC, 0666 )) == -1) goto
done;
count = sprintf( buffer, "%lu\n", timestamp );
+ ret = TRUE;
+
+ if (_wgetenv( wineprefix_overlaysW ))
+ {
+ WINE_MESSAGE( "wine: overlays are enabled, disabling prefix updates\n"
);
+ count = sprintf( buffer, "disable\n" );
+ ret = FALSE;
+ }
+
if (write( fd, buffer, count ) != count)
{
WINE_WARN( "failed to update timestamp in %s\n", debugstr_w(timestampW)
);
chsize( fd, 0 );
+ ret = FALSE;
}
- else ret = TRUE;
done:
if (fd != -1) close( fd );
@@ -1462,6 +1472,12 @@ static void update_wineprefix( BOOL force )
fstat( fd, &st );
close( fd );
+ if (_wgetenv( wineprefix_overlaysW ))
+ {
+ WINE_MESSAGE( "wine: overlays are enabled, not updating prefix\n" );
+ force = FALSE;
+ }
+
if (update_timestamp( config_dir, st.st_mtime ) || force)
{
HANDLE process;
--
2.26.0.rc2
3:09 p.m.
New subject: [RFC PATCH 4/5] wineboot: Disable prefix update if overlays are used.
On 3/30/20 7:24 AM, Rémi Bernon wrote:
Updating the prefix with overlays enabled creates all
sort of yet
unsolved problems, and will update the upper dir, defeating the purpose
of using overlays.
---
Presumably you only want to skip the parts that write files here,
though. Otherwise the registry won't get updated.
programs/wineboot/wineboot.c | 18 +++++++++++++++++-
1 file changed, 17 insertions(+), 1 deletion(-)
diff --git a/programs/wineboot/wineboot.c b/programs/wineboot/wineboot.c
index 2463b7a0e050..53a634a70614 100644
--- a/programs/wineboot/wineboot.c
+++ b/programs/wineboot/wineboot.c
@@ -86,6 +86,7 @@ static const BOOL is_64bit = sizeof(void *) > sizeof(int);
static const WCHAR winebuilddirW[] =
{'W','I','N','E','B','U','I','L','D','D','I','R',0};
static const WCHAR winedatadirW[] =
{'W','I','N','E','D','A','T','A','D','I','R',0};
static const WCHAR wineconfigdirW[] =
{'W','I','N','E','C','O','N','F','I','G','D','I','R',0};
+static const WCHAR wineprefix_overlaysW[] =
{'W','I','N','E','P','R','E','F','I','X','_','O','V','E','R','L','A','Y','S',0};
/* retrieve the path to the wine.inf file */
static WCHAR *get_wine_inf_path(void)
@@ -138,12 +139,21 @@ static BOOL update_timestamp( const WCHAR *config_dir, unsigned
long timestamp )
if ((fd = _wopen( timestampW, O_WRONLY | O_CREAT | O_TRUNC, 0666 )) == -1) goto
done;
count = sprintf( buffer, "%lu\n", timestamp );
+ ret = TRUE;
+
+ if (_wgetenv( wineprefix_overlaysW ))
+ {
+ WINE_MESSAGE( "wine: overlays are enabled, disabling prefix updates\n"
);
+ count = sprintf( buffer, "disable\n" );
+ ret = FALSE;
+ }
+
if (write( fd, buffer, count ) != count)
{
WINE_WARN( "failed to update timestamp in %s\n",
debugstr_w(timestampW) );
chsize( fd, 0 );
+ ret = FALSE;
}
- else ret = TRUE;
done:
if (fd != -1) close( fd );
@@ -1462,6 +1472,12 @@ static void update_wineprefix( BOOL force )
fstat( fd, &st );
close( fd );
+ if (_wgetenv( wineprefix_overlaysW ))
+ {
+ WINE_MESSAGE( "wine: overlays are enabled, not updating prefix\n" );
+ force = FALSE;
+ }
+
if (update_timestamp( config_dir, st.st_mtime ) || force)
{
HANDLE process;
3:17 p.m.
New subject: [RFC PATCH 4/5] wineboot: Disable prefix update if overlays are used.
On 3/30/20 5:09 PM, Zebediah Figura wrote:
On 3/30/20 7:24 AM, Rémi Bernon wrote:
I started changing setupapi to avoid unnecessary dll updates, but then
there's the nls files that gets written, and possibly other things.
There's then the problem of reconciling multiple layers with conflicting
registry updates, it's not just a question of filesystem state. It was
simpler for a PoC to just avoid these issues and say that prefix updates
are disabled when overlays are used.
I think it can still be a valid use case if the wine version is fixed
and several applications want to share the same base layer and avoid
duplication.
Updating the prefix with overlays enabled creates
all sort of yet
unsolved problems, and will update the upper dir, defeating the purpose
of using overlays.
---
Presumably you only want to skip the parts that write files here,
though. Otherwise the registry won't get updated.
12:24 p.m.
New subject: [RFC PATCH 5/5] server: Support mounting a FUSE overlayfs on drive_c.
This uses fuse-overlayfs [1] to stack layers under drive_c and save some
space. It is then possible to have a default wine prefix that will be
used as a read-only base for other wine prefixes, with only the new or
modified files saved into the upper dir.
The overlayfs is automatically mounted by the server on startup, and
unmount is requested when the server exits. If fuse-overlayfs fails for
any reason, it should fallback to normal prefix and the upper dir will
be updated.
The lower dirs are controlled by the WINEPREFIX_OVERLAYS environment
variable, which is passed as is after -olowerdir= to fuse-overlayfs. It
supports multiple layers, separated by a colon, from top to bottom.
Any other option supported by fuse-overlayfs can also be used and passed
after the lower dirs in WINEPREFIX_OVERLAYS, separated from them with
comma - for example uidmapping/gidmapping.
[1] https://github.com/containers/fuse-overlayfs
---
server/main.c | 71 +++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 71 insertions(+)
diff --git a/server/main.c b/server/main.c
index 26986d34b305..00c4c56542e5 100644
--- a/server/main.c
+++ b/server/main.c
@@ -31,6 +31,9 @@
#ifdef HAVE_GETOPT_H
# include <getopt.h>
#endif
+#ifdef HAVE_SYS_WAIT_H
+# include <sys/wait.h>
+#endif
#include "object.h"
#include "file.h"
@@ -124,6 +127,73 @@ static void sigterm_handler( int signum )
exit(1); /* make sure atexit functions get called */
}
+static void detach_overlays(void)
+{
+ const char *config_dir = getenv( "WINEPREFIX" );
+ char *argv[] = {NULL, NULL, NULL, NULL};
+ int pid;
+
+ pid = fork();
+ if (pid == -1) fatal_error( "fork" );
+ if (!pid)
+ {
+ argv[0] = strdup( "fusermount" );
+ argv[1] = strdup( "-u" );
+
+ argv[2] = malloc( sizeof("/drive_c") + strlen( config_dir ) );
+ strcpy( argv[2], config_dir );
+ strcat( argv[2], "/drive_c" );
+
+ execvp( argv[0], argv );
+ fatal_error( "could not exec fusermount\n" );
+ }
+}
+
+static void mount_overlays(void)
+{
+ char *argv[] = {NULL, NULL, NULL, NULL, NULL, NULL};
+ const char *config_dir = getenv( "WINEPREFIX" );
+ const char *overlays = getenv( "WINEPREFIX_OVERLAYS" );
+ int status;
+ int pid;
+
+ if (!overlays) return;
+
+ pid = fork();
+ if (pid == -1) fatal_error( "fork" );
+ if (!pid)
+ {
+ argv[0] = strdup( "fuse-overlayfs" );
+
+ argv[1] = malloc( sizeof("-olowerdir=") + strlen( overlays ) );
+ strcpy( argv[1], "-olowerdir=" );
+ strcat( argv[1], overlays );
+
+ argv[2] = malloc( sizeof("-oupperdir=/drive_c") + strlen( config_dir )
);
+ strcpy( argv[2], "-oupperdir=" );
+ strcat( argv[2], config_dir );
+ strcat( argv[2], "/drive_c" );
+
+ argv[3] = malloc( sizeof("-oworkdir=") + strlen( config_dir ) );
+ strcpy( argv[3], "-oworkdir=" );
+ strcat( argv[3], config_dir );
+
+ argv[4] = malloc( sizeof("/drive_c") + strlen( config_dir ) );
+ strcpy( argv[4], config_dir );
+ strcat( argv[4], "/drive_c" );
+
+ execvp( argv[0], argv );
+ fatal_error( "could not exec fuse-overlayfs\n" );
+ }
+
+ do waitpid( pid, &status, 0 );
+ while (!WIFEXITED(status));
+
+ if (WEXITSTATUS(status) != 0)
+ fatal_error( "fuse-overlayfs failed\n" );
+ atexit( detach_overlays );
+}
+
int main( int argc, char *argv[] )
{
setvbuf( stderr, NULL, _IOLBF, 0 );
@@ -139,6 +209,7 @@ int main( int argc, char *argv[] )
sock_init();
open_master_socket();
+ mount_overlays();
if (debug_level) fprintf( stderr, "wineserver: starting (pid=%ld)\n",
(long) getpid() );
init_signals();
--
2.26.0.rc2
12:50 p.m.
Hi Rémi,
I like the general idea personally. I use something similar on my setup,
which is custom though, but I use the overlayfs itself rather than the
FUSE variant (well since I could give it privileges, not suitable for
distribution).
Is there a reason you're moving the registry files? As you said, they'll
get quickly overwritten anyway and doesn't make much sense to me. Of
course, to "solve" this you'd have to just set multiple prefixes in
WINEPREFIX_OVERLAYS instead of their drive_c, and then the underlying
mechanism will:
* Overlay drive_c.
* Copy the registry (and dosdevices?) if it's not found on the new
prefix, using same rules as an overlay would (i.e. the topmost one).
Well it might be more work but it seems more right to me. The FUSE layer
will add some overhead, unfortunately. I don't know if it's significant
for Wine/Windows stuff, but I'd imagine it can be if someone installs an
application in drive_c somewhere in Program Files, which makes heavy use
of file I/O.
It's a shame we can't use the native overlayfs due to privileges...
Obviously there's no point in trying to improve this further if it's not
an idea worth pursuing for the Wine project. I just gave my 2 cents.
12:56 p.m.
On 3/30/20 2:50 PM, Gabriel Ivăncescu wrote:
Hi Rémi,
I like the general idea personally. I use something similar on my setup,
which is custom though, but I use the overlayfs itself rather than the
FUSE variant (well since I could give it privileges, not suitable for
distribution).
Thanks!
Is there a reason you're moving the registry
files? As you said, they'll
get quickly overwritten anyway and doesn't make much sense to me. Of
course, to "solve" this you'd have to just set multiple prefixes in
WINEPREFIX_OVERLAYS instead of their drive_c, and then the underlying
mechanism will:
The reason is mostly because of the disabled wineboot. In order for the
upper layer to have the correctly initialized registry on the initial
prefix creation, it has to read it from the lower dir.
Of course with a smarter prefix update that would maybe not be necessary
but I was on the easy path.
--
Rémi Bernon <rbernon(a)codeweavers.com>
31 Mar
31 Mar
4:18 p.m.
Hi,
Am 30.03.2020 um 14:24 schrieb Rémi Bernon
<rbernon(a)codeweavers.com>om>:
I have been playing with fuse-overlayfs [1] lately to reduce prefix size
and speed up wine startup time, and I thought maybe someone else could
be interested, and I would definitely be interested in some feedback,
especially if anyone already tried.
I have used fuse-overlayfs in the past for my lan party setup to have multi-user installs
of games that insist on writing to their own installation directory. One big trouble I ran
into was performance. For this reason I have eventually replaced it with AUFS.
This was ~7 years ago though, I have been using AUFS ever since and did not test the
in-kernel overlayfs yet. The big complicating factor was that I ran everything off nfsroot
and did the fuse-overlayfs mount on the client machine side, so every file lookup sent two
file lookups to the server. Two games that were horribly slow due to this were Warhammer
40k: Dawn of War (+expansions) and Crashday. Both of them have a directory with thousands
of files and open them with case-sensitivity mismatches. The regular load time of the
games was in the area of seconds whereas the fuse-overlayfs solution took minutes to load
the games.
1 Apr
1 Apr
9:07 p.m.
Hi Rémi,
On 30.03.2020 14:24, Rémi Bernon wrote:
I have been playing with fuse-overlayfs [1] lately to
reduce prefix size
and speed up wine startup time, and I thought maybe someone else could
be interested, and I would definitely be interested in some feedback,
especially if anyone already tried.
This is an interesting experiment, do we really need overlayfs to
achieve that? We control all access to files, so it seems we could
implement such copy on write in ntdll. On prefix creation we could
create only some 'markers' in drive_c that ntdll could recognize and
lookup builtin files for the actual file to open. When opening for read,
it would just open the file from Wine installation. When opening for
write, we would need to create the actual file and potentially copy it.
I'm sure there are a number of things making it tricky, but it should be
doable. This would not have any of mentioned prefix update problems and
would be more portable.
One possibility for those 'markers' is something discussed in context of
Windows symlinks. It could be a UNIX symlink with a magic target with
magic prefix in its target.
Thanks,
Jacek
1509
days inactive
1511
days old
11 comments
5 participants
participants (5)
-
Gabriel Ivăncescu -
Jacek Caban -
Rémi Bernon -
Stefan Dösinger -
Zebediah Figura