[Bug 45847] New: Warframe installer update/ download gets stuck due to corruption on large files in 'Bulk Download' mode (tries over and over again)
wine-bugs at winehq.org
wine-bugs at winehq.org
Mon Sep 17 03:23:09 CDT 2018
https://bugs.winehq.org/show_bug.cgi?id=45847
Bug ID: 45847
Summary: Warframe installer update/download gets stuck due to
corruption on large files in 'Bulk Download' mode
(tries over and over again)
Product: Wine
Version: 3.16
Hardware: x86-64
OS: Linux
Status: NEW
Severity: normal
Priority: P2
Component: wininet
Assignee: wine-bugs at winehq.org
Reporter: focht at gmx.net
Distribution: ---
Hello folks,
cloned off from bug 45734
There is troll who has no idea how Bugzilla works.
I made the mistake to comment there.
Partial copy/pasta of my initial analysis from
https://bugs.winehq.org/show_bug.cgi?id=45734#c12
Prerequisites:
* work around bug 38960 ("Multiple applications need
'kernel32.SetFileCompletionNotificationModes' implementation (Steam
'steamwebhelper.exe' and Blizzard Battle.net desktop app CEF v3.2+/Chromium
v44+ crash in Win7 mode, Sony Xperia Companion 1.7.x)"
-> 'winetricks winxp' or use Wine-Staging if you need 'Windows 7' mode for
whatever reasons
* work around bug 37797 ("Warframe updater/installer crashes (CEF3 with
Blink/Google Chrome and V8 checks requested font facename, doesn't like
Liberation replacement, Arial font)") -> dupe of bug 32342 ("Multiple
applications and games using 'Webkit' or 'Blink' web browser engine crash on
startup ('Arial' and 'Times New Roman' font face name validation)")
-> 'winetricks -q corefonts' or use Wine-Staging
--- snip ---
$ find .wine -iname *.log
.wine/winetricks.log
.wine/drive_c/windows/DirectX.log
.wine/drive_c/windows/logs/DirectX.log
.wine/drive_c/users/focht/Local Settings/Application Data/Warframe/Launcher.log
--- snip ---
'Launcher.log' is the log file one needs to check for download problems/errors.
It shows each file/chunk downloaded.
There seems to be a problem with very large files (archives) when 'Bulk
Download' mode is used. For example there is always corruption happening in
some chunks for the file
'F.TextureDx9.cache.7909D15B39359A27EDEF21DEB9D86D3B.lzma' (14 GB). Since it's
streaming LZMA it gets detected (see "Bad LZMA stream" in launcher log) and
re-downloaded over and over again.
--- snip ---
...
Queuing Z:\home\focht\Downloaded\Public\Cache.Windows\F.TextureDx9.cache
[7909D15B39359A27EDEF21DEB9D86D3B]
Queuing Z:\home\focht\Downloaded\Public\Cache.Windows\F.TextureDx9.toc
[DCBE1867172A86A365910B6CB36BFC4B]
...
HTTP GET
/Cache.Windows/F.TextureDx9.cache.7909D15B39359A27EDEF21DEB9D86D3B.lzma
GET //Cache.Windows/B.Misc.cache.17C3FB9222153CF642694D33955737BD.lzma HTTP/1.1
Host: content.warframe.com
Connection: Keep-Alive
HTTP/1.1 200 OK
Date: Sun, 16 Sep 2018 11:40:51 GMT
Connection: Keep-Alive
Accept-Ranges: bytes
Cache-Control: public, max-age=86400
Content-Length: 2035849776
Content-Type: chemical/x-cache
Last-Modified: Wed, 01 Aug 2018 20:21:44 GMT
ETag: "79589a30-572657175ce04"
X-HW: 1537098051.dop014.fr8.t,1537098051.cds035.fr8.c
GET //Cache.Windows/F.TextureDx9.cache.7909D15B39359A27EDEF21DEB9D86D3B.lzma
HTTP/1.1
Host: content.warframe.com
Connection: Keep-Alive
HTTP/1.1 200 OK
Date: Sun, 16 Sep 2018 11:40:51 GMT
Connection: Keep-Alive
Accept-Ranges: bytes
Cache-Control: public, max-age=86400
Content-Length: 14112683568
Content-Type: chemical/x-cache
Last-Modified: Wed, 01 Aug 2018 20:24:53 GMT
ETag: "3492e7630-572657cb775b4"
X-HW: 1537098051.dop002.fr8.t,1537098051.cds003.fr8.c
Downloading @ 6 MB/s
Downloading @ 6 MB/s
...
Downloading @ 6 MB/s
Bad LZMA stream in
//Cache.Windows/F.TextureDx9.cache.7909D15B39359A27EDEF21DEB9D86D3B.lzma
HTTP GET
/Cache.Windows/F.TextureDx9.cache.7909D15B39359A27EDEF21DEB9D86D3B.lzma
GET //Cache.Windows/F.TextureDx9.cache.7909D15B39359A27EDEF21DEB9D86D3B.lzma
HTTP/1.0
Host: content.warframe.com
Pragma: no-cache
Connection: Keep-Alive
HTTP/1.0 200 OK
LzmaDec_Allocate failed for
//Cache.Windows/F.TextureDx9.cache.7909D15B39359A27EDEF21DEB9D86D3B.lzma
HTTP GET
/Cache.Windows/F.TextureDx9.cache.7909D15B39359A27EDEF21DEB9D86D3B.lzma
GET //Cache.Windows/F.TextureDx9.cache.7909D15B39359A27EDEF21DEB9D86D3B.lzma
HTTP/1.1
Host: content.warframe.com
Pragma: no-cache
Connection: Keep-Alive
HTTP/1.1 200 OK
Date: Sun, 16 Sep 2018 11:54:01 GMT
Connection: Keep-Alive
Accept-Ranges: bytes
Cache-Control: public, max-age=86400
Content-Length: 14112683568
Content-Type: chemical/x-cache
Last-Modified: Wed, 01 Aug 2018 20:24:53 GMT
ETag: "3492e7630-572657cb775b4"
X-HW: 1537098841.dop017.fr8.t,1537098841.cds003.fr8.c
Downloading @ 6 MB/s
Downloading @ 6 MB/s
...
Downloading @ 6 MB/s
Bad LZMA stream in
//Cache.Windows/F.TextureDx9.cache.7909D15B39359A27EDEF21DEB9D86D3B.lzma
HTTP GET
/Cache.Windows/F.TextureDx9.cache.7909D15B39359A27EDEF21DEB9D86D3B.lzma
GET //Cache.Windows/F.TextureDx9.cache.7909D15B39359A27EDEF21DEB9D86D3B.lzma
HTTP/1.0
Host: content.warframe.com
Pragma: no-cache
Connection: Keep-Alive
HTTP/1.0 200 OK
LzmaDec_Allocate failed for
//Cache.Windows/F.TextureDx9.cache.7909D15B39359A27EDEF21DEB9D86D3B.lzma
HTTP GET
/Cache.Windows/F.TextureDx9.cache.7909D15B39359A27EDEF21DEB9D86D3B.lzma
GET //Cache.Windows/F.TextureDx9.cache.7909D15B39359A27EDEF21DEB9D86D3B.lzma
HTTP/1.1
Host: content.warframe.com
Pragma: no-cache
Connection: Keep-Alive
HTTP/1.1 200 OK
Date: Sun, 16 Sep 2018 12:05:44 GMT
Connection: Keep-Alive
Accept-Ranges: bytes
Cache-Control: public, max-age=86400
Content-Length: 14112683568
Content-Type: chemical/x-cache
Last-Modified: Wed, 01 Aug 2018 20:24:53 GMT
ETag: "3492e7630-572657cb775b4"
X-HW: 1537099544.dop009.fr8.t,1537099544.cds003.fr8.c
Downloading @ 6 MB/s
Downloading @ 6 MB/s
...
Downloading @ 6 MB/s
Bad LZMA stream in
//Cache.Windows/F.TextureDx9.cache.7909D15B39359A27EDEF21DEB9D86D3B.lzma
Downloaded 11,630,780,172 bytes total (25,743,463,740 expected) ~ 6,349,930 B/s
Finished Binaries1 (1831.7 seconds)
...
--- snip ---
To work around, uncheck the 'Bulk download' mode in settings.
Interestingly this mode is not available in 32-bit WINEPREFIX.
Both, '64-bit Mode' and 'Bulk Download' are automatically disabled in settings
there.
The downloads are ordinary webserver/webdav file requests via 'wininet'.
If you start the launcher as follows you will see all of them (lots of spam
though):
--- snip ---
$ WINEDEBUG=+loaddll,+process,+wininet wine msiexec -i Warframe.msi
--- snip ---
All the files can be downloaded without Warframe installer/launcher by tracing
the installer on +wininet debug channel. Their content is delivered as LZMA
compressed files/streams.
Get list of all files to be downloaded:
--- snip ---
$ wget -qO- /dev/null content.warframe.com//origin/310F099E/index.txt.lzma |
lzmadec
...
/Cache.Windows/F.TextureDx9.cache.7909D15B39359A27EDEF21DEB9D86D3B.lzma,14112683568
--- snip ---
NOTE: All paths are relative to your Warframe download folder. It depends on
your selection when your started the updater first time. I selected
"c:\\warframe" in my case.
--- snip ---
$ pwd
/home/focht/.wine/drive_c/warframe/Downloaded/Public
--- snip ---
Download one specific file via 'wget' from 'content.warframe.com' using the
file path from list:
--- snip ---
$ wget
content.warframe.com/Cache.Windows/F.TextureDx9.cache.7909D15B39359A27EDEF21DEB9D86D3B.lzma
--2018-09-16 14:58:15--
http://content.warframe.com/Cache.Windows/F.TextureDx9.cache.7909D15B39359A27EDEF21DEB9D86D3B.lzma
Resolving content.warframe.com (content.warframe.com)... 205.185.216.10,
205.185.216.42
Connecting to content.warframe.com (content.warframe.com)|205.185.216.10|:80...
connected.
HTTP request sent, awaiting response... 200 OK
Length: 14112683568 (13G) [chemical/x-cache]
Saving to: ‘F.TextureDx9.cache.7909D15B39359A27EDEF21DEB9D86D3B.lzma’
F.TextureDx9.cache.7909D15B39359A27EDEF21DEB9D86D
100%[==========================================================================================================>]
13.14G 6.07MB/s in 39m 42s
2018-09-16 15:37:57 (5.65 MB/s) -
‘F.TextureDx9.cache.7909D15B39359A27EDEF21DEB9D86D3B.lzma’ saved
[14112683568/14112683568]
$ lzmainfo F.TextureDx9.cache.7909D15B39359A27EDEF21DEB9D86D3B.lzma
F.TextureDx9.cache.7909D15B39359A27EDEF21DEB9D86D3B.lzma
Uncompressed size: 15749 MB (16514030011 bytes)
Dictionary size: 64 MB (2^26 bytes)
Literal context bits (lc): 3
Literal pos bits (lp): 0
Number of pos bits (pb): 2
$ xz --format=lzma --decompress
F.TextureDx9.cache.7909D15B39359A27EDEF21DEB9D86D3B.lzma
$ md5sum F.TextureDx9.cache.7909D15B39359A27EDEF21DEB9D86D3B
7909d15b39359a27edef21deb9d86d3b
F.TextureDx9.cache.7909D15B39359A27EDEF21DEB9D86D3B
$ cp F.TextureDx9.cache.7909D15B39359A27EDEF21DEB9D86D3B
~/.wine/drive_c/warframe/Downloaded/Public/Cache.Windows/F.TextureDx9.cache
--- snip ---
Essentially with a few script lines you could write your own Warframe
updater/downloader ;-)
--- snip ---
# retrieve all files for 'Cache.Windows' folder in Warframe download cache
$ wget -qO- /dev/null content.warframe.com//origin/310F099E/index.txt.lzma |
lzmadec | sed -ne 's/^\(\/Cache.Windows.*\),.*/content.warframe.com\1/p' |
xargs wget -c
# uncompress downloaded files (keep originals)
$ find . -iname "*.lzma" -exec xz --format=lzma --decompress --keep {} \;
# copy all files that have the md5sum pattern extension to target folder (while
dropping the md5sum extension)
$ find . -regextype sed -regex ".*.[A-F0-9]\{31\}$" -exec bash -c 'cp "$0"
~/".wine/drive_c/warframe/Downloaded/Public/Cache.Windows/${0%.*}"' {} \;
--- snip ---
This looks like 'wininet' related. Unfortunately 'winetricks -q wininet'
provides a very old version of 'wininet.dll' which can't be used due to Win7+
features (Warframe will refuse).
$ sha1sum Warframe.msi
cd9b81c88d0b2888772e1cab570246d5b5b29131 Warframe.msi
$ du -sh Warframe.msi
40M Warframe.msi
$ wine --version
wine-3.16
Regards
--
Do not reply to this email, post in Bugzilla using the
above URL to reply.
You are receiving this mail because:
You are watching all bug changes.
More information about the wine-bugs
mailing list