As I prepared tests, I tried other compressors that I have in here too. Results turned out to be worse than I expected...Is size gain worth having longer startup, bigger installers and risk of breaking something?
Read and discuss.
Data: a compilation of ~30 executables, 104 914 944 bytes.
All tests done with Pentium D 805 @2.66 GHZ.
Results:
Code: Select all
Program size decompression time switches used
UPX NRV2B 37060096 0.968 --best --all-filters --crp-ms=99999 --nrv2b
UPX NRV2E 36585472 1.000 --best --all-filters --crp-ms=99999 --nrv2e
UPX LZMA 28961792 6.453 --best --all-filters --crp-ms=99999 --lzma
Upack 28923468 37.422 -brute
kkrunchy error (application failed to initialaize properly)
mpress error (bad executable)
MEW error (crash after 34.828)
20to4 error (application hangs)
Upack is very slow and just slightly better than UPX LZMA, IMO has no practical uses.
Only UPX left, which mode to use?
NRV2B decompresses at 103.3 MB/s, NRV2E 100, LZMA - 15.5.
Now count in hdd / pendrive speed... it gets tougher.
At startup compressed applications have to read whole executable, uncompressed - only a part of it. How big? There's no easy answer, it highly depends on particular application. Let's try to measure it. The accurate method would be to ask Windows memory manager about it, but I don't know of any tool that would do this. I could write such myself, but now I lack access to Microsoft documentation, which is necessary.
But there's another thing that we can use...Compressed executables read whole executable and this is expressed in amount of memory that they use. The trick to run the same program 2 times, once compressed and once not and measure the difference. It's a bit less accurate though.
Important thing: Many applications are combination of .exes and .dlls. Not all of them are being loaded on the startup and we should calculate startup usage ratio(SUR) for total size of what's in use. All good taks managers let you see the list of loaded modules, so that's easy.
My test results:
Code: Select all
SumatraPDF - 16.9%
CDex - 25.3%
KMPlayer - 34.1%
PhotoScape - 35.8%
VirtualDub - 40.4%
TrueCrypt - 40.7%
WinHTTrack - 40.8%
AngelWriter - 43.0%
Rainlendar - 45.9%
Artweaver - 46.0%
Media Coder - 46.5%
DSynchronize - 46.9%
FSViewer - 47.2%
Audio Identifier - 52.7%
FreeOCR - 55.4%
InfraRecorder - 55.9%
Ant Movie Catalog - 57.0%
IcoFX - 57.0%
PStart - 57.6%
PSPad - 64.0%
HxD - 67.6%
Dev Project Manager - 72.3%
eToolz - 78.6%
Opera - 95.1%
Uncompressed application load speed:
Code: Select all
read_speed/startup_usage_ratio
Code: Select all
1/(compression_ratio/read_speed+1/decompression_speed)
Code: Select all
Compression: Speed(30% SUR) Speed(50% SUR)
Uncompressed: 100.0 MB/s 60.0 MB/s
NRV2B 46.6 MB/s 46.6 MB/s
NRV2E 46.3 MB/s 46.3 MB/s
LZMA 13.6 MB/s 13.6 MB/s
But how does it work in practice? I think that the limit of what can you notice is about 0.05s. How big program can you compress with virtually no load time increase?
Code: Select all
Compression: Size(0% SUR) Size(30% SUR) Size(50% SUR)
NRV2B 2.33 MB 4.36 MB 10.45 MB
NRV2E 2.31 MB 4.30 MB 10.09 MB
LZMA 0.67 MB 0.78 MB 0.87 MB
Variant2: slow pendrive, 10 MB/s.
Code: Select all
Compression: Speed(30% SUR) Speed(50% SUR)
Uncompressed: 33.3 MB/s 20.0 MB/s
NRV2B 22.2 MB/s 22.2 MB/s
NRV2E 22.3 MB/s 22.3 MB/s
LZMA 10.9 MB/s 10.9 MB/s
Compression: Size(0% SUR) Size(30% SUR) Size(50% SUR)
NRV2B 1.11 MB 3.33 MB every
NRV2E 1.11 MB 3.36 MB every
LZMA 0.54 MB 0.80 MB 1.18 MB
Variant3: HDD, 80 MB/s.
Code: Select all
Compression: Speed(30% SUR) Speed(50% SUR)
Uncompressed 266.7 MB/s 200.0 MB/s
NRV2B 71.0 MB/s 71.0 MB/s
NRV2E 69.7 MB/s 69.7 MB/s
LZMA 14.7 MB/s 14.7 MB/s
Compression: Size(0% SUR) Size(30% SUR) Size(50% SUR)
NRV2B 3.78 MB 4.83 MB 6.09 MB
NRV2E 3.70 MB 4.71 MB 5.89 MB
LZMA 0.74 MB 0.77 MB 0.80 MB
Claims that UPX does not slow down application startup are wrong. It usually doesn't have a big impact, but applications cases where it does happen often enough to be significant.
For very small programs you can use LZMA. You'll gain little, but there's virtually no loss.
For bigger ones use NRV2E, it compresses slightly better and because of this it's faster in pessimistic case and usually practically as fast.
Other thoughts:
1. Machines about twice slower than mine are still in use today. On them impact of decompression speed is going to be bigger, I'd say that the upper limit for UPX LZMA should be no higher than 300-400 KB unless you know what machines you'll be using it at. That's for all application's .exes and .dll together.
2. Compressed executables are loaded to memory almost sequentially, while uncompressed not necessarily. This can reduce startup performance of uncompressed programs when running from HDD, but I think it matters very rarely.
3. When loading multiple instances of the same executable OS optimizes the process: everything is loaded only once. Packing breakes this, every instance has to be loaded separately, it's slower and takes more memory.
Things like Google Chrome, where each tab is a different process shouldn't be packed. The same applies to dlls shared by programs.
4. In my test I made a one big executable containing many smaller ones. It affects the results in several ways:
- Bigger data tends to compress better, real compression ratios are going to be somewhat worse.
- Each executable contains a header which has some constant sized parts that can't be compressed. It significantly affects compression ratios of small files, while here it's not significant. (Upack scores better on smaller files )
Icon has the same impact.
- Compressors can do some optimizations that are section-specific. Putting everything in .rsrc like I did could break them. I guess that it's not the case because compression ratios are very good compared to what standalone programs can do.
5. Some programs have something called "overlay data" which can take a lot of their size. Packers can do 2 things with it: Either preserve unchanged or remove, removal can break them. This greatly affects compression and startup usage ratios, but:
a) It doesn't change load time.
b) Ratios above assumed no overlay data.
Therefore it can change size gain, but doesn't break performance calculations.
6. There's one thing that makes SUR calculations a bit less accurate. When you start application, it's first icon is practically always loaded into memory even when it's not used. Because your file manager loaded it. This can account for like 100 KB difference, but only in rare cases.
7. I remember reading somewhere that UPX does in place decompression, after it's done no memory is wasted. If this is wrong, SUR is going to be slightly higher, but usually much less than 0.1% anyway. Anyway, if anybody knows how is it, please let me know.
8. The method of calculating SUR based on memory difference is incorrect in some cases - where there are multiple executables stuffed in a single one. I.e. Sysinternals Process Explorer works this way. It often doesn't work with launchers either, many quits before you can measure something, some are written in NSIS and have plugins stuffed inside - like Process Explorer.
I didn't include any problematic app in this test.
9. Size gain can be different than it looks to be at first sight. What matters is not file size, but how much does it take from your hard drive. Real size is usually file size aligned to some value from 4-32 KB + some constant (called "metadata").
It makes no sense to compress files that have 4 KB, you'll only loose this way. With many bigger ones it's the case too.
10. UPXed files = bigger installers.
For the sake of completeness, program versions:
Code: Select all
UPX 3.03w
Upack 0.399
MEW11 SE
kkrunchy 0.23 alpha 2
Mpress 1.25
20to4 2004.04.18