Commit graph

70 commits

Author SHA1 Message Date
Christian Schneider
dcc1c648f2 Add parameters for LZMA memory and thread count
- Use at least the memory for LZMA preset 1
- Added methods for memory and thread defaults to avoid code duplication
- Fix: Precomp Comfort used invalid Compression_Method as default
- Fix: Use singular if only 1 thread is used
- Fix: precomp_dll.h had use_mp3 declared twice
2017-04-05 16:49:58 +02:00
Christian Schneider
2aab7dffa5 Preparing configurable LZMA max. memory/thread count 2017-04-05 15:30:02 +02:00
Christian Schneider
ea7af5f2fd Show LZMA block size 2017-04-05 13:44:45 +02:00
Christian Schneider
2112d8be7b Bump copyright year 2017-04-05 10:57:08 +02:00
Christian Schneider
55f78fd763 Fix garbled output with -nl
First percentage (0.00%) is shown after LZMA thread/memory usage now,
like it is in -cl mode
2017-03-28 07:21:10 +02:00
Christian Schneider
f86c9b420e Improve JPG parsing
- fixes JPG part of #52
- improves JPG parsing speed (issue #5)
2017-03-12 11:00:38 +01:00
Christian Schneider
7d4ab6554c Fix for MP3 part of #52
Incomplete MP3 frames were detected before, now they get discarded, too
2017-03-12 10:11:14 +01:00
Christian Schneider
9ddc049521 Fix for incorrectly restored multi PNGs
- Using a signed int instead of an unsigned solves the issue
- Fixes #50
2017-01-02 11:32:08 +01:00
Christian Schneider
67091fcbbd Fix for slowdown on some files
- identical_bytes and compressed_stream_size don't have to be the same
anymore, as for some files they differ although decompressed bytes match
- tolerance for penalty bytes (5*32 bytes)
2017-01-02 11:22:01 +01:00
Christian Schneider
a7a2b78feb Show lzma memory usage in MiB 2016-12-29 18:34:34 +01:00
Christian Schneider
28f3a72899 Use multithreaded lzma2 only 2016-12-29 18:31:47 +01:00
Christian Schneider
5568734be5 Use the LZMA preset that satisfies memory limits 2016-12-29 18:07:51 +01:00
Christian Schneider
52be43eaa4 Show OS and 32-/64-bit information 2016-12-28 16:52:45 +01:00
Christian Schneider
769bdc3a74 Autodetect number of threads for LZMA
- Print memory usage
- Minor changes in compress_easy_mt.cpp (indentation, comments, removing
some unnecessary code)
2016-12-28 11:14:41 +01:00
Christian Schneider
b581197c2b Renaming bz2_in/out -> otf_in/out 2016-11-09 16:39:33 +01:00
Christian Schneider
865a7c85e8 Fixed enum typo
Led to a compilation error in comfort version
2016-08-27 12:28:26 +02:00
Christian Schneider
acff85e316 Removing old commented code 2016-07-05 14:37:39 +02:00
Christian Schneider
fab8c2b488 Enum for on-the-fly compression methods 2016-07-02 18:37:31 +02:00
Christian Schneider
294fc3f128 Converting from/to liblzma
- conversions using -nl, -nx, -nm
- case cleanup (2,3,4 instead of 4,2,3)
- two switch statements replaced by if
2016-07-02 18:24:23 +02:00
Christian Schneider
8fc79d94b3 Always use liblzma
- #ifdefs removed
- Removed lzma_end call from Ctrl+C handler, cleaning up stream state
isn't necessary there.
2016-07-02 18:06:22 +02:00
Christian Schneider
a512eb30de On-the-fly liblzma compression and decompression works
- LZMA_CONCATENATED flag removed (not needed because of a single stream)
- Code adapted from liblzma decompress.cpp example
- Sleep in own_fwrite removed
2016-07-02 04:45:05 +02:00
Christian Schneider
6631a9acf8 Parantheses for clarity and against compiler warning 2016-06-26 10:41:53 +02:00
Gavin.Gao
258d484083 add otf_compression methods: lzma / xz(lzma2) / xz_MT 2016-06-19 16:53:50 +08:00
Gavin.Gao
03eb16c8c3 vs2008 vcproj
precomf &
paq8o8pre with precomp_dll
2016-06-12 23:20:00 +08:00
Christian Schneider
1ca25c6c63 Verbose mode: Print both compression and memory level
- Before: Only compression level 1-9 was shown
- After. Compression and memory level are shown similar to -zl parameter
(e.g. compression level 5, memory level 6 shows as "level combination
56")
2016-06-10 22:58:17 +02:00
Christian Schneider
a8c59cb82e Fix for "Compressed size:" output in non-verbose mode 2016-06-07 10:58:33 +02:00
Christian Schneider
d1d4263ae6 Fix for cases with worse compression II
- original compressed size is tracked now
- trying combination stop has a new constraint: new compressed size must
be the same as the original compressed size
- fixed some usages of identical_bytes_decomp (instead of
best_identical_bytes_decomp) that could lead to "No matches" although
matches were found
2016-06-05 11:49:48 +02:00
Christian Schneider
074e9fd621 Fix for cases with worse compression
- there were cases where everything was decompressed, but penalty bytes
were used and trying combinations was stopped too early
- penalty byte must now be 0 before trying combinations is stopped
2016-06-05 10:22:33 +02:00
Christian Schneider
f9afdbd7df One temporary file less
- Merged compare_file_mem and compare_files_penalty
- There was a temporary file between the two functions before which is
not needed anymore, comparisons are done in memory on 512 byte chunks
- Result on silesia.zip: 15% faster (30,7 s => 26,2 s)
- See issue #14
2016-06-02 07:07:05 +02:00
Christian Schneider
1d36916341 Removed unused variables 2016-06-01 09:37:37 +02:00
Christian Schneider
85048e1ed9 Partial matches removed for GIF
- Only some files used partial matches, on most of them, compression was
worse with them
- Now "recompression failed" is printed instead and the stream doesn't
get processed further
2016-06-01 09:32:51 +02:00
Christian Schneider
5a85735f2e Removed dead code
- compare_files_penalty for "real identical bytes" is useless now, as it
gives the same result as the compare_files_penalty done before
- file truncation is unnecessary
2016-06-01 06:56:43 +02:00
Christian Schneider
0d0b754e47 Use 2 temporary files less for partial bZip2 matches
- See issue #14
2016-05-30 13:54:09 +02:00
Christian Schneider
faa4776346 Use 2 temporary files less for partial matches
- Before, partial matches led to additional decompression and
recompression to determinate the precise size of the partial match
- We can remove these 2 steps by using a slightly less precise size that
is known before
- Results for silesia.zip: 20% faster (37->30 seconds), 0.2% overhead in
bZip2 compressed variant (64,629,217 -> 64,764,849 bytes)
2016-05-28 09:54:57 +02:00
Christian Schneider
6fdb8af409 Corrected final_compression_found handling
- Breaking out of both loops when final compression was found in all
two-loop cases (zip, gzip, brute)
- Prevents unnecessary additional try_recompress calls
2016-05-20 16:12:58 +02:00
Christian Schneider
74fc64d92a Version 0.4.6 2016-05-11 11:34:49 +02:00
Christian Schneider
1bf4d71edc Cache for MP3 parsing
- Avoid parsing of MP3 streams that have been parsed before
- For example, improves speed for testfile from issue #37 (before: 90 s,
after: < 1 s)
2016-05-06 12:12:36 +02:00
Christian Schneider
e9f370479f Slowdown fix for "big value pairs out of bounds"
- If there was a "big value pairs" error, ignore following streams with
the same position/length sum
- Fixes issue #37
2016-05-06 11:31:47 +02:00
Christian Schneider
e05f0e9e6e Validate MP3 frames
- Some of the validation code from packMP3 is done in Precomp to speed
up the process and reduce false positives
- Fixes issues #35 and #19
2016-04-29 16:23:40 +02:00
Christian Schneider
cce57ccdc9 Fix Base64 recursion bug
- Allocate memory for Base64 line lengths dynamically instead of using a
global variable that got overwritten in recursion
- Fixes issue #36
2016-04-29 09:23:54 +02:00
Christian Schneider
5edc074b4d JPG recompression ("-r") in memory for files up to 64 MB
- Related: Issue #14
2016-04-21 14:56:38 +02:00
Christian Schneider
a00e0ff5bd Compress JPG files up to 64 MB in memory
- Related: Issue #14
2016-04-21 08:21:04 +02:00
Christian Schneider
fefc6a41ea MP3 recompression ("-r") in memory for files up to 64 MB
- Helper functions renamed to fast_copy overrides
- Helper functions added to precomp.h
- Related: Issue #14
2016-04-19 06:22:48 +02:00
Christian Schneider
a75eb06711 Recompress MP3 files up to 64 MB in memory
- Added helper functions to copy data, file<->memory
- Related: Issue #14
2016-04-18 14:13:10 +02:00
Christian Schneider
c8dc5a0980 Parsing of unsupported MP3 streams improved
- For supported MP3 streams, all frame header information must be the
same for the whole stream.
- For unsupported MP3 streams, this has been removed and only the stream
type has to be the same. This will lead to some more false positives,
but give longer streams, so the log is not bloated with thousands small
false positives
2016-04-11 13:26:35 +02:00
Christian Schneider
5cf1063855 Information about unsupported MP3 formats in verbose mode
Solves issue #28
2016-04-02 12:25:40 +02:00
Christian Schneider
6e7f489439 Ignore compressed SWF version byte
Solves issue #31
2016-03-21 14:06:57 +01:00
Christian Schneider
93c1988ef3 No more tabs
Converted tabs to spaces to get a consistent code layout independent of
tab length
2016-03-19 09:13:54 +01:00
Christian Schneider
e818a6b0c8 Fix for MP3 slowdown
- Fixes issue #29
- Fixes wrong MP3/zlib lengths being written introduced by commit
491ebec
2016-03-17 10:35:48 +01:00
Christian Schneider
70e29b2f77 Header files
- Moved contrib declarations from precomp.cpp in header files in contrib
directories
- Moved Precomp declarations to precomp.h
- Renamed old precomp.h to precomp_dll.h
2016-03-05 16:15:19 +01:00