This website requires JavaScript.
d6799635a5
Merge pull request #756 from facebook/memOpt
Yann Collet and GitHub
2017-07-12 18:32:37 -07:00
7c886db0a8
changed to stderr
Paul Cruz
2017-07-12 17:28:53 -07:00
b5b18cf664
changed to malloc, added comment about adaptive compression level, and changed ternary operators
Paul Cruz
2017-07-12 17:10:58 -07:00
954d999abf
fixed up freeCCtx() removed BYTE since it wasn't being used
Paul Cruz
2017-07-12 16:50:43 -07:00
3c16edd26a
added copyright header, removed clean from makefile
Paul Cruz
2017-07-12 16:40:24 -07:00
8de82b6eb0
[ldm] Clean up versions
Stella Lau
2017-07-12 16:31:31 -07:00
74d3a6f5ae
passes tests with adaptive compression level
Paul Cruz
2017-07-12 16:18:41 -07:00
5353d350ae
working with fixed compression level and fixed dictionary size
Paul Cruz
2017-07-12 16:02:20 -07:00
8ff8cdb15b
[ldm] Clean up code
Stella Lau
2017-07-12 15:11:06 -07:00
33cecaaaca
Merge branch 'dev' of github.com:facebook/zstd into dev
Yann Collet
2017-07-12 14:38:36 -07:00
8ef666c325
slightly increased buffer pool, to cover normal "full load" scenarios
Yann Collet
2017-07-12 14:23:34 -07:00
356ddb649f
working with flush job->src.size and fixed cLevel
Paul Cruz
2017-07-12 12:21:21 -07:00
3a48ffd4fd
Fix sumToHash to use hash space more efficiently
Stella Lau
2017-07-12 10:53:19 -07:00
e0d4162464
Minor fix for non-rolling hash
Stella Lau
2017-07-12 09:50:24 -07:00
50502519fb
Switch to using rolling hash only
Stella Lau
2017-07-12 09:47:00 -07:00
583dda17a8
Update rolling hash
Stella Lau
2017-07-11 18:13:26 -07:00
052a95f77c
fix : ZSTDMT_compress_advanced() correctly generates checksum
Yann Collet
2017-07-11 17:18:26 -07:00
0a401852c4
added debug statement
Paul Cruz
2017-07-11 16:50:50 -07:00
2a62f48bf4
release input buffers from inside worker thread
Yann Collet
2017-07-11 15:56:40 -07:00
72a183efad
changed dictionary size, added debugging statements
Paul Cruz
2017-07-11 15:49:52 -07:00
57236184af
buffer pool : all buffers have same size
Yann Collet
2017-07-11 15:17:25 -07:00
7c54e09347
updated DEBUG statements
Paul Cruz
2017-07-11 15:15:41 -07:00
a3c077b8c6
added error message, updated copying dictionary into the input buffer
Paul Cruz
2017-07-11 15:00:52 -07:00
34b2b95631
zstdmt : intermediate outBuffer allocated from within worker
Yann Collet
2017-07-11 14:59:10 -07:00
16261e6951
buffer pool can be invoked from multiple threads
Yann Collet
2017-07-11 14:14:07 -07:00
fe80d45807
Merge pull request #755 from jacquesg/solaris
Yann Collet and GitHub
2017-07-11 12:11:44 -07:00
34afb9b23e
changed to using ZSTD_compressBegin_usingDict() and fixed strange issue with ZSTD_compressContinue()
Paul Cruz
2017-07-11 11:50:00 -07:00
c325c8db82
fix missing symbol 'nanosleep' for Solaris
Jacques Germishuys
2017-07-11 19:25:14 +02:00
7ec5928626
fixed an error where -c argument wasn't working for single files
Paul Cruz
2017-07-11 10:23:25 -07:00
f6c5d07fe2
Save v3
Stella Lau
2017-07-11 09:23:44 -07:00
ef0ff7fe7f
zstdmt: removed margin for improved memory usage
Yann Collet
2017-07-11 08:54:29 -07:00
6c3673f4c3
Add rolling hash
Stella Lau
2017-07-10 22:27:43 -07:00
f918545491
made some progress on improving compression ratio, but problems exist with speed limits, and for some reason higher compression levels are really slow
Paul Cruz
2017-07-10 18:16:42 -07:00
4616fad18b
improved ZSTDMT_compress() memory usage
Yann Collet
2017-07-10 17:16:41 -07:00
670b1fc547
optimized memory usage for ZSTDMT_compress()
Yann Collet
2017-07-10 16:30:55 -07:00
01fc7c4244
changed how the detection of the last job works
Paul Cruz
2017-07-10 16:27:58 -07:00
c36552ef8a
dst buffer should use ZSTD_compressBound to determine how much space it needs
Paul Cruz
2017-07-10 16:10:19 -07:00
7aa36df6df
fixed memory leak that was happening when creating jobs
Paul Cruz
2017-07-10 16:03:09 -07:00
ef2b728316
Clean up and refactor compress function
Stella Lau
2017-07-10 15:48:47 -07:00
e410d63d45
made input buffer an internal part of the compression context
Paul Cruz
2017-07-10 15:37:14 -07:00
3510efb02d
fix : custom allocator correctly propagated to child contexts
Yann Collet
2017-07-10 14:21:40 -07:00
ee3423d709
extended fuzzer MT memory tests
Yann Collet
2017-07-10 14:09:16 -07:00
88da8f1816
fix : propagate custom allocator to ZSTDMT though ZSTD_CCtx_setParameter()
Yann Collet
2017-07-10 14:02:33 -07:00
f9524cf366
added --memtest to fuzzer
Yann Collet
2017-07-10 13:48:41 -07:00
e4155b11d7
Add warning flags to makefile and clean up code to remove warnings
Stella Lau
2017-07-10 13:08:19 -07:00
10a71d9f1c
Add compression context
Stella Lau
2017-07-10 12:38:27 -07:00
e32fb0c1fe
added ZSTD_sizeof_CCtx() test
Yann Collet
2017-07-10 12:29:57 -07:00
89190ef07d
renamed pool.c to poolTests.c
Paul Cruz
2017-07-10 11:32:30 -07:00
cc7f8e4d71
small changes
Paul Cruz
2017-07-10 11:10:11 -07:00
7e09b508ff
changed name
Paul Cruz
2017-07-10 11:05:37 -07:00
ed72ea5438
removed single from Makefile
Paul Cruz
2017-07-10 10:58:03 -07:00
ced3ec5714
removed scripts
Paul Cruz
2017-07-10 10:53:02 -07:00
82f0d64bee
removed single.c
Paul Cruz
2017-07-10 10:51:50 -07:00
62ebbabd32
updated error checking in each thread
Paul Cruz
2017-07-10 09:36:22 -07:00
ae9cf235d6
Add LDM_DCtx
Stella Lau
2017-07-10 07:38:09 -07:00
5432214ee3
Minor refactoring
Stella Lau
2017-07-10 06:50:49 -07:00
b94b468e84
Merge branch 'ldm' of https://github.com/stellamplau/zstd into ldm
Stella Lau
2017-07-10 06:32:46 -07:00
474e06ac5b
Minor refactoring
Stella Lau
2017-07-10 06:32:29 -07:00
eb280cd568
Add folder for old versions
Stella Lau
2017-07-10 06:32:05 -07:00
719ccdc5a5
Update mainfile
Stella Lau
2017-07-09 22:45:54 -07:00
40156a4967
bumped version nb to v1.3.1
Yann Collet
2017-07-08 04:55:09 -07:00
0f4fc6c20a
fixed several conversion warnings
Yann Collet
2017-07-07 17:13:12 -07:00
acdeb9f302
Add compression statistics
Stella Lau
2017-07-07 17:09:28 -07:00
c3ae23d459
added ability to compress without specifying out filename
Paul Cruz
2017-07-07 17:07:05 -07:00
ed0243a63c
removed zbufftest from list of all tests
Yann Collet
2017-07-07 16:16:14 -07:00
9bde061a0b
fixed minor Visual compilation limitation
Yann Collet
2017-07-07 16:14:17 -07:00
593d517ebf
fixed minor cast warning
Yann Collet
2017-07-07 16:09:47 -07:00
7163ffafde
playing around with adapt param
Paul Cruz
2017-07-07 15:56:00 -07:00
842644e42e
target gpptest uses CXX environment variable
Yann Collet
2017-07-07 15:55:41 -07:00
d4dcace34b
Merge branch 'dev' of github.com:facebook/zstd into dev
Yann Collet
2017-07-07 15:53:20 -07:00
ead4dd48f6
new field frameHeader.headerSize
Yann Collet
2017-07-07 15:51:24 -07:00
1c9d6b2c6b
rewrote time elapsed with UTIL
Paul Cruz
2017-07-07 15:42:20 -07:00
46396523c0
ZSTD_getFrameHeader : control of windowSize limits is delegated to caller
Yann Collet
2017-07-07 15:32:12 -07:00
990449b89d
new field : ZSTD_frameHeader.frameType
Yann Collet
2017-07-07 15:21:35 -07:00
c0c236a28b
changed to using compressCCtx
Paul Cruz
2017-07-07 15:13:40 -07:00
4076be09ec
[ldm] Update to hash every position
Stella Lau
2017-07-07 14:52:40 -07:00
e622330a3b
extended frameHeader.windowSize to unsigned long long
Yann Collet
2017-07-07 14:19:01 -07:00
7945f9ee47
Fix offset overflow bug
Stella Lau
2017-07-07 14:14:01 -07:00
11fc0f4119
changed completed -> compressed
Paul Cruz
2017-07-07 13:55:38 -07:00
09d7c6a994
changed completed variables to compressed for clarity
Paul Cruz
2017-07-07 13:18:55 -07:00
f791fc27e3
Add header with compress and decompress size
Stella Lau
2017-07-07 12:44:29 -07:00
8c0eb62920
removed unnecessary comments, uncommented DEBUGLOG for later use
Paul Cruz
2017-07-07 11:47:16 -07:00
70a4153bd3
added ability to force output to stdout, wrote an additional test for this functionality
Paul Cruz
2017-07-07 11:32:14 -07:00
532f439961
cleaned up code for arguments a bit
Paul Cruz
2017-07-07 10:58:43 -07:00
f7e6b358d0
added tests that check to ensure stdout is working
Paul Cruz
2017-07-07 10:29:06 -07:00
4679132f59
updated avg compression rate, also hiding progress bar behind a flag now
Paul Cruz
2017-07-07 10:25:38 -07:00
00bc5df4e0
added compression rate to status bar
Paul Cruz
2017-07-07 09:35:39 -07:00
f351848b76
added data amount
Paul Cruz
2017-07-06 20:40:00 -07:00
2939301023
fixed problem with progress bar not persisting, added time elapsed
Paul Cruz
2017-07-06 20:30:20 -07:00
57ec0232a8
added help menu
Paul Cruz
2017-07-06 18:09:10 -07:00
b6cc084716
added really simple progress update in the corner
Paul Cruz
2017-07-06 17:48:18 -07:00
3bbfa1249e
Update compressor and decompressor
Stella Lau
2017-07-06 16:47:08 -07:00
ff9f2cd057
added some basic logic for altering compression level
Paul Cruz
2017-07-06 16:06:53 -07:00
b96ad327a4
Add simple compress and decompress functions
Stella Lau
2017-07-06 15:23:15 -07:00
a407ccc215
added ability to congregate statistics into single print statement rather than using debug
Paul Cruz
2017-07-06 13:09:17 -07:00
49d3d4f484
Merge pull request #751 from ivuk/fix-typo
Yann Collet and GitHub
2017-07-06 12:06:43 -07:00
e6e25c9507
Fix typos in README.md
Igor Vuk
2017-07-06 20:43:14 +02:00
f57849b9c6
added ability to set initial compression level
Paul Cruz
2017-07-06 11:05:51 -07:00
592a0d9495
changed to work with std out
Paul Cruz
2017-07-06 10:49:26 -07:00
94fe291b83
small changes
Paul Cruz
2017-07-06 10:29:16 -07:00