Compare commits

...
95 Commits
Author SHA1 Message Date
Yann Collet 47222eabd3 Merge pull request #186 from Cyan4973/dev
v0.6.1
2016-05-13 14:01:34 +02:00
Yann Collet d9b3cca4d3 restore openVMS support (detected by @thatsafunnyname) 2016-05-13 12:43:36 +02:00
Yann Collet 929f80288d Merge pull request #185 from inikep/dev
Dev
2016-05-13 12:40:02 +02:00
inikep 5c39327912 Merge remote-tracking branch 'refs/remotes/Cyan4973/dev' into dev 2016-05-13 12:06:36 +02:00
Yann Collet e6aa23be91 updated NEWS for v0.6.1 2016-05-13 11:29:38 +02:00
inikep 63ecd747de added common/entropy_common.c 2016-05-13 11:27:56 +02:00
Yann Collet ea7e4f0216 Merge pull request #183 from inikep/dev
added zlibWrapper - zstd wrapper for zlib
2016-05-13 11:24:14 +02:00
inikep 58a1d45aa5 Merge remote-tracking branch 'refs/remotes/Cyan4973/dev' into dev 2016-05-13 11:05:35 +02:00
inikep 0bdb6a8118 changed definition of UTIL_createFileList 2016-05-13 10:52:02 +02:00
inikep 37aba8b1d9 updated zlibWrapper/README.md (2) 2016-05-12 17:32:28 +02:00
inikep b5e6870bb5 updated zlibWrapper/README.md 2016-05-12 17:28:57 +02:00
inikep 3eabe9b474 added zlibWrapper - zstd wrapper for zlib 2016-05-12 17:15:41 +02:00
Yann Collet ccd6e86d45 improved decoding speed by inlining x_isError() functions 2016-05-12 15:55:26 +02:00
Yann Collet 16871680e3 minor refactor : more accurate variable scope 2016-05-12 14:27:00 +02:00
Yann Collet b19d5d76ec restored inline for better decompression speed 2016-05-12 14:06:31 +02:00
Yann Collet 1ceb5a9295 restored DStream init
due to strange speed regressions and warnins on gcc 5.3
2016-05-12 13:50:13 +02:00
Yann Collet ce27e85209 fixed clang conversion warning 2016-05-11 18:55:27 +02:00
Yann Collet 1032fbe714 update FSE library 2016-05-11 18:30:24 +02:00
Yann Collet 249ae0ca99 better big-endian / little-endian r/w support 2016-05-11 16:38:18 +02:00
inikep 4dbf7f4a3b dynamic memory allocation in UTIL_createFileList 2016-05-11 14:11:00 +02:00
Yann Collet a96f9fd5bc datagencli uses mem.h 2016-05-11 11:42:51 +02:00
Yann Collet 3329d98df9 fixed minor asan warning in legacy decoder 2016-05-11 11:07:54 +02:00
Yann Collet c75e4c25e8 Added : ZSTD_getErrorString(), to get error string from error enum (#168) 2016-05-10 17:47:11 +02:00
Yann Collet b3a9938d0c Merge pull request #182 from inikep/dev
support for Visual Studio 2012, 2013, 2015
2016-05-10 17:18:12 +02:00
Yann Collet b12b369817 fixed : unused argument in printf 2016-05-10 15:30:43 +02:00
inikep 95459458ed disable Visual Studio code analysis 2016-05-10 14:42:54 +02:00
inikep 3733797fcd bench.c: experimental -r (operate recursively on directories) for Windows and _POSIX_C_SOURCE >= 200112L 2016-05-10 14:22:55 +02:00
Yann Collet bd39d54576 zbuff supports byte-by-byte decompression scenarios 2016-05-10 14:14:19 +02:00
inikep ed9a08538c Merge remote-tracking branch 'refs/remotes/Cyan4973/dev' into dev
# Conflicts:
#	lib/common/util.h
#	programs/paramgrill.c
#	visual/2013/fullbench/fullbench.vcxproj.filters
#	visual/2013/fuzzer/fuzzer.vcxproj.filters
2016-05-10 13:20:01 +02:00
Yann Collet 918b934a26 minor comment update 2016-05-10 07:00:07 +02:00
Yann Collet d693117113 fixed : null-length string roundtrip 2016-05-10 05:56:09 +02:00
Yann Collet 8b23eea694 minor refactor : more accurate variables scope
Added null-length string test
2016-05-10 05:37:43 +02:00
inikep 349fcf7ea2 fixed Linux compilation 2016-05-09 18:04:07 +02:00
inikep aaaf923a17 common/util.h moved to programs/util.h
decompress/fse_decompress.c moved to common/
2016-05-09 16:19:25 +02:00
inikep d44ec48b1c detection of snprintf for Visual <= 2012 2016-05-09 14:37:41 +02:00
inikep 03bf5ad6cb added appveyor.yml 2016-05-09 14:04:40 +02:00
inikep 6e61a842b8 added support for VS 2015 2016-05-09 14:00:54 +02:00
Yann Collet d782890576 fixed datagen 2016-05-09 12:28:20 +02:00
Yann Collet ceca200c77 datagen uses mem.h 2016-05-09 12:20:50 +02:00
Yann Collet 59b6ba7677 Merge pull request #180 from thatsafunnyname/patch-1
No stdint.h on OpenVMS.
2016-05-09 11:56:42 +02:00
Yann Collet 4ff73a2eff added : midipix support (#181) 2016-05-09 11:44:00 +02:00
Yann Collet d275236e6a Fixed Windows API, as detected by @KrzysFR (#171) 2016-05-09 08:19:27 +02:00
Yann Collet f6ca09b5ff Reduced console display on loading lots of files with zstd --train. Reported by @KrzysFR, see #177 2016-05-09 04:44:45 +02:00
Yann Collet e1a69a88c0 Expose Advanced Streaming API for v0.5.x legacy 2016-05-08 09:13:40 +02:00
Yann Collet 95af06f6ef fixed legacy dictionary decompression 2016-05-08 08:23:51 +02:00
Yann Collet 0c0f314550 CLI supports dictionary decompression in legacy mode v0.5.x 2016-05-08 08:11:24 +02:00
Yann Collet f849462d4c minor refactor : more accurate variable scope 2016-05-07 22:43:40 +02:00
Yann Collet a5dacdf551 updated fse (minor refactor) 2016-05-06 16:55:27 +02:00
Yann Collet 18dedece91 Support legacy ZSTD_decompress_usingDict() (starting v0.5+) 2016-05-06 16:43:23 +02:00
Yann Collet 8283a2f0aa Fixed decoding corruption error with quad-symbols huffman on legacy decoders 2016-05-06 01:51:31 +02:00
inikep 13c8424ea0 code cleaning 2016-05-05 13:58:56 +02:00
inikep 0bd0faec32 fixed compatibility issues 2016-05-05 13:10:57 +02:00
Yann Collet 6d1d25299a fixed Huff0 quad-symbols decoder (#173) 2016-05-05 12:41:36 +02:00
inikep 9c22e57bfb Compiler Options moved to util.h 2016-05-05 11:53:42 +02:00
Yann Collet ddb8ebd5b3 Stricter gnu90 tests 2016-05-05 04:59:53 +02:00
Yann Collet 3f8ed50bf1 Added gnu90 test
work-around : disabled automatic selection of huff0 quad-decoder (see #173)
2016-05-05 03:01:13 +02:00
inikep 3163403855 UTIL_prepareFileList 2016-05-05 00:25:38 +02:00
thatsafunnyname b0e5427d1a No stdint.h on OpenVMS.
No stdint.h on OpenVMS, it is inttypes.h instead, some projects use HAVE_STDINT_H.
2016-05-04 15:43:42 +01:00
thatsafunnyname dca5cf92ac No stdint.h on OpenVMS.
No stdint.h on OpenVMS, it is inttypes.h instead, some projects use HAVE_STDINT_H.
2016-05-04 15:42:25 +01:00
thatsafunnyname eef5fdb58a No stdint.h on OpenVMS.
No stdint.h on OpenVMS, it is inttypes.h instead, some projects use HAVE_STDINT_H.
2016-05-04 15:39:31 +01:00
Yann Collet f6ac47955b removed BMI tests on Travis CI : incompatible compiler 2016-05-01 10:34:04 +02:00
Yann Collet 17d35b0bf4 added : BMI x64 test for Travis CI 2016-05-01 10:27:54 +02:00
Yann Collet 6f9c056662 Fixed BMI behavior on x86 and x32 targets, reported by @tycho (#178) 2016-05-01 10:26:30 +02:00
Yann Collet cf8584c90a Merge pull request #175 from inikep/dev
separation into compressor and decompressor
2016-04-29 23:07:45 -07:00
inikep 648d74a604 Sync 2016-04-29 15:25:00 +02:00
inikep bab4317961 util.h must the the first include to #define _POSIX_C_SOURCE 2016-04-29 15:19:40 +02:00
inikep 4f832c9f26 util.h must be the first include to #define _POSIX_C_SOURCE 2016-04-29 15:18:50 +02:00
inikep 55d047aa92 getTotalFileSize moved to common/util.h 2016-04-28 16:50:13 +02:00
inikep d5ff2c3d9a ordering of #include 2016-04-28 14:40:45 +02:00
inikep 83c76b4594 bench.c: time functions moved to common/util.h 2016-04-28 13:16:01 +02:00
inikep 69fcd7c0ae getFileSize moved to common/util.h 2016-04-28 12:23:33 +02:00
inikep 1007a1fe3d get rid of some Visual C++ warnings 2016-04-25 15:23:09 +02:00
inikep 6cb083fe3d fixed 64-bit compilation with Visual C++ 2016-04-25 14:42:15 +02:00
inikep ea4ee3eee0 added compatibility with Visual C++ 2012 2016-04-25 13:09:06 +02:00
inikep a8138fd767 updated lib/README.md part 2 2016-04-25 11:36:44 +02:00
inikep 49794316c2 updated lib/README.md 2016-04-25 11:31:28 +02:00
inikep f066816998 fix for g++ compilation 2016-04-22 18:54:05 +02:00
inikep db3964382c introduced ZSTD_NODECOMPRESS to link only compressor 2016-04-22 18:22:30 +02:00
inikep a1febea01d Error functions moved to common/zstd_common.c 2016-04-22 17:14:25 +02:00
inikep d6be2751a8 updated CMakeLists.txt 2016-04-22 13:59:21 +02:00
inikep 3c7c3527d0 introduced ZSTD_NOCOMPRESS to generate decompressor only 2016-04-22 13:59:05 +02:00
inikep 23a0889301 separation of lib/ into common/, compress/, decompress/, dictBuilder/, legacy/ 2016-04-22 12:43:18 +02:00
inikep 4699b3be03 Merge remote-tracking branch 'refs/remotes/Cyan4973/dev' into dev 2016-04-21 12:30:58 +02:00
inikep 38654988f3 minor speed improvements 2
bench.c: block size has to be bigger than 32 bytes
zstdcli.c: support for e.g. -B16k -B16m
2016-04-21 12:18:47 +02:00
inikep ef51941822 zst_opt.h: minor compression speed improvement 2016-04-21 11:08:43 +02:00
Yann Collet e02808ffc5 bumped lib version to 0.6.1
added ZSTD_VERSION_STRING
2016-04-20 22:46:16 +02:00
inikep c5e1d295ee bench.c: force at least one compression and decompression loop
fix for -i0 with small files
2016-04-19 09:37:59 +02:00
inikep 6d157f1fbe bench.c: fixed rare compression and decompression speed bug
concerns only big files with compression or decompression time longer
than 100 seconds
2016-04-15 16:54:11 +02:00
inikep 2c5eeea635 introduced ZSTD_DEAFULT_CLEVEL for (compressionLevel<=0) 2016-04-15 13:44:46 +02:00
inikep 4b3c5eed99 removed ZSTD_compressBegin_targetSrcSize 2016-04-14 13:43:51 +02:00
inikep c70c90e8a3 Merge remote-tracking branch 'refs/remotes/Cyan4973/dev' into dev 2016-04-13 10:48:26 +02:00
inikep c0d5f4eb2e bench.c: ignore directories from a file list for benchmark 2016-04-13 10:48:04 +02:00
inikep 8844e21dff Merge remote-tracking branch 'refs/remotes/Cyan4973/dev' into dev 2016-04-11 16:49:27 +02:00
inikep f7d210b2e9 cache literal prices for ZSTD_btopt 2016-04-11 16:35:13 +02:00
inikep 1eeddde625 clock() is default timer for all platforms except Windows 2016-04-08 16:55:17 +02:00
81 changed files with 6626 additions and 4642 deletions
+5 -4
View File
@@ -8,8 +8,9 @@
# Denote files that should not be modified.
*.odt binary
*.png binary
# Visual Studio
*.sln binary
*.suo binary
*.vcxproj* binary
# Visual Studio
*.sln text eol=crlf
*.vcxproj* text eol=crlf
*.suo binary
*.rc binary
+8 -24
View File
@@ -17,33 +17,17 @@
*.out
*.app
# Visual solution files
*.suo
*.user
*.VC.db
# Build results
[Dd]ebug/
[Rr]elease/
[Rr]eleases/
x64/
x86/
[Bb]in/
[Oo]bj/
# Visual C++ cache files
ipch/
*.aps
*.ncb
*.opendb
*.opensdf
*.sdf
*.cachefile
# Visual C++
visual/
# IDEA solution files
*.idea
# Other files
.directory
_codelite
_zstdbench
_codelite/
_zstdbench/
zlib_wrapper/
# CMake
contrib/cmake/
+1
View File
@@ -12,6 +12,7 @@ env:
- ZSTD_TRAVIS_CI_ENV=cmaketest
- ZSTD_TRAVIS_CI_ENV=clangtest
- ZSTD_TRAVIS_CI_ENV=gpptest
- ZSTD_TRAVIS_CI_ENV=gnu90test
- ZSTD_TRAVIS_CI_ENV=armtest-w-install
- ZSTD_TRAVIS_CI_ENV=test
- ZSTD_TRAVIS_CI_ENV="-C programs test32"
+19 -4
View File
@@ -31,7 +31,7 @@
# ################################################################
# force a version number : uncomment below export (otherwise, default to the one declared into zstd.h)
#export VERSION := 0.5.1
#export VERSION := 0.6.1
PRGDIR = programs
ZSTDDIR = lib
@@ -54,6 +54,9 @@ all:
zstdprogram:
$(MAKE) -C $(PRGDIR)
test:
$(MAKE) -C $(PRGDIR) $@
clean:
@$(MAKE) -C $(ZSTDDIR) $@ > $(VOID)
@$(MAKE) -C $(PRGDIR) $@ > $(VOID)
@@ -75,9 +78,6 @@ uninstall:
travis-install:
$(MAKE) install PREFIX=~/install_test_dir
test:
$(MAKE) -C $(PRGDIR) $@
cmaketest:
cd contrib/cmake ; cmake . ; $(MAKE)
@@ -88,6 +88,21 @@ clangtest: clean
gpptest: clean
$(MAKE) all CC=g++ CFLAGS="-O3 -Wall -Wextra -Wundef -Wshadow -Wcast-align -Werror"
gnu90test: clean
$(MAKE) all CFLAGS="-std=gnu90 -Wall -Wextra -Wcast-qual -Wcast-align -Wshadow -Wstrict-aliasing=1 -Wswitch-enum -Wstrict-prototypes -Wundef -Wdeclaration-after-statement -Werror"
c90test: clean
$(MAKE) all CFLAGS="-std=c90 -Wall -Wextra -Wcast-qual -Wcast-align -Wshadow -Wstrict-aliasing=1 -Wswitch-enum -Wstrict-prototypes -Wundef -Werror" # will fail, due to // and long long
bmix64test: clean
CFLAGS="-O3 -mbmi -Werror" $(MAKE) -C $(PRGDIR) test
bmix32test: clean
CFLAGS="-O3 -mbmi -mx32 -Werror" $(MAKE) -C $(PRGDIR) test
bmi32test: clean
CFLAGS="-O3 -mbmi -m32 -Werror" $(MAKE) -C $(PRGDIR) test
armtest: clean
$(MAKE) -C $(PRGDIR) datagen # use native, faster
$(MAKE) -C $(PRGDIR) test CC=arm-linux-gnueabi-gcc ZSTDRTTEST= MOREFLAGS="-Werror -static"
+11 -1
View File
@@ -1,3 +1,13 @@
v0.6.1
New : zlib wrapper API, thanks to Przemyslaw Skibinski
New : Ability to compile compressor / decompressor separately
Changed : new lib directory structure
Fixed : Legacy codec v0.5 compatible with dictionary decompression
Fixed : Decoder corruption error (#173)
Fixed : null-string roundtrip (#176)
New : benchmark mode can select directory as input
Experimental : midipix support, VMS support
v0.6.0
Stronger high compression modes, thanks to Przemyslaw Skibinski
API : ZSTD_getFrameParams() provides size of decompressed content
@@ -95,7 +105,7 @@ frame concatenation support
v0.1.1
fix compression bug
detects write-flush errors
git@github.com:Cyan4973/zstd.git
v0.1.0
first release
+25
View File
@@ -0,0 +1,25 @@
version: 1.0.{build}
configuration:
- Release
- Debug
platform:
- Win32
- x64
environment:
matrix:
# - PlatformToolset: v100
- PlatformToolset: v110
- PlatformToolset: v120
- PlatformToolset: v140
build_script:
- cmd: >-
ECHO PlatformToolset=%PlatformToolset%
msbuild "visual\2013\zstd.sln" /m /verbosity:minimal /property:PlatformToolset=%PlatformToolset% /t:Clean,Build /logger:"C:\Program Files\AppVeyor\BuildAgent\Appveyor.MSBuildLogger.dll"
test_script:
- cmd: >-
ECHO %APPVEYOR_BUILD_FOLDER%\visual\2013\bin\%PLATFORM%\%CONFIGURATION%\fuzzer.exe %FUZZERTEST%
SET FUZZERTEST=-T2mn
IF %CONFIGURATION%==Release (%APPVEYOR_BUILD_FOLDER%\visual\2013\bin\%PLATFORM%\%CONFIGURATION%\fuzzer.exe %FUZZERTEST%)
+31 -26
View File
@@ -47,40 +47,45 @@ SET(ROOT_DIR ../../..)
# Define library directory, where sources and header files are located
SET(LIBRARY_DIR ${ROOT_DIR}/lib)
INCLUDE_DIRECTORIES(${LIBRARY_DIR})
INCLUDE_DIRECTORIES(${LIBRARY_DIR}/common)
# Read file content
FILE(READ ${LIBRARY_DIR}/zstd.h HEADER_CONTENT)
FILE(READ ${LIBRARY_DIR}/common/zstd.h HEADER_CONTENT)
# Parse version
GetLibraryVersion("${HEADER_CONTENT}" LIBVER_MAJOR LIBVER_MINOR LIBVER_RELEASE)
MESSAGE("ZSTD VERSION ${LIBVER_MAJOR}.${LIBVER_MINOR}.${LIBVER_RELEASE}")
SET(Sources
${LIBRARY_DIR}/divsufsort.c
${LIBRARY_DIR}/fse.c
${LIBRARY_DIR}/huff0.c
${LIBRARY_DIR}/zbuff.c
${LIBRARY_DIR}/zdict.c
${LIBRARY_DIR}/zstd_compress.c
${LIBRARY_DIR}/zstd_decompress.c)
${LIBRARY_DIR}/common/entropy_common.c
${LIBRARY_DIR}/common/zstd_common.c
${LIBRARY_DIR}/common/fse_decompress.c
${LIBRARY_DIR}/compress/fse_compress.c
${LIBRARY_DIR}/compress/huf_compress.c
${LIBRARY_DIR}/compress/zbuff_compress.c
${LIBRARY_DIR}/compress/zstd_compress.c
${LIBRARY_DIR}/decompress/huf_decompress.c
${LIBRARY_DIR}/decompress/zbuff_decompress.c
${LIBRARY_DIR}/decompress/zstd_decompress.c
${LIBRARY_DIR}/dictBuilder/divsufsort.c
${LIBRARY_DIR}/dictBuilder/zdict.c)
SET(Headers
${LIBRARY_DIR}/bitstream.h
${LIBRARY_DIR}/error_private.h
${LIBRARY_DIR}/error_public.h
${LIBRARY_DIR}/fse.h
${LIBRARY_DIR}/fse_static.h
${LIBRARY_DIR}/huff0.h
${LIBRARY_DIR}/huff0_static.h
${LIBRARY_DIR}/mem.h
${LIBRARY_DIR}/zbuff.h
${LIBRARY_DIR}/zbuff_static.h
${LIBRARY_DIR}/zdict.h
${LIBRARY_DIR}/zdict_static.h
${LIBRARY_DIR}/zstd_internal.h
${LIBRARY_DIR}/zstd_static.h
${LIBRARY_DIR}/zstd.h)
${LIBRARY_DIR}/common/bitstream.h
${LIBRARY_DIR}/common/error_private.h
${LIBRARY_DIR}/common/error_public.h
${LIBRARY_DIR}/common/fse.h
${LIBRARY_DIR}/common/fse_static.h
${LIBRARY_DIR}/common/huf.h
${LIBRARY_DIR}/common/huf_static.h
${LIBRARY_DIR}/common/mem.h
${LIBRARY_DIR}/common/zbuff.h
${LIBRARY_DIR}/common/zbuff_static.h
${LIBRARY_DIR}/common/zstd_internal.h
${LIBRARY_DIR}/common/zstd_static.h
${LIBRARY_DIR}/common/zstd.h
${LIBRARY_DIR}/dictBuilder/zdict.h
${LIBRARY_DIR}/dictBuilder/zdict_static.h)
IF (ZSTD_LEGACY_SUPPORT)
SET(LIBRARY_LEGACY_DIR ${LIBRARY_DIR}/legacy)
@@ -117,8 +122,8 @@ ENDIF (MSVC)
# Define include directories
IF (NOT WORKAROUND_OUTDATED_CODE_STYLE)
TARGET_INCLUDE_DIRECTORIES(libzstd_static PUBLIC ${LIBRARY_DIR})
TARGET_INCLUDE_DIRECTORIES(libzstd_shared PUBLIC ${LIBRARY_DIR})
TARGET_INCLUDE_DIRECTORIES(libzstd_static PUBLIC ${LIBRARY_DIR}/common)
TARGET_INCLUDE_DIRECTORIES(libzstd_shared PUBLIC ${LIBRARY_DIR}/common)
IF (ZSTD_LEGACY_SUPPORT)
TARGET_INCLUDE_DIRECTORIES(libzstd_static PUBLIC ${LIBRARY_LEGACY_DIR})
TARGET_INCLUDE_DIRECTORIES(libzstd_shared PUBLIC ${LIBRARY_LEGACY_DIR})
+1 -1
View File
@@ -44,7 +44,7 @@ INCLUDE_DIRECTORIES(${PROGRAMS_DIR})
IF (WORKAROUND_OUTDATED_CODE_STYLE)
# Define library directory, where sources and header files are located
SET(LIBRARY_DIR ${ROOT_DIR}/lib)
INCLUDE_DIRECTORIES(${LIBRARY_DIR})
INCLUDE_DIRECTORIES(${LIBRARY_DIR}/common)
ENDIF (WORKAROUND_OUTDATED_CODE_STYLE)
IF (ZSTD_LEGACY_SUPPORT)
+11 -8
View File
@@ -31,9 +31,9 @@
# ################################################################
# Version numbers
LIBVER_MAJOR_SCRIPT:=`sed -n '/define ZSTD_VERSION_MAJOR/s/.*[[:blank:]]\([0-9][0-9]*\).*/\1/p' < zstd.h`
LIBVER_MINOR_SCRIPT:=`sed -n '/define ZSTD_VERSION_MINOR/s/.*[[:blank:]]\([0-9][0-9]*\).*/\1/p' < zstd.h`
LIBVER_PATCH_SCRIPT:=`sed -n '/define ZSTD_VERSION_RELEASE/s/.*[[:blank:]]\([0-9][0-9]*\).*/\1/p' < zstd.h`
LIBVER_MAJOR_SCRIPT:=`sed -n '/define ZSTD_VERSION_MAJOR/s/.*[[:blank:]]\([0-9][0-9]*\).*/\1/p' < ./common/zstd.h`
LIBVER_MINOR_SCRIPT:=`sed -n '/define ZSTD_VERSION_MINOR/s/.*[[:blank:]]\([0-9][0-9]*\).*/\1/p' < ./common/zstd.h`
LIBVER_PATCH_SCRIPT:=`sed -n '/define ZSTD_VERSION_RELEASE/s/.*[[:blank:]]\([0-9][0-9]*\).*/\1/p' < ./common/zstd.h`
LIBVER_SCRIPT:= $(LIBVER_MAJOR_SCRIPT).$(LIBVER_MINOR_SCRIPT).$(LIBVER_PATCH_SCRIPT)
LIBVER_MAJOR := $(shell echo $(LIBVER_MAJOR_SCRIPT))
LIBVER_MINOR := $(shell echo $(LIBVER_MINOR_SCRIPT))
@@ -43,7 +43,7 @@ VERSION?= $(LIBVER)
DESTDIR?=
PREFIX ?= /usr/local
CPPFLAGS= -I.
CPPFLAGS= -I./common
CFLAGS ?= -O3
CFLAGS += -std=c99 -Wall -Wextra -Wundef -Wshadow -Wcast-qual -Wcast-align -Wstrict-prototypes -Wstrict-aliasing=1
FLAGS = $(CPPFLAGS) $(CFLAGS) $(LDFLAGS) $(MOREFLAGS)
@@ -51,7 +51,10 @@ FLAGS = $(CPPFLAGS) $(CFLAGS) $(LDFLAGS) $(MOREFLAGS)
LIBDIR ?= $(PREFIX)/lib
INCLUDEDIR=$(PREFIX)/include
ZSTD_FILES := zstd_compress.c zstd_decompress.c fse.c huff0.c zbuff.c zdict.c divsufsort.c
ZSTDCOMP_FILES := compress/zstd_compress.c compress/fse_compress.c compress/huf_compress.c compress/zbuff_compress.c
ZSTDDECOMP_FILES := decompress/zstd_decompress.c common/fse_decompress.c decompress/huf_decompress.c decompress/zbuff_decompress.c
ZSTDDICT_FILES := dictBuilder/zdict.c dictBuilder/divsufsort.c
ZSTD_FILES := $(ZSTDDECOMP_FILES) common/entropy_common.c common/zstd_common.c $(ZSTDCOMP_FILES) $(ZSTDDICT_FILES)
ZSTD_LEGACY:= legacy/zstd_v01.c legacy/zstd_v02.c legacy/zstd_v03.c legacy/zstd_v04.c legacy/zstd_v05.c
ifeq ($(ZSTD_LEGACY_SUPPORT), 0)
@@ -117,9 +120,9 @@ install: libzstd libzstd.pc
@cp -a libzstd.$(SHARED_EXT) $(DESTDIR)$(LIBDIR)
@cp -a libzstd.pc $(DESTDIR)$(LIBDIR)/pkgconfig/
@install -m 644 libzstd.a $(DESTDIR)$(LIBDIR)/libzstd.a
@install -m 644 zstd.h $(DESTDIR)$(INCLUDEDIR)/zstd.h
@install -m 644 zbuff.h $(DESTDIR)$(INCLUDEDIR)/zbuff.h
@install -m 644 zdict.h $(DESTDIR)$(INCLUDEDIR)/zdict.h
@install -m 644 common/zstd.h $(DESTDIR)$(INCLUDEDIR)/zstd.h
@install -m 644 common/zbuff.h $(DESTDIR)$(INCLUDEDIR)/zbuff.h
@install -m 644 dictBuilder/zdict.h $(DESTDIR)$(INCLUDEDIR)/zdict.h
@echo zstd static and shared library installed
uninstall:
+39 -31
View File
@@ -5,53 +5,61 @@ The __lib__ directory contains several files, but depending on target use case,
#### Minimal library files
##### Shared ressources
To build the zstd library the following files are required:
- [mem.h](mem.h)
- [error_private.h](error_private.h)
- [error_public.h](error_public.h)
- [common/bitstream.h](common/bitstream.h)
- [common/error_private.h](common/error_private.h)
- [common/error_public.h](common/error_public.h)
- common/fse.h
- common/fse_decompress.c
- common/fse_static.h
- common/huf.h
- common/huf_static.h
- [common/mem.h](common/mem.h)
- [common/zstd.h]
- common/zstd_internal.h
- common/zstd_static.h
- compress/fse_compress.c
- compress/huf_compress.c
- compress/zstd_compress.c
- compress/zstd_opt.h
- decompress/huf_decompress.c
- decompress/zstd_decompress.c
##### zstd core compression
Stable API is exposed in [zstd.h].
Advanced and experimental API is exposed in `zstd_static.h`.
`zstd_static.h` API elements should be used with static linking only,
Stable API is exposed in [common/zstd.h].
Advanced and experimental API is exposed in [common/zstd_static.h].
API elements of [common/zstd_static.h] should be used with static linking only,
as their definition may change in future version of the library.
- [bitstream.h](bitstream.h)
- fse.c
- fse.h
- fse_static.h
- huff0.c
- huff0.h
- huff0_static.h
- zstd_compress.c
- zstd_decompress.c
- zstd_internal.h
- zstd_opt.h
- [zstd.h]
- zstd_static.h
[common/zstd.h]: common/zstd.h
[common/zstd_static.h]: common/zstd_static.h
#### Separate compressor and decompressor
To build a separate zstd compressor all files from `common/` and `compressor/` directories are required.
In a similar way to build a separate zstd decompressor all files from `common/` and `decompressor/` directories are needed.
[zstd.h]: zstd.h
#### Buffered streaming
This complementary API makes streaming integration easier.
It is used by `zstd` command line utility, and [7zip plugin](http://mcmilk.de/projects/7-Zip-ZStd) :
- zbuff.c
- zbuff.h
- zbuff_static.h
- common/zbuff.h
- common/zbuff_static.h
- compress/zbuff_compress.c
- decompress/zbuff_decompress.c
#### Dictionary builder
To create dictionaries from training sets :
- divsufsort.c
- divsufsort.h
- zdict.c
- zdict.h
- zdict_static.h
- dictBuilder/divsufsort.c
- dictBuilder/divsufsort.h
- dictBuilder/zdict.c
- dictBuilder/zdict.h
- dictBuilder/zdict_static.h
#### Miscellaneous
+27 -22
View File
@@ -84,7 +84,7 @@ MEM_STATIC size_t BIT_closeCStream(BIT_CStream_t* bitC);
/* Start with initCStream, providing the size of buffer to write into.
* bitStream will never write outside of this buffer.
* `dstCapacity` must be >= sizeof(size_t), otherwise @return will be an error code.
* `dstCapacity` must be >= sizeof(bitD->bitContainer), otherwise @return will be an error code.
*
* bits are first added to a local register.
* Local register is size_t, hence 64-bits on 64-bits systems, or 32-bits on 32-bits systems.
@@ -128,7 +128,7 @@ MEM_STATIC unsigned BIT_endOfDStream(const BIT_DStream_t* bitD);
* Local register size is 64-bits on 64-bits systems, 32-bits on 32-bits systems (size_t).
* You can then retrieve bitFields stored into the local register, **in reverse order**.
* Local register is explicitly reloaded from memory by the BIT_reloadDStream() method.
* A reload guarantee a minimum of ((8*sizeof(size_t))-7) bits when its result is BIT_DStream_unfinished.
* A reload guarantee a minimum of ((8*sizeof(bitD->bitContainer))-7) bits when its result is BIT_DStream_unfinished.
* Otherwise, it can be less than that, so proceed accordingly.
* Checking if DStream has reached its end can be performed with BIT_endOfDStream().
*/
@@ -263,9 +263,9 @@ MEM_STATIC size_t BIT_initDStream(BIT_DStream_t* bitD, const void* srcBuffer, si
{
if (srcSize < 1) { memset(bitD, 0, sizeof(*bitD)); return ERROR(srcSize_wrong); }
if (srcSize >= sizeof(size_t)) { /* normal case */
if (srcSize >= sizeof(bitD->bitContainer)) { /* normal case */
bitD->start = (const char*)srcBuffer;
bitD->ptr = (const char*)srcBuffer + srcSize - sizeof(size_t);
bitD->ptr = (const char*)srcBuffer + srcSize - sizeof(bitD->bitContainer);
bitD->bitContainer = MEM_readLEST(bitD->ptr);
{ BYTE const lastByte = ((const BYTE*)srcBuffer)[srcSize-1];
if (lastByte == 0) return ERROR(GENERIC); /* endMark not present */
@@ -276,53 +276,58 @@ MEM_STATIC size_t BIT_initDStream(BIT_DStream_t* bitD, const void* srcBuffer, si
bitD->bitContainer = *(const BYTE*)(bitD->start);
switch(srcSize)
{
case 7: bitD->bitContainer += (size_t)(((const BYTE*)(bitD->start))[6]) << (sizeof(size_t)*8 - 16);
case 6: bitD->bitContainer += (size_t)(((const BYTE*)(bitD->start))[5]) << (sizeof(size_t)*8 - 24);
case 5: bitD->bitContainer += (size_t)(((const BYTE*)(bitD->start))[4]) << (sizeof(size_t)*8 - 32);
case 4: bitD->bitContainer += (size_t)(((const BYTE*)(bitD->start))[3]) << 24;
case 3: bitD->bitContainer += (size_t)(((const BYTE*)(bitD->start))[2]) << 16;
case 2: bitD->bitContainer += (size_t)(((const BYTE*)(bitD->start))[1]) << 8;
case 7: bitD->bitContainer += (size_t)(((const BYTE*)(srcBuffer))[6]) << (sizeof(bitD->bitContainer)*8 - 16);
case 6: bitD->bitContainer += (size_t)(((const BYTE*)(srcBuffer))[5]) << (sizeof(bitD->bitContainer)*8 - 24);
case 5: bitD->bitContainer += (size_t)(((const BYTE*)(srcBuffer))[4]) << (sizeof(bitD->bitContainer)*8 - 32);
case 4: bitD->bitContainer += (size_t)(((const BYTE*)(srcBuffer))[3]) << 24;
case 3: bitD->bitContainer += (size_t)(((const BYTE*)(srcBuffer))[2]) << 16;
case 2: bitD->bitContainer += (size_t)(((const BYTE*)(srcBuffer))[1]) << 8;
default:;
}
{ BYTE const lastByte = ((const BYTE*)srcBuffer)[srcSize-1];
if (lastByte == 0) return ERROR(GENERIC); /* endMark not present */
bitD->bitsConsumed = 8 - BIT_highbit32(lastByte); }
bitD->bitsConsumed += (U32)(sizeof(size_t) - srcSize)*8;
bitD->bitsConsumed += (U32)(sizeof(bitD->bitContainer) - srcSize)*8;
}
return srcSize;
}
MEM_STATIC size_t BIT_getUpperBits(size_t bitD, U32 const start)
MEM_STATIC size_t BIT_getUpperBits(size_t bitContainer, U32 const start)
{
return bitD >> start;
return bitContainer >> start;
}
MEM_STATIC size_t BIT_getMiddleBits(size_t bitD, U32 const nbBits, U32 const start)
MEM_STATIC size_t BIT_getMiddleBits(size_t bitContainer, U32 const start, U32 const nbBits)
{
#if defined(__BMI__) && defined(__GNUC__) /* experimental */
return __builtin_ia32_bextr_u64(bitD, (nbBits<<8) | start );
# if defined(__x86_64__)
if (sizeof(bitContainer)==8)
return _bextr_u64(bitContainer, start, nbBits);
else
# endif
return _bextr_u32(bitContainer, start, nbBits);
#else
return (bitD >> start) & BIT_mask[nbBits];
return (bitContainer >> start) & BIT_mask[nbBits];
#endif
}
MEM_STATIC size_t BIT_getLowerBits(size_t bitD, U32 const nbBits)
MEM_STATIC size_t BIT_getLowerBits(size_t bitContainer, U32 const nbBits)
{
return bitD & BIT_mask[nbBits];
return bitContainer & BIT_mask[nbBits];
}
/*! BIT_lookBits() :
* Provides next n bits from local register.
* local register is not modified (bits are still present for next read/look).
* local register is not modified.
* On 32-bits, maxNbBits==24.
* On 64-bits, maxNbBits==56.
* @return : value extracted
*/
MEM_STATIC size_t BIT_lookBits(const BIT_DStream_t* bitD, U32 nbBits)
{
#if defined(__BMI__) && defined(__GNUC__) /* experimental */
return __builtin_ia32_bextr_u64(bitD->bitContainer, (nbBits<<8) | (64 - bitD->bitsConsumed - nbBits) );
#if defined(__BMI__) && defined(__GNUC__) /* experimental; fails if bitD->bitsConsumed + nbBits > sizeof(bitD->bitContainer)*8 */
return BIT_getMiddleBits(bitD->bitContainer, (sizeof(bitD->bitContainer)*8) - bitD->bitsConsumed - nbBits, nbBits);
#else
U32 const bitMask = sizeof(bitD->bitContainer)*8 - 1;
return ((bitD->bitContainer << (bitD->bitsConsumed & bitMask)) >> 1) >> ((bitMask-nbBits) & bitMask);
@@ -367,7 +372,7 @@ MEM_STATIC size_t BIT_readBitsFast(BIT_DStream_t* bitD, U32 nbBits)
* Refill `BIT_DStream_t` from src buffer previously defined (see BIT_initDStream() ).
* This function is safe, it guarantees it will not read beyond src buffer.
* @return : status of `BIT_DStream_t` internal register.
if status == unfinished, internal register is filled with >= (sizeof(size_t)*8 - 7) bits */
if status == unfinished, internal register is filled with >= (sizeof(bitD->bitContainer)*8 - 7) bits */
MEM_STATIC BIT_DStream_status BIT_reloadDStream(BIT_DStream_t* bitD)
{
if (bitD->bitsConsumed > (sizeof(bitD->bitContainer)*8)) /* should never happen */
+158
View File
@@ -0,0 +1,158 @@
/*
Common functions of New Generation Entropy library
Copyright (C) 2016, Yann Collet.
BSD 2-Clause License (http://www.opensource.org/licenses/bsd-license.php)
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the following disclaimer
in the documentation and/or other materials provided with the
distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
You can contact the author at :
- FSE+HUF source repository : https://github.com/Cyan4973/FiniteStateEntropy
- Public forum : https://groups.google.com/forum/#!forum/lz4c
*************************************************************************** */
/* *************************************
* Dependencies
***************************************/
#include <stdlib.h>
#include "mem.h"
#include "fse_static.h" /* FSE_MIN_TABLELOG */
#include "error_private.h"
#include "fse.h" /* declaration of FSE_isError, FSE_getErrorName */
#include "huf.h" /* declaration of HUF_isError, HUF_getErrorName */
/*-****************************************
* FSE Error Management
******************************************/
unsigned FSE_isError(size_t code) { return ERR_isError(code); }
const char* FSE_getErrorName(size_t code) { return ERR_getErrorName(code); }
/* **************************************************************
* HUF Error Management
****************************************************************/
unsigned HUF_isError(size_t code) { return ERR_isError(code); }
const char* HUF_getErrorName(size_t code) { return ERR_getErrorName(code); }
/*-**************************************************************
* FSE NCount encoding-decoding
****************************************************************/
static short FSE_abs(short a) { return a<0 ? -a : a; }
size_t FSE_readNCount (short* normalizedCounter, unsigned* maxSVPtr, unsigned* tableLogPtr,
const void* headerBuffer, size_t hbSize)
{
const BYTE* const istart = (const BYTE*) headerBuffer;
const BYTE* const iend = istart + hbSize;
const BYTE* ip = istart;
int nbBits;
int remaining;
int threshold;
U32 bitStream;
int bitCount;
unsigned charnum = 0;
int previous0 = 0;
if (hbSize < 4) return ERROR(srcSize_wrong);
bitStream = MEM_readLE32(ip);
nbBits = (bitStream & 0xF) + FSE_MIN_TABLELOG; /* extract tableLog */
if (nbBits > FSE_TABLELOG_ABSOLUTE_MAX) return ERROR(tableLog_tooLarge);
bitStream >>= 4;
bitCount = 4;
*tableLogPtr = nbBits;
remaining = (1<<nbBits)+1;
threshold = 1<<nbBits;
nbBits++;
while ((remaining>1) && (charnum<=*maxSVPtr)) {
if (previous0) {
unsigned n0 = charnum;
while ((bitStream & 0xFFFF) == 0xFFFF) {
n0+=24;
if (ip < iend-5) {
ip+=2;
bitStream = MEM_readLE32(ip) >> bitCount;
} else {
bitStream >>= 16;
bitCount+=16;
} }
while ((bitStream & 3) == 3) {
n0+=3;
bitStream>>=2;
bitCount+=2;
}
n0 += bitStream & 3;
bitCount += 2;
if (n0 > *maxSVPtr) return ERROR(maxSymbolValue_tooSmall);
while (charnum < n0) normalizedCounter[charnum++] = 0;
if ((ip <= iend-7) || (ip + (bitCount>>3) <= iend-4)) {
ip += bitCount>>3;
bitCount &= 7;
bitStream = MEM_readLE32(ip) >> bitCount;
}
else
bitStream >>= 2;
}
{ short const max = (short)((2*threshold-1)-remaining);
short count;
if ((bitStream & (threshold-1)) < (U32)max) {
count = (short)(bitStream & (threshold-1));
bitCount += nbBits-1;
} else {
count = (short)(bitStream & (2*threshold-1));
if (count >= threshold) count -= max;
bitCount += nbBits;
}
count--; /* extra accuracy */
remaining -= FSE_abs(count);
normalizedCounter[charnum++] = count;
previous0 = !count;
while (remaining < threshold) {
nbBits--;
threshold >>= 1;
}
if ((ip <= iend-7) || (ip + (bitCount>>3) <= iend-4)) {
ip += bitCount>>3;
bitCount &= 7;
} else {
bitCount -= (int)(8 * (iend - 4 - ip));
ip = iend - 4;
}
bitStream = MEM_readLE32(ip) >> (bitCount & 31);
} } /* while ((remaining>1) && (charnum<=*maxSVPtr)) */
if (remaining != 1) return ERROR(GENERIC);
*maxSVPtr = charnum-1;
ip += (bitCount+7)>>3;
if ((size_t)(ip-istart) > hbSize) return ERROR(srcSize_wrong);
return ip-istart;
}
@@ -78,17 +78,17 @@ typedef ZSTD_ErrorCode ERR_enum;
ERR_STATIC unsigned ERR_isError(size_t code) { return (code > ERROR(maxCode)); }
ERR_STATIC ERR_enum ERR_getError(size_t code) { if (!ERR_isError(code)) return (ERR_enum)0; return (ERR_enum) (0-code); }
ERR_STATIC ERR_enum ERR_getErrorCode(size_t code) { if (!ERR_isError(code)) return (ERR_enum)0; return (ERR_enum) (0-code); }
/*-****************************************
* Error Strings
******************************************/
ERR_STATIC const char* ERR_getErrorName(size_t code)
ERR_STATIC const char* ERR_getErrorString(ERR_enum code)
{
static const char* notErrorCode = "Unspecified error code";
switch( ERR_getError(code) )
switch( code )
{
case PREFIX(no_error): return "No error detected";
case PREFIX(GENERIC): return "Error (generic)";
@@ -107,10 +107,14 @@ ERR_STATIC const char* ERR_getErrorName(size_t code)
case PREFIX(maxSymbolValue_tooSmall): return "Specified maxSymbolValue is too small";
case PREFIX(dictionary_corrupted): return "Dictionary is corrupted";
case PREFIX(maxCode):
default: return notErrorCode; /* impossible, due to ERR_getError() */
default: return notErrorCode;
}
}
ERR_STATIC const char* ERR_getErrorName(size_t code)
{
return ERR_getErrorString(ERR_getErrorCode(code));
}
#if defined (__cplusplus)
}
View File
+330
View File
@@ -0,0 +1,330 @@
/* ******************************************************************
FSE : Finite State Entropy decoder
Copyright (C) 2013-2015, Yann Collet.
BSD 2-Clause License (http://www.opensource.org/licenses/bsd-license.php)
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the following disclaimer
in the documentation and/or other materials provided with the
distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
You can contact the author at :
- FSE source repository : https://github.com/Cyan4973/FiniteStateEntropy
- Public forum : https://groups.google.com/forum/#!forum/lz4c
****************************************************************** */
/* **************************************************************
* Compiler specifics
****************************************************************/
#ifdef _MSC_VER /* Visual Studio */
# define FORCE_INLINE static __forceinline
# include <intrin.h> /* For Visual 2005 */
# pragma warning(disable : 4127) /* disable: C4127: conditional expression is constant */
# pragma warning(disable : 4214) /* disable: C4214: non-int bitfields */
#else
# ifdef __GNUC__
# define GCC_VERSION (__GNUC__ * 100 + __GNUC_MINOR__)
# define FORCE_INLINE static inline __attribute__((always_inline))
# else
# define FORCE_INLINE static inline
# endif
#endif
/* **************************************************************
* Includes
****************************************************************/
#include <stdlib.h> /* malloc, free, qsort */
#include <string.h> /* memcpy, memset */
#include <stdio.h> /* printf (debug) */
#include "bitstream.h"
#include "fse_static.h"
/* **************************************************************
* Error Management
****************************************************************/
#define FSE_isError ERR_isError
#define FSE_STATIC_ASSERT(c) { enum { FSE_static_assert = 1/(int)(!!(c)) }; } /* use only *after* variable declarations */
/* **************************************************************
* Complex types
****************************************************************/
typedef U32 DTable_max_t[FSE_DTABLE_SIZE_U32(FSE_MAX_TABLELOG)];
/* **************************************************************
* Templates
****************************************************************/
/*
designed to be included
for type-specific functions (template emulation in C)
Objective is to write these functions only once, for improved maintenance
*/
/* safety checks */
#ifndef FSE_FUNCTION_EXTENSION
# error "FSE_FUNCTION_EXTENSION must be defined"
#endif
#ifndef FSE_FUNCTION_TYPE
# error "FSE_FUNCTION_TYPE must be defined"
#endif
/* Function names */
#define FSE_CAT(X,Y) X##Y
#define FSE_FUNCTION_NAME(X,Y) FSE_CAT(X,Y)
#define FSE_TYPE_NAME(X,Y) FSE_CAT(X,Y)
/* Function templates */
FSE_DTable* FSE_createDTable (unsigned tableLog)
{
if (tableLog > FSE_TABLELOG_ABSOLUTE_MAX) tableLog = FSE_TABLELOG_ABSOLUTE_MAX;
return (FSE_DTable*)malloc( FSE_DTABLE_SIZE_U32(tableLog) * sizeof (U32) );
}
void FSE_freeDTable (FSE_DTable* dt)
{
free(dt);
}
size_t FSE_buildDTable(FSE_DTable* dt, const short* normalizedCounter, unsigned maxSymbolValue, unsigned tableLog)
{
void* const tdPtr = dt+1; /* because *dt is unsigned, 32-bits aligned on 32-bits */
FSE_DECODE_TYPE* const tableDecode = (FSE_DECODE_TYPE*) (tdPtr);
U16 symbolNext[FSE_MAX_SYMBOL_VALUE+1];
U32 const maxSV1 = maxSymbolValue + 1;
U32 const tableSize = 1 << tableLog;
U32 highThreshold = tableSize-1;
/* Sanity Checks */
if (maxSymbolValue > FSE_MAX_SYMBOL_VALUE) return ERROR(maxSymbolValue_tooLarge);
if (tableLog > FSE_MAX_TABLELOG) return ERROR(tableLog_tooLarge);
/* Init, lay down lowprob symbols */
{ FSE_DTableHeader DTableH;
DTableH.tableLog = (U16)tableLog;
DTableH.fastMode = 1;
{ S16 const largeLimit= (S16)(1 << (tableLog-1));
U32 s;
for (s=0; s<maxSV1; s++) {
if (normalizedCounter[s]==-1) {
tableDecode[highThreshold--].symbol = (FSE_FUNCTION_TYPE)s;
symbolNext[s] = 1;
} else {
if (normalizedCounter[s] >= largeLimit) DTableH.fastMode=0;
symbolNext[s] = normalizedCounter[s];
} } }
memcpy(dt, &DTableH, sizeof(DTableH));
}
/* Spread symbols */
{ U32 const tableMask = tableSize-1;
U32 const step = FSE_TABLESTEP(tableSize);
U32 s, position = 0;
for (s=0; s<maxSV1; s++) {
int i;
for (i=0; i<normalizedCounter[s]; i++) {
tableDecode[position].symbol = (FSE_FUNCTION_TYPE)s;
position = (position + step) & tableMask;
while (position > highThreshold) position = (position + step) & tableMask; /* lowprob area */
} }
if (position!=0) return ERROR(GENERIC); /* position must reach all cells once, otherwise normalizedCounter is incorrect */
}
/* Build Decoding table */
{ U32 u;
for (u=0; u<tableSize; u++) {
FSE_FUNCTION_TYPE const symbol = (FSE_FUNCTION_TYPE)(tableDecode[u].symbol);
U16 nextState = symbolNext[symbol]++;
tableDecode[u].nbBits = (BYTE) (tableLog - BIT_highbit32 ((U32)nextState) );
tableDecode[u].newState = (U16) ( (nextState << tableDecode[u].nbBits) - tableSize);
} }
return 0;
}
#ifndef FSE_COMMONDEFS_ONLY
/*-*******************************************************
* Decompression (Byte symbols)
*********************************************************/
size_t FSE_buildDTable_rle (FSE_DTable* dt, BYTE symbolValue)
{
void* ptr = dt;
FSE_DTableHeader* const DTableH = (FSE_DTableHeader*)ptr;
void* dPtr = dt + 1;
FSE_decode_t* const cell = (FSE_decode_t*)dPtr;
DTableH->tableLog = 0;
DTableH->fastMode = 0;
cell->newState = 0;
cell->symbol = symbolValue;
cell->nbBits = 0;
return 0;
}
size_t FSE_buildDTable_raw (FSE_DTable* dt, unsigned nbBits)
{
void* ptr = dt;
FSE_DTableHeader* const DTableH = (FSE_DTableHeader*)ptr;
void* dPtr = dt + 1;
FSE_decode_t* const dinfo = (FSE_decode_t*)dPtr;
const unsigned tableSize = 1 << nbBits;
const unsigned tableMask = tableSize - 1;
const unsigned maxSV1 = tableMask+1;
unsigned s;
/* Sanity checks */
if (nbBits < 1) return ERROR(GENERIC); /* min size */
/* Build Decoding Table */
DTableH->tableLog = (U16)nbBits;
DTableH->fastMode = 1;
for (s=0; s<maxSV1; s++) {
dinfo[s].newState = 0;
dinfo[s].symbol = (BYTE)s;
dinfo[s].nbBits = (BYTE)nbBits;
}
return 0;
}
FORCE_INLINE size_t FSE_decompress_usingDTable_generic(
void* dst, size_t maxDstSize,
const void* cSrc, size_t cSrcSize,
const FSE_DTable* dt, const unsigned fast)
{
BYTE* const ostart = (BYTE*) dst;
BYTE* op = ostart;
BYTE* const omax = op + maxDstSize;
BYTE* const olimit = omax-3;
BIT_DStream_t bitD;
FSE_DState_t state1;
FSE_DState_t state2;
/* Init */
{ size_t const errorCode = BIT_initDStream(&bitD, cSrc, cSrcSize); /* replaced last arg by maxCompressed Size */
if (FSE_isError(errorCode)) return errorCode; }
FSE_initDState(&state1, &bitD, dt);
FSE_initDState(&state2, &bitD, dt);
#define FSE_GETSYMBOL(statePtr) fast ? FSE_decodeSymbolFast(statePtr, &bitD) : FSE_decodeSymbol(statePtr, &bitD)
/* 4 symbols per loop */
for ( ; (BIT_reloadDStream(&bitD)==BIT_DStream_unfinished) && (op<olimit) ; op+=4) {
op[0] = FSE_GETSYMBOL(&state1);
if (FSE_MAX_TABLELOG*2+7 > sizeof(bitD.bitContainer)*8) /* This test must be static */
BIT_reloadDStream(&bitD);
op[1] = FSE_GETSYMBOL(&state2);
if (FSE_MAX_TABLELOG*4+7 > sizeof(bitD.bitContainer)*8) /* This test must be static */
{ if (BIT_reloadDStream(&bitD) > BIT_DStream_unfinished) { op+=2; break; } }
op[2] = FSE_GETSYMBOL(&state1);
if (FSE_MAX_TABLELOG*2+7 > sizeof(bitD.bitContainer)*8) /* This test must be static */
BIT_reloadDStream(&bitD);
op[3] = FSE_GETSYMBOL(&state2);
}
/* tail */
/* note : BIT_reloadDStream(&bitD) >= FSE_DStream_partiallyFilled; Ends at exactly BIT_DStream_completed */
while (1) {
if (op>(omax-2)) return ERROR(dstSize_tooSmall);
*op++ = FSE_GETSYMBOL(&state1);
if (BIT_reloadDStream(&bitD)==BIT_DStream_overflow) {
*op++ = FSE_GETSYMBOL(&state2);
break;
}
if (op>(omax-2)) return ERROR(dstSize_tooSmall);
*op++ = FSE_GETSYMBOL(&state2);
if (BIT_reloadDStream(&bitD)==BIT_DStream_overflow) {
*op++ = FSE_GETSYMBOL(&state1);
break;
} }
return op-ostart;
}
size_t FSE_decompress_usingDTable(void* dst, size_t originalSize,
const void* cSrc, size_t cSrcSize,
const FSE_DTable* dt)
{
const void* ptr = dt;
const FSE_DTableHeader* DTableH = (const FSE_DTableHeader*)ptr;
const U32 fastMode = DTableH->fastMode;
/* select fast mode (static) */
if (fastMode) return FSE_decompress_usingDTable_generic(dst, originalSize, cSrc, cSrcSize, dt, 1);
return FSE_decompress_usingDTable_generic(dst, originalSize, cSrc, cSrcSize, dt, 0);
}
size_t FSE_decompress(void* dst, size_t maxDstSize, const void* cSrc, size_t cSrcSize)
{
const BYTE* const istart = (const BYTE*)cSrc;
const BYTE* ip = istart;
short counting[FSE_MAX_SYMBOL_VALUE+1];
DTable_max_t dt; /* Static analyzer seems unable to understand this table will be properly initialized later */
unsigned tableLog;
unsigned maxSymbolValue = FSE_MAX_SYMBOL_VALUE;
if (cSrcSize<2) return ERROR(srcSize_wrong); /* too small input size */
/* normal FSE decoding mode */
{ size_t const NCountLength = FSE_readNCount (counting, &maxSymbolValue, &tableLog, istart, cSrcSize);
if (FSE_isError(NCountLength)) return NCountLength;
if (NCountLength >= cSrcSize) return ERROR(srcSize_wrong); /* too small input size */
ip += NCountLength;
cSrcSize -= NCountLength;
}
{ size_t const errorCode = FSE_buildDTable (dt, counting, maxSymbolValue, tableLog);
if (FSE_isError(errorCode)) return errorCode; }
return FSE_decompress_usingDTable (dst, maxDstSize, ip, cSrcSize, dt); /* always return, even if it is an error code */
}
#endif /* FSE_COMMONDEFS_ONLY */
@@ -334,6 +334,54 @@ MEM_STATIC unsigned FSE_endOfDState(const FSE_DState_t* DStatePtr)
}
#ifndef FSE_COMMONDEFS_ONLY
/* **************************************************************
* Tuning parameters
****************************************************************/
/*!MEMORY_USAGE :
* Memory usage formula : N->2^N Bytes (examples : 10 -> 1KB; 12 -> 4KB ; 16 -> 64KB; 20 -> 1MB; etc.)
* Increasing memory usage improves compression ratio
* Reduced memory usage can improve speed, due to cache effect
* Recommended max value is 14, for 16KB, which nicely fits into Intel x86 L1 cache */
#define FSE_MAX_MEMORY_USAGE 14
#define FSE_DEFAULT_MEMORY_USAGE 13
/*!FSE_MAX_SYMBOL_VALUE :
* Maximum symbol value authorized.
* Required for proper stack allocation */
#define FSE_MAX_SYMBOL_VALUE 255
/* **************************************************************
* template functions type & suffix
****************************************************************/
#define FSE_FUNCTION_TYPE BYTE
#define FSE_FUNCTION_EXTENSION
#define FSE_DECODE_TYPE FSE_decode_t
#endif /* !FSE_COMMONDEFS_ONLY */
/* ***************************************************************
* Constants
*****************************************************************/
#define FSE_MAX_TABLELOG (FSE_MAX_MEMORY_USAGE-2)
#define FSE_MAX_TABLESIZE (1U<<FSE_MAX_TABLELOG)
#define FSE_MAXTABLESIZE_MASK (FSE_MAX_TABLESIZE-1)
#define FSE_DEFAULT_TABLELOG (FSE_DEFAULT_MEMORY_USAGE-2)
#define FSE_MIN_TABLELOG 5
#define FSE_TABLELOG_ABSOLUTE_MAX 15
#if FSE_MAX_TABLELOG > FSE_TABLELOG_ABSOLUTE_MAX
#error "FSE_MAX_TABLELOG > FSE_TABLELOG_ABSOLUTE_MAX is not supported"
#endif
#define FSE_TABLESTEP(tableSize) ((tableSize>>1) + (tableSize>>3) + 3)
#if defined (__cplusplus)
}
#endif
+6 -6
View File
@@ -1,5 +1,5 @@
/* ******************************************************************
Huff0 : Huffman coder, part of New Generation Entropy library
Huffman coder, part of New Generation Entropy library
header file
Copyright (C) 2013-2016, Yann Collet.
@@ -31,8 +31,8 @@
You can contact the author at :
- Source repository : https://github.com/Cyan4973/FiniteStateEntropy
****************************************************************** */
#ifndef HUFF0_H
#define HUFF0_H
#ifndef HUF_H
#define HUF_H
#if defined (__cplusplus)
extern "C" {
@@ -46,7 +46,7 @@ extern "C" {
/* ****************************************
* Huff0 simple functions
* HUF simple functions
******************************************/
size_t HUF_compress(void* dst, size_t dstCapacity,
const void* src, size_t srcSize);
@@ -63,7 +63,7 @@ HUF_compress() :
if HUF_isError(return), compression failed (more details using HUF_getErrorName())
HUF_decompress() :
Decompress Huff0 data from buffer 'cSrc', of size 'cSrcSize',
Decompress HUF data from buffer 'cSrc', of size 'cSrcSize',
into already allocated destination buffer 'dst', of size 'dstSize'.
`dstSize` : must be the **exact** size of original (uncompressed) data.
Note : in contrast with FSE, HUF_decompress can regenerate
@@ -94,4 +94,4 @@ size_t HUF_compress2 (void* dst, size_t dstSize, const void* src, size_t srcSize
}
#endif
#endif /* HUFF0_H */
#endif /* HUF_H */
+99 -11
View File
@@ -1,5 +1,5 @@
/* ******************************************************************
Huff0 : Huffman codec, part of New Generation Entropy library
Huffman codec, part of New Generation Entropy library
header file, for static linking only
Copyright (C) 2013-2016, Yann Collet
@@ -31,8 +31,8 @@
You can contact the author at :
- Source repository : https://github.com/Cyan4973/FiniteStateEntropy
****************************************************************** */
#ifndef HUFF0_STATIC_H
#define HUFF0_STATIC_H
#ifndef HUF_STATIC_H
#define HUF_STATIC_H
#if defined (__cplusplus)
extern "C" {
@@ -42,24 +42,26 @@ extern "C" {
/* ****************************************
* Dependency
******************************************/
#include "huff0.h"
#include "huf.h"
#include "fse.h"
#include "bitstream.h"
/* ****************************************
* Static allocation
******************************************/
/* Huff0 buffer bounds */
/* HUF buffer bounds */
#define HUF_CTABLEBOUND 129
#define HUF_BLOCKBOUND(size) (size + (size>>8) + 8) /* only true if incompressible pre-filtered with fast heuristic */
#define HUF_COMPRESSBOUND(size) (HUF_CTABLEBOUND + HUF_BLOCKBOUND(size)) /* Macro version, useful for static allocation */
/* static allocation of Huff0's Compression Table */
/* static allocation of HUF's Compression Table */
#define HUF_CREATE_STATIC_CTABLE(name, maxSymbolValue) \
U32 name##hb[maxSymbolValue+1]; \
void* name##hv = &(name##hb); \
HUF_CElt* name = (HUF_CElt*)(name##hv) /* no final ; */
/* static allocation of Huff0's DTable */
/* static allocation of HUF's DTable */
#define HUF_DTABLE_SIZE(maxTableLog) (1 + (1<<maxTableLog))
#define HUF_CREATE_STATIC_DTABLEX2(DTable, maxTableLog) \
unsigned short DTable[HUF_DTABLE_SIZE(maxTableLog)] = { maxTableLog }
@@ -74,11 +76,11 @@ extern "C" {
******************************************/
size_t HUF_decompress4X2 (void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize); /* single-symbol decoder */
size_t HUF_decompress4X4 (void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize); /* double-symbols decoder */
size_t HUF_decompress4X6 (void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize); /* quad-symbols decoder */
size_t HUF_decompress4X6 (void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize); /* quad-symbols decoder, only works for dstSize >= 64 */
/* ****************************************
* Huff0 detailed API
* HUF detailed API
******************************************/
/*!
HUF_compress() does the following:
@@ -120,7 +122,7 @@ size_t HUF_compress1X_usingCTable(void* dst, size_t dstSize, const void* src, si
size_t HUF_decompress1X2 (void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize); /* single-symbol decoder */
size_t HUF_decompress1X4 (void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize); /* double-symbol decoder */
size_t HUF_decompress1X6 (void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize); /* quad-symbol decoder */
size_t HUF_decompress1X6 (void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize); /* quad-symbols decoder, only works for dstSize >= 64 */
size_t HUF_decompress1X2_usingDTable(void* dst, size_t maxDstSize, const void* cSrc, size_t cSrcSize, const unsigned short* DTable);
size_t HUF_decompress1X4_usingDTable(void* dst, size_t maxDstSize, const void* cSrc, size_t cSrcSize, const unsigned* DTable);
@@ -132,8 +134,94 @@ size_t HUF_decompress1X6_usingDTable(void* dst, size_t maxDstSize, const void* c
size_t HUF_readCTable (HUF_CElt* CTable, unsigned maxSymbolValue, const void* src, size_t srcSize);
/* **************************************************************
* Constants
****************************************************************/
#define HUF_ABSOLUTEMAX_TABLELOG 16 /* absolute limit of HUF_MAX_TABLELOG. Beyond that value, code does not work */
#define HUF_MAX_TABLELOG 12 /* max configured tableLog (for static allocation); can be modified up to HUF_ABSOLUTEMAX_TABLELOG */
#define HUF_DEFAULT_TABLELOG HUF_MAX_TABLELOG /* tableLog by default, when not specified */
#define HUF_MAX_SYMBOL_VALUE 255
#if (HUF_MAX_TABLELOG > HUF_ABSOLUTEMAX_TABLELOG)
# error "HUF_MAX_TABLELOG is too large !"
#endif
/*! HUF_readStats() :
Read compact Huffman tree, saved by HUF_writeCTable().
`huffWeight` is destination buffer.
@return : size read from `src`
*/
MEM_STATIC size_t HUF_readStats(BYTE* huffWeight, size_t hwSize, U32* rankStats,
U32* nbSymbolsPtr, U32* tableLogPtr,
const void* src, size_t srcSize)
{
U32 weightTotal;
const BYTE* ip = (const BYTE*) src;
size_t iSize = ip[0];
size_t oSize;
//memset(huffWeight, 0, hwSize); /* is not necessary, even though some analyzer complain ... */
if (iSize >= 128) { /* special header */
if (iSize >= (242)) { /* RLE */
static U32 l[14] = { 1, 2, 3, 4, 7, 8, 15, 16, 31, 32, 63, 64, 127, 128 };
oSize = l[iSize-242];
memset(huffWeight, 1, hwSize);
iSize = 0;
}
else { /* Incompressible */
oSize = iSize - 127;
iSize = ((oSize+1)/2);
if (iSize+1 > srcSize) return ERROR(srcSize_wrong);
if (oSize >= hwSize) return ERROR(corruption_detected);
ip += 1;
{ U32 n;
for (n=0; n<oSize; n+=2) {
huffWeight[n] = ip[n/2] >> 4;
huffWeight[n+1] = ip[n/2] & 15;
} } } }
else { /* header compressed with FSE (normal case) */
if (iSize+1 > srcSize) return ERROR(srcSize_wrong);
oSize = FSE_decompress(huffWeight, hwSize-1, ip+1, iSize); /* max (hwSize-1) values decoded, as last one is implied */
if (FSE_isError(oSize)) return oSize;
}
/* collect weight stats */
memset(rankStats, 0, (HUF_ABSOLUTEMAX_TABLELOG + 1) * sizeof(U32));
weightTotal = 0;
{ U32 n; for (n=0; n<oSize; n++) {
if (huffWeight[n] >= HUF_ABSOLUTEMAX_TABLELOG) return ERROR(corruption_detected);
rankStats[huffWeight[n]]++;
weightTotal += (1 << huffWeight[n]) >> 1;
} }
/* get last non-null symbol weight (implied, total must be 2^n) */
{ U32 const tableLog = BIT_highbit32(weightTotal) + 1;
if (tableLog > HUF_ABSOLUTEMAX_TABLELOG) return ERROR(corruption_detected);
*tableLogPtr = tableLog;
/* determine last weight */
{ U32 const total = 1 << tableLog;
U32 const rest = total - weightTotal;
U32 const verif = 1 << BIT_highbit32(rest);
U32 const lastWeight = BIT_highbit32(rest) + 1;
if (verif != rest) return ERROR(corruption_detected); /* last value must be a clean power of 2 */
huffWeight[oSize] = (BYTE)lastWeight;
rankStats[lastWeight]++;
} }
/* check tree construction validity */
if ((rankStats[1] < 2) || (rankStats[1] & 1)) return ERROR(corruption_detected); /* by construction : at least 2 elts of rank 1, must be even */
/* results */
*nbSymbolsPtr = (U32)(oSize+1);
return iSize+1;
}
#if defined (__cplusplus)
}
#endif
#endif /* HUFF0_STATIC_H */
#endif /* HUF_STATIC_H */
+107 -33
View File
@@ -63,7 +63,7 @@ extern "C" {
/*-**************************************************************
* Basic Types
*****************************************************************/
#if defined (__cplusplus) || (defined (__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L) /* C99 */)
#if !defined (__VMS) && (defined (__cplusplus) || (defined (__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L) /* C99 */) )
# include <stdint.h>
typedef uint8_t BYTE;
typedef uint16_t U16;
@@ -108,8 +108,8 @@ extern "C" {
# endif
#endif
MEM_STATIC unsigned MEM_32bits(void) { return sizeof(void*)==4; }
MEM_STATIC unsigned MEM_64bits(void) { return sizeof(void*)==8; }
MEM_STATIC unsigned MEM_32bits(void) { return sizeof(size_t)==4; }
MEM_STATIC unsigned MEM_64bits(void) { return sizeof(size_t)==8; }
MEM_STATIC unsigned MEM_isLittleEndian(void)
{
@@ -187,6 +187,48 @@ MEM_STATIC void MEM_write64(void* memPtr, U64 value)
#endif /* MEM_FORCE_MEMORY_ACCESS */
MEM_STATIC U32 MEM_swap32(U32 in)
{
#if defined(_MSC_VER) /* Visual Studio */
return _byteswap_ulong(in);
#elif defined (__GNUC__)
return __builtin_bswap32(in);
#else
return ((in << 24) & 0xff000000 ) |
((in << 8) & 0x00ff0000 ) |
((in >> 8) & 0x0000ff00 ) |
((in >> 24) & 0x000000ff );
#endif
}
MEM_STATIC U64 MEM_swap64(U64 in)
{
#if defined(_MSC_VER) /* Visual Studio */
return _byteswap_uint64(in);
#elif defined (__GNUC__)
return __builtin_bswap64(in);
#else
return ((in << 56) & 0xff00000000000000ULL) |
((in << 40) & 0x00ff000000000000ULL) |
((in << 24) & 0x0000ff0000000000ULL) |
((in << 8) & 0x000000ff00000000ULL) |
((in >> 8) & 0x00000000ff000000ULL) |
((in >> 24) & 0x0000000000ff0000ULL) |
((in >> 40) & 0x000000000000ff00ULL) |
((in >> 56) & 0x00000000000000ffULL);
#endif
}
MEM_STATIC size_t MEM_swapST(size_t in)
{
if (MEM_32bits())
return (size_t)MEM_swap32((U32)in);
else
return (size_t)MEM_swap64((U64)in);
}
/*=== Little endian r/w ===*/
MEM_STATIC U16 MEM_readLE16(const void* memPtr)
{
if (MEM_isLittleEndian())
@@ -212,51 +254,32 @@ MEM_STATIC U32 MEM_readLE32(const void* memPtr)
{
if (MEM_isLittleEndian())
return MEM_read32(memPtr);
else {
const BYTE* p = (const BYTE*)memPtr;
return (U32)((U32)p[0] + ((U32)p[1]<<8) + ((U32)p[2]<<16) + ((U32)p[3]<<24));
}
else
return MEM_swap32(MEM_read32(memPtr));
}
MEM_STATIC void MEM_writeLE32(void* memPtr, U32 val32)
{
if (MEM_isLittleEndian()) {
if (MEM_isLittleEndian())
MEM_write32(memPtr, val32);
} else {
BYTE* p = (BYTE*)memPtr;
p[0] = (BYTE)val32;
p[1] = (BYTE)(val32>>8);
p[2] = (BYTE)(val32>>16);
p[3] = (BYTE)(val32>>24);
}
else
MEM_write32(memPtr, MEM_swap32(val32));
}
MEM_STATIC U64 MEM_readLE64(const void* memPtr)
{
if (MEM_isLittleEndian())
return MEM_read64(memPtr);
else {
const BYTE* p = (const BYTE*)memPtr;
return (U64)((U64)p[0] + ((U64)p[1]<<8) + ((U64)p[2]<<16) + ((U64)p[3]<<24)
+ ((U64)p[4]<<32) + ((U64)p[5]<<40) + ((U64)p[6]<<48) + ((U64)p[7]<<56));
}
else
return MEM_swap64(MEM_read64(memPtr));
}
MEM_STATIC void MEM_writeLE64(void* memPtr, U64 val64)
{
if (MEM_isLittleEndian()) {
if (MEM_isLittleEndian())
MEM_write64(memPtr, val64);
} else {
BYTE* p = (BYTE*)memPtr;
p[0] = (BYTE)val64;
p[1] = (BYTE)(val64>>8);
p[2] = (BYTE)(val64>>16);
p[3] = (BYTE)(val64>>24);
p[4] = (BYTE)(val64>>32);
p[5] = (BYTE)(val64>>40);
p[6] = (BYTE)(val64>>48);
p[7] = (BYTE)(val64>>56);
}
else
MEM_write64(memPtr, MEM_swap64(val64));
}
MEM_STATIC size_t MEM_readLEST(const void* memPtr)
@@ -275,7 +298,58 @@ MEM_STATIC void MEM_writeLEST(void* memPtr, size_t val)
MEM_writeLE64(memPtr, (U64)val);
}
/* function safe only for comparisons */
/*=== Big endian r/w ===*/
MEM_STATIC U32 MEM_readBE32(const void* memPtr)
{
if (MEM_isLittleEndian())
return MEM_swap32(MEM_read32(memPtr));
else
return MEM_read32(memPtr);
}
MEM_STATIC void MEM_writeBE32(void* memPtr, U32 val32)
{
if (MEM_isLittleEndian())
MEM_write32(memPtr, MEM_swap32(val32));
else
MEM_write32(memPtr, val32);
}
MEM_STATIC U64 MEM_readBE64(const void* memPtr)
{
if (MEM_isLittleEndian())
return MEM_swap64(MEM_read64(memPtr));
else
return MEM_read64(memPtr);
}
MEM_STATIC void MEM_writeBE64(void* memPtr, U64 val64)
{
if (MEM_isLittleEndian())
MEM_write64(memPtr, MEM_swap64(val64));
else
MEM_write64(memPtr, val64);
}
MEM_STATIC size_t MEM_readBEST(const void* memPtr)
{
if (MEM_32bits())
return (size_t)MEM_readBE32(memPtr);
else
return (size_t)MEM_readBE64(memPtr);
}
MEM_STATIC void MEM_writeBEST(void* memPtr, size_t val)
{
if (MEM_32bits())
MEM_writeBE32(memPtr, (U32)val);
else
MEM_writeBE64(memPtr, (U64)val);
}
/* function safe only for comparisons */
MEM_STATIC U32 MEM_readMINMATCH(const void* memPtr, U32 length)
{
switch (length)
+1 -1
View File
@@ -153,7 +153,7 @@ ZSTDLIB_API unsigned ZBUFF_isError(size_t errorCode);
ZSTDLIB_API const char* ZBUFF_getErrorName(size_t errorCode);
/** Functions below provide recommended buffer sizes for Compression or Decompression operations.
* These sizes are just hints, and tend to offer better latency */
* These sizes are just hints, they tend to offer better latency */
ZSTDLIB_API size_t ZBUFF_recommendedCInSize(void);
ZSTDLIB_API size_t ZBUFF_recommendedCOutSize(void);
ZSTDLIB_API size_t ZBUFF_recommendedDInSize(void);
@@ -46,6 +46,7 @@ extern "C" {
***************************************/
#include "zstd_static.h" /* ZSTD_parameters */
#include "zbuff.h"
#include "zstd_internal.h" /* MIN */
/* *************************************
@@ -55,6 +56,13 @@ ZSTDLIB_API size_t ZBUFF_compressInit_advanced(ZBUFF_CCtx* cctx,
const void* dict, size_t dictSize,
ZSTD_parameters params, U64 pledgedSrcSize);
MEM_STATIC size_t ZBUFF_limitCopy(void* dst, size_t dstCapacity, const void* src, size_t srcSize)
{
size_t length = MIN(dstCapacity, srcSize);
memcpy(dst, src, length);
return length;
}
#if defined (__cplusplus)
}
+9 -3
View File
@@ -59,9 +59,15 @@ extern "C" {
/* *************************************
* Version
***************************************/
#define ZSTD_VERSION_MAJOR 0 /* for breaking interface changes */
#define ZSTD_VERSION_MINOR 6 /* for new (non-breaking) interface capabilities */
#define ZSTD_VERSION_RELEASE 0 /* for tweaks, bug-fixes, or development */
#define ZSTD_VERSION_MAJOR 0
#define ZSTD_VERSION_MINOR 6
#define ZSTD_VERSION_RELEASE 1
#define ZSTD_LIB_VERSION ZSTD_VERSION_MAJOR.ZSTD_VERSION_MINOR.ZSTD_VERSION_RELEASE
#define ZSTD_QUOTE(str) #str
#define ZSTD_EXPAND_AND_QUOTE(str) ZSTD_QUOTE(str)
#define ZSTD_VERSION_STRING ZSTD_EXPAND_AND_QUOTE(ZSTD_LIB_VERSION)
#define ZSTD_VERSION_NUMBER (ZSTD_VERSION_MAJOR *100*100 + ZSTD_VERSION_MINOR *100 + ZSTD_VERSION_RELEASE)
ZSTDLIB_API unsigned ZSTD_versionNumber (void);
+72
View File
@@ -0,0 +1,72 @@
/*
Common functions of Zstd compression library
Copyright (C) 2015-2016, Yann Collet.
BSD 2-Clause License (http://www.opensource.org/licenses/bsd-license.php)
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the following disclaimer
in the documentation and/or other materials provided with the
distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
You can contact the author at :
- zstd homepage : http://www.zstd.net/
*/
/*-*************************************
* Dependencies
***************************************/
#include "error_private.h"
#include "zstd.h" /* declaration of ZSTD_isError, ZSTD_getErrorName */
#include "zbuff.h" /* declaration of ZBUFF_isError, ZBUFF_getErrorName */
/*-****************************************
* Version
******************************************/
unsigned ZSTD_versionNumber (void) { return ZSTD_VERSION_NUMBER; }
/*-****************************************
* ZSTD Error Management
******************************************/
/*! ZSTD_isError() :
* tells if a return value is an error code */
unsigned ZSTD_isError(size_t code) { return ERR_isError(code); }
/*! ZSTD_getErrorName() :
* provides error code string from function result (useful for debugging) */
const char* ZSTD_getErrorName(size_t code) { return ERR_getErrorName(code); }
/*! ZSTD_getError() :
* convert a `size_t` function result into a proper ZSTD_errorCode enum */
ZSTD_ErrorCode ZSTD_getErrorCode(size_t code) { return ERR_getErrorCode(code); }
/*! ZSTD_getErrorString() :
* provides error code string from enum */
const char* ZSTD_getErrorString(ZSTD_ErrorCode code) { return ERR_getErrorName(code); }
/* **************************************************************
* ZBUFF Error Management
****************************************************************/
unsigned ZBUFF_isError(size_t errorCode) { return ERR_isError(errorCode); }
const char* ZBUFF_getErrorName(size_t errorCode) { return ERR_getErrorName(errorCode); }
@@ -203,10 +203,10 @@ typedef struct {
#include ".debug/zstd_stats.h"
#else
typedef struct { U32 unused; } ZSTD_stats_t;
MEM_STATIC void ZSTD_statsPrint(ZSTD_stats_t* stats, U32 searchLength) { (void)stats; (void)searchLength; };
MEM_STATIC void ZSTD_statsInit(ZSTD_stats_t* stats) { (void)stats; };
MEM_STATIC void ZSTD_statsResetFreqs(ZSTD_stats_t* stats) { (void)stats; };
MEM_STATIC void ZSTD_statsUpdatePrices(ZSTD_stats_t* stats, size_t litLength, const BYTE* literals, size_t offset, size_t matchLength) { (void)stats; (void)litLength; (void)literals; (void)offset; (void)matchLength; };
MEM_STATIC void ZSTD_statsPrint(ZSTD_stats_t* stats, U32 searchLength) { (void)stats; (void)searchLength; }
MEM_STATIC void ZSTD_statsInit(ZSTD_stats_t* stats) { (void)stats; }
MEM_STATIC void ZSTD_statsResetFreqs(ZSTD_stats_t* stats) { (void)stats; }
MEM_STATIC void ZSTD_statsUpdatePrices(ZSTD_stats_t* stats, size_t litLength, const BYTE* literals, size_t offset, size_t matchLength) { (void)stats; (void)litLength; (void)literals; (void)offset; (void)matchLength; }
#endif
typedef struct {
@@ -242,12 +242,14 @@ typedef struct {
U32 log2litSum;
U32 log2offCodeSum;
U32 factor;
U32 cachedPrice;
U32 cachedLitLength;
const BYTE* cachedLiterals;
ZSTD_stats_t stats;
} seqStore_t;
const seqStore_t* ZSTD_getSeqStore(const ZSTD_CCtx* ctx);
void ZSTD_seqToCodes(const seqStore_t* seqStorePtr, size_t const nbSeq);
size_t ZSTD_compressBegin_targetSrcSize(ZSTD_CCtx* zc, const void* dict, size_t dictSize, size_t targetSrcSize, int compressionLevel);
#endif /* ZSTD_CCOMMON_H_MODULE */
@@ -103,7 +103,7 @@ ZSTDLIB_API unsigned ZSTD_maxCLevel (void);
/*! ZSTD_getCParams() :
* @return ZSTD_compressionParameters structure for a selected compression level and srcSize.
* `srcSize` value is optional, select 0 if not known */
ZSTD_compressionParameters ZSTD_getCParams(int compressionLevel, U64 srcSize, size_t dictSize);
ZSTDLIB_API ZSTD_compressionParameters ZSTD_getCParams(int compressionLevel, U64 srcSize, size_t dictSize);
/*! ZSTD_checkParams() :
* Ensure param values remain within authorized range */
@@ -250,8 +250,8 @@ ZSTDLIB_API size_t ZSTD_decompressContinue(ZSTD_DCtx* dctx, void* dst, size_t ds
*/
#define ZSTD_BLOCKSIZE_MAX (128 * 1024) /* define, for static allocation */
size_t ZSTD_compressBlock (ZSTD_CCtx* cctx, void* dst, size_t dstCapacity, const void* src, size_t srcSize);
size_t ZSTD_decompressBlock(ZSTD_DCtx* dctx, void* dst, size_t dstCapacity, const void* src, size_t srcSize);
ZSTDLIB_API size_t ZSTD_compressBlock (ZSTD_CCtx* cctx, void* dst, size_t dstCapacity, const void* src, size_t srcSize);
ZSTDLIB_API size_t ZSTD_decompressBlock(ZSTD_DCtx* dctx, void* dst, size_t dstCapacity, const void* src, size_t srcSize);
/*-*************************************
@@ -261,7 +261,8 @@ size_t ZSTD_decompressBlock(ZSTD_DCtx* dctx, void* dst, size_t dstCapacity, cons
/*! ZSTD_getErrorCode() :
convert a `size_t` function result into a `ZSTD_ErrorCode` enum type,
which can be used to compare directly with enum list published into "error_public.h" */
ZSTD_ErrorCode ZSTD_getError(size_t code);
ZSTDLIB_API ZSTD_ErrorCode ZSTD_getErrorCode(size_t functionResult);
ZSTDLIB_API const char* ZSTD_getErrorString(ZSTD_ErrorCode code);
#if defined (__cplusplus)
+16 -372
View File
@@ -1,5 +1,5 @@
/* ******************************************************************
FSE : Finite State Entropy coder
FSE : Finite State Entropy encoder
Copyright (C) 2013-2015, Yann Collet.
BSD 2-Clause License (http://www.opensource.org/licenses/bsd-license.php)
@@ -32,35 +32,6 @@
- Public forum : https://groups.google.com/forum/#!forum/lz4c
****************************************************************** */
#ifndef FSE_COMMONDEFS_ONLY
/* **************************************************************
* Tuning parameters
****************************************************************/
/*!MEMORY_USAGE :
* Memory usage formula : N->2^N Bytes (examples : 10 -> 1KB; 12 -> 4KB ; 16 -> 64KB; 20 -> 1MB; etc.)
* Increasing memory usage improves compression ratio
* Reduced memory usage can improve speed, due to cache effect
* Recommended max value is 14, for 16KB, which nicely fits into Intel x86 L1 cache */
#define FSE_MAX_MEMORY_USAGE 14
#define FSE_DEFAULT_MEMORY_USAGE 13
/*!FSE_MAX_SYMBOL_VALUE :
* Maximum symbol value authorized.
* Required for proper stack allocation */
#define FSE_MAX_SYMBOL_VALUE 255
/* **************************************************************
* template functions type & suffix
****************************************************************/
#define FSE_FUNCTION_TYPE BYTE
#define FSE_FUNCTION_EXTENSION
#define FSE_DECODE_TYPE FSE_decode_t
#endif /* !FSE_COMMONDEFS_ONLY */
/* **************************************************************
* Compiler specifics
****************************************************************/
@@ -89,21 +60,6 @@
#include "fse_static.h"
/* ***************************************************************
* Constants
*****************************************************************/
#define FSE_MAX_TABLELOG (FSE_MAX_MEMORY_USAGE-2)
#define FSE_MAX_TABLESIZE (1U<<FSE_MAX_TABLELOG)
#define FSE_MAXTABLESIZE_MASK (FSE_MAX_TABLESIZE-1)
#define FSE_DEFAULT_TABLELOG (FSE_DEFAULT_MEMORY_USAGE-2)
#define FSE_MIN_TABLELOG 5
#define FSE_TABLELOG_ABSOLUTE_MAX 15
#if FSE_MAX_TABLELOG > FSE_TABLELOG_ABSOLUTE_MAX
#error "FSE_MAX_TABLELOG > FSE_TABLELOG_ABSOLUTE_MAX is not supported"
#endif
/* **************************************************************
* Error Management
****************************************************************/
@@ -114,7 +70,6 @@
* Complex types
****************************************************************/
typedef U32 CTable_max_t[FSE_CTABLE_SIZE_U32(FSE_MAX_TABLELOG, FSE_MAX_SYMBOL_VALUE)];
typedef U32 DTable_max_t[FSE_DTABLE_SIZE_U32(FSE_MAX_TABLELOG)];
/* **************************************************************
@@ -141,8 +96,6 @@ typedef U32 DTable_max_t[FSE_DTABLE_SIZE_U32(FSE_MAX_TABLELOG)];
/* Function templates */
static U32 FSE_tableStep(U32 tableSize) { return (tableSize>>1) + (tableSize>>3) + 3; }
size_t FSE_buildCTable(FSE_CTable* ct, const short* normalizedCounter, unsigned maxSymbolValue, unsigned tableLog)
{
U32 const tableSize = 1 << tableLog;
@@ -151,12 +104,15 @@ size_t FSE_buildCTable(FSE_CTable* ct, const short* normalizedCounter, unsigned
U16* const tableU16 = ( (U16*) ptr) + 2;
void* const FSCT = ((U32*)ptr) + 1 /* header */ + (tableLog ? tableSize>>1 : 1) ;
FSE_symbolCompressionTransform* const symbolTT = (FSE_symbolCompressionTransform*) (FSCT);
U32 const step = FSE_tableStep(tableSize);
U32 const step = FSE_TABLESTEP(tableSize);
U32 cumul[FSE_MAX_SYMBOL_VALUE+2];
FSE_FUNCTION_TYPE tableSymbol[FSE_MAX_TABLESIZE]; /* memset() is not necessary, even if static analyzer complain about it */
U32 highThreshold = tableSize-1;
/* CTable header */
tableU16[-2] = (U16) tableLog;
tableU16[-1] = (U16) maxSymbolValue;
@@ -186,6 +142,7 @@ size_t FSE_buildCTable(FSE_CTable* ct, const short* normalizedCounter, unsigned
position = (position + step) & tableMask;
while (position > highThreshold) position = (position + step) & tableMask; /* Low proba area */
} }
if (position!=0) return ERROR(GENERIC); /* Must have gone through all positions */
}
@@ -202,6 +159,7 @@ size_t FSE_buildCTable(FSE_CTable* ct, const short* normalizedCounter, unsigned
switch (normalizedCounter[s])
{
case 0: break;
case -1:
case 1:
symbolTT[s].deltaNbBits = (tableLog << 16) - (1<<tableLog);
@@ -221,81 +179,8 @@ size_t FSE_buildCTable(FSE_CTable* ct, const short* normalizedCounter, unsigned
}
FSE_DTable* FSE_createDTable (unsigned tableLog)
{
if (tableLog > FSE_TABLELOG_ABSOLUTE_MAX) tableLog = FSE_TABLELOG_ABSOLUTE_MAX;
return (FSE_DTable*)malloc( FSE_DTABLE_SIZE_U32(tableLog) * sizeof (U32) );
}
void FSE_freeDTable (FSE_DTable* dt)
{
free(dt);
}
size_t FSE_buildDTable(FSE_DTable* dt, const short* normalizedCounter, unsigned maxSymbolValue, unsigned tableLog)
{
FSE_DTableHeader DTableH;
void* const tdPtr = dt+1; /* because dt is unsigned, 32-bits aligned on 32-bits */
FSE_DECODE_TYPE* const tableDecode = (FSE_DECODE_TYPE*) (tdPtr);
const U32 tableSize = 1 << tableLog;
const U32 tableMask = tableSize-1;
const U32 step = FSE_tableStep(tableSize);
U16 symbolNext[FSE_MAX_SYMBOL_VALUE+1];
U32 highThreshold = tableSize-1;
S16 const largeLimit= (S16)(1 << (tableLog-1));
U32 noLarge = 1;
U32 s;
/* Sanity Checks */
if (maxSymbolValue > FSE_MAX_SYMBOL_VALUE) return ERROR(maxSymbolValue_tooLarge);
if (tableLog > FSE_MAX_TABLELOG) return ERROR(tableLog_tooLarge);
/* Init, lay down lowprob symbols */
DTableH.tableLog = (U16)tableLog;
for (s=0; s<=maxSymbolValue; s++) {
if (normalizedCounter[s]==-1) {
tableDecode[highThreshold--].symbol = (FSE_FUNCTION_TYPE)s;
symbolNext[s] = 1;
} else {
if (normalizedCounter[s] >= largeLimit) noLarge=0;
symbolNext[s] = normalizedCounter[s];
} }
/* Spread symbols */
{ U32 position = 0;
for (s=0; s<=maxSymbolValue; s++) {
int i;
for (i=0; i<normalizedCounter[s]; i++) {
tableDecode[position].symbol = (FSE_FUNCTION_TYPE)s;
position = (position + step) & tableMask;
while (position > highThreshold) position = (position + step) & tableMask; /* lowprob area */
} }
if (position!=0) return ERROR(GENERIC); /* position must reach all cells once, otherwise normalizedCounter is incorrect */
}
/* Build Decoding table */
{ U32 u;
for (u=0; u<tableSize; u++) {
FSE_FUNCTION_TYPE symbol = (FSE_FUNCTION_TYPE)(tableDecode[u].symbol);
U16 nextState = symbolNext[symbol]++;
tableDecode[u].nbBits = (BYTE) (tableLog - BIT_highbit32 ((U32)nextState) );
tableDecode[u].newState = (U16) ( (nextState << tableDecode[u].nbBits) - tableSize);
} }
DTableH.fastMode = (U16)noLarge;
memcpy(dt, &DTableH, sizeof(DTableH));
return 0;
}
#ifndef FSE_COMMONDEFS_ONLY
/*-****************************************
* FSE helper functions
******************************************/
unsigned FSE_isError(size_t code) { return ERR_isError(code); }
const char* FSE_getErrorName(size_t code) { return ERR_getErrorName(code); }
/*-**************************************************************
* FSE NCount encoding-decoding
@@ -408,98 +293,6 @@ size_t FSE_writeNCount (void* buffer, size_t bufferSize, const short* normalized
}
size_t FSE_readNCount (short* normalizedCounter, unsigned* maxSVPtr, unsigned* tableLogPtr,
const void* headerBuffer, size_t hbSize)
{
const BYTE* const istart = (const BYTE*) headerBuffer;
const BYTE* const iend = istart + hbSize;
const BYTE* ip = istart;
int nbBits;
int remaining;
int threshold;
U32 bitStream;
int bitCount;
unsigned charnum = 0;
int previous0 = 0;
if (hbSize < 4) return ERROR(srcSize_wrong);
bitStream = MEM_readLE32(ip);
nbBits = (bitStream & 0xF) + FSE_MIN_TABLELOG; /* extract tableLog */
if (nbBits > FSE_TABLELOG_ABSOLUTE_MAX) return ERROR(tableLog_tooLarge);
bitStream >>= 4;
bitCount = 4;
*tableLogPtr = nbBits;
remaining = (1<<nbBits)+1;
threshold = 1<<nbBits;
nbBits++;
while ((remaining>1) && (charnum<=*maxSVPtr)) {
if (previous0) {
unsigned n0 = charnum;
while ((bitStream & 0xFFFF) == 0xFFFF) {
n0+=24;
if (ip < iend-5) {
ip+=2;
bitStream = MEM_readLE32(ip) >> bitCount;
} else {
bitStream >>= 16;
bitCount+=16;
} }
while ((bitStream & 3) == 3) {
n0+=3;
bitStream>>=2;
bitCount+=2;
}
n0 += bitStream & 3;
bitCount += 2;
if (n0 > *maxSVPtr) return ERROR(maxSymbolValue_tooSmall);
while (charnum < n0) normalizedCounter[charnum++] = 0;
if ((ip <= iend-7) || (ip + (bitCount>>3) <= iend-4)) {
ip += bitCount>>3;
bitCount &= 7;
bitStream = MEM_readLE32(ip) >> bitCount;
}
else
bitStream >>= 2;
}
{ short const max = (short)((2*threshold-1)-remaining);
short count;
if ((bitStream & (threshold-1)) < (U32)max) {
count = (short)(bitStream & (threshold-1));
bitCount += nbBits-1;
} else {
count = (short)(bitStream & (2*threshold-1));
if (count >= threshold) count -= max;
bitCount += nbBits;
}
count--; /* extra accuracy */
remaining -= FSE_abs(count);
normalizedCounter[charnum++] = count;
previous0 = !count;
while (remaining < threshold) {
nbBits--;
threshold >>= 1;
}
if ((ip <= iend-7) || (ip + (bitCount>>3) <= iend-4)) {
ip += bitCount>>3;
bitCount &= 7;
} else {
bitCount -= (int)(8 * (iend - 4 - ip));
ip = iend - 4;
}
bitStream = MEM_readLE32(ip) >> (bitCount & 31);
} }
if (remaining != 1) return ERROR(GENERIC);
*maxSVPtr = charnum-1;
ip += (bitCount+7)>>3;
if ((size_t)(ip-istart) > hbSize) return ERROR(srcSize_wrong);
return ip-istart;
}
/*-**************************************************************
* Counting histogram
@@ -519,6 +312,7 @@ static size_t FSE_count_simple(unsigned* count, unsigned* maxSymbolValuePtr,
unsigned maxSymbolValue = *maxSymbolValuePtr;
unsigned max=0;
memset(count, 0, (maxSymbolValue+1)*sizeof(*count));
if (srcSize==0) { *maxSymbolValuePtr = 0; return 0; }
@@ -542,6 +336,7 @@ static size_t FSE_count_parallel(unsigned* count, unsigned* maxSymbolValuePtr,
unsigned maxSymbolValue = *maxSymbolValuePtr;
unsigned max=0;
U32 Counting1[256] = { 0 };
U32 Counting2[256] = { 0 };
U32 Counting3[256] = { 0 };
@@ -619,6 +414,7 @@ size_t FSE_count(unsigned* count, unsigned* maxSymbolValuePtr,
}
/*-**************************************************************
* FSE Compression Code
****************************************************************/
@@ -763,6 +559,7 @@ size_t FSE_normalizeCount (short* normalizedCounter, unsigned tableLog,
if (tableLog < FSE_minTableLog(total, maxSymbolValue)) return ERROR(GENERIC); /* Too small tableLog, compression potentially impossible */
{ U32 const rtbTable[] = { 0, 473195, 504333, 520860, 550000, 700000, 750000, 830000 };
U64 const scale = 62 - tableLog;
U64 const step = ((U64)1<<62) / total; /* <== here, one division ! */
U64 const vStep = 1ULL<<(scale-20);
@@ -839,11 +636,13 @@ size_t FSE_buildCTable_raw (FSE_CTable* ct, unsigned nbBits)
/* Build Symbol Transformation Table */
{ const U32 deltaNbBits = (nbBits << 16) - (1 << nbBits);
for (s=0; s<=maxSymbolValue; s++) {
symbolTT[s].deltaNbBits = deltaNbBits;
symbolTT[s].deltaFindState = s-1;
} }
return 0;
}
@@ -878,6 +677,8 @@ static size_t FSE_compress_usingCTable_generic (void* dst, size_t dstSize,
const BYTE* const istart = (const BYTE*) src;
const BYTE* const iend = istart + srcSize;
const BYTE* ip=iend;
BIT_CStream_t bitC;
FSE_CState_t CState1, CState2;
@@ -908,6 +709,7 @@ static size_t FSE_compress_usingCTable_generic (void* dst, size_t dstSize,
/* 2 or 4 encoding per loop */
for ( ; ip>istart ; ) {
FSE_encodeSymbol(&bitC, &CState2, *--ip);
if (sizeof(bitC.bitContainer)*8 < FSE_MAX_TABLELOG*2+7 ) /* this test must be static */
@@ -998,162 +800,4 @@ size_t FSE_compress (void* dst, size_t dstSize, const void* src, size_t srcSize)
}
/*-*******************************************************
* Decompression (Byte symbols)
*********************************************************/
size_t FSE_buildDTable_rle (FSE_DTable* dt, BYTE symbolValue)
{
void* ptr = dt;
FSE_DTableHeader* const DTableH = (FSE_DTableHeader*)ptr;
void* dPtr = dt + 1;
FSE_decode_t* const cell = (FSE_decode_t*)dPtr;
DTableH->tableLog = 0;
DTableH->fastMode = 0;
cell->newState = 0;
cell->symbol = symbolValue;
cell->nbBits = 0;
return 0;
}
size_t FSE_buildDTable_raw (FSE_DTable* dt, unsigned nbBits)
{
void* ptr = dt;
FSE_DTableHeader* const DTableH = (FSE_DTableHeader*)ptr;
void* dPtr = dt + 1;
FSE_decode_t* const dinfo = (FSE_decode_t*)dPtr;
const unsigned tableSize = 1 << nbBits;
const unsigned tableMask = tableSize - 1;
const unsigned maxSymbolValue = tableMask;
unsigned s;
/* Sanity checks */
if (nbBits < 1) return ERROR(GENERIC); /* min size */
/* Build Decoding Table */
DTableH->tableLog = (U16)nbBits;
DTableH->fastMode = 1;
for (s=0; s<=maxSymbolValue; s++) {
dinfo[s].newState = 0;
dinfo[s].symbol = (BYTE)s;
dinfo[s].nbBits = (BYTE)nbBits;
}
return 0;
}
FORCE_INLINE size_t FSE_decompress_usingDTable_generic(
void* dst, size_t maxDstSize,
const void* cSrc, size_t cSrcSize,
const FSE_DTable* dt, const unsigned fast)
{
BYTE* const ostart = (BYTE*) dst;
BYTE* op = ostart;
BYTE* const omax = op + maxDstSize;
BYTE* const olimit = omax-3;
BIT_DStream_t bitD;
FSE_DState_t state1;
FSE_DState_t state2;
size_t errorCode;
/* Init */
errorCode = BIT_initDStream(&bitD, cSrc, cSrcSize); /* replaced last arg by maxCompressed Size */
if (FSE_isError(errorCode)) return errorCode;
FSE_initDState(&state1, &bitD, dt);
FSE_initDState(&state2, &bitD, dt);
#define FSE_GETSYMBOL(statePtr) fast ? FSE_decodeSymbolFast(statePtr, &bitD) : FSE_decodeSymbol(statePtr, &bitD)
/* 4 symbols per loop */
for ( ; (BIT_reloadDStream(&bitD)==BIT_DStream_unfinished) && (op<olimit) ; op+=4) {
op[0] = FSE_GETSYMBOL(&state1);
if (FSE_MAX_TABLELOG*2+7 > sizeof(bitD.bitContainer)*8) /* This test must be static */
BIT_reloadDStream(&bitD);
op[1] = FSE_GETSYMBOL(&state2);
if (FSE_MAX_TABLELOG*4+7 > sizeof(bitD.bitContainer)*8) /* This test must be static */
{ if (BIT_reloadDStream(&bitD) > BIT_DStream_unfinished) { op+=2; break; } }
op[2] = FSE_GETSYMBOL(&state1);
if (FSE_MAX_TABLELOG*2+7 > sizeof(bitD.bitContainer)*8) /* This test must be static */
BIT_reloadDStream(&bitD);
op[3] = FSE_GETSYMBOL(&state2);
}
/* tail */
/* note : BIT_reloadDStream(&bitD) >= FSE_DStream_partiallyFilled; Ends at exactly BIT_DStream_completed */
while (1) {
if (op>(omax-2)) return ERROR(dstSize_tooSmall);
*op++ = FSE_GETSYMBOL(&state1);
if (BIT_reloadDStream(&bitD)==BIT_DStream_overflow) {
*op++ = FSE_GETSYMBOL(&state2);
break;
}
if (op>(omax-2)) return ERROR(dstSize_tooSmall);
*op++ = FSE_GETSYMBOL(&state2);
if (BIT_reloadDStream(&bitD)==BIT_DStream_overflow) {
*op++ = FSE_GETSYMBOL(&state1);
break;
} }
return op-ostart;
}
size_t FSE_decompress_usingDTable(void* dst, size_t originalSize,
const void* cSrc, size_t cSrcSize,
const FSE_DTable* dt)
{
const void* ptr = dt;
const FSE_DTableHeader* DTableH = (const FSE_DTableHeader*)ptr;
const U32 fastMode = DTableH->fastMode;
/* select fast mode (static) */
if (fastMode) return FSE_decompress_usingDTable_generic(dst, originalSize, cSrc, cSrcSize, dt, 1);
return FSE_decompress_usingDTable_generic(dst, originalSize, cSrc, cSrcSize, dt, 0);
}
size_t FSE_decompress(void* dst, size_t maxDstSize, const void* cSrc, size_t cSrcSize)
{
const BYTE* const istart = (const BYTE*)cSrc;
const BYTE* ip = istart;
short counting[FSE_MAX_SYMBOL_VALUE+1];
DTable_max_t dt; /* Static analyzer seems unable to understand this table will be properly initialized later */
unsigned tableLog;
unsigned maxSymbolValue = FSE_MAX_SYMBOL_VALUE;
size_t errorCode;
if (cSrcSize<2) return ERROR(srcSize_wrong); /* too small input size */
/* normal FSE decoding mode */
errorCode = FSE_readNCount (counting, &maxSymbolValue, &tableLog, istart, cSrcSize);
if (FSE_isError(errorCode)) return errorCode;
if (errorCode >= cSrcSize) return ERROR(srcSize_wrong); /* too small input size */
ip += errorCode;
cSrcSize -= errorCode;
errorCode = FSE_buildDTable (dt, counting, maxSymbolValue, tableLog);
if (FSE_isError(errorCode)) return errorCode;
/* always return, even if it is an error code */
return FSE_decompress_usingDTable (dst, maxDstSize, ip, cSrcSize, dt);
}
#endif /* FSE_COMMONDEFS_ONLY */
+560
View File
@@ -0,0 +1,560 @@
/* ******************************************************************
Huffman encoder, part of New Generation Entropy library
Copyright (C) 2013-2016, Yann Collet.
BSD 2-Clause License (http://www.opensource.org/licenses/bsd-license.php)
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the following disclaimer
in the documentation and/or other materials provided with the
distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
You can contact the author at :
- FSE+HUF source repository : https://github.com/Cyan4973/FiniteStateEntropy
- Public forum : https://groups.google.com/forum/#!forum/lz4c
****************************************************************** */
/* **************************************************************
* Compiler specifics
****************************************************************/
#if defined (__cplusplus) || (defined (__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L) /* C99 */)
/* inline is defined */
#elif defined(_MSC_VER)
# define inline __inline
#else
# define inline /* disable inline */
#endif
#ifdef _MSC_VER /* Visual Studio */
# define FORCE_INLINE static __forceinline
# pragma warning(disable : 4127) /* disable: C4127: conditional expression is constant */
#else
# ifdef __GNUC__
# define FORCE_INLINE static inline __attribute__((always_inline))
# else
# define FORCE_INLINE static inline
# endif
#endif
/* **************************************************************
* Includes
****************************************************************/
#include <stdlib.h> /* malloc, free, qsort */
#include <string.h> /* memcpy, memset */
#include <stdio.h> /* printf (debug) */
#include "huf_static.h"
#include "bitstream.h"
#include "fse.h" /* header compression */
/* **************************************************************
* Error Management
****************************************************************/
#define HUF_STATIC_ASSERT(c) { enum { HUF_static_assert = 1/(int)(!!(c)) }; } /* use only *after* variable declarations */
/* *******************************************************
* HUF : Huffman block compression
*********************************************************/
struct HUF_CElt_s {
U16 val;
BYTE nbBits;
}; /* typedef'd to HUF_CElt within huf_static.h */
typedef struct nodeElt_s {
U32 count;
U16 parent;
BYTE byte;
BYTE nbBits;
} nodeElt;
/*! HUF_writeCTable() :
`CTable` : huffman tree to save, using huf representation.
@return : size of saved CTable */
size_t HUF_writeCTable (void* dst, size_t maxDstSize,
const HUF_CElt* CTable, U32 maxSymbolValue, U32 huffLog)
{
BYTE bitsToWeight[HUF_MAX_TABLELOG + 1];
BYTE huffWeight[HUF_MAX_SYMBOL_VALUE + 1];
U32 n;
BYTE* op = (BYTE*)dst;
size_t size;
/* check conditions */
if (maxSymbolValue > HUF_MAX_SYMBOL_VALUE + 1)
return ERROR(GENERIC);
/* convert to weight */
bitsToWeight[0] = 0;
for (n=1; n<=huffLog; n++)
bitsToWeight[n] = (BYTE)(huffLog + 1 - n);
for (n=0; n<maxSymbolValue; n++)
huffWeight[n] = bitsToWeight[CTable[n].nbBits];
size = FSE_compress(op+1, maxDstSize-1, huffWeight, maxSymbolValue); /* don't need last symbol stat : implied */
if (HUF_isError(size)) return size;
if (size >= 128) return ERROR(GENERIC); /* should never happen, since maxSymbolValue <= 255 */
if ((size <= 1) || (size >= maxSymbolValue/2)) {
if (size==1) { /* RLE */
/* only possible case : serie of 1 (because there are at least 2) */
/* can only be 2^n or (2^n-1), otherwise not an huffman tree */
BYTE code;
switch(maxSymbolValue)
{
case 1: code = 0; break;
case 2: code = 1; break;
case 3: code = 2; break;
case 4: code = 3; break;
case 7: code = 4; break;
case 8: code = 5; break;
case 15: code = 6; break;
case 16: code = 7; break;
case 31: code = 8; break;
case 32: code = 9; break;
case 63: code = 10; break;
case 64: code = 11; break;
case 127: code = 12; break;
case 128: code = 13; break;
default : return ERROR(corruption_detected);
}
op[0] = (BYTE)(255-13 + code);
return 1;
}
/* Not compressible */
if (maxSymbolValue > (241-128)) return ERROR(GENERIC); /* not implemented (not possible with current format) */
if (((maxSymbolValue+1)/2) + 1 > maxDstSize) return ERROR(dstSize_tooSmall); /* not enough space within dst buffer */
op[0] = (BYTE)(128 /*special case*/ + 0 /* Not Compressible */ + (maxSymbolValue-1));
huffWeight[maxSymbolValue] = 0; /* to be sure it doesn't cause issue in final combination */
for (n=0; n<maxSymbolValue; n+=2)
op[(n/2)+1] = (BYTE)((huffWeight[n] << 4) + huffWeight[n+1]);
return ((maxSymbolValue+1)/2) + 1;
}
/* normal header case */
op[0] = (BYTE)size;
return size+1;
}
size_t HUF_readCTable (HUF_CElt* CTable, U32 maxSymbolValue, const void* src, size_t srcSize)
{
BYTE huffWeight[HUF_MAX_SYMBOL_VALUE + 1];
U32 rankVal[HUF_ABSOLUTEMAX_TABLELOG + 1]; /* large enough for values from 0 to 16 */
U32 tableLog = 0;
size_t readSize;
U32 nbSymbols = 0;
//memset(huffWeight, 0, sizeof(huffWeight)); /* is not necessary, even though some analyzer complain ... */
/* get symbol weights */
readSize = HUF_readStats(huffWeight, HUF_MAX_SYMBOL_VALUE+1, rankVal, &nbSymbols, &tableLog, src, srcSize);
if (HUF_isError(readSize)) return readSize;
/* check result */
if (tableLog > HUF_MAX_TABLELOG) return ERROR(tableLog_tooLarge);
if (nbSymbols > maxSymbolValue+1) return ERROR(maxSymbolValue_tooSmall);
/* Prepare base value per rank */
{ U32 n, nextRankStart = 0;
for (n=1; n<=tableLog; n++) {
U32 current = nextRankStart;
nextRankStart += (rankVal[n] << (n-1));
rankVal[n] = current;
} }
/* fill nbBits */
{ U32 n; for (n=0; n<nbSymbols; n++) {
const U32 w = huffWeight[n];
CTable[n].nbBits = (BYTE)(tableLog + 1 - w);
}}
/* fill val */
{ U16 nbPerRank[HUF_MAX_TABLELOG+1] = {0};
U16 valPerRank[HUF_MAX_TABLELOG+1] = {0};
{ U32 n; for (n=0; n<nbSymbols; n++) nbPerRank[CTable[n].nbBits]++; }
/* determine stating value per rank */
{ U16 min = 0;
U32 n; for (n=HUF_MAX_TABLELOG; n>0; n--) {
valPerRank[n] = min; /* get starting value within each rank */
min += nbPerRank[n];
min >>= 1;
} }
/* assign value within rank, symbol order */
{ U32 n; for (n=0; n<=maxSymbolValue; n++) CTable[n].val = valPerRank[CTable[n].nbBits]++; }
}
return readSize;
}
static U32 HUF_setMaxHeight(nodeElt* huffNode, U32 lastNonNull, U32 maxNbBits)
{
const U32 largestBits = huffNode[lastNonNull].nbBits;
if (largestBits <= maxNbBits) return largestBits; /* early exit : no elt > maxNbBits */
/* there are several too large elements (at least >= 2) */
{ int totalCost = 0;
const U32 baseCost = 1 << (largestBits - maxNbBits);
U32 n = lastNonNull;
while (huffNode[n].nbBits > maxNbBits) {
totalCost += baseCost - (1 << (largestBits - huffNode[n].nbBits));
huffNode[n].nbBits = (BYTE)maxNbBits;
n --;
} /* n stops at huffNode[n].nbBits <= maxNbBits */
while (huffNode[n].nbBits == maxNbBits) n--; /* n end at index of smallest symbol using < maxNbBits */
/* renorm totalCost */
totalCost >>= (largestBits - maxNbBits); /* note : totalCost is necessarily a multiple of baseCost */
/* repay normalized cost */
{ U32 const noSymbol = 0xF0F0F0F0;
U32 rankLast[HUF_MAX_TABLELOG+1];
int pos;
/* Get pos of last (smallest) symbol per rank */
memset(rankLast, 0xF0, sizeof(rankLast));
{ U32 currentNbBits = maxNbBits;
for (pos=n ; pos >= 0; pos--) {
if (huffNode[pos].nbBits >= currentNbBits) continue;
currentNbBits = huffNode[pos].nbBits; /* < maxNbBits */
rankLast[maxNbBits-currentNbBits] = pos;
} }
while (totalCost > 0) {
U32 nBitsToDecrease = BIT_highbit32(totalCost) + 1;
for ( ; nBitsToDecrease > 1; nBitsToDecrease--) {
U32 highPos = rankLast[nBitsToDecrease];
U32 lowPos = rankLast[nBitsToDecrease-1];
if (highPos == noSymbol) continue;
if (lowPos == noSymbol) break;
{ U32 const highTotal = huffNode[highPos].count;
U32 const lowTotal = 2 * huffNode[lowPos].count;
if (highTotal <= lowTotal) break;
} }
/* only triggered when no more rank 1 symbol left => find closest one (note : there is necessarily at least one !) */
while ((nBitsToDecrease<=HUF_MAX_TABLELOG) && (rankLast[nBitsToDecrease] == noSymbol)) /* HUF_MAX_TABLELOG test just to please gcc 5+; but it should not be necessary */
nBitsToDecrease ++;
totalCost -= 1 << (nBitsToDecrease-1);
if (rankLast[nBitsToDecrease-1] == noSymbol)
rankLast[nBitsToDecrease-1] = rankLast[nBitsToDecrease]; /* this rank is no longer empty */
huffNode[rankLast[nBitsToDecrease]].nbBits ++;
if (rankLast[nBitsToDecrease] == 0) /* special case, reached largest symbol */
rankLast[nBitsToDecrease] = noSymbol;
else {
rankLast[nBitsToDecrease]--;
if (huffNode[rankLast[nBitsToDecrease]].nbBits != maxNbBits-nBitsToDecrease)
rankLast[nBitsToDecrease] = noSymbol; /* this rank is now empty */
} } /* while (totalCost > 0) */
while (totalCost < 0) { /* Sometimes, cost correction overshoot */
if (rankLast[1] == noSymbol) { /* special case : no rank 1 symbol (using maxNbBits-1); let's create one from largest rank 0 (using maxNbBits) */
while (huffNode[n].nbBits == maxNbBits) n--;
huffNode[n+1].nbBits--;
rankLast[1] = n+1;
totalCost++;
continue;
}
huffNode[ rankLast[1] + 1 ].nbBits--;
rankLast[1]++;
totalCost ++;
} } } /* there are several too large elements (at least >= 2) */
return maxNbBits;
}
typedef struct {
U32 base;
U32 current;
} rankPos;
static void HUF_sort(nodeElt* huffNode, const U32* count, U32 maxSymbolValue)
{
rankPos rank[32];
U32 n;
memset(rank, 0, sizeof(rank));
for (n=0; n<=maxSymbolValue; n++) {
U32 r = BIT_highbit32(count[n] + 1);
rank[r].base ++;
}
for (n=30; n>0; n--) rank[n-1].base += rank[n].base;
for (n=0; n<32; n++) rank[n].current = rank[n].base;
for (n=0; n<=maxSymbolValue; n++) {
U32 const c = count[n];
U32 const r = BIT_highbit32(c+1) + 1;
U32 pos = rank[r].current++;
while ((pos > rank[r].base) && (c > huffNode[pos-1].count)) huffNode[pos]=huffNode[pos-1], pos--;
huffNode[pos].count = c;
huffNode[pos].byte = (BYTE)n;
}
}
#define STARTNODE (HUF_MAX_SYMBOL_VALUE+1)
size_t HUF_buildCTable (HUF_CElt* tree, const U32* count, U32 maxSymbolValue, U32 maxNbBits)
{
nodeElt huffNode0[2*HUF_MAX_SYMBOL_VALUE+1 +1];
nodeElt* huffNode = huffNode0 + 1;
U32 n, nonNullRank;
int lowS, lowN;
U16 nodeNb = STARTNODE;
U32 nodeRoot;
/* safety checks */
if (maxNbBits == 0) maxNbBits = HUF_DEFAULT_TABLELOG;
if (maxSymbolValue > HUF_MAX_SYMBOL_VALUE) return ERROR(GENERIC);
memset(huffNode0, 0, sizeof(huffNode0));
/* sort, decreasing order */
HUF_sort(huffNode, count, maxSymbolValue);
/* init for parents */
nonNullRank = maxSymbolValue;
while(huffNode[nonNullRank].count == 0) nonNullRank--;
lowS = nonNullRank; nodeRoot = nodeNb + lowS - 1; lowN = nodeNb;
huffNode[nodeNb].count = huffNode[lowS].count + huffNode[lowS-1].count;
huffNode[lowS].parent = huffNode[lowS-1].parent = nodeNb;
nodeNb++; lowS-=2;
for (n=nodeNb; n<=nodeRoot; n++) huffNode[n].count = (U32)(1U<<30);
huffNode0[0].count = (U32)(1U<<31);
/* create parents */
while (nodeNb <= nodeRoot) {
U32 n1 = (huffNode[lowS].count < huffNode[lowN].count) ? lowS-- : lowN++;
U32 n2 = (huffNode[lowS].count < huffNode[lowN].count) ? lowS-- : lowN++;
huffNode[nodeNb].count = huffNode[n1].count + huffNode[n2].count;
huffNode[n1].parent = huffNode[n2].parent = nodeNb;
nodeNb++;
}
/* distribute weights (unlimited tree height) */
huffNode[nodeRoot].nbBits = 0;
for (n=nodeRoot-1; n>=STARTNODE; n--)
huffNode[n].nbBits = huffNode[ huffNode[n].parent ].nbBits + 1;
for (n=0; n<=nonNullRank; n++)
huffNode[n].nbBits = huffNode[ huffNode[n].parent ].nbBits + 1;
/* enforce maxTableLog */
maxNbBits = HUF_setMaxHeight(huffNode, nonNullRank, maxNbBits);
/* fill result into tree (val, nbBits) */
{ U16 nbPerRank[HUF_MAX_TABLELOG+1] = {0};
U16 valPerRank[HUF_MAX_TABLELOG+1] = {0};
if (maxNbBits > HUF_MAX_TABLELOG) return ERROR(GENERIC); /* check fit into table */
for (n=0; n<=nonNullRank; n++)
nbPerRank[huffNode[n].nbBits]++;
/* determine stating value per rank */
{ U16 min = 0;
for (n=maxNbBits; n>0; n--) {
valPerRank[n] = min; /* get starting value within each rank */
min += nbPerRank[n];
min >>= 1;
} }
for (n=0; n<=maxSymbolValue; n++)
tree[huffNode[n].byte].nbBits = huffNode[n].nbBits; /* push nbBits per symbol, symbol order */
for (n=0; n<=maxSymbolValue; n++)
tree[n].val = valPerRank[tree[n].nbBits]++; /* assign value within rank, symbol order */
}
return maxNbBits;
}
static void HUF_encodeSymbol(BIT_CStream_t* bitCPtr, U32 symbol, const HUF_CElt* CTable)
{
BIT_addBitsFast(bitCPtr, CTable[symbol].val, CTable[symbol].nbBits);
}
size_t HUF_compressBound(size_t size) { return HUF_COMPRESSBOUND(size); }
#define HUF_FLUSHBITS(s) (fast ? BIT_flushBitsFast(s) : BIT_flushBits(s))
#define HUF_FLUSHBITS_1(stream) \
if (sizeof((stream)->bitContainer)*8 < HUF_MAX_TABLELOG*2+7) HUF_FLUSHBITS(stream)
#define HUF_FLUSHBITS_2(stream) \
if (sizeof((stream)->bitContainer)*8 < HUF_MAX_TABLELOG*4+7) HUF_FLUSHBITS(stream)
size_t HUF_compress1X_usingCTable(void* dst, size_t dstSize, const void* src, size_t srcSize, const HUF_CElt* CTable)
{
const BYTE* ip = (const BYTE*) src;
BYTE* const ostart = (BYTE*)dst;
BYTE* const oend = ostart + dstSize;
BYTE* op = ostart;
size_t n;
const unsigned fast = (dstSize >= HUF_BLOCKBOUND(srcSize));
BIT_CStream_t bitC;
/* init */
if (dstSize < 8) return 0; /* not enough space to compress */
{ size_t const errorCode = BIT_initCStream(&bitC, op, oend-op);
if (HUF_isError(errorCode)) return 0; }
n = srcSize & ~3; /* join to mod 4 */
switch (srcSize & 3)
{
case 3 : HUF_encodeSymbol(&bitC, ip[n+ 2], CTable);
HUF_FLUSHBITS_2(&bitC);
case 2 : HUF_encodeSymbol(&bitC, ip[n+ 1], CTable);
HUF_FLUSHBITS_1(&bitC);
case 1 : HUF_encodeSymbol(&bitC, ip[n+ 0], CTable);
HUF_FLUSHBITS(&bitC);
case 0 :
default: ;
}
for (; n>0; n-=4) { /* note : n&3==0 at this stage */
HUF_encodeSymbol(&bitC, ip[n- 1], CTable);
HUF_FLUSHBITS_1(&bitC);
HUF_encodeSymbol(&bitC, ip[n- 2], CTable);
HUF_FLUSHBITS_2(&bitC);
HUF_encodeSymbol(&bitC, ip[n- 3], CTable);
HUF_FLUSHBITS_1(&bitC);
HUF_encodeSymbol(&bitC, ip[n- 4], CTable);
HUF_FLUSHBITS(&bitC);
}
return BIT_closeCStream(&bitC);
}
size_t HUF_compress4X_usingCTable(void* dst, size_t dstSize, const void* src, size_t srcSize, const HUF_CElt* CTable)
{
size_t segmentSize = (srcSize+3)/4; /* first 3 segments */
const BYTE* ip = (const BYTE*) src;
const BYTE* const iend = ip + srcSize;
BYTE* const ostart = (BYTE*) dst;
BYTE* const oend = ostart + dstSize;
BYTE* op = ostart;
size_t errorCode;
if (dstSize < 6 + 1 + 1 + 1 + 8) return 0; /* minimum space to compress successfully */
if (srcSize < 12) return 0; /* no saving possible : too small input */
op += 6; /* jumpTable */
errorCode = HUF_compress1X_usingCTable(op, oend-op, ip, segmentSize, CTable);
if (HUF_isError(errorCode)) return errorCode;
if (errorCode==0) return 0;
MEM_writeLE16(ostart, (U16)errorCode);
ip += segmentSize;
op += errorCode;
errorCode = HUF_compress1X_usingCTable(op, oend-op, ip, segmentSize, CTable);
if (HUF_isError(errorCode)) return errorCode;
if (errorCode==0) return 0;
MEM_writeLE16(ostart+2, (U16)errorCode);
ip += segmentSize;
op += errorCode;
errorCode = HUF_compress1X_usingCTable(op, oend-op, ip, segmentSize, CTable);
if (HUF_isError(errorCode)) return errorCode;
if (errorCode==0) return 0;
MEM_writeLE16(ostart+4, (U16)errorCode);
ip += segmentSize;
op += errorCode;
errorCode = HUF_compress1X_usingCTable(op, oend-op, ip, iend-ip, CTable);
if (HUF_isError(errorCode)) return errorCode;
if (errorCode==0) return 0;
op += errorCode;
return op-ostart;
}
static size_t HUF_compress_internal (
void* dst, size_t dstSize,
const void* src, size_t srcSize,
unsigned maxSymbolValue, unsigned huffLog,
unsigned singleStream)
{
BYTE* const ostart = (BYTE*)dst;
BYTE* const oend = ostart + dstSize;
BYTE* op = ostart;
U32 count[HUF_MAX_SYMBOL_VALUE+1];
HUF_CElt CTable[HUF_MAX_SYMBOL_VALUE+1];
size_t errorCode;
/* checks & inits */
if (srcSize < 1) return 0; /* Uncompressed - note : 1 means rle, so first byte must be correct */
if (dstSize < 1) return 0; /* not compressible within dst budget */
if (srcSize > 128 * 1024) return ERROR(srcSize_wrong); /* current block size limit */
if (huffLog > HUF_MAX_TABLELOG) return ERROR(tableLog_tooLarge);
if (!maxSymbolValue) maxSymbolValue = HUF_MAX_SYMBOL_VALUE;
if (!huffLog) huffLog = HUF_DEFAULT_TABLELOG;
/* Scan input and build symbol stats */
errorCode = FSE_count (count, &maxSymbolValue, (const BYTE*)src, srcSize);
if (HUF_isError(errorCode)) return errorCode;
if (errorCode == srcSize) { *ostart = ((const BYTE*)src)[0]; return 1; }
if (errorCode <= (srcSize >> 7)+1) return 0; /* Heuristic : not compressible enough */
/* Build Huffman Tree */
errorCode = HUF_buildCTable (CTable, count, maxSymbolValue, huffLog);
if (HUF_isError(errorCode)) return errorCode;
huffLog = (U32)errorCode;
/* Write table description header */
errorCode = HUF_writeCTable (op, dstSize, CTable, maxSymbolValue, huffLog);
if (HUF_isError(errorCode)) return errorCode;
if (errorCode + 12 >= srcSize) return 0; /* not useful to try compression */
op += errorCode;
/* Compress */
if (singleStream)
errorCode = HUF_compress1X_usingCTable(op, oend - op, src, srcSize, CTable); /* single segment */
else
errorCode = HUF_compress4X_usingCTable(op, oend - op, src, srcSize, CTable);
if (HUF_isError(errorCode)) return errorCode;
if (errorCode==0) return 0;
op += errorCode;
/* check compressibility */
if ((size_t)(op-ostart) >= srcSize-1)
return 0;
return op-ostart;
}
size_t HUF_compress1X (void* dst, size_t dstSize,
const void* src, size_t srcSize,
unsigned maxSymbolValue, unsigned huffLog)
{
return HUF_compress_internal(dst, dstSize, src, srcSize, maxSymbolValue, huffLog, 1);
}
size_t HUF_compress2 (void* dst, size_t dstSize,
const void* src, size_t srcSize,
unsigned maxSymbolValue, unsigned huffLog)
{
return HUF_compress_internal(dst, dstSize, src, srcSize, maxSymbolValue, huffLog, 0);
}
size_t HUF_compress (void* dst, size_t maxDstSize, const void* src, size_t srcSize)
{
return HUF_compress2(dst, maxDstSize, src, (U32)srcSize, 255, HUF_DEFAULT_TABLELOG);
}
@@ -169,13 +169,6 @@ size_t ZBUFF_compressInit(ZBUFF_CCtx* zbc, int compressionLevel)
/* *** Compression *** */
static size_t ZBUFF_limitCopy(void* dst, size_t dstCapacity, const void* src, size_t srcSize)
{
size_t length = MIN(dstCapacity, srcSize);
memcpy(dst, src, length);
return length;
}
static size_t ZBUFF_compressContinue_generic(ZBUFF_CCtx* zbc,
void* dst, size_t* dstCapacityPtr,
const void* src, size_t* srcSizePtr,
@@ -290,215 +283,9 @@ size_t ZBUFF_compressEnd(ZBUFF_CCtx* zbc, void* dst, size_t* dstCapacityPtr)
}
/*-***************************************************************************
* Streaming decompression howto
*
* A ZBUFF_DCtx object is required to track streaming operations.
* Use ZBUFF_createDCtx() and ZBUFF_freeDCtx() to create/release resources.
* Use ZBUFF_decompressInit() to start a new decompression operation,
* or ZBUFF_decompressInitDictionary() if decompression requires a dictionary.
* Note that ZBUFF_DCtx objects can be re-init multiple times.
*
* Use ZBUFF_decompressContinue() repetitively to consume your input.
* *srcSizePtr and *dstCapacityPtr can be any size.
* The function will report how many bytes were read or written by modifying *srcSizePtr and *dstCapacityPtr.
* Note that it may not consume the entire input, in which case it's up to the caller to present remaining input again.
* The content of @dst will be overwritten (up to *dstCapacityPtr) at each function call, so save its content if it matters, or change @dst.
* @return : a hint to preferred nb of bytes to use as input for next function call (it's only a hint, to help latency),
* or 0 when a frame is completely decoded,
* or an error code, which can be tested using ZBUFF_isError().
*
* Hint : recommended buffer sizes (not compulsory) : ZBUFF_recommendedDInSize() and ZBUFF_recommendedDOutSize()
* output : ZBUFF_recommendedDOutSize==128 KB block size is the internal unit, it ensures it's always possible to write a full block when decoded.
* input : ZBUFF_recommendedDInSize == 128KB + 3;
* just follow indications from ZBUFF_decompressContinue() to minimize latency. It should always be <= 128 KB + 3 .
* *******************************************************************************/
typedef enum { ZBUFFds_init, ZBUFFds_readHeader,
ZBUFFds_read, ZBUFFds_load, ZBUFFds_flush } ZBUFF_dStage;
/* *** Resource management *** */
struct ZBUFF_DCtx_s {
ZSTD_DCtx* zd;
ZSTD_frameParams fParams;
size_t blockSize;
char* inBuff;
size_t inBuffSize;
size_t inPos;
char* outBuff;
size_t outBuffSize;
size_t outStart;
size_t outEnd;
ZBUFF_dStage stage;
}; /* typedef'd to ZBUFF_DCtx within "zstd_buffered.h" */
ZBUFF_DCtx* ZBUFF_createDCtx(void)
{
ZBUFF_DCtx* zbd = (ZBUFF_DCtx*)malloc(sizeof(ZBUFF_DCtx));
if (zbd==NULL) return NULL;
memset(zbd, 0, sizeof(*zbd));
zbd->zd = ZSTD_createDCtx();
zbd->stage = ZBUFFds_init;
return zbd;
}
size_t ZBUFF_freeDCtx(ZBUFF_DCtx* zbd)
{
if (zbd==NULL) return 0; /* support free on null */
ZSTD_freeDCtx(zbd->zd);
free(zbd->inBuff);
free(zbd->outBuff);
free(zbd);
return 0;
}
/* *** Initialization *** */
size_t ZBUFF_decompressInitDictionary(ZBUFF_DCtx* zbd, const void* dict, size_t dictSize)
{
zbd->stage = ZBUFFds_readHeader;
zbd->inPos = zbd->outStart = zbd->outEnd = 0;
return ZSTD_decompressBegin_usingDict(zbd->zd, dict, dictSize);
}
size_t ZBUFF_decompressInit(ZBUFF_DCtx* zbd)
{
return ZBUFF_decompressInitDictionary(zbd, NULL, 0);
}
/* *** Decompression *** */
size_t ZBUFF_decompressContinue(ZBUFF_DCtx* zbd,
void* dst, size_t* dstCapacityPtr,
const void* src, size_t* srcSizePtr)
{
const char* const istart = (const char*)src;
const char* const iend = istart + *srcSizePtr;
const char* ip = istart;
char* const ostart = (char*)dst;
char* const oend = ostart + *dstCapacityPtr;
char* op = ostart;
U32 notDone = 1;
while (notDone) {
switch(zbd->stage)
{
case ZBUFFds_init :
return ERROR(init_missing);
case ZBUFFds_readHeader :
/* read header from src */
{ size_t const headerSize = ZSTD_getFrameParams(&(zbd->fParams), src, *srcSizePtr);
if (ZSTD_isError(headerSize)) return headerSize;
if (headerSize) {
/* not enough input to decode header : needs headerSize > *srcSizePtr */
*dstCapacityPtr = 0;
*srcSizePtr = 0;
return headerSize;
} }
/* Frame header instruct buffer sizes */
{ size_t const blockSize = MIN(1 << zbd->fParams.windowLog, ZSTD_BLOCKSIZE_MAX);
zbd->blockSize = blockSize;
if (zbd->inBuffSize < blockSize) {
free(zbd->inBuff);
zbd->inBuffSize = blockSize;
zbd->inBuff = (char*)malloc(blockSize);
if (zbd->inBuff == NULL) return ERROR(memory_allocation);
}
{ size_t const neededOutSize = ((size_t)1 << zbd->fParams.windowLog) + blockSize;
if (zbd->outBuffSize < neededOutSize) {
free(zbd->outBuff);
zbd->outBuffSize = neededOutSize;
zbd->outBuff = (char*)malloc(neededOutSize);
if (zbd->outBuff == NULL) return ERROR(memory_allocation);
} } }
zbd->stage = ZBUFFds_read;
case ZBUFFds_read:
{ size_t const neededInSize = ZSTD_nextSrcSizeToDecompress(zbd->zd);
if (neededInSize==0) { /* end of frame */
zbd->stage = ZBUFFds_init;
notDone = 0;
break;
}
if ((size_t)(iend-ip) >= neededInSize) {
/* directly decode from src */
size_t const decodedSize = ZSTD_decompressContinue(zbd->zd,
zbd->outBuff + zbd->outStart, zbd->outBuffSize - zbd->outStart,
ip, neededInSize);
if (ZSTD_isError(decodedSize)) return decodedSize;
ip += neededInSize;
if (!decodedSize) break; /* this was just a header */
zbd->outEnd = zbd->outStart + decodedSize;
zbd->stage = ZBUFFds_flush;
break;
}
if (ip==iend) { notDone = 0; break; } /* no more input */
zbd->stage = ZBUFFds_load;
}
case ZBUFFds_load:
{ size_t const neededInSize = ZSTD_nextSrcSizeToDecompress(zbd->zd);
size_t const toLoad = neededInSize - zbd->inPos; /* should always be <= remaining space within inBuff */
size_t loadedSize;
if (toLoad > zbd->inBuffSize - zbd->inPos) return ERROR(corruption_detected); /* should never happen */
loadedSize = ZBUFF_limitCopy(zbd->inBuff + zbd->inPos, toLoad, ip, iend-ip);
ip += loadedSize;
zbd->inPos += loadedSize;
if (loadedSize < toLoad) { notDone = 0; break; } /* not enough input, wait for more */
/* decode loaded input */
{ size_t const decodedSize = ZSTD_decompressContinue(zbd->zd,
zbd->outBuff + zbd->outStart, zbd->outBuffSize - zbd->outStart,
zbd->inBuff, neededInSize);
if (ZSTD_isError(decodedSize)) return decodedSize;
zbd->inPos = 0; /* input is consumed */
if (!decodedSize) { zbd->stage = ZBUFFds_read; break; } /* this was just a header */
zbd->outEnd = zbd->outStart + decodedSize;
zbd->stage = ZBUFFds_flush;
// break; /* ZBUFFds_flush follows */
} }
case ZBUFFds_flush:
{ size_t const toFlushSize = zbd->outEnd - zbd->outStart;
size_t const flushedSize = ZBUFF_limitCopy(op, oend-op, zbd->outBuff + zbd->outStart, toFlushSize);
op += flushedSize;
zbd->outStart += flushedSize;
if (flushedSize == toFlushSize) {
zbd->stage = ZBUFFds_read;
if (zbd->outStart + zbd->blockSize > zbd->outBuffSize)
zbd->outStart = zbd->outEnd = 0;
break;
}
/* cannot flush everything */
notDone = 0;
break;
}
default: return ERROR(GENERIC); /* impossible */
} }
/* result */
*srcSizePtr = ip-istart;
*dstCapacityPtr = op-ostart;
{ size_t nextSrcSizeHint = ZSTD_nextSrcSizeToDecompress(zbd->zd);
if (nextSrcSizeHint > ZSTD_blockHeaderSize) nextSrcSizeHint+= ZSTD_blockHeaderSize; /* get following block header too */
nextSrcSizeHint -= zbd->inPos; /* already loaded*/
return nextSrcSizeHint;
}
}
/* *************************************
* Tool functions
***************************************/
unsigned ZBUFF_isError(size_t errorCode) { return ERR_isError(errorCode); }
const char* ZBUFF_getErrorName(size_t errorCode) { return ERR_getErrorName(errorCode); }
size_t ZBUFF_recommendedCInSize(void) { return ZSTD_BLOCKSIZE_MAX; }
size_t ZBUFF_recommendedCOutSize(void) { return ZSTD_compressBound(ZSTD_BLOCKSIZE_MAX) + ZSTD_blockHeaderSize + ZBUFF_endFrameSize; }
size_t ZBUFF_recommendedDInSize(void) { return ZSTD_BLOCKSIZE_MAX + ZSTD_blockHeaderSize /* block header size*/ ; }
size_t ZBUFF_recommendedDOutSize(void) { return ZSTD_BLOCKSIZE_MAX; }
@@ -55,7 +55,7 @@
#include <string.h> /* memset */
#include "mem.h"
#include "fse_static.h"
#include "huff0_static.h"
#include "huf_static.h"
#include "zstd_internal.h"
@@ -218,7 +218,7 @@ static size_t ZSTD_resetCCtx_advanced (ZSTD_CCtx* zc,
const size_t maxNbSeq = blockSize / divider;
const size_t tokenSpace = blockSize + 11*maxNbSeq;
const size_t chainSize = (params.cParams.strategy == ZSTD_fast) ? 0 : (1 << params.cParams.chainLog);
const size_t hSize = 1 << params.cParams.hashLog;
const size_t hSize = ((size_t)1) << params.cParams.hashLog;
const size_t h3Size = (zc->hashLog3) ? 1 << zc->hashLog3 : 0;
const size_t tableSpace = (chainSize + hSize + h3Size) * sizeof(U32);
@@ -291,7 +291,7 @@ size_t ZSTD_copyCCtx(ZSTD_CCtx* dstCCtx, const ZSTD_CCtx* srcCCtx)
/* copy tables */
{ const size_t chainSize = (srcCCtx->params.cParams.strategy == ZSTD_fast) ? 0 : (1 << srcCCtx->params.cParams.chainLog);
const size_t hSize = 1 << srcCCtx->params.cParams.hashLog;
const size_t hSize = ((size_t)1) << srcCCtx->params.cParams.hashLog;
const size_t h3Size = (srcCCtx->hashLog3) ? 1 << srcCCtx->hashLog3 : 0;
const size_t tableSpace = (chainSize + hSize + h3Size) * sizeof(U32);
memcpy(dstCCtx->workSpace, srcCCtx->workSpace, tableSpace);
@@ -852,7 +852,7 @@ MEM_STATIC void ZSTD_storeSeq(seqStore_t* seqStorePtr, size_t litLength, const B
static const BYTE* g_start = NULL;
const U32 pos = (U32)(literals - g_start);
if (g_start==NULL) g_start = literals;
if ((pos > 5810300) && (pos < 5810500))
if ((pos > 2587900) && (pos < 2588050))
printf("Cpos %6u :%5u literals & match %3u bytes at distance %6u \n",
pos, (U32)litLength, (U32)matchCode+MINMATCH, (U32)offsetCode);
#endif
@@ -1619,9 +1619,6 @@ FORCE_INLINE size_t ZSTD_HcFindBestMatch_extDict_selectMLS (
}
}
/* The optimal parser */
#include "zstd_opt.h"
/* *******************************
* Common parser - lazy strategy
@@ -1756,12 +1753,6 @@ _storeSequence:
}
static void ZSTD_compressBlock_btopt(ZSTD_CCtx* ctx, const void* src, size_t srcSize)
{
ZSTD_compressBlock_opt_generic(ctx, src, srcSize);
}
static void ZSTD_compressBlock_btlazy2(ZSTD_CCtx* ctx, const void* src, size_t srcSize)
{
ZSTD_compressBlock_lazy_generic(ctx, src, srcSize, 1, 2);
@@ -1975,6 +1966,16 @@ static void ZSTD_compressBlock_btlazy2_extDict(ZSTD_CCtx* ctx, const void* src,
ZSTD_compressBlock_lazy_extDict_generic(ctx, src, srcSize, 1, 2);
}
/* The optimal parser */
#include "zstd_opt.h"
static void ZSTD_compressBlock_btopt(ZSTD_CCtx* ctx, const void* src, size_t srcSize)
{
ZSTD_compressBlock_opt_generic(ctx, src, srcSize);
}
static void ZSTD_compressBlock_btopt_extDict(ZSTD_CCtx* ctx, const void* src, size_t srcSize)
{
ZSTD_compressBlock_opt_extDict_generic(ctx, src, srcSize);
@@ -2309,17 +2310,6 @@ size_t ZSTD_compressBegin_usingDict(ZSTD_CCtx* zc, const void* dict, size_t dict
}
size_t ZSTD_compressBegin_targetSrcSize(ZSTD_CCtx* zc, const void* dict, size_t dictSize, size_t targetSrcSize, int compressionLevel)
{
ZSTD_parameters params;
params.cParams = ZSTD_getCParams(compressionLevel, targetSrcSize, dictSize);
params.fParams.contentSizeFlag = 1;
ZSTD_adjustCParams(&params.cParams, targetSrcSize, dictSize);
ZSTD_LOG_BLOCK("%p: ZSTD_compressBegin_targetSrcSize compressionLevel=%d\n", zc->base, compressionLevel);
return ZSTD_compressBegin_internal(zc, dict, dictSize, params, targetSrcSize);
}
size_t ZSTD_compressBegin(ZSTD_CCtx* zc, int compressionLevel)
{
ZSTD_LOG_BLOCK("%p: ZSTD_compressBegin compressionLevel=%d\n", zc->base, compressionLevel);
@@ -2442,7 +2432,8 @@ size_t ZSTD_compress(void* dst, size_t dstCapacity, const void* src, size_t srcS
/*-===== Pre-defined compression levels =====-*/
#define ZSTD_MAX_CLEVEL 22
#define ZSTD_DEFAULT_CLEVEL 5
#define ZSTD_MAX_CLEVEL 22
unsigned ZSTD_maxCLevel(void) { return ZSTD_MAX_CLEVEL; }
static const ZSTD_compressionParameters ZSTD_defaultCParameters[4][ZSTD_MAX_CLEVEL+1] = {
@@ -2561,7 +2552,8 @@ ZSTD_compressionParameters ZSTD_getCParams(int compressionLevel, U64 srcSize, si
size_t const addedSize = srcSize ? 0 : 500;
U64 const rSize = srcSize+dictSize ? srcSize+dictSize+addedSize : (U64)-1;
U32 const tableID = (rSize <= 256 KB) + (rSize <= 128 KB) + (rSize <= 16 KB); /* intentional underflow for srcSizeHint == 0 */
if (compressionLevel<=0) compressionLevel = 1;
if (compressionLevel < 0) compressionLevel = ZSTD_DEFAULT_CLEVEL;
if (compressionLevel==0) compressionLevel = 1;
if (compressionLevel > ZSTD_MAX_CLEVEL) compressionLevel = ZSTD_MAX_CLEVEL;
cp = ZSTD_defaultCParameters[tableID][compressionLevel];
if (MEM_32bits()) { /* auto-correction, for 32-bits mode */
+86 -87
View File
@@ -53,6 +53,9 @@ MEM_STATIC void ZSTD_rescaleFreqs(seqStore_t* ssPtr)
{
unsigned u;
ssPtr->cachedLiterals = NULL;
ssPtr->cachedPrice = ssPtr->cachedLitLength = 0;
if (ssPtr->litLengthSum == 0) {
ssPtr->litSum = (2<<Litbits);
ssPtr->litLengthSum = MaxLL+1;
@@ -98,17 +101,33 @@ MEM_STATIC void ZSTD_rescaleFreqs(seqStore_t* ssPtr)
}
FORCE_INLINE U32 ZSTD_getLiteralPrice(seqStore_t* seqStorePtr, U32 litLength, const BYTE* literals)
FORCE_INLINE U32 ZSTD_getLiteralPrice(seqStore_t* ssPtr, U32 litLength, const BYTE* literals)
{
U32 price, u;
if (litLength == 0)
return seqStorePtr->log2litLengthSum - ZSTD_highbit(seqStorePtr->litLengthFreq[0]+1);
return ssPtr->log2litLengthSum - ZSTD_highbit(ssPtr->litLengthFreq[0]+1);
/* literals */
price = litLength * seqStorePtr->log2litSum;
for (u=0; u < litLength; u++)
price -= ZSTD_highbit(seqStorePtr->litFreq[literals[u]]+1);
if (ssPtr->cachedLiterals == literals) {
U32 additional = litLength - ssPtr->cachedLitLength;
const BYTE* literals2 = ssPtr->cachedLiterals + ssPtr->cachedLitLength;
price = ssPtr->cachedPrice + additional * ssPtr->log2litSum;
for (u=0; u < additional; u++)
price -= ZSTD_highbit(ssPtr->litFreq[literals2[u]]+1);
ssPtr->cachedPrice = price;
ssPtr->cachedLitLength = litLength;
} else {
price = litLength * ssPtr->log2litSum;
for (u=0; u < litLength; u++)
price -= ZSTD_highbit(ssPtr->litFreq[literals[u]]+1);
if (litLength >= 12) {
ssPtr->cachedLiterals = literals;
ssPtr->cachedPrice = price;
ssPtr->cachedLitLength = litLength;
}
}
/* literal Length */
{ static const BYTE LL_Code[64] = { 0, 1, 2, 3, 4, 5, 6, 7,
@@ -121,7 +140,7 @@ FORCE_INLINE U32 ZSTD_getLiteralPrice(seqStore_t* seqStorePtr, U32 litLength, co
24, 24, 24, 24, 24, 24, 24, 24 };
const BYTE LL_deltaCode = 19;
const BYTE llCode = (litLength>63) ? (BYTE)ZSTD_highbit(litLength) + LL_deltaCode : LL_Code[litLength];
price += LL_bits[llCode] + seqStorePtr->log2litLengthSum - ZSTD_highbit(seqStorePtr->litLengthFreq[llCode]+1);
price += LL_bits[llCode] + ssPtr->log2litLengthSum - ZSTD_highbit(ssPtr->litLengthFreq[llCode]+1);
}
return price;
@@ -177,9 +196,10 @@ MEM_STATIC void ZSTD_updatePrice(seqStore_t* seqStorePtr, U32 litLength, const B
}
/* match offset */
seqStorePtr->offCodeSum++;
BYTE offCode = (BYTE)ZSTD_highbit(offset+1);
seqStorePtr->offCodeFreq[offCode]++;
{ BYTE offCode = (BYTE)ZSTD_highbit(offset+1);
seqStorePtr->offCodeSum++;
seqStorePtr->offCodeFreq[offCode]++;
}
/* match Length */
{ static const BYTE ML_Code[128] = { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15,
@@ -241,7 +261,7 @@ static U32 ZSTD_insertBtAndGetAllMatches (
ZSTD_CCtx* zc,
const BYTE* const ip, const BYTE* const iLimit,
U32 nbCompares, const U32 mls,
U32 extDict, ZSTD_match_t* matches)
U32 extDict, ZSTD_match_t* matches, const U32 minMatchLen)
{
const BYTE* const base = zc->base;
const U32 current = (U32)(ip-base);
@@ -266,7 +286,7 @@ static U32 ZSTD_insertBtAndGetAllMatches (
U32 mnum = 0;
const U32 minMatch = (mls == 3) ? 3 : 4;
size_t bestLength = minMatch-1;
size_t bestLength = minMatchLen-1;
if (minMatch == 3) { /* HC3 match finder */
U32 const matchIndex3 = ZSTD_insertAndFindFirstIndexHash3 (zc, ip);
@@ -366,26 +386,26 @@ update:
static U32 ZSTD_BtGetAllMatches (
ZSTD_CCtx* zc,
const BYTE* const ip, const BYTE* const iLimit,
const U32 maxNbAttempts, const U32 mls, ZSTD_match_t* matches)
const U32 maxNbAttempts, const U32 mls, ZSTD_match_t* matches, const U32 minMatchLen)
{
if (ip < zc->base + zc->nextToUpdate) return 0; /* skipped area */
ZSTD_updateTree(zc, ip, iLimit, maxNbAttempts, mls);
return ZSTD_insertBtAndGetAllMatches(zc, ip, iLimit, maxNbAttempts, mls, 0, matches);
return ZSTD_insertBtAndGetAllMatches(zc, ip, iLimit, maxNbAttempts, mls, 0, matches, minMatchLen);
}
static U32 ZSTD_BtGetAllMatches_selectMLS (
ZSTD_CCtx* zc, /* Index table will be updated */
const BYTE* ip, const BYTE* const iHighLimit,
const U32 maxNbAttempts, const U32 matchLengthSearch, ZSTD_match_t* matches)
const U32 maxNbAttempts, const U32 matchLengthSearch, ZSTD_match_t* matches, const U32 minMatchLen)
{
switch(matchLengthSearch)
{
case 3 : return ZSTD_BtGetAllMatches(zc, ip, iHighLimit, maxNbAttempts, 3, matches);
case 3 : return ZSTD_BtGetAllMatches(zc, ip, iHighLimit, maxNbAttempts, 3, matches, minMatchLen);
default :
case 4 : return ZSTD_BtGetAllMatches(zc, ip, iHighLimit, maxNbAttempts, 4, matches);
case 5 : return ZSTD_BtGetAllMatches(zc, ip, iHighLimit, maxNbAttempts, 5, matches);
case 6 : return ZSTD_BtGetAllMatches(zc, ip, iHighLimit, maxNbAttempts, 6, matches);
case 4 : return ZSTD_BtGetAllMatches(zc, ip, iHighLimit, maxNbAttempts, 4, matches, minMatchLen);
case 5 : return ZSTD_BtGetAllMatches(zc, ip, iHighLimit, maxNbAttempts, 5, matches, minMatchLen);
case 6 : return ZSTD_BtGetAllMatches(zc, ip, iHighLimit, maxNbAttempts, 6, matches, minMatchLen);
}
}
@@ -393,26 +413,26 @@ static U32 ZSTD_BtGetAllMatches_selectMLS (
static U32 ZSTD_BtGetAllMatches_extDict (
ZSTD_CCtx* zc,
const BYTE* const ip, const BYTE* const iLimit,
const U32 maxNbAttempts, const U32 mls, ZSTD_match_t* matches)
const U32 maxNbAttempts, const U32 mls, ZSTD_match_t* matches, const U32 minMatchLen)
{
if (ip < zc->base + zc->nextToUpdate) return 0; /* skipped area */
ZSTD_updateTree_extDict(zc, ip, iLimit, maxNbAttempts, mls);
return ZSTD_insertBtAndGetAllMatches(zc, ip, iLimit, maxNbAttempts, mls, 1, matches);
return ZSTD_insertBtAndGetAllMatches(zc, ip, iLimit, maxNbAttempts, mls, 1, matches, minMatchLen);
}
static U32 ZSTD_BtGetAllMatches_selectMLS_extDict (
ZSTD_CCtx* zc, /* Index table will be updated */
const BYTE* ip, const BYTE* const iHighLimit,
const U32 maxNbAttempts, const U32 matchLengthSearch, ZSTD_match_t* matches)
const U32 maxNbAttempts, const U32 matchLengthSearch, ZSTD_match_t* matches, const U32 minMatchLen)
{
switch(matchLengthSearch)
{
case 3 : return ZSTD_BtGetAllMatches_extDict(zc, ip, iHighLimit, maxNbAttempts, 3, matches);
case 3 : return ZSTD_BtGetAllMatches_extDict(zc, ip, iHighLimit, maxNbAttempts, 3, matches, minMatchLen);
default :
case 4 : return ZSTD_BtGetAllMatches_extDict(zc, ip, iHighLimit, maxNbAttempts, 4, matches);
case 5 : return ZSTD_BtGetAllMatches_extDict(zc, ip, iHighLimit, maxNbAttempts, 5, matches);
case 6 : return ZSTD_BtGetAllMatches_extDict(zc, ip, iHighLimit, maxNbAttempts, 6, matches);
case 4 : return ZSTD_BtGetAllMatches_extDict(zc, ip, iHighLimit, maxNbAttempts, 4, matches, minMatchLen);
case 5 : return ZSTD_BtGetAllMatches_extDict(zc, ip, iHighLimit, maxNbAttempts, 5, matches, minMatchLen);
case 6 : return ZSTD_BtGetAllMatches_extDict(zc, ip, iHighLimit, maxNbAttempts, 6, matches, minMatchLen);
}
}
@@ -428,7 +448,6 @@ void ZSTD_compressBlock_opt_generic(ZSTD_CCtx* ctx,
const BYTE* const istart = (const BYTE*)src;
const BYTE* ip = istart;
const BYTE* anchor = istart;
const BYTE* litstart;
const BYTE* const iend = istart + srcSize;
const BYTE* const ilimit = iend - 8;
const BYTE* const base = ctx->base;
@@ -442,10 +461,9 @@ void ZSTD_compressBlock_opt_generic(ZSTD_CCtx* ctx,
ZSTD_optimal_t* opt = seqStorePtr->priceTable;
ZSTD_match_t* matches = seqStorePtr->matchTable;
const BYTE* inr;
U32 cur, match_num, last_pos, litlen, price;
/* init */
U32 rep[ZSTD_REP_INIT];
U32 offset, rep[ZSTD_REP_INIT];
{ U32 i; for (i=0; i<ZSTD_REP_INIT; i++) rep[i]=REPCODE_STARTVALUE; }
ctx->nextToUpdate3 = ctx->nextToUpdate;
@@ -457,47 +475,37 @@ void ZSTD_compressBlock_opt_generic(ZSTD_CCtx* ctx,
/* Match Loop */
while (ip < ilimit) {
U32 u;
U32 mlen=0;
U32 best_mlen=0;
U32 best_off=0;
U32 cur, match_num, last_pos, litlen, price;
U32 u, mlen, best_mlen, best_off, litLength;
memset(opt, 0, sizeof(ZSTD_optimal_t));
last_pos = 0;
inr = ip;
litstart = ((U32)(ip - anchor) > 128) ? ip - 128 : anchor;
opt[0].litlen = (U32)(ip - litstart);
litlen = (U32)(ip - anchor);
/* check repCode */
{ U32 i; for (i=0; i<ZSTD_REP_NUM; i++)
if (MEM_readMINMATCH(ip, minMatch) == MEM_readMINMATCH(ip - rep[i], minMatch)) {
/* repcode : we take it */
mlen = (U32)ZSTD_count(ip+minMatch, ip+minMatch-rep[i], iend) + minMatch;
ZSTD_LOG_PARSER("%d: start try REP rep[%d]=%d mlen=%d\n", (int)(ip-base), i, (int)rep[i], (int)mlen);
if (mlen > sufficient_len || mlen >= ZSTD_OPT_NUM) {
best_mlen = mlen; best_off = i; cur = 0; last_pos = 1;
goto _storeSequence;
}
best_off = (i<=1 && ip == anchor) ? 1-i : i;
litlen = opt[0].litlen;
do {
price = ZSTD_getPrice(seqStorePtr, litlen, litstart, best_off, mlen - MINMATCH);
price = ZSTD_getPrice(seqStorePtr, litlen, anchor, best_off, mlen - MINMATCH);
if (mlen > last_pos || price < opt[mlen].price)
SET_PRICE(mlen, mlen, i, litlen, price); /* note : macro modifies last_pos */
mlen--;
} while (mlen >= minMatch);
} }
match_num = ZSTD_BtGetAllMatches_selectMLS(ctx, ip, iend, maxSearches, mls, matches); /* first search (depth 0) */
match_num = ZSTD_BtGetAllMatches_selectMLS(ctx, ip, iend, maxSearches, mls, matches, minMatch);
ZSTD_LOG_PARSER("%d: match_num=%d last_pos=%d\n", (int)(ip-base), match_num, last_pos);
if (!last_pos && !match_num) { ip++; continue; }
{ U32 i ; for (i=0; i<ZSTD_REP_INIT; i++) opt[0].rep[i] = rep[i]; }
opt[0].mlen = 1;
if (match_num && matches[match_num-1].len > sufficient_len) {
if (match_num && (matches[match_num-1].len > sufficient_len || matches[match_num-1].len >= ZSTD_OPT_NUM)) {
best_mlen = matches[match_num-1].len;
best_off = matches[match_num-1].off;
cur = 0;
@@ -505,23 +513,26 @@ void ZSTD_compressBlock_opt_generic(ZSTD_CCtx* ctx,
goto _storeSequence;
}
best_mlen = (last_pos) ? last_pos : minMatch;
// set prices using matches at position = 0
for (u = 0; u < match_num; u++) {
/* set prices using matches at position = 0 */
best_mlen = (last_pos) ? last_pos : minMatch;
for (u = 0; u < match_num; u++) {
mlen = (u>0) ? matches[u-1].len+1 : best_mlen;
best_mlen = (matches[u].len < ZSTD_OPT_NUM) ? matches[u].len : ZSTD_OPT_NUM;
best_mlen = matches[u].len;
ZSTD_LOG_PARSER("%d: start Found mlen=%d off=%d best_mlen=%d last_pos=%d\n", (int)(ip-base), matches[u].len, matches[u].off, (int)best_mlen, (int)last_pos);
litlen = opt[0].litlen;
while (mlen <= best_mlen) {
price = ZSTD_getPrice(seqStorePtr, litlen, litstart, matches[u].off, mlen - MINMATCH);
price = ZSTD_getPrice(seqStorePtr, litlen, anchor, matches[u].off, mlen - MINMATCH);
if (mlen > last_pos || price < opt[mlen].price)
SET_PRICE(mlen, mlen, matches[u].off, litlen, price);
SET_PRICE(mlen, mlen, matches[u].off, litlen, price); /* note : macro modifies last_pos */
mlen++;
} }
if (last_pos < minMatch) { ip++; continue; }
/* initialize opt[0] */
{ U32 i ; for (i=0; i<ZSTD_REP_INIT; i++) opt[0].rep[i] = rep[i]; }
opt[0].mlen = 1;
opt[0].litlen = litlen;
/* check further positions */
for (cur = 1; cur <= last_pos; cur++) {
inr = ip + cur;
@@ -531,7 +542,7 @@ void ZSTD_compressBlock_opt_generic(ZSTD_CCtx* ctx,
if (cur > litlen) {
price = opt[cur - litlen].price + ZSTD_getLiteralPrice(seqStorePtr, litlen, inr-litlen);
} else
price = ZSTD_getLiteralPrice(seqStorePtr, litlen, litstart);
price = ZSTD_getLiteralPrice(seqStorePtr, litlen, anchor);
} else {
litlen = 1;
price = opt[cur - 1].price + ZSTD_getLiteralPrice(seqStorePtr, litlen, inr-1);
@@ -559,8 +570,8 @@ void ZSTD_compressBlock_opt_generic(ZSTD_CCtx* ctx,
}
ZSTD_LOG_PARSER("%d: CURRENT_NoExt price[%d/%d]=%d off=%d mlen=%d litlen=%d rep[0]=%d rep[1]=%d\n", (int)(inr-base), cur, last_pos, opt[cur].price, opt[cur].off, opt[cur].mlen, opt[cur].litlen, opt[cur].rep[0], opt[cur].rep[1]);
best_mlen = 0;
best_mlen = minMatch;
{ U32 i; for (i=0; i<ZSTD_REP_NUM; i++)
if (MEM_readMINMATCH(inr, minMatch) == MEM_readMINMATCH(inr - opt[cur].rep[i], minMatch)) { /* check rep */
mlen = (U32)ZSTD_count(inr+minMatch, inr+minMatch - opt[cur].rep[i], iend) + minMatch;
@@ -578,13 +589,13 @@ void ZSTD_compressBlock_opt_generic(ZSTD_CCtx* ctx,
if (cur > litlen) {
price = opt[cur - litlen].price + ZSTD_getPrice(seqStorePtr, litlen, inr-litlen, best_off, mlen - MINMATCH);
} else
price = ZSTD_getPrice(seqStorePtr, litlen, litstart, best_off, mlen - MINMATCH);
price = ZSTD_getPrice(seqStorePtr, litlen, anchor, best_off, mlen - MINMATCH);
} else {
litlen = 0;
price = opt[cur].price + ZSTD_getPrice(seqStorePtr, 0, NULL, best_off, mlen - MINMATCH);
}
best_mlen = mlen;
if (mlen > best_mlen) best_mlen = mlen;
ZSTD_LOG_PARSER("%d: Found REP mlen=%d off=%d price=%d litlen=%d\n", (int)(inr-base), mlen, best_off, price, litlen);
do {
@@ -594,23 +605,20 @@ void ZSTD_compressBlock_opt_generic(ZSTD_CCtx* ctx,
} while (mlen >= minMatch);
} }
match_num = ZSTD_BtGetAllMatches_selectMLS(ctx, inr, iend, maxSearches, mls, matches);
match_num = ZSTD_BtGetAllMatches_selectMLS(ctx, inr, iend, maxSearches, mls, matches, best_mlen);
ZSTD_LOG_PARSER("%d: ZSTD_GetAllMatches match_num=%d\n", (int)(inr-base), match_num);
if (match_num > 0 && matches[match_num-1].len > sufficient_len) {
if (match_num > 0 && (matches[match_num-1].len > sufficient_len || cur + matches[match_num-1].len >= ZSTD_OPT_NUM)) {
best_mlen = matches[match_num-1].len;
best_off = matches[match_num-1].off;
last_pos = cur + 1;
goto _storeSequence;
}
best_mlen = (best_mlen > minMatch) ? best_mlen : minMatch;
/* set prices using matches at position = cur */
for (u = 0; u < match_num; u++) {
mlen = (u>0) ? matches[u-1].len+1 : best_mlen;
best_mlen = (cur + matches[u].len < ZSTD_OPT_NUM) ? matches[u].len : ZSTD_OPT_NUM - cur;
best_mlen = matches[u].len;
// ZSTD_LOG_PARSER("%d: Found1 cur=%d mlen=%d off=%d best_mlen=%d last_pos=%d\n", (int)(inr-base), cur, matches[u].len, matches[u].off, best_mlen, last_pos);
while (mlen <= best_mlen) {
@@ -619,7 +627,7 @@ void ZSTD_compressBlock_opt_generic(ZSTD_CCtx* ctx,
if (cur > litlen)
price = opt[cur - litlen].price + ZSTD_getPrice(seqStorePtr, litlen, ip+cur-litlen, matches[u].off, mlen - MINMATCH);
else
price = ZSTD_getPrice(seqStorePtr, litlen, litstart, matches[u].off, mlen - MINMATCH);
price = ZSTD_getPrice(seqStorePtr, litlen, anchor, matches[u].off, mlen - MINMATCH);
} else {
litlen = 0;
price = opt[cur].price + ZSTD_getPrice(seqStorePtr, 0, NULL, matches[u].off, mlen - MINMATCH);
@@ -643,7 +651,6 @@ _storeSequence: /* cur, last_pos, best_mlen, best_off have to be set */
ZSTD_LOG_PARSER("%d: cur=%d/%d best_mlen=%d best_off=%d rep[0]=%d\n", (int)(ip-base+cur), (int)cur, (int)last_pos, (int)best_mlen, (int)best_off, opt[cur].rep[0]);
opt[0].mlen = 1;
U32 offset;
while (1) {
mlen = opt[cur].mlen;
@@ -667,8 +674,7 @@ _storeSequence: /* cur, last_pos, best_mlen, best_off have to be set */
if (mlen == 1) { ip++; cur++; continue; }
offset = opt[cur].off;
cur += mlen;
U32 const litLength = (U32)(ip - anchor);
litLength = (U32)(ip - anchor);
// ZSTD_LOG_ENCODE("%d/%d: ENCODE literals=%d mlen=%d off=%d rep[0]=%d rep[1]=%d\n", (int)(ip-base), (int)(iend-base), (int)(litLength), (int)mlen, (int)(offset), (int)rep[0], (int)rep[1]);
if (offset >= ZSTD_REP_NUM) {
@@ -723,7 +729,6 @@ void ZSTD_compressBlock_opt_extDict_generic(ZSTD_CCtx* ctx,
const BYTE* const istart = (const BYTE*)src;
const BYTE* ip = istart;
const BYTE* anchor = istart;
const BYTE* litstart;
const BYTE* const iend = istart + srcSize;
const BYTE* const ilimit = iend - 8;
const BYTE* const base = ctx->base;
@@ -740,10 +745,9 @@ void ZSTD_compressBlock_opt_extDict_generic(ZSTD_CCtx* ctx,
ZSTD_optimal_t* opt = seqStorePtr->priceTable;
ZSTD_match_t* matches = seqStorePtr->matchTable;
const BYTE* inr;
U32 cur, match_num, last_pos, litlen, price;
/* init */
U32 rep[ZSTD_REP_INIT];
U32 offset, rep[ZSTD_REP_INIT];
{ U32 i; for (i=0; i<ZSTD_REP_INIT; i++) rep[i]=REPCODE_STARTVALUE; }
ctx->nextToUpdate3 = ctx->nextToUpdate;
@@ -755,16 +759,13 @@ void ZSTD_compressBlock_opt_extDict_generic(ZSTD_CCtx* ctx,
/* Match Loop */
while (ip < ilimit) {
U32 u;
U32 mlen=0;
U32 best_mlen=0;
U32 best_off=0;
U32 cur, match_num, last_pos, litlen, price;
U32 u, mlen, best_mlen, best_off, litLength;
U32 current = (U32)(ip-base);
memset(opt, 0, sizeof(ZSTD_optimal_t));
last_pos = 0;
inr = ip;
litstart = ((U32)(ip - anchor) > 128) ? ip - 128 : anchor;
opt[0].litlen = (U32)(ip - litstart);
opt[0].litlen = (U32)(ip - anchor);
/* check repCode */
{ U32 i; for (i=0; i<ZSTD_REP_NUM; i++) {
@@ -786,14 +787,14 @@ void ZSTD_compressBlock_opt_extDict_generic(ZSTD_CCtx* ctx,
best_off = (i<=1 && ip == anchor) ? 1-i : i;
litlen = opt[0].litlen;
do {
price = ZSTD_getPrice(seqStorePtr, litlen, litstart, best_off, mlen - MINMATCH);
price = ZSTD_getPrice(seqStorePtr, litlen, anchor, best_off, mlen - MINMATCH);
if (mlen > last_pos || price < opt[mlen].price)
SET_PRICE(mlen, mlen, i, litlen, price); /* note : macro modifies last_pos */
mlen--;
} while (mlen >= minMatch);
} } }
match_num = ZSTD_BtGetAllMatches_selectMLS_extDict(ctx, ip, iend, maxSearches, mls, matches); /* first search (depth 0) */
match_num = ZSTD_BtGetAllMatches_selectMLS_extDict(ctx, ip, iend, maxSearches, mls, matches, minMatch); /* first search (depth 0) */
ZSTD_LOG_PARSER("%d: match_num=%d last_pos=%d\n", (int)(ip-base), match_num, last_pos);
if (!last_pos && !match_num) { ip++; continue; }
@@ -801,7 +802,7 @@ void ZSTD_compressBlock_opt_extDict_generic(ZSTD_CCtx* ctx,
{ U32 i; for (i=0; i<ZSTD_REP_INIT; i++) opt[0].rep[i] = rep[i]; }
opt[0].mlen = 1;
if (match_num && matches[match_num-1].len > sufficient_len) {
if (match_num && (matches[match_num-1].len > sufficient_len || matches[match_num-1].len >= ZSTD_OPT_NUM)) {
best_mlen = matches[match_num-1].len;
best_off = matches[match_num-1].off;
cur = 0;
@@ -814,11 +815,11 @@ void ZSTD_compressBlock_opt_extDict_generic(ZSTD_CCtx* ctx,
// set prices using matches at position = 0
for (u = 0; u < match_num; u++) {
mlen = (u>0) ? matches[u-1].len+1 : best_mlen;
best_mlen = (matches[u].len < ZSTD_OPT_NUM) ? matches[u].len : ZSTD_OPT_NUM;
best_mlen = matches[u].len;
ZSTD_LOG_PARSER("%d: start Found mlen=%d off=%d best_mlen=%d last_pos=%d\n", (int)(ip-base), matches[u].len, matches[u].off, (int)best_mlen, (int)last_pos);
litlen = opt[0].litlen;
while (mlen <= best_mlen) {
price = ZSTD_getPrice(seqStorePtr, litlen, litstart, matches[u].off, mlen - MINMATCH);
price = ZSTD_getPrice(seqStorePtr, litlen, anchor, matches[u].off, mlen - MINMATCH);
if (mlen > last_pos || price < opt[mlen].price)
SET_PRICE(mlen, mlen, matches[u].off, litlen, price);
mlen++;
@@ -838,7 +839,7 @@ void ZSTD_compressBlock_opt_extDict_generic(ZSTD_CCtx* ctx,
if (cur > litlen) {
price = opt[cur - litlen].price + ZSTD_getLiteralPrice(seqStorePtr, litlen, inr-litlen);
} else
price = ZSTD_getLiteralPrice(seqStorePtr, litlen, litstart);
price = ZSTD_getLiteralPrice(seqStorePtr, litlen, anchor);
} else {
litlen = 1;
price = opt[cur - 1].price + ZSTD_getLiteralPrice(seqStorePtr, litlen, inr-1);
@@ -891,7 +892,7 @@ void ZSTD_compressBlock_opt_extDict_generic(ZSTD_CCtx* ctx,
if (cur > litlen) {
price = opt[cur - litlen].price + ZSTD_getPrice(seqStorePtr, litlen, inr-litlen, best_off, mlen - MINMATCH);
} else
price = ZSTD_getPrice(seqStorePtr, litlen, litstart, best_off, mlen - MINMATCH);
price = ZSTD_getPrice(seqStorePtr, litlen, anchor, best_off, mlen - MINMATCH);
} else {
litlen = 0;
price = opt[cur].price + ZSTD_getPrice(seqStorePtr, 0, NULL, best_off, mlen - MINMATCH);
@@ -907,7 +908,7 @@ void ZSTD_compressBlock_opt_extDict_generic(ZSTD_CCtx* ctx,
} while (mlen >= minMatch);
} } }
match_num = ZSTD_BtGetAllMatches_selectMLS_extDict(ctx, inr, iend, maxSearches, mls, matches);
match_num = ZSTD_BtGetAllMatches_selectMLS_extDict(ctx, inr, iend, maxSearches, mls, matches, minMatch);
ZSTD_LOG_PARSER("%d: ZSTD_GetAllMatches match_num=%d\n", (int)(inr-base), match_num);
if (match_num > 0 && matches[match_num-1].len > sufficient_len) {
@@ -931,7 +932,7 @@ void ZSTD_compressBlock_opt_extDict_generic(ZSTD_CCtx* ctx,
if (cur > litlen)
price = opt[cur - litlen].price + ZSTD_getPrice(seqStorePtr, litlen, ip+cur-litlen, matches[u].off, mlen - MINMATCH);
else
price = ZSTD_getPrice(seqStorePtr, litlen, litstart, matches[u].off, mlen - MINMATCH);
price = ZSTD_getPrice(seqStorePtr, litlen, anchor, matches[u].off, mlen - MINMATCH);
} else {
litlen = 0;
price = opt[cur].price + ZSTD_getPrice(seqStorePtr, 0, NULL, matches[u].off, mlen - MINMATCH);
@@ -955,7 +956,6 @@ _storeSequence: /* cur, last_pos, best_mlen, best_off have to be set */
ZSTD_LOG_PARSER("%d: cur=%d/%d best_mlen=%d best_off=%d rep[0]=%d\n", (int)(ip-base+cur), (int)cur, (int)last_pos, (int)best_mlen, (int)best_off, opt[cur].rep[0]);
opt[0].mlen = 1;
U32 offset;
while (1) {
mlen = opt[cur].mlen;
@@ -979,8 +979,7 @@ _storeSequence: /* cur, last_pos, best_mlen, best_off have to be set */
if (mlen == 1) { ip++; cur++; continue; }
offset = opt[cur].off;
cur += mlen;
U32 const litLength = (U32)(ip - anchor);
litLength = (U32)(ip - anchor);
// ZSTD_LOG_ENCODE("%d/%d: ENCODE1 literals=%d mlen=%d off=%d rep[0]=%d rep[1]=%d\n", (int)(ip-base), (int)(iend-base), (int)(litLength), (int)mlen, (int)(offset), (int)rep[0], (int)rep[1]);
if (offset >= ZSTD_REP_NUM) {
+96 -657
View File
@@ -1,5 +1,5 @@
/* ******************************************************************
Huff0 : Huffman coder, part of New Generation Entropy library
Huffman decoder, part of New Generation Entropy library
Copyright (C) 2013-2016, Yann Collet.
BSD 2-Clause License (http://www.opensource.org/licenses/bsd-license.php)
@@ -28,7 +28,7 @@
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
You can contact the author at :
- FSE+Huff0 source repository : https://github.com/Cyan4973/FiniteStateEntropy
- FSE+HUF source repository : https://github.com/Cyan4973/FiniteStateEntropy
- Public forum : https://groups.google.com/forum/#!forum/lz4c
****************************************************************** */
@@ -62,524 +62,21 @@
#include <stdlib.h> /* malloc, free, qsort */
#include <string.h> /* memcpy, memset */
#include <stdio.h> /* printf (debug) */
#include "huff0_static.h"
#include "huf_static.h"
#include "bitstream.h"
#include "fse.h" /* header compression */
/* **************************************************************
* Constants
****************************************************************/
#define HUF_ABSOLUTEMAX_TABLELOG 16 /* absolute limit of HUF_MAX_TABLELOG. Beyond that value, code does not work */
#define HUF_MAX_TABLELOG 12 /* max configured tableLog (for static allocation); can be modified up to HUF_ABSOLUTEMAX_TABLELOG */
#define HUF_DEFAULT_TABLELOG HUF_MAX_TABLELOG /* tableLog by default, when not specified */
#define HUF_MAX_SYMBOL_VALUE 255
#if (HUF_MAX_TABLELOG > HUF_ABSOLUTEMAX_TABLELOG)
# error "HUF_MAX_TABLELOG is too large !"
#endif
/* **************************************************************
* Error Management
****************************************************************/
unsigned HUF_isError(size_t code) { return ERR_isError(code); }
const char* HUF_getErrorName(size_t code) { return ERR_getErrorName(code); }
#define HUF_STATIC_ASSERT(c) { enum { HUF_static_assert = 1/(int)(!!(c)) }; } /* use only *after* variable declarations */
/* *******************************************************
* Huff0 : Huffman block compression
*********************************************************/
struct HUF_CElt_s {
U16 val;
BYTE nbBits;
}; /* typedef'd to HUF_CElt within huff0_static.h */
typedef struct nodeElt_s {
U32 count;
U16 parent;
BYTE byte;
BYTE nbBits;
} nodeElt;
/*! HUF_writeCTable() :
`CTable` : huffman tree to save, using huff0 representation.
@return : size of saved CTable */
size_t HUF_writeCTable (void* dst, size_t maxDstSize,
const HUF_CElt* CTable, U32 maxSymbolValue, U32 huffLog)
{
BYTE bitsToWeight[HUF_MAX_TABLELOG + 1];
BYTE huffWeight[HUF_MAX_SYMBOL_VALUE + 1];
U32 n;
BYTE* op = (BYTE*)dst;
size_t size;
/* check conditions */
if (maxSymbolValue > HUF_MAX_SYMBOL_VALUE + 1)
return ERROR(GENERIC);
/* convert to weight */
bitsToWeight[0] = 0;
for (n=1; n<=huffLog; n++)
bitsToWeight[n] = (BYTE)(huffLog + 1 - n);
for (n=0; n<maxSymbolValue; n++)
huffWeight[n] = bitsToWeight[CTable[n].nbBits];
size = FSE_compress(op+1, maxDstSize-1, huffWeight, maxSymbolValue); /* don't need last symbol stat : implied */
if (HUF_isError(size)) return size;
if (size >= 128) return ERROR(GENERIC); /* should never happen, since maxSymbolValue <= 255 */
if ((size <= 1) || (size >= maxSymbolValue/2)) {
if (size==1) { /* RLE */
/* only possible case : serie of 1 (because there are at least 2) */
/* can only be 2^n or (2^n-1), otherwise not an huffman tree */
BYTE code;
switch(maxSymbolValue)
{
case 1: code = 0; break;
case 2: code = 1; break;
case 3: code = 2; break;
case 4: code = 3; break;
case 7: code = 4; break;
case 8: code = 5; break;
case 15: code = 6; break;
case 16: code = 7; break;
case 31: code = 8; break;
case 32: code = 9; break;
case 63: code = 10; break;
case 64: code = 11; break;
case 127: code = 12; break;
case 128: code = 13; break;
default : return ERROR(corruption_detected);
}
op[0] = (BYTE)(255-13 + code);
return 1;
}
/* Not compressible */
if (maxSymbolValue > (241-128)) return ERROR(GENERIC); /* not implemented (not possible with current format) */
if (((maxSymbolValue+1)/2) + 1 > maxDstSize) return ERROR(dstSize_tooSmall); /* not enough space within dst buffer */
op[0] = (BYTE)(128 /*special case*/ + 0 /* Not Compressible */ + (maxSymbolValue-1));
huffWeight[maxSymbolValue] = 0; /* to be sure it doesn't cause issue in final combination */
for (n=0; n<maxSymbolValue; n+=2)
op[(n/2)+1] = (BYTE)((huffWeight[n] << 4) + huffWeight[n+1]);
return ((maxSymbolValue+1)/2) + 1;
}
/* normal header case */
op[0] = (BYTE)size;
return size+1;
}
static size_t HUF_readStats(BYTE* huffWeight, size_t hwSize, U32* rankStats,
U32* nbSymbolsPtr, U32* tableLogPtr,
const void* src, size_t srcSize);
size_t HUF_readCTable (HUF_CElt* CTable, U32 maxSymbolValue, const void* src, size_t srcSize)
{
BYTE huffWeight[HUF_MAX_SYMBOL_VALUE + 1];
U32 rankVal[HUF_ABSOLUTEMAX_TABLELOG + 1]; /* large enough for values from 0 to 16 */
U32 tableLog = 0;
size_t readSize;
U32 nbSymbols = 0;
//memset(huffWeight, 0, sizeof(huffWeight)); /* is not necessary, even though some analyzer complain ... */
/* get symbol weights */
readSize = HUF_readStats(huffWeight, HUF_MAX_SYMBOL_VALUE+1, rankVal, &nbSymbols, &tableLog, src, srcSize);
if (HUF_isError(readSize)) return readSize;
/* check result */
if (tableLog > HUF_MAX_TABLELOG) return ERROR(tableLog_tooLarge);
if (nbSymbols > maxSymbolValue+1) return ERROR(maxSymbolValue_tooSmall);
/* Prepare base value per rank */
{ U32 n, nextRankStart = 0;
for (n=1; n<=tableLog; n++) {
U32 current = nextRankStart;
nextRankStart += (rankVal[n] << (n-1));
rankVal[n] = current;
} }
/* fill nbBits */
{ U32 n; for (n=0; n<nbSymbols; n++) {
const U32 w = huffWeight[n];
CTable[n].nbBits = (BYTE)(tableLog + 1 - w);
}}
/* fill val */
{ U16 nbPerRank[HUF_MAX_TABLELOG+1] = {0};
U16 valPerRank[HUF_MAX_TABLELOG+1] = {0};
{ U32 n; for (n=0; n<nbSymbols; n++) nbPerRank[CTable[n].nbBits]++; }
/* determine stating value per rank */
{ U16 min = 0;
U32 n; for (n=HUF_MAX_TABLELOG; n>0; n--) {
valPerRank[n] = min; /* get starting value within each rank */
min += nbPerRank[n];
min >>= 1;
} }
/* assign value within rank, symbol order */
{ U32 n; for (n=0; n<=maxSymbolValue; n++) CTable[n].val = valPerRank[CTable[n].nbBits]++; }
}
return readSize;
}
static U32 HUF_setMaxHeight(nodeElt* huffNode, U32 lastNonNull, U32 maxNbBits)
{
const U32 largestBits = huffNode[lastNonNull].nbBits;
if (largestBits <= maxNbBits) return largestBits; /* early exit : no elt > maxNbBits */
/* there are several too large elements (at least >= 2) */
{ int totalCost = 0;
const U32 baseCost = 1 << (largestBits - maxNbBits);
U32 n = lastNonNull;
while (huffNode[n].nbBits > maxNbBits) {
totalCost += baseCost - (1 << (largestBits - huffNode[n].nbBits));
huffNode[n].nbBits = (BYTE)maxNbBits;
n --;
} /* n stops at huffNode[n].nbBits <= maxNbBits */
while (huffNode[n].nbBits == maxNbBits) n--; /* n end at index of smallest symbol using < maxNbBits */
/* renorm totalCost */
totalCost >>= (largestBits - maxNbBits); /* note : totalCost is necessarily a multiple of baseCost */
/* repay normalized cost */
{ U32 const noSymbol = 0xF0F0F0F0;
U32 rankLast[HUF_MAX_TABLELOG+1];
int pos;
/* Get pos of last (smallest) symbol per rank */
memset(rankLast, 0xF0, sizeof(rankLast));
{ U32 currentNbBits = maxNbBits;
for (pos=n ; pos >= 0; pos--) {
if (huffNode[pos].nbBits >= currentNbBits) continue;
currentNbBits = huffNode[pos].nbBits; /* < maxNbBits */
rankLast[maxNbBits-currentNbBits] = pos;
} }
while (totalCost > 0) {
U32 nBitsToDecrease = BIT_highbit32(totalCost) + 1;
for ( ; nBitsToDecrease > 1; nBitsToDecrease--) {
U32 highPos = rankLast[nBitsToDecrease];
U32 lowPos = rankLast[nBitsToDecrease-1];
if (highPos == noSymbol) continue;
if (lowPos == noSymbol) break;
{ U32 const highTotal = huffNode[highPos].count;
U32 const lowTotal = 2 * huffNode[lowPos].count;
if (highTotal <= lowTotal) break;
} }
/* only triggered when no more rank 1 symbol left => find closest one (note : there is necessarily at least one !) */
while ((nBitsToDecrease<=HUF_MAX_TABLELOG) && (rankLast[nBitsToDecrease] == noSymbol)) /* HUF_MAX_TABLELOG test just to please gcc 5+; but it should not be necessary */
nBitsToDecrease ++;
totalCost -= 1 << (nBitsToDecrease-1);
if (rankLast[nBitsToDecrease-1] == noSymbol)
rankLast[nBitsToDecrease-1] = rankLast[nBitsToDecrease]; /* this rank is no longer empty */
huffNode[rankLast[nBitsToDecrease]].nbBits ++;
if (rankLast[nBitsToDecrease] == 0) /* special case, reached largest symbol */
rankLast[nBitsToDecrease] = noSymbol;
else {
rankLast[nBitsToDecrease]--;
if (huffNode[rankLast[nBitsToDecrease]].nbBits != maxNbBits-nBitsToDecrease)
rankLast[nBitsToDecrease] = noSymbol; /* this rank is now empty */
} } /* while (totalCost > 0) */
while (totalCost < 0) { /* Sometimes, cost correction overshoot */
if (rankLast[1] == noSymbol) { /* special case : no rank 1 symbol (using maxNbBits-1); let's create one from largest rank 0 (using maxNbBits) */
while (huffNode[n].nbBits == maxNbBits) n--;
huffNode[n+1].nbBits--;
rankLast[1] = n+1;
totalCost++;
continue;
}
huffNode[ rankLast[1] + 1 ].nbBits--;
rankLast[1]++;
totalCost ++;
} } } /* there are several too large elements (at least >= 2) */
return maxNbBits;
}
typedef struct {
U32 base;
U32 current;
} rankPos;
static void HUF_sort(nodeElt* huffNode, const U32* count, U32 maxSymbolValue)
{
rankPos rank[32];
U32 n;
memset(rank, 0, sizeof(rank));
for (n=0; n<=maxSymbolValue; n++) {
U32 r = BIT_highbit32(count[n] + 1);
rank[r].base ++;
}
for (n=30; n>0; n--) rank[n-1].base += rank[n].base;
for (n=0; n<32; n++) rank[n].current = rank[n].base;
for (n=0; n<=maxSymbolValue; n++) {
U32 const c = count[n];
U32 const r = BIT_highbit32(c+1) + 1;
U32 pos = rank[r].current++;
while ((pos > rank[r].base) && (c > huffNode[pos-1].count)) huffNode[pos]=huffNode[pos-1], pos--;
huffNode[pos].count = c;
huffNode[pos].byte = (BYTE)n;
}
}
#define STARTNODE (HUF_MAX_SYMBOL_VALUE+1)
size_t HUF_buildCTable (HUF_CElt* tree, const U32* count, U32 maxSymbolValue, U32 maxNbBits)
{
nodeElt huffNode0[2*HUF_MAX_SYMBOL_VALUE+1 +1];
nodeElt* huffNode = huffNode0 + 1;
U32 n, nonNullRank;
int lowS, lowN;
U16 nodeNb = STARTNODE;
U32 nodeRoot;
/* safety checks */
if (maxNbBits == 0) maxNbBits = HUF_DEFAULT_TABLELOG;
if (maxSymbolValue > HUF_MAX_SYMBOL_VALUE) return ERROR(GENERIC);
memset(huffNode0, 0, sizeof(huffNode0));
/* sort, decreasing order */
HUF_sort(huffNode, count, maxSymbolValue);
/* init for parents */
nonNullRank = maxSymbolValue;
while(huffNode[nonNullRank].count == 0) nonNullRank--;
lowS = nonNullRank; nodeRoot = nodeNb + lowS - 1; lowN = nodeNb;
huffNode[nodeNb].count = huffNode[lowS].count + huffNode[lowS-1].count;
huffNode[lowS].parent = huffNode[lowS-1].parent = nodeNb;
nodeNb++; lowS-=2;
for (n=nodeNb; n<=nodeRoot; n++) huffNode[n].count = (U32)(1U<<30);
huffNode0[0].count = (U32)(1U<<31);
/* create parents */
while (nodeNb <= nodeRoot) {
U32 n1 = (huffNode[lowS].count < huffNode[lowN].count) ? lowS-- : lowN++;
U32 n2 = (huffNode[lowS].count < huffNode[lowN].count) ? lowS-- : lowN++;
huffNode[nodeNb].count = huffNode[n1].count + huffNode[n2].count;
huffNode[n1].parent = huffNode[n2].parent = nodeNb;
nodeNb++;
}
/* distribute weights (unlimited tree height) */
huffNode[nodeRoot].nbBits = 0;
for (n=nodeRoot-1; n>=STARTNODE; n--)
huffNode[n].nbBits = huffNode[ huffNode[n].parent ].nbBits + 1;
for (n=0; n<=nonNullRank; n++)
huffNode[n].nbBits = huffNode[ huffNode[n].parent ].nbBits + 1;
/* enforce maxTableLog */
maxNbBits = HUF_setMaxHeight(huffNode, nonNullRank, maxNbBits);
/* fill result into tree (val, nbBits) */
{ U16 nbPerRank[HUF_MAX_TABLELOG+1] = {0};
U16 valPerRank[HUF_MAX_TABLELOG+1] = {0};
if (maxNbBits > HUF_MAX_TABLELOG) return ERROR(GENERIC); /* check fit into table */
for (n=0; n<=nonNullRank; n++)
nbPerRank[huffNode[n].nbBits]++;
/* determine stating value per rank */
{ U16 min = 0;
for (n=maxNbBits; n>0; n--) {
valPerRank[n] = min; /* get starting value within each rank */
min += nbPerRank[n];
min >>= 1;
} }
for (n=0; n<=maxSymbolValue; n++)
tree[huffNode[n].byte].nbBits = huffNode[n].nbBits; /* push nbBits per symbol, symbol order */
for (n=0; n<=maxSymbolValue; n++)
tree[n].val = valPerRank[tree[n].nbBits]++; /* assign value within rank, symbol order */
}
return maxNbBits;
}
static void HUF_encodeSymbol(BIT_CStream_t* bitCPtr, U32 symbol, const HUF_CElt* CTable)
{
BIT_addBitsFast(bitCPtr, CTable[symbol].val, CTable[symbol].nbBits);
}
size_t HUF_compressBound(size_t size) { return HUF_COMPRESSBOUND(size); }
#define HUF_FLUSHBITS(s) (fast ? BIT_flushBitsFast(s) : BIT_flushBits(s))
#define HUF_FLUSHBITS_1(stream) \
if (sizeof((stream)->bitContainer)*8 < HUF_MAX_TABLELOG*2+7) HUF_FLUSHBITS(stream)
#define HUF_FLUSHBITS_2(stream) \
if (sizeof((stream)->bitContainer)*8 < HUF_MAX_TABLELOG*4+7) HUF_FLUSHBITS(stream)
size_t HUF_compress1X_usingCTable(void* dst, size_t dstSize, const void* src, size_t srcSize, const HUF_CElt* CTable)
{
const BYTE* ip = (const BYTE*) src;
BYTE* const ostart = (BYTE*)dst;
BYTE* const oend = ostart + dstSize;
BYTE* op = ostart;
size_t n;
const unsigned fast = (dstSize >= HUF_BLOCKBOUND(srcSize));
BIT_CStream_t bitC;
/* init */
if (dstSize < 8) return 0; /* not enough space to compress */
{ size_t const errorCode = BIT_initCStream(&bitC, op, oend-op);
if (HUF_isError(errorCode)) return 0; }
n = srcSize & ~3; /* join to mod 4 */
switch (srcSize & 3)
{
case 3 : HUF_encodeSymbol(&bitC, ip[n+ 2], CTable);
HUF_FLUSHBITS_2(&bitC);
case 2 : HUF_encodeSymbol(&bitC, ip[n+ 1], CTable);
HUF_FLUSHBITS_1(&bitC);
case 1 : HUF_encodeSymbol(&bitC, ip[n+ 0], CTable);
HUF_FLUSHBITS(&bitC);
case 0 :
default: ;
}
for (; n>0; n-=4) { /* note : n&3==0 at this stage */
HUF_encodeSymbol(&bitC, ip[n- 1], CTable);
HUF_FLUSHBITS_1(&bitC);
HUF_encodeSymbol(&bitC, ip[n- 2], CTable);
HUF_FLUSHBITS_2(&bitC);
HUF_encodeSymbol(&bitC, ip[n- 3], CTable);
HUF_FLUSHBITS_1(&bitC);
HUF_encodeSymbol(&bitC, ip[n- 4], CTable);
HUF_FLUSHBITS(&bitC);
}
return BIT_closeCStream(&bitC);
}
size_t HUF_compress4X_usingCTable(void* dst, size_t dstSize, const void* src, size_t srcSize, const HUF_CElt* CTable)
{
size_t segmentSize = (srcSize+3)/4; /* first 3 segments */
const BYTE* ip = (const BYTE*) src;
const BYTE* const iend = ip + srcSize;
BYTE* const ostart = (BYTE*) dst;
BYTE* const oend = ostart + dstSize;
BYTE* op = ostart;
size_t errorCode;
if (dstSize < 6 + 1 + 1 + 1 + 8) return 0; /* minimum space to compress successfully */
if (srcSize < 12) return 0; /* no saving possible : too small input */
op += 6; /* jumpTable */
errorCode = HUF_compress1X_usingCTable(op, oend-op, ip, segmentSize, CTable);
if (HUF_isError(errorCode)) return errorCode;
if (errorCode==0) return 0;
MEM_writeLE16(ostart, (U16)errorCode);
ip += segmentSize;
op += errorCode;
errorCode = HUF_compress1X_usingCTable(op, oend-op, ip, segmentSize, CTable);
if (HUF_isError(errorCode)) return errorCode;
if (errorCode==0) return 0;
MEM_writeLE16(ostart+2, (U16)errorCode);
ip += segmentSize;
op += errorCode;
errorCode = HUF_compress1X_usingCTable(op, oend-op, ip, segmentSize, CTable);
if (HUF_isError(errorCode)) return errorCode;
if (errorCode==0) return 0;
MEM_writeLE16(ostart+4, (U16)errorCode);
ip += segmentSize;
op += errorCode;
errorCode = HUF_compress1X_usingCTable(op, oend-op, ip, iend-ip, CTable);
if (HUF_isError(errorCode)) return errorCode;
if (errorCode==0) return 0;
op += errorCode;
return op-ostart;
}
static size_t HUF_compress_internal (
void* dst, size_t dstSize,
const void* src, size_t srcSize,
unsigned maxSymbolValue, unsigned huffLog,
unsigned singleStream)
{
BYTE* const ostart = (BYTE*)dst;
BYTE* const oend = ostart + dstSize;
BYTE* op = ostart;
U32 count[HUF_MAX_SYMBOL_VALUE+1];
HUF_CElt CTable[HUF_MAX_SYMBOL_VALUE+1];
size_t errorCode;
/* checks & inits */
if (srcSize < 1) return 0; /* Uncompressed - note : 1 means rle, so first byte must be correct */
if (dstSize < 1) return 0; /* not compressible within dst budget */
if (srcSize > 128 * 1024) return ERROR(srcSize_wrong); /* current block size limit */
if (huffLog > HUF_MAX_TABLELOG) return ERROR(tableLog_tooLarge);
if (!maxSymbolValue) maxSymbolValue = HUF_MAX_SYMBOL_VALUE;
if (!huffLog) huffLog = HUF_DEFAULT_TABLELOG;
/* Scan input and build symbol stats */
errorCode = FSE_count (count, &maxSymbolValue, (const BYTE*)src, srcSize);
if (HUF_isError(errorCode)) return errorCode;
if (errorCode == srcSize) { *ostart = ((const BYTE*)src)[0]; return 1; }
if (errorCode <= (srcSize >> 7)+1) return 0; /* Heuristic : not compressible enough */
/* Build Huffman Tree */
errorCode = HUF_buildCTable (CTable, count, maxSymbolValue, huffLog);
if (HUF_isError(errorCode)) return errorCode;
huffLog = (U32)errorCode;
/* Write table description header */
errorCode = HUF_writeCTable (op, dstSize, CTable, maxSymbolValue, huffLog);
if (HUF_isError(errorCode)) return errorCode;
if (errorCode + 12 >= srcSize) return 0; /* not useful to try compression */
op += errorCode;
/* Compress */
if (singleStream)
errorCode = HUF_compress1X_usingCTable(op, oend - op, src, srcSize, CTable); /* single segment */
else
errorCode = HUF_compress4X_usingCTable(op, oend - op, src, srcSize, CTable);
if (HUF_isError(errorCode)) return errorCode;
if (errorCode==0) return 0;
op += errorCode;
/* check compressibility */
if ((size_t)(op-ostart) >= srcSize-1)
return 0;
return op-ostart;
}
size_t HUF_compress1X (void* dst, size_t dstSize,
const void* src, size_t srcSize,
unsigned maxSymbolValue, unsigned huffLog)
{
return HUF_compress_internal(dst, dstSize, src, srcSize, maxSymbolValue, huffLog, 1);
}
size_t HUF_compress2 (void* dst, size_t dstSize,
const void* src, size_t srcSize,
unsigned maxSymbolValue, unsigned huffLog)
{
return HUF_compress_internal(dst, dstSize, src, srcSize, maxSymbolValue, huffLog, 0);
}
size_t HUF_compress (void* dst, size_t maxDstSize, const void* src, size_t srcSize)
{
return HUF_compress2(dst, maxDstSize, src, (U32)srcSize, 255, HUF_DEFAULT_TABLELOG);
}
/* *******************************************************
* Huff0 : Huffman block decompression
* HUF : Huffman block decompression
*********************************************************/
typedef struct { BYTE byte; BYTE nbBits; } HUF_DEltX2; /* single-symbol decoding */
@@ -587,77 +84,6 @@ typedef struct { U16 sequence; BYTE nbBits; BYTE length; } HUF_DEltX4; /* doubl
typedef struct { BYTE symbol; BYTE weight; } sortedSymbol_t;
/*! HUF_readStats() :
Read compact Huffman tree, saved by HUF_writeCTable().
`huffWeight` is destination buffer.
@return : size read from `src`
*/
static size_t HUF_readStats(BYTE* huffWeight, size_t hwSize, U32* rankStats,
U32* nbSymbolsPtr, U32* tableLogPtr,
const void* src, size_t srcSize)
{
U32 weightTotal;
U32 tableLog;
const BYTE* ip = (const BYTE*) src;
size_t iSize = ip[0];
size_t oSize;
//memset(huffWeight, 0, hwSize); /* is not necessary, even though some analyzer complain ... */
if (iSize >= 128) { /* special header */
if (iSize >= (242)) { /* RLE */
static U32 l[14] = { 1, 2, 3, 4, 7, 8, 15, 16, 31, 32, 63, 64, 127, 128 };
oSize = l[iSize-242];
memset(huffWeight, 1, hwSize);
iSize = 0;
}
else { /* Incompressible */
oSize = iSize - 127;
iSize = ((oSize+1)/2);
if (iSize+1 > srcSize) return ERROR(srcSize_wrong);
if (oSize >= hwSize) return ERROR(corruption_detected);
ip += 1;
{ U32 n;
for (n=0; n<oSize; n+=2) {
huffWeight[n] = ip[n/2] >> 4;
huffWeight[n+1] = ip[n/2] & 15;
} } } }
else { /* header compressed with FSE (normal case) */
if (iSize+1 > srcSize) return ERROR(srcSize_wrong);
oSize = FSE_decompress(huffWeight, hwSize-1, ip+1, iSize); /* max (hwSize-1) values decoded, as last one is implied */
if (FSE_isError(oSize)) return oSize;
}
/* collect weight stats */
memset(rankStats, 0, (HUF_ABSOLUTEMAX_TABLELOG + 1) * sizeof(U32));
weightTotal = 0;
{ U32 n; for (n=0; n<oSize; n++) {
if (huffWeight[n] >= HUF_ABSOLUTEMAX_TABLELOG) return ERROR(corruption_detected);
rankStats[huffWeight[n]]++;
weightTotal += (1 << huffWeight[n]) >> 1;
}}
/* get last non-null symbol weight (implied, total must be 2^n) */
tableLog = BIT_highbit32(weightTotal) + 1;
if (tableLog > HUF_ABSOLUTEMAX_TABLELOG) return ERROR(corruption_detected);
/* determine last weight */
{ U32 const total = 1 << tableLog;
U32 const rest = total - weightTotal;
U32 const verif = 1 << BIT_highbit32(rest);
U32 const lastWeight = BIT_highbit32(rest) + 1;
if (verif != rest) return ERROR(corruption_detected); /* last value must be a clean power of 2 */
huffWeight[oSize] = (BYTE)lastWeight;
rankStats[lastWeight]++;
}
/* check tree construction validity */
if ((rankStats[1] < 2) || (rankStats[1] & 1)) return ERROR(corruption_detected); /* by construction : at least 2 elts of rank 1, must be even */
/* results */
*nbSymbolsPtr = (U32)(oSize+1);
*tableLogPtr = tableLog;
return iSize+1;
}
/*-***************************/
@@ -688,7 +114,7 @@ size_t HUF_readDTableX2 (U16* DTable, const void* src, size_t srcSize)
/* Prepare ranks */
nextRankStart = 0;
for (n=1; n<=tableLog; n++) {
for (n=1; n<tableLog+1; n++) {
U32 current = nextRankStart;
nextRankStart += (rankVal[n] << (n-1));
rankVal[n] = current;
@@ -1016,7 +442,7 @@ size_t HUF_readDTableX4 (U32* DTable, const void* src, size_t srcSize)
/* Get start index of each weight */
{ U32 w, nextRankStart = 0;
for (w=1; w<=maxW; w++) {
for (w=1; w<maxW+1; w++) {
U32 current = nextRankStart;
nextRankStart += rankStats[w];
rankStart[w] = current;
@@ -1028,8 +454,8 @@ size_t HUF_readDTableX4 (U32* DTable, const void* src, size_t srcSize)
/* sort symbols by weight */
{ U32 s;
for (s=0; s<nbSymbols; s++) {
U32 w = weightList[s];
U32 r = rankStart[w]++;
U32 const w = weightList[s];
U32 const r = rankStart[w]++;
sortedSymbol[r].symbol = (BYTE)s;
sortedSymbol[r].weight = (BYTE)w;
}
@@ -1037,21 +463,23 @@ size_t HUF_readDTableX4 (U32* DTable, const void* src, size_t srcSize)
}
/* Build rankVal */
{ const U32 minBits = tableLog+1 - maxW;
U32 nextRankVal = 0;
U32 w, consumed;
const int rescale = (memLog-tableLog) - 1; /* tableLog <= memLog */
U32* rankVal0 = rankVal[0];
for (w=1; w<=maxW; w++) {
U32 current = nextRankVal;
nextRankVal += rankStats[w] << (w+rescale);
rankVal0[w] = current;
}
for (consumed = minBits; consumed <= memLog - minBits; consumed++) {
U32* rankValPtr = rankVal[consumed];
for (w = 1; w <= maxW; w++) {
rankValPtr[w] = rankVal0[w] >> consumed;
} } }
{ U32* const rankVal0 = rankVal[0];
{ int const rescale = (memLog-tableLog) - 1; /* tableLog <= memLog */
U32 nextRankVal = 0;
U32 w;
for (w=1; w<maxW+1; w++) {
U32 current = nextRankVal;
nextRankVal += rankStats[w] << (w+rescale);
rankVal0[w] = current;
} }
{ U32 const minBits = tableLog+1 - maxW;
U32 consumed;
for (consumed = minBits; consumed < memLog - minBits + 1; consumed++) {
U32* const rankValPtr = rankVal[consumed];
U32 w;
for (w = 1; w < maxW+1; w++) {
rankValPtr[w] = rankVal0[w] >> consumed;
} } } }
HUF_fillDTableX4(dt, memLog,
sortedSymbol, sizeOfSort,
@@ -1362,7 +790,7 @@ size_t HUF_readDTableX6 (U32* DTable, const void* src, size_t srcSize)
/* Get start index of each weight */
{ U32 w, nextRankStart = 0;
for (w=1; w<=maxW; w++) {
for (w=1; w<maxW+1; w++) {
U32 current = nextRankStart;
nextRankStart += rankStats[w];
rankStart[w] = current;
@@ -1388,14 +816,14 @@ size_t HUF_readDTableX6 (U32* DTable, const void* src, size_t srcSize)
U32 w, consumed;
const int rescale = (memLog-tableLog) - 1; /* tableLog <= memLog */
U32* rankVal0 = rankVal[0];
for (w=1; w<=maxW; w++) {
for (w=1; w<maxW+1; w++) {
U32 current = nextRankVal;
nextRankVal += rankStats[w] << (w+rescale);
rankVal0[w] = current;
}
for (consumed = minBits; consumed <= memLog - minBits; consumed++) {
U32* rankValPtr = rankVal[consumed];
for (w = 1; w <= maxW; w++) {
for (w = 1; w < maxW+1; w++) {
rankValPtr[w] = rankVal0[w] >> consumed;
} } }
@@ -1421,17 +849,17 @@ size_t HUF_readDTableX6 (U32* DTable, const void* src, size_t srcSize)
static U32 HUF_decodeSymbolX6(void* op, BIT_DStream_t* DStream, const HUF_DDescX6* dd, const HUF_DSeqX6* ds, const U32 dtLog)
{
const size_t val = BIT_lookBitsFast(DStream, dtLog); /* note : dtLog >= 1 */
size_t const val = BIT_lookBitsFast(DStream, dtLog); /* note : dtLog >= 1 */
memcpy(op, ds+val, sizeof(HUF_DSeqX6));
BIT_skipBits(DStream, dd[val].nbBits);
return dd[val].nbBytes;
}
static U32 HUF_decodeLastSymbolsX6(void* op, const U32 maxL, BIT_DStream_t* DStream,
static U32 HUF_decodeLastSymbolsX6(void* op, U32 const maxL, BIT_DStream_t* DStream,
const HUF_DDescX6* dd, const HUF_DSeqX6* ds, const U32 dtLog)
{
const size_t val = BIT_lookBitsFast(DStream, dtLog); /* note : dtLog >= 1 */
U32 length = dd[val].nbBytes;
size_t const val = BIT_lookBitsFast(DStream, dtLog); /* note : dtLog >= 1 */
U32 const length = dd[val].nbBytes;
if (length <= maxL) {
memcpy(op, ds+val, length);
BIT_skipBits(DStream, dd[val].nbBits);
@@ -1484,7 +912,6 @@ static inline size_t HUF_decodeStreamX6(BYTE* p, BIT_DStream_t* bitDPtr, BYTE* c
return p-pStart;
}
size_t HUF_decompress1X6_usingDTable(
void* dst, size_t dstSize,
const void* cSrc, size_t cSrcSize,
@@ -1493,17 +920,15 @@ size_t HUF_decompress1X6_usingDTable(
const BYTE* const istart = (const BYTE*) cSrc;
BYTE* const ostart = (BYTE*) dst;
BYTE* const oend = ostart + dstSize;
const U32 dtLog = DTable[0];
size_t errorCode;
BIT_DStream_t bitD;
/* Init */
BIT_DStream_t bitD;
errorCode = BIT_initDStream(&bitD, istart, cSrcSize);
if (HUF_isError(errorCode)) return errorCode;
{ size_t const errorCode = BIT_initDStream(&bitD, istart, cSrcSize);
if (HUF_isError(errorCode)) return errorCode; }
/* finish bitStreams one by one */
HUF_decodeStreamX6(ostart, &bitD, oend, DTable, dtLog);
{ U32 const dtLog = DTable[0];
HUF_decodeStreamX6(ostart, &bitD, oend, DTable, dtLog); }
/* check */
if (!BIT_endOfDStream(&bitD)) return ERROR(corruption_detected);
@@ -1517,7 +942,7 @@ size_t HUF_decompress1X6 (void* dst, size_t dstSize, const void* cSrc, size_t cS
HUF_CREATE_STATIC_DTABLEX6(DTable, HUF_MAX_TABLELOG);
const BYTE* ip = (const BYTE*) cSrc;
size_t hSize = HUF_readDTableX6 (DTable, cSrc, cSrcSize);
size_t const hSize = HUF_readDTableX6 (DTable, cSrc, cSrcSize);
if (HUF_isError(hSize)) return hSize;
if (hSize >= cSrcSize) return ERROR(srcSize_wrong);
ip += hSize;
@@ -1527,6 +952,24 @@ size_t HUF_decompress1X6 (void* dst, size_t dstSize, const void* cSrc, size_t cS
}
#define HUF_DECODE_ROUNDX6 \
HUF_DECODE_SYMBOLX6_2(op1, &bitD1); \
HUF_DECODE_SYMBOLX6_2(op2, &bitD2); \
HUF_DECODE_SYMBOLX6_2(op3, &bitD3); \
HUF_DECODE_SYMBOLX6_2(op4, &bitD4); \
HUF_DECODE_SYMBOLX6_1(op1, &bitD1); \
HUF_DECODE_SYMBOLX6_1(op2, &bitD2); \
HUF_DECODE_SYMBOLX6_1(op3, &bitD3); \
HUF_DECODE_SYMBOLX6_1(op4, &bitD4); \
HUF_DECODE_SYMBOLX6_2(op1, &bitD1); \
HUF_DECODE_SYMBOLX6_2(op2, &bitD2); \
HUF_DECODE_SYMBOLX6_2(op3, &bitD3); \
HUF_DECODE_SYMBOLX6_2(op4, &bitD4); \
HUF_DECODE_SYMBOLX6_0(op1, &bitD1); \
HUF_DECODE_SYMBOLX6_0(op2, &bitD2); \
HUF_DECODE_SYMBOLX6_0(op3, &bitD3); \
HUF_DECODE_SYMBOLX6_0(op4, &bitD4);
size_t HUF_decompress4X6_usingDTable(
void* dst, size_t dstSize,
const void* cSrc, size_t cSrcSize,
@@ -1534,6 +977,7 @@ size_t HUF_decompress4X6_usingDTable(
{
/* Check */
if (cSrcSize < 10) return ERROR(corruption_detected); /* strict minimum : jump table + 1 byte per stream */
if (dstSize < 64) return ERROR(dstSize_tooSmall); /* only work for dstSize >= 64 */
{ const BYTE* const istart = (const BYTE*) cSrc;
BYTE* const ostart = (BYTE*) dst;
@@ -1544,7 +988,6 @@ size_t HUF_decompress4X6_usingDTable(
const HUF_DDescX6* dd = (const HUF_DDescX6*)ddPtr;
const void* const dsPtr = DTable + 1 + ((size_t)1<<(dtLog-1));
const HUF_DSeqX6* ds = (const HUF_DSeqX6*)dsPtr;
size_t errorCode;
/* Init */
BIT_DStream_t bitD1;
@@ -1571,43 +1014,41 @@ size_t HUF_decompress4X6_usingDTable(
length4 = cSrcSize - (length1 + length2 + length3 + 6);
if (length4 > cSrcSize) return ERROR(corruption_detected); /* overflow */
errorCode = BIT_initDStream(&bitD1, istart1, length1);
if (HUF_isError(errorCode)) return errorCode;
errorCode = BIT_initDStream(&bitD2, istart2, length2);
if (HUF_isError(errorCode)) return errorCode;
errorCode = BIT_initDStream(&bitD3, istart3, length3);
if (HUF_isError(errorCode)) return errorCode;
errorCode = BIT_initDStream(&bitD4, istart4, length4);
if (HUF_isError(errorCode)) return errorCode;
{ size_t const errorCode = BIT_initDStream(&bitD1, istart1, length1);
if (HUF_isError(errorCode)) return errorCode; }
{ size_t const errorCode = BIT_initDStream(&bitD2, istart2, length2);
if (HUF_isError(errorCode)) return errorCode; }
{ size_t const errorCode = BIT_initDStream(&bitD3, istart3, length3);
if (HUF_isError(errorCode)) return errorCode; }
{ size_t const errorCode = BIT_initDStream(&bitD4, istart4, length4);
if (HUF_isError(errorCode)) return errorCode; }
/* 16-64 symbols per loop (4-16 symbols per stream) */
/* 4-64 symbols per loop (1-16 symbols per stream) */
endSignal = BIT_reloadDStream(&bitD1) | BIT_reloadDStream(&bitD2) | BIT_reloadDStream(&bitD3) | BIT_reloadDStream(&bitD4);
for ( ; (op3 <= opStart4) && (endSignal==BIT_DStream_unfinished) && (op4<=(oend-16)) ; ) {
HUF_DECODE_SYMBOLX6_2(op1, &bitD1);
HUF_DECODE_SYMBOLX6_2(op2, &bitD2);
HUF_DECODE_SYMBOLX6_2(op3, &bitD3);
HUF_DECODE_SYMBOLX6_2(op4, &bitD4);
HUF_DECODE_SYMBOLX6_1(op1, &bitD1);
HUF_DECODE_SYMBOLX6_1(op2, &bitD2);
HUF_DECODE_SYMBOLX6_1(op3, &bitD3);
HUF_DECODE_SYMBOLX6_1(op4, &bitD4);
HUF_DECODE_SYMBOLX6_2(op1, &bitD1);
HUF_DECODE_SYMBOLX6_2(op2, &bitD2);
HUF_DECODE_SYMBOLX6_2(op3, &bitD3);
HUF_DECODE_SYMBOLX6_2(op4, &bitD4);
HUF_DECODE_SYMBOLX6_0(op1, &bitD1);
HUF_DECODE_SYMBOLX6_0(op2, &bitD2);
HUF_DECODE_SYMBOLX6_0(op3, &bitD3);
HUF_DECODE_SYMBOLX6_0(op4, &bitD4);
endSignal = BIT_reloadDStream(&bitD1) | BIT_reloadDStream(&bitD2) | BIT_reloadDStream(&bitD3) | BIT_reloadDStream(&bitD4);
}
if (endSignal==BIT_DStream_unfinished) {
HUF_DECODE_ROUNDX6;
if (sizeof(bitD1.bitContainer)==4) { /* need to decode at least 4 bytes per stream */
endSignal = BIT_reloadDStream(&bitD1) | BIT_reloadDStream(&bitD2) | BIT_reloadDStream(&bitD3) | BIT_reloadDStream(&bitD4);
HUF_DECODE_ROUNDX6;
}
{ U32 const saved2 = MEM_read32(opStart2); /* saved from overwrite */
U32 const saved3 = MEM_read32(opStart3);
U32 const saved4 = MEM_read32(opStart4);
endSignal = BIT_reloadDStream(&bitD1) | BIT_reloadDStream(&bitD2) | BIT_reloadDStream(&bitD3) | BIT_reloadDStream(&bitD4);
for ( ; (op3 <= opStart4) && (endSignal==BIT_DStream_unfinished) && (op4<=(oend-16)) ; ) {
HUF_DECODE_ROUNDX6;
endSignal = BIT_reloadDStream(&bitD1) | BIT_reloadDStream(&bitD2) | BIT_reloadDStream(&bitD3) | BIT_reloadDStream(&bitD4);
}
MEM_write32(opStart2, saved2);
MEM_write32(opStart3, saved3);
MEM_write32(opStart4, saved4);
} }
/* check corruption */
if (op1 > opStart2) return ERROR(corruption_detected);
if (op2 > opStart3) return ERROR(corruption_detected);
if (op3 > opStart4) return ERROR(corruption_detected);
/* note : op4 supposed already verified within main loop */
/* note : op4 already verified within main loop */
/* finish bitStreams one by one */
HUF_decodeStreamX6(op1, &bitD1, opStart2, DTable, dtLog);
@@ -1671,12 +1112,7 @@ typedef size_t (*decompressionAlgo)(void* dst, size_t dstSize, const void* cSrc,
size_t HUF_decompress (void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize)
{
static const decompressionAlgo decompress[3] = { HUF_decompress4X2, HUF_decompress4X4, HUF_decompress4X6 };
/* estimate decompression time */
U32 Q;
const U32 D256 = (U32)(dstSize >> 8);
U32 Dtime[3];
U32 algoNb = 0;
int n;
U32 Dtime[3]; /* decompression time estimation */
/* validation checks */
if (dstSize == 0) return ERROR(dstSize_tooSmall);
@@ -1685,16 +1121,19 @@ size_t HUF_decompress (void* dst, size_t dstSize, const void* cSrc, size_t cSrcS
if (cSrcSize == 1) { memset(dst, *(const BYTE*)cSrc, dstSize); return dstSize; } /* RLE */
/* decoder timing evaluation */
Q = (U32)(cSrcSize * 16 / dstSize); /* Q < 16 since dstSize > cSrcSize */
for (n=0; n<3; n++)
Dtime[n] = algoTime[Q][n].tableTime + (algoTime[Q][n].decode256Time * D256);
{ U32 const Q = (U32)(cSrcSize * 16 / dstSize); /* Q < 16 since dstSize > cSrcSize */
U32 const D256 = (U32)(dstSize >> 8);
U32 n; for (n=0; n<3; n++)
Dtime[n] = algoTime[Q][n].tableTime + (algoTime[Q][n].decode256Time * D256);
}
Dtime[1] += Dtime[1] >> 4; Dtime[2] += Dtime[2] >> 3; /* advantage to algorithms using less memory, for cache eviction */
if (Dtime[1] < Dtime[0]) algoNb = 1;
if (Dtime[2] < Dtime[algoNb]) algoNb = 2;
return decompress[algoNb](dst, dstSize, cSrc, cSrcSize);
{ U32 algoNb = 0;
if (Dtime[1] < Dtime[0]) algoNb = 1;
// if (Dtime[2] < Dtime[algoNb]) algoNb = 2; /* current speed of HUF_decompress4X6 is not good */
return decompress[algoNb](dst, dstSize, cSrc, cSrcSize);
}
//return HUF_decompress4X2(dst, dstSize, cSrc, cSrcSize); /* multi-streams single-symbol decoding */
//return HUF_decompress4X4(dst, dstSize, cSrc, cSrcSize); /* multi-streams double-symbols decoding */
+265
View File
@@ -0,0 +1,265 @@
/*
Buffered version of Zstd compression library
Copyright (C) 2015-2016, Yann Collet.
BSD 2-Clause License (http://www.opensource.org/licenses/bsd-license.php)
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the following disclaimer
in the documentation and/or other materials provided with the
distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
You can contact the author at :
- zstd homepage : http://www.zstd.net/
*/
/* *************************************
* Dependencies
***************************************/
#include <stdlib.h>
#include "error_private.h"
#include "zstd_internal.h" /* MIN, ZSTD_blockHeaderSize */
#include "zstd_static.h" /* ZSTD_BLOCKSIZE_MAX */
#include "zbuff_static.h"
/*-***************************************************************************
* Streaming decompression howto
*
* A ZBUFF_DCtx object is required to track streaming operations.
* Use ZBUFF_createDCtx() and ZBUFF_freeDCtx() to create/release resources.
* Use ZBUFF_decompressInit() to start a new decompression operation,
* or ZBUFF_decompressInitDictionary() if decompression requires a dictionary.
* Note that ZBUFF_DCtx objects can be re-init multiple times.
*
* Use ZBUFF_decompressContinue() repetitively to consume your input.
* *srcSizePtr and *dstCapacityPtr can be any size.
* The function will report how many bytes were read or written by modifying *srcSizePtr and *dstCapacityPtr.
* Note that it may not consume the entire input, in which case it's up to the caller to present remaining input again.
* The content of @dst will be overwritten (up to *dstCapacityPtr) at each function call, so save its content if it matters, or change @dst.
* @return : a hint to preferred nb of bytes to use as input for next function call (it's only a hint, to help latency),
* or 0 when a frame is completely decoded,
* or an error code, which can be tested using ZBUFF_isError().
*
* Hint : recommended buffer sizes (not compulsory) : ZBUFF_recommendedDInSize() and ZBUFF_recommendedDOutSize()
* output : ZBUFF_recommendedDOutSize==128 KB block size is the internal unit, it ensures it's always possible to write a full block when decoded.
* input : ZBUFF_recommendedDInSize == 128KB + 3;
* just follow indications from ZBUFF_decompressContinue() to minimize latency. It should always be <= 128 KB + 3 .
* *******************************************************************************/
typedef enum { ZBUFFds_init, ZBUFFds_loadHeader,
ZBUFFds_read, ZBUFFds_load, ZBUFFds_flush } ZBUFF_dStage;
/* *** Resource management *** */
struct ZBUFF_DCtx_s {
ZSTD_DCtx* zd;
ZSTD_frameParams fParams;
ZBUFF_dStage stage;
char* inBuff;
size_t inBuffSize;
size_t inPos;
char* outBuff;
size_t outBuffSize;
size_t outStart;
size_t outEnd;
size_t blockSize;
BYTE headerBuffer[ZSTD_FRAMEHEADERSIZE_MAX];
size_t lhSize;
}; /* typedef'd to ZBUFF_DCtx within "zstd_buffered.h" */
ZBUFF_DCtx* ZBUFF_createDCtx(void)
{
ZBUFF_DCtx* zbd = (ZBUFF_DCtx*)malloc(sizeof(ZBUFF_DCtx));
if (zbd==NULL) return NULL;
memset(zbd, 0, sizeof(*zbd));
zbd->zd = ZSTD_createDCtx();
zbd->stage = ZBUFFds_init;
return zbd;
}
size_t ZBUFF_freeDCtx(ZBUFF_DCtx* zbd)
{
if (zbd==NULL) return 0; /* support free on null */
ZSTD_freeDCtx(zbd->zd);
free(zbd->inBuff);
free(zbd->outBuff);
free(zbd);
return 0;
}
/* *** Initialization *** */
size_t ZBUFF_decompressInitDictionary(ZBUFF_DCtx* zbd, const void* dict, size_t dictSize)
{
zbd->stage = ZBUFFds_loadHeader;
zbd->lhSize = zbd->inPos = zbd->outStart = zbd->outEnd = 0;
return ZSTD_decompressBegin_usingDict(zbd->zd, dict, dictSize);
}
size_t ZBUFF_decompressInit(ZBUFF_DCtx* zbd)
{
return ZBUFF_decompressInitDictionary(zbd, NULL, 0);
}
/* *** Decompression *** */
size_t ZBUFF_decompressContinue(ZBUFF_DCtx* zbd,
void* dst, size_t* dstCapacityPtr,
const void* src, size_t* srcSizePtr)
{
const char* const istart = (const char*)src;
const char* const iend = istart + *srcSizePtr;
const char* ip = istart;
char* const ostart = (char*)dst;
char* const oend = ostart + *dstCapacityPtr;
char* op = ostart;
U32 notDone = 1;
while (notDone) {
switch(zbd->stage)
{
case ZBUFFds_init :
return ERROR(init_missing);
case ZBUFFds_loadHeader :
{ size_t const hSize = ZSTD_getFrameParams(&(zbd->fParams), zbd->headerBuffer, zbd->lhSize);
if (hSize != 0) {
size_t const toLoad = hSize - zbd->lhSize; /* if hSize!=0, hSize > zbd->lhSize */
if (ZSTD_isError(hSize)) return hSize;
if (toLoad > (size_t)(iend-ip)) { /* not enough input to load full header */
memcpy(zbd->headerBuffer + zbd->lhSize, ip, iend-ip);
zbd->lhSize += iend-ip; ip = iend; notDone = 0;
*dstCapacityPtr = 0;
return (hSize - zbd->lhSize) + ZSTD_blockHeaderSize; /* remaining header bytes + next block header */
}
memcpy(zbd->headerBuffer + zbd->lhSize, ip, toLoad); zbd->lhSize = hSize; ip += toLoad;
break;
} }
/* Consume header */
{ size_t const h1Size = ZSTD_nextSrcSizeToDecompress(zbd->zd); /* == ZSTD_frameHeaderSize_min */
size_t const h1Result = ZSTD_decompressContinue(zbd->zd, NULL, 0, zbd->headerBuffer, h1Size);
if (ZSTD_isError(h1Result)) return h1Result;
if (h1Size < zbd->lhSize) { /* long header */
size_t const h2Size = ZSTD_nextSrcSizeToDecompress(zbd->zd);
size_t const h2Result = ZSTD_decompressContinue(zbd->zd, NULL, 0, zbd->headerBuffer+h1Size, h2Size);
if (ZSTD_isError(h2Result)) return h2Result;
} }
/* Frame header instruct buffer sizes */
{ size_t const blockSize = MIN(1 << zbd->fParams.windowLog, ZSTD_BLOCKSIZE_MAX);
zbd->blockSize = blockSize;
if (zbd->inBuffSize < blockSize) {
free(zbd->inBuff);
zbd->inBuffSize = blockSize;
zbd->inBuff = (char*)malloc(blockSize);
if (zbd->inBuff == NULL) return ERROR(memory_allocation);
}
{ size_t const neededOutSize = ((size_t)1 << zbd->fParams.windowLog) + blockSize;
if (zbd->outBuffSize < neededOutSize) {
free(zbd->outBuff);
zbd->outBuffSize = neededOutSize;
zbd->outBuff = (char*)malloc(neededOutSize);
if (zbd->outBuff == NULL) return ERROR(memory_allocation);
} } }
zbd->stage = ZBUFFds_read;
case ZBUFFds_read:
{ size_t const neededInSize = ZSTD_nextSrcSizeToDecompress(zbd->zd);
if (neededInSize==0) { /* end of frame */
zbd->stage = ZBUFFds_init;
notDone = 0;
break;
}
if ((size_t)(iend-ip) >= neededInSize) { /* decode directly from src */
size_t const decodedSize = ZSTD_decompressContinue(zbd->zd,
zbd->outBuff + zbd->outStart, zbd->outBuffSize - zbd->outStart,
ip, neededInSize);
if (ZSTD_isError(decodedSize)) return decodedSize;
ip += neededInSize;
if (!decodedSize) break; /* this was just a header */
zbd->outEnd = zbd->outStart + decodedSize;
zbd->stage = ZBUFFds_flush;
break;
}
if (ip==iend) { notDone = 0; break; } /* no more input */
zbd->stage = ZBUFFds_load;
}
case ZBUFFds_load:
{ size_t const neededInSize = ZSTD_nextSrcSizeToDecompress(zbd->zd);
size_t const toLoad = neededInSize - zbd->inPos; /* should always be <= remaining space within inBuff */
size_t loadedSize;
if (toLoad > zbd->inBuffSize - zbd->inPos) return ERROR(corruption_detected); /* should never happen */
loadedSize = ZBUFF_limitCopy(zbd->inBuff + zbd->inPos, toLoad, ip, iend-ip);
ip += loadedSize;
zbd->inPos += loadedSize;
if (loadedSize < toLoad) { notDone = 0; break; } /* not enough input, wait for more */
/* decode loaded input */
{ size_t const decodedSize = ZSTD_decompressContinue(zbd->zd,
zbd->outBuff + zbd->outStart, zbd->outBuffSize - zbd->outStart,
zbd->inBuff, neededInSize);
if (ZSTD_isError(decodedSize)) return decodedSize;
zbd->inPos = 0; /* input is consumed */
if (!decodedSize) { zbd->stage = ZBUFFds_read; break; } /* this was just a header */
zbd->outEnd = zbd->outStart + decodedSize;
zbd->stage = ZBUFFds_flush;
// break; /* ZBUFFds_flush follows */
} }
case ZBUFFds_flush:
{ size_t const toFlushSize = zbd->outEnd - zbd->outStart;
size_t const flushedSize = ZBUFF_limitCopy(op, oend-op, zbd->outBuff + zbd->outStart, toFlushSize);
op += flushedSize;
zbd->outStart += flushedSize;
if (flushedSize == toFlushSize) {
zbd->stage = ZBUFFds_read;
if (zbd->outStart + zbd->blockSize > zbd->outBuffSize)
zbd->outStart = zbd->outEnd = 0;
break;
}
/* cannot flush everything */
notDone = 0;
break;
}
default: return ERROR(GENERIC); /* impossible */
} }
/* result */
*srcSizePtr = ip-istart;
*dstCapacityPtr = op-ostart;
{ size_t nextSrcSizeHint = ZSTD_nextSrcSizeToDecompress(zbd->zd);
if (nextSrcSizeHint > ZSTD_blockHeaderSize) nextSrcSizeHint+= ZSTD_blockHeaderSize; /* get following block header too */
nextSrcSizeHint -= zbd->inPos; /* already loaded*/
return nextSrcSizeHint;
}
}
/* *************************************
* Tool functions
***************************************/
size_t ZBUFF_recommendedDInSize(void) { return ZSTD_BLOCKSIZE_MAX + ZSTD_blockHeaderSize /* block header size*/ ; }
size_t ZBUFF_recommendedDOutSize(void) { return ZSTD_BLOCKSIZE_MAX; }
@@ -59,7 +59,7 @@
#include "mem.h" /* low level memory routines */
#include "zstd_internal.h"
#include "fse_static.h"
#include "huff0_static.h"
#include "huf_static.h"
#if defined(ZSTD_LEGACY_SUPPORT) && (ZSTD_LEGACY_SUPPORT==1)
# include "zstd_legacy.h"
@@ -83,30 +83,20 @@
#endif
/*-*************************************
* Macros
***************************************/
#define ZSTD_isError ERR_isError /* for inlining */
#define FSE_isError ERR_isError
#define HUF_isError ERR_isError
/*_*******************************************************
* Memory operations
**********************************************************/
static void ZSTD_copy4(void* dst, const void* src) { memcpy(dst, src, 4); }
/*-*************************************
* Error Management
***************************************/
unsigned ZSTD_versionNumber (void) { return ZSTD_VERSION_NUMBER; }
/*! ZSTD_isError() :
* tells if a return value is an error code */
unsigned ZSTD_isError(size_t code) { return ERR_isError(code); }
/*! ZSTD_getError() :
* convert a `size_t` function result into a proper ZSTD_errorCode enum */
ZSTD_ErrorCode ZSTD_getError(size_t code) { return ERR_getError(code); }
/*! ZSTD_getErrorName() :
* provides error code string (useful for debugging) */
const char* ZSTD_getErrorName(size_t code) { return ERR_getErrorName(code); }
/*-*************************************************************
* Context management
***************************************************************/
@@ -278,10 +268,9 @@ void ZSTD_copyDCtx(ZSTD_DCtx* dstDCtx, const ZSTD_DCtx* srcDCtx)
* @return : size of the Frame Header */
static size_t ZSTD_frameHeaderSize(const void* src, size_t srcSize)
{
U32 fcsId;
if (srcSize < ZSTD_frameHeaderSize_min) return ERROR(srcSize_wrong);
fcsId = (((const BYTE*)src)[4]) >> 6;
return ZSTD_frameHeaderSize_min + ZSTD_fcs_fieldSize[fcsId];
{ U32 const fcsId = (((const BYTE*)src)[4]) >> 6;
return ZSTD_frameHeaderSize_min + ZSTD_fcs_fieldSize[fcsId]; }
}
@@ -670,7 +659,8 @@ static void ZSTD_decodeSequence(seq_t* seq, seqState_t* seqState)
}
FORCE_INLINE size_t ZSTD_execSequence(BYTE* op,
FORCE_INLINE
size_t ZSTD_execSequence(BYTE* op,
BYTE* const oend, seq_t sequence,
const BYTE** litPtr, const BYTE* const litLimit_8,
const BYTE* const base, const BYTE* const vBase, const BYTE* const dictEnd)
@@ -855,28 +845,22 @@ size_t ZSTD_decompressBlock(ZSTD_DCtx* dctx,
}
/*! ZSTD_decompress_continueDCtx() :
* `dctx` must have been properly initialized */
/*! ZSTD_decompressFrame() :
* `dctx` must be properly initialized */
static size_t ZSTD_decompressFrame(ZSTD_DCtx* dctx,
void* dst, size_t dstCapacity,
const void* src, size_t srcSize)
{
const BYTE* ip = (const BYTE*)src;
const BYTE* iend = ip + srcSize;
const BYTE* const iend = ip + srcSize;
BYTE* const ostart = (BYTE* const)dst;
BYTE* op = ostart;
BYTE* const oend = ostart + dstCapacity;
size_t remainingSize = srcSize;
blockProperties_t blockProperties;
blockProperties_t blockProperties = { bt_compressed, 0 };
/* check */
if (srcSize < ZSTD_frameHeaderSize_min+ZSTD_blockHeaderSize) return ERROR(srcSize_wrong);
#if defined(ZSTD_LEGACY_SUPPORT) && (ZSTD_LEGACY_SUPPORT==1)
{ const U32 magicNumber = MEM_readLE32(src);
if (ZSTD_isLegacy(magicNumber))
return ZSTD_decompressLegacy(dst, dstCapacity, src, srcSize, magicNumber);
}
#endif
/* Frame Header */
{ size_t const frameHeaderSize = ZSTD_frameHeaderSize(src, ZSTD_frameHeaderSize_min);
@@ -941,6 +925,12 @@ size_t ZSTD_decompress_usingDict(ZSTD_DCtx* dctx,
const void* src, size_t srcSize,
const void* dict, size_t dictSize)
{
#if defined(ZSTD_LEGACY_SUPPORT) && (ZSTD_LEGACY_SUPPORT==1)
{ const U32 magicNumber = MEM_readLE32(src);
if (ZSTD_isLegacy(magicNumber))
return ZSTD_decompressLegacy(dst, dstCapacity, src, srcSize, dict, dictSize, magicNumber);
}
#endif
ZSTD_decompressBegin_usingDict(dctx, dict, dictSize);
ZSTD_checkContinuity(dctx, dst);
return ZSTD_decompressFrame(dctx, dst, dstCapacity, src, srcSize);
@@ -962,7 +952,7 @@ size_t ZSTD_decompress(void* dst, size_t dstCapacity, const void* src, size_t sr
regenSize = ZSTD_decompressDCtx(dctx, dst, dstCapacity, src, srcSize);
ZSTD_freeDCtx(dctx);
return regenSize;
#else
#else /* stack mode */
ZSTD_DCtx dctx;
return ZSTD_decompressDCtx(&dctx, dst, dstCapacity, src, srcSize);
#endif
@@ -977,31 +967,29 @@ size_t ZSTD_nextSrcSizeToDecompress(ZSTD_DCtx* dctx)
return dctx->expected;
}
size_t ZSTD_decompressContinue(ZSTD_DCtx* dctx, void* dst, size_t maxDstSize, const void* src, size_t srcSize)
size_t ZSTD_decompressContinue(ZSTD_DCtx* dctx, void* dst, size_t dstCapacity, const void* src, size_t srcSize)
{
/* Sanity check */
if (srcSize != dctx->expected) return ERROR(srcSize_wrong);
ZSTD_checkContinuity(dctx, dst);
if (dstCapacity) ZSTD_checkContinuity(dctx, dst);
/* Decompress : frame header; part 1 */
switch (dctx->stage)
{
case ZSTDds_getFrameHeaderSize :
{
if (srcSize != ZSTD_frameHeaderSize_min) return ERROR(srcSize_wrong); /* impossible */
dctx->headerSize = ZSTD_frameHeaderSize(src, ZSTD_frameHeaderSize_min);
if (ZSTD_isError(dctx->headerSize)) return dctx->headerSize;
memcpy(dctx->headerBuffer, src, ZSTD_frameHeaderSize_min);
if (dctx->headerSize > ZSTD_frameHeaderSize_min) {
dctx->expected = dctx->headerSize - ZSTD_frameHeaderSize_min;
dctx->stage = ZSTDds_decodeFrameHeader;
return 0;
}
dctx->expected = 0; /* not necessary to copy more */
if (srcSize != ZSTD_frameHeaderSize_min) return ERROR(srcSize_wrong); /* impossible */
dctx->headerSize = ZSTD_frameHeaderSize(src, ZSTD_frameHeaderSize_min);
if (ZSTD_isError(dctx->headerSize)) return dctx->headerSize;
memcpy(dctx->headerBuffer, src, ZSTD_frameHeaderSize_min);
if (dctx->headerSize > ZSTD_frameHeaderSize_min) {
dctx->expected = dctx->headerSize - ZSTD_frameHeaderSize_min;
dctx->stage = ZSTDds_decodeFrameHeader;
return 0;
}
dctx->expected = 0; /* not necessary to copy more */
case ZSTDds_decodeFrameHeader:
{
size_t result;
{ size_t result;
memcpy(dctx->headerBuffer + ZSTD_frameHeaderSize_min, src, dctx->expected);
result = ZSTD_decodeFrameHeader(dctx, dctx->headerBuffer, dctx->headerSize);
if (ZSTD_isError(result)) return result;
@@ -1010,8 +998,7 @@ size_t ZSTD_decompressContinue(ZSTD_DCtx* dctx, void* dst, size_t maxDstSize, co
return 0;
}
case ZSTDds_decodeBlockHeader:
{
blockProperties_t bp;
{ blockProperties_t bp;
size_t const cBlockSize = ZSTD_getcBlockSize(src, ZSTD_blockHeaderSize, &bp);
if (ZSTD_isError(cBlockSize)) return cBlockSize;
if (bp.blockType == bt_end) {
@@ -1025,16 +1012,14 @@ size_t ZSTD_decompressContinue(ZSTD_DCtx* dctx, void* dst, size_t maxDstSize, co
return 0;
}
case ZSTDds_decompressBlock:
{
/* Decompress : block content */
size_t rSize;
{ size_t rSize;
switch(dctx->bType)
{
case bt_compressed:
rSize = ZSTD_decompressBlock_internal(dctx, dst, maxDstSize, src, srcSize);
rSize = ZSTD_decompressBlock_internal(dctx, dst, dstCapacity, src, srcSize);
break;
case bt_raw :
rSize = ZSTD_copyRawBlock(dst, maxDstSize, src, srcSize);
rSize = ZSTD_copyRawBlock(dst, dstCapacity, src, srcSize);
break;
case bt_rle :
return ERROR(GENERIC); /* not yet handled */
@@ -1066,37 +1051,40 @@ static void ZSTD_refDictContent(ZSTD_DCtx* dctx, const void* dict, size_t dictSi
static size_t ZSTD_loadEntropy(ZSTD_DCtx* dctx, const void* dict, size_t dictSize)
{
size_t hSize, offcodeHeaderSize, matchlengthHeaderSize, errorCode, litlengthHeaderSize;
short offcodeNCount[MaxOff+1];
U32 offcodeMaxValue=MaxOff, offcodeLog=OffFSELog;
short matchlengthNCount[MaxML+1];
unsigned matchlengthMaxValue = MaxML, matchlengthLog = MLFSELog;
short litlengthNCount[MaxLL+1];
unsigned litlengthMaxValue = MaxLL, litlengthLog = LLFSELog;
size_t hSize, offcodeHeaderSize, matchlengthHeaderSize, litlengthHeaderSize;
hSize = HUF_readDTableX4(dctx->hufTableX4, dict, dictSize);
if (HUF_isError(hSize)) return ERROR(dictionary_corrupted);
dict = (const char*)dict + hSize;
dictSize -= hSize;
offcodeHeaderSize = FSE_readNCount(offcodeNCount, &offcodeMaxValue, &offcodeLog, dict, dictSize);
if (FSE_isError(offcodeHeaderSize)) return ERROR(dictionary_corrupted);
errorCode = FSE_buildDTable(dctx->OffTable, offcodeNCount, offcodeMaxValue, offcodeLog);
if (FSE_isError(errorCode)) return ERROR(dictionary_corrupted);
dict = (const char*)dict + offcodeHeaderSize;
dictSize -= offcodeHeaderSize;
{ short offcodeNCount[MaxOff+1];
U32 offcodeMaxValue=MaxOff, offcodeLog=OffFSELog;
offcodeHeaderSize = FSE_readNCount(offcodeNCount, &offcodeMaxValue, &offcodeLog, dict, dictSize);
if (FSE_isError(offcodeHeaderSize)) return ERROR(dictionary_corrupted);
{ size_t const errorCode = FSE_buildDTable(dctx->OffTable, offcodeNCount, offcodeMaxValue, offcodeLog);
if (FSE_isError(errorCode)) return ERROR(dictionary_corrupted); }
dict = (const char*)dict + offcodeHeaderSize;
dictSize -= offcodeHeaderSize;
}
matchlengthHeaderSize = FSE_readNCount(matchlengthNCount, &matchlengthMaxValue, &matchlengthLog, dict, dictSize);
if (FSE_isError(matchlengthHeaderSize)) return ERROR(dictionary_corrupted);
errorCode = FSE_buildDTable(dctx->MLTable, matchlengthNCount, matchlengthMaxValue, matchlengthLog);
if (FSE_isError(errorCode)) return ERROR(dictionary_corrupted);
dict = (const char*)dict + matchlengthHeaderSize;
dictSize -= matchlengthHeaderSize;
{ short matchlengthNCount[MaxML+1];
unsigned matchlengthMaxValue = MaxML, matchlengthLog = MLFSELog;
matchlengthHeaderSize = FSE_readNCount(matchlengthNCount, &matchlengthMaxValue, &matchlengthLog, dict, dictSize);
if (FSE_isError(matchlengthHeaderSize)) return ERROR(dictionary_corrupted);
{ size_t const errorCode = FSE_buildDTable(dctx->MLTable, matchlengthNCount, matchlengthMaxValue, matchlengthLog);
if (FSE_isError(errorCode)) return ERROR(dictionary_corrupted); }
dict = (const char*)dict + matchlengthHeaderSize;
dictSize -= matchlengthHeaderSize;
}
litlengthHeaderSize = FSE_readNCount(litlengthNCount, &litlengthMaxValue, &litlengthLog, dict, dictSize);
if (FSE_isError(litlengthHeaderSize)) return ERROR(dictionary_corrupted);
errorCode = FSE_buildDTable(dctx->LLTable, litlengthNCount, litlengthMaxValue, litlengthLog);
if (FSE_isError(errorCode)) return ERROR(dictionary_corrupted);
{ short litlengthNCount[MaxLL+1];
unsigned litlengthMaxValue = MaxLL, litlengthLog = LLFSELog;
litlengthHeaderSize = FSE_readNCount(litlengthNCount, &litlengthMaxValue, &litlengthLog, dict, dictSize);
if (FSE_isError(litlengthHeaderSize)) return ERROR(dictionary_corrupted);
{ size_t const errorCode = FSE_buildDTable(dctx->LLTable, litlengthNCount, litlengthMaxValue, litlengthLog);
if (FSE_isError(errorCode)) return ERROR(dictionary_corrupted); }
}
dctx->flagRepeatTable = 1;
return hSize + offcodeHeaderSize + matchlengthHeaderSize + litlengthHeaderSize;
+1 -10
View File
@@ -54,26 +54,17 @@
#include <stdlib.h> /* malloc, free */
#include <string.h> /* memset */
#include <stdio.h> /* fprintf, fopen, ftello64 */
#include <sys/types.h> /* stat64 */
#include <sys/stat.h> /* stat64 */
#include <time.h> /* clock */
#include "mem.h" /* read */
#include "error_private.h"
#include "fse.h"
#include "huff0_static.h"
#include "huf_static.h"
#include "zstd_internal.h"
#include "divsufsort.h"
#include "zdict_static.h"
/*-*************************************
* Compiler specifics
***************************************/
#if !defined(S_ISREG)
# define S_ISREG(x) (((x) & S_IFMT) == S_IFREG)
#endif
/*-*************************************
* Constants
+20 -8
View File
@@ -1,7 +1,7 @@
/*
zstd_legacy - decoder for legacy format
Header File
Copyright (C) 2015, Yann Collet.
Copyright (C) 2015-2016, Yann Collet.
BSD 2-Clause License (http://www.opensource.org/licenses/bsd-license.php)
@@ -48,16 +48,20 @@ extern "C" {
#include "zstd_v04.h"
#include "zstd_v05.h"
/** ZSTD_isLegacy() :
@return : > 0 if supported by legacy decoder. 0 otherwise.
return value is the version.
*/
MEM_STATIC unsigned ZSTD_isLegacy (U32 magicNumberLE)
{
switch(magicNumberLE)
{
case ZSTDv01_magicNumberLE :
case ZSTDv02_magicNumber :
case ZSTDv03_magicNumber :
case ZSTDv04_magicNumber :
case ZSTDv05_MAGICNUMBER :
return 1;
case ZSTDv01_magicNumberLE:return 1;
case ZSTDv02_magicNumber : return 2;
case ZSTDv03_magicNumber : return 3;
case ZSTDv04_magicNumber : return 4;
case ZSTDv05_MAGICNUMBER : return 5;
default : return 0;
}
}
@@ -66,6 +70,7 @@ MEM_STATIC unsigned ZSTD_isLegacy (U32 magicNumberLE)
MEM_STATIC size_t ZSTD_decompressLegacy(
void* dst, size_t dstCapacity,
const void* src, size_t compressedSize,
const void* dict,size_t dictSize,
U32 magicNumberLE)
{
switch(magicNumberLE)
@@ -79,7 +84,14 @@ MEM_STATIC size_t ZSTD_decompressLegacy(
case ZSTDv04_magicNumber :
return ZSTDv04_decompress(dst, dstCapacity, src, compressedSize);
case ZSTDv05_MAGICNUMBER :
return ZSTDv05_decompress(dst, dstCapacity, src, compressedSize);
{
size_t result;
ZSTDv05_DCtx* zd = ZSTDv05_createDCtx();
if (zd==NULL) return ERROR(memory_allocation);
result = ZSTDv05_decompress_usingDict(zd, dst, dstCapacity, src, compressedSize, dict, dictSize);
ZSTDv05_freeDCtx(zd);
return result;
}
default :
return ERROR(prefix_unknown);
}
+2 -2
View File
@@ -407,7 +407,7 @@ FORCE_INLINE unsigned FSE_highbit32 (register U32 val)
static U32 FSE_tableStep(U32 tableSize) { return (tableSize>>1) + (tableSize>>3) + 3; }
#define FSE_DECODE_TYPE FSE_TYPE_NAME(FSE_decode_t, FSE_FUNCTION_EXTENSION)
#define FSE_DECODE_TYPE FSE_decode_t
typedef struct {
@@ -415,7 +415,7 @@ typedef struct {
U16 fastMode;
} FSE_DTableHeader; /* sizeof U32 */
static size_t FSE_FUNCTION_NAME(FSE_buildDTable, FSE_FUNCTION_EXTENSION)
static size_t FSE_buildDTable
(FSE_DTable* dt, const short* normalizedCounter, unsigned maxSymbolValue, unsigned tableLog)
{
void* ptr = dt;
+2 -2
View File
@@ -1266,11 +1266,11 @@ typedef U32 DTable_max_t[FSE_DTABLE_SIZE_U32(FSE_MAX_TABLELOG)];
/* Function templates */
#define FSE_DECODE_TYPE FSE_TYPE_NAME(FSE_decode_t, FSE_FUNCTION_EXTENSION)
#define FSE_DECODE_TYPE FSE_decode_t
static U32 FSE_tableStep(U32 tableSize) { return (tableSize>>1) + (tableSize>>3) + 3; }
static size_t FSE_FUNCTION_NAME(FSE_buildDTable, FSE_FUNCTION_EXTENSION)
static size_t FSE_buildDTable
(FSE_DTable* dt, const short* normalizedCounter, unsigned maxSymbolValue, unsigned tableLog)
{
void* ptr = dt+1;
+3 -361
View File
@@ -991,7 +991,6 @@ extern "C" {
******************************************/
static size_t HUF_decompress4X2 (void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize); /* single-symbol decoder */
static size_t HUF_decompress4X4 (void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize); /* double-symbols decoder */
static size_t HUF_decompress4X6 (void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize); /* quad-symbols decoder */
#if defined (__cplusplus)
@@ -1266,11 +1265,11 @@ typedef U32 DTable_max_t[FSE_DTABLE_SIZE_U32(FSE_MAX_TABLELOG)];
/* Function templates */
#define FSE_DECODE_TYPE FSE_TYPE_NAME(FSE_decode_t, FSE_FUNCTION_EXTENSION)
#define FSE_DECODE_TYPE FSE_decode_t
static U32 FSE_tableStep(U32 tableSize) { return (tableSize>>1) + (tableSize>>3) + 3; }
static size_t FSE_FUNCTION_NAME(FSE_buildDTable, FSE_FUNCTION_EXTENSION)
static size_t FSE_buildDTable
(FSE_DTable* dt, const short* normalizedCounter, unsigned maxSymbolValue, unsigned tableLog)
{
void* ptr = dt+1;
@@ -2371,362 +2370,6 @@ static size_t HUF_decompress4X4 (void* dst, size_t dstSize, const void* cSrc, si
}
/**********************************/
/* quad-symbol decoding */
/**********************************/
typedef struct { BYTE nbBits; BYTE nbBytes; } HUF_DDescX6;
typedef union { BYTE byte[4]; U32 sequence; } HUF_DSeqX6;
/* recursive, up to level 3; may benefit from <template>-like strategy to nest each level inline */
static void HUF_fillDTableX6LevelN(HUF_DDescX6* DDescription, HUF_DSeqX6* DSequence, int sizeLog,
const rankVal_t rankValOrigin, const U32 consumed, const int minWeight, const U32 maxWeight,
const sortedSymbol_t* sortedSymbols, const U32 sortedListSize, const U32* rankStart,
const U32 nbBitsBaseline, HUF_DSeqX6 baseSeq, HUF_DDescX6 DDesc)
{
const int scaleLog = nbBitsBaseline - sizeLog; /* note : targetLog >= (nbBitsBaseline-1), hence scaleLog <= 1 */
const int minBits = nbBitsBaseline - maxWeight;
const U32 level = DDesc.nbBytes;
U32 rankVal[HUF_ABSOLUTEMAX_TABLELOG + 1];
U32 symbolStartPos, s;
/* local rankVal, will be modified */
memcpy(rankVal, rankValOrigin[consumed], sizeof(rankVal));
/* fill skipped values */
if (minWeight>1)
{
U32 i;
const U32 skipSize = rankVal[minWeight];
for (i = 0; i < skipSize; i++)
{
DSequence[i] = baseSeq;
DDescription[i] = DDesc;
}
}
/* fill DTable */
DDesc.nbBytes++;
symbolStartPos = rankStart[minWeight];
for (s=symbolStartPos; s<sortedListSize; s++)
{
const BYTE symbol = sortedSymbols[s].symbol;
const U32 weight = sortedSymbols[s].weight; /* >= 1 (sorted) */
const int nbBits = nbBitsBaseline - weight; /* >= 1 (by construction) */
const int totalBits = consumed+nbBits;
const U32 start = rankVal[weight];
const U32 length = 1 << (sizeLog-nbBits);
baseSeq.byte[level] = symbol;
DDesc.nbBits = (BYTE)totalBits;
if ((level<3) && (sizeLog-totalBits >= minBits)) /* enough room for another symbol */
{
int nextMinWeight = totalBits + scaleLog;
if (nextMinWeight < 1) nextMinWeight = 1;
HUF_fillDTableX6LevelN(DDescription+start, DSequence+start, sizeLog-nbBits,
rankValOrigin, totalBits, nextMinWeight, maxWeight,
sortedSymbols, sortedListSize, rankStart,
nbBitsBaseline, baseSeq, DDesc); /* recursive (max : level 3) */
}
else
{
U32 i;
const U32 end = start + length;
for (i = start; i < end; i++)
{
DDescription[i] = DDesc;
DSequence[i] = baseSeq;
}
}
rankVal[weight] += length;
}
}
/* note : same preparation as X4 */
static size_t HUF_readDTableX6 (U32* DTable, const void* src, size_t srcSize)
{
BYTE weightList[HUF_MAX_SYMBOL_VALUE + 1];
sortedSymbol_t sortedSymbol[HUF_MAX_SYMBOL_VALUE + 1];
U32 rankStats[HUF_ABSOLUTEMAX_TABLELOG + 1] = { 0 };
U32 rankStart0[HUF_ABSOLUTEMAX_TABLELOG + 2] = { 0 };
U32* const rankStart = rankStart0+1;
U32 tableLog, maxW, sizeOfSort, nbSymbols;
rankVal_t rankVal;
const U32 memLog = DTable[0];
const BYTE* ip = (const BYTE*) src;
size_t iSize = ip[0];
if (memLog > HUF_ABSOLUTEMAX_TABLELOG) return ERROR(tableLog_tooLarge);
//memset(weightList, 0, sizeof(weightList)); /* is not necessary, even though some analyzer complain ... */
iSize = HUF_readStats(weightList, HUF_MAX_SYMBOL_VALUE + 1, rankStats, &nbSymbols, &tableLog, src, srcSize);
if (HUF_isError(iSize)) return iSize;
/* check result */
if (tableLog > memLog) return ERROR(tableLog_tooLarge); /* DTable is too small */
/* find maxWeight */
for (maxW = tableLog; rankStats[maxW]==0; maxW--)
{ if (!maxW) return ERROR(GENERIC); } /* necessarily finds a solution before maxW==0 */
/* Get start index of each weight */
{
U32 w, nextRankStart = 0;
for (w=1; w<=maxW; w++)
{
U32 current = nextRankStart;
nextRankStart += rankStats[w];
rankStart[w] = current;
}
rankStart[0] = nextRankStart; /* put all 0w symbols at the end of sorted list*/
sizeOfSort = nextRankStart;
}
/* sort symbols by weight */
{
U32 s;
for (s=0; s<nbSymbols; s++)
{
U32 w = weightList[s];
U32 r = rankStart[w]++;
sortedSymbol[r].symbol = (BYTE)s;
sortedSymbol[r].weight = (BYTE)w;
}
rankStart[0] = 0; /* forget 0w symbols; this is beginning of weight(1) */
}
/* Build rankVal */
{
const U32 minBits = tableLog+1 - maxW;
U32 nextRankVal = 0;
U32 w, consumed;
const int rescale = (memLog-tableLog) - 1; /* tableLog <= memLog */
U32* rankVal0 = rankVal[0];
for (w=1; w<=maxW; w++)
{
U32 current = nextRankVal;
nextRankVal += rankStats[w] << (w+rescale);
rankVal0[w] = current;
}
for (consumed = minBits; consumed <= memLog - minBits; consumed++)
{
U32* rankValPtr = rankVal[consumed];
for (w = 1; w <= maxW; w++)
{
rankValPtr[w] = rankVal0[w] >> consumed;
}
}
}
/* fill tables */
{
void* ddPtr = DTable+1;
HUF_DDescX6* DDescription = (HUF_DDescX6*)(ddPtr);
void* dsPtr = DTable + 1 + ((size_t)1<<(memLog-1));
HUF_DSeqX6* DSequence = (HUF_DSeqX6*)(dsPtr);
HUF_DSeqX6 DSeq;
HUF_DDescX6 DDesc;
DSeq.sequence = 0;
DDesc.nbBits = 0;
DDesc.nbBytes = 0;
HUF_fillDTableX6LevelN(DDescription, DSequence, memLog,
(const U32 (*)[HUF_ABSOLUTEMAX_TABLELOG + 1])rankVal, 0, 1, maxW,
sortedSymbol, sizeOfSort, rankStart0,
tableLog+1, DSeq, DDesc);
}
return iSize;
}
static U32 HUF_decodeSymbolX6(void* op, BIT_DStream_t* DStream, const HUF_DDescX6* dd, const HUF_DSeqX6* ds, const U32 dtLog)
{
const size_t val = BIT_lookBitsFast(DStream, dtLog); /* note : dtLog >= 1 */
memcpy(op, ds+val, sizeof(HUF_DSeqX6));
BIT_skipBits(DStream, dd[val].nbBits);
return dd[val].nbBytes;
}
static U32 HUF_decodeLastSymbolsX6(void* op, const U32 maxL, BIT_DStream_t* DStream,
const HUF_DDescX6* dd, const HUF_DSeqX6* ds, const U32 dtLog)
{
const size_t val = BIT_lookBitsFast(DStream, dtLog); /* note : dtLog >= 1 */
U32 length = dd[val].nbBytes;
if (length <= maxL)
{
memcpy(op, ds+val, length);
BIT_skipBits(DStream, dd[val].nbBits);
return length;
}
memcpy(op, ds+val, maxL);
if (DStream->bitsConsumed < (sizeof(DStream->bitContainer)*8))
{
BIT_skipBits(DStream, dd[val].nbBits);
if (DStream->bitsConsumed > (sizeof(DStream->bitContainer)*8))
DStream->bitsConsumed = (sizeof(DStream->bitContainer)*8); /* ugly hack; works only because it's the last symbol. Note : can't easily extract nbBits from just this symbol */
}
return maxL;
}
#define HUF_DECODE_SYMBOLX6_0(ptr, DStreamPtr) \
ptr += HUF_decodeSymbolX6(ptr, DStreamPtr, dd, ds, dtLog)
#define HUF_DECODE_SYMBOLX6_1(ptr, DStreamPtr) \
if (MEM_64bits() || (HUF_MAX_TABLELOG<=12)) \
HUF_DECODE_SYMBOLX6_0(ptr, DStreamPtr)
#define HUF_DECODE_SYMBOLX6_2(ptr, DStreamPtr) \
if (MEM_64bits()) \
HUF_DECODE_SYMBOLX6_0(ptr, DStreamPtr)
static inline size_t HUF_decodeStreamX6(BYTE* p, BIT_DStream_t* bitDPtr, BYTE* const pEnd, const U32* DTable, const U32 dtLog)
{
const void* ddPtr = DTable+1;
const HUF_DDescX6* dd = (const HUF_DDescX6*)(ddPtr);
const void* dsPtr = DTable + 1 + ((size_t)1<<(dtLog-1));
const HUF_DSeqX6* ds = (const HUF_DSeqX6*)(dsPtr);
BYTE* const pStart = p;
/* up to 16 symbols at a time */
while ((BIT_reloadDStream(bitDPtr) == BIT_DStream_unfinished) && (p <= pEnd-16))
{
HUF_DECODE_SYMBOLX6_2(p, bitDPtr);
HUF_DECODE_SYMBOLX6_1(p, bitDPtr);
HUF_DECODE_SYMBOLX6_2(p, bitDPtr);
HUF_DECODE_SYMBOLX6_0(p, bitDPtr);
}
/* closer to the end, up to 4 symbols at a time */
while ((BIT_reloadDStream(bitDPtr) == BIT_DStream_unfinished) && (p <= pEnd-4))
HUF_DECODE_SYMBOLX6_0(p, bitDPtr);
while (p <= pEnd-4)
HUF_DECODE_SYMBOLX6_0(p, bitDPtr); /* no need to reload : reached the end of DStream */
while (p < pEnd)
p += HUF_decodeLastSymbolsX6(p, (U32)(pEnd-p), bitDPtr, dd, ds, dtLog);
return p-pStart;
}
static size_t HUF_decompress4X6_usingDTable(
void* dst, size_t dstSize,
const void* cSrc, size_t cSrcSize,
const U32* DTable)
{
if (cSrcSize < 10) return ERROR(corruption_detected); /* strict minimum : jump table + 1 byte per stream */
{
const BYTE* const istart = (const BYTE*) cSrc;
BYTE* const ostart = (BYTE*) dst;
BYTE* const oend = ostart + dstSize;
const U32 dtLog = DTable[0];
const void* ddPtr = DTable+1;
const HUF_DDescX6* dd = (const HUF_DDescX6*)(ddPtr);
const void* dsPtr = DTable + 1 + ((size_t)1<<(dtLog-1));
const HUF_DSeqX6* ds = (const HUF_DSeqX6*)(dsPtr);
size_t errorCode;
/* Init */
BIT_DStream_t bitD1;
BIT_DStream_t bitD2;
BIT_DStream_t bitD3;
BIT_DStream_t bitD4;
const size_t length1 = MEM_readLE16(istart);
const size_t length2 = MEM_readLE16(istart+2);
const size_t length3 = MEM_readLE16(istart+4);
size_t length4;
const BYTE* const istart1 = istart + 6; /* jumpTable */
const BYTE* const istart2 = istart1 + length1;
const BYTE* const istart3 = istart2 + length2;
const BYTE* const istart4 = istart3 + length3;
const size_t segmentSize = (dstSize+3) / 4;
BYTE* const opStart2 = ostart + segmentSize;
BYTE* const opStart3 = opStart2 + segmentSize;
BYTE* const opStart4 = opStart3 + segmentSize;
BYTE* op1 = ostart;
BYTE* op2 = opStart2;
BYTE* op3 = opStart3;
BYTE* op4 = opStart4;
U32 endSignal;
length4 = cSrcSize - (length1 + length2 + length3 + 6);
if (length4 > cSrcSize) return ERROR(corruption_detected); /* overflow */
errorCode = BIT_initDStream(&bitD1, istart1, length1);
if (HUF_isError(errorCode)) return errorCode;
errorCode = BIT_initDStream(&bitD2, istart2, length2);
if (HUF_isError(errorCode)) return errorCode;
errorCode = BIT_initDStream(&bitD3, istart3, length3);
if (HUF_isError(errorCode)) return errorCode;
errorCode = BIT_initDStream(&bitD4, istart4, length4);
if (HUF_isError(errorCode)) return errorCode;
/* 16-64 symbols per loop (4-16 symbols per stream) */
endSignal = BIT_reloadDStream(&bitD1) | BIT_reloadDStream(&bitD2) | BIT_reloadDStream(&bitD3) | BIT_reloadDStream(&bitD4);
for ( ; (op3 <= opStart4) && (endSignal==BIT_DStream_unfinished) && (op4<=(oend-16)) ; )
{
HUF_DECODE_SYMBOLX6_2(op1, &bitD1);
HUF_DECODE_SYMBOLX6_2(op2, &bitD2);
HUF_DECODE_SYMBOLX6_2(op3, &bitD3);
HUF_DECODE_SYMBOLX6_2(op4, &bitD4);
HUF_DECODE_SYMBOLX6_1(op1, &bitD1);
HUF_DECODE_SYMBOLX6_1(op2, &bitD2);
HUF_DECODE_SYMBOLX6_1(op3, &bitD3);
HUF_DECODE_SYMBOLX6_1(op4, &bitD4);
HUF_DECODE_SYMBOLX6_2(op1, &bitD1);
HUF_DECODE_SYMBOLX6_2(op2, &bitD2);
HUF_DECODE_SYMBOLX6_2(op3, &bitD3);
HUF_DECODE_SYMBOLX6_2(op4, &bitD4);
HUF_DECODE_SYMBOLX6_0(op1, &bitD1);
HUF_DECODE_SYMBOLX6_0(op2, &bitD2);
HUF_DECODE_SYMBOLX6_0(op3, &bitD3);
HUF_DECODE_SYMBOLX6_0(op4, &bitD4);
endSignal = BIT_reloadDStream(&bitD1) | BIT_reloadDStream(&bitD2) | BIT_reloadDStream(&bitD3) | BIT_reloadDStream(&bitD4);
}
/* check corruption */
if (op1 > opStart2) return ERROR(corruption_detected);
if (op2 > opStart3) return ERROR(corruption_detected);
if (op3 > opStart4) return ERROR(corruption_detected);
/* note : op4 supposed already verified within main loop */
/* finish bitStreams one by one */
HUF_decodeStreamX6(op1, &bitD1, opStart2, DTable, dtLog);
HUF_decodeStreamX6(op2, &bitD2, opStart3, DTable, dtLog);
HUF_decodeStreamX6(op3, &bitD3, opStart4, DTable, dtLog);
HUF_decodeStreamX6(op4, &bitD4, oend, DTable, dtLog);
/* check */
endSignal = BIT_endOfDStream(&bitD1) & BIT_endOfDStream(&bitD2) & BIT_endOfDStream(&bitD3) & BIT_endOfDStream(&bitD4);
if (!endSignal) return ERROR(corruption_detected);
/* decoded size */
return dstSize;
}
}
static size_t HUF_decompress4X6 (void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize)
{
HUF_CREATE_STATIC_DTABLEX6(DTable, HUF_MAX_TABLELOG);
const BYTE* ip = (const BYTE*) cSrc;
size_t hSize = HUF_readDTableX6 (DTable, cSrc, cSrcSize);
if (HUF_isError(hSize)) return hSize;
if (hSize >= cSrcSize) return ERROR(srcSize_wrong);
ip += hSize;
cSrcSize -= hSize;
return HUF_decompress4X6_usingDTable (dst, dstSize, ip, cSrcSize, DTable);
}
/**********************************/
/* Generic decompression selector */
/**********************************/
@@ -2757,7 +2400,7 @@ typedef size_t (*decompressionAlgo)(void* dst, size_t dstSize, const void* cSrc,
static size_t HUF_decompress (void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize)
{
static const decompressionAlgo decompress[3] = { HUF_decompress4X2, HUF_decompress4X4, HUF_decompress4X6 };
static const decompressionAlgo decompress[3] = { HUF_decompress4X2, HUF_decompress4X4, NULL };
/* estimate decompression time */
U32 Q;
const U32 D256 = (U32)(dstSize >> 8);
@@ -2779,7 +2422,6 @@ static size_t HUF_decompress (void* dst, size_t dstSize, const void* cSrc, size_
Dtime[1] += Dtime[1] >> 4; Dtime[2] += Dtime[2] >> 3; /* advantage to algorithms using less memory, for cache eviction */
if (Dtime[1] < Dtime[0]) algoNb = 1;
if (Dtime[2] < Dtime[algoNb]) algoNb = 2;
return decompress[algoNb](dst, dstSize, cSrc, cSrcSize);
+1 -360
View File
@@ -2004,11 +2004,6 @@ extern "C" {
#endif
/* ****************************************
* Dependency
******************************************/
#include "huff0.h"
/* ****************************************
* Static allocation macros
@@ -2028,7 +2023,6 @@ extern "C" {
******************************************/
static size_t HUF_decompress4X2 (void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize); /* single-symbol decoder */
static size_t HUF_decompress4X4 (void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize); /* double-symbols decoder */
static size_t HUF_decompress4X6 (void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize); /* quad-symbols decoder */
/* ****************************************
@@ -2043,11 +2037,9 @@ HUF_decompress() does the following:
*/
static size_t HUF_readDTableX2 (unsigned short* DTable, const void* src, size_t srcSize);
static size_t HUF_readDTableX4 (unsigned* DTable, const void* src, size_t srcSize);
static size_t HUF_readDTableX6 (unsigned* DTable, const void* src, size_t srcSize);
static size_t HUF_decompress4X2_usingDTable(void* dst, size_t maxDstSize, const void* cSrc, size_t cSrcSize, const unsigned short* DTable);
static size_t HUF_decompress4X4_usingDTable(void* dst, size_t maxDstSize, const void* cSrc, size_t cSrcSize, const unsigned* DTable);
static size_t HUF_decompress4X6_usingDTable(void* dst, size_t maxDstSize, const void* cSrc, size_t cSrcSize, const unsigned* DTable);
#if defined (__cplusplus)
@@ -2798,356 +2790,6 @@ static size_t HUF_decompress4X4 (void* dst, size_t dstSize, const void* cSrc, si
}
/**********************************/
/* quad-symbol decoding */
/**********************************/
typedef struct { BYTE nbBits; BYTE nbBytes; } HUF_DDescX6;
typedef union { BYTE byte[4]; U32 sequence; } HUF_DSeqX6;
/* recursive, up to level 3; may benefit from <template>-like strategy to nest each level inline */
static void HUF_fillDTableX6LevelN(HUF_DDescX6* DDescription, HUF_DSeqX6* DSequence, int sizeLog,
const rankVal_t rankValOrigin, const U32 consumed, const int minWeight, const U32 maxWeight,
const sortedSymbol_t* sortedSymbols, const U32 sortedListSize, const U32* rankStart,
const U32 nbBitsBaseline, HUF_DSeqX6 baseSeq, HUF_DDescX6 DDesc)
{
const int scaleLog = nbBitsBaseline - sizeLog; /* note : targetLog >= (nbBitsBaseline-1), hence scaleLog <= 1 */
const int minBits = nbBitsBaseline - maxWeight;
const U32 level = DDesc.nbBytes;
U32 rankVal[HUF_ABSOLUTEMAX_TABLELOG + 1];
U32 symbolStartPos, s;
/* local rankVal, will be modified */
memcpy(rankVal, rankValOrigin[consumed], sizeof(rankVal));
/* fill skipped values */
if (minWeight>1)
{
U32 i;
const U32 skipSize = rankVal[minWeight];
for (i = 0; i < skipSize; i++)
{
DSequence[i] = baseSeq;
DDescription[i] = DDesc;
}
}
/* fill DTable */
DDesc.nbBytes++;
symbolStartPos = rankStart[minWeight];
for (s=symbolStartPos; s<sortedListSize; s++)
{
const BYTE symbol = sortedSymbols[s].symbol;
const U32 weight = sortedSymbols[s].weight; /* >= 1 (sorted) */
const int nbBits = nbBitsBaseline - weight; /* >= 1 (by construction) */
const int totalBits = consumed+nbBits;
const U32 start = rankVal[weight];
const U32 length = 1 << (sizeLog-nbBits);
baseSeq.byte[level] = symbol;
DDesc.nbBits = (BYTE)totalBits;
if ((level<3) && (sizeLog-totalBits >= minBits)) /* enough room for another symbol */
{
int nextMinWeight = totalBits + scaleLog;
if (nextMinWeight < 1) nextMinWeight = 1;
HUF_fillDTableX6LevelN(DDescription+start, DSequence+start, sizeLog-nbBits,
rankValOrigin, totalBits, nextMinWeight, maxWeight,
sortedSymbols, sortedListSize, rankStart,
nbBitsBaseline, baseSeq, DDesc); /* recursive (max : level 3) */
}
else
{
U32 i;
const U32 end = start + length;
for (i = start; i < end; i++)
{
DDescription[i] = DDesc;
DSequence[i] = baseSeq;
}
}
rankVal[weight] += length;
}
}
/* note : same preparation as X4 */
static size_t HUF_readDTableX6 (U32* DTable, const void* src, size_t srcSize)
{
BYTE weightList[HUF_MAX_SYMBOL_VALUE + 1];
sortedSymbol_t sortedSymbol[HUF_MAX_SYMBOL_VALUE + 1];
U32 rankStats[HUF_ABSOLUTEMAX_TABLELOG + 1] = { 0 };
U32 rankStart0[HUF_ABSOLUTEMAX_TABLELOG + 2] = { 0 };
U32* const rankStart = rankStart0+1;
U32 tableLog, maxW, sizeOfSort, nbSymbols;
rankVal_t rankVal;
const U32 memLog = DTable[0];
size_t iSize;
if (memLog > HUF_ABSOLUTEMAX_TABLELOG) return ERROR(tableLog_tooLarge);
//memset(weightList, 0, sizeof(weightList)); /* is not necessary, even though some analyzer complain ... */
iSize = HUF_readStats(weightList, HUF_MAX_SYMBOL_VALUE + 1, rankStats, &nbSymbols, &tableLog, src, srcSize);
if (HUF_isError(iSize)) return iSize;
/* check result */
if (tableLog > memLog) return ERROR(tableLog_tooLarge); /* DTable is too small */
/* find maxWeight */
for (maxW = tableLog; rankStats[maxW]==0; maxW--)
{ if (!maxW) return ERROR(GENERIC); } /* necessarily finds a solution before maxW==0 */
/* Get start index of each weight */
{
U32 w, nextRankStart = 0;
for (w=1; w<=maxW; w++)
{
U32 current = nextRankStart;
nextRankStart += rankStats[w];
rankStart[w] = current;
}
rankStart[0] = nextRankStart; /* put all 0w symbols at the end of sorted list*/
sizeOfSort = nextRankStart;
}
/* sort symbols by weight */
{
U32 s;
for (s=0; s<nbSymbols; s++)
{
U32 w = weightList[s];
U32 r = rankStart[w]++;
sortedSymbol[r].symbol = (BYTE)s;
sortedSymbol[r].weight = (BYTE)w;
}
rankStart[0] = 0; /* forget 0w symbols; this is beginning of weight(1) */
}
/* Build rankVal */
{
const U32 minBits = tableLog+1 - maxW;
U32 nextRankVal = 0;
U32 w, consumed;
const int rescale = (memLog-tableLog) - 1; /* tableLog <= memLog */
U32* rankVal0 = rankVal[0];
for (w=1; w<=maxW; w++)
{
U32 current = nextRankVal;
nextRankVal += rankStats[w] << (w+rescale);
rankVal0[w] = current;
}
for (consumed = minBits; consumed <= memLog - minBits; consumed++)
{
U32* rankValPtr = rankVal[consumed];
for (w = 1; w <= maxW; w++)
{
rankValPtr[w] = rankVal0[w] >> consumed;
}
}
}
/* fill tables */
{
void* ddPtr = DTable+1;
HUF_DDescX6* DDescription = (HUF_DDescX6*)ddPtr;
void* dsPtr = DTable + 1 + ((size_t)1<<(memLog-1));
HUF_DSeqX6* DSequence = (HUF_DSeqX6*)dsPtr;
HUF_DSeqX6 DSeq;
HUF_DDescX6 DDesc;
DSeq.sequence = 0;
DDesc.nbBits = 0;
DDesc.nbBytes = 0;
HUF_fillDTableX6LevelN(DDescription, DSequence, memLog,
(const U32 (*)[HUF_ABSOLUTEMAX_TABLELOG + 1])rankVal, 0, 1, maxW,
sortedSymbol, sizeOfSort, rankStart0,
tableLog+1, DSeq, DDesc);
}
return iSize;
}
static U32 HUF_decodeSymbolX6(void* op, BIT_DStream_t* DStream, const HUF_DDescX6* dd, const HUF_DSeqX6* ds, const U32 dtLog)
{
const size_t val = BIT_lookBitsFast(DStream, dtLog); /* note : dtLog >= 1 */
memcpy(op, ds+val, sizeof(HUF_DSeqX6));
BIT_skipBits(DStream, dd[val].nbBits);
return dd[val].nbBytes;
}
static U32 HUF_decodeLastSymbolsX6(void* op, const U32 maxL, BIT_DStream_t* DStream,
const HUF_DDescX6* dd, const HUF_DSeqX6* ds, const U32 dtLog)
{
const size_t val = BIT_lookBitsFast(DStream, dtLog); /* note : dtLog >= 1 */
U32 length = dd[val].nbBytes;
if (length <= maxL)
{
memcpy(op, ds+val, length);
BIT_skipBits(DStream, dd[val].nbBits);
return length;
}
memcpy(op, ds+val, maxL);
if (DStream->bitsConsumed < (sizeof(DStream->bitContainer)*8))
{
BIT_skipBits(DStream, dd[val].nbBits);
if (DStream->bitsConsumed > (sizeof(DStream->bitContainer)*8))
DStream->bitsConsumed = (sizeof(DStream->bitContainer)*8); /* ugly hack; works only because it's the last symbol. Note : can't easily extract nbBits from just this symbol */
}
return maxL;
}
#define HUF_DECODE_SYMBOLX6_0(ptr, DStreamPtr) \
ptr += HUF_decodeSymbolX6(ptr, DStreamPtr, dd, ds, dtLog)
#define HUF_DECODE_SYMBOLX6_1(ptr, DStreamPtr) \
if (MEM_64bits() || (HUF_MAX_TABLELOG<=12)) \
HUF_DECODE_SYMBOLX6_0(ptr, DStreamPtr)
#define HUF_DECODE_SYMBOLX6_2(ptr, DStreamPtr) \
if (MEM_64bits()) \
HUF_DECODE_SYMBOLX6_0(ptr, DStreamPtr)
static inline size_t HUF_decodeStreamX6(BYTE* p, BIT_DStream_t* bitDPtr, BYTE* const pEnd, const U32* DTable, const U32 dtLog)
{
const void* const ddPtr = DTable+1;
const HUF_DDescX6* dd = (const HUF_DDescX6*)ddPtr;
const void* const dsPtr = DTable + 1 + ((size_t)1<<(dtLog-1));
const HUF_DSeqX6* ds = (const HUF_DSeqX6*)dsPtr;
BYTE* const pStart = p;
/* up to 16 symbols at a time */
while ((BIT_reloadDStream(bitDPtr) == BIT_DStream_unfinished) && (p <= pEnd-16))
{
HUF_DECODE_SYMBOLX6_2(p, bitDPtr);
HUF_DECODE_SYMBOLX6_1(p, bitDPtr);
HUF_DECODE_SYMBOLX6_2(p, bitDPtr);
HUF_DECODE_SYMBOLX6_0(p, bitDPtr);
}
/* closer to the end, up to 4 symbols at a time */
while ((BIT_reloadDStream(bitDPtr) == BIT_DStream_unfinished) && (p <= pEnd-4))
HUF_DECODE_SYMBOLX6_0(p, bitDPtr);
while ((BIT_reloadDStream(bitDPtr) <= BIT_DStream_endOfBuffer) && (p < pEnd))
p += HUF_decodeLastSymbolsX6(p, (U32)(pEnd-p), bitDPtr, dd, ds, dtLog);
return p-pStart;
}
static size_t HUF_decompress4X6_usingDTable(
void* dst, size_t dstSize,
const void* cSrc, size_t cSrcSize,
const U32* DTable)
{
if (cSrcSize < 10) return ERROR(corruption_detected); /* strict minimum : jump table + 1 byte per stream */
{
const BYTE* const istart = (const BYTE*) cSrc;
BYTE* const ostart = (BYTE*) dst;
BYTE* const oend = ostart + dstSize;
const U32 dtLog = DTable[0];
const void* const ddPtr = DTable+1;
const HUF_DDescX6* dd = (const HUF_DDescX6*)ddPtr;
const void* const dsPtr = DTable + 1 + ((size_t)1<<(dtLog-1));
const HUF_DSeqX6* ds = (const HUF_DSeqX6*)dsPtr;
size_t errorCode;
/* Init */
BIT_DStream_t bitD1;
BIT_DStream_t bitD2;
BIT_DStream_t bitD3;
BIT_DStream_t bitD4;
const size_t length1 = MEM_readLE16(istart);
const size_t length2 = MEM_readLE16(istart+2);
const size_t length3 = MEM_readLE16(istart+4);
size_t length4;
const BYTE* const istart1 = istart + 6; /* jumpTable */
const BYTE* const istart2 = istart1 + length1;
const BYTE* const istart3 = istart2 + length2;
const BYTE* const istart4 = istart3 + length3;
const size_t segmentSize = (dstSize+3) / 4;
BYTE* const opStart2 = ostart + segmentSize;
BYTE* const opStart3 = opStart2 + segmentSize;
BYTE* const opStart4 = opStart3 + segmentSize;
BYTE* op1 = ostart;
BYTE* op2 = opStart2;
BYTE* op3 = opStart3;
BYTE* op4 = opStart4;
U32 endSignal;
length4 = cSrcSize - (length1 + length2 + length3 + 6);
if (length4 > cSrcSize) return ERROR(corruption_detected); /* overflow */
errorCode = BIT_initDStream(&bitD1, istart1, length1);
if (HUF_isError(errorCode)) return errorCode;
errorCode = BIT_initDStream(&bitD2, istart2, length2);
if (HUF_isError(errorCode)) return errorCode;
errorCode = BIT_initDStream(&bitD3, istart3, length3);
if (HUF_isError(errorCode)) return errorCode;
errorCode = BIT_initDStream(&bitD4, istart4, length4);
if (HUF_isError(errorCode)) return errorCode;
/* 16-64 symbols per loop (4-16 symbols per stream) */
endSignal = BIT_reloadDStream(&bitD1) | BIT_reloadDStream(&bitD2) | BIT_reloadDStream(&bitD3) | BIT_reloadDStream(&bitD4);
for ( ; (op3 <= opStart4) && (endSignal==BIT_DStream_unfinished) && (op4<=(oend-16)) ; )
{
HUF_DECODE_SYMBOLX6_2(op1, &bitD1);
HUF_DECODE_SYMBOLX6_2(op2, &bitD2);
HUF_DECODE_SYMBOLX6_2(op3, &bitD3);
HUF_DECODE_SYMBOLX6_2(op4, &bitD4);
HUF_DECODE_SYMBOLX6_1(op1, &bitD1);
HUF_DECODE_SYMBOLX6_1(op2, &bitD2);
HUF_DECODE_SYMBOLX6_1(op3, &bitD3);
HUF_DECODE_SYMBOLX6_1(op4, &bitD4);
HUF_DECODE_SYMBOLX6_2(op1, &bitD1);
HUF_DECODE_SYMBOLX6_2(op2, &bitD2);
HUF_DECODE_SYMBOLX6_2(op3, &bitD3);
HUF_DECODE_SYMBOLX6_2(op4, &bitD4);
HUF_DECODE_SYMBOLX6_0(op1, &bitD1);
HUF_DECODE_SYMBOLX6_0(op2, &bitD2);
HUF_DECODE_SYMBOLX6_0(op3, &bitD3);
HUF_DECODE_SYMBOLX6_0(op4, &bitD4);
endSignal = BIT_reloadDStream(&bitD1) | BIT_reloadDStream(&bitD2) | BIT_reloadDStream(&bitD3) | BIT_reloadDStream(&bitD4);
}
/* check corruption */
if (op1 > opStart2) return ERROR(corruption_detected);
if (op2 > opStart3) return ERROR(corruption_detected);
if (op3 > opStart4) return ERROR(corruption_detected);
/* note : op4 supposed already verified within main loop */
/* finish bitStreams one by one */
HUF_decodeStreamX6(op1, &bitD1, opStart2, DTable, dtLog);
HUF_decodeStreamX6(op2, &bitD2, opStart3, DTable, dtLog);
HUF_decodeStreamX6(op3, &bitD3, opStart4, DTable, dtLog);
HUF_decodeStreamX6(op4, &bitD4, oend, DTable, dtLog);
/* check */
endSignal = BIT_endOfDStream(&bitD1) & BIT_endOfDStream(&bitD2) & BIT_endOfDStream(&bitD3) & BIT_endOfDStream(&bitD4);
if (!endSignal) return ERROR(corruption_detected);
/* decoded size */
return dstSize;
}
}
static size_t HUF_decompress4X6 (void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize)
{
HUF_CREATE_STATIC_DTABLEX6(DTable, HUF_MAX_TABLELOG);
const BYTE* ip = (const BYTE*) cSrc;
size_t hSize = HUF_readDTableX6 (DTable, cSrc, cSrcSize);
if (HUF_isError(hSize)) return hSize;
if (hSize >= cSrcSize) return ERROR(srcSize_wrong);
ip += hSize;
cSrcSize -= hSize;
return HUF_decompress4X6_usingDTable (dst, dstSize, ip, cSrcSize, DTable);
}
/**********************************/
/* Generic decompression selector */
/**********************************/
@@ -3178,7 +2820,7 @@ typedef size_t (*decompressionAlgo)(void* dst, size_t dstSize, const void* cSrc,
static size_t HUF_decompress (void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize)
{
static const decompressionAlgo decompress[3] = { HUF_decompress4X2, HUF_decompress4X4, HUF_decompress4X6 };
static const decompressionAlgo decompress[3] = { HUF_decompress4X2, HUF_decompress4X4, NULL };
/* estimate decompression time */
U32 Q;
const U32 D256 = (U32)(dstSize >> 8);
@@ -3200,7 +2842,6 @@ static size_t HUF_decompress (void* dst, size_t dstSize, const void* cSrc, size_
Dtime[1] += Dtime[1] >> 4; Dtime[2] += Dtime[2] >> 3; /* advantage to algorithms using less memory, for cache eviction */
if (Dtime[1] < Dtime[0]) algoNb = 1;
if (Dtime[2] < Dtime[algoNb]) algoNb = 2;
return decompress[algoNb](dst, dstSize, cSrc, cSrcSize);
+6 -384
View File
@@ -2023,7 +2023,6 @@ extern "C" {
******************************************/
size_t HUFv05_decompress4X2 (void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize); /* single-symbol decoder */
size_t HUFv05_decompress4X4 (void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize); /* double-symbols decoder */
size_t HUFv05_decompress4X6 (void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize); /* quad-symbols decoder */
/* ****************************************
@@ -2037,22 +2036,18 @@ HUFv05_decompress() does the following:
*/
size_t HUFv05_readDTableX2 (unsigned short* DTable, const void* src, size_t srcSize);
size_t HUFv05_readDTableX4 (unsigned* DTable, const void* src, size_t srcSize);
size_t HUFv05_readDTableX6 (unsigned* DTable, const void* src, size_t srcSize);
size_t HUFv05_decompress4X2_usingDTable(void* dst, size_t maxDstSize, const void* cSrc, size_t cSrcSize, const unsigned short* DTable);
size_t HUFv05_decompress4X4_usingDTable(void* dst, size_t maxDstSize, const void* cSrc, size_t cSrcSize, const unsigned* DTable);
size_t HUFv05_decompress4X6_usingDTable(void* dst, size_t maxDstSize, const void* cSrc, size_t cSrcSize, const unsigned* DTable);
/* single stream variants */
size_t HUFv05_decompress1X2 (void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize); /* single-symbol decoder */
size_t HUFv05_decompress1X4 (void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize); /* double-symbol decoder */
size_t HUFv05_decompress1X6 (void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize); /* quad-symbol decoder */
size_t HUFv05_decompress1X2_usingDTable(void* dst, size_t maxDstSize, const void* cSrc, size_t cSrcSize, const unsigned short* DTable);
size_t HUFv05_decompress1X4_usingDTable(void* dst, size_t maxDstSize, const void* cSrc, size_t cSrcSize, const unsigned* DTable);
size_t HUFv05_decompress1X6_usingDTable(void* dst, size_t maxDstSize, const void* cSrc, size_t cSrcSize, const unsigned* DTable);
@@ -2372,9 +2367,6 @@ size_t HUFv05_decompress4X2_usingDTable(
const U32 dtLog = DTable[0];
size_t errorCode;
/* Check */
if (cSrcSize < 10) return ERROR(corruption_detected); /* strict minimum : jump table + 1 byte per stream */
/* Init */
BITv05_DStream_t bitD1;
BITv05_DStream_t bitD2;
@@ -2398,6 +2390,9 @@ size_t HUFv05_decompress4X2_usingDTable(
BYTE* op4 = opStart4;
U32 endSignal;
/* Check */
if (cSrcSize < 10) return ERROR(corruption_detected); /* strict minimum : jump table + 1 byte per stream */
length4 = cSrcSize - (length1 + length2 + length3 + 6);
if (length4 > cSrcSize) return ERROR(corruption_detected); /* overflow */
errorCode = BITv05_initDStream(&bitD1, istart1, length1);
@@ -2850,376 +2845,6 @@ size_t HUFv05_decompress4X4 (void* dst, size_t dstSize, const void* cSrc, size_t
}
/* ********************************/
/* quad-symbol decoding */
/* ********************************/
typedef struct { BYTE nbBits; BYTE nbBytes; } HUFv05_DDescX6;
typedef union { BYTE byte[4]; U32 sequence; } HUFv05_DSeqX6;
/* recursive, up to level 3; may benefit from <template>-like strategy to nest each level inline */
static void HUFv05_fillDTableX6LevelN(HUFv05_DDescX6* DDescription, HUFv05_DSeqX6* DSequence, int sizeLog,
const rankVal_t rankValOrigin, const U32 consumed, const int minWeight, const U32 maxWeight,
const sortedSymbol_t* sortedSymbols, const U32 sortedListSize, const U32* rankStart,
const U32 nbBitsBaseline, HUFv05_DSeqX6 baseSeq, HUFv05_DDescX6 DDesc)
{
const int scaleLog = nbBitsBaseline - sizeLog; /* note : targetLog >= (nbBitsBaseline-1), hence scaleLog <= 1 */
const int minBits = nbBitsBaseline - maxWeight;
const U32 level = DDesc.nbBytes;
U32 rankVal[HUFv05_ABSOLUTEMAX_TABLELOG + 1];
U32 symbolStartPos, s;
/* local rankVal, will be modified */
memcpy(rankVal, rankValOrigin[consumed], sizeof(rankVal));
/* fill skipped values */
if (minWeight>1) {
U32 i;
const U32 skipSize = rankVal[minWeight];
for (i = 0; i < skipSize; i++) {
DSequence[i] = baseSeq;
DDescription[i] = DDesc;
} }
/* fill DTable */
DDesc.nbBytes++;
symbolStartPos = rankStart[minWeight];
for (s=symbolStartPos; s<sortedListSize; s++) {
const BYTE symbol = sortedSymbols[s].symbol;
const U32 weight = sortedSymbols[s].weight; /* >= 1 (sorted) */
const int nbBits = nbBitsBaseline - weight; /* >= 1 (by construction) */
const int totalBits = consumed+nbBits;
const U32 start = rankVal[weight];
const U32 length = 1 << (sizeLog-nbBits);
baseSeq.byte[level] = symbol;
DDesc.nbBits = (BYTE)totalBits;
if ((level<3) && (sizeLog-totalBits >= minBits)) { /* enough room for another symbol */
int nextMinWeight = totalBits + scaleLog;
if (nextMinWeight < 1) nextMinWeight = 1;
HUFv05_fillDTableX6LevelN(DDescription+start, DSequence+start, sizeLog-nbBits,
rankValOrigin, totalBits, nextMinWeight, maxWeight,
sortedSymbols, sortedListSize, rankStart,
nbBitsBaseline, baseSeq, DDesc); /* recursive (max : level 3) */
} else {
U32 i;
const U32 end = start + length;
for (i = start; i < end; i++) {
DDescription[i] = DDesc;
DSequence[i] = baseSeq;
} }
rankVal[weight] += length;
}
}
/* note : same preparation as X4 */
size_t HUFv05_readDTableX6 (U32* DTable, const void* src, size_t srcSize)
{
BYTE weightList[HUFv05_MAX_SYMBOL_VALUE + 1];
sortedSymbol_t sortedSymbol[HUFv05_MAX_SYMBOL_VALUE + 1];
U32 rankStats[HUFv05_ABSOLUTEMAX_TABLELOG + 1] = { 0 };
U32 rankStart0[HUFv05_ABSOLUTEMAX_TABLELOG + 2] = { 0 };
U32* const rankStart = rankStart0+1;
U32 tableLog, maxW, sizeOfSort, nbSymbols;
rankVal_t rankVal;
const U32 memLog = DTable[0];
size_t iSize;
if (memLog > HUFv05_ABSOLUTEMAX_TABLELOG) return ERROR(tableLog_tooLarge);
//memset(weightList, 0, sizeof(weightList)); /* is not necessary, even though some analyzer complain ... */
iSize = HUFv05_readStats(weightList, HUFv05_MAX_SYMBOL_VALUE + 1, rankStats, &nbSymbols, &tableLog, src, srcSize);
if (HUFv05_isError(iSize)) return iSize;
/* check result */
if (tableLog > memLog) return ERROR(tableLog_tooLarge); /* DTable is too small */
/* find maxWeight */
for (maxW = tableLog; rankStats[maxW]==0; maxW--) {} /* necessarily finds a solution before 0 */
/* Get start index of each weight */
{
U32 w, nextRankStart = 0;
for (w=1; w<=maxW; w++) {
U32 current = nextRankStart;
nextRankStart += rankStats[w];
rankStart[w] = current;
}
rankStart[0] = nextRankStart; /* put all 0w symbols at the end of sorted list*/
sizeOfSort = nextRankStart;
}
/* sort symbols by weight */
{
U32 s;
for (s=0; s<nbSymbols; s++) {
U32 w = weightList[s];
U32 r = rankStart[w]++;
sortedSymbol[r].symbol = (BYTE)s;
sortedSymbol[r].weight = (BYTE)w;
}
rankStart[0] = 0; /* forget 0w symbols; this is beginning of weight(1) */
}
/* Build rankVal */
{
const U32 minBits = tableLog+1 - maxW;
U32 nextRankVal = 0;
U32 w, consumed;
const int rescale = (memLog-tableLog) - 1; /* tableLog <= memLog */
U32* rankVal0 = rankVal[0];
for (w=1; w<=maxW; w++) {
U32 current = nextRankVal;
nextRankVal += rankStats[w] << (w+rescale);
rankVal0[w] = current;
}
for (consumed = minBits; consumed <= memLog - minBits; consumed++) {
U32* rankValPtr = rankVal[consumed];
for (w = 1; w <= maxW; w++) {
rankValPtr[w] = rankVal0[w] >> consumed;
} } }
/* fill tables */
{
void* ddPtr = DTable+1;
HUFv05_DDescX6* DDescription = (HUFv05_DDescX6*)ddPtr;
void* dsPtr = DTable + 1 + ((size_t)1<<(memLog-1));
HUFv05_DSeqX6* DSequence = (HUFv05_DSeqX6*)dsPtr;
HUFv05_DSeqX6 DSeq;
HUFv05_DDescX6 DDesc;
DSeq.sequence = 0;
DDesc.nbBits = 0;
DDesc.nbBytes = 0;
HUFv05_fillDTableX6LevelN(DDescription, DSequence, memLog,
(const U32 (*)[HUFv05_ABSOLUTEMAX_TABLELOG + 1])rankVal, 0, 1, maxW,
sortedSymbol, sizeOfSort, rankStart0,
tableLog+1, DSeq, DDesc);
}
return iSize;
}
static U32 HUFv05_decodeSymbolX6(void* op, BITv05_DStream_t* DStream, const HUFv05_DDescX6* dd, const HUFv05_DSeqX6* ds, const U32 dtLog)
{
const size_t val = BITv05_lookBitsFast(DStream, dtLog); /* note : dtLog >= 1 */
memcpy(op, ds+val, sizeof(HUFv05_DSeqX6));
BITv05_skipBits(DStream, dd[val].nbBits);
return dd[val].nbBytes;
}
static U32 HUFv05_decodeLastSymbolsX6(void* op, const U32 maxL, BITv05_DStream_t* DStream,
const HUFv05_DDescX6* dd, const HUFv05_DSeqX6* ds, const U32 dtLog)
{
const size_t val = BITv05_lookBitsFast(DStream, dtLog); /* note : dtLog >= 1 */
U32 length = dd[val].nbBytes;
if (length <= maxL) {
memcpy(op, ds+val, length);
BITv05_skipBits(DStream, dd[val].nbBits);
return length;
}
memcpy(op, ds+val, maxL);
if (DStream->bitsConsumed < (sizeof(DStream->bitContainer)*8)) {
BITv05_skipBits(DStream, dd[val].nbBits);
if (DStream->bitsConsumed > (sizeof(DStream->bitContainer)*8))
DStream->bitsConsumed = (sizeof(DStream->bitContainer)*8); /* ugly hack; works only because it's the last symbol. Note : can't easily extract nbBits from just this symbol */
}
return maxL;
}
#define HUFv05_DECODE_SYMBOLX6_0(ptr, DStreamPtr) \
ptr += HUFv05_decodeSymbolX6(ptr, DStreamPtr, dd, ds, dtLog)
#define HUFv05_DECODE_SYMBOLX6_1(ptr, DStreamPtr) \
if (MEM_64bits() || (HUFv05_MAX_TABLELOG<=12)) \
HUFv05_DECODE_SYMBOLX6_0(ptr, DStreamPtr)
#define HUFv05_DECODE_SYMBOLX6_2(ptr, DStreamPtr) \
if (MEM_64bits()) \
HUFv05_DECODE_SYMBOLX6_0(ptr, DStreamPtr)
static inline size_t HUFv05_decodeStreamX6(BYTE* p, BITv05_DStream_t* bitDPtr, BYTE* const pEnd, const U32* DTable, const U32 dtLog)
{
const void* const ddPtr = DTable+1;
const HUFv05_DDescX6* dd = (const HUFv05_DDescX6*)ddPtr;
const void* const dsPtr = DTable + 1 + ((size_t)1<<(dtLog-1));
const HUFv05_DSeqX6* ds = (const HUFv05_DSeqX6*)dsPtr;
BYTE* const pStart = p;
/* up to 16 symbols at a time */
while ((BITv05_reloadDStream(bitDPtr) == BITv05_DStream_unfinished) && (p <= pEnd-16)) {
HUFv05_DECODE_SYMBOLX6_2(p, bitDPtr);
HUFv05_DECODE_SYMBOLX6_1(p, bitDPtr);
HUFv05_DECODE_SYMBOLX6_2(p, bitDPtr);
HUFv05_DECODE_SYMBOLX6_0(p, bitDPtr);
}
/* closer to the end, up to 4 symbols at a time */
while ((BITv05_reloadDStream(bitDPtr) == BITv05_DStream_unfinished) && (p <= pEnd-4))
HUFv05_DECODE_SYMBOLX6_0(p, bitDPtr);
while ((BITv05_reloadDStream(bitDPtr) <= BITv05_DStream_endOfBuffer) && (p < pEnd))
p += HUFv05_decodeLastSymbolsX6(p, (U32)(pEnd-p), bitDPtr, dd, ds, dtLog);
return p-pStart;
}
size_t HUFv05_decompress1X6_usingDTable(
void* dst, size_t dstSize,
const void* cSrc, size_t cSrcSize,
const U32* DTable)
{
const BYTE* const istart = (const BYTE*) cSrc;
BYTE* const ostart = (BYTE*) dst;
BYTE* const oend = ostart + dstSize;
const U32 dtLog = DTable[0];
size_t errorCode;
/* Init */
BITv05_DStream_t bitD;
errorCode = BITv05_initDStream(&bitD, istart, cSrcSize);
if (HUFv05_isError(errorCode)) return errorCode;
/* finish bitStreams one by one */
HUFv05_decodeStreamX6(ostart, &bitD, oend, DTable, dtLog);
/* check */
if (!BITv05_endOfDStream(&bitD)) return ERROR(corruption_detected);
/* decoded size */
return dstSize;
}
size_t HUFv05_decompress1X6 (void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize)
{
HUFv05_CREATE_STATIC_DTABLEX6(DTable, HUFv05_MAX_TABLELOG);
const BYTE* ip = (const BYTE*) cSrc;
size_t hSize = HUFv05_readDTableX6 (DTable, cSrc, cSrcSize);
if (HUFv05_isError(hSize)) return hSize;
if (hSize >= cSrcSize) return ERROR(srcSize_wrong);
ip += hSize;
cSrcSize -= hSize;
return HUFv05_decompress1X6_usingDTable (dst, dstSize, ip, cSrcSize, DTable);
}
size_t HUFv05_decompress4X6_usingDTable(
void* dst, size_t dstSize,
const void* cSrc, size_t cSrcSize,
const U32* DTable)
{
const BYTE* const istart = (const BYTE*) cSrc;
BYTE* const ostart = (BYTE*) dst;
BYTE* const oend = ostart + dstSize;
const U32 dtLog = DTable[0];
const void* const ddPtr = DTable+1;
const HUFv05_DDescX6* dd = (const HUFv05_DDescX6*)ddPtr;
const void* const dsPtr = DTable + 1 + ((size_t)1<<(dtLog-1));
const HUFv05_DSeqX6* ds = (const HUFv05_DSeqX6*)dsPtr;
size_t errorCode;
/* Check */
if (cSrcSize < 10) return ERROR(corruption_detected); /* strict minimum : jump table + 1 byte per stream */
/* Init */
BITv05_DStream_t bitD1;
BITv05_DStream_t bitD2;
BITv05_DStream_t bitD3;
BITv05_DStream_t bitD4;
const size_t length1 = MEM_readLE16(istart);
const size_t length2 = MEM_readLE16(istart+2);
const size_t length3 = MEM_readLE16(istart+4);
size_t length4;
const BYTE* const istart1 = istart + 6; /* jumpTable */
const BYTE* const istart2 = istart1 + length1;
const BYTE* const istart3 = istart2 + length2;
const BYTE* const istart4 = istart3 + length3;
const size_t segmentSize = (dstSize+3) / 4;
BYTE* const opStart2 = ostart + segmentSize;
BYTE* const opStart3 = opStart2 + segmentSize;
BYTE* const opStart4 = opStart3 + segmentSize;
BYTE* op1 = ostart;
BYTE* op2 = opStart2;
BYTE* op3 = opStart3;
BYTE* op4 = opStart4;
U32 endSignal;
length4 = cSrcSize - (length1 + length2 + length3 + 6);
if (length4 > cSrcSize) return ERROR(corruption_detected); /* overflow */
errorCode = BITv05_initDStream(&bitD1, istart1, length1);
if (HUFv05_isError(errorCode)) return errorCode;
errorCode = BITv05_initDStream(&bitD2, istart2, length2);
if (HUFv05_isError(errorCode)) return errorCode;
errorCode = BITv05_initDStream(&bitD3, istart3, length3);
if (HUFv05_isError(errorCode)) return errorCode;
errorCode = BITv05_initDStream(&bitD4, istart4, length4);
if (HUFv05_isError(errorCode)) return errorCode;
/* 16-64 symbols per loop (4-16 symbols per stream) */
endSignal = BITv05_reloadDStream(&bitD1) | BITv05_reloadDStream(&bitD2) | BITv05_reloadDStream(&bitD3) | BITv05_reloadDStream(&bitD4);
for ( ; (op3 <= opStart4) && (endSignal==BITv05_DStream_unfinished) && (op4<=(oend-16)) ; ) {
HUFv05_DECODE_SYMBOLX6_2(op1, &bitD1);
HUFv05_DECODE_SYMBOLX6_2(op2, &bitD2);
HUFv05_DECODE_SYMBOLX6_2(op3, &bitD3);
HUFv05_DECODE_SYMBOLX6_2(op4, &bitD4);
HUFv05_DECODE_SYMBOLX6_1(op1, &bitD1);
HUFv05_DECODE_SYMBOLX6_1(op2, &bitD2);
HUFv05_DECODE_SYMBOLX6_1(op3, &bitD3);
HUFv05_DECODE_SYMBOLX6_1(op4, &bitD4);
HUFv05_DECODE_SYMBOLX6_2(op1, &bitD1);
HUFv05_DECODE_SYMBOLX6_2(op2, &bitD2);
HUFv05_DECODE_SYMBOLX6_2(op3, &bitD3);
HUFv05_DECODE_SYMBOLX6_2(op4, &bitD4);
HUFv05_DECODE_SYMBOLX6_0(op1, &bitD1);
HUFv05_DECODE_SYMBOLX6_0(op2, &bitD2);
HUFv05_DECODE_SYMBOLX6_0(op3, &bitD3);
HUFv05_DECODE_SYMBOLX6_0(op4, &bitD4);
endSignal = BITv05_reloadDStream(&bitD1) | BITv05_reloadDStream(&bitD2) | BITv05_reloadDStream(&bitD3) | BITv05_reloadDStream(&bitD4);
}
/* check corruption */
if (op1 > opStart2) return ERROR(corruption_detected);
if (op2 > opStart3) return ERROR(corruption_detected);
if (op3 > opStart4) return ERROR(corruption_detected);
/* note : op4 supposed already verified within main loop */
/* finish bitStreams one by one */
HUFv05_decodeStreamX6(op1, &bitD1, opStart2, DTable, dtLog);
HUFv05_decodeStreamX6(op2, &bitD2, opStart3, DTable, dtLog);
HUFv05_decodeStreamX6(op3, &bitD3, opStart4, DTable, dtLog);
HUFv05_decodeStreamX6(op4, &bitD4, oend, DTable, dtLog);
/* check */
endSignal = BITv05_endOfDStream(&bitD1) & BITv05_endOfDStream(&bitD2) & BITv05_endOfDStream(&bitD3) & BITv05_endOfDStream(&bitD4);
if (!endSignal) return ERROR(corruption_detected);
/* decoded size */
return dstSize;
}
size_t HUFv05_decompress4X6 (void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize)
{
HUFv05_CREATE_STATIC_DTABLEX6(DTable, HUFv05_MAX_TABLELOG);
const BYTE* ip = (const BYTE*) cSrc;
size_t hSize = HUFv05_readDTableX6 (DTable, cSrc, cSrcSize);
if (HUFv05_isError(hSize)) return hSize;
if (hSize >= cSrcSize) return ERROR(srcSize_wrong);
ip += hSize;
cSrcSize -= hSize;
return HUFv05_decompress4X6_usingDTable (dst, dstSize, ip, cSrcSize, DTable);
}
/* ********************************/
/* Generic decompression selector */
/* ********************************/
@@ -3250,7 +2875,7 @@ typedef size_t (*decompressionAlgo)(void* dst, size_t dstSize, const void* cSrc,
size_t HUFv05_decompress (void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize)
{
static const decompressionAlgo decompress[3] = { HUFv05_decompress4X2, HUFv05_decompress4X4, HUFv05_decompress4X6 };
static const decompressionAlgo decompress[3] = { HUFv05_decompress4X2, HUFv05_decompress4X4, NULL };
/* estimate decompression time */
U32 Q;
const U32 D256 = (U32)(dstSize >> 8);
@@ -3260,8 +2885,7 @@ size_t HUFv05_decompress (void* dst, size_t dstSize, const void* cSrc, size_t cS
/* validation checks */
if (dstSize == 0) return ERROR(dstSize_tooSmall);
if (cSrcSize > dstSize) return ERROR(corruption_detected); /* invalid */
if (cSrcSize == dstSize) { memcpy(dst, cSrc, dstSize); return dstSize; } /* not compressed */
if (cSrcSize >= dstSize) return ERROR(corruption_detected); /* invalid, or not compressed, but not compressed already dealt with */
if (cSrcSize == 1) { memset(dst, *(const BYTE*)cSrc, dstSize); return dstSize; } /* RLE */
/* decoder timing evaluation */
@@ -3272,7 +2896,6 @@ size_t HUFv05_decompress (void* dst, size_t dstSize, const void* cSrc, size_t cS
Dtime[1] += Dtime[1] >> 4; Dtime[2] += Dtime[2] >> 3; /* advantage to algorithms using less memory, for cache eviction */
if (Dtime[1] < Dtime[0]) algoNb = 1;
if (Dtime[2] < Dtime[algoNb]) algoNb = 2;
return decompress[algoNb](dst, dstSize, cSrc, cSrcSize);
@@ -4706,8 +4329,7 @@ size_t ZBUFFv05_decompressContinue(ZBUFFv05_DCtx* zbc, void* dst, size_t* maxDst
*srcSizePtr = ip-istart;
*maxDstSizePtr = op-ostart;
{
size_t nextSrcSizeHint = ZSTDv05_nextSrcSizeToDecompress(zbc->zc);
{ size_t nextSrcSizeHint = ZSTDv05_nextSrcSizeToDecompress(zbc->zc);
if (nextSrcSizeHint > ZBUFFv05_blockHeaderSize) nextSrcSizeHint+= ZBUFFv05_blockHeaderSize; /* get next block header too */
nextSrcSizeHint -= zbc->inPos; /* already loaded*/
return nextSrcSizeHint;
+17 -10
View File
@@ -69,7 +69,7 @@ const char* ZSTDv05_getErrorName(size_t code); /*!< provides readable string
/** Decompression context */
typedef struct ZSTDv05_DCtx_s ZSTDv05_DCtx;
ZSTDv05_DCtx* ZSTDv05_createDCtx(void);
size_t ZSTDv05_freeDCtx(ZSTDv05_DCtx* dctx); /*!< @return : errorCode */
size_t ZSTDv05_freeDCtx(ZSTDv05_DCtx* dctx); /*!< @return : errorCode */
/** ZSTDv05_decompressDCtx() :
* Same as ZSTDv05_decompress(), but requires an already allocated ZSTDv05_DCtx (see ZSTDv05_createDCtx()) */
@@ -77,25 +77,32 @@ size_t ZSTDv05_decompressDCtx(ZSTDv05_DCtx* ctx, void* dst, size_t dstCapacity,
/*-***********************
* Dictionary API
* Simple Dictionary API
*************************/
/*! ZSTDv05_decompress_usingDict() :
* Decompression using a pre-defined Dictionary content (see dictBuilder).
* Dictionary must be identical to the one used during compression, otherwise regenerated data will be corrupted.
* Note : dict can be NULL, in which case, it's equivalent to ZSTDv05_decompressDCtx() */
size_t ZSTDv05_decompress_usingDict(ZSTDv05_DCtx* dctx,
void* dst, size_t dstCapacity,
const void* src, size_t srcSize,
const void* dict,size_t dictSize);
void* dst, size_t dstCapacity,
const void* src, size_t srcSize,
const void* dict,size_t dictSize);
/*-************************
* Advanced Streaming API
***************************/
size_t ZSTDv05_decompressBegin_usingDict(ZSTDv05_DCtx* dctx, const void* dict, size_t dictSize);
void ZSTDv05_copyDCtx(ZSTDv05_DCtx* dstDCtx, const ZSTDv05_DCtx* srcDCtx);
size_t ZSTDv05_nextSrcSizeToDecompress(ZSTDv05_DCtx* dctx);
size_t ZSTDv05_decompressContinue(ZSTDv05_DCtx* dctx, void* dst, size_t dstCapacity, const void* src, size_t srcSize);
/*-***********************
* ZBUFF API
*************************/
typedef struct ZBUFFv05_DCtx_s ZBUFFv05_DCtx;
ZBUFFv05_DCtx* ZBUFFv05_createDCtx(void);
size_t ZBUFFv05_freeDCtx(ZBUFFv05_DCtx* dctx);
size_t ZBUFFv05_freeDCtx(ZBUFFv05_DCtx* dctx);
size_t ZBUFFv05_decompressInit(ZBUFFv05_DCtx* dctx);
size_t ZBUFFv05_decompressInitDictionary(ZBUFFv05_DCtx* dctx, const void* dict, size_t dictSize);
+24 -16
View File
@@ -32,9 +32,9 @@
# ##########################################################################
# Version numbers
LIBVER_MAJOR_SCRIPT:=`sed -n '/define ZSTD_VERSION_MAJOR/s/.*[[:blank:]]\([0-9][0-9]*\).*/\1/p' < ../lib/zstd.h`
LIBVER_MINOR_SCRIPT:=`sed -n '/define ZSTD_VERSION_MINOR/s/.*[[:blank:]]\([0-9][0-9]*\).*/\1/p' < ../lib/zstd.h`
LIBVER_PATCH_SCRIPT:=`sed -n '/define ZSTD_VERSION_RELEASE/s/.*[[:blank:]]\([0-9][0-9]*\).*/\1/p' < ../lib/zstd.h`
LIBVER_MAJOR_SCRIPT:=`sed -n '/define ZSTD_VERSION_MAJOR/s/.*[[:blank:]]\([0-9][0-9]*\).*/\1/p' < ../lib/common/zstd.h`
LIBVER_MINOR_SCRIPT:=`sed -n '/define ZSTD_VERSION_MINOR/s/.*[[:blank:]]\([0-9][0-9]*\).*/\1/p' < ../lib/common/zstd.h`
LIBVER_PATCH_SCRIPT:=`sed -n '/define ZSTD_VERSION_RELEASE/s/.*[[:blank:]]\([0-9][0-9]*\).*/\1/p' < ../lib/common/zstd.h`
LIBVER_SCRIPT:= $(LIBVER_MAJOR_SCRIPT).$(LIBVER_MINOR_SCRIPT).$(LIBVER_PATCH_SCRIPT)
LIBVER_MAJOR := $(shell echo $(LIBVER_MAJOR_SCRIPT))
LIBVER_MINOR := $(shell echo $(LIBVER_MINOR_SCRIPT))
@@ -44,7 +44,7 @@ VERSION?= $(LIBVER)
DESTDIR?=
PREFIX ?= /usr/local
CPPFLAGS= -I../lib -DZSTD_VERSION=\"$(VERSION)\"
CPPFLAGS= -I../lib/common -DZSTD_VERSION=\"$(VERSION)\"
CFLAGS ?= -O3 # -falign-loops=32 # not always beneficial
CFLAGS += -std=c99 -Wall -Wextra -Wcast-qual -Wcast-align -Wshadow -Wstrict-aliasing=1 -Wswitch-enum -Wstrict-prototypes -Wundef
FLAGS = $(CPPFLAGS) $(CFLAGS) $(LDFLAGS) $(MOREFLAGS)
@@ -53,7 +53,11 @@ BINDIR = $(PREFIX)/bin
MANDIR = $(PREFIX)/share/man/man1
ZSTDDIR = ../lib
ZSTD_FILES := $(ZSTDDIR)/huff0.c $(ZSTDDIR)/fse.c $(ZSTDDIR)/zstd_decompress.c $(ZSTDDIR)/zstd_compress.c
ZSTDCOMP_FILES := $(ZSTDDIR)/common/fse_decompress.c $(ZSTDDIR)/compress/zstd_compress.c $(ZSTDDIR)/compress/fse_compress.c $(ZSTDDIR)/compress/huf_compress.c $(ZSTDDIR)/common/zstd_common.c $(ZSTDDIR)/common/entropy_common.c
ZSTDDECOMP_FILES := $(ZSTDDIR)/decompress/zstd_decompress.c $(ZSTDDIR)/common/fse_decompress.c $(ZSTDDIR)/decompress/huf_decompress.c $(ZSTDDIR)/common/zstd_common.c $(ZSTDDIR)/common/entropy_common.c
ZDICT_FILES := $(ZSTDDIR)/dictBuilder/zdict.c $(ZSTDDIR)/dictBuilder/divsufsort.c
ZBUFF_FILES := $(ZSTDDIR)/compress/zbuff_compress.c $(ZSTDDIR)/decompress/zbuff_decompress.c
ZSTD_FILES := $(ZSTDDECOMP_FILES) $(ZSTDCOMP_FILES)
ifeq ($(ZSTD_LEGACY_SUPPORT), 0)
CPPFLAGS += -DZSTD_LEGACY_SUPPORT=0
@@ -83,13 +87,13 @@ ZSTDRTTEST= --test-large-data
default: zstd
all: zstd zstd32 fullbench fullbench32 fuzzer fuzzer32 zbufftest zbufftest32 paramgrill datagen
all: zstd fullbench fuzzer zbufftest paramgrill datagen zstd32 fullbench32 fuzzer32 zbufftest32
zstd : $(ZSTD_FILES) $(ZSTD_FILES_LEGACY) $(ZSTDDIR)/zbuff.c $(ZSTDDIR)/zdict.c $(ZSTDDIR)/divsufsort.c \
zstd : $(ZSTD_FILES) $(ZSTD_FILES_LEGACY) $(ZBUFF_FILES) $(ZDICT_FILES) \
zstdcli.c fileio.c bench.c xxhash.c datagen.c dibio.c
$(CC) $(FLAGS) -DZSTD_LEGACY_SUPPORT=$(ZSTD_LEGACY_SUPPORT) $^ -o $@$(EXT)
zstd32: $(ZSTD_FILES) $(ZSTD_FILES_LEGACY) $(ZSTDDIR)/zbuff.c $(ZSTDDIR)/zdict.c $(ZSTDDIR)/divsufsort.c \
zstd32: $(ZSTD_FILES) $(ZSTD_FILES_LEGACY) $(ZBUFF_FILES) $(ZDICT_FILES) \
zstdcli.c fileio.c bench.c xxhash.c datagen.c dibio.c
$(CC) -m32 $(FLAGS) -DZSTD_LEGACY_SUPPORT=$(ZSTD_LEGACY_SUPPORT) $^ -o $@$(EXT)
@@ -107,18 +111,22 @@ zstd-pgo : clean zstd
rm zstd
$(MAKE) zstd MOREFLAGS=-fprofile-use
zstd-frugal: $(ZSTD_FILES) $(ZSTDDIR)/zbuff.c zstdcli.c fileio.c
zstd-frugal: $(ZSTD_FILES) $(ZBUFF_FILES) zstdcli.c fileio.c
$(CC) $(FLAGS) -DZSTD_NOBENCH -DZSTD_NODICT -DZSTD_LEGACY_SUPPORT=0 $^ -o zstd$(EXT)
zstd-compress: $(ZSTDCOMP_FILES) $(ZSTDDIR)/compress/zbuff_compress.c zstdcli.c fileio.c
$(CC) $(FLAGS) -DZSTD_NOBENCH -DZSTD_NODICT -DZSTD_NODECOMPRESS -DZSTD_LEGACY_SUPPORT=0 $^ -o $@$(EXT)
zstd-decompress: $(ZSTDDECOMP_FILES) $(ZSTDDIR)/decompress/zbuff_decompress.c zstdcli.c fileio.c
$(CC) $(FLAGS) -DZSTD_NOBENCH -DZSTD_NODICT -DZSTD_NOCOMPRESS -DZSTD_LEGACY_SUPPORT=0 $^ -o $@$(EXT)
zstd-small: clean
CFLAGS="-Os -s" $(MAKE) zstd-frugal
fullbench : $(ZSTD_FILES) $(ZSTDDIR)/zbuff.c \
datagen.c fullbench.c
fullbench : $(ZSTD_FILES) $(ZBUFF_FILES) datagen.c fullbench.c
$(CC) $(FLAGS) $^ -o $@$(EXT)
fullbench32: $(ZSTD_FILES) $(ZSTDDIR)/zbuff.c \
datagen.c fullbench.c
fullbench32: $(ZSTD_FILES) $(ZBUFF_FILES) datagen.c fullbench.c
$(CC) -m32 $(FLAGS) $^ -o $@$(EXT)
fuzzer : $(ZSTD_FILES) \
@@ -129,11 +137,11 @@ fuzzer32: $(ZSTD_FILES) \
datagen.c xxhash.c fuzzer.c
$(CC) -m32 $(FLAGS) $^ -o $@$(EXT)
zbufftest : $(ZSTD_FILES) $(ZSTDDIR)/zbuff.c \
zbufftest : $(ZSTD_FILES) $(ZBUFF_FILES) \
datagen.c xxhash.c zbufftest.c
$(CC) $(FLAGS) $^ -o $@$(EXT)
zbufftest32: $(ZSTD_FILES) $(ZSTDDIR)/zbuff.c \
zbufftest32: $(ZSTD_FILES) $(ZBUFF_FILES) \
datagen.c xxhash.c zbufftest.c
$(CC) -m32 $(FLAGS) $^ -o $@$(EXT)
@@ -146,7 +154,7 @@ datagen : datagen.c datagencli.c
clean:
@rm -f core *.o tmp* result* *.gcda dictionary *.zst \
zstd$(EXT) zstd32$(EXT) \
zstd$(EXT) zstd32$(EXT) zstd-compress$(EXT) zstd-decompress$(EXT) \
fullbench$(EXT) fullbench32$(EXT) \
fuzzer$(EXT) fuzzer32$(EXT) zbufftest$(EXT) zbufftest32$(EXT) \
datagen$(EXT) paramgrill$(EXT)
+83 -162
View File
@@ -23,95 +23,20 @@
- zstd source repository : https://github.com/Cyan4973/zstd
*/
/* **************************************
* Compiler Options
****************************************/
/* Disable some Visual warning messages */
#ifdef _MSC_VER
# define _CRT_SECURE_NO_WARNINGS /* fopen */
# pragma warning(disable : 4127) /* disable: C4127: conditional expression is constant */
#endif
/* Unix Large Files support (>4GB) */
#define _FILE_OFFSET_BITS 64
#if (defined(__sun__) && (!defined(__LP64__))) /* Sun Solaris 32-bits requires specific definitions */
# define _LARGEFILE_SOURCE
#elif ! defined(__LP64__) /* No point defining Large file for 64 bit */
# define _LARGEFILE64_SOURCE
#endif
/* *************************************
* Includes
***************************************/
#define _POSIX_C_SOURCE 199309L /* before <time.h> - needed for nanosleep() */
#include "util.h" /* Compiler options, UTIL_GetFileSize, UTIL_HAS_CREATEFILELIST, UTIL_sleep */
#include <stdlib.h> /* malloc, free */
#include <string.h> /* memset */
#include <stdio.h> /* fprintf, fopen, ftello64 */
#include <sys/types.h> /* stat64 */
#include <sys/stat.h> /* stat64 */
#include <time.h> /* clock_t, nanosleep, clock, CLOCKS_PER_SEC */
/* sleep : posix - windows - others */
#if !defined(_WIN32) && (defined(__unix__) || defined(__unix) || (defined(__APPLE__) && defined(__MACH__)))
# include <unistd.h>
# include <sys/resource.h> /* setpriority */
# define BMK_sleep(s) sleep(s)
# define mili_sleep(mili) { struct timespec t; t.tv_sec=0; t.tv_nsec=mili*1000000ULL; nanosleep(&t, NULL); }
# define SET_HIGH_PRIORITY setpriority(PRIO_PROCESS, 0, -20)
#elif defined(_WIN32)
# include <windows.h>
# define BMK_sleep(s) Sleep(1000*s)
# define mili_sleep(mili) Sleep(mili)
# define SET_HIGH_PRIORITY SetPriorityClass(GetCurrentProcess(), REALTIME_PRIORITY_CLASS)
#else
# define BMK_sleep(s) /* disabled */
# define mili_sleep(mili) /* disabled */
# define SET_HIGH_PRIORITY /* disabled */
#endif
#if !defined(_WIN32) && (defined(__unix__) || defined(__unix) || (defined(__APPLE__) && defined(__MACH__)))
typedef clock_t BMK_time_t;
# define BMK_initTimer(ticksPerSecond) ticksPerSecond=0
# define BMK_getTime(x) x = clock()
# define BMK_getSpanTimeMicro(ticksPerSecond, clockStart, clockEnd) (1000000ULL * (clockEnd - clockStart) / CLOCKS_PER_SEC)
# define BMK_getSpanTimeNano(ticksPerSecond, clockStart, clockEnd) (1000000000ULL * (clockEnd - clockStart) / CLOCKS_PER_SEC)
#elif defined(_WIN32)
typedef LARGE_INTEGER BMK_time_t;
# define BMK_initTimer(x) if (!QueryPerformanceFrequency(&x)) { fprintf(stderr, "ERROR: QueryPerformance not present\n"); }
# define BMK_getTime(x) QueryPerformanceCounter(&x)
# define BMK_getSpanTimeMicro(ticksPerSecond, clockStart, clockEnd) (1000000ULL*(clockEnd.QuadPart - clockStart.QuadPart)/ticksPerSecond.QuadPart)
# define BMK_getSpanTimeNano(ticksPerSecond, clockStart, clockEnd) (1000000000ULL*(clockEnd.QuadPart - clockStart.QuadPart)/ticksPerSecond.QuadPart)
#else
typedef int BMK_time_t;
# define BMK_initTimer(ticksPerSecond) ticksPerSecond=0
# define BMK_getTimeMicro(clockStart) clockStart=1
# define BMK_getSpanTimeMicro(ticksPerSecond, clockStart, clockEnd) (TIMELOOP_S*1000000ULL+clockEnd-clockStart)
# define BMK_getSpanTimeNano(ticksPerSecond, clockStart, clockEnd) (TIMELOOP_S*1000000000ULL+clockEnd-clockStart)
#endif
#include "mem.h"
#include "zstd_static.h"
#include "zstd_internal.h" /* ZSTD_compressBegin_targetSrcSize */
#include "datagen.h" /* RDG_genBuffer */
#include "datagen.h" /* RDG_genBuffer */
#include "xxhash.h"
/* *************************************
* Compiler specifics
***************************************/
#if !defined(S_ISREG)
# define S_ISREG(x) (((x) & S_IFMT) == S_IFREG)
#endif
#if defined(_MSC_VER)
# define snprintf sprintf_s
#elif defined (__cplusplus) || (defined (__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L))
/* part of <stdio.h> */
#else
extern int snprintf (char* s, size_t maxlen, const char* format, ...); /* not declared in <stdio.h> when C version < c99 */
#endif
/* *************************************
* Constants
@@ -120,10 +45,10 @@
# define ZSTD_VERSION ""
#endif
#define NBLOOPS 3
#define TIMELOOP_S 1
#define ACTIVEPERIOD_S 70
#define COOLPERIOD_S 10
#define NBLOOPS 3
#define TIMELOOP_MICROSEC 1*1000000ULL /* 1 second */
#define ACTIVEPERIOD_MICROSEC 70*1000000ULL /* 70 seconds */
#define COOLPERIOD_SEC 10
#define KB *(1 <<10)
#define MB *(1 <<20)
@@ -183,34 +108,6 @@ void BMK_SetBlockSize(size_t blockSize)
}
/* ********************************************************
* Private functions
**********************************************************/
/* returns time span in microseconds */
static U64 BMK_clockSpan( BMK_time_t clockStart, BMK_time_t ticksPerSecond )
{
BMK_time_t clockEnd;
(void)ticksPerSecond;
BMK_getTime(clockEnd);
return BMK_getSpanTimeMicro(ticksPerSecond, clockStart, clockEnd);
}
static U64 BMK_getFileSize(const char* infilename)
{
int r;
#if defined(_MSC_VER)
struct _stat64 statbuf;
r = _stat64(infilename, &statbuf);
#else
struct stat statbuf;
r = stat(infilename, &statbuf);
#endif
if (r || !S_ISREG(statbuf.st_mode)) return 0; /* No good... */
return (U64)statbuf.st_size;
}
/* ********************************************************
* Bench functions
**********************************************************/
@@ -242,7 +139,7 @@ static int BMK_benchMem(const void* srcBuffer, size_t srcSize,
const size_t* fileSizes, U32 nbFiles,
const void* dictBuffer, size_t dictBufferSize, benchResult_t *result)
{
size_t const blockSize = (g_blockSize ? g_blockSize : srcSize) + (!srcSize); /* avoid div by 0 */
size_t const blockSize = (g_blockSize>=32 ? g_blockSize : srcSize) + (!srcSize); /* avoid div by 0 */
U32 const maxNbBlocks = (U32) ((srcSize + (blockSize-1)) / blockSize) + nbFiles;
blockParam_t* const blockTable = (blockParam_t*) malloc(maxNbBlocks * sizeof(blockParam_t));
size_t const maxCompressedSize = ZSTD_compressBound(srcSize) + (maxNbBlocks * 1024); /* add some room for safety */
@@ -253,7 +150,7 @@ static int BMK_benchMem(const void* srcBuffer, size_t srcSize,
ZSTD_DCtx* refDCtx = ZSTD_createDCtx();
ZSTD_DCtx* dctx = ZSTD_createDCtx();
U32 nbBlocks;
BMK_time_t ticksPerSecond;
UTIL_time_t ticksPerSecond;
/* checks */
if (!compressedBuffer || !resultBuffer || !blockTable || !refCtx || !ctx || !refDCtx || !dctx)
@@ -261,7 +158,7 @@ static int BMK_benchMem(const void* srcBuffer, size_t srcSize,
/* init */
if (strlen(displayName)>17) displayName += strlen(displayName)-17; /* can only display 17 characters */
BMK_initTimer(ticksPerSecond);
UTIL_initTimer(&ticksPerSecond);
/* Init blockTable data */
{ const char* srcPtr = (const char*)srcBuffer;
@@ -289,51 +186,56 @@ static int BMK_benchMem(const void* srcBuffer, size_t srcSize,
RDG_genBuffer(compressedBuffer, maxCompressedSize, 0.10, 0.50, 1);
/* Bench */
{ double fastestC = 100000000., fastestD = 100000000.;
{ U64 fastestC = (U64)(-1LL), fastestD = (U64)(-1LL);
U64 const crcOrig = XXH64(srcBuffer, srcSize, 0);
U64 crcCheck = 0;
BMK_time_t coolTime;
UTIL_time_t coolTime;
U32 testNb;
size_t cSize = 0;
double ratio = 0.;
BMK_getTime(coolTime);
UTIL_getTime(&coolTime);
DISPLAYLEVEL(2, "\r%79s\r", "");
for (testNb = 1; testNb <= (g_nbIterations + !g_nbIterations); testNb++) {
BMK_time_t clockStart, clockEnd;
U64 clockLoop = g_nbIterations ? TIMELOOP_S*1000000ULL : 10;
UTIL_time_t clockStart;
U64 clockLoop = g_nbIterations ? TIMELOOP_MICROSEC : 1;
/* overheat protection */
if (BMK_clockSpan(coolTime, ticksPerSecond) > ACTIVEPERIOD_S*1000000ULL) {
if (UTIL_clockSpanMicro(coolTime, ticksPerSecond) > ACTIVEPERIOD_MICROSEC) {
DISPLAY("\rcooling down ... \r");
BMK_sleep(COOLPERIOD_S);
BMK_getTime(coolTime);
UTIL_sleep(COOLPERIOD_SEC);
UTIL_getTime(&coolTime);
}
/* Compression */
DISPLAYLEVEL(2, "%2i-%-17.17s :%10u ->\r", testNb, displayName, (U32)srcSize);
memset(compressedBuffer, 0xE5, maxCompressedSize); /* warm up and erase result buffer */
mili_sleep(1); /* give processor time to other processes */
BMK_getTime(clockStart);
do { BMK_getTime(clockEnd); }
while (BMK_getSpanTimeNano(ticksPerSecond, clockStart, clockEnd) == 0);
BMK_getTime(clockStart);
UTIL_sleepMilli(1); /* give processor time to other processes */
UTIL_waitForNextTick(ticksPerSecond);
UTIL_getTime(&clockStart);
{ U32 nbLoops;
for (nbLoops = 0 ; BMK_clockSpan(clockStart, ticksPerSecond) < clockLoop ; nbLoops++) {
{ U32 nbLoops = 0;
do {
U32 blockNb;
ZSTD_compressBegin_targetSrcSize(refCtx, dictBuffer, dictBufferSize, blockSize, cLevel);
// ZSTD_compressBegin_usingDict(refCtx, dictBuffer, dictBufferSize, cLevel);
{ ZSTD_parameters params;
params.cParams = ZSTD_getCParams(cLevel, blockSize, dictBufferSize);
params.fParams.contentSizeFlag = 1;
ZSTD_adjustCParams(&params.cParams, blockSize, dictBufferSize);
{ size_t const initResult = ZSTD_compressBegin_advanced(refCtx, dictBuffer, dictBufferSize, params, blockSize);
if (ZSTD_isError(initResult)) break;
} }
for (blockNb=0; blockNb<nbBlocks; blockNb++) {
size_t const rSize = ZSTD_compress_usingPreparedCCtx(ctx, refCtx,
blockTable[blockNb].cPtr, blockTable[blockNb].cRoom,
blockTable[blockNb].srcPtr,blockTable[blockNb].srcSize);
if (ZSTD_isError(rSize)) EXM_THROW(1, "ZSTD_compress_usingPreparedCCtx() failed : %s", ZSTD_getErrorName(rSize));
blockTable[blockNb].cSize = rSize;
} }
{ U64 const clockSpan = BMK_clockSpan(clockStart, ticksPerSecond);
if ((double)clockSpan < fastestC*nbLoops) fastestC = (double)clockSpan / nbLoops;
}
nbLoops++;
} while (UTIL_clockSpanMicro(clockStart, ticksPerSecond) < clockLoop);
{ U64 const clockSpan = UTIL_clockSpanMicro(clockStart, ticksPerSecond);
if (clockSpan < fastestC*nbLoops) fastestC = clockSpan / nbLoops;
} }
cSize = 0;
@@ -348,14 +250,12 @@ static int BMK_benchMem(const void* srcBuffer, size_t srcSize,
/* Decompression */
memset(resultBuffer, 0xD6, srcSize); /* warm result buffer */
mili_sleep(1); /* give processor time to other processes */
BMK_getTime(clockStart);
do { BMK_getTime(clockEnd); }
while (BMK_getSpanTimeNano(ticksPerSecond, clockStart, clockEnd) == 0);
BMK_getTime(clockStart);
UTIL_sleepMilli(1); /* give processor time to other processes */
UTIL_waitForNextTick(ticksPerSecond);
UTIL_getTime(&clockStart);
{ U32 nbLoops;
for (nbLoops = 0 ; BMK_clockSpan(clockStart, ticksPerSecond) < clockLoop ; nbLoops++) {
{ U32 nbLoops = 0;
do {
U32 blockNb;
ZSTD_decompressBegin_usingDict(refDCtx, dictBuffer, dictBufferSize);
for (blockNb=0; blockNb<nbBlocks; blockNb++) {
@@ -369,9 +269,11 @@ static int BMK_benchMem(const void* srcBuffer, size_t srcSize,
break;
}
blockTable[blockNb].resSize = regenSize;
} }
{ U64 const clockSpan = BMK_clockSpan(clockStart, ticksPerSecond);
if ((double)clockSpan < fastestD*nbLoops) fastestD = (double)clockSpan / nbLoops;
}
nbLoops++;
} while (UTIL_clockSpanMicro(clockStart, ticksPerSecond) < clockLoop);
{ U64 const clockSpan = UTIL_clockSpanMicro(clockStart, ticksPerSecond);
if (clockSpan < fastestD*nbLoops) fastestD = clockSpan / nbLoops;
} }
DISPLAYLEVEL(2, "%2i-%-17.17s :%10u ->%10u (%5.3f),%6.1f MB/s ,%6.1f MB/s\r",
@@ -416,6 +318,7 @@ static int BMK_benchMem(const void* srcBuffer, size_t srcSize,
} /* Bench */
/* clean up */
free(blockTable);
free(compressedBuffer);
free(resultBuffer);
ZSTD_freeCCtx(refCtx);
@@ -452,12 +355,12 @@ static void BMK_benchCLevel(void* srcBuffer, size_t benchedSize,
benchResult_t result, total;
int l;
SET_HIGH_PRIORITY;
const char* pch = strrchr(displayName, '\\'); /* Windows */
if (!pch) pch = strrchr(displayName, '/'); /* Linux */
if (pch) displayName = pch+1;
SET_HIGH_PRIORITY;
memset(&result, 0, sizeof(result));
memset(&total, 0, sizeof(total));
@@ -473,9 +376,9 @@ static void BMK_benchCLevel(void* srcBuffer, size_t benchedSize,
dictBuffer, dictBufferSize, &result);
if (g_displayLevel == 1) {
if (g_additionalParam)
DISPLAY("%-3i%11i (%5.3f) %6.1f MB/s %6.1f MB/s %s (param=%d)\n", -l, (int)result.cSize, result.ratio, result.cSpeed, result.dSpeed, displayName, g_additionalParam);
DISPLAY("%-3i%11i (%5.3f) %6.2f MB/s %6.1f MB/s %s (param=%d)\n", -l, (int)result.cSize, result.ratio, result.cSpeed, result.dSpeed, displayName, g_additionalParam);
else
DISPLAY("%-3i%11i (%5.3f) %6.1f MB/s %6.1f MB/s %s\n", -l, (int)result.cSize, result.ratio, result.cSpeed, result.dSpeed, displayName);
DISPLAY("%-3i%11i (%5.3f) %6.2f MB/s %6.1f MB/s %s\n", -l, (int)result.cSize, result.ratio, result.cSpeed, result.dSpeed, displayName);
total.cSize += result.cSize;
total.cSpeed += result.cSpeed;
total.dSpeed += result.dSpeed;
@@ -486,18 +389,10 @@ static void BMK_benchCLevel(void* srcBuffer, size_t benchedSize,
total.cSpeed /= 1+cLevelLast-cLevel;
total.dSpeed /= 1+cLevelLast-cLevel;
total.ratio /= 1+cLevelLast-cLevel;
DISPLAY("avg%11i (%5.3f) %6.1f MB/s %6.1f MB/s %s\n", (int)total.cSize, total.ratio, total.cSpeed, total.dSpeed, displayName);
DISPLAY("avg%11i (%5.3f) %6.2f MB/s %6.1f MB/s %s\n", (int)total.cSize, total.ratio, total.cSpeed, total.dSpeed, displayName);
}
}
static U64 BMK_getTotalFileSize(const char** fileNamesTable, unsigned nbFiles)
{
U64 total = 0;
unsigned n;
for (n=0; n<nbFiles; n++)
total += BMK_getFileSize(fileNamesTable[n]);
return total;
}
/*! BMK_loadFiles() :
Loads `buffer` with content of files listed within `fileNamesTable`.
@@ -506,12 +401,17 @@ static void BMK_loadFiles(void* buffer, size_t bufferSize,
size_t* fileSizes,
const char** fileNamesTable, unsigned nbFiles)
{
size_t pos = 0;
size_t pos = 0, totalSize = 0;
FILE* f;
unsigned n;
for (n=0; n<nbFiles; n++) {
U64 fileSize = BMK_getFileSize(fileNamesTable[n]);
FILE* const f = fopen(fileNamesTable[n], "rb");
U64 fileSize = UTIL_getFileSize(fileNamesTable[n]);
if (UTIL_isDirectory(fileNamesTable[n])) {
DISPLAYLEVEL(2, "Ignoring %s directory... \n", fileNamesTable[n]);
fileSizes[n] = 0;
continue;
}
f = fopen(fileNamesTable[n], "rb");
if (f==NULL) EXM_THROW(10, "impossible to open file %s", fileNamesTable[n]);
DISPLAYLEVEL(2, "Loading %s... \r", fileNamesTable[n]);
if (fileSize > bufferSize-pos) fileSize = bufferSize-pos, nbFiles=n; /* buffer too small - stop after this file */
@@ -519,8 +419,11 @@ static void BMK_loadFiles(void* buffer, size_t bufferSize,
if (readSize != (size_t)fileSize) EXM_THROW(11, "could not read %s", fileNamesTable[n]);
pos += readSize; }
fileSizes[n] = (size_t)fileSize;
totalSize += (size_t)fileSize;
fclose(f);
}
if (totalSize == 0) EXM_THROW(12, "no data to bench");
}
static void BMK_benchFileTable(const char** fileNamesTable, unsigned nbFiles,
@@ -531,7 +434,7 @@ static void BMK_benchFileTable(const char** fileNamesTable, unsigned nbFiles,
void* dictBuffer = NULL;
size_t dictBufferSize = 0;
size_t* fileSizes = (size_t*)malloc(nbFiles * sizeof(size_t));
U64 totalSizeToLoad = BMK_getTotalFileSize(fileNamesTable, nbFiles);
U64 totalSizeToLoad = UTIL_getTotalFileSize(fileNamesTable, nbFiles);
char mfName[20] = {0};
const char* displayName = NULL;
@@ -539,7 +442,7 @@ static void BMK_benchFileTable(const char** fileNamesTable, unsigned nbFiles,
/* Load dictionary */
if (dictFileName != NULL) {
U64 dictFileSize = BMK_getFileSize(dictFileName);
U64 dictFileSize = UTIL_getFileSize(dictFileName);
if (dictFileSize > 64 MB) EXM_THROW(10, "dictionary file %s too large", dictFileName);
dictBufferSize = (size_t)dictFileSize;
dictBuffer = malloc(dictBufferSize);
@@ -597,14 +500,32 @@ static void BMK_syntheticTest(int cLevel, int cLevelLast, double compressibility
int BMK_benchFiles(const char** fileNamesTable, unsigned nbFiles,
const char* dictFileName, int cLevel, int cLevelLast)
const char* dictFileName, int cLevel, int cLevelLast, int recursive)
{
double const compressibility = (double)g_compressibilityDefault / 100;
if (nbFiles == 0)
BMK_syntheticTest(cLevel, cLevelLast, compressibility);
else
{
#ifdef UTIL_HAS_CREATEFILELIST
if (recursive) {
char* buf;
const char** filenameTable;
unsigned i;
filenameTable = UTIL_createFileList(fileNamesTable, nbFiles, &buf, &nbFiles);
if (filenameTable) {
for (i=0; i<nbFiles; i++) DISPLAYLEVEL(3, "%d %s\n", i, filenameTable[i]);
BMK_benchFileTable(filenameTable, nbFiles, dictFileName, cLevel, cLevelLast);
UTIL_freeFileList(filenameTable, buf);
}
}
else BMK_benchFileTable(fileNamesTable, nbFiles, dictFileName, cLevel, cLevelLast);
#else
(void)recursive;
BMK_benchFileTable(fileNamesTable, nbFiles, dictFileName, cLevel, cLevelLast);
#endif
}
return 0;
}
+3 -4
View File
@@ -1,6 +1,6 @@
/*
bench.h - Demo program to benchmark open-source compression algorithm
Copyright (C) Yann Collet 2012-2015
Copyright (C) Yann Collet 2012-2016
GPL v2 License
@@ -19,15 +19,14 @@
51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
You can contact the author at :
- LZ4 source repository : http://code.google.com/p/lz4/
- LZ4 public forum : https://group.google.com/forum/#!forum/lz4c
- ZSTD homepage : http://www.zstd.net/
*/
#pragma once
/* Main function */
int BMK_benchFiles(const char** fileNamesTable, unsigned nbFiles,
const char* dictFileName, int cLevel, int cLevelLast);
const char* dictFileName, int cLevel, int cLevelLast, int recursive);
/* Set Parameters */
void BMK_SetNbIterations(unsigned nbLoops);
+8 -33
View File
@@ -29,25 +29,7 @@
#include <stdlib.h> /* malloc */
#include <stdio.h> /* FILE, fwrite */
#include <string.h> /* memcpy */
/*-************************************
* Basic Types
**************************************/
#if defined (__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L) /* C99 */
# include <stdint.h>
typedef uint8_t BYTE;
typedef uint16_t U16;
typedef uint32_t U32;
typedef int32_t S32;
typedef uint64_t U64;
#else
typedef unsigned char BYTE;
typedef unsigned short U16;
typedef unsigned int U32;
typedef signed int S32;
typedef unsigned long long U64;
#endif
#include "mem.h"
/*-************************************
@@ -63,7 +45,7 @@
/*-************************************
* Constants
* Macros
**************************************/
#define KB *(1 <<10)
@@ -77,7 +59,6 @@
typedef BYTE litDistribTable[LTSIZE];
/*-*******************************************************
* Local Functions
*********************************************************/
@@ -98,15 +79,10 @@ static unsigned int RDG_rand(U32* src)
static void RDG_fillLiteralDistrib(litDistribTable lt, double ld)
{
U32 i = 0;
BYTE character = '0';
BYTE firstChar = '(';
BYTE lastChar = '}';
BYTE character = (ld<=0.0) ? 0 : '0';
BYTE const firstChar = (ld<=0.0) ? 0 : '(';
BYTE const lastChar = (ld<=0.0) ?255: '}';
if (ld<=0.0) {
character = 0;
firstChar = 0;
lastChar =255;
}
while (i<LTSIZE) {
U32 weight = (U32)((double)(LTSIZE - i) * ld) + 1;
U32 end;
@@ -161,7 +137,7 @@ void RDG_genBlock(void* buffer, size_t buffSize, size_t prefixSize, double match
/* Copy (within 32K) */
size_t match;
size_t d;
int length = RDG_RANDLENGTH + 4;
size_t const length = RDG_RANDLENGTH + 4;
U32 offset = RDG_RAND15BITS + 1;
U32 repeatOffset = (RDG_rand(seed) & 15) == 2;
if (repeatOffset) offset = prevOffset;
@@ -173,9 +149,8 @@ void RDG_genBlock(void* buffer, size_t buffSize, size_t prefixSize, double match
while (pos < d) buffPtr[pos++] = buffPtr[match++]; /* correctly manages overlaps */
} else {
/* Literal (noise) */
size_t d;
size_t length = RDG_RANDLENGTH;
d = pos + length;
size_t const length = RDG_RANDLENGTH;
size_t d = pos + length;
if (d > buffSize) d = buffSize;
while (pos < d) buffPtr[pos++] = RDG_genChar(seed, lt);
} }
+18 -62
View File
@@ -24,39 +24,17 @@
- Public forum : https://groups.google.com/forum/#!forum/lz4c
*/
/**************************************
/*-************************************
* Includes
**************************************/
#include <stdio.h> /* fprintf, stderr */
#include "mem.h"
#include "datagen.h" /* RDG_generate */
/**************************************
* Basic Types
**************************************/
#if defined (__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L) /* C99 */
# include <stdint.h>
typedef uint8_t BYTE;
typedef uint16_t U16;
typedef uint32_t U32;
typedef int32_t S32;
typedef uint64_t U64;
#else
typedef unsigned char BYTE;
typedef unsigned short U16;
typedef unsigned int U32;
typedef signed int S32;
typedef unsigned long long U64;
#endif
/**************************************
/*-************************************
* Constants
**************************************/
#ifndef ZSTD_VERSION
# define ZSTD_VERSION "r1"
#endif
#define KB *(1 <<10)
#define MB *(1 <<20)
#define GB *(1U<<30)
@@ -66,7 +44,7 @@
#define COMPRESSIBILITY_DEFAULT 50
/**************************************
/*-************************************
* Macros
**************************************/
#define DISPLAY(...) fprintf(stderr, __VA_ARGS__)
@@ -74,10 +52,10 @@
static unsigned displayLevel = 2;
/*********************************************************
/*-*******************************************************
* Command line
*********************************************************/
static int usage(char* programName)
static int usage(const char* programName)
{
DISPLAY( "Compressible data generator\n");
DISPLAY( "Usage :\n");
@@ -92,29 +70,26 @@ static int usage(char* programName)
}
int main(int argc, char** argv)
int main(int argc, const char** argv)
{
int argNb;
double proba = (double)COMPRESSIBILITY_DEFAULT / 100;
double litProba = 0.0;
U64 size = SIZE_DEFAULT;
U32 seed = SEED_DEFAULT;
char* programName;
const char* programName;
/* Check command line */
programName = argv[0];
for(argNb=1; argNb<argc; argNb++)
{
char* argument = argv[argNb];
for(argNb=1; argNb<argc; argNb++) {
const char* argument = argv[argNb];
if(!argument) continue; /* Protection if argument empty */
/* Handle commands. Aggregated commands are allowed */
if (*argument=='-')
{
if (*argument=='-') {
argument++;
while (*argument!=0)
{
while (*argument!=0) {
switch(*argument)
{
case 'h':
@@ -123,11 +98,7 @@ int main(int argc, char** argv)
argument++;
size=0;
while ((*argument>='0') && (*argument<='9'))
{
size *= 10;
size += *argument - '0';
argument++;
}
size *= 10, size += *argument++ - '0';
if (*argument=='K') { size <<= 10; argument++; }
if (*argument=='M') { size <<= 20; argument++; }
if (*argument=='G') { size <<= 30; argument++; }
@@ -137,21 +108,13 @@ int main(int argc, char** argv)
argument++;
seed=0;
while ((*argument>='0') && (*argument<='9'))
{
seed *= 10;
seed += *argument - '0';
argument++;
}
seed *= 10, seed += *argument++ - '0';
break;
case 'P':
argument++;
proba=0.0;
while ((*argument>='0') && (*argument<='9'))
{
proba *= 10;
proba += *argument - '0';
argument++;
}
proba *= 10, proba += *argument++ - '0';
if (proba>100.) proba=100.;
proba /= 100.;
break;
@@ -159,11 +122,7 @@ int main(int argc, char** argv)
argument++;
litProba=0.;
while ((*argument>='0') && (*argument<='9'))
{
litProba *= 10;
litProba += *argument - '0';
argument++;
}
litProba *= 10, litProba += *argument++ - '0';
if (litProba>100.) litProba=100.;
litProba /= 100.;
break;
@@ -174,12 +133,9 @@ int main(int argc, char** argv)
default:
return usage(programName);
}
}
} } } /* for(argNb=1; argNb<argc; argNb++) */
}
}
DISPLAYLEVEL(4, "Data Generator %s \n", ZSTD_VERSION);
DISPLAYLEVEL(4, "Data Generator \n");
DISPLAYLEVEL(3, "Seed = %u \n", seed);
if (proba!=COMPRESSIBILITY_DEFAULT) DISPLAYLEVEL(3, "Compressibility : %i%%\n", (U32)(proba*100));
+24 -66
View File
@@ -22,45 +22,18 @@
- zstd homepage : http://www.zstd.net/
*/
/*-**************************************
* Compiler Options
****************************************/
/* Disable some Visual warning messages */
#ifdef _MSC_VER
# define _CRT_SECURE_NO_WARNINGS /* fopen */
# pragma warning(disable : 4127) /* disable: C4127: conditional expression is constant */
#endif
/* Unix Large Files support (>4GB) */
#define _FILE_OFFSET_BITS 64
#if (defined(__sun__) && (!defined(__LP64__))) /* Sun Solaris 32-bits requires specific definitions */
# define _LARGEFILE_SOURCE
#elif ! defined(__LP64__) /* No point defining Large file for 64 bit */
# define _LARGEFILE64_SOURCE
#endif
/*-*************************************
* Includes
***************************************/
#include "util.h" /* Compiler options, UTIL_GetFileSize, UTIL_getTotalFileSize */
#include <stdlib.h> /* malloc, free */
#include <string.h> /* memset */
#include <stdio.h> /* fprintf, fopen, ftello64 */
#include <sys/types.h> /* stat64 */
#include <sys/stat.h> /* stat64 */
#include <time.h> /* clock */
#include <time.h> /* clock_t, clock, CLOCKS_PER_SEC */
#include "mem.h" /* read */
#include "error_private.h"
#include "zdict_static.h"
/*-*************************************
* Compiler specifics
***************************************/
#if !defined(S_ISREG)
# define S_ISREG(x) (((x) & S_IFMT) == S_IFREG)
#endif
#include "dibio.h"
/*-*************************************
@@ -86,6 +59,19 @@ static const size_t maxMemory = (sizeof(size_t) == 4) ? (2 GB - 64 MB) : ((size_
#define DISPLAYLEVEL(l, ...) if (g_displayLevel>=l) { DISPLAY(__VA_ARGS__); }
static unsigned g_displayLevel = 0; /* 0 : no display; 1: errors; 2: default; 4: full information */
#define DISPLAYUPDATE(l, ...) if (g_displayLevel>=l) { \
if ((DIB_GetMilliSpan(g_time) > refreshRate) || (g_displayLevel>=4)) \
{ g_time = clock(); DISPLAY(__VA_ARGS__); \
if (g_displayLevel>=4) fflush(stdout); } }
static const unsigned refreshRate = 150;
static clock_t g_time = 0;
static unsigned DIB_GetMilliSpan(clock_t nPrevious)
{
clock_t const nCurrent = clock();
return (unsigned)(((nCurrent - nPrevious) * 1000) / CLOCKS_PER_SEC);
}
/*-*************************************
* Exceptions
@@ -115,31 +101,6 @@ const char* DiB_getErrorName(size_t errorCode) { return ERR_getErrorName(errorCo
/* ********************************************************
* File related operations
**********************************************************/
static unsigned long long DiB_getFileSize(const char* infilename)
{
int r;
#if defined(_MSC_VER)
struct _stat64 statbuf;
r = _stat64(infilename, &statbuf);
#else
struct stat statbuf;
r = stat(infilename, &statbuf);
#endif
if (r || !S_ISREG(statbuf.st_mode)) return 0; /* No good... */
return (unsigned long long)statbuf.st_size;
}
static unsigned long long DiB_getTotalFileSize(const char** fileNamesTable, unsigned nbFiles)
{
unsigned long long total = 0;
unsigned n;
for (n=0; n<nbFiles; n++)
total += DiB_getFileSize(fileNamesTable[n]);
return total;
}
static void DiB_loadFiles(void* buffer, size_t bufferSize,
size_t* fileSizes,
const char** fileNamesTable, unsigned nbFiles)
@@ -150,10 +111,10 @@ static void DiB_loadFiles(void* buffer, size_t bufferSize,
for (n=0; n<nbFiles; n++) {
size_t readSize;
unsigned long long fileSize = DiB_getFileSize(fileNamesTable[n]);
unsigned long long fileSize = UTIL_getFileSize(fileNamesTable[n]);
FILE* f = fopen(fileNamesTable[n], "rb");
if (f==NULL) EXM_THROW(10, "impossible to open file %s", fileNamesTable[n]);
DISPLAYLEVEL(2, "Loading %s... \r", fileNamesTable[n]);
DISPLAYUPDATE(2, "Loading %s... \r", fileNamesTable[n]);
if (fileSize > bufferSize-pos) fileSize = 0; /* stop there, not enough memory to load all files */
readSize = fread(buff+pos, 1, (size_t)fileSize, f);
if (readSize != (size_t)fileSize) EXM_THROW(11, "could not read %s", fileNamesTable[n]);
@@ -201,17 +162,14 @@ static void DiB_fillNoise(void* buffer, size_t length)
static void DiB_saveDict(const char* dictFileName,
const void* buff, size_t buffSize)
{
FILE* f;
size_t n;
f = fopen(dictFileName, "wb");
FILE* f = fopen(dictFileName, "wb");
if (f==NULL) EXM_THROW(3, "cannot open %s ", dictFileName);
n = fwrite(buff, 1, buffSize, f);
if (n!=buffSize) EXM_THROW(4, "%s : write error", dictFileName)
{ size_t const n = fwrite(buff, 1, buffSize, f);
if (n!=buffSize) EXM_THROW(4, "%s : write error", dictFileName) }
n = (size_t)fclose(f);
if (n!=0) EXM_THROW(5, "%s : flush error", dictFileName)
{ size_t const n = (size_t)fclose(f);
if (n!=0) EXM_THROW(5, "%s : flush error", dictFileName) }
}
@@ -234,7 +192,7 @@ int DiB_trainFromFiles(const char* dictFileName, unsigned maxDictSize,
void* srcBuffer;
size_t benchedSize;
size_t* fileSizes = (size_t*)malloc(nbFiles * sizeof(size_t));
unsigned long long totalSizeToLoad = DiB_getTotalFileSize(fileNamesTable, nbFiles);
unsigned long long totalSizeToLoad = UTIL_getTotalFileSize(fileNamesTable, nbFiles);
void* dictBuffer = malloc(maxDictSize);
size_t dictSize;
int result = 0;
+1 -1
View File
@@ -32,7 +32,7 @@
/*-*************************************
* Dependencies
***************************************/
#include "zdict_static.h" /* ZDICT_params_t */
#include "../lib/dictBuilder/zdict_static.h" /* ZDICT_params_t */
/*-*************************************
+73 -115
View File
@@ -41,27 +41,19 @@
/* *************************************
* Compiler Options
***************************************/
/* Disable some Visual warning messages */
#ifdef _MSC_VER
# define _CRT_SECURE_NO_WARNINGS
# define _CRT_SECURE_NO_DEPRECATE /* VS2005 */
# pragma warning(disable : 4127) /* disable: C4127: conditional expression is constant */
#endif
#define _FILE_OFFSET_BITS 64 /* Large file support on 32-bits unix */
#define _POSIX_SOURCE 1 /* enable fileno() within <stdio.h> on unix */
#define _POSIX_SOURCE 1 /* enable %llu on Windows */
/*-*************************************
* Includes
***************************************/
#include "util.h" /* Compiler options, UTIL_GetFileSize */
#include <stdio.h> /* fprintf, fopen, fread, _fileno, stdin, stdout */
#include <stdlib.h> /* malloc, free */
#include <string.h> /* strcmp, strlen */
#include <time.h> /* clock */
#include <errno.h> /* errno */
#include <sys/types.h> /* stat64 */
#include <sys/stat.h> /* stat64 */
#include "mem.h"
#include "fileio.h"
#include "zstd_static.h" /* ZSTD_magicNumber, ZSTD_frameHeaderSize_max */
@@ -80,25 +72,15 @@
# include <fcntl.h> /* _O_BINARY */
# include <io.h> /* _setmode, _isatty */
# define SET_BINARY_MODE(file) { int unused = _setmode(_fileno(file), _O_BINARY); (void)unused; }
# define IS_CONSOLE(stdStream) _isatty(_fileno(stdStream))
#else
# include <unistd.h> /* isatty */
# define SET_BINARY_MODE(file)
# define IS_CONSOLE(stdStream) isatty(fileno(stdStream))
#endif
#if !defined(S_ISREG)
# define S_ISREG(x) (((x) & S_IFMT) == S_IFREG)
#endif
/*-*************************************
* Constants
***************************************/
#define KB *(1U<<10)
#define MB *(1U<<20)
#define GB *(1U<<30)
#define _1BIT 0x01
#define _2BITS 0x03
#define _3BITS 0x07
@@ -106,9 +88,6 @@
#define _6BITS 0x3F
#define _8BITS 0xFF
#define BIT6 0x40
#define BIT7 0x80
#define BLOCKSIZE (128 KB)
#define ROLLBUFFERSIZE (BLOCKSIZE*8*64)
@@ -119,6 +98,8 @@
#define MAX_DICT_SIZE (1 MB) /* protection against large input (attack scenario) ; can be changed */
#define FNSPACE 30
/*-*************************************
* Macros
@@ -135,7 +116,11 @@ void FIO_setNotificationLevel(unsigned level) { g_displayLevel=level; }
static const unsigned refreshRate = 150;
static clock_t g_time = 0;
#define MAX(a,b) ((a)>(b)?(a):(b))
static unsigned FIO_GetMilliSpan(clock_t nPrevious)
{
clock_t const nCurrent = clock();
return (unsigned)(((nCurrent - nPrevious) * 1000) / CLOCKS_PER_SEC);
}
/*-*************************************
@@ -167,29 +152,6 @@ void FIO_setMaxWLog(unsigned maxWLog) { g_maxWLog = maxWLog; }
/*-*************************************
* Functions
***************************************/
static unsigned FIO_GetMilliSpan(clock_t nPrevious)
{
clock_t nCurrent = clock();
unsigned nSpan = (unsigned)(((nCurrent - nPrevious) * 1000) / CLOCKS_PER_SEC);
return nSpan;
}
static U64 FIO_getFileSize(const char* infilename)
{
int r;
#if defined(_MSC_VER)
struct _stat64 statbuf;
r = _stat64(infilename, &statbuf);
#else
struct stat statbuf;
r = stat(infilename, &statbuf);
#endif
if (r || !S_ISREG(statbuf.st_mode)) return 0;
return (U64)statbuf.st_size;
}
static FILE* FIO_openSrcFile(const char* srcFileName)
{
FILE* f;
@@ -258,7 +220,7 @@ static size_t FIO_loadFile(void** bufferPtr, const char* fileName)
DISPLAYLEVEL(4,"Loading %s as dictionary \n", fileName);
fileHandle = fopen(fileName, "rb");
if (fileHandle==0) EXM_THROW(31, "Error opening file %s", fileName);
fileSize = FIO_getFileSize(fileName);
fileSize = UTIL_getFileSize(fileName);
if (fileSize > MAX_DICT_SIZE) {
int seekResult;
if (fileSize > 1 GB) EXM_THROW(32, "Dictionary file %s is too large", fileName); /* avoid extreme cases */
@@ -275,6 +237,7 @@ static size_t FIO_loadFile(void** bufferPtr, const char* fileName)
return (size_t)fileSize;
}
#ifndef ZSTD_NOCOMPRESS
/*-**********************************************************************
* Compression
@@ -331,21 +294,20 @@ static int FIO_compressFilename_internal(cRess_t ress,
const char* dstFileName, const char* srcFileName,
int cLevel)
{
FILE* srcFile = ress.srcFile;
FILE* dstFile = ress.dstFile;
FILE* const srcFile = ress.srcFile;
FILE* const dstFile = ress.dstFile;
U64 readsize = 0;
U64 compressedfilesize = 0;
size_t dictSize = ress.dictBufferSize;
size_t sizeCheck;
ZSTD_parameters params;
U64 const fileSize = FIO_getFileSize(srcFileName);
U64 const fileSize = UTIL_getFileSize(srcFileName);
/* init */
params.cParams = ZSTD_getCParams(cLevel, fileSize, dictSize);
params.fParams.contentSizeFlag = 1;
if (g_maxWLog) if (params.cParams.windowLog > g_maxWLog) params.cParams.windowLog = g_maxWLog;
{ size_t const errorCode = ZBUFF_compressInit_advanced(ress.ctx, ress.dictBuffer, ress.dictBufferSize, params, fileSize);
if (ZBUFF_isError(errorCode)) EXM_THROW(21, "Error initializing compression : %s", ZBUFF_getErrorName(errorCode)); }
{ ZSTD_parameters params;
params.cParams = ZSTD_getCParams(cLevel, fileSize, ress.dictBufferSize);
params.fParams.contentSizeFlag = 1;
if (g_maxWLog) if (params.cParams.windowLog > g_maxWLog) params.cParams.windowLog = g_maxWLog;
{ size_t const errorCode = ZBUFF_compressInit_advanced(ress.ctx, ress.dictBuffer, ress.dictBufferSize, params, fileSize);
if (ZBUFF_isError(errorCode)) EXM_THROW(21, "Error initializing compression : %s", ZBUFF_getErrorName(errorCode)); }
}
/* Main compression loop */
readsize = 0;
@@ -359,16 +321,15 @@ static int FIO_compressFilename_internal(cRess_t ress,
{ /* Compress using buffered streaming */
size_t usedInSize = inSize;
size_t cSize = ress.dstBufferSize;
size_t result = ZBUFF_compressContinue(ress.ctx, ress.dstBuffer, &cSize, ress.srcBuffer, &usedInSize);
if (ZBUFF_isError(result))
EXM_THROW(23, "Compression error : %s ", ZBUFF_getErrorName(result));
{ size_t const result = ZBUFF_compressContinue(ress.ctx, ress.dstBuffer, &cSize, ress.srcBuffer, &usedInSize);
if (ZBUFF_isError(result)) EXM_THROW(23, "Compression error : %s ", ZBUFF_getErrorName(result)); }
if (inSize != usedInSize)
/* inBuff should be entirely consumed since buffer sizes are recommended ones */
EXM_THROW(24, "Compression error : input block not fully consumed");
/* Write cBlock */
sizeCheck = fwrite(ress.dstBuffer, 1, cSize, dstFile);
if (sizeCheck!=cSize) EXM_THROW(25, "Write error : cannot write compressed block into %s", dstFileName);
{ size_t const sizeCheck = fwrite(ress.dstBuffer, 1, cSize, dstFile);
if (sizeCheck!=cSize) EXM_THROW(25, "Write error : cannot write compressed block into %s", dstFileName); }
compressedfilesize += cSize;
}
DISPLAYUPDATE(2, "\rRead : %u MB ==> %.2f%% ", (U32)(readsize>>20), (double)compressedfilesize/readsize*100);
@@ -379,8 +340,8 @@ static int FIO_compressFilename_internal(cRess_t ress,
size_t const result = ZBUFF_compressEnd(ress.ctx, ress.dstBuffer, &cSize);
if (result!=0) EXM_THROW(26, "Compression error : cannot create frame end");
sizeCheck = fwrite(ress.dstBuffer, 1, cSize, dstFile);
if (sizeCheck!=cSize) EXM_THROW(27, "Write error : cannot write frame end into %s", dstFileName);
{ size_t const sizeCheck = fwrite(ress.dstBuffer, 1, cSize, dstFile);
if (sizeCheck!=cSize) EXM_THROW(27, "Write error : cannot write frame end into %s", dstFileName); }
compressedfilesize += cSize;
}
@@ -410,7 +371,6 @@ static int FIO_compressFilename_srcFile(cRess_t ress,
result = FIO_compressFilename_internal(ress, dstFileName, srcFileName, cLevel);
/* clean */
fclose(ress.srcFile);
return result;
}
@@ -443,7 +403,7 @@ static int FIO_compressFilename_extRess(cRess_t ress,
int FIO_compressFilename(const char* dstFileName, const char* srcFileName,
const char* dictFileName, int compressionLevel)
{
clock_t start, end;
clock_t start;
cRess_t ress;
int issueWithSrcFile = 0;
@@ -451,33 +411,25 @@ int FIO_compressFilename(const char* dstFileName, const char* srcFileName,
start = clock();
ress = FIO_createCResources(dictFileName);
/* Compress File */
issueWithSrcFile += FIO_compressFilename_extRess(ress, dstFileName, srcFileName, compressionLevel);
/* Free resources */
FIO_freeCResources(ress);
/* Final Status */
end = clock();
{
double seconds = (double)(end - start) / CLOCKS_PER_SEC;
{ double seconds = (double)(clock() - start) / CLOCKS_PER_SEC;
DISPLAYLEVEL(4, "Completed in %.2f sec \n", seconds);
}
return issueWithSrcFile;
}
#define FNSPACE 30
int FIO_compressMultipleFilenames(const char** inFileNamesTable, unsigned nbFiles,
const char* suffix,
const char* dictFileName, int compressionLevel)
{
unsigned u;
int missed_files = 0;
char* dstFileName = (char*)malloc(FNSPACE);
char* dstFileName = (char*)malloc(FNSPACE);
size_t dfnSize = FNSPACE;
const size_t suffixSize = suffix ? strlen(suffix) : 0;
size_t const suffixSize = suffix ? strlen(suffix) : 0;
cRess_t ress;
/* init */
@@ -485,12 +437,14 @@ int FIO_compressMultipleFilenames(const char** inFileNamesTable, unsigned nbFile
/* loop on each file */
if (!strcmp(suffix, stdoutmark)) {
unsigned u;
ress.dstFile = stdout;
for (u=0; u<nbFiles; u++)
missed_files += FIO_compressFilename_srcFile(ress, stdoutmark,
inFileNamesTable[u], compressionLevel);
if (fclose(ress.dstFile)) EXM_THROW(29, "Write error : cannot properly close %s", stdoutmark);
} else {
unsigned u;
for (u=0; u<nbFiles; u++) {
size_t ifnSize = strlen(inFileNamesTable[u]);
if (dfnSize <= ifnSize+suffixSize+1) { free(dstFileName); dfnSize = ifnSize + 20; dstFileName = (char*)malloc(dfnSize); }
@@ -507,6 +461,11 @@ int FIO_compressMultipleFilenames(const char** inFileNamesTable, unsigned nbFile
return missed_files;
}
#endif /* #ifndef ZSTD_NOCOMPRESS */
#ifndef ZSTD_NODECOMPRESS
/* **************************************************************************
* Decompression
@@ -545,7 +504,7 @@ static dRess_t FIO_createDResources(const char* dictFileName)
static void FIO_freeDResources(dRess_t ress)
{
size_t errorCode = ZBUFF_freeDCtx(ress.dctx);
size_t const errorCode = ZBUFF_freeDCtx(ress.dctx);
if (ZBUFF_isError(errorCode)) EXM_THROW(69, "Error : can't free ZBUFF context resource : %s", ZBUFF_getErrorName(errorCode));
free(ress.srcBuffer);
free(ress.dstBuffer);
@@ -564,18 +523,17 @@ unsigned long long FIO_decompressFrame(dRess_t ress,
ZBUFF_decompressInitDictionary(ress.dctx, ress.dictBuffer, ress.dictBufferSize);
/* Complete Header loading */
{ size_t const toLoad = ZSTD_frameHeaderSize_max - alreadyLoaded; /* assumption : alreadyLoaded <= ZSTD_frameHeaderSize_max */
size_t const checkSize = fread(((char*)ress.srcBuffer) + alreadyLoaded, 1, toLoad, finput);
if (checkSize != toLoad) EXM_THROW(32, "Read error"); /* assumption : srcSize >= ZSTD_frameHeaderSize_max */
/* Header loading (optional, saves one loop) */
{ size_t const toLoad = ZSTD_frameHeaderSize_min - alreadyLoaded; /* assumption : ZSTD_frameHeaderSize_min >= alreadyLoaded */
size_t const loadedSize = fread(((char*)ress.srcBuffer) + alreadyLoaded, 1, toLoad, finput);
readSize = alreadyLoaded + loadedSize;
}
readSize = ZSTD_frameHeaderSize_max;
/* Main decompression Loop */
while (1) {
/* Decode */
size_t inSize=readSize, decodedSize=ress.dstBufferSize;
size_t toRead = ZBUFF_decompressContinue(ress.dctx, ress.dstBuffer, &decodedSize, ress.srcBuffer, &inSize);
size_t const toRead = ZBUFF_decompressContinue(ress.dctx, ress.dstBuffer, &decodedSize, ress.srcBuffer, &inSize);
if (ZBUFF_isError(toRead)) EXM_THROW(36, "Decoding error : %s", ZBUFF_getErrorName(toRead));
readSize -= inSize;
@@ -607,8 +565,8 @@ unsigned long long FIO_decompressFrame(dRess_t ress,
static int FIO_decompressSrcFile(dRess_t ress, const char* srcFileName)
{
unsigned long long filesize = 0;
FILE* dstFile = ress.dstFile;
FILE* srcFile = FIO_openSrcFile(srcFileName);
FILE* const dstFile = ress.dstFile;
FILE* const srcFile = FIO_openSrcFile(srcFileName);
if (srcFile==0) return 1;
/* for each frame */
@@ -618,16 +576,17 @@ static int FIO_decompressSrcFile(dRess_t ress, const char* srcFileName)
size_t const sizeCheck = fread(ress.srcBuffer, (size_t)1, toRead, srcFile);
if (sizeCheck==0) break; /* no more input */
if (sizeCheck != toRead) EXM_THROW(31, "zstd: %s read error : cannot read header", srcFileName);
#if defined(ZSTD_LEGACY_SUPPORT) && (ZSTD_LEGACY_SUPPORT==1)
if (ZSTD_isLegacy(MEM_readLE32(ress.srcBuffer))) {
filesize += FIO_decompressLegacyFrame(dstFile, srcFile, MEM_readLE32(ress.srcBuffer));
continue;
}
#endif /* ZSTD_LEGACY_SUPPORT */
if (MEM_readLE32(ress.srcBuffer) != ZSTD_MAGICNUMBER) {
DISPLAYLEVEL(1, "zstd: %s: not in zstd format \n", srcFileName);
return 1;
}
{ U32 const magic = MEM_readLE32(ress.srcBuffer);
#if defined(ZSTD_LEGACY_SUPPORT) && (ZSTD_LEGACY_SUPPORT>=1)
if (ZSTD_isLegacy(magic)) {
filesize += FIO_decompressLegacyFrame(dstFile, srcFile, ress.dictBuffer, ress.dictBufferSize, magic);
continue;
}
#endif
if (magic != ZSTD_MAGICNUMBER) {
DISPLAYLEVEL(1, "zstd: %s: not in zstd format \n", srcFileName);
return 1;
} }
filesize += FIO_decompressFrame(ress, dstFile, srcFile, toRead);
}
@@ -654,9 +613,7 @@ static int FIO_decompressFile_extRess(dRess_t ress,
if (ress.dstFile==0) return 1;
result = FIO_decompressSrcFile(ress, srcFileName);
if (result != 0) {
remove(dstFileName);
}
if (result != 0) remove(dstFileName);
if (fclose(ress.dstFile)) EXM_THROW(38, "Write error : cannot properly close %s", dstFileName);
return result;
@@ -681,28 +638,27 @@ int FIO_decompressMultipleFilenames(const char** srcNamesTable, unsigned nbFiles
const char* suffix,
const char* dictFileName)
{
unsigned u;
int skippedFiles = 0;
int missingFiles = 0;
char* dstFileName = (char*)malloc(FNSPACE);
size_t dfnSize = FNSPACE;
const size_t suffixSize = suffix ? strlen(suffix) : 0;
dRess_t ress;
if (dstFileName==NULL) EXM_THROW(70, "not enough memory for dstFileName");
ress = FIO_createDResources(dictFileName);
dRess_t ress = FIO_createDResources(dictFileName);
if (!strcmp(suffix, stdoutmark) || !strcmp(suffix, nulmark)) {
unsigned u;
ress.dstFile = FIO_openDstFile(suffix);
if (ress.dstFile == 0) EXM_THROW(71, "cannot open %s", suffix);
for (u=0; u<nbFiles; u++)
missingFiles += FIO_decompressSrcFile(ress, srcNamesTable[u]);
if (fclose(ress.dstFile)) EXM_THROW(39, "Write error : cannot properly close %s", stdoutmark);
} else {
size_t const suffixSize = suffix ? strlen(suffix) : 0;
size_t dfnSize = FNSPACE;
unsigned u;
char* dstFileName = (char*)malloc(FNSPACE);
if (dstFileName==NULL) EXM_THROW(70, "not enough memory for dstFileName");
for (u=0; u<nbFiles; u++) { /* create dstFileName */
const char* srcFileName = srcNamesTable[u];
size_t sfnSize = strlen(srcFileName);
const char* suffixPtr = srcFileName + sfnSize - suffixSize;
const char* const srcFileName = srcNamesTable[u];
size_t const sfnSize = strlen(srcFileName);
const char* const suffixPtr = srcFileName + sfnSize - suffixSize;
if (dfnSize+suffixSize <= sfnSize+1) {
free(dstFileName);
dfnSize = sfnSize + 20;
@@ -718,10 +674,12 @@ int FIO_decompressMultipleFilenames(const char** srcNamesTable, unsigned nbFiles
dstFileName[sfnSize-suffixSize] = '\0';
missingFiles += FIO_decompressFile_extRess(ress, dstFileName, srcFileName);
} }
}
free(dstFileName);
}
FIO_freeDResources(ress);
free(dstFileName);
return missingFiles + skippedFiles;
}
#endif /* #ifndef ZSTD_NODECOMPRESS */
+6 -47
View File
@@ -22,31 +22,14 @@
- zstd homepage : http://www.zstd.net
*/
/*_************************************
* Compiler Options
**************************************/
/* Disable some Visual warning messages */
#define _CRT_SECURE_NO_WARNINGS
#define _CRT_SECURE_NO_DEPRECATE /* VS2005 */
/* Unix Large Files support (>4GB) */
#if (defined(__sun__) && (!defined(__LP64__))) /* Sun Solaris 32-bits requires specific definitions */
# define _LARGEFILE_SOURCE
# define _FILE_OFFSET_BITS 64
#elif ! defined(__LP64__) /* No point defining Large file for 64 bit */
# define _LARGEFILE64_SOURCE
#endif
/*_************************************
* Includes
**************************************/
#include <stdlib.h> /* malloc */
#include <stdio.h> /* fprintf, fopen, ftello64 */
#include <sys/types.h> /* stat64 */
#include <sys/stat.h> /* stat64 */
#include <string.h> /* strcmp */
#include <time.h> /* clock_t, clock, CLOCKS_PER_SEC */
#include "util.h" /* Compiler options, UTIL_GetFileSize */
#include <stdlib.h> /* malloc */
#include <stdio.h> /* fprintf, fopen, ftello64 */
#include <string.h> /* strcmp */
#include <time.h> /* clock_t, clock, CLOCKS_PER_SEC */
#include "mem.h"
#include "zstd_static.h"
@@ -55,15 +38,6 @@
#include "datagen.h"
/*_************************************
* Compiler Options
**************************************/
/* S_ISREG & gettimeofday() are not supported by MSVC */
#if !defined(S_ISREG)
# define S_ISREG(x) (((x) & S_IFMT) == S_IFREG)
#endif
/*_************************************
* Constants
**************************************/
@@ -135,21 +109,6 @@ static size_t BMK_findMaxMem(U64 requiredMem)
}
static U64 BMK_GetFileSize(const char* infilename)
{
int r;
#if defined(_MSC_VER)
struct _stat64 statbuf;
r = _stat64(infilename, &statbuf);
#else
struct stat statbuf;
r = stat(infilename, &statbuf);
#endif
if (r || !S_ISREG(statbuf.st_mode)) return 0; /* No good... */
return (U64)statbuf.st_size;
}
/*_*******************************************************
* Benchmark wrappers
*********************************************************/
@@ -446,7 +405,7 @@ static int benchFiles(const char** fileNamesTable, const int nbFiles, U32 benchN
if (inFile==NULL) { DISPLAY( "Pb opening %s\n", inFileName); return 11; }
/* Memory allocation & restrictions */
inFileSize = BMK_GetFileSize(inFileName);
inFileSize = UTIL_getFileSize(inFileName);
benchedSize = BMK_findMaxMem(inFileSize*3) / 3;
if ((U64)benchedSize > inFileSize) benchedSize = (size_t)inFileSize;
if (benchedSize < inFileSize)
+18 -18
View File
@@ -38,8 +38,6 @@
# pragma warning(disable : 4127) /* disable: C4127: conditional expression is constant */
#endif
#define GCC_VERSION (__GNUC__ * 100 + __GNUC_MINOR__)
#define _FILE_OFFSET_BITS 64 /* Large file support on 32-bits unix */
#define _POSIX_SOURCE 1 /* enable fileno() within <stdio.h> on unix */
@@ -92,8 +90,8 @@
#define BIT6 0x40
#define BIT7 0x80
#define FIO_FRAMEHEADERSIZE 5 /* as a define, because needed to allocated table on stack */
#define FSE_CHECKSUM_SEED 0
#define FIO_FRAMEHEADERSIZE 5 /* as a define, because needed to allocated table on stack */
#define FSE_CHECKSUM_SEED 0
#define CACHELINE 64
@@ -149,7 +147,7 @@ static unsigned FIO_GetMilliSpan(clock_t nPrevious)
unsigned long long FIOv01_decompressFrame(FILE* foutput, FILE* finput)
{
size_t outBuffSize = 512 KB;
size_t const outBuffSize = 512 KB;
BYTE* outBuff = (BYTE*)malloc(outBuffSize);
size_t inBuffSize = 128 KB + 8;
BYTE inBuff[128 KB + 8];
@@ -208,7 +206,7 @@ unsigned long long FIOv01_decompressFrame(FILE* foutput, FILE* finput)
unsigned long long FIOv02_decompressFrame(FILE* foutput, FILE* finput)
{
size_t outBuffSize = 512 KB;
size_t const outBuffSize = 512 KB;
BYTE* outBuff = (BYTE*)malloc(outBuffSize);
size_t inBuffSize = 128 KB + 8;
BYTE inBuff[128 KB + 8];
@@ -266,7 +264,7 @@ unsigned long long FIOv02_decompressFrame(FILE* foutput, FILE* finput)
unsigned long long FIOv03_decompressFrame(FILE* foutput, FILE* finput)
{
size_t outBuffSize = 512 KB;
size_t const outBuffSize = 512 KB;
BYTE* outBuff = (BYTE*)malloc(outBuffSize);
size_t inBuffSize = 128 KB + 8;
BYTE inBuff[128 KB + 8];
@@ -407,7 +405,7 @@ typedef struct {
size_t srcBufferSize;
void* dstBuffer;
size_t dstBufferSize;
void* dictBuffer;
const void* dictBuffer;
size_t dictBufferSize;
ZBUFFv05_DCtx* dctx;
} dRessv05_t;
@@ -437,7 +435,6 @@ static void FIOv05_freeDResources(dRessv05_t ress)
if (ZBUFFv05_isError(errorCode)) EXM_THROW(69, "Error : can't free ZBUFF context resource : %s", ZBUFFv05_getErrorName(errorCode));
free(ress.srcBuffer);
free(ress.dstBuffer);
free(ress.dictBuffer);
}
@@ -445,22 +442,21 @@ unsigned long long FIOv05_decompressFrame(dRessv05_t ress,
FILE* foutput, FILE* finput)
{
U64 frameSize = 0;
size_t readSize = 4;
size_t readSize = 4;
MEM_writeLE32(ress.srcBuffer, ZSTDv05_MAGICNUMBER);
ZBUFFv05_decompressInitDictionary(ress.dctx, ress.dictBuffer, ress.dictBufferSize);
while (1) {
/* Decode */
size_t sizeCheck;
size_t inSize=readSize, decodedSize=ress.dstBufferSize;
size_t toRead = ZBUFFv05_decompressContinue(ress.dctx, ress.dstBuffer, &decodedSize, ress.srcBuffer, &inSize);
if (ZBUFFv05_isError(toRead)) EXM_THROW(36, "Decoding error : %s", ZBUFFv05_getErrorName(toRead));
readSize -= inSize;
/* Write block */
sizeCheck = fwrite(ress.dstBuffer, 1, decodedSize, foutput);
if (sizeCheck != decodedSize) EXM_THROW(37, "Write error : unable to write data block to destination file");
{ size_t const sizeCheck = fwrite(ress.dstBuffer, 1, decodedSize, foutput);
if (sizeCheck != decodedSize) EXM_THROW(37, "Write error : unable to write data block to destination file"); }
frameSize += decodedSize;
DISPLAYUPDATE(2, "\rDecoded : %u MB... ", (U32)(frameSize>>20) );
@@ -479,7 +475,9 @@ unsigned long long FIOv05_decompressFrame(dRessv05_t ress,
/*===== General legacy dispatcher =====*/
unsigned long long FIO_decompressLegacyFrame(FILE* foutput, FILE* finput, U32 magicNumberLE)
unsigned long long FIO_decompressLegacyFrame(FILE* foutput, FILE* finput,
const void* dictBuffer, size_t dictSize,
U32 magicNumberLE)
{
switch(magicNumberLE)
{
@@ -497,10 +495,12 @@ unsigned long long FIO_decompressLegacyFrame(FILE* foutput, FILE* finput, U32 ma
}
case ZSTDv05_MAGICNUMBER :
{ dRessv05_t r = FIOv05_createDResources();
unsigned long long const s = FIOv05_decompressFrame(r, foutput, finput);
FIOv05_freeDResources(r);
return s;
}
r.dictBuffer = dictBuffer;
r.dictBufferSize = dictSize;
{ unsigned long long const s = FIOv05_decompressFrame(r, foutput, finput);
FIOv05_freeDResources(r);
return s;
} }
default :
return ERROR(prefix_unknown);
}
+3 -1
View File
@@ -38,7 +38,9 @@ void FIO_legacy_setNotificationLevel(unsigned level);
/* *************************************
* Stream/File functions
***************************************/
unsigned long long FIO_decompressLegacyFrame(FILE* foutput, FILE* finput, U32 magicNumberLE);
unsigned long long FIO_decompressLegacyFrame(FILE* foutput, FILE* finput,
const void* dictBuffer, size_t dictSize,
U32 magicNumberLE);
#if defined (__cplusplus)
+5 -45
View File
@@ -25,35 +25,18 @@
/*-************************************
* Compiler Options
**************************************/
/* Disable some Visual warning messages */
#define _CRT_SECURE_NO_WARNINGS
#define _CRT_SECURE_NO_DEPRECATE /* VS2005 */
/* Unix Large Files support (>4GB) */
#if (defined(__sun__) && (!defined(__LP64__))) /* Sun Solaris 32-bits requires specific definitions */
# define _LARGEFILE_SOURCE
# define _FILE_OFFSET_BITS 64
#elif ! defined(__LP64__) /* No point defining Large file for 64 bit */
# define _LARGEFILE64_SOURCE
#endif
/* S_ISREG & gettimeofday() are not supported by MSVC */
/* gettimeofday() are not supported by MSVC */
#if defined(_MSC_VER) || defined(_WIN32)
# define BMK_LEGACY_TIMER 1
#endif
#if defined(_MSC_VER)
# define snprintf _snprintf /* snprintf unsupported by Visual <= 2012 */
#endif
/*-************************************
* Dependencies
**************************************/
#include "util.h" /* Compiler options, UTIL_GetFileSize */
#include <stdlib.h> /* malloc */
#include <stdio.h> /* fprintf, fopen, ftello64 */
#include <sys/types.h> /* stat64 */
#include <sys/stat.h> /* stat64 */
#include <string.h> /* strcmp */
#include <math.h> /* log */
@@ -70,19 +53,10 @@
#include "xxhash.h"
/*-************************************
* Compiler Options
**************************************/
/* S_ISREG & gettimeofday() are not supported by MSVC */
#if !defined(S_ISREG)
# define S_ISREG(x) (((x) & S_IFMT) == S_IFREG)
#endif
/*-************************************
* Constants
**************************************/
#define PROGRAM_DESCRIPTION "ZSTD_HC parameters tester"
#define PROGRAM_DESCRIPTION "ZSTD parameters tester"
#ifndef ZSTD_VERSION
# define ZSTD_VERSION ""
#endif
@@ -197,20 +171,6 @@ static size_t BMK_findMaxMem(U64 requiredMem)
}
static U64 BMK_GetFileSize(char* infilename)
{
int r;
#if defined(_MSC_VER)
struct _stat64 statbuf;
r = _stat64(infilename, &statbuf);
#else
struct stat statbuf;
r = stat(infilename, &statbuf);
#endif
if (r || !S_ISREG(statbuf.st_mode)) return 0; /* No good... */
return (U64)statbuf.st_size;
}
# define FUZ_rotl32(x,r) ((x << r) | (x >> (32 - r)))
U32 FUZ_rand(U32* src)
{
@@ -790,7 +750,7 @@ int benchFiles(char** fileNamesTable, int nbFiles)
}
/* Memory allocation & restrictions */
inFileSize = BMK_GetFileSize(inFileName);
inFileSize = UTIL_getFileSize(inFileName);
benchedSize = BMK_findMaxMem(inFileSize*3) / 3;
if ((U64)benchedSize > inFileSize) benchedSize = (size_t)inFileSize;
if (benchedSize < inFileSize)
@@ -841,7 +801,7 @@ int optimizeForSize(char* inFileName)
}
/* Memory allocation & restrictions */
inFileSize = BMK_GetFileSize(inFileName);
inFileSize = UTIL_getFileSize(inFileName);
benchedSize = (size_t) BMK_findMaxMem(inFileSize*3) / 3;
if ((U64)benchedSize > inFileSize) benchedSize = (size_t)inFileSize;
if (benchedSize < inFileSize)
+5 -2
View File
@@ -29,8 +29,11 @@ $ZSTD -f tmp # trivial compression case, creates tmp
$ZSTD -df tmp.zst # trivial decompression case (overwrites tmp)
echo "test : too large compression level (must fail)"
$ZSTD -99 tmp && die "too large compression level undetected"
$ZSTD tmp -c > tmpCompressed # compression using stdout
$ZSTD tmp --stdout > tmpCompressed # compressoin using stdout, long format
echo "test : compress to stdout"
$ZSTD tmp -c > tmpCompressed
$ZSTD tmp --stdout > tmpCompressed # long command format
echo "test : null-length file roundtrip"
echo -n '' | $ZSTD - --stdout | $ZSTD -d --stdout
echo "test : decompress file with wrong suffix (must fail)"
$ZSTD -d tmpCompressed && die "wrong suffix error not detected!"
$ZSTD -d tmpCompressed -c > tmpResult # decompression using stdout
+399
View File
@@ -0,0 +1,399 @@
/* ******************************************************************
util.h - utility functions
Copyright (C) 2016, Przemyslaw Skibinski, Yann Collet.
GPL v2 License
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License along
with this program; if not, write to the Free Software Foundation, Inc.,
51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
You can contact the author at :
- ZSTD homepage : http://www.zstd.net/
*/
#ifndef UTIL_H_MODULE
#define UTIL_H_MODULE
#if defined (__cplusplus)
extern "C" {
#endif
/* **************************************
* Compiler Options
****************************************/
#if defined(_MSC_VER)
# define _CRT_SECURE_NO_WARNINGS /* Disable some Visual warning messages for fopen, strncpy */
# define _CRT_SECURE_NO_DEPRECATE /* VS2005 */
# pragma warning(disable : 4127) /* disable: C4127: conditional expression is constant */
#if _MSC_VER <= 1800 /* (1800 = Visual Studio 2013) */
#define snprintf sprintf_s /* snprintf unsupported by Visual <= 2013 */
//#define snprintf _snprintf
#endif
#endif
/* Unix Large Files support (>4GB) */
#if !defined(__LP64__) /* No point defining Large file for 64 bit */
# define _FILE_OFFSET_BITS 64 /* turn off_t into a 64-bit type for ftello, fseeko */
# if defined(__sun__) /* Sun Solaris 32-bits requires specific definitions */
# define _LARGEFILE_SOURCE /* fseeko, ftello */
# else
# define _LARGEFILE64_SOURCE /* off64_t, fseeko64, ftello64 */
# endif
#endif
/*-****************************************
* Dependencies
******************************************/
#include <stdlib.h> /* features.h with _POSIX_C_SOURCE, malloc */
#include <stdio.h> /* fprintf */
#include <sys/types.h> /* stat */
#include <sys/stat.h> /* stat */
#include "mem.h" /* U32, U64 */
/* *************************************
* Constants
***************************************/
#define LIST_SIZE_INCREASE (8*1024)
/*-****************************************
* Compiler specifics
******************************************/
#if defined(__GNUC__)
# define UTIL_STATIC static __attribute__((unused))
#elif defined (__cplusplus) || (defined (__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L) /* C99 */)
# define UTIL_STATIC static inline
#elif defined(_MSC_VER)
# define UTIL_STATIC static __inline
#else
# define UTIL_STATIC static /* this version may generate warnings for unused static functions; disable the relevant warning */
#endif
/*-****************************************
* Sleep functions: Windows - Posix - others
******************************************/
#if defined(_WIN32)
# include <windows.h>
# define SET_HIGH_PRIORITY SetPriorityClass(GetCurrentProcess(), REALTIME_PRIORITY_CLASS)
# define UTIL_sleep(s) Sleep(1000*s)
# define UTIL_sleepMilli(milli) Sleep(milli)
#elif (defined(__unix__) || defined(__unix) || defined(__midipix__) || (defined(__APPLE__) && defined(__MACH__)))
# include <unistd.h>
# include <sys/resource.h> /* setpriority */
# include <time.h> /* clock_t, nanosleep, clock, CLOCKS_PER_SEC */
# define SET_HIGH_PRIORITY setpriority(PRIO_PROCESS, 0, -20)
# define UTIL_sleep(s) sleep(s)
# if defined(_POSIX_C_SOURCE) && (_POSIX_C_SOURCE >= 199309L)
# define UTIL_sleepMilli(milli) { struct timespec t; t.tv_sec=0; t.tv_nsec=milli*1000000ULL; nanosleep(&t, NULL); }
# else
# define UTIL_sleepMilli(milli) /* disabled */
# endif
#else
# define SET_HIGH_PRIORITY /* disabled */
# define UTIL_sleep(s) /* disabled */
# define UTIL_sleepMilli(milli) /* disabled */
#endif
/*-****************************************
* Time functions
******************************************/
#if !defined(_WIN32)
typedef clock_t UTIL_time_t;
UTIL_STATIC void UTIL_initTimer(UTIL_time_t* ticksPerSecond) { *ticksPerSecond=0; }
UTIL_STATIC void UTIL_getTime(UTIL_time_t* x) { *x = clock(); }
UTIL_STATIC U64 UTIL_getSpanTimeMicro(UTIL_time_t ticksPerSecond, UTIL_time_t clockStart, UTIL_time_t clockEnd) { (void)ticksPerSecond; return 1000000ULL * (clockEnd - clockStart) / CLOCKS_PER_SEC; }
UTIL_STATIC U64 UTIL_getSpanTimeNano(UTIL_time_t ticksPerSecond, UTIL_time_t clockStart, UTIL_time_t clockEnd) { (void)ticksPerSecond; return 1000000000ULL * (clockEnd - clockStart) / CLOCKS_PER_SEC; }
#else
typedef LARGE_INTEGER UTIL_time_t;
UTIL_STATIC void UTIL_initTimer(UTIL_time_t* ticksPerSecond) { if (!QueryPerformanceFrequency(ticksPerSecond)) fprintf(stderr, "ERROR: QueryPerformance not present\n"); }
UTIL_STATIC void UTIL_getTime(UTIL_time_t* x) { QueryPerformanceCounter(x); }
UTIL_STATIC U64 UTIL_getSpanTimeMicro(UTIL_time_t ticksPerSecond, UTIL_time_t clockStart, UTIL_time_t clockEnd) { return 1000000ULL*(clockEnd.QuadPart - clockStart.QuadPart)/ticksPerSecond.QuadPart; }
UTIL_STATIC U64 UTIL_getSpanTimeNano(UTIL_time_t ticksPerSecond, UTIL_time_t clockStart, UTIL_time_t clockEnd) { return 1000000000ULL*(clockEnd.QuadPart - clockStart.QuadPart)/ticksPerSecond.QuadPart; }
#endif
/* returns time span in microseconds */
UTIL_STATIC U64 UTIL_clockSpanMicro( UTIL_time_t clockStart, UTIL_time_t ticksPerSecond )
{
UTIL_time_t clockEnd;
UTIL_getTime(&clockEnd);
return UTIL_getSpanTimeMicro(ticksPerSecond, clockStart, clockEnd);
}
UTIL_STATIC void UTIL_waitForNextTick(UTIL_time_t ticksPerSecond)
{
UTIL_time_t clockStart, clockEnd;
UTIL_getTime(&clockStart);
do {
UTIL_getTime(&clockEnd);
} while (UTIL_getSpanTimeNano(ticksPerSecond, clockStart, clockEnd) == 0);
}
/*-****************************************
* File functions
******************************************/
UTIL_STATIC U64 UTIL_getFileSize(const char* infilename)
{
int r;
#if defined(_MSC_VER)
struct _stat64 statbuf;
r = _stat64(infilename, &statbuf);
if (r || !(statbuf.st_mode & S_IFREG)) return 0; /* No good... */
#else
struct stat statbuf;
r = stat(infilename, &statbuf);
if (r || !S_ISREG(statbuf.st_mode)) return 0; /* No good... */
#endif
return (U64)statbuf.st_size;
}
UTIL_STATIC U64 UTIL_getTotalFileSize(const char** fileNamesTable, unsigned nbFiles)
{
U64 total = 0;
unsigned n;
for (n=0; n<nbFiles; n++)
total += UTIL_getFileSize(fileNamesTable[n]);
return total;
}
UTIL_STATIC int UTIL_doesFileExists(const char* infilename)
{
int r;
#if defined(_MSC_VER)
struct _stat64 statbuf;
r = _stat64(infilename, &statbuf);
if (r || !(statbuf.st_mode & S_IFREG)) return 0; /* No good... */
#else
struct stat statbuf;
r = stat(infilename, &statbuf);
if (r || !S_ISREG(statbuf.st_mode)) return 0; /* No good... */
#endif
return 1;
}
UTIL_STATIC U32 UTIL_isDirectory(const char* infilename)
{
int r;
#if defined(_MSC_VER)
struct _stat64 statbuf;
r = _stat64(infilename, &statbuf);
if (!r && (statbuf.st_mode & _S_IFDIR)) return 1;
#else
struct stat statbuf;
r = stat(infilename, &statbuf);
if (!r && S_ISDIR(statbuf.st_mode)) return 1;
#endif
return 0;
}
#ifdef _WIN32
# define UTIL_HAS_CREATEFILELIST
UTIL_STATIC int UTIL_prepareFileList(const char *dirName, char** bufStart, size_t* pos, char** bufEnd)
{
char path[MAX_PATH];
int pathLength, nbFiles = 0;
WIN32_FIND_DATA cFile;
HANDLE hFile;
pathLength = snprintf(path, MAX_PATH, "%s\\*", dirName);
if (pathLength < 0 || pathLength >= MAX_PATH) {
fprintf(stderr, "Path length has got too long.\n");
return 0;
}
hFile=FindFirstFile(path, &cFile);
if (hFile == INVALID_HANDLE_VALUE) {
fprintf(stderr, "Cannot open directory '%s'\n", dirName);
return 0;
}
do {
pathLength = snprintf(path, MAX_PATH, "%s\\%s", dirName, cFile.cFileName);
if (pathLength < 0 || pathLength >= MAX_PATH) {
fprintf(stderr, "Path length has got too long.\n");
continue;
}
if (cFile.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY) {
if (strcmp (cFile.cFileName, "..") == 0 ||
strcmp (cFile.cFileName, ".") == 0) continue;
// printf ("[%s]\n", path);
nbFiles += UTIL_prepareFileList(path, bufStart, pos, bufEnd); /* Recursively call "UTIL_prepareFileList" with the new path. */
if (*bufStart == NULL) { FindClose(hFile); return 0; }
}
else if ((cFile.dwFileAttributes & FILE_ATTRIBUTE_NORMAL) || (cFile.dwFileAttributes & FILE_ATTRIBUTE_ARCHIVE) || (cFile.dwFileAttributes & FILE_ATTRIBUTE_COMPRESSED)) {
if (*bufStart + *pos + pathLength >= *bufEnd) {
ptrdiff_t newListSize = (*bufEnd - *bufStart) + LIST_SIZE_INCREASE;
*bufStart = (char*)realloc(*bufStart, newListSize);
*bufEnd = *bufStart + newListSize;
if (*bufStart == NULL) { FindClose(hFile); return 0; }
}
if (*bufStart + *pos + pathLength < *bufEnd) {
strncpy(*bufStart + *pos, path, *bufEnd - (*bufStart + *pos));
*pos += pathLength + 1;
nbFiles++;
}
// printf ("%s\\%s nbFiles=%d left=%d\n", dirName, cFile.cFileName, nbFiles, (int)(bufEnd - *bufStart));
}
} while (FindNextFile(hFile, &cFile));
FindClose(hFile);
return nbFiles;
}
#elif (defined(__unix__) || defined(__unix) || defined(__midipix__) || (defined(__APPLE__) && defined(__MACH__))) && defined(_POSIX_C_SOURCE) && (_POSIX_C_SOURCE >= 200112L) /* snprintf, opendir */
# define UTIL_HAS_CREATEFILELIST
# include <dirent.h> /* opendir, readdir */
# include <limits.h> /* PATH_MAX */
# include <errno.h>
UTIL_STATIC int UTIL_prepareFileList(const char *dirName, char** bufStart, size_t* pos, char** bufEnd)
{
DIR *dir;
struct dirent *entry;
char path[PATH_MAX];
int pathLength, nbFiles = 0;
if (!(dir = opendir(dirName))) {
fprintf(stderr, "Cannot open directory '%s': %s\n", dirName, strerror(errno));
return 0;
}
while ((entry = readdir(dir)) != NULL) {
if (strcmp (entry->d_name, "..") == 0 ||
strcmp (entry->d_name, ".") == 0) continue;
pathLength = snprintf(path, PATH_MAX, "%s/%s", dirName, entry->d_name);
if (pathLength < 0 || pathLength >= PATH_MAX) {
fprintf(stderr, "Path length has got too long.\n");
continue;
}
if (UTIL_isDirectory(path)) {
// printf ("[%s]\n", path);
nbFiles += UTIL_prepareFileList(path, bufStart, pos, bufEnd); /* Recursively call "UTIL_prepareFileList" with the new path. */
if (*bufStart == NULL) { closedir(dir); return 0; }
} else {
if (*bufStart + *pos + pathLength >= *bufEnd) {
ptrdiff_t newListSize = (*bufEnd - *bufStart) + LIST_SIZE_INCREASE;
*bufStart = (char*)realloc(*bufStart, newListSize);
*bufEnd = *bufStart + newListSize;
if (*bufStart == NULL) { closedir(dir); return 0; }
}
if (*bufStart + *pos + pathLength < *bufEnd) {
strncpy(*bufStart + *pos, path, *bufEnd - (*bufStart + *pos));
*pos += pathLength + 1;
nbFiles++;
}
// printf ("%s/%s nbFiles=%d left=%d\n", dirName, entry->d_name, nbFiles, (int)(bufEnd - *bufStart));
}
}
closedir(dir);
return nbFiles;
}
#else
UTIL_STATIC int UTIL_prepareFileList(const char *dirName, char** bufStart, size_t* pos, char** bufEnd)
{
(void)bufStart; (void)bufEnd; (void)pos;
fprintf(stderr, "Directory %s ignored (zstd compiled without _WIN32 or _POSIX_C_SOURCE)\n", dirName);
return 0;
}
#endif // #ifdef _WIN32
/*
* UTIL_createFileList - takes a list of files and directories (params: inputNames, inputNamesNb), scans directories,
* and returns a new list of files (params: return value, allocatedBuffer, allocatedNamesNb).
* After finishing usage of the list the structures should be freed with UTIL_freeFileList(params: return value, allocatedBuffer)
* In case of error UTIL_createFileList returns NULL and UTIL_freeFileList should not be called.
*/
UTIL_STATIC const char** UTIL_createFileList(const char **inputNames, unsigned inputNamesNb, char** allocatedBuffer, unsigned* allocatedNamesNb)
{
size_t pos;
unsigned i, nbFiles;
char *bufend, *buf;
const char** fileTable;
buf = (char*)malloc(LIST_SIZE_INCREASE);
if (!buf) return NULL;
bufend = buf + LIST_SIZE_INCREASE;
for (i=0, pos=0, nbFiles=0; i<inputNamesNb; i++) {
if (UTIL_doesFileExists(inputNames[i])) {
// printf ("UTIL_doesFileExists=[%s]\n", inputNames[i]);
size_t len = strlen(inputNames[i]);
if (buf + pos + len >= bufend) {
ptrdiff_t newListSize = (bufend - buf) + LIST_SIZE_INCREASE;
buf = (char*)realloc(buf, newListSize);
bufend = buf + newListSize;
if (!buf) return NULL;
}
if (buf + pos + len < bufend) {
strncpy(buf + pos, inputNames[i], bufend - (buf + pos));
pos += len + 1;
nbFiles++;
}
}
else
{
nbFiles += UTIL_prepareFileList(inputNames[i], &buf, &pos, &bufend);
if (buf == NULL) return NULL;
}
}
if (nbFiles == 0) { free(buf); return NULL; }
fileTable = (const char**)malloc((nbFiles+1) * sizeof(const char*));
if (!fileTable) { free(buf); return NULL; }
for (i=0, pos=0; i<nbFiles; i++)
{
fileTable[i] = buf + pos;
pos += strlen(fileTable[i]) + 1;
}
if (buf + pos > bufend) { free(buf); free((void*)fileTable); return NULL; }
*allocatedBuffer = buf;
*allocatedNamesNb = nbFiles;
return fileTable;
}
UTIL_STATIC void UTIL_freeFileList(const char** filenameTable, char* allocatedBuffer)
{
if (allocatedBuffer) free(allocatedBuffer);
if (filenameTable) free((void*)filenameTable);
}
#if defined (__cplusplus)
}
#endif
#endif /* UTIL_H_MODULE */
+1 -1
View File
@@ -123,7 +123,7 @@ static void* XXH_memcpy(void* dest, const void* src, size_t size) { return memcp
***************************************/
#ifndef MEM_MODULE
# define MEM_MODULE
# if defined (__STDC_VERSION__) && __STDC_VERSION__ >= 199901L /* C99 */
# if !defined (__VMS) && ( defined (__STDC_VERSION__) && __STDC_VERSION__ >= 199901L /* C99 */ )
# include <stdint.h>
typedef uint8_t BYTE;
typedef uint16_t U16;
+27
View File
@@ -188,6 +188,33 @@ static int basicUnitTests(U32 seed, double compressibility)
DISPLAYLEVEL(4, "OK \n");
}
/* Byte-by-byte decompression test */
DISPLAYLEVEL(4, "test%3i : decompress byte-by-byte : ", testNb++);
ZBUFF_decompressInitDictionary(zd, CNBuffer, 128 KB);
{ size_t r = 1, pIn=0, pOut=0;
while (r) {
size_t inS = 1;
size_t outS = 1;
r = ZBUFF_decompressContinue(zd, ((BYTE*)decodedBuffer)+pOut, &outS, ((BYTE*)compressedBuffer)+pIn, &inS);
pIn += inS;
pOut += outS;
}
readSize = pIn;
genSize = pOut;
}
if (genSize != CNBufferSize) goto _output_error; /* should regenerate the same amount */
if (readSize != cSize) goto _output_error; /* should have read the entire frame */
DISPLAYLEVEL(4, "OK \n");
/* check regenerated data is byte exact */
{ size_t i;
DISPLAYLEVEL(4, "test%3i : check decompressed result : ", testNb++);
for (i=0; i<CNBufferSize; i++) {
if (((BYTE*)decodedBuffer)[i] != ((BYTE*)CNBuffer)[i]) goto _output_error;;
}
DISPLAYLEVEL(4, "OK \n");
}
_end:
ZBUFF_freeCCtx(zc);
ZBUFF_freeDCtx(zd);
+55 -41
View File
@@ -28,29 +28,23 @@
*/
/*-************************************
* Compiler Options
**************************************/
#define _CRT_SECURE_NO_WARNINGS /* Visual : removes warning from strcpy */
#define _POSIX_SOURCE 1 /* triggers fileno() within <stdio.h> on unix */
/*-************************************
* Includes
**************************************/
#include <stdio.h> /* fprintf, getchar */
#include <stdlib.h> /* exit, calloc, free */
#include "util.h" /* Compiler options, UTIL_HAS_CREATEFILELIST */
#include <string.h> /* strcmp, strlen */
#include <ctype.h> /* toupper */
#include "fileio.h"
#ifndef ZSTD_NOBENCH
# include "bench.h" /* BMK_benchFiles, BMK_SetNbIterations */
#endif
#include "zstd_static.h" /* ZSTD_maxCLevel, ZSTD version numbers */
#ifndef ZSTD_NODICT
# include "dibio.h" /* BMK_benchFiles, BMK_SetNbIterations */
# include "dibio.h"
#endif
/*-************************************
* OS-specific Includes
**************************************/
@@ -60,6 +54,7 @@
# define SET_BINARY_MODE(file) _setmode(_fileno(file), _O_BINARY)
# define IS_CONSOLE(stdStream) _isatty(_fileno(stdStream))
#else
extern int fileno(FILE *stream); /* triggers fileno() within <stdio.h> on POSIX */
# include <unistd.h> /* isatty */
# define SET_BINARY_MODE(file)
# define IS_CONSOLE(stdStream) isatty(fileno(stdStream))
@@ -71,13 +66,10 @@
**************************************/
#define COMPRESSOR_NAME "zstd command line interface"
#ifndef ZSTD_VERSION
# define LIB_VERSION ZSTD_VERSION_MAJOR.ZSTD_VERSION_MINOR.ZSTD_VERSION_RELEASE
# define QUOTE(str) #str
# define EXPAND_AND_QUOTE(str) QUOTE(str)
# define ZSTD_VERSION "v" EXPAND_AND_QUOTE(LIB_VERSION)
# define ZSTD_VERSION "v" ZSTD_VERSION_STRING
#endif
#define AUTHOR "Yann Collet"
#define WELCOME_MESSAGE "*** %s %i-bits %s, by %s ***\n", COMPRESSOR_NAME, (int)(sizeof(void*)*8), ZSTD_VERSION, AUTHOR
#define WELCOME_MESSAGE "*** %s %i-bits %s, by %s ***\n", COMPRESSOR_NAME, (int)(sizeof(size_t)*8), ZSTD_VERSION, AUTHOR
#define ZSTD_EXTENSION ".zst"
#define ZSTD_CAT "zstdcat"
@@ -87,7 +79,7 @@
#define MB *(1 <<20)
#define GB *(1U<<30)
static const char* g_defaultDictName = "dictionary";
static const char* g_defaultDictName = "dictionary";
static const unsigned g_defaultMaxDictSize = 110 KB;
static const unsigned g_defaultDictCLevel = 5;
static const unsigned g_defaultSelectivityLevel = 9;
@@ -113,8 +105,12 @@ static int usage(const char* programName)
DISPLAY( "FILE : a filename\n");
DISPLAY( " with no FILE, or when FILE is - , read standard input\n");
DISPLAY( "Arguments :\n");
#ifndef ZSTD_NOCOMPRESS
DISPLAY( " -# : # compression level (1-%u, default:1) \n", ZSTD_maxCLevel());
#endif
#ifndef ZSTD_NODECOMPRESS
DISPLAY( " -d : decompression \n");
#endif
DISPLAY( " -D file: use `file` as Dictionary \n");
DISPLAY( " -o file: result stored into `file` (only if 1 input file) \n");
DISPLAY( " -f : overwrite output without prompting \n");
@@ -133,7 +129,9 @@ static int usage_advanced(const char* programName)
DISPLAY( " -v : verbose mode\n");
DISPLAY( " -q : suppress warnings; specify twice to suppress errors too\n");
DISPLAY( " -c : force write to standard output, even if it is the console\n");
#ifndef ZSTD_NOCOMPRESS
DISPLAY( "--ultra : enable ultra modes (requires more memory to decompress)\n");
#endif
#ifndef ZSTD_NODICT
DISPLAY( "Dictionary builder :\n");
DISPLAY( "--train : create a dictionary from a training set of files \n");
@@ -144,8 +142,11 @@ static int usage_advanced(const char* programName)
#ifndef ZSTD_NOBENCH
DISPLAY( "Benchmark arguments :\n");
DISPLAY( " -b# : benchmark file(s), using # compression level (default : 1) \n");
DISPLAY( " -r# : test all compression levels from -bX to # (default: 1)\n");
DISPLAY( " -e# : test all compression levels from -bX to # (default: 1)\n");
DISPLAY( " -i# : iteration loops [1-9](default : 3)\n");
#ifdef UTIL_HAS_CREATEFILELIST
DISPLAY( " -r : operate recursively on directories\n");
#endif
DISPLAY( " -B# : cut file into independent blocks of size # (default: no block)\n");
#endif
return 0;
@@ -184,6 +185,7 @@ int main(int argCount, const char** argv)
nextArgumentIsMaxDict=0;
unsigned cLevel = 1;
unsigned cLevelLast = 1;
unsigned recursive = 0;
const char** filenameTable = (const char**)malloc(argCount * sizeof(const char*)); /* argCount >= 1 */
unsigned filenameIdx = 0;
const char* programName = argv[0];
@@ -195,7 +197,8 @@ int main(int argCount, const char** argv)
unsigned dictSelect = g_defaultSelectivityLevel;
/* init */
(void)cLevelLast; (void)dictCLevel; /* not used when ZSTD_NOBENCH / ZSTD_NODICT set */
(void)recursive; (void)cLevelLast; (void)dictCLevel; /* not used when ZSTD_NOBENCH / ZSTD_NODICT set */
(void)decode; (void)cLevel; /* not used when ZSTD_NOCOMPRESS set */
if (filenameTable==NULL) { DISPLAY("not enough memory\n"); exit(1); }
displayOut = stderr;
/* Pick out program name from path. Don't rely on stdlib because of conflicting behavior */
@@ -235,6 +238,7 @@ int main(int argCount, const char** argv)
argument++;
while (argument[0]!=0) {
#ifndef ZSTD_NOCOMPRESS
/* compression Level */
if ((*argument>='0') && (*argument<='9')) {
cLevel = 0;
@@ -248,6 +252,7 @@ int main(int argCount, const char** argv)
CLEAN_RETURN(badusage(programName));
continue;
}
#endif
switch(argument[0])
{
@@ -287,6 +292,17 @@ int main(int argCount, const char** argv)
/* Benchmark */
case 'b': bench=1; argument++; break;
/* range bench (benchmark only) */
case 'e':
/* compression Level */
argument++;
if ((*argument>='0') && (*argument<='9')) {
cLevelLast = 0;
while ((*argument >= '0') && (*argument <= '9'))
cLevelLast *= 10, cLevelLast += *argument++ - '0';
}
break;
/* Modify Nb Iterations (benchmark only) */
case 'i':
{ U32 iters= 0;
@@ -298,30 +314,22 @@ int main(int argCount, const char** argv)
}
break;
/* recursive */
case 'r': recursive=1; argument++; break;
/* cut input into blocks (benchmark only) */
case 'B':
{ size_t bSize = 0;
argument++;
while ((*argument >='0') && (*argument <='9'))
bSize *= 10, bSize += *argument++ - '0';
if (*argument=='K') bSize<<=10, argument++; /* allows using KB notation */
if (*argument=='M') bSize<<=20, argument++;
if (*argument=='B') argument++;
if (toupper(*argument)=='K') bSize<<=10, argument++; /* allows using KB notation */
if (toupper(*argument)=='M') bSize<<=20, argument++;
if (toupper(*argument)=='B') argument++;
BMK_setNotificationLevel(displayLevel);
BMK_SetBlockSize(bSize);
}
break;
/* range bench (benchmark only) */
case 'r':
/* compression Level */
argument++;
if ((*argument>='0') && (*argument<='9')) {
cLevelLast = 0;
while ((*argument >= '0') && (*argument <= '9'))
cLevelLast *= 10, cLevelLast += *argument++ - '0';
}
break;
#endif /* ZSTD_NOBENCH */
/* Selection level */
@@ -368,8 +376,7 @@ int main(int argCount, const char** argv)
maxDictSize = 0;
while ((*argument>='0') && (*argument<='9'))
maxDictSize = maxDictSize * 10 + (*argument - '0'), argument++;
if (*argument=='k' || *argument=='K')
maxDictSize <<= 10;
if (toupper(*argument)=='K') maxDictSize <<= 10;
continue;
}
@@ -384,7 +391,7 @@ int main(int argCount, const char** argv)
if (bench) {
#ifndef ZSTD_NOBENCH
BMK_setNotificationLevel(displayLevel);
BMK_benchFiles(filenameTable, filenameIdx, dictFileName, cLevel, cLevelLast);
BMK_benchFiles(filenameTable, filenameIdx, dictFileName, cLevel, cLevelLast, recursive);
#endif
goto _end;
}
@@ -420,16 +427,23 @@ int main(int argCount, const char** argv)
/* IO Stream/File */
FIO_setNotificationLevel(displayLevel);
if (decode) {
if (filenameIdx==1 && outFileName)
operationResult = FIO_decompressFilename(outFileName, filenameTable[0], dictFileName);
else
operationResult = FIO_decompressMultipleFilenames(filenameTable, filenameIdx, outFileName ? outFileName : ZSTD_EXTENSION, dictFileName);
} else { /* compression */
#ifndef ZSTD_NOCOMPRESS
if (!decode) {
if (filenameIdx==1 && outFileName)
operationResult = FIO_compressFilename(outFileName, filenameTable[0], dictFileName, cLevel);
else
operationResult = FIO_compressMultipleFilenames(filenameTable, filenameIdx, outFileName ? outFileName : ZSTD_EXTENSION, dictFileName, cLevel);
} else
#endif
{ /* decompression */
#ifndef ZSTD_NODECOMPRESS
if (filenameIdx==1 && outFileName)
operationResult = FIO_decompressFilename(outFileName, filenameTable[0], dictFileName);
else
operationResult = FIO_decompressMultipleFilenames(filenameTable, filenameIdx, outFileName ? outFileName : ZSTD_EXTENSION, dictFileName);
#else
DISPLAY("Decompression not supported\n");
#endif
}
_end:
+189 -188
View File
@@ -1,189 +1,190 @@
<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Build" ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup Label="ProjectConfigurations">
<ProjectConfiguration Include="Debug|Win32">
<Configuration>Debug</Configuration>
<Platform>Win32</Platform>
</ProjectConfiguration>
<ProjectConfiguration Include="Debug|x64">
<Configuration>Debug</Configuration>
<Platform>x64</Platform>
</ProjectConfiguration>
<ProjectConfiguration Include="Release|Win32">
<Configuration>Release</Configuration>
<Platform>Win32</Platform>
</ProjectConfiguration>
<ProjectConfiguration Include="Release|x64">
<Configuration>Release</Configuration>
<Platform>x64</Platform>
</ProjectConfiguration>
</ItemGroup>
<PropertyGroup Label="Globals">
<ProjectGuid>{61ABD629-1CC8-4FD7-9281-6B8DBB9D3DF8}</ProjectGuid>
<Keyword>Win32Proj</Keyword>
<RootNamespace>fullbench</RootNamespace>
<OutDir>$(SolutionDir)bin\$(Platform)\$(Configuration)\</OutDir>
</PropertyGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.Default.props" />
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'" Label="Configuration">
<ConfigurationType>Application</ConfigurationType>
<UseDebugLibraries>true</UseDebugLibraries>
<PlatformToolset>v120</PlatformToolset>
<CharacterSet>Unicode</CharacterSet>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'" Label="Configuration">
<ConfigurationType>Application</ConfigurationType>
<UseDebugLibraries>true</UseDebugLibraries>
<PlatformToolset>v120</PlatformToolset>
<CharacterSet>Unicode</CharacterSet>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'" Label="Configuration">
<ConfigurationType>Application</ConfigurationType>
<UseDebugLibraries>false</UseDebugLibraries>
<PlatformToolset>v120</PlatformToolset>
<WholeProgramOptimization>true</WholeProgramOptimization>
<CharacterSet>Unicode</CharacterSet>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'" Label="Configuration">
<ConfigurationType>Application</ConfigurationType>
<UseDebugLibraries>false</UseDebugLibraries>
<PlatformToolset>v120</PlatformToolset>
<WholeProgramOptimization>true</WholeProgramOptimization>
<CharacterSet>Unicode</CharacterSet>
</PropertyGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.props" />
<ImportGroup Label="ExtensionSettings">
</ImportGroup>
<ImportGroup Label="PropertySheets" Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
</ImportGroup>
<ImportGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'" Label="PropertySheets">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
</ImportGroup>
<ImportGroup Label="PropertySheets" Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
</ImportGroup>
<ImportGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'" Label="PropertySheets">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
</ImportGroup>
<PropertyGroup Label="UserMacros" />
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
<LinkIncremental>true</LinkIncremental>
<IncludePath>$(SolutionDir)..\..\lib;$(SolutionDir)..\..\lib\legacy;$(VCInstallDir)include;$(VCInstallDir)atlmfc\include;$(WindowsSDK_IncludePath);</IncludePath>
<RunCodeAnalysis>true</RunCodeAnalysis>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">
<LinkIncremental>true</LinkIncremental>
<IncludePath>$(SolutionDir)..\..\lib\legacy;$(SolutionDir)..\..\lib;$(VCInstallDir)include;$(VCInstallDir)atlmfc\include;$(WindowsSDK_IncludePath);</IncludePath>
<RunCodeAnalysis>true</RunCodeAnalysis>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
<LinkIncremental>false</LinkIncremental>
<IncludePath>$(SolutionDir)..\..\lib\legacy;$(SolutionDir)..\..\lib;$(VCInstallDir)include;$(VCInstallDir)atlmfc\include;$(WindowsSDK_IncludePath);</IncludePath>
<RunCodeAnalysis>true</RunCodeAnalysis>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'">
<LinkIncremental>false</LinkIncremental>
<IncludePath>$(SolutionDir)..\..\lib\legacy;$(SolutionDir)..\..\lib;$(VCInstallDir)include;$(VCInstallDir)atlmfc\include;$(WindowsSDK_IncludePath);</IncludePath>
<RunCodeAnalysis>true</RunCodeAnalysis>
</PropertyGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
<ClCompile>
<PrecompiledHeader>
</PrecompiledHeader>
<WarningLevel>Level4</WarningLevel>
<Optimization>Disabled</Optimization>
<PreprocessorDefinitions>WIN32;_DEBUG;_CONSOLE;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<TreatWarningAsError>true</TreatWarningAsError>
<EnablePREfast>true</EnablePREfast>
<AdditionalOptions>/analyze:stacksize25000 %(AdditionalOptions)</AdditionalOptions>
</ClCompile>
<Link>
<SubSystem>Console</SubSystem>
<GenerateDebugInformation>true</GenerateDebugInformation>
</Link>
</ItemDefinitionGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">
<ClCompile>
<PrecompiledHeader>
</PrecompiledHeader>
<WarningLevel>Level4</WarningLevel>
<Optimization>Disabled</Optimization>
<PreprocessorDefinitions>WIN32;_DEBUG;_CONSOLE;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<TreatWarningAsError>true</TreatWarningAsError>
<EnablePREfast>true</EnablePREfast>
<AdditionalOptions>/analyze:stacksize25000 %(AdditionalOptions)</AdditionalOptions>
</ClCompile>
<Link>
<SubSystem>Console</SubSystem>
<GenerateDebugInformation>true</GenerateDebugInformation>
</Link>
</ItemDefinitionGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
<ClCompile>
<WarningLevel>Level4</WarningLevel>
<PrecompiledHeader>
</PrecompiledHeader>
<Optimization>MaxSpeed</Optimization>
<FunctionLevelLinking>true</FunctionLevelLinking>
<IntrinsicFunctions>true</IntrinsicFunctions>
<PreprocessorDefinitions>WIN32;NDEBUG;_CONSOLE;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<EnablePREfast>true</EnablePREfast>
<TreatWarningAsError>true</TreatWarningAsError>
<AdditionalOptions>/analyze:stacksize25000 %(AdditionalOptions)</AdditionalOptions>
</ClCompile>
<Link>
<SubSystem>Console</SubSystem>
<GenerateDebugInformation>true</GenerateDebugInformation>
<EnableCOMDATFolding>true</EnableCOMDATFolding>
<OptimizeReferences>true</OptimizeReferences>
</Link>
</ItemDefinitionGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'">
<ClCompile>
<WarningLevel>Level4</WarningLevel>
<PrecompiledHeader>
</PrecompiledHeader>
<Optimization>MaxSpeed</Optimization>
<FunctionLevelLinking>true</FunctionLevelLinking>
<IntrinsicFunctions>true</IntrinsicFunctions>
<PreprocessorDefinitions>WIN32;NDEBUG;_CONSOLE;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<TreatWarningAsError>true</TreatWarningAsError>
<EnablePREfast>true</EnablePREfast>
<AdditionalOptions>/analyze:stacksize25000 %(AdditionalOptions)</AdditionalOptions>
</ClCompile>
<Link>
<SubSystem>Console</SubSystem>
<GenerateDebugInformation>true</GenerateDebugInformation>
<EnableCOMDATFolding>true</EnableCOMDATFolding>
<OptimizeReferences>true</OptimizeReferences>
</Link>
</ItemDefinitionGroup>
<ItemGroup>
<ClCompile Include="..\..\..\lib\fse.c" />
<ClCompile Include="..\..\..\lib\huff0.c" />
<ClCompile Include="..\..\..\lib\zbuff.c" />
<ClCompile Include="..\..\..\lib\zstd_compress.c" />
<ClCompile Include="..\..\..\lib\zstd_decompress.c" />
<ClCompile Include="..\..\..\programs\datagen.c" />
<ClCompile Include="..\..\..\programs\fullbench.c" />
</ItemGroup>
<ItemGroup>
<ClInclude Include="..\..\..\lib\fse.h" />
<ClInclude Include="..\..\..\lib\fse_static.h" />
<ClInclude Include="..\..\..\lib\huff0.h" />
<ClInclude Include="..\..\..\lib\huff0_static.h" />
<ClInclude Include="..\..\..\lib\legacy\zstd_legacy.h" />
<ClInclude Include="..\..\..\lib\legacy\zstd_v01.h" />
<ClInclude Include="..\..\..\lib\legacy\zstd_v02.h" />
<ClInclude Include="..\..\..\lib\legacy\zstd_v03.h" />
<ClInclude Include="..\..\..\lib\zbuff.h" />
<ClInclude Include="..\..\..\lib\zbuff_static.h" />
<ClInclude Include="..\..\..\lib\zstd.h" />
<ClInclude Include="..\..\..\lib\zstd_static.h" />
<ClInclude Include="..\..\..\programs\datagen.h" />
</ItemGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.targets" />
<ImportGroup Label="ExtensionTargets">
</ImportGroup>
<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Build" ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup Label="ProjectConfigurations">
<ProjectConfiguration Include="Debug|Win32">
<Configuration>Debug</Configuration>
<Platform>Win32</Platform>
</ProjectConfiguration>
<ProjectConfiguration Include="Debug|x64">
<Configuration>Debug</Configuration>
<Platform>x64</Platform>
</ProjectConfiguration>
<ProjectConfiguration Include="Release|Win32">
<Configuration>Release</Configuration>
<Platform>Win32</Platform>
</ProjectConfiguration>
<ProjectConfiguration Include="Release|x64">
<Configuration>Release</Configuration>
<Platform>x64</Platform>
</ProjectConfiguration>
</ItemGroup>
<PropertyGroup Label="Globals">
<ProjectGuid>{61ABD629-1CC8-4FD7-9281-6B8DBB9D3DF8}</ProjectGuid>
<Keyword>Win32Proj</Keyword>
<RootNamespace>fullbench</RootNamespace>
<OutDir>$(SolutionDir)bin\$(Platform)\$(Configuration)\</OutDir>
</PropertyGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.Default.props" />
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'" Label="Configuration">
<ConfigurationType>Application</ConfigurationType>
<UseDebugLibraries>true</UseDebugLibraries>
<CharacterSet>MultiByte</CharacterSet>
<PlatformToolset>v110</PlatformToolset>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'" Label="Configuration">
<ConfigurationType>Application</ConfigurationType>
<UseDebugLibraries>true</UseDebugLibraries>
<PlatformToolset>v110</PlatformToolset>
<CharacterSet>MultiByte</CharacterSet>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'" Label="Configuration">
<ConfigurationType>Application</ConfigurationType>
<UseDebugLibraries>false</UseDebugLibraries>
<PlatformToolset>v110</PlatformToolset>
<WholeProgramOptimization>true</WholeProgramOptimization>
<CharacterSet>MultiByte</CharacterSet>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'" Label="Configuration">
<ConfigurationType>Application</ConfigurationType>
<UseDebugLibraries>false</UseDebugLibraries>
<PlatformToolset>v110</PlatformToolset>
<WholeProgramOptimization>true</WholeProgramOptimization>
<CharacterSet>MultiByte</CharacterSet>
</PropertyGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.props" />
<ImportGroup Label="ExtensionSettings">
</ImportGroup>
<ImportGroup Label="PropertySheets" Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
</ImportGroup>
<ImportGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'" Label="PropertySheets">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
</ImportGroup>
<ImportGroup Label="PropertySheets" Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
</ImportGroup>
<ImportGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'" Label="PropertySheets">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
</ImportGroup>
<PropertyGroup Label="UserMacros" />
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
<LinkIncremental>true</LinkIncremental>
<IncludePath>$(WindowsSdkDir)\include;$(UniversalCRT_IncludePath);$(SolutionDir)..\..\lib\common;$(SolutionDir)..\..\lib\legacy;$(VCInstallDir)include;$(VCInstallDir)atlmfc\include;$(WindowsSDK_IncludePath)</IncludePath>
<RunCodeAnalysis>false</RunCodeAnalysis>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">
<LinkIncremental>true</LinkIncremental>
<IncludePath>$(WindowsSdkDir)\include;$(UniversalCRT_IncludePath);$(SolutionDir)..\..\lib\common;$(SolutionDir)..\..\lib\legacy;$(VCInstallDir)include;$(VCInstallDir)atlmfc\include;$(WindowsSDK_IncludePath)</IncludePath>
<RunCodeAnalysis>false</RunCodeAnalysis>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
<LinkIncremental>false</LinkIncremental>
<IncludePath>$(WindowsSdkDir)\include;$(UniversalCRT_IncludePath);$(SolutionDir)..\..\lib\common;$(SolutionDir)..\..\lib\legacy;$(VCInstallDir)include;$(VCInstallDir)atlmfc\include;$(WindowsSDK_IncludePath)</IncludePath>
<RunCodeAnalysis>false</RunCodeAnalysis>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'">
<LinkIncremental>false</LinkIncremental>
<IncludePath>$(WindowsSdkDir)\include;$(UniversalCRT_IncludePath);$(SolutionDir)..\..\lib\common;$(SolutionDir)..\..\lib\legacy;$(VCInstallDir)include;$(VCInstallDir)atlmfc\include;$(WindowsSDK_IncludePath)</IncludePath>
<RunCodeAnalysis>false</RunCodeAnalysis>
</PropertyGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
<ClCompile>
<PrecompiledHeader>
</PrecompiledHeader>
<WarningLevel>Level4</WarningLevel>
<Optimization>Disabled</Optimization>
<PreprocessorDefinitions>WIN32;_DEBUG;_CONSOLE;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<TreatWarningAsError>true</TreatWarningAsError>
<EnablePREfast>false</EnablePREfast>
</ClCompile>
<Link>
<SubSystem>Console</SubSystem>
<GenerateDebugInformation>true</GenerateDebugInformation>
</Link>
</ItemDefinitionGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">
<ClCompile>
<PrecompiledHeader>
</PrecompiledHeader>
<WarningLevel>Level4</WarningLevel>
<Optimization>Disabled</Optimization>
<PreprocessorDefinitions>WIN32;_DEBUG;_CONSOLE;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<TreatWarningAsError>true</TreatWarningAsError>
<EnablePREfast>false</EnablePREfast>
</ClCompile>
<Link>
<SubSystem>Console</SubSystem>
<GenerateDebugInformation>true</GenerateDebugInformation>
</Link>
</ItemDefinitionGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
<ClCompile>
<WarningLevel>Level4</WarningLevel>
<PrecompiledHeader>
</PrecompiledHeader>
<Optimization>MaxSpeed</Optimization>
<FunctionLevelLinking>true</FunctionLevelLinking>
<IntrinsicFunctions>true</IntrinsicFunctions>
<PreprocessorDefinitions>WIN32;NDEBUG;_CONSOLE;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<EnablePREfast>false</EnablePREfast>
<TreatWarningAsError>true</TreatWarningAsError>
</ClCompile>
<Link>
<SubSystem>Console</SubSystem>
<GenerateDebugInformation>true</GenerateDebugInformation>
<EnableCOMDATFolding>true</EnableCOMDATFolding>
<OptimizeReferences>true</OptimizeReferences>
</Link>
</ItemDefinitionGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'">
<ClCompile>
<WarningLevel>Level4</WarningLevel>
<PrecompiledHeader>
</PrecompiledHeader>
<Optimization>MaxSpeed</Optimization>
<FunctionLevelLinking>true</FunctionLevelLinking>
<IntrinsicFunctions>true</IntrinsicFunctions>
<PreprocessorDefinitions>WIN32;NDEBUG;_CONSOLE;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<TreatWarningAsError>true</TreatWarningAsError>
<EnablePREfast>false</EnablePREfast>
</ClCompile>
<Link>
<SubSystem>Console</SubSystem>
<GenerateDebugInformation>true</GenerateDebugInformation>
<EnableCOMDATFolding>true</EnableCOMDATFolding>
<OptimizeReferences>true</OptimizeReferences>
</Link>
</ItemDefinitionGroup>
<ItemGroup>
<ClCompile Include="..\..\..\lib\common\entropy_common.c" />
<ClCompile Include="..\..\..\lib\common\zstd_common.c" />
<ClCompile Include="..\..\..\lib\common\fse_decompress.c" />
<ClCompile Include="..\..\..\lib\compress\fse_compress.c" />
<ClCompile Include="..\..\..\lib\compress\huf_compress.c" />
<ClCompile Include="..\..\..\lib\compress\zbuff_compress.c" />
<ClCompile Include="..\..\..\lib\compress\zstd_compress.c" />
<ClCompile Include="..\..\..\lib\decompress\huf_decompress.c" />
<ClCompile Include="..\..\..\lib\decompress\zbuff_decompress.c" />
<ClCompile Include="..\..\..\lib\decompress\zstd_decompress.c" />
<ClCompile Include="..\..\..\programs\datagen.c" />
<ClCompile Include="..\..\..\programs\fullbench.c" />
</ItemGroup>
<ItemGroup>
<ClInclude Include="..\..\..\lib\common\fse.h" />
<ClInclude Include="..\..\..\lib\common\fse_static.h" />
<ClInclude Include="..\..\..\lib\common\huf.h" />
<ClInclude Include="..\..\..\lib\common\huf_static.h" />
<ClInclude Include="..\..\..\lib\common\zbuff.h" />
<ClInclude Include="..\..\..\lib\common\zbuff_static.h" />
<ClInclude Include="..\..\..\lib\common\zstd.h" />
<ClInclude Include="..\..\..\lib\common\zstd_internal.h" />
<ClInclude Include="..\..\..\lib\common\zstd_static.h" />
<ClInclude Include="..\..\..\lib\compress\zstd_opt.h" />
<ClInclude Include="..\..\..\lib\legacy\zstd_legacy.h" />
<ClInclude Include="..\..\..\programs\datagen.h" />
<ClInclude Include="..\..\..\programs\util.h" />
</ItemGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.targets" />
<ImportGroup Label="ExtensionTargets">
</ImportGroup>
</Project>
+91 -80
View File
@@ -1,81 +1,92 @@
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup>
<Filter Include="Fichiers sources">
<UniqueIdentifier>{4FC737F1-C7A5-4376-A066-2A32D752A2FF}</UniqueIdentifier>
<Extensions>cpp;c;cc;cxx;def;odl;idl;hpj;bat;asm;asmx</Extensions>
</Filter>
<Filter Include="Fichiers d%27en-tête">
<UniqueIdentifier>{93995380-89BD-4b04-88EB-625FBE52EBFB}</UniqueIdentifier>
<Extensions>h;hpp;hxx;hm;inl;inc;xsd</Extensions>
</Filter>
<Filter Include="Fichiers de ressources">
<UniqueIdentifier>{67DA6AB6-F800-4c08-8B7A-83BB121AAD01}</UniqueIdentifier>
<Extensions>rc;ico;cur;bmp;dlg;rc2;rct;bin;rgs;gif;jpg;jpeg;jpe;resx;tiff;tif;png;wav;mfcribbon-ms</Extensions>
</Filter>
</ItemGroup>
<ItemGroup>
<ClCompile Include="..\..\..\lib\fse.c">
<Filter>Fichiers sources</Filter>
</ClCompile>
<ClCompile Include="..\..\..\programs\fullbench.c">
<Filter>Fichiers sources</Filter>
</ClCompile>
<ClCompile Include="..\..\..\programs\datagen.c">
<Filter>Fichiers sources</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\huff0.c">
<Filter>Fichiers sources</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\zstd_compress.c">
<Filter>Fichiers sources</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\zstd_decompress.c">
<Filter>Fichiers sources</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\zbuff.c">
<Filter>Fichiers sources</Filter>
</ClCompile>
</ItemGroup>
<ItemGroup>
<ClInclude Include="..\..\..\lib\fse.h">
<Filter>Fichiers d%27en-tête</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\fse_static.h">
<Filter>Fichiers d%27en-tête</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\zstd.h">
<Filter>Fichiers d%27en-tête</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\zstd_static.h">
<Filter>Fichiers d%27en-tête</Filter>
</ClInclude>
<ClInclude Include="..\..\..\programs\datagen.h">
<Filter>Fichiers d%27en-tête</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\legacy\zstd_v01.h">
<Filter>Fichiers d%27en-tête</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\huff0.h">
<Filter>Fichiers d%27en-tête</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\huff0_static.h">
<Filter>Fichiers d%27en-tête</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\legacy\zstd_legacy.h">
<Filter>Fichiers d%27en-tête</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\legacy\zstd_v02.h">
<Filter>Fichiers d%27en-tête</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\legacy\zstd_v03.h">
<Filter>Fichiers d%27en-tête</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\zbuff.h">
<Filter>Fichiers d%27en-tête</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\zbuff_static.h">
<Filter>Fichiers d%27en-tête</Filter>
</ClInclude>
</ItemGroup>
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup>
<Filter Include="Header Files">
<UniqueIdentifier>{93995380-89BD-4b04-88EB-625FBE52EBFB}</UniqueIdentifier>
<Extensions>h;hpp;hxx;hm;inl;inc;xsd</Extensions>
</Filter>
<Filter Include="Source Files">
<UniqueIdentifier>{4FC737F1-C7A5-4376-A066-2A32D752A2FF}</UniqueIdentifier>
<Extensions>cpp;c;cc;cxx;def;odl;idl;hpj;bat;asm;asmx</Extensions>
</Filter>
</ItemGroup>
<ItemGroup>
<ClCompile Include="..\..\..\lib\common\zstd_common.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\compress\fse_compress.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\programs\fullbench.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\programs\datagen.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\compress\huf_compress.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\decompress\huf_decompress.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\compress\zstd_compress.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\decompress\zstd_decompress.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\compress\zbuff_compress.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\decompress\zbuff_decompress.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\common\fse_decompress.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\common\entropy_common.c">
<Filter>Source Files</Filter>
</ClCompile>
</ItemGroup>
<ItemGroup>
<ClInclude Include="..\..\..\lib\common\fse.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\common\fse_static.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\common\zstd.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\common\zstd_static.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\programs\datagen.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\common\huf.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\common\huf_static.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\legacy\zstd_legacy.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\common\zbuff.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\common\zbuff_static.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\compress\zstd_opt.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\common\zstd_internal.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\programs\util.h">
<Filter>Header Files</Filter>
</ClInclude>
</ItemGroup>
</Project>
+189 -191
View File
@@ -1,192 +1,190 @@
<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Build" ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup Label="ProjectConfigurations">
<ProjectConfiguration Include="Debug|Win32">
<Configuration>Debug</Configuration>
<Platform>Win32</Platform>
</ProjectConfiguration>
<ProjectConfiguration Include="Debug|x64">
<Configuration>Debug</Configuration>
<Platform>x64</Platform>
</ProjectConfiguration>
<ProjectConfiguration Include="Release|Win32">
<Configuration>Release</Configuration>
<Platform>Win32</Platform>
</ProjectConfiguration>
<ProjectConfiguration Include="Release|x64">
<Configuration>Release</Configuration>
<Platform>x64</Platform>
</ProjectConfiguration>
</ItemGroup>
<PropertyGroup Label="Globals">
<ProjectGuid>{6FD4352B-346C-4703-96EA-D4A8B9A6976E}</ProjectGuid>
<Keyword>Win32Proj</Keyword>
<RootNamespace>fuzzer</RootNamespace>
<OutDir>$(SolutionDir)bin\$(Platform)\$(Configuration)\</OutDir>
</PropertyGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.Default.props" />
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'" Label="Configuration">
<ConfigurationType>Application</ConfigurationType>
<UseDebugLibraries>true</UseDebugLibraries>
<PlatformToolset>v120</PlatformToolset>
<CharacterSet>Unicode</CharacterSet>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'" Label="Configuration">
<ConfigurationType>Application</ConfigurationType>
<UseDebugLibraries>true</UseDebugLibraries>
<PlatformToolset>v120</PlatformToolset>
<CharacterSet>Unicode</CharacterSet>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'" Label="Configuration">
<ConfigurationType>Application</ConfigurationType>
<UseDebugLibraries>false</UseDebugLibraries>
<PlatformToolset>v120</PlatformToolset>
<WholeProgramOptimization>true</WholeProgramOptimization>
<CharacterSet>Unicode</CharacterSet>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'" Label="Configuration">
<ConfigurationType>Application</ConfigurationType>
<UseDebugLibraries>false</UseDebugLibraries>
<PlatformToolset>v120</PlatformToolset>
<WholeProgramOptimization>true</WholeProgramOptimization>
<CharacterSet>Unicode</CharacterSet>
</PropertyGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.props" />
<ImportGroup Label="ExtensionSettings">
</ImportGroup>
<ImportGroup Label="PropertySheets" Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
</ImportGroup>
<ImportGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'" Label="PropertySheets">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
</ImportGroup>
<ImportGroup Label="PropertySheets" Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
</ImportGroup>
<ImportGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'" Label="PropertySheets">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
</ImportGroup>
<PropertyGroup Label="UserMacros" />
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
<LinkIncremental>true</LinkIncremental>
<IncludePath>$(SolutionDir)..\..\lib;$(SolutionDir)..\..\lib\legacy;$(VCInstallDir)include;$(VCInstallDir)atlmfc\include;$(WindowsSDK_IncludePath);</IncludePath>
<RunCodeAnalysis>true</RunCodeAnalysis>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">
<LinkIncremental>true</LinkIncremental>
<IncludePath>$(SolutionDir)..\..\lib\legacy;$(SolutionDir)..\..\lib;$(VCInstallDir)include;$(VCInstallDir)atlmfc\include;$(WindowsSDK_IncludePath);</IncludePath>
<RunCodeAnalysis>true</RunCodeAnalysis>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
<LinkIncremental>false</LinkIncremental>
<IncludePath>$(SolutionDir)..\..\lib\legacy;$(SolutionDir)..\..\lib;$(VCInstallDir)include;$(VCInstallDir)atlmfc\include;$(WindowsSDK_IncludePath);</IncludePath>
<RunCodeAnalysis>true</RunCodeAnalysis>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'">
<LinkIncremental>false</LinkIncremental>
<IncludePath>$(SolutionDir)..\..\lib\legacy;$(SolutionDir)..\..\lib;$(VCInstallDir)include;$(VCInstallDir)atlmfc\include;$(WindowsSDK_IncludePath);</IncludePath>
<RunCodeAnalysis>true</RunCodeAnalysis>
</PropertyGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
<ClCompile>
<PrecompiledHeader>
</PrecompiledHeader>
<WarningLevel>Level4</WarningLevel>
<Optimization>Disabled</Optimization>
<PreprocessorDefinitions>WIN32;_DEBUG;_CONSOLE;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<TreatWarningAsError>true</TreatWarningAsError>
<EnablePREfast>true</EnablePREfast>
<AdditionalOptions>/analyze:stacksize25000 %(AdditionalOptions)</AdditionalOptions>
</ClCompile>
<Link>
<SubSystem>Console</SubSystem>
<GenerateDebugInformation>true</GenerateDebugInformation>
</Link>
</ItemDefinitionGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">
<ClCompile>
<PrecompiledHeader>
</PrecompiledHeader>
<WarningLevel>Level4</WarningLevel>
<Optimization>Disabled</Optimization>
<PreprocessorDefinitions>WIN32;_DEBUG;_CONSOLE;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<TreatWarningAsError>true</TreatWarningAsError>
<EnablePREfast>true</EnablePREfast>
<AdditionalOptions>/analyze:stacksize25000 %(AdditionalOptions)</AdditionalOptions>
</ClCompile>
<Link>
<SubSystem>Console</SubSystem>
<GenerateDebugInformation>true</GenerateDebugInformation>
</Link>
</ItemDefinitionGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
<ClCompile>
<WarningLevel>Level4</WarningLevel>
<PrecompiledHeader>
</PrecompiledHeader>
<Optimization>MaxSpeed</Optimization>
<FunctionLevelLinking>true</FunctionLevelLinking>
<IntrinsicFunctions>true</IntrinsicFunctions>
<PreprocessorDefinitions>WIN32;NDEBUG;_CONSOLE;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<EnablePREfast>true</EnablePREfast>
<TreatWarningAsError>true</TreatWarningAsError>
<AdditionalOptions>/analyze:stacksize25000 %(AdditionalOptions)</AdditionalOptions>
</ClCompile>
<Link>
<SubSystem>Console</SubSystem>
<GenerateDebugInformation>true</GenerateDebugInformation>
<EnableCOMDATFolding>true</EnableCOMDATFolding>
<OptimizeReferences>true</OptimizeReferences>
</Link>
</ItemDefinitionGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'">
<ClCompile>
<WarningLevel>Level4</WarningLevel>
<PrecompiledHeader>
</PrecompiledHeader>
<Optimization>MaxSpeed</Optimization>
<FunctionLevelLinking>true</FunctionLevelLinking>
<IntrinsicFunctions>true</IntrinsicFunctions>
<PreprocessorDefinitions>WIN32;NDEBUG;_CONSOLE;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<TreatWarningAsError>true</TreatWarningAsError>
<EnablePREfast>true</EnablePREfast>
<AdditionalOptions>/analyze:stacksize25000 %(AdditionalOptions)</AdditionalOptions>
</ClCompile>
<Link>
<SubSystem>Console</SubSystem>
<GenerateDebugInformation>true</GenerateDebugInformation>
<EnableCOMDATFolding>true</EnableCOMDATFolding>
<OptimizeReferences>true</OptimizeReferences>
</Link>
</ItemDefinitionGroup>
<ItemGroup>
<ClCompile Include="..\..\..\lib\fse.c" />
<ClCompile Include="..\..\..\lib\huff0.c" />
<ClCompile Include="..\..\..\lib\legacy\zstd_v01.c" />
<ClCompile Include="..\..\..\lib\legacy\zstd_v02.c" />
<ClCompile Include="..\..\..\lib\legacy\zstd_v03.c" />
<ClCompile Include="..\..\..\lib\zstd_compress.c" />
<ClCompile Include="..\..\..\lib\zstd_decompress.c" />
<ClCompile Include="..\..\..\programs\datagen.c" />
<ClCompile Include="..\..\..\programs\fuzzer.c" />
<ClCompile Include="..\..\..\programs\xxhash.c" />
</ItemGroup>
<ItemGroup>
<ClInclude Include="..\..\..\lib\fse.h" />
<ClInclude Include="..\..\..\lib\fse_static.h" />
<ClInclude Include="..\..\..\lib\huff0.h" />
<ClInclude Include="..\..\..\lib\huff0_static.h" />
<ClInclude Include="..\..\..\lib\legacy\zstd_legacy.h" />
<ClInclude Include="..\..\..\lib\legacy\zstd_v01.h" />
<ClInclude Include="..\..\..\lib\legacy\zstd_v02.h" />
<ClInclude Include="..\..\..\lib\legacy\zstd_v03.h" />
<ClInclude Include="..\..\..\lib\zstd.h" />
<ClInclude Include="..\..\..\lib\zstd_internal.h" />
<ClInclude Include="..\..\..\lib\zstd_static.h" />
<ClInclude Include="..\..\..\programs\datagen.h" />
<ClInclude Include="..\..\..\programs\xxhash.h" />
</ItemGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.targets" />
<ImportGroup Label="ExtensionTargets">
</ImportGroup>
<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Build" ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup Label="ProjectConfigurations">
<ProjectConfiguration Include="Debug|Win32">
<Configuration>Debug</Configuration>
<Platform>Win32</Platform>
</ProjectConfiguration>
<ProjectConfiguration Include="Debug|x64">
<Configuration>Debug</Configuration>
<Platform>x64</Platform>
</ProjectConfiguration>
<ProjectConfiguration Include="Release|Win32">
<Configuration>Release</Configuration>
<Platform>Win32</Platform>
</ProjectConfiguration>
<ProjectConfiguration Include="Release|x64">
<Configuration>Release</Configuration>
<Platform>x64</Platform>
</ProjectConfiguration>
</ItemGroup>
<PropertyGroup Label="Globals">
<ProjectGuid>{6FD4352B-346C-4703-96EA-D4A8B9A6976E}</ProjectGuid>
<Keyword>Win32Proj</Keyword>
<RootNamespace>fuzzer</RootNamespace>
<OutDir>$(SolutionDir)bin\$(Platform)\$(Configuration)\</OutDir>
</PropertyGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.Default.props" />
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'" Label="Configuration">
<ConfigurationType>Application</ConfigurationType>
<UseDebugLibraries>true</UseDebugLibraries>
<PlatformToolset>v110</PlatformToolset>
<CharacterSet>MultiByte</CharacterSet>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'" Label="Configuration">
<ConfigurationType>Application</ConfigurationType>
<UseDebugLibraries>true</UseDebugLibraries>
<PlatformToolset>v110</PlatformToolset>
<CharacterSet>MultiByte</CharacterSet>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'" Label="Configuration">
<ConfigurationType>Application</ConfigurationType>
<UseDebugLibraries>false</UseDebugLibraries>
<PlatformToolset>v110</PlatformToolset>
<WholeProgramOptimization>true</WholeProgramOptimization>
<CharacterSet>MultiByte</CharacterSet>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'" Label="Configuration">
<ConfigurationType>Application</ConfigurationType>
<UseDebugLibraries>false</UseDebugLibraries>
<PlatformToolset>v110</PlatformToolset>
<WholeProgramOptimization>true</WholeProgramOptimization>
<CharacterSet>MultiByte</CharacterSet>
</PropertyGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.props" />
<ImportGroup Label="ExtensionSettings">
</ImportGroup>
<ImportGroup Label="PropertySheets" Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
</ImportGroup>
<ImportGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'" Label="PropertySheets">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
</ImportGroup>
<ImportGroup Label="PropertySheets" Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
</ImportGroup>
<ImportGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'" Label="PropertySheets">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
</ImportGroup>
<PropertyGroup Label="UserMacros" />
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
<LinkIncremental>true</LinkIncremental>
<IncludePath>$(WindowsSdkDir)\include;$(UniversalCRT_IncludePath);$(SolutionDir)..\..\lib\common;$(SolutionDir)..\..\lib\legacy;$(VCInstallDir)include;$(VCInstallDir)atlmfc\include;$(WindowsSDK_IncludePath)</IncludePath>
<RunCodeAnalysis>false</RunCodeAnalysis>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">
<LinkIncremental>true</LinkIncremental>
<IncludePath>$(WindowsSdkDir)\include;$(UniversalCRT_IncludePath);$(SolutionDir)..\..\lib\common;$(SolutionDir)..\..\lib\legacy;$(VCInstallDir)include;$(VCInstallDir)atlmfc\include;$(WindowsSDK_IncludePath)</IncludePath>
<RunCodeAnalysis>false</RunCodeAnalysis>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
<LinkIncremental>false</LinkIncremental>
<IncludePath>$(WindowsSdkDir)\include;$(UniversalCRT_IncludePath);$(SolutionDir)..\..\lib\common;$(SolutionDir)..\..\lib\legacy;$(VCInstallDir)include;$(VCInstallDir)atlmfc\include;$(WindowsSDK_IncludePath)</IncludePath>
<RunCodeAnalysis>false</RunCodeAnalysis>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'">
<LinkIncremental>false</LinkIncremental>
<IncludePath>$(WindowsSdkDir)\include;$(UniversalCRT_IncludePath);$(SolutionDir)..\..\lib\common;$(SolutionDir)..\..\lib\legacy;$(VCInstallDir)include;$(VCInstallDir)atlmfc\include;$(WindowsSDK_IncludePath)</IncludePath>
<RunCodeAnalysis>false</RunCodeAnalysis>
</PropertyGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
<ClCompile>
<PrecompiledHeader>
</PrecompiledHeader>
<WarningLevel>Level4</WarningLevel>
<Optimization>Disabled</Optimization>
<PreprocessorDefinitions>WIN32;_DEBUG;_CONSOLE;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<TreatWarningAsError>true</TreatWarningAsError>
<EnablePREfast>false</EnablePREfast>
</ClCompile>
<Link>
<SubSystem>Console</SubSystem>
<GenerateDebugInformation>true</GenerateDebugInformation>
</Link>
</ItemDefinitionGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">
<ClCompile>
<PrecompiledHeader>
</PrecompiledHeader>
<WarningLevel>Level4</WarningLevel>
<Optimization>Disabled</Optimization>
<PreprocessorDefinitions>WIN32;_DEBUG;_CONSOLE;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<TreatWarningAsError>true</TreatWarningAsError>
<EnablePREfast>false</EnablePREfast>
</ClCompile>
<Link>
<SubSystem>Console</SubSystem>
<GenerateDebugInformation>true</GenerateDebugInformation>
</Link>
</ItemDefinitionGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
<ClCompile>
<WarningLevel>Level4</WarningLevel>
<PrecompiledHeader>
</PrecompiledHeader>
<Optimization>MaxSpeed</Optimization>
<FunctionLevelLinking>true</FunctionLevelLinking>
<IntrinsicFunctions>true</IntrinsicFunctions>
<PreprocessorDefinitions>WIN32;NDEBUG;_CONSOLE;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<EnablePREfast>false</EnablePREfast>
<TreatWarningAsError>true</TreatWarningAsError>
</ClCompile>
<Link>
<SubSystem>Console</SubSystem>
<GenerateDebugInformation>true</GenerateDebugInformation>
<EnableCOMDATFolding>true</EnableCOMDATFolding>
<OptimizeReferences>true</OptimizeReferences>
</Link>
</ItemDefinitionGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'">
<ClCompile>
<WarningLevel>Level4</WarningLevel>
<PrecompiledHeader>
</PrecompiledHeader>
<Optimization>MaxSpeed</Optimization>
<FunctionLevelLinking>true</FunctionLevelLinking>
<IntrinsicFunctions>true</IntrinsicFunctions>
<PreprocessorDefinitions>WIN32;NDEBUG;_CONSOLE;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<TreatWarningAsError>true</TreatWarningAsError>
<EnablePREfast>false</EnablePREfast>
</ClCompile>
<Link>
<SubSystem>Console</SubSystem>
<GenerateDebugInformation>true</GenerateDebugInformation>
<EnableCOMDATFolding>true</EnableCOMDATFolding>
<OptimizeReferences>true</OptimizeReferences>
</Link>
</ItemDefinitionGroup>
<ItemGroup>
<ClCompile Include="..\..\..\lib\common\entropy_common.c" />
<ClCompile Include="..\..\..\lib\common\fse_decompress.c" />
<ClCompile Include="..\..\..\lib\common\zstd_common.c" />
<ClCompile Include="..\..\..\lib\compress\fse_compress.c" />
<ClCompile Include="..\..\..\lib\compress\huf_compress.c" />
<ClCompile Include="..\..\..\lib\compress\zstd_compress.c" />
<ClCompile Include="..\..\..\lib\decompress\huf_decompress.c" />
<ClCompile Include="..\..\..\lib\decompress\zstd_decompress.c" />
<ClCompile Include="..\..\..\programs\datagen.c" />
<ClCompile Include="..\..\..\programs\fuzzer.c" />
<ClCompile Include="..\..\..\programs\xxhash.c" />
</ItemGroup>
<ItemGroup>
<ClInclude Include="..\..\..\lib\common\fse.h" />
<ClInclude Include="..\..\..\lib\common\fse_static.h" />
<ClInclude Include="..\..\..\lib\common\huf.h" />
<ClInclude Include="..\..\..\lib\common\huf_static.h" />
<ClInclude Include="..\..\..\lib\common\zbuff.h" />
<ClInclude Include="..\..\..\lib\common\zstd_internal.h" />
<ClInclude Include="..\..\..\lib\common\zbuff_static.h" />
<ClInclude Include="..\..\..\lib\common\zstd.h" />
<ClInclude Include="..\..\..\lib\common\zstd_static.h" />
<ClInclude Include="..\..\..\lib\compress\zstd_opt.h" />
<ClInclude Include="..\..\..\lib\legacy\zstd_legacy.h" />
<ClInclude Include="..\..\..\programs\datagen.h" />
<ClInclude Include="..\..\..\programs\xxhash.h" />
<ClInclude Include="..\..\..\programs\util.h" />
</ItemGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.targets" />
<ImportGroup Label="ExtensionTargets">
</ImportGroup>
</Project>
+91 -89
View File
@@ -1,90 +1,92 @@
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup>
<Filter Include="Fichiers sources">
<UniqueIdentifier>{4FC737F1-C7A5-4376-A066-2A32D752A2FF}</UniqueIdentifier>
<Extensions>cpp;c;cc;cxx;def;odl;idl;hpj;bat;asm;asmx</Extensions>
</Filter>
<Filter Include="Fichiers d%27en-tête">
<UniqueIdentifier>{93995380-89BD-4b04-88EB-625FBE52EBFB}</UniqueIdentifier>
<Extensions>h;hpp;hxx;hm;inl;inc;xsd</Extensions>
</Filter>
<Filter Include="Fichiers de ressources">
<UniqueIdentifier>{67DA6AB6-F800-4c08-8B7A-83BB121AAD01}</UniqueIdentifier>
<Extensions>rc;ico;cur;bmp;dlg;rc2;rct;bin;rgs;gif;jpg;jpeg;jpe;resx;tiff;tif;png;wav;mfcribbon-ms</Extensions>
</Filter>
</ItemGroup>
<ItemGroup>
<ClCompile Include="..\..\..\lib\fse.c">
<Filter>Fichiers sources</Filter>
</ClCompile>
<ClCompile Include="..\..\..\programs\fuzzer.c">
<Filter>Fichiers sources</Filter>
</ClCompile>
<ClCompile Include="..\..\..\programs\xxhash.c">
<Filter>Fichiers sources</Filter>
</ClCompile>
<ClCompile Include="..\..\..\programs\datagen.c">
<Filter>Fichiers sources</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\legacy\zstd_v01.c">
<Filter>Fichiers sources</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\huff0.c">
<Filter>Fichiers sources</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\legacy\zstd_v02.c">
<Filter>Fichiers sources</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\zstd_compress.c">
<Filter>Fichiers sources</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\zstd_decompress.c">
<Filter>Fichiers sources</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\legacy\zstd_v03.c">
<Filter>Fichiers sources</Filter>
</ClCompile>
</ItemGroup>
<ItemGroup>
<ClInclude Include="..\..\..\lib\fse.h">
<Filter>Fichiers d%27en-tête</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\fse_static.h">
<Filter>Fichiers d%27en-tête</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\zstd.h">
<Filter>Fichiers d%27en-tête</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\zstd_static.h">
<Filter>Fichiers d%27en-tête</Filter>
</ClInclude>
<ClInclude Include="..\..\..\programs\xxhash.h">
<Filter>Fichiers d%27en-tête</Filter>
</ClInclude>
<ClInclude Include="..\..\..\programs\datagen.h">
<Filter>Fichiers d%27en-tête</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\legacy\zstd_v01.h">
<Filter>Fichiers d%27en-tête</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\huff0.h">
<Filter>Fichiers d%27en-tête</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\huff0_static.h">
<Filter>Fichiers d%27en-tête</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\zstd_internal.h">
<Filter>Fichiers d%27en-tête</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\legacy\zstd_legacy.h">
<Filter>Fichiers d%27en-tête</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\legacy\zstd_v02.h">
<Filter>Fichiers d%27en-tête</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\legacy\zstd_v03.h">
<Filter>Fichiers d%27en-tête</Filter>
</ClInclude>
</ItemGroup>
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup>
<Filter Include="Header Files">
<UniqueIdentifier>{93995380-89BD-4b04-88EB-625FBE52EBFB}</UniqueIdentifier>
<Extensions>h;hpp;hxx;hm;inl;inc;xsd</Extensions>
</Filter>
<Filter Include="Source Files">
<UniqueIdentifier>{4FC737F1-C7A5-4376-A066-2A32D752A2FF}</UniqueIdentifier>
<Extensions>cpp;c;cc;cxx;def;odl;idl;hpj;bat;asm;asmx</Extensions>
</Filter>
</ItemGroup>
<ItemGroup>
<ClCompile Include="..\..\..\programs\fuzzer.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\programs\xxhash.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\programs\datagen.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\common\zstd_common.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\compress\fse_compress.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\compress\huf_compress.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\decompress\huf_decompress.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\compress\zstd_compress.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\decompress\zstd_decompress.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\common\fse_decompress.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\common\entropy_common.c">
<Filter>Source Files</Filter>
</ClCompile>
</ItemGroup>
<ItemGroup>
<ClInclude Include="..\..\..\programs\xxhash.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\programs\datagen.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\legacy\zstd_legacy.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\common\fse.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\common\fse_static.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\common\huf.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\common\huf_static.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\common\zbuff.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\common\zstd_internal.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\common\zbuff_static.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\common\zstd.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\common\zstd_static.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\compress\zstd_opt.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\programs\util.h">
<Filter>Header Files</Filter>
</ClInclude>
</ItemGroup>
</Project>
+56 -58
View File
@@ -1,58 +1,56 @@
Microsoft Visual Studio Solution File, Format Version 12.00
# Visual Studio 2013
VisualStudioVersion = 12.0.40629.0
MinimumVisualStudioVersion = 10.0.40219.1
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "zstd", "zstd\zstd.vcxproj", "{4E52A41A-F33B-4C7A-8C36-A1A6B4F4277C}"
EndProject
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "fuzzer", "fuzzer\fuzzer.vcxproj", "{6FD4352B-346C-4703-96EA-D4A8B9A6976E}"
EndProject
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "fullbench", "fullbench\fullbench.vcxproj", "{61ABD629-1CC8-4FD7-9281-6B8DBB9D3DF8}"
EndProject
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "zstdlib", "zstdlib\zstdlib.vcxproj", "{8BFD8150-94D5-4BF9-8A50-7BD9929A0850}"
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution
Debug|Win32 = Debug|Win32
Debug|x64 = Debug|x64
Release|Win32 = Release|Win32
Release|x64 = Release|x64
EndGlobalSection
GlobalSection(ProjectConfigurationPlatforms) = postSolution
{4E52A41A-F33B-4C7A-8C36-A1A6B4F4277C}.Debug|Win32.ActiveCfg = Debug|Win32
{4E52A41A-F33B-4C7A-8C36-A1A6B4F4277C}.Debug|Win32.Build.0 = Debug|Win32
{4E52A41A-F33B-4C7A-8C36-A1A6B4F4277C}.Debug|x64.ActiveCfg = Debug|x64
{4E52A41A-F33B-4C7A-8C36-A1A6B4F4277C}.Debug|x64.Build.0 = Debug|x64
{4E52A41A-F33B-4C7A-8C36-A1A6B4F4277C}.Release|Win32.ActiveCfg = Release|Win32
{4E52A41A-F33B-4C7A-8C36-A1A6B4F4277C}.Release|Win32.Build.0 = Release|Win32
{4E52A41A-F33B-4C7A-8C36-A1A6B4F4277C}.Release|x64.ActiveCfg = Release|x64
{4E52A41A-F33B-4C7A-8C36-A1A6B4F4277C}.Release|x64.Build.0 = Release|x64
{6FD4352B-346C-4703-96EA-D4A8B9A6976E}.Debug|Win32.ActiveCfg = Debug|Win32
{6FD4352B-346C-4703-96EA-D4A8B9A6976E}.Debug|Win32.Build.0 = Debug|Win32
{6FD4352B-346C-4703-96EA-D4A8B9A6976E}.Debug|x64.ActiveCfg = Debug|x64
{6FD4352B-346C-4703-96EA-D4A8B9A6976E}.Debug|x64.Build.0 = Debug|x64
{6FD4352B-346C-4703-96EA-D4A8B9A6976E}.Release|Win32.ActiveCfg = Release|Win32
{6FD4352B-346C-4703-96EA-D4A8B9A6976E}.Release|Win32.Build.0 = Release|Win32
{6FD4352B-346C-4703-96EA-D4A8B9A6976E}.Release|x64.ActiveCfg = Release|x64
{6FD4352B-346C-4703-96EA-D4A8B9A6976E}.Release|x64.Build.0 = Release|x64
{61ABD629-1CC8-4FD7-9281-6B8DBB9D3DF8}.Debug|Win32.ActiveCfg = Debug|Win32
{61ABD629-1CC8-4FD7-9281-6B8DBB9D3DF8}.Debug|Win32.Build.0 = Debug|Win32
{61ABD629-1CC8-4FD7-9281-6B8DBB9D3DF8}.Debug|x64.ActiveCfg = Debug|x64
{61ABD629-1CC8-4FD7-9281-6B8DBB9D3DF8}.Debug|x64.Build.0 = Debug|x64
{61ABD629-1CC8-4FD7-9281-6B8DBB9D3DF8}.Release|Win32.ActiveCfg = Release|Win32
{61ABD629-1CC8-4FD7-9281-6B8DBB9D3DF8}.Release|Win32.Build.0 = Release|Win32
{61ABD629-1CC8-4FD7-9281-6B8DBB9D3DF8}.Release|x64.ActiveCfg = Release|x64
{61ABD629-1CC8-4FD7-9281-6B8DBB9D3DF8}.Release|x64.Build.0 = Release|x64
{8BFD8150-94D5-4BF9-8A50-7BD9929A0850}.Debug|Win32.ActiveCfg = Debug|Win32
{8BFD8150-94D5-4BF9-8A50-7BD9929A0850}.Debug|Win32.Build.0 = Debug|Win32
{8BFD8150-94D5-4BF9-8A50-7BD9929A0850}.Debug|x64.ActiveCfg = Debug|x64
{8BFD8150-94D5-4BF9-8A50-7BD9929A0850}.Debug|x64.Build.0 = Debug|x64
{8BFD8150-94D5-4BF9-8A50-7BD9929A0850}.Release|Win32.ActiveCfg = Release|Win32
{8BFD8150-94D5-4BF9-8A50-7BD9929A0850}.Release|Win32.Build.0 = Release|Win32
{8BFD8150-94D5-4BF9-8A50-7BD9929A0850}.Release|x64.ActiveCfg = Release|x64
{8BFD8150-94D5-4BF9-8A50-7BD9929A0850}.Release|x64.Build.0 = Release|x64
EndGlobalSection
GlobalSection(SolutionProperties) = preSolution
HideSolutionNode = FALSE
EndGlobalSection
EndGlobal
Microsoft Visual Studio Solution File, Format Version 12.00
# Visual Studio Express 2012 for Windows Desktop
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "zstd", "zstd\zstd.vcxproj", "{4E52A41A-F33B-4C7A-8C36-A1A6B4F4277C}"
EndProject
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "fuzzer", "fuzzer\fuzzer.vcxproj", "{6FD4352B-346C-4703-96EA-D4A8B9A6976E}"
EndProject
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "fullbench", "fullbench\fullbench.vcxproj", "{61ABD629-1CC8-4FD7-9281-6B8DBB9D3DF8}"
EndProject
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "zstdlib", "zstdlib\zstdlib.vcxproj", "{8BFD8150-94D5-4BF9-8A50-7BD9929A0850}"
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution
Debug|Win32 = Debug|Win32
Debug|x64 = Debug|x64
Release|Win32 = Release|Win32
Release|x64 = Release|x64
EndGlobalSection
GlobalSection(ProjectConfigurationPlatforms) = postSolution
{4E52A41A-F33B-4C7A-8C36-A1A6B4F4277C}.Debug|Win32.ActiveCfg = Debug|Win32
{4E52A41A-F33B-4C7A-8C36-A1A6B4F4277C}.Debug|Win32.Build.0 = Debug|Win32
{4E52A41A-F33B-4C7A-8C36-A1A6B4F4277C}.Debug|x64.ActiveCfg = Debug|x64
{4E52A41A-F33B-4C7A-8C36-A1A6B4F4277C}.Debug|x64.Build.0 = Debug|x64
{4E52A41A-F33B-4C7A-8C36-A1A6B4F4277C}.Release|Win32.ActiveCfg = Release|Win32
{4E52A41A-F33B-4C7A-8C36-A1A6B4F4277C}.Release|Win32.Build.0 = Release|Win32
{4E52A41A-F33B-4C7A-8C36-A1A6B4F4277C}.Release|x64.ActiveCfg = Release|x64
{4E52A41A-F33B-4C7A-8C36-A1A6B4F4277C}.Release|x64.Build.0 = Release|x64
{6FD4352B-346C-4703-96EA-D4A8B9A6976E}.Debug|Win32.ActiveCfg = Debug|Win32
{6FD4352B-346C-4703-96EA-D4A8B9A6976E}.Debug|Win32.Build.0 = Debug|Win32
{6FD4352B-346C-4703-96EA-D4A8B9A6976E}.Debug|x64.ActiveCfg = Debug|x64
{6FD4352B-346C-4703-96EA-D4A8B9A6976E}.Debug|x64.Build.0 = Debug|x64
{6FD4352B-346C-4703-96EA-D4A8B9A6976E}.Release|Win32.ActiveCfg = Release|Win32
{6FD4352B-346C-4703-96EA-D4A8B9A6976E}.Release|Win32.Build.0 = Release|Win32
{6FD4352B-346C-4703-96EA-D4A8B9A6976E}.Release|x64.ActiveCfg = Release|x64
{6FD4352B-346C-4703-96EA-D4A8B9A6976E}.Release|x64.Build.0 = Release|x64
{61ABD629-1CC8-4FD7-9281-6B8DBB9D3DF8}.Debug|Win32.ActiveCfg = Debug|Win32
{61ABD629-1CC8-4FD7-9281-6B8DBB9D3DF8}.Debug|Win32.Build.0 = Debug|Win32
{61ABD629-1CC8-4FD7-9281-6B8DBB9D3DF8}.Debug|x64.ActiveCfg = Debug|x64
{61ABD629-1CC8-4FD7-9281-6B8DBB9D3DF8}.Debug|x64.Build.0 = Debug|x64
{61ABD629-1CC8-4FD7-9281-6B8DBB9D3DF8}.Release|Win32.ActiveCfg = Release|Win32
{61ABD629-1CC8-4FD7-9281-6B8DBB9D3DF8}.Release|Win32.Build.0 = Release|Win32
{61ABD629-1CC8-4FD7-9281-6B8DBB9D3DF8}.Release|x64.ActiveCfg = Release|x64
{61ABD629-1CC8-4FD7-9281-6B8DBB9D3DF8}.Release|x64.Build.0 = Release|x64
{8BFD8150-94D5-4BF9-8A50-7BD9929A0850}.Debug|Win32.ActiveCfg = Debug|Win32
{8BFD8150-94D5-4BF9-8A50-7BD9929A0850}.Debug|Win32.Build.0 = Debug|Win32
{8BFD8150-94D5-4BF9-8A50-7BD9929A0850}.Debug|x64.ActiveCfg = Debug|x64
{8BFD8150-94D5-4BF9-8A50-7BD9929A0850}.Debug|x64.Build.0 = Debug|x64
{8BFD8150-94D5-4BF9-8A50-7BD9929A0850}.Release|Win32.ActiveCfg = Release|Win32
{8BFD8150-94D5-4BF9-8A50-7BD9929A0850}.Release|Win32.Build.0 = Release|Win32
{8BFD8150-94D5-4BF9-8A50-7BD9929A0850}.Release|x64.ActiveCfg = Release|x64
{8BFD8150-94D5-4BF9-8A50-7BD9929A0850}.Release|x64.Build.0 = Release|x64
EndGlobalSection
GlobalSection(SolutionProperties) = preSolution
HideSolutionNode = FALSE
EndGlobalSection
EndGlobal
+218 -215
View File
@@ -1,216 +1,219 @@
<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Build" ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup Label="ProjectConfigurations">
<ProjectConfiguration Include="Debug|Win32">
<Configuration>Debug</Configuration>
<Platform>Win32</Platform>
</ProjectConfiguration>
<ProjectConfiguration Include="Debug|x64">
<Configuration>Debug</Configuration>
<Platform>x64</Platform>
</ProjectConfiguration>
<ProjectConfiguration Include="Release|Win32">
<Configuration>Release</Configuration>
<Platform>Win32</Platform>
</ProjectConfiguration>
<ProjectConfiguration Include="Release|x64">
<Configuration>Release</Configuration>
<Platform>x64</Platform>
</ProjectConfiguration>
</ItemGroup>
<ItemGroup>
<ClCompile Include="..\..\..\lib\divsufsort.c" />
<ClCompile Include="..\..\..\lib\fse.c" />
<ClCompile Include="..\..\..\lib\huff0.c" />
<ClCompile Include="..\..\..\lib\legacy\zstd_v01.c" />
<ClCompile Include="..\..\..\lib\legacy\zstd_v02.c" />
<ClCompile Include="..\..\..\lib\legacy\zstd_v03.c" />
<ClCompile Include="..\..\..\lib\legacy\zstd_v04.c" />
<ClCompile Include="..\..\..\lib\legacy\zstd_v05.c" />
<ClCompile Include="..\..\..\lib\zbuff.c" />
<ClCompile Include="..\..\..\lib\zdict.c" />
<ClCompile Include="..\..\..\lib\zstd_compress.c" />
<ClCompile Include="..\..\..\lib\zstd_decompress.c" />
<ClCompile Include="..\..\..\programs\bench.c" />
<ClCompile Include="..\..\..\programs\datagen.c" />
<ClCompile Include="..\..\..\programs\dibio.c" />
<ClCompile Include="..\..\..\programs\fileio.c" />
<ClCompile Include="..\..\..\programs\legacy\fileio_legacy.c" />
<ClCompile Include="..\..\..\programs\xxhash.c" />
<ClCompile Include="..\..\..\programs\zstdcli.c" />
</ItemGroup>
<ItemGroup>
<ClInclude Include="..\..\..\lib\divsufsort.h" />
<ClInclude Include="..\..\..\lib\fse.h" />
<ClInclude Include="..\..\..\lib\fse_static.h" />
<ClInclude Include="..\..\..\lib\huff0.h" />
<ClInclude Include="..\..\..\lib\huff0_static.h" />
<ClInclude Include="..\..\..\lib\legacy\zstd_legacy.h" />
<ClInclude Include="..\..\..\lib\legacy\zstd_v01.h" />
<ClInclude Include="..\..\..\lib\legacy\zstd_v02.h" />
<ClInclude Include="..\..\..\lib\legacy\zstd_v03.h" />
<ClInclude Include="..\..\..\lib\legacy\zstd_v04.h" />
<ClInclude Include="..\..\..\lib\legacy\zstd_v05.h" />
<ClInclude Include="..\..\..\lib\zbuff.h" />
<ClInclude Include="..\..\..\lib\zbuff_static.h" />
<ClInclude Include="..\..\..\lib\zdict.h" />
<ClInclude Include="..\..\..\lib\zdict_static.h" />
<ClInclude Include="..\..\..\lib\zstd.h" />
<ClInclude Include="..\..\..\lib\zstd_buffered.h" />
<ClInclude Include="..\..\..\lib\zstd_buffered_static.h" />
<ClInclude Include="..\..\..\lib\zstd_internal.h" />
<ClInclude Include="..\..\..\lib\zstd_static.h" />
<ClInclude Include="..\..\..\programs\bench.h" />
<ClInclude Include="..\..\..\programs\datagen.h" />
<ClInclude Include="..\..\..\programs\dibio.h" />
<ClInclude Include="..\..\..\programs\fileio.h" />
<ClInclude Include="..\..\..\programs\legacy\fileio_legacy.h" />
<ClInclude Include="..\..\..\programs\xxhash.h" />
</ItemGroup>
<PropertyGroup Label="Globals">
<ProjectGuid>{4E52A41A-F33B-4C7A-8C36-A1A6B4F4277C}</ProjectGuid>
<Keyword>Win32Proj</Keyword>
<RootNamespace>zstd</RootNamespace>
<OutDir>$(SolutionDir)bin\$(Platform)\$(Configuration)\</OutDir>
</PropertyGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.Default.props" />
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'" Label="Configuration">
<ConfigurationType>Application</ConfigurationType>
<UseDebugLibraries>true</UseDebugLibraries>
<CharacterSet>Unicode</CharacterSet>
<PlatformToolset>v120</PlatformToolset>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'" Label="Configuration">
<ConfigurationType>Application</ConfigurationType>
<UseDebugLibraries>true</UseDebugLibraries>
<PlatformToolset>v120</PlatformToolset>
<CharacterSet>Unicode</CharacterSet>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'" Label="Configuration">
<ConfigurationType>Application</ConfigurationType>
<UseDebugLibraries>false</UseDebugLibraries>
<WholeProgramOptimization>true</WholeProgramOptimization>
<PlatformToolset>v120_xp</PlatformToolset>
<CharacterSet>Unicode</CharacterSet>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'" Label="Configuration">
<ConfigurationType>Application</ConfigurationType>
<UseDebugLibraries>false</UseDebugLibraries>
<WholeProgramOptimization>true</WholeProgramOptimization>
<PlatformToolset>v120</PlatformToolset>
<CharacterSet>Unicode</CharacterSet>
</PropertyGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.props" />
<ImportGroup Label="ExtensionSettings">
</ImportGroup>
<ImportGroup Label="PropertySheets" Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
</ImportGroup>
<ImportGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'" Label="PropertySheets">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
</ImportGroup>
<ImportGroup Label="PropertySheets" Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
</ImportGroup>
<ImportGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'" Label="PropertySheets">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
</ImportGroup>
<PropertyGroup Label="UserMacros" />
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
<LinkIncremental>true</LinkIncremental>
<IncludePath>$(SolutionDir)..\..\programs\legacy;$(SolutionDir)..\..\lib\legacy;$(SolutionDir)..\..\lib;$(VCInstallDir)include;$(VCInstallDir)atlmfc\include;$(WindowsSDK_IncludePath);</IncludePath>
<RunCodeAnalysis>true</RunCodeAnalysis>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">
<LinkIncremental>true</LinkIncremental>
<IncludePath>$(SolutionDir)..\..\programs\legacy;$(SolutionDir)..\..\lib\legacy;$(SolutionDir)..\..\lib;$(VCInstallDir)include;$(VCInstallDir)atlmfc\include;$(WindowsSDK_IncludePath);</IncludePath>
<RunCodeAnalysis>true</RunCodeAnalysis>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
<LinkIncremental>false</LinkIncremental>
<IncludePath>$(SolutionDir)..\..\programs\legacy;$(SolutionDir)..\..\lib\legacy;$(SolutionDir)..\..\lib;$(VCInstallDir)include;$(VCInstallDir)atlmfc\include;$(WindowsSDK_IncludePath);</IncludePath>
<RunCodeAnalysis>false</RunCodeAnalysis>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'">
<LinkIncremental>false</LinkIncremental>
<IncludePath>$(SolutionDir)..\..\programs\legacy;$(SolutionDir)..\..\lib\legacy;$(SolutionDir)..\..\lib;$(VCInstallDir)include;$(VCInstallDir)atlmfc\include;$(WindowsSDK_IncludePath);</IncludePath>
<RunCodeAnalysis>false</RunCodeAnalysis>
</PropertyGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
<ClCompile>
<PrecompiledHeader>
</PrecompiledHeader>
<WarningLevel>Level4</WarningLevel>
<Optimization>Disabled</Optimization>
<PreprocessorDefinitions>WIN32;_DEBUG;_CONSOLE;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<TreatWarningAsError>true</TreatWarningAsError>
<EnablePREfast>true</EnablePREfast>
<AdditionalOptions>/analyze:stacksize25000 %(AdditionalOptions)</AdditionalOptions>
</ClCompile>
<Link>
<SubSystem>Console</SubSystem>
<GenerateDebugInformation>true</GenerateDebugInformation>
</Link>
</ItemDefinitionGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">
<ClCompile>
<PrecompiledHeader>
</PrecompiledHeader>
<WarningLevel>Level4</WarningLevel>
<Optimization>Disabled</Optimization>
<PreprocessorDefinitions>WIN32;_DEBUG;_CONSOLE;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<TreatWarningAsError>false</TreatWarningAsError>
<EnablePREfast>true</EnablePREfast>
<AdditionalOptions>/analyze:stacksize19000 %(AdditionalOptions)</AdditionalOptions>
</ClCompile>
<Link>
<SubSystem>Console</SubSystem>
<GenerateDebugInformation>true</GenerateDebugInformation>
</Link>
</ItemDefinitionGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
<ClCompile>
<WarningLevel>Level4</WarningLevel>
<PrecompiledHeader>
</PrecompiledHeader>
<Optimization>Full</Optimization>
<FunctionLevelLinking>true</FunctionLevelLinking>
<IntrinsicFunctions>true</IntrinsicFunctions>
<PreprocessorDefinitions>WIN32;NDEBUG;_CONSOLE;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<EnablePREfast>false</EnablePREfast>
<TreatWarningAsError>true</TreatWarningAsError>
<RuntimeLibrary>MultiThreaded</RuntimeLibrary>
</ClCompile>
<Link>
<SubSystem>Console</SubSystem>
<GenerateDebugInformation>true</GenerateDebugInformation>
<EnableCOMDATFolding>true</EnableCOMDATFolding>
<OptimizeReferences>true</OptimizeReferences>
<AdditionalDependencies>setargv.obj; kernel32.lib;user32.lib;gdi32.lib;winspool.lib;comdlg32.lib;advapi32.lib;shell32.lib;ole32.lib;oleaut32.lib;uuid.lib;odbc32.lib;odbccp32.lib;%(AdditionalDependencies)</AdditionalDependencies>
</Link>
</ItemDefinitionGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'">
<ClCompile>
<WarningLevel>Level4</WarningLevel>
<PrecompiledHeader>
</PrecompiledHeader>
<Optimization>MaxSpeed</Optimization>
<FunctionLevelLinking>true</FunctionLevelLinking>
<IntrinsicFunctions>true</IntrinsicFunctions>
<PreprocessorDefinitions>WIN32;NDEBUG;_CONSOLE;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<TreatWarningAsError>false</TreatWarningAsError>
<EnablePREfast>false</EnablePREfast>
<RuntimeLibrary>MultiThreaded</RuntimeLibrary>
</ClCompile>
<Link>
<SubSystem>Console</SubSystem>
<GenerateDebugInformation>true</GenerateDebugInformation>
<EnableCOMDATFolding>true</EnableCOMDATFolding>
<OptimizeReferences>true</OptimizeReferences>
<AdditionalDependencies>setargv.obj;kernel32.lib;user32.lib;gdi32.lib;winspool.lib;comdlg32.lib;advapi32.lib;shell32.lib;ole32.lib;oleaut32.lib;uuid.lib;odbc32.lib;odbccp32.lib;%(AdditionalDependencies)</AdditionalDependencies>
</Link>
</ItemDefinitionGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.targets" />
<ImportGroup Label="ExtensionTargets">
</ImportGroup>
<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Build" ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup Label="ProjectConfigurations">
<ProjectConfiguration Include="Debug|Win32">
<Configuration>Debug</Configuration>
<Platform>Win32</Platform>
</ProjectConfiguration>
<ProjectConfiguration Include="Debug|x64">
<Configuration>Debug</Configuration>
<Platform>x64</Platform>
</ProjectConfiguration>
<ProjectConfiguration Include="Release|Win32">
<Configuration>Release</Configuration>
<Platform>Win32</Platform>
</ProjectConfiguration>
<ProjectConfiguration Include="Release|x64">
<Configuration>Release</Configuration>
<Platform>x64</Platform>
</ProjectConfiguration>
</ItemGroup>
<ItemGroup>
<ClCompile Include="..\..\..\lib\common\entropy_common.c" />
<ClCompile Include="..\..\..\lib\common\zstd_common.c" />
<ClCompile Include="..\..\..\lib\common\fse_decompress.c" />
<ClCompile Include="..\..\..\lib\compress\fse_compress.c" />
<ClCompile Include="..\..\..\lib\compress\huf_compress.c" />
<ClCompile Include="..\..\..\lib\compress\zbuff_compress.c" />
<ClCompile Include="..\..\..\lib\compress\zstd_compress.c" />
<ClCompile Include="..\..\..\lib\decompress\huf_decompress.c" />
<ClCompile Include="..\..\..\lib\decompress\zbuff_decompress.c" />
<ClCompile Include="..\..\..\lib\decompress\zstd_decompress.c" />
<ClCompile Include="..\..\..\lib\dictBuilder\divsufsort.c" />
<ClCompile Include="..\..\..\lib\dictBuilder\zdict.c" />
<ClCompile Include="..\..\..\lib\legacy\zstd_v01.c" />
<ClCompile Include="..\..\..\lib\legacy\zstd_v02.c" />
<ClCompile Include="..\..\..\lib\legacy\zstd_v03.c" />
<ClCompile Include="..\..\..\lib\legacy\zstd_v04.c" />
<ClCompile Include="..\..\..\lib\legacy\zstd_v05.c" />
<ClCompile Include="..\..\..\programs\bench.c" />
<ClCompile Include="..\..\..\programs\datagen.c" />
<ClCompile Include="..\..\..\programs\dibio.c" />
<ClCompile Include="..\..\..\programs\fileio.c" />
<ClCompile Include="..\..\..\programs\legacy\fileio_legacy.c" />
<ClCompile Include="..\..\..\programs\xxhash.c" />
<ClCompile Include="..\..\..\programs\zstdcli.c" />
</ItemGroup>
<ItemGroup>
<ClInclude Include="..\..\..\lib\dictBuilder\zdict.h" />
<ClInclude Include="..\..\..\lib\dictBuilder\zdict_static.h" />
<ClInclude Include="..\..\..\lib\dictBuilder\divsufsort.h" />
<ClInclude Include="..\..\..\lib\common\fse.h" />
<ClInclude Include="..\..\..\lib\common\fse_static.h" />
<ClInclude Include="..\..\..\lib\common\huf.h" />
<ClInclude Include="..\..\..\lib\common\huf_static.h" />
<ClInclude Include="..\..\..\lib\common\zbuff.h" />
<ClInclude Include="..\..\..\lib\common\zbuff_static.h" />
<ClInclude Include="..\..\..\lib\common\zstd.h" />
<ClInclude Include="..\..\..\lib\common\zstd_internal.h" />
<ClInclude Include="..\..\..\lib\common\zstd_static.h" />
<ClInclude Include="..\..\..\lib\compress\zstd_opt.h" />
<ClInclude Include="..\..\..\lib\legacy\zstd_legacy.h" />
<ClInclude Include="..\..\..\lib\legacy\zstd_v01.h" />
<ClInclude Include="..\..\..\lib\legacy\zstd_v02.h" />
<ClInclude Include="..\..\..\lib\legacy\zstd_v03.h" />
<ClInclude Include="..\..\..\lib\legacy\zstd_v04.h" />
<ClInclude Include="..\..\..\lib\legacy\zstd_v05.h" />
<ClInclude Include="..\..\..\programs\bench.h" />
<ClInclude Include="..\..\..\programs\datagen.h" />
<ClInclude Include="..\..\..\programs\dibio.h" />
<ClInclude Include="..\..\..\programs\fileio.h" />
<ClInclude Include="..\..\..\programs\legacy\fileio_legacy.h" />
<ClInclude Include="..\..\..\programs\xxhash.h" />
<ClInclude Include="..\..\..\programs\util.h" />
</ItemGroup>
<PropertyGroup Label="Globals">
<ProjectGuid>{4E52A41A-F33B-4C7A-8C36-A1A6B4F4277C}</ProjectGuid>
<Keyword>Win32Proj</Keyword>
<RootNamespace>zstd</RootNamespace>
<OutDir>$(SolutionDir)bin\$(Platform)\$(Configuration)\</OutDir>
</PropertyGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.Default.props" />
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'" Label="Configuration">
<ConfigurationType>Application</ConfigurationType>
<UseDebugLibraries>true</UseDebugLibraries>
<PlatformToolset>v110</PlatformToolset>
<CharacterSet>MultiByte</CharacterSet>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'" Label="Configuration">
<ConfigurationType>Application</ConfigurationType>
<UseDebugLibraries>true</UseDebugLibraries>
<PlatformToolset>v110</PlatformToolset>
<CharacterSet>MultiByte</CharacterSet>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'" Label="Configuration">
<ConfigurationType>Application</ConfigurationType>
<UseDebugLibraries>false</UseDebugLibraries>
<WholeProgramOptimization>true</WholeProgramOptimization>
<PlatformToolset>v110</PlatformToolset>
<CharacterSet>MultiByte</CharacterSet>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'" Label="Configuration">
<ConfigurationType>Application</ConfigurationType>
<UseDebugLibraries>false</UseDebugLibraries>
<WholeProgramOptimization>true</WholeProgramOptimization>
<PlatformToolset>v110</PlatformToolset>
<CharacterSet>MultiByte</CharacterSet>
</PropertyGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.props" />
<ImportGroup Label="ExtensionSettings">
</ImportGroup>
<ImportGroup Label="PropertySheets" Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
</ImportGroup>
<ImportGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'" Label="PropertySheets">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
</ImportGroup>
<ImportGroup Label="PropertySheets" Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
</ImportGroup>
<ImportGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'" Label="PropertySheets">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
</ImportGroup>
<PropertyGroup Label="UserMacros" />
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
<LinkIncremental>true</LinkIncremental>
<IncludePath>$(WindowsSdkDir)\include;$(UniversalCRT_IncludePath);$(SolutionDir)..\..\programs\legacy;$(SolutionDir)..\..\lib\legacy;$(SolutionDir)..\..\lib\common;$(VCInstallDir)include;$(VCInstallDir)atlmfc\include;$(WindowsSDK_IncludePath);</IncludePath>
<RunCodeAnalysis>false</RunCodeAnalysis>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">
<LinkIncremental>true</LinkIncremental>
<IncludePath>$(WindowsSdkDir)\include;$(UniversalCRT_IncludePath);$(SolutionDir)..\..\programs\legacy;$(SolutionDir)..\..\lib\legacy;$(SolutionDir)..\..\lib\common;$(VCInstallDir)include;$(VCInstallDir)atlmfc\include;$(WindowsSDK_IncludePath);</IncludePath>
<RunCodeAnalysis>false</RunCodeAnalysis>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
<LinkIncremental>false</LinkIncremental>
<IncludePath>$(WindowsSdkDir)\include;$(UniversalCRT_IncludePath);$(SolutionDir)..\..\programs\legacy;$(SolutionDir)..\..\lib\legacy;$(SolutionDir)..\..\lib\common;$(VCInstallDir)include;$(VCInstallDir)atlmfc\include;$(WindowsSDK_IncludePath);</IncludePath>
<RunCodeAnalysis>false</RunCodeAnalysis>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'">
<LinkIncremental>false</LinkIncremental>
<IncludePath>$(WindowsSdkDir)\include;$(UniversalCRT_IncludePath);$(SolutionDir)..\..\programs\legacy;$(SolutionDir)..\..\lib\legacy;$(SolutionDir)..\..\lib\common;$(VCInstallDir)include;$(VCInstallDir)atlmfc\include;$(WindowsSDK_IncludePath);</IncludePath>
<RunCodeAnalysis>false</RunCodeAnalysis>
</PropertyGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
<ClCompile>
<PrecompiledHeader>
</PrecompiledHeader>
<WarningLevel>Level4</WarningLevel>
<Optimization>Disabled</Optimization>
<PreprocessorDefinitions>WIN32;_DEBUG;_CONSOLE;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<TreatWarningAsError>true</TreatWarningAsError>
<EnablePREfast>false</EnablePREfast>
</ClCompile>
<Link>
<SubSystem>Console</SubSystem>
<GenerateDebugInformation>true</GenerateDebugInformation>
</Link>
</ItemDefinitionGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">
<ClCompile>
<PrecompiledHeader>
</PrecompiledHeader>
<WarningLevel>Level4</WarningLevel>
<Optimization>Disabled</Optimization>
<PreprocessorDefinitions>WIN32;_DEBUG;_CONSOLE;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<TreatWarningAsError>false</TreatWarningAsError>
<EnablePREfast>false</EnablePREfast>
</ClCompile>
<Link>
<SubSystem>Console</SubSystem>
<GenerateDebugInformation>true</GenerateDebugInformation>
</Link>
</ItemDefinitionGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
<ClCompile>
<WarningLevel>Level4</WarningLevel>
<PrecompiledHeader>
</PrecompiledHeader>
<Optimization>Full</Optimization>
<FunctionLevelLinking>true</FunctionLevelLinking>
<IntrinsicFunctions>true</IntrinsicFunctions>
<PreprocessorDefinitions>WIN32;NDEBUG;_CONSOLE;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<EnablePREfast>false</EnablePREfast>
<TreatWarningAsError>true</TreatWarningAsError>
<RuntimeLibrary>MultiThreaded</RuntimeLibrary>
</ClCompile>
<Link>
<SubSystem>Console</SubSystem>
<GenerateDebugInformation>true</GenerateDebugInformation>
<EnableCOMDATFolding>true</EnableCOMDATFolding>
<OptimizeReferences>true</OptimizeReferences>
<AdditionalDependencies>setargv.obj; kernel32.lib;user32.lib;gdi32.lib;winspool.lib;comdlg32.lib;advapi32.lib;shell32.lib;ole32.lib;oleaut32.lib;uuid.lib;odbc32.lib;odbccp32.lib;%(AdditionalDependencies)</AdditionalDependencies>
</Link>
</ItemDefinitionGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'">
<ClCompile>
<WarningLevel>Level4</WarningLevel>
<PrecompiledHeader>
</PrecompiledHeader>
<Optimization>MaxSpeed</Optimization>
<FunctionLevelLinking>true</FunctionLevelLinking>
<IntrinsicFunctions>true</IntrinsicFunctions>
<PreprocessorDefinitions>WIN32;NDEBUG;_CONSOLE;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<TreatWarningAsError>false</TreatWarningAsError>
<EnablePREfast>false</EnablePREfast>
<RuntimeLibrary>MultiThreaded</RuntimeLibrary>
</ClCompile>
<Link>
<SubSystem>Console</SubSystem>
<GenerateDebugInformation>true</GenerateDebugInformation>
<EnableCOMDATFolding>true</EnableCOMDATFolding>
<OptimizeReferences>true</OptimizeReferences>
<AdditionalDependencies>setargv.obj;kernel32.lib;user32.lib;gdi32.lib;winspool.lib;comdlg32.lib;advapi32.lib;shell32.lib;ole32.lib;oleaut32.lib;uuid.lib;odbc32.lib;odbccp32.lib;%(AdditionalDependencies)</AdditionalDependencies>
</Link>
</ItemDefinitionGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.targets" />
<ImportGroup Label="ExtensionTargets">
</ImportGroup>
</Project>
+166 -155
View File
@@ -1,156 +1,167 @@
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup>
<Filter Include="Fichiers sources">
<UniqueIdentifier>{4FC737F1-C7A5-4376-A066-2A32D752A2FF}</UniqueIdentifier>
<Extensions>cpp;c;cc;cxx;def;odl;idl;hpj;bat;asm;asmx</Extensions>
</Filter>
<Filter Include="Fichiers d%27en-tête">
<UniqueIdentifier>{93995380-89BD-4b04-88EB-625FBE52EBFB}</UniqueIdentifier>
<Extensions>h;hpp;hxx;hm;inl;inc;xsd</Extensions>
</Filter>
<Filter Include="Fichiers de ressources">
<UniqueIdentifier>{67DA6AB6-F800-4c08-8B7A-83BB121AAD01}</UniqueIdentifier>
<Extensions>rc;ico;cur;bmp;dlg;rc2;rct;bin;rgs;gif;jpg;jpeg;jpe;resx;tiff;tif;png;wav;mfcribbon-ms</Extensions>
</Filter>
</ItemGroup>
<ItemGroup>
<ClCompile Include="..\..\..\lib\fse.c">
<Filter>Fichiers sources</Filter>
</ClCompile>
<ClCompile Include="..\..\..\programs\bench.c">
<Filter>Fichiers sources</Filter>
</ClCompile>
<ClCompile Include="..\..\..\programs\fileio.c">
<Filter>Fichiers sources</Filter>
</ClCompile>
<ClCompile Include="..\..\..\programs\xxhash.c">
<Filter>Fichiers sources</Filter>
</ClCompile>
<ClCompile Include="..\..\..\programs\zstdcli.c">
<Filter>Fichiers sources</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\legacy\zstd_v01.c">
<Filter>Fichiers sources</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\huff0.c">
<Filter>Fichiers sources</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\legacy\zstd_v02.c">
<Filter>Fichiers sources</Filter>
</ClCompile>
<ClCompile Include="..\..\..\programs\legacy\fileio_legacy.c">
<Filter>Fichiers sources</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\zstd_compress.c">
<Filter>Fichiers sources</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\zstd_decompress.c">
<Filter>Fichiers sources</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\legacy\zstd_v03.c">
<Filter>Fichiers sources</Filter>
</ClCompile>
<ClCompile Include="..\..\..\programs\datagen.c">
<Filter>Fichiers sources</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\legacy\zstd_v04.c">
<Filter>Fichiers sources</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\divsufsort.c">
<Filter>Fichiers sources</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\zbuff.c">
<Filter>Fichiers sources</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\zdict.c">
<Filter>Fichiers sources</Filter>
</ClCompile>
<ClCompile Include="..\..\..\programs\dibio.c">
<Filter>Fichiers sources</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\legacy\zstd_v05.c">
<Filter>Fichiers sources</Filter>
</ClCompile>
</ItemGroup>
<ItemGroup>
<ClInclude Include="..\..\..\lib\fse.h">
<Filter>Fichiers d%27en-tête</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\fse_static.h">
<Filter>Fichiers d%27en-tête</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\zstd.h">
<Filter>Fichiers d%27en-tête</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\zstd_static.h">
<Filter>Fichiers d%27en-tête</Filter>
</ClInclude>
<ClInclude Include="..\..\..\programs\bench.h">
<Filter>Fichiers d%27en-tête</Filter>
</ClInclude>
<ClInclude Include="..\..\..\programs\fileio.h">
<Filter>Fichiers d%27en-tête</Filter>
</ClInclude>
<ClInclude Include="..\..\..\programs\xxhash.h">
<Filter>Fichiers d%27en-tête</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\legacy\zstd_v01.h">
<Filter>Fichiers d%27en-tête</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\huff0.h">
<Filter>Fichiers d%27en-tête</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\huff0_static.h">
<Filter>Fichiers d%27en-tête</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\zstd_internal.h">
<Filter>Fichiers d%27en-tête</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\legacy\zstd_legacy.h">
<Filter>Fichiers d%27en-tête</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\legacy\zstd_v02.h">
<Filter>Fichiers d%27en-tête</Filter>
</ClInclude>
<ClInclude Include="..\..\..\programs\legacy\fileio_legacy.h">
<Filter>Fichiers d%27en-tête</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\zstd_buffered.h">
<Filter>Fichiers d%27en-tête</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\zstd_buffered_static.h">
<Filter>Fichiers d%27en-tête</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\legacy\zstd_v03.h">
<Filter>Fichiers d%27en-tête</Filter>
</ClInclude>
<ClInclude Include="..\..\..\programs\datagen.h">
<Filter>Fichiers d%27en-tête</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\legacy\zstd_v04.h">
<Filter>Fichiers d%27en-tête</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\divsufsort.h">
<Filter>Fichiers sources</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\zbuff.h">
<Filter>Fichiers sources</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\zbuff_static.h">
<Filter>Fichiers sources</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\zdict.h">
<Filter>Fichiers sources</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\zdict_static.h">
<Filter>Fichiers sources</Filter>
</ClInclude>
<ClInclude Include="..\..\..\programs\dibio.h">
<Filter>Fichiers d%27en-tête</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\legacy\zstd_v05.h">
<Filter>Fichiers d%27en-tête</Filter>
</ClInclude>
</ItemGroup>
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup>
<Filter Include="Header Files">
<UniqueIdentifier>{93995380-89BD-4b04-88EB-625FBE52EBFB}</UniqueIdentifier>
<Extensions>h;hpp;hxx;hm;inl;inc;xsd</Extensions>
</Filter>
<Filter Include="Source Files">
<UniqueIdentifier>{4FC737F1-C7A5-4376-A066-2A32D752A2FF}</UniqueIdentifier>
<Extensions>cpp;c;cc;cxx;def;odl;idl;hpj;bat;asm;asmx</Extensions>
</Filter>
</ItemGroup>
<ItemGroup>
<ClCompile Include="..\..\..\programs\bench.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\programs\fileio.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\programs\xxhash.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\programs\zstdcli.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\legacy\zstd_v01.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\legacy\zstd_v02.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\programs\legacy\fileio_legacy.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\legacy\zstd_v03.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\programs\datagen.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\legacy\zstd_v04.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\programs\dibio.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\legacy\zstd_v05.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\common\zstd_common.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\compress\fse_compress.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\compress\huf_compress.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\compress\zbuff_compress.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\compress\zstd_compress.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\decompress\huf_decompress.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\decompress\zbuff_decompress.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\decompress\zstd_decompress.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\dictBuilder\divsufsort.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\dictBuilder\zdict.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\common\fse_decompress.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\common\entropy_common.c">
<Filter>Source Files</Filter>
</ClCompile>
</ItemGroup>
<ItemGroup>
<ClInclude Include="..\..\..\programs\bench.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\programs\fileio.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\programs\xxhash.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\legacy\zstd_v01.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\legacy\zstd_legacy.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\legacy\zstd_v02.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\programs\legacy\fileio_legacy.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\legacy\zstd_v03.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\programs\datagen.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\legacy\zstd_v04.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\programs\dibio.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\dictBuilder\zdict.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\dictBuilder\zdict_static.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\dictBuilder\divsufsort.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\common\fse.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\common\fse_static.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\common\huf.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\common\huf_static.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\common\zbuff.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\common\zbuff_static.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\common\zstd.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\common\zstd_internal.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\common\zstd_static.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\compress\zstd_opt.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\legacy\zstd_v05.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\programs\util.h">
<Filter>Header Files</Filter>
</ClInclude>
</ItemGroup>
</Project>
Binary file not shown.
+217 -211
View File
@@ -1,212 +1,218 @@
<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Build" ToolsVersion="12.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup Label="ProjectConfigurations">
<ProjectConfiguration Include="Debug|Win32">
<Configuration>Debug</Configuration>
<Platform>Win32</Platform>
</ProjectConfiguration>
<ProjectConfiguration Include="Debug|x64">
<Configuration>Debug</Configuration>
<Platform>x64</Platform>
</ProjectConfiguration>
<ProjectConfiguration Include="Release|Win32">
<Configuration>Release</Configuration>
<Platform>Win32</Platform>
</ProjectConfiguration>
<ProjectConfiguration Include="Release|x64">
<Configuration>Release</Configuration>
<Platform>x64</Platform>
</ProjectConfiguration>
</ItemGroup>
<ItemGroup>
<ClCompile Include="..\..\..\lib\fse.c" />
<ClCompile Include="..\..\..\lib\huff0.c" />
<ClCompile Include="..\..\..\lib\zbuff.c" />
<ClCompile Include="..\..\..\lib\zstd_compress.c" />
<ClCompile Include="..\..\..\lib\zstd_decompress.c" />
<ClInclude Include="..\..\..\lib\bitstream.h" />
<ClInclude Include="..\..\..\lib\error_private.h" />
<ClInclude Include="..\..\..\lib\error_public.h" />
<ClInclude Include="..\..\..\lib\fse.h" />
<ClInclude Include="..\..\..\lib\fse_static.h" />
<ClInclude Include="..\..\..\lib\huff0.h" />
<ClInclude Include="..\..\..\lib\huff0_static.h" />
<ClInclude Include="..\..\..\lib\mem.h" />
<ClInclude Include="..\..\..\lib\zbuff.h" />
<ClInclude Include="..\..\..\lib\zbuff_static.h" />
<ClInclude Include="..\..\..\lib\zstd.h" />
<ClInclude Include="..\..\..\lib\zstd_internal.h" />
<ClInclude Include="..\..\..\lib\zstd_static.h" />
<ClInclude Include="resource.h" />
</ItemGroup>
<ItemGroup>
<ResourceCompile Include="zstdlib.rc" />
</ItemGroup>
<PropertyGroup Label="Globals">
<ProjectGuid>{8BFD8150-94D5-4BF9-8A50-7BD9929A0850}</ProjectGuid>
<Keyword>Win32Proj</Keyword>
<RootNamespace>zstdlib</RootNamespace>
<OutDir>$(SolutionDir)bin\$(Platform)\$(Configuration)\</OutDir>
</PropertyGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.Default.props" />
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'" Label="Configuration">
<ConfigurationType>DynamicLibrary</ConfigurationType>
<UseDebugLibraries>true</UseDebugLibraries>
<CharacterSet>Unicode</CharacterSet>
<PlatformToolset>v120</PlatformToolset>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'" Label="Configuration">
<ConfigurationType>DynamicLibrary</ConfigurationType>
<UseDebugLibraries>true</UseDebugLibraries>
<PlatformToolset>v120</PlatformToolset>
<CharacterSet>Unicode</CharacterSet>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'" Label="Configuration">
<ConfigurationType>DynamicLibrary</ConfigurationType>
<UseDebugLibraries>false</UseDebugLibraries>
<WholeProgramOptimization>true</WholeProgramOptimization>
<PlatformToolset>v120</PlatformToolset>
<CharacterSet>Unicode</CharacterSet>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'" Label="Configuration">
<ConfigurationType>DynamicLibrary</ConfigurationType>
<UseDebugLibraries>false</UseDebugLibraries>
<WholeProgramOptimization>true</WholeProgramOptimization>
<PlatformToolset>v120</PlatformToolset>
<CharacterSet>Unicode</CharacterSet>
</PropertyGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.props" />
<ImportGroup Label="ExtensionSettings">
</ImportGroup>
<ImportGroup Label="PropertySheets" Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
</ImportGroup>
<ImportGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'" Label="PropertySheets">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
</ImportGroup>
<ImportGroup Label="PropertySheets" Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
</ImportGroup>
<ImportGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'" Label="PropertySheets">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
</ImportGroup>
<PropertyGroup Label="UserMacros" />
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
<LinkIncremental>true</LinkIncremental>
<TargetName>zstdlib_x86</TargetName>
<IntDir>$(Platform)\$(Configuration)\</IntDir>
<IncludePath>$(SolutionDir)..\..\programs\legacy;$(SolutionDir)..\..\lib\legacy;$(SolutionDir)..\..\lib;$(VCInstallDir)include;$(VCInstallDir)atlmfc\include;$(WindowsSDK_IncludePath);</IncludePath>
<RunCodeAnalysis>true</RunCodeAnalysis>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">
<LinkIncremental>true</LinkIncremental>
<TargetName>zstdlib_x64</TargetName>
<IntDir>$(Platform)\$(Configuration)\</IntDir>
<IncludePath>$(SolutionDir)..\..\programs\legacy;$(SolutionDir)..\..\lib\legacy;$(SolutionDir)..\..\lib;$(VCInstallDir)include;$(VCInstallDir)atlmfc\include;$(WindowsSDK_IncludePath);</IncludePath>
<RunCodeAnalysis>true</RunCodeAnalysis>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
<LinkIncremental>false</LinkIncremental>
<TargetName>zstdlib_x86</TargetName>
<IntDir>$(Platform)\$(Configuration)\</IntDir>
<IncludePath>$(SolutionDir)..\..\programs\legacy;$(SolutionDir)..\..\lib\legacy;$(SolutionDir)..\..\lib;$(VCInstallDir)include;$(VCInstallDir)atlmfc\include;$(WindowsSDK_IncludePath);</IncludePath>
<RunCodeAnalysis>false</RunCodeAnalysis>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'">
<LinkIncremental>false</LinkIncremental>
<TargetName>zstdlib_x64</TargetName>
<IntDir>$(Platform)\$(Configuration)\</IntDir>
<IncludePath>$(SolutionDir)..\..\programs\legacy;$(SolutionDir)..\..\lib\legacy;$(SolutionDir)..\..\lib;$(VCInstallDir)include;$(VCInstallDir)atlmfc\include;$(WindowsSDK_IncludePath);</IncludePath>
<RunCodeAnalysis>false</RunCodeAnalysis>
</PropertyGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
<ClCompile>
<PrecompiledHeader>
</PrecompiledHeader>
<WarningLevel>Level4</WarningLevel>
<Optimization>Disabled</Optimization>
<PreprocessorDefinitions>ZSTD_DLL_EXPORT=1;ZSTD_HEAPMODE=0;ZSTD_LEGACY_SUPPORT=0;WIN32;_DEBUG;_CONSOLE;_CRT_SECURE_NO_WARNINGS;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<MinimalRebuild>true</MinimalRebuild>
<BasicRuntimeChecks>EnableFastChecks</BasicRuntimeChecks>
<RuntimeLibrary>MultiThreadedDebugDLL</RuntimeLibrary>
<DebugInformationFormat>EditAndContinue</DebugInformationFormat>
<TreatWarningAsError>true</TreatWarningAsError>
<EnablePREfast>true</EnablePREfast>
<AdditionalOptions>/analyze:stacksize25000 %(AdditionalOptions)</AdditionalOptions>
</ClCompile>
<Link>
<SubSystem>Console</SubSystem>
<GenerateDebugInformation>true</GenerateDebugInformation>
<TargetMachine>MachineX86</TargetMachine>
</Link>
</ItemDefinitionGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">
<ClCompile>
<PrecompiledHeader>
</PrecompiledHeader>
<WarningLevel>Level4</WarningLevel>
<Optimization>Disabled</Optimization>
<PreprocessorDefinitions>ZSTD_DLL_EXPORT=1;ZSTD_HEAPMODE=0;ZSTD_LEGACY_SUPPORT=0;WIN32;_DEBUG;_CONSOLE;_CRT_SECURE_NO_WARNINGS;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<TreatWarningAsError>false</TreatWarningAsError>
<BasicRuntimeChecks>EnableFastChecks</BasicRuntimeChecks>
<RuntimeLibrary>MultiThreadedDebugDLL</RuntimeLibrary>
<DebugInformationFormat>ProgramDatabase</DebugInformationFormat>
<EnablePREfast>true</EnablePREfast>
<AdditionalOptions>/analyze:stacksize25000 %(AdditionalOptions)</AdditionalOptions>
</ClCompile>
<Link>
<SubSystem>Console</SubSystem>
<GenerateDebugInformation>true</GenerateDebugInformation>
</Link>
</ItemDefinitionGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
<ClCompile>
<WarningLevel>Level4</WarningLevel>
<PrecompiledHeader>
</PrecompiledHeader>
<Optimization>MaxSpeed</Optimization>
<FunctionLevelLinking>true</FunctionLevelLinking>
<IntrinsicFunctions>true</IntrinsicFunctions>
<PreprocessorDefinitions>ZSTD_DLL_EXPORT=1;ZSTD_HEAPMODE=0;ZSTD_LEGACY_SUPPORT=0;WIN32;NDEBUG;_CONSOLE;_CRT_SECURE_NO_WARNINGS;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<EnablePREfast>true</EnablePREfast>
<AdditionalOptions>/analyze:stacksize25000 %(AdditionalOptions)</AdditionalOptions>
<RuntimeLibrary>MultiThreadedDLL</RuntimeLibrary>
<DebugInformationFormat>ProgramDatabase</DebugInformationFormat>
<AssemblerOutput>All</AssemblerOutput>
</ClCompile>
<Link>
<SubSystem>Console</SubSystem>
<GenerateDebugInformation>true</GenerateDebugInformation>
<EnableCOMDATFolding>true</EnableCOMDATFolding>
<OptimizeReferences>true</OptimizeReferences>
<TargetMachine>MachineX86</TargetMachine>
</Link>
</ItemDefinitionGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'">
<ClCompile>
<WarningLevel>Level4</WarningLevel>
<PrecompiledHeader>
</PrecompiledHeader>
<Optimization>MaxSpeed</Optimization>
<FunctionLevelLinking>true</FunctionLevelLinking>
<IntrinsicFunctions>true</IntrinsicFunctions>
<PreprocessorDefinitions>ZSTD_DLL_EXPORT=1;ZSTD_HEAPMODE=0;ZSTD_LEGACY_SUPPORT=0;WIN32;NDEBUG;_CONSOLE;_CRT_SECURE_NO_WARNINGS;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<TreatWarningAsError>false</TreatWarningAsError>
<EnablePREfast>false</EnablePREfast>
<RuntimeLibrary>MultiThreadedDLL</RuntimeLibrary>
<DebugInformationFormat>ProgramDatabase</DebugInformationFormat>
<WholeProgramOptimization>true</WholeProgramOptimization>
<OmitFramePointers>true</OmitFramePointers>
<AssemblerOutput>All</AssemblerOutput>
</ClCompile>
<Link>
<SubSystem>Console</SubSystem>
<GenerateDebugInformation>true</GenerateDebugInformation>
<EnableCOMDATFolding>true</EnableCOMDATFolding>
<OptimizeReferences>true</OptimizeReferences>
</Link>
</ItemDefinitionGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.targets" />
<ImportGroup Label="ExtensionTargets">
</ImportGroup>
<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Build" ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup Label="ProjectConfigurations">
<ProjectConfiguration Include="Debug|Win32">
<Configuration>Debug</Configuration>
<Platform>Win32</Platform>
</ProjectConfiguration>
<ProjectConfiguration Include="Debug|x64">
<Configuration>Debug</Configuration>
<Platform>x64</Platform>
</ProjectConfiguration>
<ProjectConfiguration Include="Release|Win32">
<Configuration>Release</Configuration>
<Platform>Win32</Platform>
</ProjectConfiguration>
<ProjectConfiguration Include="Release|x64">
<Configuration>Release</Configuration>
<Platform>x64</Platform>
</ProjectConfiguration>
</ItemGroup>
<ItemGroup>
<ClCompile Include="..\..\..\lib\common\entropy_common.c" />
<ClCompile Include="..\..\..\lib\common\zstd_common.c" />
<ClCompile Include="..\..\..\lib\common\fse_decompress.c" />
<ClCompile Include="..\..\..\lib\compress\fse_compress.c" />
<ClCompile Include="..\..\..\lib\compress\huf_compress.c" />
<ClCompile Include="..\..\..\lib\compress\zbuff_compress.c" />
<ClCompile Include="..\..\..\lib\compress\zstd_compress.c" />
<ClCompile Include="..\..\..\lib\decompress\huf_decompress.c" />
<ClCompile Include="..\..\..\lib\decompress\zbuff_decompress.c" />
<ClCompile Include="..\..\..\lib\decompress\zstd_decompress.c" />
<ClCompile Include="..\..\..\lib\dictBuilder\divsufsort.c" />
<ClCompile Include="..\..\..\lib\dictBuilder\zdict.c" />
<ClInclude Include="..\..\..\lib\common\bitstream.h" />
<ClInclude Include="..\..\..\lib\common\error_private.h" />
<ClInclude Include="..\..\..\lib\common\error_public.h" />
<ClInclude Include="..\..\..\lib\common\mem.h" />
<ClInclude Include="..\..\..\lib\common\fse.h" />
<ClInclude Include="..\..\..\lib\common\fse_static.h" />
<ClInclude Include="..\..\..\lib\common\huf.h" />
<ClInclude Include="..\..\..\lib\common\huf_static.h" />
<ClInclude Include="..\..\..\lib\common\zbuff.h" />
<ClInclude Include="..\..\..\lib\common\zbuff_static.h" />
<ClInclude Include="..\..\..\lib\common\zstd.h" />
<ClInclude Include="..\..\..\lib\common\zstd_internal.h" />
<ClInclude Include="..\..\..\lib\common\zstd_static.h" />
<ClInclude Include="..\..\..\lib\compress\zstd_opt.h" />
<ClInclude Include="..\..\..\programs\util.h" />
<ClInclude Include="resource.h" />
</ItemGroup>
<ItemGroup>
<ResourceCompile Include="zstdlib.rc" />
</ItemGroup>
<PropertyGroup Label="Globals">
<ProjectGuid>{8BFD8150-94D5-4BF9-8A50-7BD9929A0850}</ProjectGuid>
<Keyword>Win32Proj</Keyword>
<RootNamespace>zstdlib</RootNamespace>
<OutDir>$(SolutionDir)bin\$(Platform)\$(Configuration)\</OutDir>
</PropertyGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.Default.props" />
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'" Label="Configuration">
<ConfigurationType>DynamicLibrary</ConfigurationType>
<UseDebugLibraries>true</UseDebugLibraries>
<PlatformToolset>v110</PlatformToolset>
<CharacterSet>MultiByte</CharacterSet>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'" Label="Configuration">
<ConfigurationType>DynamicLibrary</ConfigurationType>
<UseDebugLibraries>true</UseDebugLibraries>
<PlatformToolset>v110</PlatformToolset>
<CharacterSet>MultiByte</CharacterSet>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'" Label="Configuration">
<ConfigurationType>DynamicLibrary</ConfigurationType>
<UseDebugLibraries>false</UseDebugLibraries>
<WholeProgramOptimization>true</WholeProgramOptimization>
<PlatformToolset>v110</PlatformToolset>
<CharacterSet>MultiByte</CharacterSet>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'" Label="Configuration">
<ConfigurationType>DynamicLibrary</ConfigurationType>
<UseDebugLibraries>false</UseDebugLibraries>
<WholeProgramOptimization>true</WholeProgramOptimization>
<PlatformToolset>v110</PlatformToolset>
<CharacterSet>MultiByte</CharacterSet>
</PropertyGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.props" />
<ImportGroup Label="ExtensionSettings">
</ImportGroup>
<ImportGroup Label="PropertySheets" Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
</ImportGroup>
<ImportGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'" Label="PropertySheets">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
</ImportGroup>
<ImportGroup Label="PropertySheets" Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
</ImportGroup>
<ImportGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'" Label="PropertySheets">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
</ImportGroup>
<PropertyGroup Label="UserMacros" />
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
<LinkIncremental>true</LinkIncremental>
<TargetName>zstdlib_x86</TargetName>
<IntDir>$(Platform)\$(Configuration)\</IntDir>
<IncludePath>$(WindowsSdkDir)\include;$(UniversalCRT_IncludePath);$(SolutionDir)..\..\programs\legacy;$(SolutionDir)..\..\lib\legacy;$(SolutionDir)..\..\lib\common;$(VCInstallDir)include;$(VCInstallDir)atlmfc\include;$(WindowsSDK_IncludePath);</IncludePath>
<RunCodeAnalysis>false</RunCodeAnalysis>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">
<LinkIncremental>true</LinkIncremental>
<TargetName>zstdlib_x64</TargetName>
<IntDir>$(Platform)\$(Configuration)\</IntDir>
<IncludePath>$(WindowsSdkDir)\include;$(UniversalCRT_IncludePath);$(SolutionDir)..\..\programs\legacy;$(SolutionDir)..\..\lib\legacy;$(SolutionDir)..\..\lib\common;$(VCInstallDir)include;$(VCInstallDir)atlmfc\include;$(WindowsSDK_IncludePath);</IncludePath>
<RunCodeAnalysis>false</RunCodeAnalysis>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
<LinkIncremental>false</LinkIncremental>
<TargetName>zstdlib_x86</TargetName>
<IntDir>$(Platform)\$(Configuration)\</IntDir>
<IncludePath>$(WindowsSdkDir)\include;$(UniversalCRT_IncludePath);$(SolutionDir)..\..\programs\legacy;$(SolutionDir)..\..\lib\legacy;$(SolutionDir)..\..\lib\common;$(VCInstallDir)include;$(VCInstallDir)atlmfc\include;$(WindowsSDK_IncludePath);</IncludePath>
<RunCodeAnalysis>false</RunCodeAnalysis>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'">
<LinkIncremental>false</LinkIncremental>
<TargetName>zstdlib_x64</TargetName>
<IntDir>$(Platform)\$(Configuration)\</IntDir>
<IncludePath>$(WindowsSdkDir)\include;$(UniversalCRT_IncludePath);$(SolutionDir)..\..\programs\legacy;$(SolutionDir)..\..\lib\legacy;$(SolutionDir)..\..\lib\common;$(VCInstallDir)include;$(VCInstallDir)atlmfc\include;$(WindowsSDK_IncludePath);</IncludePath>
<RunCodeAnalysis>false</RunCodeAnalysis>
</PropertyGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
<ClCompile>
<PrecompiledHeader>
</PrecompiledHeader>
<WarningLevel>Level4</WarningLevel>
<Optimization>Disabled</Optimization>
<PreprocessorDefinitions>ZSTD_DLL_EXPORT=1;ZSTD_HEAPMODE=0;ZSTD_LEGACY_SUPPORT=0;WIN32;_DEBUG;_CONSOLE;_CRT_SECURE_NO_WARNINGS;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<MinimalRebuild>true</MinimalRebuild>
<BasicRuntimeChecks>EnableFastChecks</BasicRuntimeChecks>
<RuntimeLibrary>MultiThreadedDebugDLL</RuntimeLibrary>
<DebugInformationFormat>EditAndContinue</DebugInformationFormat>
<TreatWarningAsError>true</TreatWarningAsError>
<EnablePREfast>false</EnablePREfast>
</ClCompile>
<Link>
<SubSystem>Console</SubSystem>
<GenerateDebugInformation>true</GenerateDebugInformation>
<TargetMachine>MachineX86</TargetMachine>
</Link>
</ItemDefinitionGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">
<ClCompile>
<PrecompiledHeader>
</PrecompiledHeader>
<WarningLevel>Level4</WarningLevel>
<Optimization>Disabled</Optimization>
<PreprocessorDefinitions>ZSTD_DLL_EXPORT=1;ZSTD_HEAPMODE=0;ZSTD_LEGACY_SUPPORT=0;WIN32;_DEBUG;_CONSOLE;_CRT_SECURE_NO_WARNINGS;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<TreatWarningAsError>false</TreatWarningAsError>
<BasicRuntimeChecks>EnableFastChecks</BasicRuntimeChecks>
<RuntimeLibrary>MultiThreadedDebugDLL</RuntimeLibrary>
<DebugInformationFormat>ProgramDatabase</DebugInformationFormat>
<EnablePREfast>false</EnablePREfast>
</ClCompile>
<Link>
<SubSystem>Console</SubSystem>
<GenerateDebugInformation>true</GenerateDebugInformation>
</Link>
</ItemDefinitionGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
<ClCompile>
<WarningLevel>Level4</WarningLevel>
<PrecompiledHeader>
</PrecompiledHeader>
<Optimization>MaxSpeed</Optimization>
<FunctionLevelLinking>true</FunctionLevelLinking>
<IntrinsicFunctions>true</IntrinsicFunctions>
<PreprocessorDefinitions>ZSTD_DLL_EXPORT=1;ZSTD_HEAPMODE=0;ZSTD_LEGACY_SUPPORT=0;WIN32;NDEBUG;_CONSOLE;_CRT_SECURE_NO_WARNINGS;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<EnablePREfast>false</EnablePREfast>
<RuntimeLibrary>MultiThreadedDLL</RuntimeLibrary>
<DebugInformationFormat>ProgramDatabase</DebugInformationFormat>
<AssemblerOutput>All</AssemblerOutput>
</ClCompile>
<Link>
<SubSystem>Console</SubSystem>
<GenerateDebugInformation>true</GenerateDebugInformation>
<EnableCOMDATFolding>true</EnableCOMDATFolding>
<OptimizeReferences>true</OptimizeReferences>
<TargetMachine>MachineX86</TargetMachine>
</Link>
</ItemDefinitionGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'">
<ClCompile>
<WarningLevel>Level4</WarningLevel>
<PrecompiledHeader>
</PrecompiledHeader>
<Optimization>MaxSpeed</Optimization>
<FunctionLevelLinking>true</FunctionLevelLinking>
<IntrinsicFunctions>true</IntrinsicFunctions>
<PreprocessorDefinitions>ZSTD_DLL_EXPORT=1;ZSTD_HEAPMODE=0;ZSTD_LEGACY_SUPPORT=0;WIN32;NDEBUG;_CONSOLE;_CRT_SECURE_NO_WARNINGS;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<TreatWarningAsError>false</TreatWarningAsError>
<EnablePREfast>false</EnablePREfast>
<RuntimeLibrary>MultiThreadedDLL</RuntimeLibrary>
<DebugInformationFormat>ProgramDatabase</DebugInformationFormat>
<WholeProgramOptimization>true</WholeProgramOptimization>
<OmitFramePointers>true</OmitFramePointers>
<AssemblerOutput>All</AssemblerOutput>
</ClCompile>
<Link>
<SubSystem>Console</SubSystem>
<GenerateDebugInformation>true</GenerateDebugInformation>
<EnableCOMDATFolding>true</EnableCOMDATFolding>
<OptimizeReferences>true</OptimizeReferences>
</Link>
</ItemDefinitionGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.targets" />
<ImportGroup Label="ExtensionTargets">
</ImportGroup>
</Project>
+109 -82
View File
@@ -1,83 +1,110 @@
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup>
<Filter Include="Source Files">
<UniqueIdentifier>{4FC737F1-C7A5-4376-A066-2A32D752A2FF}</UniqueIdentifier>
<Extensions>cpp;c;cc;cxx;def;odl;idl;hpj;bat;asm;asmx</Extensions>
</Filter>
<Filter Include="Header Files">
<UniqueIdentifier>{93995380-89BD-4b04-88EB-625FBE52EBFB}</UniqueIdentifier>
<Extensions>h;hh;hpp;hxx;hm;inl;inc;xsd</Extensions>
</Filter>
<Filter Include="Resource Files">
<UniqueIdentifier>{67DA6AB6-F800-4c08-8B7A-83BB121AAD01}</UniqueIdentifier>
<Extensions>rc;ico;cur;bmp;dlg;rc2;rct;bin;rgs;gif;jpg;jpeg;jpe;resx;tiff;tif;png;wav;mfcribbon-ms</Extensions>
</Filter>
</ItemGroup>
<ItemGroup>
<ClCompile Include="..\..\..\lib\fse.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\huff0.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\zstd_compress.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\zstd_decompress.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\zbuff.c">
<Filter>Source Files</Filter>
</ClCompile>
</ItemGroup>
<ItemGroup>
<ClInclude Include="..\..\..\lib\fse.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\fse_static.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\zstd.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\zstd_static.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\huff0.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\huff0_static.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="resource.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\bitstream.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\zstd_internal.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\mem.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\error_private.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\error_public.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\zbuff.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\zbuff_static.h">
<Filter>Header Files</Filter>
</ClInclude>
</ItemGroup>
<ItemGroup>
<ResourceCompile Include="zstdlib.rc">
<Filter>Resource Files</Filter>
</ResourceCompile>
</ItemGroup>
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup>
<Filter Include="Source Files">
<UniqueIdentifier>{4FC737F1-C7A5-4376-A066-2A32D752A2FF}</UniqueIdentifier>
<Extensions>cpp;c;cc;cxx;def;odl;idl;hpj;bat;asm;asmx</Extensions>
</Filter>
<Filter Include="Header Files">
<UniqueIdentifier>{93995380-89BD-4b04-88EB-625FBE52EBFB}</UniqueIdentifier>
<Extensions>h;hh;hpp;hxx;hm;inl;inc;xsd</Extensions>
</Filter>
<Filter Include="Resource Files">
<UniqueIdentifier>{67DA6AB6-F800-4c08-8B7A-83BB121AAD01}</UniqueIdentifier>
<Extensions>rc;ico;cur;bmp;dlg;rc2;rct;bin;rgs;gif;jpg;jpeg;jpe;resx;tiff;tif;png;wav;mfcribbon-ms</Extensions>
</Filter>
</ItemGroup>
<ItemGroup>
<ClCompile Include="..\..\..\lib\common\zstd_common.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\compress\fse_compress.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\compress\huf_compress.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\compress\zbuff_compress.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\compress\zstd_compress.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\decompress\huf_decompress.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\decompress\zbuff_decompress.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\decompress\zstd_decompress.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\dictBuilder\divsufsort.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\dictBuilder\zdict.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\common\fse_decompress.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\lib\common\entropy_common.c">
<Filter>Source Files</Filter>
</ClCompile>
</ItemGroup>
<ItemGroup>
<ClInclude Include="resource.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\common\bitstream.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\common\error_private.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\common\error_public.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\common\mem.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\common\fse.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\common\fse_static.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\common\huf.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\common\huf_static.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\common\zbuff.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\common\zbuff_static.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\common\zstd.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\common\zstd_internal.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\common\zstd_static.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\lib\compress\zstd_opt.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\programs\util.h">
<Filter>Header Files</Filter>
</ClInclude>
</ItemGroup>
<ItemGroup>
<ResourceCompile Include="zstdlib.rc">
<Filter>Resource Files</Filter>
</ResourceCompile>
</ItemGroup>
</Project>
+26
View File
@@ -0,0 +1,26 @@
# Object files
*.o
*.ko
# Libraries
*.lib
*.a
# Shared objects (inc. Windows DLLs)
*.dll
*.so
*.so.*
*.dylib
# Executables
*.exe
*.out
*.app
# Default result files
_*
# Misc files
*.bat
*.zip
examples/example2.c
+54
View File
@@ -0,0 +1,54 @@
# Makefile for example of using zstd wrapper for zlib
#
# make - compiles statically and dynamically linked examples/example.c
# make test testdll - compiles and runs statically and dynamically linked examples/example.c
# make LOC=-DZWRAP_USE_ZSTD=1 - compiles statically and dynamically linked examples/example.c with zstd compression turned on
# Paths to static and dynamic zlib and zstd libraries
ifneq (,$(filter Windows%,$(OS)))
STATICLIB = ../../zlib/libz.a ../lib/libzstd.a
IMPLIB = ../../zlib/libz.dll.a ../lib/libzstd.a
else
STATICLIB = -static -lz ../lib/libzstd.a
IMPLIB = -lz ../lib/libzstd.a
endif
ZLIBWRAPPER_PATH = .
EXAMPLE_PATH = examples/
CC = gcc
CFLAGS = $(LOC) -I../lib/common -I$(ZLIBWRAPPER_PATH) -O3 -Wall -std=gnu89
LDFLAGS = $(LOC)
RM = rm -f
all: example example_d
test: example
./example
testdll: example_d
./example_d
.c.o:
$(CC) $(CFLAGS) -c -o $@ $<
example: $(EXAMPLE_PATH)/example.o $(ZLIBWRAPPER_PATH)/zstd_zlibwrapper.o
$(CC) $(LDFLAGS) -o $@ $(EXAMPLE_PATH)/example.o $(ZLIBWRAPPER_PATH)/zstd_zlibwrapper.o $(STATICLIB)
example_d: $(EXAMPLE_PATH)/example.o $(ZLIBWRAPPER_PATH)/zstd_zlibwrapper.o
$(CC) $(LDFLAGS) -o $@ $(EXAMPLE_PATH)/example.o $(ZLIBWRAPPER_PATH)/zstd_zlibwrapper.o $(IMPLIB)
$(EXAMPLE_PATH)/example.o: $(EXAMPLE_PATH)/example.c
$(CC) $(CFLAGS) -I. -c -o $@ $(EXAMPLE_PATH)/example.c
$(ZLIBWRAPPER_PATH)/zstd_zlibwrapper.o: $(ZLIBWRAPPER_PATH)/zstd_zlibwrapper.c $(ZLIBWRAPPER_PATH)/zstd_zlibwrapper.h
$(CC) $(CFLAGS) -I. -c -o $@ $(ZLIBWRAPPER_PATH)/zstd_zlibwrapper.c
clean:
-$(RM) $(ZLIBWRAPPER_PATH)/*.o
-$(RM) $(EXAMPLE_PATH)/*.o
-$(RM) *.o
-$(RM) *.exe
-$(RM) foo.gz
+105
View File
@@ -0,0 +1,105 @@
zstd wrapper for zlib
================================
The main objective of creating a zstd wrapper for [zlib](http://zlib.net/) is to allow a quick and smooth transition to zstd for projects already using zlib.
#### Required files
To build the zstd wrapper for zlib the following files are required:
- zlib.h
- a static or dynamic zlib library
- zlibWrapper/zstd_zlibwrapper.h
- zlibWrapper/zstd_zlibwrapper.c
- a static or dynamic zstd library
The first two files are required by all projects using zlib and they are not included with the zstd distribution.
The further files are supplied with the zstd distribution.
#### Embedding the zstd wrapper within your project
Let's assume that your project that uses zlib is compiled with:
```gcc project.o -lz```
To compile the zstd wrapper with your project you have to do the following:
- change all references with ```#include "zlib.h"``` to ```#include "zstd_zlibwrapper.h"```
- compile your project with zlib_wrapper.c and a static or dynamic zstd library
The linking should be changed to:
```gcc project.o zlib_wrapper.o -lz -lzstd```
#### Enabling zstd compression within your project
After embedding the zstd wrapper within your project the zstd library is turned off by default.
Your project should work as before with zlib. There are two options to enable zstd compression:
- compilation with ```-DZWRAP_USE_ZSTD=1``` (or using ```#define ZWRAP_USE_ZSTD 1``` before ```#include "zstd_zlibwrapper.h"```)
- using the ```void useZSTD(int turn_on)``` function (declared in ```#include "zstd_zlibwrapper.h"```)
There is no switch for zstd decompression because zlib and zstd streams are automatically detected and decompressed using a proper library.
#### Example
We have take the file ```test/example.c``` from [the zlib library distribution](http://zlib.net/) and copied it to [zlibWrapper/examples/example.c](examples/example.c).
After compilation and execution it shows the following results:
```
zlib version 1.2.8 = 0x1280, compile flags = 0x65
uncompress(): hello, hello!
gzread(): hello, hello!
gzgets() after gzseek: hello!
inflate(): hello, hello!
large_inflate(): OK
after inflateSync(): hello, hello!
inflate with dictionary: hello, hello!
```
Then we have changed ```#include "zlib.h"``` to ```#include "zstd_zlibwrapper.h"```, compiled the [example.c](examples/example.c) file
with ```-DZWRAP_USE_ZSTD=1``` and linked with additional ```zlib_wrapper.o -lzstd```.
We were forced to turn off the following functions: ```test_gzio```, ```test_flush```, ```test_sync``` which use currently unsupported features.
After running it shows the following results:
```
zlib version 1.2.8 = 0x1280, compile flags = 0x65
uncompress(): hello, hello!
inflate(): hello, hello!
large_inflate(): OK
inflate with dictionary: hello, hello!
```
The script used for compilation can be found at [zlibWrapper/Makefile](Makefile).
#### Compatibility issues
After enabling zstd compression not all native zlib functions are supported. When calling unsupported methods they print error message and return an error value.
Supported methods:
- deflateInit
- deflate (with exception of Z_FULL_FLUSH)
- deflateSetDictionary
- deflateEnd
- deflateBound
- inflateInit
- inflate
- inflateSetDictionary
- compress
- compress2
- compressBound
- uncompress
Ignored methods (they do nothing):
- deflateParams
Unsupported methods:
- gzip file access functions
- deflateCopy
- deflateReset
- deflateTune
- deflatePending
- deflatePrime
- deflateSetHeader
- inflateGetDictionary
- inflateCopy
- inflateReset
- inflateReset2
- inflatePrime
- inflateMark
- inflateGetHeader
- inflateBackInit
- inflateBack
- inflateBackEnd
+607
View File
@@ -0,0 +1,607 @@
/* the file contains minimal changes required to be compiled with zstd wrapper for zlib */
/* example.c -- usage example of the zlib compression library
* Copyright (C) 1995-2006, 2011 Jean-loup Gailly.
* For conditions of distribution and use, see copyright notice in zlib.h
*/
/* @(#) $Id$ */
#include "zstd_zlibwrapper.h"
#include <stdio.h>
#ifdef STDC
# include <string.h>
# include <stdlib.h>
#endif
#if defined(VMS) || defined(RISCOS)
# define TESTFILE "foo-gz"
#else
# define TESTFILE "foo.gz"
#endif
#define CHECK_ERR(err, msg) { \
if (err != Z_OK) { \
fprintf(stderr, "%s error: %d\n", msg, err); \
exit(1); \
} \
}
z_const char hello[] = "hello, hello!";
/* "hello world" would be more standard, but the repeated "hello"
* stresses the compression code better, sorry...
*/
const char dictionary[] = "hello";
uLong dictId; /* Adler32 value of the dictionary */
void test_deflate OF((Byte *compr, uLong comprLen));
void test_inflate OF((Byte *compr, uLong comprLen,
Byte *uncompr, uLong uncomprLen));
void test_large_deflate OF((Byte *compr, uLong comprLen,
Byte *uncompr, uLong uncomprLen));
void test_large_inflate OF((Byte *compr, uLong comprLen,
Byte *uncompr, uLong uncomprLen));
void test_flush OF((Byte *compr, uLong *comprLen));
void test_sync OF((Byte *compr, uLong comprLen,
Byte *uncompr, uLong uncomprLen));
void test_dict_deflate OF((Byte *compr, uLong comprLen));
void test_dict_inflate OF((Byte *compr, uLong comprLen,
Byte *uncompr, uLong uncomprLen));
int main OF((int argc, char *argv[]));
#ifdef Z_SOLO
void *myalloc OF((void *, unsigned, unsigned));
void myfree OF((void *, void *));
void *myalloc(q, n, m)
void *q;
unsigned n, m;
{
q = Z_NULL;
return calloc(n, m);
}
void myfree(void *q, void *p)
{
q = Z_NULL;
free(p);
}
static alloc_func zalloc = myalloc;
static free_func zfree = myfree;
#else /* !Z_SOLO */
static alloc_func zalloc = (alloc_func)0;
static free_func zfree = (free_func)0;
void test_compress OF((Byte *compr, uLong comprLen,
Byte *uncompr, uLong uncomprLen));
void test_gzio OF((const char *fname,
Byte *uncompr, uLong uncomprLen));
/* ===========================================================================
* Test compress() and uncompress()
*/
void test_compress(compr, comprLen, uncompr, uncomprLen)
Byte *compr, *uncompr;
uLong comprLen, uncomprLen;
{
int err;
uLong len = (uLong)strlen(hello)+1;
err = compress(compr, &comprLen, (const Bytef*)hello, len);
CHECK_ERR(err, "compress");
strcpy((char*)uncompr, "garbage");
err = uncompress(uncompr, &uncomprLen, compr, comprLen);
CHECK_ERR(err, "uncompress");
if (strcmp((char*)uncompr, hello)) {
fprintf(stderr, "bad uncompress\n");
exit(1);
} else {
printf("uncompress(): %s\n", (char *)uncompr);
}
}
/* ===========================================================================
* Test read/write of .gz files
*/
void test_gzio(fname, uncompr, uncomprLen)
const char *fname; /* compressed file name */
Byte *uncompr;
uLong uncomprLen;
{
#ifdef NO_GZCOMPRESS
fprintf(stderr, "NO_GZCOMPRESS -- gz* functions cannot compress\n");
#else
int err;
int len = (int)strlen(hello)+1;
gzFile file;
z_off_t pos;
file = gzopen(fname, "wb");
if (file == NULL) {
fprintf(stderr, "gzopen error\n");
exit(1);
}
gzputc(file, 'h');
if (gzputs(file, "ello") != 4) {
fprintf(stderr, "gzputs err: %s\n", gzerror(file, &err));
exit(1);
}
if (gzprintf(file, ", %s!", "hello") != 8) {
fprintf(stderr, "gzprintf err: %s\n", gzerror(file, &err));
exit(1);
}
gzseek(file, 1L, SEEK_CUR); /* add one zero byte */
gzclose(file);
file = gzopen(fname, "rb");
if (file == NULL) {
fprintf(stderr, "gzopen error\n");
exit(1);
}
strcpy((char*)uncompr, "garbage");
if (gzread(file, uncompr, (unsigned)uncomprLen) != len) {
fprintf(stderr, "gzread err: %s\n", gzerror(file, &err));
exit(1);
}
if (strcmp((char*)uncompr, hello)) {
fprintf(stderr, "bad gzread: %s\n", (char*)uncompr);
exit(1);
} else {
printf("gzread(): %s\n", (char*)uncompr);
}
pos = gzseek(file, -8L, SEEK_CUR);
if (pos != 6 || gztell(file) != pos) {
fprintf(stderr, "gzseek error, pos=%ld, gztell=%ld\n",
(long)pos, (long)gztell(file));
exit(1);
}
if (gzgetc(file) != ' ') {
fprintf(stderr, "gzgetc error\n");
exit(1);
}
if (gzungetc(' ', file) != ' ') {
fprintf(stderr, "gzungetc error\n");
exit(1);
}
gzgets(file, (char*)uncompr, (int)uncomprLen);
if (strlen((char*)uncompr) != 7) { /* " hello!" */
fprintf(stderr, "gzgets err after gzseek: %s\n", gzerror(file, &err));
exit(1);
}
if (strcmp((char*)uncompr, hello + 6)) {
fprintf(stderr, "bad gzgets after gzseek\n");
exit(1);
} else {
printf("gzgets() after gzseek: %s\n", (char*)uncompr);
}
gzclose(file);
#endif
}
#endif /* Z_SOLO */
/* ===========================================================================
* Test deflate() with small buffers
*/
void test_deflate(compr, comprLen)
Byte *compr;
uLong comprLen;
{
z_stream c_stream; /* compression stream */
int err;
uLong len = (uLong)strlen(hello)+1;
c_stream.zalloc = zalloc;
c_stream.zfree = zfree;
c_stream.opaque = (voidpf)0;
err = deflateInit(&c_stream, Z_DEFAULT_COMPRESSION);
CHECK_ERR(err, "deflateInit");
c_stream.next_in = (z_const unsigned char *)hello;
c_stream.next_out = compr;
while (c_stream.total_in != len && c_stream.total_out < comprLen) {
c_stream.avail_in = c_stream.avail_out = 1; /* force small buffers */
err = deflate(&c_stream, Z_NO_FLUSH);
CHECK_ERR(err, "deflate");
}
/* Finish the stream, still forcing small buffers: */
for (;;) {
c_stream.avail_out = 1;
err = deflate(&c_stream, Z_FINISH);
if (err == Z_STREAM_END) break;
CHECK_ERR(err, "deflate");
}
err = deflateEnd(&c_stream);
CHECK_ERR(err, "deflateEnd");
}
/* ===========================================================================
* Test inflate() with small buffers
*/
void test_inflate(compr, comprLen, uncompr, uncomprLen)
Byte *compr, *uncompr;
uLong comprLen, uncomprLen;
{
int err;
z_stream d_stream; /* decompression stream */
strcpy((char*)uncompr, "garbage");
d_stream.zalloc = zalloc;
d_stream.zfree = zfree;
d_stream.opaque = (voidpf)0;
d_stream.next_in = compr;
d_stream.avail_in = 0;
d_stream.next_out = uncompr;
err = inflateInit(&d_stream);
CHECK_ERR(err, "inflateInit");
while (d_stream.total_out < uncomprLen && d_stream.total_in < comprLen) {
d_stream.avail_in = d_stream.avail_out = 1; /* force small buffers */
err = inflate(&d_stream, Z_NO_FLUSH);
if (err == Z_STREAM_END) break;
CHECK_ERR(err, "inflate");
}
err = inflateEnd(&d_stream);
CHECK_ERR(err, "inflateEnd");
if (strcmp((char*)uncompr, hello)) {
fprintf(stderr, "bad inflate\n");
exit(1);
} else {
printf("inflate(): %s\n", (char *)uncompr);
}
}
/* ===========================================================================
* Test deflate() with large buffers and dynamic change of compression level
*/
void test_large_deflate(compr, comprLen, uncompr, uncomprLen)
Byte *compr, *uncompr;
uLong comprLen, uncomprLen;
{
z_stream c_stream; /* compression stream */
int err;
c_stream.zalloc = zalloc;
c_stream.zfree = zfree;
c_stream.opaque = (voidpf)0;
err = deflateInit(&c_stream, Z_BEST_SPEED);
CHECK_ERR(err, "deflateInit");
c_stream.next_out = compr;
c_stream.avail_out = (uInt)comprLen;
/* At this point, uncompr is still mostly zeroes, so it should compress
* very well:
*/
c_stream.next_in = uncompr;
c_stream.avail_in = (uInt)uncomprLen;
err = deflate(&c_stream, Z_NO_FLUSH);
CHECK_ERR(err, "deflate");
if (c_stream.avail_in != 0) {
fprintf(stderr, "deflate not greedy\n");
exit(1);
}
/* Feed in already compressed data and switch to no compression: */
deflateParams(&c_stream, Z_NO_COMPRESSION, Z_DEFAULT_STRATEGY);
c_stream.next_in = compr;
c_stream.avail_in = (uInt)comprLen/2;
err = deflate(&c_stream, Z_NO_FLUSH);
CHECK_ERR(err, "deflate");
/* Switch back to compressing mode: */
deflateParams(&c_stream, Z_BEST_COMPRESSION, Z_FILTERED);
c_stream.next_in = uncompr;
c_stream.avail_in = (uInt)uncomprLen;
err = deflate(&c_stream, Z_NO_FLUSH);
CHECK_ERR(err, "deflate");
err = deflate(&c_stream, Z_FINISH);
if (err != Z_STREAM_END) {
fprintf(stderr, "deflate should report Z_STREAM_END\n");
exit(1);
}
err = deflateEnd(&c_stream);
CHECK_ERR(err, "deflateEnd");
}
/* ===========================================================================
* Test inflate() with large buffers
*/
void test_large_inflate(compr, comprLen, uncompr, uncomprLen)
Byte *compr, *uncompr;
uLong comprLen, uncomprLen;
{
int err;
z_stream d_stream; /* decompression stream */
strcpy((char*)uncompr, "garbage");
d_stream.zalloc = zalloc;
d_stream.zfree = zfree;
d_stream.opaque = (voidpf)0;
d_stream.next_in = compr;
d_stream.avail_in = (uInt)comprLen;
err = inflateInit(&d_stream);
CHECK_ERR(err, "inflateInit");
for (;;) {
d_stream.next_out = uncompr; /* discard the output */
d_stream.avail_out = (uInt)uncomprLen;
err = inflate(&d_stream, Z_NO_FLUSH);
if (err == Z_STREAM_END) break;
CHECK_ERR(err, "large inflate");
}
err = inflateEnd(&d_stream);
CHECK_ERR(err, "inflateEnd");
if (d_stream.total_out != 2*uncomprLen + comprLen/2) {
fprintf(stderr, "bad large inflate: %ld\n", d_stream.total_out);
exit(1);
} else {
printf("large_inflate(): OK\n");
}
}
/* ===========================================================================
* Test deflate() with full flush
*/
void test_flush(compr, comprLen)
Byte *compr;
uLong *comprLen;
{
z_stream c_stream; /* compression stream */
int err;
uInt len = (uInt)strlen(hello)+1;
c_stream.zalloc = zalloc;
c_stream.zfree = zfree;
c_stream.opaque = (voidpf)0;
err = deflateInit(&c_stream, Z_DEFAULT_COMPRESSION);
CHECK_ERR(err, "deflateInit");
c_stream.next_in = (z_const unsigned char *)hello;
c_stream.next_out = compr;
c_stream.avail_in = 3;
c_stream.avail_out = (uInt)*comprLen;
err = deflate(&c_stream, Z_FULL_FLUSH);
CHECK_ERR(err, "deflate");
compr[3]++; /* force an error in first compressed block */
c_stream.avail_in = len - 3;
err = deflate(&c_stream, Z_FINISH);
if (err != Z_STREAM_END) {
CHECK_ERR(err, "deflate");
}
err = deflateEnd(&c_stream);
CHECK_ERR(err, "deflateEnd");
*comprLen = c_stream.total_out;
}
/* ===========================================================================
* Test inflateSync()
*/
void test_sync(compr, comprLen, uncompr, uncomprLen)
Byte *compr, *uncompr;
uLong comprLen, uncomprLen;
{
int err;
z_stream d_stream; /* decompression stream */
strcpy((char*)uncompr, "garbage");
d_stream.zalloc = zalloc;
d_stream.zfree = zfree;
d_stream.opaque = (voidpf)0;
d_stream.next_in = compr;
d_stream.avail_in = 2; /* just read the zlib header */
err = inflateInit(&d_stream);
CHECK_ERR(err, "inflateInit");
d_stream.next_out = uncompr;
d_stream.avail_out = (uInt)uncomprLen;
inflate(&d_stream, Z_NO_FLUSH);
CHECK_ERR(err, "inflate");
d_stream.avail_in = (uInt)comprLen-2; /* read all compressed data */
err = inflateSync(&d_stream); /* but skip the damaged part */
CHECK_ERR(err, "inflateSync");
err = inflate(&d_stream, Z_FINISH);
if (err != Z_DATA_ERROR) {
fprintf(stderr, "inflate should report DATA_ERROR\n");
/* Because of incorrect adler32 */
exit(1);
}
err = inflateEnd(&d_stream);
CHECK_ERR(err, "inflateEnd");
printf("after inflateSync(): hel%s\n", (char *)uncompr);
}
/* ===========================================================================
* Test deflate() with preset dictionary
*/
void test_dict_deflate(compr, comprLen)
Byte *compr;
uLong comprLen;
{
z_stream c_stream; /* compression stream */
int err;
c_stream.zalloc = zalloc;
c_stream.zfree = zfree;
c_stream.opaque = (voidpf)0;
err = deflateInit(&c_stream, Z_BEST_COMPRESSION);
CHECK_ERR(err, "deflateInit");
err = deflateSetDictionary(&c_stream,
(const Bytef*)dictionary, (int)sizeof(dictionary));
CHECK_ERR(err, "deflateSetDictionary");
dictId = c_stream.adler;
c_stream.next_out = compr;
c_stream.avail_out = (uInt)comprLen;
c_stream.next_in = (z_const unsigned char *)hello;
c_stream.avail_in = (uInt)strlen(hello)+1;
err = deflate(&c_stream, Z_FINISH);
if (err != Z_STREAM_END) {
fprintf(stderr, "deflate should report Z_STREAM_END\n");
exit(1);
}
err = deflateEnd(&c_stream);
CHECK_ERR(err, "deflateEnd");
}
/* ===========================================================================
* Test inflate() with a preset dictionary
*/
void test_dict_inflate(compr, comprLen, uncompr, uncomprLen)
Byte *compr, *uncompr;
uLong comprLen, uncomprLen;
{
int err;
z_stream d_stream; /* decompression stream */
strcpy((char*)uncompr, "garbage");
d_stream.zalloc = zalloc;
d_stream.zfree = zfree;
d_stream.opaque = (voidpf)0;
d_stream.next_in = compr;
d_stream.avail_in = (uInt)comprLen;
err = inflateInit(&d_stream);
CHECK_ERR(err, "inflateInit");
d_stream.next_out = uncompr;
d_stream.avail_out = (uInt)uncomprLen;
for (;;) {
err = inflate(&d_stream, Z_NO_FLUSH);
if (err == Z_STREAM_END) break;
if (err == Z_NEED_DICT) {
if (d_stream.adler != dictId) {
fprintf(stderr, "unexpected dictionary");
exit(1);
}
err = inflateSetDictionary(&d_stream, (const Bytef*)dictionary,
(int)sizeof(dictionary));
}
CHECK_ERR(err, "inflate with dict");
}
err = inflateEnd(&d_stream);
CHECK_ERR(err, "inflateEnd");
if (strcmp((char*)uncompr, hello)) {
fprintf(stderr, "bad inflate with dict\n");
exit(1);
} else {
printf("inflate with dictionary: %s\n", (char *)uncompr);
}
}
/* ===========================================================================
* Usage: example [output.gz [input.gz]]
*/
int main(argc, argv)
int argc;
char *argv[];
{
Byte *compr, *uncompr;
uLong comprLen = 10000*sizeof(int); /* don't overflow on MSDOS */
uLong uncomprLen = comprLen;
static const char* myVersion = ZLIB_VERSION;
if (zlibVersion()[0] != myVersion[0]) {
fprintf(stderr, "incompatible zlib version\n");
exit(1);
} else if (strcmp(zlibVersion(), ZLIB_VERSION) != 0) {
fprintf(stderr, "warning: different zlib version\n");
}
printf("zlib version %s = 0x%04x, compile flags = 0x%lx\n",
ZLIB_VERSION, ZLIB_VERNUM, zlibCompileFlags());
if (isUsingZSTD()) printf("zstd version %s\n", zstdVersion());
compr = (Byte*)calloc((uInt)comprLen, 1);
uncompr = (Byte*)calloc((uInt)uncomprLen, 1);
/* compr and uncompr are cleared to avoid reading uninitialized
* data and to ensure that uncompr compresses well.
*/
if (compr == Z_NULL || uncompr == Z_NULL) {
printf("out of memory\n");
exit(1);
}
#ifdef Z_SOLO
argc = strlen(argv[0]);
#else
test_compress(compr, comprLen, uncompr, uncomprLen);
if (!isUsingZSTD())
test_gzio((argc > 1 ? argv[1] : TESTFILE),
uncompr, uncomprLen);
#endif
test_deflate(compr, comprLen);
test_inflate(compr, comprLen, uncompr, uncomprLen);
test_large_deflate(compr, comprLen, uncompr, uncomprLen);
test_large_inflate(compr, comprLen, uncompr, uncomprLen);
if (!isUsingZSTD()) {
test_flush(compr, &comprLen);
test_sync(compr, comprLen, uncompr, uncomprLen);
}
comprLen = uncomprLen;
test_dict_deflate(compr, comprLen);
test_dict_inflate(compr, comprLen, uncompr, uncomprLen);
free(compr);
free(uncompr);
return 0;
}
+601
View File
@@ -0,0 +1,601 @@
/* example.c -- usage example of the zlib compression library
* Copyright (C) 1995-2006, 2011 Jean-loup Gailly.
* For conditions of distribution and use, see copyright notice in zlib.h
*/
/* @(#) $Id$ */
#include "zlib.h"
#include <stdio.h>
#ifdef STDC
# include <string.h>
# include <stdlib.h>
#endif
#if defined(VMS) || defined(RISCOS)
# define TESTFILE "foo-gz"
#else
# define TESTFILE "foo.gz"
#endif
#define CHECK_ERR(err, msg) { \
if (err != Z_OK) { \
fprintf(stderr, "%s error: %d\n", msg, err); \
exit(1); \
} \
}
z_const char hello[] = "hello, hello!";
/* "hello world" would be more standard, but the repeated "hello"
* stresses the compression code better, sorry...
*/
const char dictionary[] = "hello";
uLong dictId; /* Adler32 value of the dictionary */
void test_deflate OF((Byte *compr, uLong comprLen));
void test_inflate OF((Byte *compr, uLong comprLen,
Byte *uncompr, uLong uncomprLen));
void test_large_deflate OF((Byte *compr, uLong comprLen,
Byte *uncompr, uLong uncomprLen));
void test_large_inflate OF((Byte *compr, uLong comprLen,
Byte *uncompr, uLong uncomprLen));
void test_flush OF((Byte *compr, uLong *comprLen));
void test_sync OF((Byte *compr, uLong comprLen,
Byte *uncompr, uLong uncomprLen));
void test_dict_deflate OF((Byte *compr, uLong comprLen));
void test_dict_inflate OF((Byte *compr, uLong comprLen,
Byte *uncompr, uLong uncomprLen));
int main OF((int argc, char *argv[]));
#ifdef Z_SOLO
void *myalloc OF((void *, unsigned, unsigned));
void myfree OF((void *, void *));
void *myalloc(q, n, m)
void *q;
unsigned n, m;
{
q = Z_NULL;
return calloc(n, m);
}
void myfree(void *q, void *p)
{
q = Z_NULL;
free(p);
}
static alloc_func zalloc = myalloc;
static free_func zfree = myfree;
#else /* !Z_SOLO */
static alloc_func zalloc = (alloc_func)0;
static free_func zfree = (free_func)0;
void test_compress OF((Byte *compr, uLong comprLen,
Byte *uncompr, uLong uncomprLen));
void test_gzio OF((const char *fname,
Byte *uncompr, uLong uncomprLen));
/* ===========================================================================
* Test compress() and uncompress()
*/
void test_compress(compr, comprLen, uncompr, uncomprLen)
Byte *compr, *uncompr;
uLong comprLen, uncomprLen;
{
int err;
uLong len = (uLong)strlen(hello)+1;
err = compress(compr, &comprLen, (const Bytef*)hello, len);
CHECK_ERR(err, "compress");
strcpy((char*)uncompr, "garbage");
err = uncompress(uncompr, &uncomprLen, compr, comprLen);
CHECK_ERR(err, "uncompress");
if (strcmp((char*)uncompr, hello)) {
fprintf(stderr, "bad uncompress\n");
exit(1);
} else {
printf("uncompress(): %s\n", (char *)uncompr);
}
}
/* ===========================================================================
* Test read/write of .gz files
*/
void test_gzio(fname, uncompr, uncomprLen)
const char *fname; /* compressed file name */
Byte *uncompr;
uLong uncomprLen;
{
#ifdef NO_GZCOMPRESS
fprintf(stderr, "NO_GZCOMPRESS -- gz* functions cannot compress\n");
#else
int err;
int len = (int)strlen(hello)+1;
gzFile file;
z_off_t pos;
file = gzopen(fname, "wb");
if (file == NULL) {
fprintf(stderr, "gzopen error\n");
exit(1);
}
gzputc(file, 'h');
if (gzputs(file, "ello") != 4) {
fprintf(stderr, "gzputs err: %s\n", gzerror(file, &err));
exit(1);
}
if (gzprintf(file, ", %s!", "hello") != 8) {
fprintf(stderr, "gzprintf err: %s\n", gzerror(file, &err));
exit(1);
}
gzseek(file, 1L, SEEK_CUR); /* add one zero byte */
gzclose(file);
file = gzopen(fname, "rb");
if (file == NULL) {
fprintf(stderr, "gzopen error\n");
exit(1);
}
strcpy((char*)uncompr, "garbage");
if (gzread(file, uncompr, (unsigned)uncomprLen) != len) {
fprintf(stderr, "gzread err: %s\n", gzerror(file, &err));
exit(1);
}
if (strcmp((char*)uncompr, hello)) {
fprintf(stderr, "bad gzread: %s\n", (char*)uncompr);
exit(1);
} else {
printf("gzread(): %s\n", (char*)uncompr);
}
pos = gzseek(file, -8L, SEEK_CUR);
if (pos != 6 || gztell(file) != pos) {
fprintf(stderr, "gzseek error, pos=%ld, gztell=%ld\n",
(long)pos, (long)gztell(file));
exit(1);
}
if (gzgetc(file) != ' ') {
fprintf(stderr, "gzgetc error\n");
exit(1);
}
if (gzungetc(' ', file) != ' ') {
fprintf(stderr, "gzungetc error\n");
exit(1);
}
gzgets(file, (char*)uncompr, (int)uncomprLen);
if (strlen((char*)uncompr) != 7) { /* " hello!" */
fprintf(stderr, "gzgets err after gzseek: %s\n", gzerror(file, &err));
exit(1);
}
if (strcmp((char*)uncompr, hello + 6)) {
fprintf(stderr, "bad gzgets after gzseek\n");
exit(1);
} else {
printf("gzgets() after gzseek: %s\n", (char*)uncompr);
}
gzclose(file);
#endif
}
#endif /* Z_SOLO */
/* ===========================================================================
* Test deflate() with small buffers
*/
void test_deflate(compr, comprLen)
Byte *compr;
uLong comprLen;
{
z_stream c_stream; /* compression stream */
int err;
uLong len = (uLong)strlen(hello)+1;
c_stream.zalloc = zalloc;
c_stream.zfree = zfree;
c_stream.opaque = (voidpf)0;
err = deflateInit(&c_stream, Z_DEFAULT_COMPRESSION);
CHECK_ERR(err, "deflateInit");
c_stream.next_in = (z_const unsigned char *)hello;
c_stream.next_out = compr;
while (c_stream.total_in != len && c_stream.total_out < comprLen) {
c_stream.avail_in = c_stream.avail_out = 1; /* force small buffers */
err = deflate(&c_stream, Z_NO_FLUSH);
CHECK_ERR(err, "deflate");
}
/* Finish the stream, still forcing small buffers: */
for (;;) {
c_stream.avail_out = 1;
err = deflate(&c_stream, Z_FINISH);
if (err == Z_STREAM_END) break;
CHECK_ERR(err, "deflate");
}
err = deflateEnd(&c_stream);
CHECK_ERR(err, "deflateEnd");
}
/* ===========================================================================
* Test inflate() with small buffers
*/
void test_inflate(compr, comprLen, uncompr, uncomprLen)
Byte *compr, *uncompr;
uLong comprLen, uncomprLen;
{
int err;
z_stream d_stream; /* decompression stream */
strcpy((char*)uncompr, "garbage");
d_stream.zalloc = zalloc;
d_stream.zfree = zfree;
d_stream.opaque = (voidpf)0;
d_stream.next_in = compr;
d_stream.avail_in = 0;
d_stream.next_out = uncompr;
err = inflateInit(&d_stream);
CHECK_ERR(err, "inflateInit");
while (d_stream.total_out < uncomprLen && d_stream.total_in < comprLen) {
d_stream.avail_in = d_stream.avail_out = 1; /* force small buffers */
err = inflate(&d_stream, Z_NO_FLUSH);
if (err == Z_STREAM_END) break;
CHECK_ERR(err, "inflate");
}
err = inflateEnd(&d_stream);
CHECK_ERR(err, "inflateEnd");
if (strcmp((char*)uncompr, hello)) {
fprintf(stderr, "bad inflate\n");
exit(1);
} else {
printf("inflate(): %s\n", (char *)uncompr);
}
}
/* ===========================================================================
* Test deflate() with large buffers and dynamic change of compression level
*/
void test_large_deflate(compr, comprLen, uncompr, uncomprLen)
Byte *compr, *uncompr;
uLong comprLen, uncomprLen;
{
z_stream c_stream; /* compression stream */
int err;
c_stream.zalloc = zalloc;
c_stream.zfree = zfree;
c_stream.opaque = (voidpf)0;
err = deflateInit(&c_stream, Z_BEST_SPEED);
CHECK_ERR(err, "deflateInit");
c_stream.next_out = compr;
c_stream.avail_out = (uInt)comprLen;
/* At this point, uncompr is still mostly zeroes, so it should compress
* very well:
*/
c_stream.next_in = uncompr;
c_stream.avail_in = (uInt)uncomprLen;
err = deflate(&c_stream, Z_NO_FLUSH);
CHECK_ERR(err, "deflate");
if (c_stream.avail_in != 0) {
fprintf(stderr, "deflate not greedy\n");
exit(1);
}
/* Feed in already compressed data and switch to no compression: */
deflateParams(&c_stream, Z_NO_COMPRESSION, Z_DEFAULT_STRATEGY);
c_stream.next_in = compr;
c_stream.avail_in = (uInt)comprLen/2;
err = deflate(&c_stream, Z_NO_FLUSH);
CHECK_ERR(err, "deflate");
/* Switch back to compressing mode: */
deflateParams(&c_stream, Z_BEST_COMPRESSION, Z_FILTERED);
c_stream.next_in = uncompr;
c_stream.avail_in = (uInt)uncomprLen;
err = deflate(&c_stream, Z_NO_FLUSH);
CHECK_ERR(err, "deflate");
err = deflate(&c_stream, Z_FINISH);
if (err != Z_STREAM_END) {
fprintf(stderr, "deflate should report Z_STREAM_END\n");
exit(1);
}
err = deflateEnd(&c_stream);
CHECK_ERR(err, "deflateEnd");
}
/* ===========================================================================
* Test inflate() with large buffers
*/
void test_large_inflate(compr, comprLen, uncompr, uncomprLen)
Byte *compr, *uncompr;
uLong comprLen, uncomprLen;
{
int err;
z_stream d_stream; /* decompression stream */
strcpy((char*)uncompr, "garbage");
d_stream.zalloc = zalloc;
d_stream.zfree = zfree;
d_stream.opaque = (voidpf)0;
d_stream.next_in = compr;
d_stream.avail_in = (uInt)comprLen;
err = inflateInit(&d_stream);
CHECK_ERR(err, "inflateInit");
for (;;) {
d_stream.next_out = uncompr; /* discard the output */
d_stream.avail_out = (uInt)uncomprLen;
err = inflate(&d_stream, Z_NO_FLUSH);
if (err == Z_STREAM_END) break;
CHECK_ERR(err, "large inflate");
}
err = inflateEnd(&d_stream);
CHECK_ERR(err, "inflateEnd");
if (d_stream.total_out != 2*uncomprLen + comprLen/2) {
fprintf(stderr, "bad large inflate: %ld\n", d_stream.total_out);
exit(1);
} else {
printf("large_inflate(): OK\n");
}
}
/* ===========================================================================
* Test deflate() with full flush
*/
void test_flush(compr, comprLen)
Byte *compr;
uLong *comprLen;
{
z_stream c_stream; /* compression stream */
int err;
uInt len = (uInt)strlen(hello)+1;
c_stream.zalloc = zalloc;
c_stream.zfree = zfree;
c_stream.opaque = (voidpf)0;
err = deflateInit(&c_stream, Z_DEFAULT_COMPRESSION);
CHECK_ERR(err, "deflateInit");
c_stream.next_in = (z_const unsigned char *)hello;
c_stream.next_out = compr;
c_stream.avail_in = 3;
c_stream.avail_out = (uInt)*comprLen;
err = deflate(&c_stream, Z_FULL_FLUSH);
CHECK_ERR(err, "deflate");
compr[3]++; /* force an error in first compressed block */
c_stream.avail_in = len - 3;
err = deflate(&c_stream, Z_FINISH);
if (err != Z_STREAM_END) {
CHECK_ERR(err, "deflate");
}
err = deflateEnd(&c_stream);
CHECK_ERR(err, "deflateEnd");
*comprLen = c_stream.total_out;
}
/* ===========================================================================
* Test inflateSync()
*/
void test_sync(compr, comprLen, uncompr, uncomprLen)
Byte *compr, *uncompr;
uLong comprLen, uncomprLen;
{
int err;
z_stream d_stream; /* decompression stream */
strcpy((char*)uncompr, "garbage");
d_stream.zalloc = zalloc;
d_stream.zfree = zfree;
d_stream.opaque = (voidpf)0;
d_stream.next_in = compr;
d_stream.avail_in = 2; /* just read the zlib header */
err = inflateInit(&d_stream);
CHECK_ERR(err, "inflateInit");
d_stream.next_out = uncompr;
d_stream.avail_out = (uInt)uncomprLen;
inflate(&d_stream, Z_NO_FLUSH);
CHECK_ERR(err, "inflate");
d_stream.avail_in = (uInt)comprLen-2; /* read all compressed data */
err = inflateSync(&d_stream); /* but skip the damaged part */
CHECK_ERR(err, "inflateSync");
err = inflate(&d_stream, Z_FINISH);
if (err != Z_DATA_ERROR) {
fprintf(stderr, "inflate should report DATA_ERROR\n");
/* Because of incorrect adler32 */
exit(1);
}
err = inflateEnd(&d_stream);
CHECK_ERR(err, "inflateEnd");
printf("after inflateSync(): hel%s\n", (char *)uncompr);
}
/* ===========================================================================
* Test deflate() with preset dictionary
*/
void test_dict_deflate(compr, comprLen)
Byte *compr;
uLong comprLen;
{
z_stream c_stream; /* compression stream */
int err;
c_stream.zalloc = zalloc;
c_stream.zfree = zfree;
c_stream.opaque = (voidpf)0;
err = deflateInit(&c_stream, Z_BEST_COMPRESSION);
CHECK_ERR(err, "deflateInit");
err = deflateSetDictionary(&c_stream,
(const Bytef*)dictionary, (int)sizeof(dictionary));
CHECK_ERR(err, "deflateSetDictionary");
dictId = c_stream.adler;
c_stream.next_out = compr;
c_stream.avail_out = (uInt)comprLen;
c_stream.next_in = (z_const unsigned char *)hello;
c_stream.avail_in = (uInt)strlen(hello)+1;
err = deflate(&c_stream, Z_FINISH);
if (err != Z_STREAM_END) {
fprintf(stderr, "deflate should report Z_STREAM_END\n");
exit(1);
}
err = deflateEnd(&c_stream);
CHECK_ERR(err, "deflateEnd");
}
/* ===========================================================================
* Test inflate() with a preset dictionary
*/
void test_dict_inflate(compr, comprLen, uncompr, uncomprLen)
Byte *compr, *uncompr;
uLong comprLen, uncomprLen;
{
int err;
z_stream d_stream; /* decompression stream */
strcpy((char*)uncompr, "garbage");
d_stream.zalloc = zalloc;
d_stream.zfree = zfree;
d_stream.opaque = (voidpf)0;
d_stream.next_in = compr;
d_stream.avail_in = (uInt)comprLen;
err = inflateInit(&d_stream);
CHECK_ERR(err, "inflateInit");
d_stream.next_out = uncompr;
d_stream.avail_out = (uInt)uncomprLen;
for (;;) {
err = inflate(&d_stream, Z_NO_FLUSH);
if (err == Z_STREAM_END) break;
if (err == Z_NEED_DICT) {
if (d_stream.adler != dictId) {
fprintf(stderr, "unexpected dictionary");
exit(1);
}
err = inflateSetDictionary(&d_stream, (const Bytef*)dictionary,
(int)sizeof(dictionary));
}
CHECK_ERR(err, "inflate with dict");
}
err = inflateEnd(&d_stream);
CHECK_ERR(err, "inflateEnd");
if (strcmp((char*)uncompr, hello)) {
fprintf(stderr, "bad inflate with dict\n");
exit(1);
} else {
printf("inflate with dictionary: %s\n", (char *)uncompr);
}
}
/* ===========================================================================
* Usage: example [output.gz [input.gz]]
*/
int main(argc, argv)
int argc;
char *argv[];
{
Byte *compr, *uncompr;
uLong comprLen = 10000*sizeof(int); /* don't overflow on MSDOS */
uLong uncomprLen = comprLen;
static const char* myVersion = ZLIB_VERSION;
if (zlibVersion()[0] != myVersion[0]) {
fprintf(stderr, "incompatible zlib version\n");
exit(1);
} else if (strcmp(zlibVersion(), ZLIB_VERSION) != 0) {
fprintf(stderr, "warning: different zlib version\n");
}
printf("zlib version %s = 0x%04x, compile flags = 0x%lx\n",
ZLIB_VERSION, ZLIB_VERNUM, zlibCompileFlags());
compr = (Byte*)calloc((uInt)comprLen, 1);
uncompr = (Byte*)calloc((uInt)uncomprLen, 1);
/* compr and uncompr are cleared to avoid reading uninitialized
* data and to ensure that uncompr compresses well.
*/
if (compr == Z_NULL || uncompr == Z_NULL) {
printf("out of memory\n");
exit(1);
}
#ifdef Z_SOLO
argc = strlen(argv[0]);
#else
test_compress(compr, comprLen, uncompr, uncomprLen);
test_gzio((argc > 1 ? argv[1] : TESTFILE),
uncompr, uncomprLen);
#endif
test_deflate(compr, comprLen);
test_inflate(compr, comprLen, uncompr, uncomprLen);
test_large_deflate(compr, comprLen, uncompr, uncomprLen);
test_large_inflate(compr, comprLen, uncompr, uncomprLen);
test_flush(compr, &comprLen);
test_sync(compr, comprLen, uncompr, uncomprLen);
comprLen = uncomprLen;
test_dict_deflate(compr, comprLen);
test_dict_inflate(compr, comprLen, uncompr, uncomprLen);
free(compr);
free(uncompr);
return 0;
}
+905
View File
@@ -0,0 +1,905 @@
/*
zstd_zlibwrapper.c - zstd wrapper for zlib
Copyright (C) 2016, Przemyslaw Skibinski.
BSD 2-Clause License (http://www.opensource.org/licenses/bsd-license.php)
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the following disclaimer
in the documentation and/or other materials provided with the
distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
You can contact the author at :
- zstd source repository : https://github.com/Cyan4973/zstd
*/
#include <stdio.h> /* fprintf */
#include <stdlib.h> /* malloc */
#include <zlib.h>
#include "zstd_zlibwrapper.h"
#include "zstd.h"
#include "zstd_static.h" /* ZSTD_MAGICNUMBER */
#include "zstd_internal.h" /* MIN */
#include "zbuff.h"
#define Z_INFLATE_SYNC 8
#define ZWRAP_HEADERSIZE 4
#define ZSTD_FRAMEHEADERSIZE_MIN 5
#define LOG_WRAPPER(...) // printf(__VA_ARGS__)
#define FINISH_WITH_ERR(msg) { \
fprintf(stderr, "ERROR: %s\n", msg); \
return Z_MEM_ERROR; \
}
#define FINISH_WITH_NULL_ERR(msg) { \
fprintf(stderr, "ERROR: %s\n", msg); \
return NULL; \
}
#ifndef ZWRAP_USE_ZSTD
#define ZWRAP_USE_ZSTD 0
#endif
static int g_useZSTD = ZWRAP_USE_ZSTD; /* 0 = don't use ZSTD */
void useZSTD(int turn_on) { g_useZSTD = turn_on; }
int isUsingZSTD() { return g_useZSTD; }
const char * zstdVersion() { return ZSTD_VERSION_STRING; }
ZEXTERN const char * ZEXPORT z_zlibVersion OF((void)) { return zlibVersion(); }
/* *** Compression *** */
typedef struct {
ZBUFF_CCtx* zbc;
size_t bytesLeft;
int compressionLevel;
} ZWRAP_CCtx;
ZWRAP_CCtx* ZWRAP_createCCtx()
{
ZWRAP_CCtx* zwc = (ZWRAP_CCtx*)malloc(sizeof(ZWRAP_CCtx));
if (zwc==NULL) return NULL;
memset(zwc, 0, sizeof(*zwc));
zwc->zbc = ZBUFF_createCCtx();
return zwc;
}
size_t ZWRAP_freeCCtx(ZWRAP_CCtx* zwc)
{
if (zwc==NULL) return 0; /* support free on NULL */
ZBUFF_freeCCtx(zwc->zbc);
free(zwc);
return 0;
}
ZEXTERN int ZEXPORT z_deflateInit_ OF((z_streamp strm, int level,
const char *version, int stream_size))
{
ZWRAP_CCtx* zwc;
if (!g_useZSTD) {
LOG_WRAPPER("- deflateInit level=%d\n", level);
return deflateInit_((strm), (level), version, stream_size);
}
LOG_WRAPPER("- deflateInit level=%d\n", level);
zwc = ZWRAP_createCCtx();
if (zwc == NULL) return Z_MEM_ERROR;
{ size_t const errorCode = ZBUFF_compressInit(zwc->zbc, level);
if (ZSTD_isError(errorCode)) return Z_MEM_ERROR; }
zwc->compressionLevel = level;
strm->opaque = (voidpf) zwc;
strm->total_in = 0;
strm->total_out = 0;
return Z_OK;
}
ZEXTERN int ZEXPORT z_deflateInit2_ OF((z_streamp strm, int level, int method,
int windowBits, int memLevel,
int strategy, const char *version,
int stream_size))
{
if (!g_useZSTD)
return deflateInit2_(strm, level, method, windowBits, memLevel, strategy, version, stream_size);
return z_deflateInit_ (strm, level, version, stream_size);
}
ZEXTERN int ZEXPORT z_deflateSetDictionary OF((z_streamp strm,
const Bytef *dictionary,
uInt dictLength))
{
if (!g_useZSTD)
return deflateSetDictionary(strm, dictionary, dictLength);
{ ZWRAP_CCtx* zwc = (ZWRAP_CCtx*) strm->opaque;
LOG_WRAPPER("- deflateSetDictionary level=%d\n", (int)strm->data_type);
{ size_t const errorCode = ZBUFF_compressInitDictionary(zwc->zbc, dictionary, dictLength, zwc->compressionLevel);
if (ZSTD_isError(errorCode)) return Z_MEM_ERROR; }
}
return Z_OK;
}
ZEXTERN int ZEXPORT z_deflate OF((z_streamp strm, int flush))
{
ZWRAP_CCtx* zwc;
if (!g_useZSTD) {
int res = deflate(strm, flush);
LOG_WRAPPER("- avail_in=%d total_in=%d total_out=%d\n", (int)strm->avail_in, (int)strm->total_in, (int)strm->total_out);
return res;
}
zwc = (ZWRAP_CCtx*) strm->opaque;
LOG_WRAPPER("deflate flush=%d avail_in=%d avail_out=%d total_in=%d total_out=%d\n", (int)flush, (int)strm->avail_in, (int)strm->avail_out, (int)strm->total_in, (int)strm->total_out);
if (strm->avail_in > 0) {
size_t dstCapacity = strm->avail_out;
size_t srcSize = strm->avail_in;
size_t const errorCode = ZBUFF_compressContinue(zwc->zbc, strm->next_out, &dstCapacity, strm->next_in, &srcSize);
LOG_WRAPPER("ZBUFF_compressContinue srcSize=%d dstCapacity=%d\n", (int)srcSize, (int)dstCapacity);
if (ZSTD_isError(errorCode)) return Z_MEM_ERROR;
strm->next_out += dstCapacity;
strm->total_out += dstCapacity;
strm->avail_out -= dstCapacity;
strm->total_in += srcSize;
strm->next_in += srcSize;
strm->avail_in -= srcSize;
}
if (flush == Z_FULL_FLUSH) FINISH_WITH_ERR("Z_FULL_FLUSH is not supported!");
if (flush == Z_FINISH || flush == Z_FULL_FLUSH) {
size_t bytesLeft;
size_t dstCapacity = strm->avail_out;
if (zwc->bytesLeft) {
bytesLeft = ZBUFF_compressFlush(zwc->zbc, strm->next_out, &dstCapacity);
LOG_WRAPPER("ZBUFF_compressFlush avail_out=%d dstCapacity=%d bytesLeft=%d\n", (int)strm->avail_out, (int)dstCapacity, (int)bytesLeft);
} else {
bytesLeft = ZBUFF_compressEnd(zwc->zbc, strm->next_out, &dstCapacity);
LOG_WRAPPER("ZBUFF_compressEnd dstCapacity=%d bytesLeft=%d\n", (int)dstCapacity, (int)bytesLeft);
// { size_t const errorCode = ZBUFF_compressInit(zwc->zbc, 1);
// if (ZSTD_isError(errorCode)) return Z_MEM_ERROR; }
}
if (ZSTD_isError(bytesLeft)) return Z_MEM_ERROR;
strm->next_out += dstCapacity;
strm->total_out += dstCapacity;
strm->avail_out -= dstCapacity;
if (flush == Z_FINISH && bytesLeft == 0) return Z_STREAM_END;
zwc->bytesLeft = bytesLeft;
}
return Z_OK;
}
ZEXTERN int ZEXPORT z_deflateEnd OF((z_streamp strm))
{
if (!g_useZSTD) {
LOG_WRAPPER("- deflateEnd\n");
return deflateEnd(strm);
}
LOG_WRAPPER("- deflateEnd total_in=%d total_out=%d\n", (int)(strm->total_in), (int)(strm->total_out));
{ ZWRAP_CCtx* zwc = (ZWRAP_CCtx*) strm->opaque;
size_t const errorCode = ZWRAP_freeCCtx(zwc);
if (ZSTD_isError(errorCode)) return Z_MEM_ERROR;
}
return Z_OK;
}
ZEXTERN uLong ZEXPORT z_deflateBound OF((z_streamp strm,
uLong sourceLen))
{
if (!g_useZSTD)
return deflateBound(strm, sourceLen);
return ZSTD_compressBound(sourceLen);
}
ZEXTERN int ZEXPORT z_deflateParams OF((z_streamp strm,
int level,
int strategy))
{
if (!g_useZSTD) {
LOG_WRAPPER("- deflateParams level=%d strategy=%d\n", level, strategy);
return deflateParams(strm, level, strategy);
}
return Z_OK;
}
/* *** Decompression *** */
typedef struct {
ZBUFF_DCtx* zbd;
char headerBuf[ZSTD_FRAMEHEADERSIZE_MIN];
int errorCount;
int requiredHeaderSize;
/* zlib params */
int stream_size;
char *version;
int windowBits;
alloc_func zalloc; /* used to allocate the internal state */
free_func zfree; /* used to free the internal state */
voidpf opaque; /* private data object passed to zalloc and zfree */
} ZWRAP_DCtx;
ZWRAP_DCtx* ZWRAP_createDCtx(void)
{
ZWRAP_DCtx* zwd = (ZWRAP_DCtx*)malloc(sizeof(ZWRAP_DCtx));
if (zwd==NULL) return NULL;
memset(zwd, 0, sizeof(*zwd));
return zwd;
}
size_t ZWRAP_freeDCtx(ZWRAP_DCtx* zwd)
{
if (zwd==NULL) return 0; /* support free on null */
if (zwd->version) free(zwd->version);
if (zwd->zbd) ZBUFF_freeDCtx(zwd->zbd);
free(zwd);
return 0;
}
ZEXTERN int ZEXPORT z_inflateInit_ OF((z_streamp strm,
const char *version, int stream_size))
{
ZWRAP_DCtx* zwd = ZWRAP_createDCtx();
LOG_WRAPPER("- inflateInit\n");
if (zwd == NULL) return Z_MEM_ERROR;
zwd->stream_size = stream_size;
zwd->version = strdup(version);
zwd->zalloc = strm->zalloc;
zwd->zfree = strm->zfree;
zwd->opaque = strm->opaque;
zwd->requiredHeaderSize = ZWRAP_HEADERSIZE;
strm->state = (struct internal_state*) zwd;
strm->total_in = 0;
strm->total_out = 0;
strm->reserved = 1; /* mark as unknown steam */
return Z_OK;
}
ZEXTERN int ZEXPORT z_inflateInit2_ OF((z_streamp strm, int windowBits,
const char *version, int stream_size))
{
int ret = z_inflateInit_ (strm, version, stream_size);
if (ret == Z_OK) {
ZWRAP_DCtx* zwd = (ZWRAP_DCtx*)strm->state;
zwd->windowBits = windowBits;
}
return ret;
}
ZEXTERN int ZEXPORT z_inflateSetDictionary OF((z_streamp strm,
const Bytef *dictionary,
uInt dictLength))
{
if (!strm->reserved)
return inflateSetDictionary(strm, dictionary, dictLength);
LOG_WRAPPER("- inflateSetDictionary\n");
{ ZWRAP_DCtx* zwd = (ZWRAP_DCtx*) strm->state;
size_t errorCode = ZBUFF_decompressInitDictionary(zwd->zbd, dictionary, dictLength);
if (ZSTD_isError(errorCode)) return Z_MEM_ERROR;
if (strm->total_in == ZSTD_frameHeaderSize_min) {
size_t dstCapacity = 0;
size_t srcSize = strm->total_in;
errorCode = ZBUFF_decompressContinue(zwd->zbd, strm->next_out, &dstCapacity, zwd->headerBuf, &srcSize);
LOG_WRAPPER("ZBUFF_decompressContinue3 errorCode=%d srcSize=%d dstCapacity=%d\n", (int)errorCode, (int)srcSize, (int)dstCapacity);
if (dstCapacity > 0 || ZSTD_isError(errorCode)) {
LOG_WRAPPER("ERROR: ZBUFF_decompressContinue %s\n", ZSTD_getErrorName(errorCode));
return Z_MEM_ERROR;
}
}
}
return Z_OK;
}
ZEXTERN int ZEXPORT z_inflate OF((z_streamp strm, int flush))
{
if (!strm->reserved)
return inflate(strm, flush);
if (strm->avail_in > 0) {
size_t errorCode, dstCapacity, srcSize;
ZWRAP_DCtx* zwd = (ZWRAP_DCtx*) strm->state;
LOG_WRAPPER("inflate avail_in=%d avail_out=%d total_in=%d total_out=%d\n", (int)strm->avail_in, (int)strm->avail_out, (int)strm->total_in, (int)strm->total_out);
while (strm->total_in < ZSTD_FRAMEHEADERSIZE_MIN)
{
srcSize = MIN(strm->avail_in, zwd->requiredHeaderSize - strm->total_in);
memcpy(zwd->headerBuf+strm->total_in, strm->next_in, srcSize);
strm->total_in += srcSize;
strm->next_in += srcSize;
strm->avail_in -= srcSize;
if (strm->total_in < zwd->requiredHeaderSize) return Z_OK;
if (MEM_readLE32(zwd->headerBuf) != ZSTD_MAGICNUMBER) {
z_stream strm2;
strm2.next_in = strm->next_in;
strm2.avail_in = strm->avail_in;
strm2.next_out = strm->next_out;
strm2.avail_out = strm->avail_out;
if (zwd->windowBits)
errorCode = inflateInit2_(strm, zwd->windowBits, zwd->version, zwd->stream_size);
else
errorCode = inflateInit_(strm, zwd->version, zwd->stream_size);
LOG_WRAPPER("ZLIB inflateInit errorCode=%d\n", (int)errorCode);
if (errorCode != Z_OK) return errorCode;
/* inflate header */
strm->next_in = (unsigned char*)zwd->headerBuf;
strm->avail_in = ZWRAP_HEADERSIZE;
strm->avail_out = 0;
errorCode = inflate(strm, Z_NO_FLUSH);
LOG_WRAPPER("ZLIB inflate errorCode=%d strm->avail_in=%d\n", (int)errorCode, (int)strm->avail_in);
if (errorCode != Z_OK) return errorCode;
if (strm->avail_in > 0) return Z_MEM_ERROR;
strm->next_in = strm2.next_in;
strm->avail_in = strm2.avail_in;
strm->next_out = strm2.next_out;
strm->avail_out = strm2.avail_out;
strm->reserved = 0; /* mark as zlib stream */
if (flush == Z_INFLATE_SYNC) return inflateSync(strm);
return inflate(strm, flush);
}
if (zwd->requiredHeaderSize < ZSTD_FRAMEHEADERSIZE_MIN) {
zwd->requiredHeaderSize = ZSTD_FRAMEHEADERSIZE_MIN;
continue;
}
zwd->zbd = ZBUFF_createDCtx();
{ size_t const errorCode = ZBUFF_decompressInit(zwd->zbd);
if (ZSTD_isError(errorCode)) return Z_MEM_ERROR; }
srcSize = ZSTD_FRAMEHEADERSIZE_MIN;
dstCapacity = 0;
errorCode = ZBUFF_decompressContinue(zwd->zbd, strm->next_out, &dstCapacity, zwd->headerBuf, &srcSize);
LOG_WRAPPER("ZBUFF_decompressContinue1 errorCode=%d srcSize=%d dstCapacity=%d\n", (int)errorCode, (int)srcSize, (int)dstCapacity);
if (ZSTD_isError(errorCode)) {
LOG_WRAPPER("ERROR: ZBUFF_decompressContinue %s\n", ZSTD_getErrorName(errorCode));
return Z_MEM_ERROR;
}
if (strm->avail_in == 0) return Z_OK;
break;
}
srcSize = strm->avail_in;
dstCapacity = strm->avail_out;
errorCode = ZBUFF_decompressContinue(zwd->zbd, strm->next_out, &dstCapacity, strm->next_in, &srcSize);
LOG_WRAPPER("ZBUFF_decompressContinue2 errorCode=%d srcSize=%d dstCapacity=%d\n", (int)errorCode, (int)srcSize, (int)dstCapacity);
if (ZSTD_isError(errorCode)) {
LOG_WRAPPER("ERROR: ZBUFF_decompressContinue %s\n", ZSTD_getErrorName(errorCode));
zwd->errorCount++;
return (zwd->errorCount<=1) ? Z_NEED_DICT : Z_MEM_ERROR;
}
strm->next_out += dstCapacity;
strm->total_out += dstCapacity;
strm->avail_out -= dstCapacity;
strm->total_in += srcSize;
strm->next_in += srcSize;
strm->avail_in -= srcSize;
if (errorCode == 0) return Z_STREAM_END;
}
return Z_OK;
}
ZEXTERN int ZEXPORT z_inflateEnd OF((z_streamp strm))
{
int ret = Z_OK;
if (!strm->reserved)
ret = inflateEnd(strm);
LOG_WRAPPER("- inflateEnd total_in=%d total_out=%d\n", (int)(strm->total_in), (int)(strm->total_out));
{ ZWRAP_DCtx* zwd = (ZWRAP_DCtx*) strm->state;
size_t const errorCode = ZWRAP_freeDCtx(zwd);
if (ZSTD_isError(errorCode)) return Z_MEM_ERROR;
}
return ret;
}
ZEXTERN int ZEXPORT z_inflateSync OF((z_streamp strm))
{
return z_inflate(strm, Z_INFLATE_SYNC);
}
/* Advanced compression functions */
ZEXTERN int ZEXPORT z_deflateCopy OF((z_streamp dest,
z_streamp source))
{
if (!g_useZSTD)
return deflateCopy(dest, source);
FINISH_WITH_ERR("deflateCopy is not supported!");
}
ZEXTERN int ZEXPORT z_deflateReset OF((z_streamp strm))
{
if (!g_useZSTD)
return deflateReset(strm);
FINISH_WITH_ERR("deflateReset is not supported!");
}
ZEXTERN int ZEXPORT z_deflateTune OF((z_streamp strm,
int good_length,
int max_lazy,
int nice_length,
int max_chain))
{
if (!g_useZSTD)
return deflateTune(strm, good_length, max_lazy, nice_length, max_chain);
FINISH_WITH_ERR("deflateTune is not supported!");
}
ZEXTERN int ZEXPORT z_deflatePending OF((z_streamp strm,
unsigned *pending,
int *bits))
{
if (!g_useZSTD)
return deflatePending(strm, pending, bits);
FINISH_WITH_ERR("deflatePending is not supported!");
}
ZEXTERN int ZEXPORT z_deflatePrime OF((z_streamp strm,
int bits,
int value))
{
if (!g_useZSTD)
return deflatePrime(strm, bits, value);
FINISH_WITH_ERR("deflatePrime is not supported!");
}
ZEXTERN int ZEXPORT z_deflateSetHeader OF((z_streamp strm,
gz_headerp head))
{
if (!g_useZSTD)
return deflateSetHeader(strm, head);
FINISH_WITH_ERR("deflateSetHeader is not supported!");
}
/* Advanced compression functions */
ZEXTERN int ZEXPORT z_inflateGetDictionary OF((z_streamp strm,
Bytef *dictionary,
uInt *dictLength))
{
if (!strm->reserved)
return inflateGetDictionary(strm, dictionary, dictLength);
FINISH_WITH_ERR("inflateGetDictionary is not supported!");
}
ZEXTERN int ZEXPORT z_inflateCopy OF((z_streamp dest,
z_streamp source))
{
if (!g_useZSTD)
return inflateCopy(dest, source);
FINISH_WITH_ERR("inflateCopy is not supported!");
}
ZEXTERN int ZEXPORT z_inflateReset OF((z_streamp strm))
{
if (!strm->reserved)
return inflateReset(strm);
FINISH_WITH_ERR("inflateReset is not supported!");
}
ZEXTERN int ZEXPORT z_inflateReset2 OF((z_streamp strm,
int windowBits))
{
if (!strm->reserved)
return inflateReset2(strm, windowBits);
FINISH_WITH_ERR("inflateReset2 is not supported!");
}
ZEXTERN int ZEXPORT z_inflatePrime OF((z_streamp strm,
int bits,
int value))
{
if (!strm->reserved)
return inflatePrime(strm, bits, value);
FINISH_WITH_ERR("inflatePrime is not supported!");
}
ZEXTERN long ZEXPORT z_inflateMark OF((z_streamp strm))
{
if (!strm->reserved)
return inflateMark(strm);
FINISH_WITH_ERR("inflateMark is not supported!");
}
ZEXTERN int ZEXPORT z_inflateGetHeader OF((z_streamp strm,
gz_headerp head))
{
if (!strm->reserved)
return inflateGetHeader(strm, head);
FINISH_WITH_ERR("inflateGetHeader is not supported!");
}
ZEXTERN int ZEXPORT z_inflateBackInit_ OF((z_streamp strm, int windowBits,
unsigned char FAR *window,
const char *version,
int stream_size))
{
if (!strm->reserved)
return inflateBackInit_(strm, windowBits, window, version, stream_size);
FINISH_WITH_ERR("inflateBackInit is not supported!");
}
ZEXTERN int ZEXPORT z_inflateBack OF((z_streamp strm,
in_func in, void FAR *in_desc,
out_func out, void FAR *out_desc))
{
if (!strm->reserved)
return inflateBack(strm, in, in_desc, out, out_desc);
FINISH_WITH_ERR("inflateBack is not supported!");
}
ZEXTERN int ZEXPORT z_inflateBackEnd OF((z_streamp strm))
{
if (!strm->reserved)
return inflateBackEnd(strm);
FINISH_WITH_ERR("inflateBackEnd is not supported!");
}
ZEXTERN uLong ZEXPORT z_zlibCompileFlags OF((void)) { return zlibCompileFlags(); };
/* utility functions */
#ifndef Z_SOLO
ZEXTERN int ZEXPORT z_compress OF((Bytef *dest, uLongf *destLen,
const Bytef *source, uLong sourceLen))
{
if (!g_useZSTD)
return compress(dest, destLen, source, sourceLen);
size_t dstCapacity = *destLen;
LOG_WRAPPER("z_compress sourceLen=%d dstCapacity=%d\n", (int)sourceLen, (int)dstCapacity);
{ size_t const errorCode = ZSTD_compress(dest, dstCapacity, source, sourceLen, -1);
if (ZSTD_isError(errorCode)) return Z_MEM_ERROR;
*destLen = errorCode;
}
return Z_OK;
}
ZEXTERN int ZEXPORT z_compress2 OF((Bytef *dest, uLongf *destLen,
const Bytef *source, uLong sourceLen,
int level))
{
if (!g_useZSTD)
return compress2(dest, destLen, source, sourceLen, level);
size_t dstCapacity = *destLen;
{ size_t const errorCode = ZSTD_compress(dest, dstCapacity, source, sourceLen, level);
if (ZSTD_isError(errorCode)) return Z_MEM_ERROR;
*destLen = errorCode;
}
return Z_OK;
}
ZEXTERN uLong ZEXPORT z_compressBound OF((uLong sourceLen))
{
if (!g_useZSTD)
return compressBound(sourceLen);
return ZSTD_compressBound(sourceLen);
}
ZEXTERN int ZEXPORT z_uncompress OF((Bytef *dest, uLongf *destLen,
const Bytef *source, uLong sourceLen))
{
if (sourceLen < 4 || MEM_readLE32(source) != ZSTD_MAGICNUMBER)
// if (!g_useZSTD)
return uncompress(dest, destLen, source, sourceLen);
size_t dstCapacity = *destLen;
{ size_t const errorCode = ZSTD_decompress(dest, dstCapacity, source, sourceLen);
if (ZSTD_isError(errorCode)) return Z_MEM_ERROR;
*destLen = errorCode;
}
return Z_OK;
}
/* gzip file access functions */
ZEXTERN gzFile ZEXPORT z_gzopen OF((const char *path, const char *mode))
{
if (!g_useZSTD)
return gzopen(path, mode);
FINISH_WITH_NULL_ERR("gzopen is not supported!");
}
ZEXTERN gzFile ZEXPORT z_gzdopen OF((int fd, const char *mode))
{
if (!g_useZSTD)
return gzdopen(fd, mode);
FINISH_WITH_NULL_ERR("gzdopen is not supported!");
}
ZEXTERN int ZEXPORT z_gzbuffer OF((gzFile file, unsigned size))
{
if (!g_useZSTD)
return gzbuffer(file, size);
FINISH_WITH_ERR("gzbuffer is not supported!");
}
ZEXTERN int ZEXPORT z_gzsetparams OF((gzFile file, int level, int strategy))
{
if (!g_useZSTD)
return gzsetparams(file, level, strategy);
FINISH_WITH_ERR("gzsetparams is not supported!");
}
ZEXTERN int ZEXPORT z_gzread OF((gzFile file, voidp buf, unsigned len))
{
if (!g_useZSTD)
return gzread(file, buf, len);
FINISH_WITH_ERR("gzread is not supported!");
}
ZEXTERN int ZEXPORT z_gzwrite OF((gzFile file,
voidpc buf, unsigned len))
{
if (!g_useZSTD)
return gzwrite(file, buf, len);
FINISH_WITH_ERR("gzwrite is not supported!");
}
ZEXTERN int ZEXPORTVA z_gzprintf Z_ARG((gzFile file, const char *format, ...))
{
if (!g_useZSTD) {
int ret;
char buf[1024];
va_list args;
va_start (args, format);
ret = vsprintf (buf, format, args);
va_end (args);
ret = gzprintf(file, buf);
// printf("gzprintf ret=%d\n", ret);
return ret;
}
FINISH_WITH_ERR("gzprintf is not supported!");
}
ZEXTERN int ZEXPORT z_gzputs OF((gzFile file, const char *s))
{
if (!g_useZSTD)
return gzputs(file, s);
FINISH_WITH_ERR("gzputs is not supported!");
}
ZEXTERN char * ZEXPORT z_gzgets OF((gzFile file, char *buf, int len))
{
if (!g_useZSTD)
return gzgets(file, buf, len);
FINISH_WITH_NULL_ERR("gzgets is not supported!");
}
ZEXTERN int ZEXPORT z_gzputc OF((gzFile file, int c))
{
if (!g_useZSTD)
return gzputc(file, c);
FINISH_WITH_ERR("gzputc is not supported!");
}
ZEXTERN int ZEXPORT z_gzgetc OF((gzFile file))
{
if (!g_useZSTD)
return gzgetc(file);
FINISH_WITH_ERR("gzgetc is not supported!");
}
ZEXTERN int ZEXPORT z_gzungetc OF((int c, gzFile file))
{
if (!g_useZSTD)
return gzungetc(c, file);
FINISH_WITH_ERR("gzungetc is not supported!");
}
ZEXTERN int ZEXPORT z_gzflush OF((gzFile file, int flush))
{
if (!g_useZSTD)
return gzflush(file, flush);
FINISH_WITH_ERR("gzflush is not supported!");
}
ZEXTERN z_off_t ZEXPORT z_gzseek OF((gzFile file, z_off_t offset, int whence))
{
if (!g_useZSTD)
return gzseek(file, offset, whence);
FINISH_WITH_ERR("gzseek is not supported!");
}
ZEXTERN int ZEXPORT z_gzrewind OF((gzFile file))
{
if (!g_useZSTD)
return gzrewind(file);
FINISH_WITH_ERR("gzrewind is not supported!");
}
ZEXTERN z_off_t ZEXPORT z_gztell OF((gzFile file))
{
if (!g_useZSTD)
return gztell(file);
FINISH_WITH_ERR("gztell is not supported!");
}
ZEXTERN z_off_t ZEXPORT z_gzoffset OF((gzFile file))
{
if (!g_useZSTD)
return gzoffset(file);
FINISH_WITH_ERR("gzoffset is not supported!");
}
ZEXTERN int ZEXPORT z_gzeof OF((gzFile file))
{
if (!g_useZSTD)
return gzeof(file);
FINISH_WITH_ERR("gzeof is not supported!");
}
ZEXTERN int ZEXPORT z_gzdirect OF((gzFile file))
{
if (!g_useZSTD)
return gzdirect(file);
FINISH_WITH_ERR("gzdirect is not supported!");
}
ZEXTERN int ZEXPORT z_gzclose OF((gzFile file))
{
if (!g_useZSTD)
return gzclose(file);
FINISH_WITH_ERR("gzclose is not supported!");
}
ZEXTERN int ZEXPORT z_gzclose_r OF((gzFile file))
{
if (!g_useZSTD)
return gzclose_r(file);
FINISH_WITH_ERR("gzclose_r is not supported!");
}
ZEXTERN int ZEXPORT z_gzclose_w OF((gzFile file))
{
if (!g_useZSTD)
return gzclose_w(file);
FINISH_WITH_ERR("gzclose_w is not supported!");
}
ZEXTERN const char * ZEXPORT z_gzerror OF((gzFile file, int *errnum))
{
if (!g_useZSTD)
return gzerror(file, errnum);
FINISH_WITH_NULL_ERR("gzerror is not supported!");
}
ZEXTERN void ZEXPORT z_gzclearerr OF((gzFile file))
{
if (!g_useZSTD)
gzclearerr(file);
}
#endif /* !Z_SOLO */
/* checksum functions */
ZEXTERN uLong ZEXPORT z_adler32 OF((uLong adler, const Bytef *buf, uInt len))
{
return adler32(adler, buf, len);
}
ZEXTERN uLong ZEXPORT z_crc32 OF((uLong crc, const Bytef *buf, uInt len))
{
return crc32(crc, buf, len);
}
+53
View File
@@ -0,0 +1,53 @@
/*
zstd_zlibwrapper.h - zstd wrapper for zlib
Copyright (C) 2016, Przemyslaw Skibinski.
BSD 2-Clause License (http://www.opensource.org/licenses/bsd-license.php)
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the following disclaimer
in the documentation and/or other materials provided with the
distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
You can contact the author at :
- zstd source repository : https://github.com/Cyan4973/zstd
*/
#ifndef ZSTD_ZLIBWRAPPER_H
#define ZSTD_ZLIBWRAPPER_H
#if defined (__cplusplus)
extern "C" {
#endif
#define Z_PREFIX
#include <zlib.h>
void useZSTD(int turn_on);
int isUsingZSTD();
const char * zstdVersion();
#if defined (__cplusplus)
}
#endif
#endif /* ZSTD_ZLIBWRAPPER_H */