ddidderr/zstd-rs
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
Files
bdd35c838d6e9ba5a6e0c59d1f6e0e08dbaccd5f
zstd-rs/tests/Makefile
T
ddidderr bdd35c838d build(rust): add legacy feature scaffolding 
The legacy decoders (lib/legacy/zstd_v01.c .. zstd_v07.c) are next in
the Rust migration. Each of those files is a frozen snapshot of the
FSE/Huff0 entropy coders and frame logic of one historical release, so
their ports must not reuse the modern Rust entropy modules and must not
share code with each other: outputs and error codes have to stay
byte-identical to the frozen C forever. This commit installs the
build-system scaffolding so seven per-version ports can land
independently, each adding only its own module file plus a one-line
registration in rust/src/legacy/mod.rs.

Cargo grows features legacy-v01 .. legacy-v07. They are never default
features: the C build defaults differ per build system, so each build
system passes the list explicitly, derived from its own legacy
configuration:

- lib/Makefile and programs/Makefile map ZSTD_LEGACY_SUPPORT=N to the
  features for versions N..7 (0 disables legacy), mirroring the
  ZSTD_LEGACY_FILES selection in lib/libzstd.mk.
- tests/Makefile always enables all seven features because its
  ZSTDLEGACY_FILES wildcard compiles every lib/legacy/*.c regardless of
  the dispatch level.
- build/meson maps legacy_level exactly like the makefiles; build/cmake
  enables all seven whenever ZSTD_LEGACY_SUPPORT is ON because it
  always compiles all seven C files (ZSTD_LEGACY_LEVEL only selects the
  C dispatch).

Every build system also encodes the legacy selection in the Rust target
directory name (e.g. c1-d1-default-legacy5), for the same reason the
HUF mode is encoded there: a cached archive built for one configuration
must never be linked into a build expecting another. In tests/Makefile
the legacy level additionally flows into the existing HUF C-mode stamp,
so the flat C test objects (which bake -DZSTD_LEGACY_SUPPORT into the
dispatch) are rebuilt whenever the level changes. In programs/Makefile
the compress-only, decompress-only, and CLI archives keep
level-independent directories (RUST_HUF_MODE) because they are only
linked into ZSTD_LEGACY_SUPPORT=0 program variants and carry no legacy
features.

A feature whose version has not been ported yet gates nothing: the
module registration in rust/src/legacy/mod.rs is added by each port,
so enabling e.g. legacy-v05 today simply leaves that decoder in C.
This is what makes mixed C/Rust legacy levels link cleanly while the
seven ports land in any order.

Test plan:
- cd rust && cargo fmt --check && cargo clippy --all-targets
  -- -D warnings && cargo test --all-targets
- cargo clippy with --no-default-features --features
  decompression,legacy-v01 and with all seven legacy features
- make -C tests fuzzer && ./tests/fuzzer -i1 --no-big-tests
- make -C tests test-rust-lib-smoke; make -C tests test-legacy
- make -C lib libzstd.a with ZSTD_LEGACY_SUPPORT=0, 1 and default (5)
- cmake configure and meson setup (including -Dlegacy_level=1) emit the
  expected --features lists and legacy-suffixed target directories
2026-07-11 23:07:55 +02:00

628 lines
23 KiB
Makefile
Raw Blame History

# ################################################################
# Copyright (c) Meta Platforms, Inc. and affiliates.
# All rights reserved.
#
# This source code is licensed under both the BSD-style license (found in the
# LICENSE file in the root directory of this source tree) and the GPLv2 (found
# in the COPYING file in the root directory of this source tree).
# You may select, at your option, one of the above-listed licenses.
# ################################################################
# datagen : Synthetic and parametrable data generator, for tests
# fullbench : Precisely measure speed for each zstd inner functions
# fullbench32: Same as fullbench, but forced to compile in 32-bits mode
# fuzzer : Test tool, to check zstd integrity on target platform
# fuzzer32: Same as fuzzer, but forced to compile in 32-bits mode
# paramgrill : parameter tester for zstd
# test-zstd-speed.py : script for testing zstd speed difference between commits
# versionsTest : compatibility test between zstd versions stored on Github (v0.1+)
# zstreamtest : Fuzzer test tool for zstd streaming API
# zstreamtest32: Same as zstreamtest, but forced to compile in 32-bits mode
# ##########################################################################
ZSTD_LEGACY_SUPPORT ?= 5
export ZSTD_LEGACY_SUPPORT
DEBUGLEVEL ?= 2
export DEBUGLEVEL # transmit value to sub-makefiles
.PHONY: default
default: fullbench
LIBZSTD_MK_DIR := ../lib
include $(LIBZSTD_MK_DIR)/libzstd.mk
PRGDIR = ../programs
PYTHON ?= python3
TESTARTEFACT := versionsTest
# The Rust implementation is linked into C compatibility tests. Keep the
# archive a real prerequisite so a changed Rust source always relinks a test.
CARGO ?= cargo
RUST_DIR := ../rust
RUST_MANIFEST := $(RUST_DIR)/Cargo.toml
RUST_SOURCES := $(RUST_MANIFEST) $(RUST_DIR)/Cargo.lock \
$(shell find $(RUST_DIR)/src -type f -name '*.rs' -print)
# Keep Rust's HUF implementation in lockstep with libzstd.mk's C selection.
# Forced modes may arrive as libzstd.mk variables or as direct -D flags in
# MOREFLAGS/CFLAGS/CPPFLAGS, as used by the C-only build matrix.
RUST_HUF_FORCE_X1 := $(HUF_FORCE_DECOMPRESS_X1)
RUST_HUF_FORCE_X2 := $(HUF_FORCE_DECOMPRESS_X2)
ifneq ($(filter -DHUF_FORCE_DECOMPRESS_X1 -DHUF_FORCE_DECOMPRESS_X1=%,$(CPPFLAGS) $(CFLAGS) $(MOREFLAGS)),)
RUST_HUF_FORCE_X1 := 1
endif
ifneq ($(filter -DHUF_FORCE_DECOMPRESS_X2 -DHUF_FORCE_DECOMPRESS_X2=%,$(CPPFLAGS) $(CFLAGS) $(MOREFLAGS)),)
RUST_HUF_FORCE_X2 := 1
endif
# A separate target directory per mode prevents a forced build from reusing the
# default archive, and also gives link targets a configuration-specific input.
ifneq ($(RUST_HUF_FORCE_X1),0)
ifneq ($(RUST_HUF_FORCE_X2),0)
$(error HUF_FORCE_DECOMPRESS_X1 and HUF_FORCE_DECOMPRESS_X2 are mutually exclusive)
endif
endif
RUST_HUF_FEATURE :=
RUST_HUF_MODE := default
ifneq ($(RUST_HUF_FORCE_X1),0)
RUST_HUF_FEATURE := huf-force-decompress-x1
RUST_HUF_MODE := huf-force-decompress-x1
endif
ifneq ($(RUST_HUF_FORCE_X2),0)
RUST_HUF_FEATURE := huf-force-decompress-x2
RUST_HUF_MODE := huf-force-decompress-x2
endif
# Test binaries compile every lib/legacy/*.c file regardless of the dispatch
# level (ZSTDLEGACY_FILES is a plain wildcard below), so the Rust archive must
# always carry all ported legacy decoders too. The build configuration still
# encodes the level because the flat C objects bake -DZSTD_LEGACY_SUPPORT into
# the dispatch code and must never outlive a level change.
RUST_LEGACY_FEATURES := legacy-v01,legacy-v02,legacy-v03,legacy-v04,legacy-v05,legacy-v06,legacy-v07
RUST_BUILD_CONFIG := $(RUST_HUF_MODE)-legacy$(ZSTD_LEGACY_SUPPORT)
RUST_TARGET_DIR := $(RUST_DIR)/target/$(RUST_BUILD_CONFIG)
RUST_STATICLIB := $(RUST_TARGET_DIR)/release/libzstd_rs.a
RUST_TARGET_32 ?= i686-unknown-linux-gnu
RUST_STATICLIB_32 := $(RUST_TARGET_DIR)/$(RUST_TARGET_32)/release/libzstd_rs.a
# Tests build every library module, so they use the crate's default feature
# set (compression, decompression, and dict-builder) plus any forced HUF mode.
RUST_CARGO_FLAGS := --manifest-path $(RUST_MANIFEST) --release \
--target-dir $(RUST_TARGET_DIR)
ifneq ($(RUST_HUF_FEATURE),)
RUST_CARGO_FLAGS += --features $(RUST_HUF_FEATURE)
endif
RUST_CARGO_FLAGS += --features $(RUST_LEGACY_FEATURES)
$(RUST_STATICLIB): $(RUST_SOURCES)
$(CARGO) build $(RUST_CARGO_FLAGS)
$(RUST_STATICLIB_32): $(RUST_SOURCES)
$(CARGO) build $(RUST_CARGO_FLAGS) --target $(RUST_TARGET_32)
# Program-only helpers that were C sources shared with the tests (timefn,
# benchfn) now live in the Rust CLI package. The tests link a helpers-only
# archive, built without the `cli` feature: the parser/dispatch layer needs
# the C fileio backend, which test binaries do not provide.
RUST_CLI_DIR := $(RUST_DIR)/cli
RUST_CLI_MANIFEST := $(RUST_CLI_DIR)/Cargo.toml
RUST_CLI_HELPER_SOURCES := $(RUST_CLI_MANIFEST) $(RUST_CLI_DIR)/Cargo.lock \
$(RUST_CLI_DIR)/src/lib.rs \
$(RUST_DIR)/src/timefn.rs $(RUST_DIR)/src/benchfn.rs
RUST_CLI_HELPERS_TARGET_DIR := $(RUST_DIR)/target/cli-helpers
RUST_CLI_HELPERS_STATICLIB := $(RUST_CLI_HELPERS_TARGET_DIR)/release/libzstd_cli_rs.a
RUST_CLI_HELPERS_STATICLIB_32 := $(RUST_CLI_HELPERS_TARGET_DIR)/$(RUST_TARGET_32)/release/libzstd_cli_rs.a
RUST_CLI_HELPERS_CARGO_FLAGS := --manifest-path $(RUST_CLI_MANIFEST) --release \
--target-dir $(RUST_CLI_HELPERS_TARGET_DIR) \
--no-default-features
$(RUST_CLI_HELPERS_STATICLIB): $(RUST_CLI_HELPER_SOURCES)
$(CARGO) build $(RUST_CLI_HELPERS_CARGO_FLAGS)
$(RUST_CLI_HELPERS_STATICLIB_32): $(RUST_CLI_HELPER_SOURCES)
$(CARGO) build $(RUST_CLI_HELPERS_CARGO_FLAGS) --target $(RUST_TARGET_32)
# These test objects have flat filenames, unlike the configuration-hashed
# program objects. Track the HUF mode separately so a C object set compiled
# for one decoder is never relinked with a Rust archive for another decoder.
# The stamp is an empty archive, so normal linker `$^` recipes can consume it.
RUST_HUF_C_MODE_FILE := $(RUST_DIR)/target/.huf-c-mode-tests
RUST_HUF_C_MODE_STAMP := $(RUST_DIR)/target/.huf-c-mode-tests.a
.PHONY: FORCE_RUST_HUF_C_MODE
FORCE_RUST_HUF_C_MODE:
$(RUST_HUF_C_MODE_STAMP): FORCE_RUST_HUF_C_MODE
@mkdir -p $(@D)
@if test ! -r $(RUST_HUF_C_MODE_FILE) \
|| test "$$(cat $(RUST_HUF_C_MODE_FILE))" != "$(RUST_BUILD_CONFIG)" \
|| test ! -f $@; then \
printf '%s\n' "$(RUST_BUILD_CONFIG)" > $(RUST_HUF_C_MODE_FILE); \
$(AR) rcs $@; \
touch $@; \
fi
CLEAN += $(RUST_HUF_C_MODE_FILE) $(RUST_HUF_C_MODE_STAMP)
DEBUGFLAGS += -g -Wno-c++-compat
CPPFLAGS += -I$(LIB_SRCDIR) -I$(LIB_SRCDIR)/common -I$(LIB_SRCDIR)/compress -I$(LIB_SRCDIR)/legacy \
-I$(LIB_SRCDIR)/dictBuilder -I$(LIB_SRCDIR)/deprecated -I$(PRGDIR) \
-DZSTD_WINDOW_OVERFLOW_CORRECT_FREQUENTLY=1
ZSTDCOMMON_FILES := $(sort $(ZSTD_COMMON_FILES))
ZSTDCOMP_FILES := $(sort $(ZSTD_COMPRESS_FILES))
ZSTDDECOMP_FILES := $(sort $(ZSTD_DECOMPRESS_FILES))
ZSTDLEGACY_FILES := $(sort $(wildcard $(LIB_SRCDIR)/legacy/*.c))
ZSTD_FILES := $(ZSTDDECOMP_FILES) $(ZSTDCOMMON_FILES) $(ZSTDCOMP_FILES) $(ZSTDLEGACY_FILES)
ZDICT_FILES := $(sort $(ZSTD_DICTBUILDER_FILES))
ZSTD_F1 := $(sort $(wildcard $(ZSTD_FILES)))
ZSTD_OBJ1 := $(subst $(LIB_SRCDIR)/common/,zstdm_,$(ZSTD_F1))
ZSTD_OBJ2 := $(subst $(LIB_SRCDIR)/compress/,zstdc_,$(ZSTD_OBJ1))
ZSTD_OBJ3 := $(subst $(LIB_SRCDIR)/decompress/,zstdd_,$(ZSTD_OBJ2))
ZSTD_OBJ4 := $(subst $(LIB_SRCDIR)/legacy/,zstdl_,$(ZSTD_OBJ3))
ZSTD_OBJ5 := $(ZSTD_OBJ4:.c=.o)
ZSTD_OBJECTS := $(ZSTD_OBJ5:.S=.o)
ZSTDMT_OBJ1 := $(subst $(LIB_SRCDIR)/common/,zstdmt_m_,$(ZSTD_F1))
ZSTDMT_OBJ2 := $(subst $(LIB_SRCDIR)/compress/,zstdmt_c_,$(ZSTDMT_OBJ1))
ZSTDMT_OBJ3 := $(subst $(LIB_SRCDIR)/decompress/,zstdmt_d_,$(ZSTDMT_OBJ2))
ZSTDMT_OBJ4 := $(subst $(LIB_SRCDIR)/legacy/,zstdmt_l_,$(ZSTDMT_OBJ3))
ZSTDMT_OBJ5 := $(ZSTDMT_OBJ4:.c=.o)
ZSTDMT_OBJECTS := $(ZSTDMT_OBJ5:.S=.o)
$(ZSTD_OBJECTS) $(ZSTDMT_OBJECTS): $(RUST_HUF_C_MODE_STAMP)
# Define *.exe as extension for Windows systems
ifneq (,$(filter Windows%,$(OS)))
EXT =.exe
MULTITHREAD_CPP = -DZSTD_MULTITHREAD
MULTITHREAD_LD =
else
EXT =
MULTITHREAD_CPP = -DZSTD_MULTITHREAD
MULTITHREAD_LD = -pthread
endif
MULTITHREAD = $(MULTITHREAD_CPP) $(MULTITHREAD_LD)
VOID = /dev/null
ZSTREAM_TESTTIME ?= -T90s
FUZZERTEST ?= -T200s
ZSTDRTTEST = --test-large-data
DECODECORPUS_TESTTIME ?= -T30
.PHONY: all
all: fullbench fuzzer zstreamtest paramgrill datagen decodecorpus roundTripCrash poolTests
.PHONY: all32
all32: fullbench32 fuzzer32 zstreamtest32
.PHONY: allnothread
allnothread: MULTITHREAD_CPP=
allnothread: MULTITHREAD_LD=
allnothread: fullbench fuzzer paramgrill datagen decodecorpus
# note : broken : requires symbols unavailable from dynamic library
.PHONY: dll
dll: fuzzer-dll zstreamtest-dll
.PHONY: zstd zstd32 zstd-nolegacy # only external makefile knows how to build or update them
zstd zstd32 zstd-nolegacy zstd-dll:
$(MAKE) -C $(PRGDIR) $@ MOREFLAGS+="$(DEBUGFLAGS)"
.PHONY: libzstd
libzstd :
$(MAKE) -C $(LIB_SRCDIR) libzstd MOREFLAGS+="$(DEBUGFLAGS)"
%-dll : libzstd
%-dll : LDFLAGS += -L$(LIB_BINDIR) -lzstd
$(LIB_BINDIR)/libzstd.a :
$(MAKE) -C $(LIB_SRCDIR) libzstd.a
zstdm_%.o : $(LIB_SRCDIR)/common/%.c
$(CC) -c $(CPPFLAGS) $(CFLAGS) $< -o $@
zstdc_%.o : $(LIB_SRCDIR)/compress/%.c
$(CC) -c $(CPPFLAGS) $(CFLAGS) $< -o $@
zstdd_%.o : $(LIB_SRCDIR)/decompress/%.c
$(CC) -c $(CPPFLAGS) $(CFLAGS) $< -o $@
zstdd_%.o : $(LIB_SRCDIR)/decompress/%.S
$(CC) -c $(CPPFLAGS) $(ASFLAGS) $< -o $@
zstdl_%.o : $(LIB_SRCDIR)/legacy/%.c
$(CC) -c $(CPPFLAGS) $(CFLAGS) $< -o $@
zstdmt%.o : CPPFLAGS += $(MULTITHREAD_CPP)
zstdmt_m_%.o : $(LIB_SRCDIR)/common/%.c
$(CC) -c $(CPPFLAGS) $(CFLAGS) $< -o $@
zstdmt_c_%.o : $(LIB_SRCDIR)/compress/%.c
$(CC) -c $(CPPFLAGS) $(CFLAGS) $< -o $@
zstdmt_d_%.o : $(LIB_SRCDIR)/decompress/%.c
$(CC) -c $(CPPFLAGS) $(CFLAGS) $< -o $@
zstdmt_d_%.o : $(LIB_SRCDIR)/decompress/%.S
$(CC) -c $(CPPFLAGS) $(ASFLAGS) $< -o $@
zstdmt_l_%.o : $(LIB_SRCDIR)/legacy/%.c
$(CC) -c $(CPPFLAGS) $(CFLAGS) $< -o $@
FULLBENCHS := fullbench fullbench32
CLEAN += $(FULLBENCHS)
fullbench32: CPPFLAGS += -m32
$(FULLBENCHS) : CPPFLAGS += $(MULTITHREAD_CPP) -Wno-deprecated-declarations
$(FULLBENCHS) : LDFLAGS += $(MULTITHREAD_LD)
$(FULLBENCHS) : DEBUGFLAGS = -DNDEBUG # turn off assert() for speed measurements
$(FULLBENCHS) : DEBUGLEVEL = 0 # turn off assert() for speed measurements
$(FULLBENCHS) : $(ZSTD_FILES)
$(FULLBENCHS) : $(PRGDIR)/datagen.c $(PRGDIR)/lorem.c $(PRGDIR)/util.c $(PRGDIR)/timefn.c $(PRGDIR)/benchfn.c fullbench.c
$(LINK.c) $^ -o $@$(EXT)
CLEAN += fullbench-lib
fullbench-lib : CPPFLAGS += -DXXH_NAMESPACE=ZSTD_
fullbench-lib : $(PRGDIR)/datagen.c $(PRGDIR)/lorem.c $(PRGDIR)/util.c $(PRGDIR)/timefn.c $(PRGDIR)/benchfn.c $(LIB_SRCDIR)/libzstd.a fullbench.c
$(LINK.c) $^ -o $@$(EXT)
# note : broken : requires symbols unavailable from dynamic library
fullbench-dll: $(PRGDIR)/datagen.c $(PRGDIR)/util.c $(PRGDIR)/benchfn.c $(PRGDIR)/timefn.c fullbench.c
# $(CC) $(FLAGS) $(filter %.c,$^) -o $@$(EXT) -DZSTD_DLL_IMPORT=1 $(LIB_SRCDIR)/dll/libzstd.dll
$(LINK.c) $^ $(LDLIBS) -o $@$(EXT)
CLEAN += fuzzer fuzzer32
fuzzer : CPPFLAGS += $(MULTITHREAD_CPP) -Wno-deprecated-declarations
fuzzer : LDFLAGS += $(MULTITHREAD_LD)
fuzzer : $(ZSTDMT_OBJECTS)
fuzzer fuzzer32 : $(ZDICT_FILES) $(PRGDIR)/util.c $(PRGDIR)/timefn.c $(PRGDIR)/datagen.c fuzzer.c
fuzzer32 : CFLAGS += -m32 $(MULTITHREAD)
fuzzer32 : $(ZSTD_FILES)
$(LINK.c) $^ -o $@$(EXT)
# note : broken : requires symbols unavailable from dynamic library
fuzzer-dll : $(LIB_SRCDIR)/common/xxhash.c $(PRGDIR)/util.c $(PRGDIR)/timefn.c $(PRGDIR)/datagen.c fuzzer.c $(RUST_CLI_HELPERS_STATICLIB)
$(CC) $(CPPFLAGS) $(CFLAGS) $(filter %.c,$^) $(RUST_CLI_HELPERS_STATICLIB) $(LDFLAGS) -o $@$(EXT)
CLEAN += zstreamtest zstreamtest32
ZSTREAM_LOCAL_FILES := $(PRGDIR)/datagen.c $(PRGDIR)/util.c $(PRGDIR)/timefn.c seqgen.c zstreamtest.c external_matchfinder.c
ZSTREAM_PROPER_FILES := $(ZDICT_FILES) $(ZSTREAM_LOCAL_FILES)
ZSTREAMFILES := $(ZSTD_FILES) $(ZSTREAM_PROPER_FILES)
zstreamtest32 : CFLAGS += -m32
zstreamtest zstreamtest32 : CPPFLAGS += $(MULTITHREAD_CPP)
zstreamtest zstreamtest32 : LDFLAGS += $(MULTITHREAD_LD)
zstreamtest : $(ZSTDMT_OBJECTS) $(ZSTREAM_PROPER_FILES)
zstreamtest32 : $(ZSTREAMFILES)
zstreamtest zstreamtest32 :
$(LINK.c) $^ -o $@$(EXT)
CLEAN += zstreamtest_asan
zstreamtest_asan : CFLAGS += -fsanitize=address
zstreamtest_asan : $(ZSTREAMFILES)
$(LINK.c) $(MULTITHREAD) $^ -o $@$(EXT)
CLEAN += zstreamtest_tsan
zstreamtest_tsan : CFLAGS += -fsanitize=thread
zstreamtest_tsan : $(ZSTREAMFILES)
$(LINK.c) $(MULTITHREAD) $^ -o $@$(EXT)
CLEAN += zstreamtest_ubsan
zstreamtest_ubsan : CFLAGS += -fsanitize=undefined
zstreamtest_ubsan : $(ZSTREAMFILES)
$(LINK.c) $(MULTITHREAD) $^ -o $@$(EXT)
# note : broken : requires symbols unavailable from dynamic library
zstreamtest-dll : $(LIB_SRCDIR)/common/xxhash.c # xxh symbols not exposed from dll
zstreamtest-dll : $(ZSTREAM_LOCAL_FILES) $(RUST_CLI_HELPERS_STATICLIB)
$(CC) $(CPPFLAGS) $(CFLAGS) $(filter %.c,$^) $(RUST_CLI_HELPERS_STATICLIB) $(LDFLAGS) -o $@$(EXT)
CLEAN += paramgrill
paramgrill : DEBUGFLAGS = # turn off debug for speed measurements
paramgrill : LDLIBS += -lm
paramgrill : $(ZSTD_FILES) $(PRGDIR)/util.c $(PRGDIR)/timefn.c $(PRGDIR)/benchfn.c $(PRGDIR)/benchzstd.c $(PRGDIR)/datagen.c $(PRGDIR)/lorem.c paramgrill.c
CLEAN += datagen
datagen : $(PRGDIR)/datagen.c $(PRGDIR)/lorem.c loremOut.c datagencli.c
$(LINK.c) $^ -o $@$(EXT)
CLEAN += roundTripCrash
roundTripCrash: CFLAGS += $(MULTITHREAD)
roundTripCrash : $(ZSTD_OBJECTS) roundTripCrash.c
CLEAN += longmatch
longmatch : $(ZSTD_OBJECTS) longmatch.c
CLEAN += largeDictionary
largeDictionary: CFLAGS += $(MULTITHREAD)
largeDictionary: $(ZSTDMT_OBJECTS) $(PRGDIR)/datagen.c largeDictionary.c
CLEAN += invalidDictionaries
invalidDictionaries : $(ZSTD_OBJECTS) invalidDictionaries.c
# Exercise the public static archive as an external C consumer would. This
# catches a nested Rust archive or unresolved C/Rust cycle that direct object
# links cannot expose.
CLEAN += rustLibSmoke
rustLibSmoke : $(LIB_BINDIR)/libzstd.a rustLibSmoke.c
$(LINK.c) $(filter %.c,$^) $(filter %.a,$^) -o $@$(EXT)
CLEAN += legacy
legacy : CPPFLAGS += -UZSTD_LEGACY_SUPPORT -DZSTD_LEGACY_SUPPORT=4
legacy : $(ZSTD_FILES) legacy.c
CLEAN += decodecorpus
decodecorpus : LDLIBS += -lm
decodecorpus : $(filter-out zstdc_zstd_compress.o, $(ZSTD_OBJECTS)) $(ZDICT_FILES) $(PRGDIR)/util.c $(PRGDIR)/timefn.c decodecorpus.c
CLEAN += poolTests
poolTests : CPPFLAGS += -D_POSIX_C_SOURCE=200809L $(MULTITHREAD_CPP)
poolTests : $(ZSTDMT_OBJECTS) $(PRGDIR)/util.c $(PRGDIR)/timefn.c poolTests.c
$(LINK.c) $(MULTITHREAD_LD) $^ -o $@$(EXT)
# These static C test executables exercise Rust replacements. The normal
# prerequisite is also an archive input, placed after C sources by `$^`, and
# rebuilding Rust therefore relinks rather than leaving a stale implementation.
RUST_LINK_TARGETS := fullbench fuzzer zstreamtest \
zstreamtest_asan zstreamtest_tsan zstreamtest_ubsan \
paramgrill roundTripCrash longmatch largeDictionary \
invalidDictionaries legacy decodecorpus poolTests
$(RUST_LINK_TARGETS): $(RUST_STATICLIB)
RUST_LINK_TARGETS_32 := fullbench32 fuzzer32 zstreamtest32
$(RUST_LINK_TARGETS_32): $(RUST_STATICLIB_32)
$(RUST_LINK_TARGETS) $(RUST_LINK_TARGETS_32): $(RUST_HUF_C_MODE_STAMP)
# Tests that compile the timefn/benchfn C shims also link the Rust CLI
# helpers archive, which owns those implementations. The archive is a
# prerequisite so `$^` places it after every C object referencing its symbols.
RUST_CLI_LINK_TARGETS := fullbench fullbench-lib fullbench-dll fuzzer \
zstreamtest zstreamtest_asan zstreamtest_tsan \
zstreamtest_ubsan paramgrill decodecorpus poolTests
$(RUST_CLI_LINK_TARGETS): $(RUST_CLI_HELPERS_STATICLIB)
RUST_CLI_LINK_TARGETS_32 := fullbench32 fuzzer32 zstreamtest32
$(RUST_CLI_LINK_TARGETS_32): $(RUST_CLI_HELPERS_STATICLIB_32)
.PHONY: versionsTest
versionsTest: clean
$(PYTHON) test-zstd-versions.py
.PHONY: automated_benchmarking
automated_benchmarking: clean
$(PYTHON) automated_benchmarking.py
# make checkTag : check that release tag corresponds to release version
CLEAN += checkTag
checkTag.o : $(LIB_SRCDIR)/zstd.h
.PHONY: clean
clean:
$(MAKE) -C $(LIB_SRCDIR) clean
$(MAKE) -C $(PRGDIR) clean
$(MAKE) -C fuzz clean
$(RM) -R $(TESTARTEFACT)
$(RM) -r tmp* # some test directories are named tmp*
$(RM) $(CLEAN) core *.o *.tmp result* *.gcda dictionary *.zst \
$(PRGDIR)/zstd$(EXT) $(PRGDIR)/zstd32$(EXT) \
fullbench-dll$(EXT) fuzzer-dll$(EXT) zstreamtest-dll$(EXT)
@echo Cleaning completed
#----------------------------------------------------------------------------------
# valgrind tests validated only for some posix platforms
#----------------------------------------------------------------------------------
UNAME := $(shell sh -c 'MSYSTEM="MSYS" uname')
ifneq (,$(filter Linux Darwin GNU/kFreeBSD GNU OpenBSD FreeBSD NetBSD DragonFly SunOS AIX CYGWIN_NT%,$(UNAME)))
HOST_OS = POSIX
.PHONY: test-valgrind
test-valgrind: VALGRIND = valgrind --leak-check=full --show-leak-kinds=all --error-exitcode=1
test-valgrind: zstd datagen fuzzer fullbench
@echo "\n ---- valgrind tests : memory analyzer ----"
$(VALGRIND) ./datagen -g50M > $(VOID)
$(VALGRIND) $(PRGDIR)/zstd ; if [ $$? -eq 0 ] ; then echo "zstd without argument should have failed"; false; fi
./datagen -g80 | $(VALGRIND) $(PRGDIR)/zstd - -c > $(VOID)
./datagen -g16KB | $(VALGRIND) $(PRGDIR)/zstd -vf - -c > $(VOID)
./datagen -g2930KB | $(VALGRIND) $(PRGDIR)/zstd -5 -vf - -o tmp
$(VALGRIND) $(PRGDIR)/zstd -vdf tmp -c > $(VOID)
./datagen -g64MB | $(VALGRIND) $(PRGDIR)/zstd -vf - -c > $(VOID)
$(RM) tmp
$(VALGRIND) ./fuzzer -T1mn -t1
$(VALGRIND) ./fullbench -i1
endif
ifneq (,$(filter MINGW% MSYS%,$(UNAME)))
HOST_OS = MSYS
endif
#-----------------------------------------------------------------------------
# make tests validated only for below targets
#-----------------------------------------------------------------------------
ifneq (,$(filter MSYS POSIX,$(HOST_OS)))
DIFF:=diff
ifneq (,$(filter SunOS,$(UNAME)))
DIFF:=gdiff
endif
.PHONY: list
list:
@$(MAKE) -pRrq -f $(lastword $(MAKEFILE_LIST)) : 2>/dev/null | awk -v RS= -F: '/^# File/,/^# Finished Make data base/ {if ($$1 !~ "^[#.]") {print $$1}}' | sort | egrep -v -e '^[^[:alnum:]]' -e '^$@$$' | xargs
.PHONY: check
check: ZSTDRTTEST= # remove long tests
check: test-zstd
@echo "\n******************************"
@echo "All tests completed successfully"
@echo "******************************"
.PHONY: fuzztest
fuzztest: test-fuzzer test-zstream test-decodecorpus
.PHONY: test
test: test-zstd test-cli-tests test-fullbench test-fuzzer test-zstream test-invalidDictionaries test-rust-lib-smoke test-legacy test-decodecorpus
ifeq ($(QEMU_SYS),)
test: test-pool
endif
@echo "\n******************************"
@echo "All tests completed successfully"
@echo "******************************"
.PHONY: test32
test32: test-zstd32 test-fullbench32 test-fuzzer32 test-zstream32
.PHONY: test-all
test-all: test test32 test-decodecorpus-cli
.PHONY: test-zstd test-zstd32 test-zstd-nolegacy
test-zstd: ZSTD = $(PRGDIR)/zstd
test-zstd: zstd
.PHONY: test-zstd-dll
test-zstd-dll: ZSTD = $(PRGDIR)/zstd
test-zstd-dll: zstd-dll
test-zstd32: ZSTD = $(PRGDIR)/zstd32
test-zstd32: zstd32
test-zstd-nolegacy: ZSTD = $(PRGDIR)/zstd-nolegacy
test-zstd-nolegacy: zstd-nolegacy
test-zstd test-zstd32 test-zstd-nolegacy test-zstd-dll: datagen
file $(ZSTD)
EXE_PREFIX="$(QEMU_SYS)" ZSTD_BIN="$(ZSTD)" DATAGEN_BIN=./datagen ./playTests.sh $(ZSTDRTTEST)
.PHONY: test-cli-tests
test-cli-tests: ZSTD = $(PRGDIR)/zstd
test-cli-tests: zstd datagen
file $(ZSTD)
./cli-tests/run.py --exec-prefix="$(QEMU_SYS)" --zstd="$(ZSTD)" --datagen=./datagen
.PHONY: test-fullbench
test-fullbench: fullbench datagen
$(QEMU_SYS) ./fullbench -i1
$(QEMU_SYS) ./fullbench -i1 -P0
.PHONY: test-fullbench32
test-fullbench32: fullbench32 datagen
$(QEMU_SYS) ./fullbench32 -i1
$(QEMU_SYS) ./fullbench32 -i1 -P0
.PHONY: test-fuzzer
test-fuzzer: fuzzer
$(QEMU_SYS) ./fuzzer -v $(FUZZERTEST) $(FUZZER_FLAGS)
# Note : this test presumes `fuzzer` will be built
.PHONY: test-fuzzer-stackmode
test-fuzzer-stackmode: MOREFLAGS += -DZSTD_HEAPMODE=0
test-fuzzer-stackmode: test-fuzzer
.PHONY: test-fuzzer32
test-fuzzer32: fuzzer32
$(QEMU_SYS) ./fuzzer32 -v $(FUZZERTEST) $(FUZZER_FLAGS)
.PHONY: test-zstream
test-zstream: zstreamtest
$(QEMU_SYS) ./zstreamtest -v $(ZSTREAM_TESTTIME) $(FUZZER_FLAGS)
$(QEMU_SYS) ./zstreamtest --newapi -t1 $(ZSTREAM_TESTTIME) $(FUZZER_FLAGS)
test-zstream32: zstreamtest32
$(QEMU_SYS) ./zstreamtest32 -v $(ZSTREAM_TESTTIME) $(FUZZER_FLAGS)
test-longmatch: longmatch
$(QEMU_SYS) ./longmatch
test-largeDictionary: largeDictionary
$(QEMU_SYS) ./largeDictionary
test-invalidDictionaries: invalidDictionaries
$(QEMU_SYS) ./invalidDictionaries
.PHONY: test-rust-lib-smoke
test-rust-lib-smoke: rustLibSmoke
$(QEMU_SYS) ./rustLibSmoke
test-legacy: legacy
$(QEMU_SYS) ./legacy
test-decodecorpus: decodecorpus
$(QEMU_SYS) ./decodecorpus -t $(DECODECORPUS_TESTTIME)
test-decodecorpus-cli: decodecorpus
@echo "\n ---- decodecorpus basic cli tests ----"
@mkdir testdir
./decodecorpus -n5 -otestdir -ptestdir
@cd testdir && \
$(ZSTD) -d z000000.zst -o tmp0 && \
$(ZSTD) -d z000001.zst -o tmp1 && \
$(ZSTD) -d z000002.zst -o tmp2 && \
$(ZSTD) -d z000003.zst -o tmp3 && \
$(ZSTD) -d z000004.zst -o tmp4 && \
diff z000000 tmp0 && \
diff z000001 tmp1 && \
diff z000002 tmp2 && \
diff z000003 tmp3 && \
diff z000004 tmp4 && \
rm ./* && \
cd ..
@echo "\n ---- decodecorpus dictionary cli tests ----"
./decodecorpus -n5 -otestdir -ptestdir --use-dict=1MB
@cd testdir && \
$(ZSTD) -d z000000.zst -D dictionary -o tmp0 && \
$(ZSTD) -d z000001.zst -D dictionary -o tmp1 && \
$(ZSTD) -d z000002.zst -D dictionary -o tmp2 && \
$(ZSTD) -d z000003.zst -D dictionary -o tmp3 && \
$(ZSTD) -d z000004.zst -D dictionary -o tmp4 && \
diff z000000 tmp0 && \
diff z000001 tmp1 && \
diff z000002 tmp2 && \
diff z000003 tmp3 && \
diff z000004 tmp4 && \
cd ..
@rm -rf testdir
test-pool: poolTests
$(QEMU_SYS) ./poolTests
test-lz4: ZSTD = LD_LIBRARY_PATH=/usr/local/lib $(PRGDIR)/zstd
test-lz4: ZSTD_LZ4 = LD_LIBRARY_PATH=/usr/local/lib ./lz4
test-lz4: ZSTD_UNLZ4 = LD_LIBRARY_PATH=/usr/local/lib ./unlz4
test-lz4: zstd decodecorpus datagen
[ -f lz4 ] || ln -s $(PRGDIR)/zstd lz4
[ -f unlz4 ] || ln -s $(PRGDIR)/zstd unlz4
./decodecorpus -ptmp
# lz4 -> zstd
lz4 < tmp | \
$(ZSTD) -d | \
cmp - tmp
lz4 < tmp | \
$(ZSTD_UNLZ4) | \
cmp - tmp
# zstd -> lz4
$(ZSTD) --format=lz4 < tmp | \
lz4 -d | \
cmp - tmp
$(ZSTD_LZ4) < tmp | \
lz4 -d | \
cmp - tmp
# zstd -> zstd
$(ZSTD) --format=lz4 < tmp | \
$(ZSTD) -d | \
cmp - tmp
# zstd -> zstd
$(ZSTD) < tmp | \
$(ZSTD) -d | \
cmp - tmp
./datagen -g384KB | $(ZSTD) --format=lz4 | $(ZSTD) -d > /dev/null
rm tmp lz4 unlz4
endif
Reference in New Issue View Git Blame Copy Permalink