ddidderr/zstd-rs
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
Files
4f6a8f018505317a116447d1fff83451f2d79808
zstd-rs/build/meson/lib/meson.build
T
ddidderr eecd2e168a build(rust): add dict-builder cargo feature dimension 
The dictionary-builder sources (lib/dictBuilder) are about to start moving
to Rust, beginning with divsufsort. The Rust crate previously only modeled
the compression/decompression module split plus the forced-HUF decoder
modes, so no build could express "this C configuration includes (or
excludes) dictBuilder" to Cargo. Without that, a Rust archive could carry
dictBuilder modules into a build whose C side disabled them, or worse,
omit a migrated implementation from a build whose C shims require it.

Add a `dict-builder` cargo feature and thread it through every build that
consumes the Rust static archive, mirroring exactly how each build system
already gates the dictBuilder C sources:

- rust/Cargo.toml: new `dict-builder` feature, included in the default
  set because the C library builds dictBuilder by default
  (ZSTD_LIB_DICTBUILDER ?= 1). The feature is empty until the first
  dictBuilder module lands.
- lib/Makefile: RUST_CARGO_FEATURES gains dict-builder when
  ZSTD_LIB_DICTBUILDER is enabled, following the existing
  ZSTD_LIB_COMPRESSION/ZSTD_LIB_DECOMPRESSION pattern. The archive
  directory naming grows a matching `b<0|1>` dimension
  (c1-d1-b1-default etc.) so differently configured archives never
  collide; the repeated config prefix is factored into
  RUST_MODULE_CONFIG.
- programs/Makefile: the full-featured archives now request
  compression,decompression,dict-builder (equal to the default set, so
  the target directory stays shared with tests). The partial-library
  variants gain the `b0` name dimension, and zstd-dictBuilder gets its
  own lib-c1-d0-b1 archive because it compiles the dictBuilder C sources
  without decompression; it previously shared the compression-only
  archive, which will lack the migrated dictBuilder symbols.
- tests/Makefile: no flag change needed since tests use the crate default
  feature set; a comment now records that dict-builder arrives that way.
- build/cmake/lib/CMakeLists.txt: ZSTD_BUILD_DICTBUILDER now adds the
  dict-builder feature and a `b<0|1>` component in the Rust build-config
  directory name, in lockstep with the DictBuilderSources gating.
- build/meson/lib/meson.build: meson compiles the dictBuilder sources
  unconditionally, so the feature list and config name gain dict-builder
  unconditionally (c1-d1-b1-<huf-mode>).

The `dict-builder` feature deliberately does not imply `compression`.
lib/Makefile forces ZSTD_LIB_DICTBUILDER=0 when compression is disabled,
but CMake does not couple the two options, so encoding the C-side
constraint in Cargo would make the Rust archive diverge from the C source
list in that (already unsupported) CMake configuration.

Test plan:
- cd rust && cargo build --release
- cargo build --release --no-default-features \
    --features compression,decompression
- cargo build --release --no-default-features \
    --features compression,dict-builder
- Full validation (fuzzer, smoke tests, dictionary byte-identity) runs
  with the follow-up commit that ports divsufsort onto this scaffolding.
2026-07-11 09:38:04 +02:00

311 lines
12 KiB
Meson
Raw Blame History

# #############################################################################
# Copyright (c) 2018-present Dima Krasner <dima@dimakrasner.com>
# lzutao <taolzu(at)gmail.com>
# All rights reserved.
#
# This source code is licensed under both the BSD-style license (found in the
# LICENSE file in the root directory of this source tree) and the GPLv2 (found
# in the COPYING file in the root directory of this source tree).
# #############################################################################
zstd_rootdir = '../../..'
libzstd_includes = [include_directories(join_paths(zstd_rootdir,'lib'),
join_paths(zstd_rootdir, 'lib/common'),
join_paths(zstd_rootdir, 'lib/compress'),
join_paths(zstd_rootdir, 'lib/decompress'),
join_paths(zstd_rootdir, 'lib/dictBuilder'))]
libzstd_sources = [join_paths(zstd_rootdir, 'lib/common/entropy_common.c'),
join_paths(zstd_rootdir, 'lib/common/fse_decompress.c'),
join_paths(zstd_rootdir, 'lib/common/threading.c'),
join_paths(zstd_rootdir, 'lib/common/pool.c'),
join_paths(zstd_rootdir, 'lib/common/zstd_common.c'),
join_paths(zstd_rootdir, 'lib/common/error_private.c'),
join_paths(zstd_rootdir, 'lib/common/xxhash.c'),
join_paths(zstd_rootdir, 'lib/compress/hist.c'),
join_paths(zstd_rootdir, 'lib/compress/fse_compress.c'),
join_paths(zstd_rootdir, 'lib/compress/huf_compress.c'),
join_paths(zstd_rootdir, 'lib/compress/zstd_compress.c'),
join_paths(zstd_rootdir, 'lib/compress/zstd_compress_literals.c'),
join_paths(zstd_rootdir, 'lib/compress/zstd_compress_sequences.c'),
join_paths(zstd_rootdir, 'lib/compress/zstd_compress_superblock.c'),
join_paths(zstd_rootdir, 'lib/compress/zstd_preSplit.c'),
join_paths(zstd_rootdir, 'lib/compress/zstdmt_compress.c'),
join_paths(zstd_rootdir, 'lib/compress/zstd_fast.c'),
join_paths(zstd_rootdir, 'lib/compress/zstd_double_fast.c'),
join_paths(zstd_rootdir, 'lib/compress/zstd_lazy.c'),
join_paths(zstd_rootdir, 'lib/compress/zstd_opt.c'),
join_paths(zstd_rootdir, 'lib/compress/zstd_ldm.c'),
join_paths(zstd_rootdir, 'lib/decompress/huf_decompress.c'),
join_paths(zstd_rootdir, 'lib/decompress/zstd_decompress.c'),
join_paths(zstd_rootdir, 'lib/decompress/zstd_decompress_block.c'),
join_paths(zstd_rootdir, 'lib/decompress/zstd_ddict.c'),
join_paths(zstd_rootdir, 'lib/dictBuilder/cover.c'),
join_paths(zstd_rootdir, 'lib/dictBuilder/fastcover.c'),
join_paths(zstd_rootdir, 'lib/dictBuilder/divsufsort.c'),
join_paths(zstd_rootdir, 'lib/dictBuilder/zdict.c')]
# C compatibility shims in the source list above are implemented by the Rust
# static library. Build a Cargo archive in the Meson build directory so its
# configuration cannot leak into another Meson setup.
rust_huf_force_x1 = get_option('huf_force_decompress_x1')
rust_huf_force_x2 = get_option('huf_force_decompress_x2')
rust_uses_m32 = false
foreach c_arg : get_option('c_args')
if c_arg == '-m32'
rust_uses_m32 = true
endif
if c_arg == '-DHUF_FORCE_DECOMPRESS_X1' or \
c_arg.startswith('-DHUF_FORCE_DECOMPRESS_X1=') or \
c_arg == '/DHUF_FORCE_DECOMPRESS_X1' or \
c_arg.startswith('/DHUF_FORCE_DECOMPRESS_X1=')
rust_huf_force_x1 = true
endif
if c_arg == '-DHUF_FORCE_DECOMPRESS_X2' or \
c_arg.startswith('-DHUF_FORCE_DECOMPRESS_X2=') or \
c_arg == '/DHUF_FORCE_DECOMPRESS_X2' or \
c_arg.startswith('/DHUF_FORCE_DECOMPRESS_X2=')
rust_huf_force_x2 = true
endif
endforeach
if rust_huf_force_x1 and rust_huf_force_x2
error('HUF_FORCE_DECOMPRESS_X1 and HUF_FORCE_DECOMPRESS_X2 are mutually exclusive')
endif
# Meson always compiles the dictBuilder sources above, so the Rust archive
# must always carry the matching dict-builder module set.
rust_features = ['compression', 'decompression', 'dict-builder']
rust_huf_mode = 'default'
rust_huf_c_args = []
if rust_huf_force_x1
rust_features += 'huf-force-decompress-x1'
rust_huf_mode = 'huf-force-decompress-x1'
rust_huf_c_args += '-DHUF_FORCE_DECOMPRESS_X1'
elif rust_huf_force_x2
rust_features += 'huf-force-decompress-x2'
rust_huf_mode = 'huf-force-decompress-x2'
rust_huf_c_args += '-DHUF_FORCE_DECOMPRESS_X2'
endif
rust_target = get_option('zstd_rust_target')
if rust_target == ''
if host_machine_os == os_linux and \
(host_machine.cpu_family() == 'x86' or rust_uses_m32)
rust_target = 'i686-unknown-linux-gnu'
elif meson.is_cross_build()
error('Set -Dzstd_rust_target to the Cargo target triple for this cross build')
endif
endif
rust_build_config = 'c1-d1-b1-' + rust_huf_mode
rust_target_dir = join_paths(meson.current_build_dir(), 'rust-target', rust_build_config)
is_msvc = cc_id == compiler_msvc or cc_id == 'clang-cl'
rust_staticlib_name = is_msvc ? 'zstd_rs.lib' : 'libzstd_rs.a'
zstd_staticlib_name = is_msvc ? 'zstd.lib' : 'libzstd.a'
cargo = find_program('cargo', required: true)
python = find_program('python3', 'python', required: true)
rust_manifest = files(join_paths(zstd_rootdir, 'rust/Cargo.toml'))
rust_lockfile = files(join_paths(zstd_rootdir, 'rust/Cargo.lock'))
cargo_staticlib_py = files('cargo_staticlib.py')
rust_archive_command = [
python,
cargo_staticlib_py,
'--cargo', cargo,
'--manifest-path', rust_manifest,
'--target-dir', rust_target_dir,
'--staticlib-name', rust_staticlib_name,
'--output', '@OUTPUT@',
'--features', ','.join(rust_features),
]
if rust_target != ''
rust_archive_command += ['--target', rust_target]
endif
rust_archive = custom_target('zstd_rust_archive',
output: rust_staticlib_name,
command: rust_archive_command,
depend_files: [rust_lockfile],
build_always_stale: true)
# Meson's c_args controls compilation only. A native -m32 configuration also
# needs its final C and C++ link commands to select the i686 linker emulation.
if rust_uses_m32
m32_link_args = cc.get_supported_link_arguments('-m32')
add_project_link_arguments(m32_link_args, language: ['c', 'cpp'])
endif
# really we need anything that defines __GNUC__ as that is what ZSTD_ASM_SUPPORTED is gated on
# but these are the two compilers that are supported in tree and actually handle this correctly.
# The assembly source is AMD64-only: do not add it for an i686 (-m32) or other
# cross target, even when the build host itself is x86_64.
if [compiler_gcc, compiler_clang].contains(cc_id) and \
['x86_64', 'amd64'].contains(host_machine.cpu_family()) and not rust_uses_m32
libzstd_sources += join_paths(zstd_rootdir, 'lib/decompress/huf_decompress_amd64.S')
else
add_project_arguments('-DZSTD_DISABLE_ASM', language: 'c')
endif
# Explicit define legacy support
add_project_arguments('-DZSTD_LEGACY_SUPPORT=@0@'.format(legacy_level),
language: 'c')
if legacy_level == 0
message('Legacy support: DISABLED')
else
# See ZSTD_LEGACY_SUPPORT of lib/README.md
message('Enable legacy support back to version 0.@0@'.format(legacy_level))
libzstd_includes += [ include_directories(join_paths(zstd_rootdir, 'lib/legacy')) ]
foreach i : [1, 2, 3, 4, 5, 6, 7]
if legacy_level <= i
libzstd_sources += join_paths(zstd_rootdir, 'lib/legacy/zstd_v0@0@.c'.format(i))
endif
endforeach
endif
libzstd_deps = []
if use_multi_thread
message('Enable multi-threading support')
add_project_arguments('-DZSTD_MULTITHREAD', language: 'c')
libzstd_deps = [ thread_dep ]
endif
libzstd_c_args = []
if cc_id == compiler_msvc
if default_library_type != 'static'
libzstd_sources += [windows_mod.compile_resources(
join_paths(zstd_rootdir, 'build/VS2010/libzstd-dll/libzstd-dll.rc'),
include_directories: libzstd_includes)]
libzstd_c_args += ['-DZSTD_DLL_EXPORT=1',
'-DZSTD_HEAPMODE=0',
'-D_CONSOLE',
'-D_CRT_SECURE_NO_WARNINGS']
else
libzstd_c_args += ['-DZSTD_HEAPMODE=0',
'-D_CRT_SECURE_NO_WARNINGS']
endif
endif
mingw_ansi_stdio_flags = []
if host_machine_os == os_windows and cc_id == compiler_gcc
mingw_ansi_stdio_flags = [ '-D__USE_MINGW_ANSI_STDIO' ]
endif
libzstd_c_args += mingw_ansi_stdio_flags
libzstd_c_args += rust_huf_c_args
libzstd_debug_cflags = []
if use_debug
libzstd_c_args += '-DDEBUGLEVEL=@0@'.format(debug_level)
if cc_id == compiler_gcc or cc_id == compiler_clang
libzstd_debug_cflags = ['-Wstrict-aliasing=1', '-Wswitch-enum',
'-Wdeclaration-after-statement', '-Wstrict-prototypes',
'-Wundef', '-Wpointer-arith', '-Wvla',
'-Wformat=2', '-Winit-self', '-Wfloat-equal', '-Wwrite-strings',
'-Wredundant-decls', '-Wmissing-prototypes', '-Wc++-compat']
endif
endif
libzstd_c_args += cc.get_supported_arguments(libzstd_debug_cflags)
# Keep the C archive separate until Cargo has finished. The final public
# static archive below contains the object members of both archives, which is
# necessary because C shims call Rust and Rust calls C helpers.
build_static = default_library_type != 'shared'
libzstd_c_static = static_library('zstd_c',
libzstd_sources,
include_directories: libzstd_includes,
c_args: libzstd_c_args,
gnu_symbol_visibility: 'hidden',
dependencies: libzstd_deps,
build_by_default: build_static)
rust_archiver_name = get_option('zstd_rust_archiver')
if rust_archiver_name == ''
rust_archiver_name = is_msvc ? 'lib' : 'ar'
endif
rust_archiver = find_program(rust_archiver_name, required: true)
merge_rust_archive_py = files('merge_rust_archive.py')
merge_rust_archive_command = [
python,
merge_rust_archive_py,
'--archiver', rust_archiver,
'--c-archive', '@INPUT0@',
'--rust-archive', '@INPUT1@',
'--output', '@OUTPUT@',
]
if is_msvc
merge_rust_archive_command += '--msvc'
endif
libzstd_static = custom_target('zstd_static',
input: [libzstd_c_static, rust_archive],
output: zstd_staticlib_name,
command: merge_rust_archive_command,
install: build_static,
install_dir: get_option('libdir'),
build_by_default: build_static)
libzstd_static_dep = declare_dependency(
sources: libzstd_static,
dependencies: libzstd_deps,
include_directories: join_paths(zstd_rootdir, 'lib'))
# We link to both a shared library (for public symbols) and the C-only static
# archive (for private symbols). The latter must stay C-only here: the Rust
# symbols are already supplied by the shared library. MSVC rejects this
# combination, so its internal users link the flattened static archive.
if default_library_type == 'static'
libzstd_dep = libzstd_static_dep
libzstd_internal_dep = declare_dependency(
sources: libzstd_static,
dependencies: libzstd_deps,
include_directories: libzstd_includes)
else
if host_machine_os == os_darwin
rust_shared_link_args = ['-Wl,-force_load,' + rust_archive.full_path()]
elif is_msvc
rust_shared_link_args = ['/WHOLEARCHIVE:' + rust_archive.full_path()]
else
rust_shared_link_args = [
'-Wl,--whole-archive',
rust_archive.full_path(),
'-Wl,--no-whole-archive',
]
endif
libzstd_shared = shared_library('zstd',
libzstd_sources,
include_directories: libzstd_includes,
c_args: libzstd_c_args,
gnu_symbol_visibility: 'hidden',
dependencies: libzstd_deps,
link_args: rust_shared_link_args,
link_depends: rust_archive,
install: true,
version: zstd_libversion)
libzstd_dep = declare_dependency(link_with: libzstd_shared,
include_directories: join_paths(zstd_rootdir, 'lib')) # Do not expose private headers
if is_msvc
libzstd_internal_dep = declare_dependency(
sources: libzstd_static,
dependencies: libzstd_deps,
include_directories: libzstd_includes)
else
libzstd_internal_dep = declare_dependency(link_with: libzstd_shared,
# the static library must be linked after the shared one
dependencies: declare_dependency(link_with: libzstd_c_static),
include_directories: libzstd_includes)
endif
endif
pkgconfig.generate(
name: 'libzstd',
filebase: 'libzstd',
description: 'fast lossless compression algorithm library',
version: zstd_libversion,
url: 'https://facebook.github.io/zstd/',
libraries: ['-L${libdir}', '-lzstd'],
libraries_private: libzstd_deps)
install_headers(join_paths(zstd_rootdir, 'lib/zstd.h'),
join_paths(zstd_rootdir, 'lib/zdict.h'),
join_paths(zstd_rootdir, 'lib/zstd_errors.h'))
Reference in New Issue View Git Blame Copy Permalink