Commit Graph
161 Commits
Author SHA1 Message Date
Yann Collet 59a7116cc2 benchfn dependencies reduced to only timefn
benchfn used to rely on mem.h, and util,
which in turn relied on platform.h.
Using benchfn outside of zstd required to bring all these dependencies.

Now, dependency is reduced to timefn only.
This required to create a separate timefn from util,
and rewrite benchfn and timefn to no longer need mem.h.

Separating timefn from util has a wide effect accross the code base,
as usage of time functions is widespread.
A lot of build scripts had to be updated to also include timefn.
2019-04-10 12:37:03 -07:00
Peter (Stig) EdwardsandGitHub 4a9e0502e6 -Wformat-security not needed with -Wformat=2 2019-02-01 09:29:08 +00:00
Yann Collet ededcfca57 fix confusion between unsigned <-> U32
as suggested in #1441.

generally U32 and unsigned are the same thing,
except when they are not ...

case : 32-bit compilation for MIPS (uint32_t == unsigned long)

A vast majority of transformation consists in transforming U32 into unsigned.
In rare cases, it's the other way around (typically for internal code, such as seeds).

Among a few issues this patches solves :
- some parameters were declared with type `unsigned` in *.h,
  but with type `U32` in their implementation *.c .
- some parameters have type unsigned*,
  but the caller user a pointer to U32 instead.

These fixes are useful.

However, the bulk of changes is about %u formating,
which requires unsigned type,
but generally receives U32 values instead,
often just for brevity (U32 is shorter than unsigned).
These changes are generally minor, or even annoying.

As a consequence, the amount of code changed is larger than I would expect for such a patch.

Testing is also a pain :
it requires manually modifying `mem.h`,
in order to lie about `U32`
and force it to be an `unsigned long` typically.
On a 64-bit system, this will break the equivalence unsigned == U32.
Unfortunately, it will also break a few static_assert(), controlling structure sizes.
So it also requires modifying `debug.h` to make `static_assert()` a noop.
And then reverting these changes.

So it's inconvenient, and as a consequence,
this property is currently not checked during CI tests.
Therefore, these problems can emerge again in the future.

I wonder if it is worth ensuring proper distinction of U32 != unsigned in CI tests.
It's another restriction for coding, adding more frustration during merge tests,
since most platforms don't need this distinction (hence contributor will not see it),
and while this can matter in theory, the number of platforms impacted seems minimal.

Thoughts ?
2018-12-21 18:09:41 -08:00
Yann Collet 3583d19c4e changed parameter names from ZSTD_p_* to ZSTD_c_*
for naming consistency
2018-12-05 17:26:02 -08:00
Rohit Jain 705e0b18ab Making changes to make it compile on my laptop 2018-10-11 15:51:57 -07:00
Ryan Schmidt b567ce9d68 Fix name of macOS 2018-06-09 14:31:17 -05:00
Yann Collet 5373e44ba7 fixed contrib/adaptive-compression 2018-03-15 17:10:15 -07:00
Yann Collet b0cb081dc8 last batch of header files changed to reflect new license (#825)
only remains to update contrib/linux-kernel (@terrelln)
2017-08-31 12:20:50 -07:00
Yann Collet e21384fffb fixed more file headers after license change (#825) 2017-08-31 12:11:57 -07:00
Paul Cruz 733ca51360 Updating README.md 2017-08-15 17:48:23 -07:00
Paul Cruz fc90469587 updated program name print statement 2017-08-10 16:11:59 -07:00
Paul Cruz 7069bb9458 Merge branch 'adapt-approach-4' into bug-fixes 2017-08-07 13:18:53 -07:00
Paul Cruz e100a311eb removed direct assignment of 22, used ZSTD_maxCLevel() instead 2017-08-07 13:11:07 -07:00
Paul Cruz 01237e3b35 changed multi to zstd-adaptive in the help menu 2017-08-03 15:13:49 -07:00
Paul Cruz 8be7bba08c added mutex for compression level to avoid data race 2017-08-02 10:27:33 -07:00
Paul Cruz 69ef22c0ac added detach statements to prevent resource leak 2017-08-01 17:36:13 -07:00
Paul Cruz 0295737ad7 change signal to broadcast for jobCompressed condition varaible since multiple threads waiting 2017-07-31 13:43:03 -07:00
Paul Cruz 9ea7df03de add install target in makefile 2017-07-31 11:04:17 -07:00
Paul Cruz f60cd3f99b print defaults and range, remove EXT 2017-07-31 09:47:09 -07:00
Paul Cruz e22b60cb76 removed ternary operation, added assert statement, check to make sure initial compression level is within bounds 2017-07-28 17:46:51 -07:00
Paul Cruz cb9af53e77 delete empty line 2017-07-28 17:28:25 -07:00
Paul Cruz 51788225db remove exe extension from makefile, reinclude pthread flag 2017-07-28 17:27:36 -07:00
Paul Cruz 4d904ac800 add flags for multithreading 2017-07-28 16:12:58 -07:00
Paul Cruz 0f4cb67b00 add tests for compression bounds, fix another warning 2017-07-28 15:55:02 -07:00
Paul Cruz ff54fced64 patched style errors, add ability to bound compression level variation 2017-07-28 15:30:46 -07:00
Paul Cruz 2320e7378a remove unused variable, add documentation for context fields 2017-07-26 17:02:47 -07:00
Paul Cruz ab5a78547e fix leaky abstraction regarding measuring completion 2017-07-26 16:40:05 -07:00
Paul Cruz 715f36ca81 added definitions for conversion constants, moved forced compression check to top of adaptCompressionLevel, used ZSTD_BLOCKSIZE_MAX 2017-07-26 15:52:15 -07:00
Paul Cruz 6c1c1242fc set the window log value before performing compression 2017-07-26 14:29:59 -07:00
Paul Cruz a959cc881a moved reset of completion to right after wait 2017-07-26 10:34:48 -07:00
Paul Cruz 305d5ee70f change to >= convergence counter 2017-07-26 10:20:29 -07:00
Paul Cruz be92a38d6a decrease completion requirements for change, move create thread wait, merge cases where compression thread should wait 2017-07-26 10:05:10 -07:00
Paul Cruz 0b18d21e03 building on readme, added another help tip in the menu 2017-07-25 17:47:02 -07:00
Paul Cruz 7cc74e0b27 adding more to readme 2017-07-25 16:55:16 -07:00
Paul Cruz 8dbb07d822 updated progress bar with better representation of time, added const 2017-07-25 16:03:43 -07:00
Paul Cruz 9a132707af changing time units to seconds 2017-07-25 15:26:26 -07:00
Paul Cruz 31a9ed9883 updated const values, added more comments 2017-07-25 14:53:40 -07:00
Paul Cruz 5cfbf609a4 removed old debug statements no longer being used 2017-07-25 14:31:48 -07:00
Paul Cruz 0882cd1981 progress bar -- don't print num jobs, time elapsed shown in seconds 2017-07-25 14:26:55 -07:00
Paul Cruz 310c12d07e moved debug statements to a compiler flag 2017-07-25 14:08:39 -07:00
Paul Cruz e02c79f833 started using decrease cooldown so that compression level would not decrease several times in a row 2017-07-25 11:16:27 -07:00
Paul Cruz 85d7c919f6 created independent function for controlling how completion relates to compression level change 2017-07-25 10:32:14 -07:00
Paul Cruz 6f1e260edd added mechanism for getting rid of spikes 2017-07-25 10:01:10 -07:00
Paul Cruz 700758d676 added help statement for -p, switched it to hide progress bar now that progress bar is default 2017-07-24 16:26:20 -07:00
Paul Cruz df3754b6ed add quiet option, make progress bar default 2017-07-24 16:19:07 -07:00
Paul Cruz 4dc83ca64c compression thread should take measurements independently based on whether or not the create/write thread will actually bottleneck performance 2017-07-24 15:14:58 -07:00
Paul Cruz 0ee3f8c2f8 adding more debug 2017-07-24 15:06:11 -07:00
Paul Cruz 8328f8192a updating debug statements again 2017-07-24 14:40:23 -07:00
Paul Cruz d3d759301f changing position of endline for debug 2017-07-24 13:47:39 -07:00
Paul Cruz e508f632d6 updated comments and debug statements 2017-07-24 11:01:36 -07:00