Add doc/note/zero-initialization.md

This commit is contained in:
Nigel Tao committed 2019-11-14 20:45:48 +11:00
1 parent cb7e9e0ce9
commit f10849d20b
4 files changed
+65 -32

No files matched your search

+60
View File
@@ -0,0 +1,60 @@
# Zero Initialization
Memory-safe programming languages typically initialize their variables and
fields to zero, so that there is no way to read an uninitialized value. By
default, Wuffs does so too, but in Wuffs' C/C++ form, there is the option to
leave some struct fields uninitialized (although local variables are always
zero-initialized), for performance reasons:
```
uint32_t flags = 0;
etc
// Uncomment this line to take the option.
// flags |= WUFFS_INITIALIZE__LEAVE_INTERNAL_BUFFERS_UNINITIALIZED;
wuffs_foo__bar__initialize(etc, flags);
```
With or without this flag bit set, the Wuffs compiler still enforces bounds and
arithmetic overflow checks. It's just that for potentially-uninitialized struct
fields, the compiler has weaker starting assumptions: their numeric types
cannot be
[refined](https://github.com/google/wuffs/blob/master/doc/glossary.md#refinement-type).
Even with this flag bit set, the Wuffs standard library also considers reading
from an uninitialized buffer to be a bug, and strives to never do so, but
unlike buffer out-of-bounds reads or writes, it is not a bug class that the
Wuffs compiler eliminates.
For those paranoid about security, leave this flag bit unset, so that
`wuffs_foo__bar__initialize` will zero-initialize the entire struct (unless the
`WUFFS_INITIALIZE__ALREADY_ZEROED` flag bit is also set).
Setting this flag bit (avoiding a fixed-size cost) gives a small absolute
improvement on micro-benchmarks, mostly noticable (in relative terms) only when
the actual work to do (the input) is also small. Look for
`WUFFS_INITIALIZE__LEAVE_INTERNAL_BUFFERS_UNINITIALIZED` in the
[benchmarks](https://github.com/google/wuffs/blob/master/doc/benchmarks.md) for
performance numbers.
In Wuffs code, a struct definition has two parts, although the second part's
`()` parentheses may be omitted if empty:
```
pub struct bar?(
// Fields in the first part are always zero-initialized.
x : etc,
y : etc,
)(
// Fields in the second part are optionally uninitialized, but are still
// zero-initialized by default.
//
// Valid types for these fields are either unrefined numerical types, or
// arrays of a valid type.
// - "base.u8" is ok.
// - "array[123] base.u8" is ok.
// - "array[123] base.u8[0 ..= 99]" is not, due to the refinement.
z : etc,
)
```
+2 -15
View File
@@ -87,21 +87,8 @@
// that the buffer is contained by the receiver struct, as opposed to being
// passed as a separately allocated "work buffer".
//
// With or without this bit set, the Wuffs compiler still enforces that no
// reads or writes will overflow internal buffers' bounds. Even with this bit
// set, the Wuffs standard library also considers reading from an uninitialized
// buffer to be a bug, and strives to never do so, but unlike buffer overflows,
// it is not a bug class that the Wuffs compiler eliminates.
//
// For those paranoid about security, leave this bit unset, so that
// wuffs_foo__bar__initialize will initialize the entire struct value to zeroes
// (unless WUFFS_INITIALIZE__ALREADY_ZEROED is set).
//
// Setting this bit (avoiding a fixed-size cost) gives a small absolute
// improvement on micro-benchmarks, mostly noticable (in relative terms) only
// when the actual work to do (i.e. the input) is also small. Look for
// WUFFS_INITIALIZE__LEAVE_INTERNAL_BUFFERS_UNINITIALIZED in
// https://github.com/google/wuffs/blob/master/doc/benchmarks.md for numbers.
// For more detail, see:
// https://github.com/google/wuffs/blob/master/doc/note/zero-initialization.md
#define WUFFS_INITIALIZE__LEAVE_INTERNAL_BUFFERS_UNINITIALIZED \
((uint32_t)0x00000002)
+1 -2
View File
@@ -64,8 +64,7 @@ const baseCorePublicH = "" +
"// Wuffs assumes that:\n// - converting a uint32_t to a size_t will never overflow.\n// - converting a size_t to a uint64_t will never overflow.\n#ifdef __WORDSIZE\n#if (__WORDSIZE != 32) && (__WORDSIZE != 64)\n#error \"Wuffs requires a word size of either 32 or 64 bits\"\n#endif\n#endif\n\n// WUFFS_VERSION is the major.minor.patch version, as per https://semver.org/,\n// as a uint64_t. The major number is the high 32 bits. The minor number is the\n// middle 16 bits. The patch number is the low 16 bits. The pre-release label\n// and build metadata are part of the string representation (such as\n// \"1.2.3-beta+456.20181231\") but not the uint64_t representation.\n//\n// WUFFS_VERSION_PRE_RELEASE_LABEL (such as \"\", \"beta\" or \"rc.1\") being\n// non-empty denotes a developer preview, not a release version, and has no\n// backwards or forwards compatibility guarantees.\n//\n// WUFFS_VERSION_BUILD_METADATA_XXX, if non-zero, are the number of commits and\n// the last commit date in the repository used to build this library. Within\n// eac" +
"h major.minor branch, the commit count should increase monotonically.\n//\n// !! Some code generation programs can override WUFFS_VERSION.\n#define WUFFS_VERSION ((uint64_t)0)\n#define WUFFS_VERSION_MAJOR ((uint64_t)0)\n#define WUFFS_VERSION_MINOR ((uint64_t)0)\n#define WUFFS_VERSION_PATCH ((uint64_t)0)\n#define WUFFS_VERSION_PRE_RELEASE_LABEL \"work.in.progress\"\n#define WUFFS_VERSION_BUILD_METADATA_COMMIT_COUNT 0\n#define WUFFS_VERSION_BUILD_METADATA_COMMIT_DATE 0\n#define WUFFS_VERSION_STRING \"0.0.0+0.00000000\"\n\n// Define WUFFS_CONFIG__STATIC_FUNCTIONS to make all of Wuffs' functions have\n// static storage. The motivation is discussed in the \"ALLOW STATIC\n// IMPLEMENTATION\" section of\n// https://raw.githubusercontent.com/nothings/stb/master/docs/stb_howto.txt\n#ifdef WUFFS_CONFIG__STATIC_FUNCTIONS\n#define WUFFS_BASE__MAYBE_STATIC static\n#else\n#define WUFFS_BASE__MAYBE_STATIC\n#endif\n\n#if defined(__clang__)\n#define WUFFS_BASE__POTENTIALLY_UNUSED_FIELD __attribute__((unused))\n#else\n#define WUFFS_BASE__POTENTIALLY_UNUSED_" +
"FIELD\n#endif\n\n// Clang also defines \"__GNUC__\".\n#if defined(__GNUC__)\n#define WUFFS_BASE__POTENTIALLY_UNUSED __attribute__((unused))\n#define WUFFS_BASE__WARN_UNUSED_RESULT __attribute__((warn_unused_result))\n#else\n#define WUFFS_BASE__POTENTIALLY_UNUSED\n#define WUFFS_BASE__WARN_UNUSED_RESULT\n#endif\n\n// Flags for wuffs_foo__bar__initialize functions.\n\n#define WUFFS_INITIALIZE__DEFAULT_OPTIONS ((uint32_t)0x00000000)\n\n// WUFFS_INITIALIZE__ALREADY_ZEROED means that the \"self\" receiver struct value\n// has already been set to all zeroes.\n#define WUFFS_INITIALIZE__ALREADY_ZEROED ((uint32_t)0x00000001)\n\n// WUFFS_INITIALIZE__LEAVE_INTERNAL_BUFFERS_UNINITIALIZED means that, absent\n// WUFFS_INITIALIZE__ALREADY_ZEROED, only some of the \"self\" receiver struct\n// value will be set to all zeroes. Internal buffers, which tend to be a large\n// proportion of the struct's size, will be left uninitialized. Internal means\n// that the buffer is contained by the receiver struct, as opposed to being\n// passed as a separately allocate" +
"d \"work buffer\".\n//\n// With or without this bit set, the Wuffs compiler still enforces that no\n// reads or writes will overflow internal buffers' bounds. Even with this bit\n// set, the Wuffs standard library also considers reading from an uninitialized\n// buffer to be a bug, and strives to never do so, but unlike buffer overflows,\n// it is not a bug class that the Wuffs compiler eliminates.\n//\n// For those paranoid about security, leave this bit unset, so that\n// wuffs_foo__bar__initialize will initialize the entire struct value to zeroes\n// (unless WUFFS_INITIALIZE__ALREADY_ZEROED is set).\n//\n// Setting this bit (avoiding a fixed-size cost) gives a small absolute\n// improvement on micro-benchmarks, mostly noticable (in relative terms) only\n// when the actual work to do (i.e. the input) is also small. Look for\n// WUFFS_INITIALIZE__LEAVE_INTERNAL_BUFFERS_UNINITIALIZED in\n// https://github.com/google/wuffs/blob/master/doc/benchmarks.md for numbers.\n#define WUFFS_INITIALIZE__LEAVE_INTERNAL_BUFFERS_UNINITIALIZED " +
"\\\n ((uint32_t)0x00000002)\n\n" +
"d \"work buffer\".\n//\n// For more detail, see:\n// https://github.com/google/wuffs/blob/master/doc/note/zero-initialization.md\n#define WUFFS_INITIALIZE__LEAVE_INTERNAL_BUFFERS_UNINITIALIZED \\\n ((uint32_t)0x00000002)\n\n" +
"" +
"// --------\n\n// wuffs_base__empty_struct is used when a Wuffs function returns an empty\n// struct. In C, if a function f returns void, you can't say \"x = f()\", but in\n// Wuffs, if a function g returns empty, you can say \"y = g()\".\ntypedef struct {\n // private_impl is a placeholder field. It isn't explicitly used, except that\n // without it, the sizeof a struct with no fields can differ across C/C++\n // compilers, and it is undefined behavior in C99. For example, gcc says that\n // the sizeof an empty struct is 0, and g++ says that it is 1. This leads to\n // ABI incompatibility if a Wuffs .c file is processed by one compiler and\n // its .h file with another compiler.\n //\n // Instead, we explicitly insert an otherwise unused field, so that the\n // sizeof this struct is always 1.\n uint8_t private_impl;\n} wuffs_base__empty_struct;\n\nstatic inline wuffs_base__empty_struct //\nwuffs_base__make_empty_struct() {\n wuffs_base__empty_struct ret;\n ret.private_impl = 0;\n return ret;\n}\n\n// wuffs_base__utility is" +
" a placeholder receiver type. It enables what Java\n// calls static methods, as opposed to regular methods.\ntypedef struct {\n // private_impl is a placeholder field. It isn't explicitly used, except that\n // without it, the sizeof a struct with no fields can differ across C/C++\n // compilers, and it is undefined behavior in C99. For example, gcc says that\n // the sizeof an empty struct is 0, and g++ says that it is 1. This leads to\n // ABI incompatibility if a Wuffs .c file is processed by one compiler and\n // its .h file with another compiler.\n //\n // Instead, we explicitly insert an otherwise unused field, so that the\n // sizeof this struct is always 1.\n uint8_t private_impl;\n} wuffs_base__utility;\n\n" +
+2 -15
View File
@@ -109,21 +109,8 @@ extern "C" {
// that the buffer is contained by the receiver struct, as opposed to being
// passed as a separately allocated "work buffer".
//
// With or without this bit set, the Wuffs compiler still enforces that no
// reads or writes will overflow internal buffers' bounds. Even with this bit
// set, the Wuffs standard library also considers reading from an uninitialized
// buffer to be a bug, and strives to never do so, but unlike buffer overflows,
// it is not a bug class that the Wuffs compiler eliminates.
//
// For those paranoid about security, leave this bit unset, so that
// wuffs_foo__bar__initialize will initialize the entire struct value to zeroes
// (unless WUFFS_INITIALIZE__ALREADY_ZEROED is set).
//
// Setting this bit (avoiding a fixed-size cost) gives a small absolute
// improvement on micro-benchmarks, mostly noticable (in relative terms) only
// when the actual work to do (i.e. the input) is also small. Look for
// WUFFS_INITIALIZE__LEAVE_INTERNAL_BUFFERS_UNINITIALIZED in
// https://github.com/google/wuffs/blob/master/doc/benchmarks.md for numbers.
// For more detail, see:
// https://github.com/google/wuffs/blob/master/doc/note/zero-initialization.md
#define WUFFS_INITIALIZE__LEAVE_INTERNAL_BUFFERS_UNINITIALIZED \
((uint32_t)0x00000002)