From b9fd20bfe4c949d11b6eb31897180324dbfea0a6 Mon Sep 17 00:00:00 2001 From: Nigel Tao Date: Thu, 30 Nov 2017 18:15:36 +1100 Subject: [PATCH] Rename PUFFS to WUFFS, part 10 of N --- README.md | 108 +++++++++++++++++++------------------- doc/benchmarks.md | 72 ++++++++++++------------- doc/binary-size.md | 12 ++--- doc/compatibility.md | 2 +- doc/related-work.md | 54 +++++++++---------- doc/wuffs-the-language.md | 34 ++++++------ 6 files changed, 141 insertions(+), 141 deletions(-) diff --git a/README.md b/README.md index 8adb79bb..9b046819 100644 --- a/README.md +++ b/README.md @@ -1,16 +1,16 @@ # Wrangling Untrusted File Formats Safely ([Formerly known as -PUFFS](https://groups.google.com/d/topic/puffslang/cSrH-s7UqwA/discussion): +Puffs](https://groups.google.com/d/topic/puffslang/cSrH-s7UqwA/discussion): Parsing Untrusted File Formats Safely). -Puffs is a domain-specific language and library for parsing untrusted file +Wuffs is a domain-specific language and library for parsing untrusted file formats safely. Examples of such file formats include images, audio, video, fonts and compressed archives. -Unlike the C programming language, Puffs is safe with respect to buffer +Unlike the C programming language, Wuffs is safe with respect to buffer overflows, integer arithmetic overflows and null pointer dereferences. The key -difference between Puffs and other memory-safe languages is that all such +difference between Wuffs and other memory-safe languages is that all such checks are done at compile time, not at run time. *If it compiles, it is safe*, with respect to those three bug classes. @@ -20,37 +20,37 @@ are used. This includes very large C/C++ products, such as popular web browsers and operating systems (using that term to include desktop and mobile user interfaces, not just the kernel). -The trade-off in aiming for both safety and speed is that Puffs programs take +The trade-off in aiming for both safety and speed is that Wuffs programs take longer for a programmer to write, as they have to explicitly annotate their programs with proofs of safety. A statement like `x += 1` unsurprisingly means -to increment the variable `x` by `1`. However, in Puffs, such a statement is a +to increment the variable `x` by `1`. However, in Wuffs, such a statement is a compile time error unless the compiler can also prove that `x` is not the maximal value of `x`'s type (e.g. `x` is not `255` if `x` is a `u8`), as the increment would otherwise overflow. Similarly, an integer arithmetic expression like `x / y` is a compile time error unless the compiler can also prove that `y` is not zero. -Puffs is not a general purpose programming language. While technically -possible, it is unlikely that a Puffs compiler would be worth writing in Puffs. +Wuffs is not a general purpose programming language. While technically +possible, it is unlikely that a Wuffs compiler would be worth writing in Wuffs. -## What Does Puffs Code Look Like? +## What Does Wuffs Code Look Like? -The [`std/gif/decode_lzw.puffs`](./std/gif/decode_lzw.puffs) file is a good +The [`std/gif/decode_lzw.wuffs`](./std/gif/decode_lzw.wuffs) file is a good example. See the "Poking Around" section below for more guidance. ## What Does Compile Time Checking Look Like? For example, making this one-line edit to the GIF codec leads to a compile time -error. `puffs gen` fails to generate the C code, i.e. fails to compile -(transpile) the Puffs code to C code: +error. `wuffs gen` fails to generate the C code, i.e. fails to compile +(transpile) the Wuffs code to C code: ```diff -diff --git a/std/gif/decode_lzw.puffs b/std/gif/decode_lzw.puffs +diff --git a/std/gif/decode_lzw.wuffs b/std/gif/decode_lzw.wuffs index f878c5e..f10dcee 100644 ---- a/std/gif/decode_lzw.puffs -+++ b/std/gif/decode_lzw.puffs +--- a/std/gif/decode_lzw.wuffs ++++ b/std/gif/decode_lzw.wuffs @@ -98,7 +98,7 @@ pub func lzw_decoder.decode?(dst ptr buf1, src ptr buf1, src_final bool)() { in.dst.write?(x:s) @@ -62,9 +62,9 @@ index f878c5e..f10dcee 100644 ``` ```bash -$ puffs gen std/gif +$ wuffs gen std/gif check: expression "(c + 1) as u8" bounds [1..256] is not within bounds [0..255] at -/home/n/go/src/github.com/google/wuffs/std/gif/decode_lzw.puffs:101. Facts: +/home/n/go/src/github.com/google/wuffs/std/gif/decode_lzw.wuffs:101. Facts: n_bits < 8 c < 256 this.stack[s] == (c as u8) @@ -75,10 +75,10 @@ In comparison, this two-line edit will compile (but the "does it decode GIF correctly" tests then fail): ```diff -diff --git a/std/gif/decode_lzw.puffs b/std/gif/decode_lzw.puffs +diff --git a/std/gif/decode_lzw.wuffs b/std/gif/decode_lzw.wuffs index f878c5e..b43443d 100644 ---- a/std/gif/decode_lzw.puffs -+++ b/std/gif/decode_lzw.puffs +--- a/std/gif/decode_lzw.wuffs ++++ b/std/gif/decode_lzw.wuffs @@ -97,8 +97,8 @@ pub func lzw_decoder.decode?(dst ptr buf1, src ptr buf1, src_final bool)() { // type checking, bounds checking and code generation for it). in.dst.write?(x:s) @@ -92,10 +92,10 @@ index f878c5e..b43443d 100644 ``` ```bash -$ puffs gen std/gif +$ wuffs gen std/gif gen wrote: /home/n/go/src/github.com/google/wuffs/gen/c/gif.c gen unchanged: /home/n/go/src/github.com/google/wuffs/gen/h/gif.h -$ puffs test std/gif +$ wuffs test std/gif gen unchanged: /home/n/go/src/github.com/google/wuffs/gen/c/gif.c gen unchanged: /home/n/go/src/github.com/google/wuffs/gen/h/gif.h test: /home/n/go/src/github.com/google/wuffs/test/c/gif @@ -125,8 +125,8 @@ excerpts of got (above) versus want (below): 000020: eded 8f91 9191 9090 9090 9090 9191 9191 ................ 000030: 9191 9191 9191 9193 93f0 f0f0 f1f1 f2f2 ................ -puffs-test-c: some tests failed -puffs test: some tests failed +wuffs-test-c: some tests failed +wuffs test: some tests failed ``` # Background @@ -186,15 +186,15 @@ The run time cost is small, measured in nanoseconds. But if an image decoding library has to eat this cost per pixel, and you have a megapixel image, then nanoseconds become milliseconds, and milliseconds can matter. -In comparison, in Puffs, all bounds checks and arithmetic overflow checks +In comparison, in Wuffs, all bounds checks and arithmetic overflow checks happen at compile time, with zero run time overhead. # Getting Started -Puffs code (that is proved safe via explicit assertions) is compiled to C code +Wuffs code (that is proved safe via explicit assertions) is compiled to C code (with those assertions removed) - it is transpiled. If you are a C/C++ -programmer and just want to *use* the C edition of the Puffs standard library, +programmer and just want to *use* the C edition of the Wuffs standard library, then clone the repository and look at the files in the `gen/c` and `gen/h` directories. No other software tools are required and there are no library dependencies, other than C standard library concepts like ``'s @@ -213,38 +213,38 @@ gzip or zlib, roughly speaking) without necessarily processing PNG images. ## Getting Deeper -If you want to modify the Puffs standard library, or compile your own Puffs +If you want to modify the Wuffs standard library, or compile your own Wuffs code, you will need to do a little more work, and will have to install at least -the Go toolchain in order to build the Puffs tools. To run the test suite, you +the Go toolchain in order to build the Wuffs tools. To run the test suite, you might also have to install C compilers like clang and gcc, as well as C libraries (and their .h files) like libjpeg and libpng, as some tests compare -that Puffs produces exactly the same output as these other libraries. +that Wuffs produces exactly the same output as these other libraries. Running `go get -v github.com/google/wuffs/cmd/...` will download and install -the Puffs tools. Change `get` to `install` to re-install those programs without +the Wuffs tools. Change `get` to `install` to re-install those programs without downloading, e.g. after you've modified their source code, or after a manually -issued `git pull`. The Puffs tools that you'll most often use are `puffsfmt` -(analogous to `clang-format`, `gofmt` or `rustfmt`) and `puffs` (roughly +issued `git pull`. The Wuffs tools that you'll most often use are `wuffsfmt` +(analogous to `clang-format`, `gofmt` or `rustfmt`) and `wuffs` (roughly analogous to `make`, `go` or `cargo`). -You should now be able to run `puffs test`. If all goes well, you should see +You should now be able to run `wuffs test`. If all goes well, you should see some output containing the word "PASS" multiple times. ## Poking Around -Feel free to edit the `std/gif/decode_lzw.puffs` file, which implements the GIF -LZW decoder. After editing, run `puffs gen std/gif` or `puffs test std/gif` to -re-generate the C edition of the Puffs standard library's GIF codec, and +Feel free to edit the `std/gif/decode_lzw.wuffs` file, which implements the GIF +LZW decoder. After editing, run `wuffs gen std/gif` or `wuffs test std/gif` to +re-generate the C edition of the Wuffs standard library's GIF codec, and optionally run its tests. -Try deleting an assert statement and re-running `puffs gen`. The result should +Try deleting an assert statement and re-running `wuffs gen`. The result should be syntactically valid, but a compile error, as some bounds checks can no longer be proven. Find the line `var bits u32`, which declares the bits variable and initializes it to zero. Try adding `bits -= 1` on a new line of code after it. Again, -`puffs gen` should fail, as the computation can underflow. +`wuffs gen` should fail, as the computation can underflow. Similarly, replacing the line `var n_bits u32` with `var n_bits u32 = 10` should fail, as an `n_bits < 8` assertion, a pre-condition, a few lines further @@ -254,8 +254,8 @@ Similarly, changing the `4095` in `var prev_code u32[..4095]` either higher or lower should fail. Try adding `assert false` at various places, which should obviously fail, but -should also cause `puffs gen` to print what facts the compiler can prove at -that point. This can be useful when debugging why Puffs can't prove something +should also cause `wuffs gen` to print what facts the compiler can prove at +that point. This can be useful when debugging why Wuffs can't prove something you think it should be able to. @@ -265,14 +265,14 @@ If you've changed any of the tools (i.e. changed any `.go` code), re-run `go install -v github.com/google/wuffs/cmd/...` and `go test github.com/google/wuffs/lang/...`. -If you've changed any of the libraries (i.e. changed any `.puffs` code), run -`puffs test` or, ideally, `puffs test -mimic` to also check that Puffs' output +If you've changed any of the libraries (i.e. changed any `.wuffs` code), run +`wuffs test` or, ideally, `wuffs test -mimic` to also check that Wuffs' output mimics (i.e. exactly matches) other libraries' output, such as giflib for GIF, libpng for PNG, etc. -If your library change is an optimization, run `puffs bench` or `puffs bench +If your library change is an optimization, run `wuffs bench` or `wuffs bench -mimic` both before and after your change to quantify the improvement. The -mimic benchmark numbers should't change if you're only changing `.puffs` code, +mimic benchmark numbers should't change if you're only changing `.wuffs` code, but seeing zero change in those numbers is a sanity check on any unrelated system variance, such as software updates or virus checkers running in the background. @@ -280,30 +280,30 @@ background. ## Directory Layout -- `lang` holds the Go libraries that implement the Puffs language: tokenizer, - AST, parser, renderer, etc. The Puffs tools are written in Go, but as - mentioned above, Puffs transpiles to C code, and Go is not necessarily - involved if all you want is to use the C edition of Puffs. -- `cmd` holds Puffs' command line tools, also written in Go. -- `std` holds the Puffs standard library's code. The initial focus is on +- `lang` holds the Go libraries that implement the Wuffs language: tokenizer, + AST, parser, renderer, etc. The Wuffs tools are written in Go, but as + mentioned above, Wuffs transpiles to C code, and Go is not necessarily + involved if all you want is to use the C edition of Wuffs. +- `cmd` holds Wuffs' command line tools, also written in Go. +- `std` holds the Wuffs standard library's code. The initial focus is on popular image codecs: BMP, GIF, JPEG, PNG, TIFF and WEBP. - `gen` holds the transpiled editions of that standard library. The initial focus is generating C code. Later on, the repository might include generated Go and Rust code. -- `test` holds the tests for the Puffs standard library. +- `test` holds the tests for the Wuffs standard library. - `script` holds miscellaneous utility programs. - `doc` holds documentation. - `example` holds example programs. For a guide on how various things work together, the "99ff8e2 Let fields have -default values" commit is an example of adding new Puffs syntax and threading +default values" commit is an example of adding new Wuffs syntax and threading that all the way through to C code generation and testing. # Documentation - [Wuffs the Language](./doc/wuffs-the-language.md) -- Puffs the Library (TODO) +- Wuffs the Library (TODO) - [Related Work](./doc/related-work.md) - [Roadmap](./doc/roadmap.md) diff --git a/doc/benchmarks.md b/doc/benchmarks.md index 64f0ae05..3354e2b0 100644 --- a/doc/benchmarks.md +++ b/doc/benchmarks.md @@ -1,9 +1,9 @@ # Benchmarks -*Preliminary* `puffs bench -mimic` summarized throughput numbers for various +*Preliminary* `wuffs bench -mimic` summarized throughput numbers for various codecs are below. Higher is better. -"Mimic" tests check that Puffs' output mimics (i.e. exactly matches) other +"Mimic" tests check that Wuffs' output mimics (i.e. exactly matches) other libraries' output. "Mimic" benchmarks give the numbers for those other libraries, as shipped with the OS. These were measured on a Debian Testing system as of November 2017, which means these compiler versions: @@ -27,7 +27,7 @@ The benchmark programs aim to be runnable "out of the box" without any configuration or installation. For example, to run the `std/flate` benchmarks: git clone https://github.com/google/wuffs.git - cd puffs/test/c/std + cd wuffs/test/c/std gcc -O3 flate.c ./a.out -bench rm a.out @@ -42,24 +42,24 @@ code change. To install it, first install Go, then run `go get golang.org/x/perf/cmd/benchstat`. -## puffs bench +## wuffs bench As mentioned above, individual benchmark programs can be run manually. However, the canonical way to run the benchmarks (across multiple compilers and multiple -packages like GIF and PNG) is to use the `puffs` command line tool, as it will -also re-generate (transpile) the C code whenever you edit the \*.puffs code. -Running `go install -v github.com/google/wuffs/cmd/...` will install the Puffs +packages like GIF and PNG) is to use the `wuffs` command line tool, as it will +also re-generate (transpile) the C code whenever you edit the \*.wuffs code. +Running `go install -v github.com/google/wuffs/cmd/...` will install the Wuffs tools. After that, you can say - puffs bench + wuffs bench or - puffs bench -mimic std/flate + wuffs bench -mimic std/flate or - puffs bench -ccompilers=gcc -reps=3 -focus=Benchmarkpuffs_gif_lzw std/gif + wuffs bench -ccompilers=gcc -reps=3 -focus=Benchmarkwuffs_gif_lzw std/gif ## CPU Scaling @@ -82,21 +82,21 @@ output. name speed - puffs_adler32_10k/clang 2.42GB/s ± 3% - puffs_adler32_100k/clang 2.42GB/s ± 3% - puffs_flate_decode_1k/clang 133MB/s ± 3% - puffs_flate_decode_10k/clang 199MB/s ± 3% - puffs_flate_decode_100k/clang 229MB/s ± 4% - puffs_zlib_decode_10k/clang 185MB/s ± 1% - puffs_zlib_decode_100k/clang 210MB/s ± 3% + wuffs_adler32_10k/clang 2.42GB/s ± 3% + wuffs_adler32_100k/clang 2.42GB/s ± 3% + wuffs_flate_decode_1k/clang 133MB/s ± 3% + wuffs_flate_decode_10k/clang 199MB/s ± 3% + wuffs_flate_decode_100k/clang 229MB/s ± 4% + wuffs_zlib_decode_10k/clang 185MB/s ± 1% + wuffs_zlib_decode_100k/clang 210MB/s ± 3% - puffs_adler32_10k/gcc 3.22GB/s ± 1% - puffs_adler32_100k/gcc 3.23GB/s ± 0% - puffs_flate_decode_1k/gcc 152MB/s ± 1% - puffs_flate_decode_10k/gcc 250MB/s ± 1% - puffs_flate_decode_100k/gcc 298MB/s ± 1% - puffs_zlib_decode_10k/gcc 238MB/s ± 2% - puffs_zlib_decode_100k/gcc 270MB/s ± 2% + wuffs_adler32_10k/gcc 3.22GB/s ± 1% + wuffs_adler32_100k/gcc 3.23GB/s ± 0% + wuffs_flate_decode_1k/gcc 152MB/s ± 1% + wuffs_flate_decode_10k/gcc 250MB/s ± 1% + wuffs_flate_decode_100k/gcc 298MB/s ± 1% + wuffs_zlib_decode_10k/gcc 238MB/s ± 2% + wuffs_zlib_decode_100k/gcc 270MB/s ± 2% mimic_adler32_10k 3.00GB/s ± 1% mimic_adler32_100k 2.91GB/s ± 2% @@ -116,15 +116,15 @@ is 1 byte per pixel. name speed - puffs_gif_decode_1k/clang 346MB/s ± 1% - puffs_gif_decode_10k/clang 137MB/s ± 0% - puffs_gif_decode_100k/clang 118MB/s ± 0% - puffs_gif_decode_1000k/clang 120MB/s ± 0% + wuffs_gif_decode_1k/clang 346MB/s ± 1% + wuffs_gif_decode_10k/clang 137MB/s ± 0% + wuffs_gif_decode_100k/clang 118MB/s ± 0% + wuffs_gif_decode_1000k/clang 120MB/s ± 0% - puffs_gif_decode_1k/gcc 399MB/s ± 1% - puffs_gif_decode_10k/gcc 141MB/s ± 0% - puffs_gif_decode_100k/gcc 128MB/s ± 0% - puffs_gif_decode_1000k/gcc 131MB/s ± 0% + wuffs_gif_decode_1k/gcc 399MB/s ± 1% + wuffs_gif_decode_10k/gcc 141MB/s ± 0% + wuffs_gif_decode_100k/gcc 128MB/s ± 0% + wuffs_gif_decode_1000k/gcc 131MB/s ± 0% mimic_gif_decode_1k 147MB/s ± 0% mimic_gif_decode_10k 90.7MB/s ± 0% @@ -134,10 +134,10 @@ is 1 byte per pixel. TODO: investigate why gcc 4.8 (Ubuntu Trusty) seems to generate faster code than gcc 7.2 (Debian Testing): - puffs_gif_decode_1k/gcc 411MB/s ± 2% - puffs_gif_decode_10k/gcc 162MB/s ± 0% - puffs_gif_decode_100k/gcc 138MB/s ± 0% - puffs_gif_decode_1000k/gcc 141MB/s ± 0% + wuffs_gif_decode_1k/gcc 411MB/s ± 2% + wuffs_gif_decode_10k/gcc 162MB/s ± 0% + wuffs_gif_decode_100k/gcc 138MB/s ± 0% + wuffs_gif_decode_1000k/gcc 141MB/s ± 0% --- diff --git a/doc/binary-size.md b/doc/binary-size.md index d1e21afd..36706d4e 100644 --- a/doc/binary-size.md +++ b/doc/binary-size.md @@ -1,25 +1,25 @@ # Binary Size -*Preliminary* measurements of `puffs genlib` libraries' binary size on x86_64 +*Preliminary* measurements of `wuffs genlib` libraries' binary size on x86_64 are below. Lower is better. clang-dynamic: - -rwxr-xr-x 1 nigeltao eng 38352 Nov 9 22:59 libpuffs.so + -rwxr-xr-x 1 nigeltao eng 38352 Nov 9 22:59 libwuffs.so -rw-r--r-- 1 nigeltao eng 22624 Nov 9 22:59 std-flate.lo -rw-r--r-- 1 nigeltao eng 9456 Nov 9 22:59 std-gif.lo clang-static: - -rw-r----- 1 nigeltao eng 32966 Nov 9 22:59 libpuffs.a + -rw-r----- 1 nigeltao eng 32966 Nov 9 22:59 libwuffs.a -rw-r--r-- 1 nigeltao eng 22648 Nov 9 22:59 std-flate.o -rw-r--r-- 1 nigeltao eng 9480 Nov 9 22:59 std-gif.o gcc-dynamic: - -rwxr-xr-x 1 nigeltao eng 42504 Nov 9 22:59 libpuffs.so + -rwxr-xr-x 1 nigeltao eng 42504 Nov 9 22:59 libwuffs.so -rw-r--r-- 1 nigeltao eng 24776 Nov 9 22:59 std-flate.lo -rw-r--r-- 1 nigeltao eng 13520 Nov 9 22:59 std-gif.lo gcc-static: - -rw-r----- 1 nigeltao eng 39102 Nov 9 22:59 libpuffs.a + -rw-r----- 1 nigeltao eng 39102 Nov 9 22:59 libwuffs.a -rw-r--r-- 1 nigeltao eng 24728 Nov 9 22:59 std-flate.o -rw-r--r-- 1 nigeltao eng 13536 Nov 9 22:59 std-gif.o @@ -29,7 +29,7 @@ are below. Lower is better. Below are some standard C libraries shipped as part of Debian Testing as of November 2017. The numbers aren't directly comparable, as these libraries have a richer API, especially in providing an encoder and not just a decoder. Still, -it is a reference point for e.g. Puffs flate vs libz and Puffs gif vs libgif. +it is a reference point for e.g. Wuffs flate vs libz and Wuffs gif vs libgif. dynamic: -rw-r--r-- 1 root root 105088 Jan 29 2017 /lib/x86_64-linux-gnu/libz.so.1.2.8 diff --git a/doc/compatibility.md b/doc/compatibility.md index 0306076c..ab53b5b5 100644 --- a/doc/compatibility.md +++ b/doc/compatibility.md @@ -1,7 +1,7 @@ # Compatibility TODO: get a decently large corpus of test images from a web crawl. Ensure that -Puffs produces 100% identical decodings to the suite of giflib, libjpeg, +Wuffs produces 100% identical decodings to the suite of giflib, libjpeg, libpng, etc. TODO: also trawl through Go's bug tracker for "this image failed to load". diff --git a/doc/related-work.md b/doc/related-work.md index fca5a2ef..16c2f3d4 100644 --- a/doc/related-work.md +++ b/doc/related-work.md @@ -1,34 +1,34 @@ # Related Work -Puffs is an industrial project, not an academic project, and this Related Work +Wuffs is an industrial project, not an academic project, and this Related Work section is not as extensive or as rigorous as would be found in an academic thesis or journal article. The underlying goal is usefulness, not novelty. For -Puffs' users, it'd perfectly fine to have multiple implementations of a good +Wuffs' users, it'd perfectly fine to have multiple implementations of a good idea, even if the later ones wouldn't earn a Ph. D. -A number of other projects are listed below. Puffs is not exactly like any of +A number of other projects are listed below. Wuffs is not exactly like any of them, for one reason or another. We're not claiming that "use a state of the art theorem prover" or "write it in Rust" are unworkable approaches. They're just different approaches, with different trade-offs. -For example, Puffs is primarily concerned about *safety* and *performance*. +For example, Wuffs is primarily concerned about *safety* and *performance*. Formal *correctness*, such as being able to mathematically express that "when this function returns, that array's elements will be sorted in increasing -order", is less of a concern for Puffs the Language than for other projects. +order", is less of a concern for Wuffs the Language than for other projects. This is especially as higher level concerns like "correctly implements the JPEG specification" are less amenable to formalization than foundational concepts like searching and sorting. -For Puffs the Library, confidence about correctness is instead based on +For Wuffs the Library, confidence about correctness is instead based on testing, both unit testing and for e.g. media codecs, corpora of test files. A -small corpus is included in the Puffs source code repository. Other, much +small corpus is included in the Wuffs source code repository. Other, much larger corpora (e.g. based on a sample of a web crawl), are not included for download size and copyright reasons. ## Static Checkers -Puffs is a language by itself, integrated with a compiler, not something +Wuffs is a language by itself, integrated with a compiler, not something embedded in the comments of another language's program, supported by a separate program checker, as the bounds check elimination affects the generated code. @@ -37,18 +37,18 @@ similar goal of safety, but aims to be backwards compatible with C, including the idiomatic ways in which C code plays fast and loose with pointers and arrays, and implicitly inserting run time checks while maintaining existing data layout (e.g. the sizeof a struct type) for interoperability with other -systems. This is a much harder problem than Puffs' approach of a new, simple, +systems. This is a much harder problem than Wuffs' approach of a new, simple, domain-specific programming language. Extended Static Checker for Java (ESC/Java) and its successor [OpenJML](http://www.openjml.org/), which obviously target the Java programming language, similarly has to analyze a language that is more complicated that -Puffs. Java is also not commonly used for writing e.g. low level image codecs, +Wuffs. Java is also not commonly used for writing e.g. low level image codecs, as the language lacks unsigned integer types, it is garbage collected and idiomatic code often allocates. Similarly, [Whiley](http://whiley.org/) is a programming language with extended -static checking. In contrast to Puffs, its `int` type uses unbounded +static checking. In contrast to Wuffs, its `int` type uses unbounded arithmetic, instead of e.g. 32-bit twos-complement representation. That can be easier for theorem provers to reason about, but this can have a performance impact when such integers are used in inner loops. In theory, a compiler could @@ -57,7 +57,7 @@ potentially 'big integer', but even so, it may improve performance to explicitly use a `uint32` here even when a `uint8` would do. [Spark ADA](http://libre.adacore.com/tools/spark-gpl-edition/) targets a subset -of the Ada programming language, which again is more complicated than Puffs. It +of the Ada programming language, which again is more complicated than Wuffs. It also allows for pluggable theorem provers such as Z3, similar to [Dafny](http://research.microsoft.com/dafny). This can increase programmer productivity in that it can take less effort to prove more programs safe, but @@ -67,19 +67,19 @@ trusted computing base. It's also not obvious up front what effect different theorem provers, and their different heuristics, will have on compile times or proof times. -In constrast, Puffs' proof system is fast (and dumb) instead of smart (and +In constrast, Wuffs' proof system is fast (and dumb) instead of smart (and slow). For example, [Vale (Verified Assembly Language for Everest)](https://github.com/project-everest/vale) is a promising approach that uses Dafny to verify implementations of cryptographic algorithms. However, ["Vale: Verifying High-Performance Cryptographic Assembly Code"](http://www.andrew.cmu.edu/user/bparno/papers/vale.pdf) reports in Table -1 that verification times are measured in *minutes*. Puffs aims for +1 that verification times are measured in *minutes*. Wuffs aims for *sub-second* compilation times, including proofs of (memory) safety. Figure 15 in that paper suggests that Vale's verification times can grow exponentially in some cases, and also goes on to say, "Dafny/Z3 fails to verify the AES key inversion for 6 unrolled iterations and 9 unrolled iterations, indicating that SMT solvers like Z3 are still occasionally unpredictable". Of course, Vale is -also tackling a much harder problem than Puffs: proving correctness, not just +also tackling a much harder problem than Wuffs: proving correctness, not just safety. Once again, we're not claiming that these other approaches are unworkable, or @@ -91,17 +91,17 @@ not useful, just different, with different trade-offs. Simpler languages are easier to prove things about. Macros, inheritance, closures, generics, operator overloading, goto's and built-in container types are all useful features, but as mentioned in the top-level -[README](../README.md), Puffs is not a general purpose programming language. +[README](../README.md), Wuffs is not a general purpose programming language. Instead, its focus is on compile-time provable memory safety, with C-like performance, for CPU-intensive file format decoders. -Nonetheless, Puffs is an imperative language, not a functional language, +Nonetheless, Wuffs is an imperative language, not a functional language, despite the long history of functional languages and provability. Inner loop performance usage matters, and it is easier to match the optimization techniques of incumbent C libraries with a C-like language than with a functional language. Compared to something like [F\*](https://www.fstar-lang.org/), [Idris](https://www.idris-lang.org/) or -[Liquid Haskell](https://ucsd-progsys.github.io/liquidhaskell-blog/), Puffs has +[Liquid Haskell](https://ucsd-progsys.github.io/liquidhaskell-blog/), Wuffs has no garbage collector overhead, as it has no garbage collector. Memory usage also matters, again considering per-pixel costs can be multiplied @@ -110,8 +110,8 @@ u8 values (or one u32), not as four naturally-sized-for-the-CPU integers or four 'big integers'. [F\* / KreMLin](https://github.com/FStarLang/kremlin) is a subset of F\* that -is similar in some sense to Puffs, in that it transpiles to C and cares about -both safety and performance. Unlike Puffs, it is a functional language (with +is similar in some sense to Wuffs, in that it transpiles to C and cares about +both safety and performance. Unlike Wuffs, it is a functional language (with dependent types), not an imperative one (with a simpler type system), and uses a sophisticated theorem prover like Z3, with the same trade-offs as discussed above for Spark ADA and Dafny. It also tackles a more complicated problem, in @@ -122,7 +122,7 @@ Low-Level Programming Embedded in F\*"](https://arxiv.org/pdf/1703.00053.pdf). [Cryptol](https://github.com/GaloisInc/cryptol) is another project that, like Everest (including F\* / KreMLin and its HACL\* sub-project, and Dafny / Vale), -focuses on cryptographic algorithms, rather than Puffs' focus on file formats, +focuses on cryptographic algorithms, rather than Wuffs' focus on file formats, and also relies on a sophisticated theorem prover like Z3. Once again, we're not claiming that these other approaches are unworkable, or @@ -131,9 +131,9 @@ not useful, just different, with different trade-offs. ## Why Not Rust? -Puffs transpiles to standard C99 code with no dependencies, and should run +Wuffs transpiles to standard C99 code with no dependencies, and should run anywhere that has a C99 compliant C compiler. An existing C/C++ project, such -as the Chromium web browser, can choose to replace e.g. libpng with Puffs PNG, +as the Chromium web browser, can choose to replace e.g. libpng with Wuffs PNG, without needing any additional toolchains. Sure, languages like D and Rust have great binary-level interop with C code, and Mozilla are reporting progress with parsing media formats in Rust, but it's still an operational hurdle to grow a @@ -141,18 +141,18 @@ project's build process and to assess build times and binary sizes. [Nom](https://github.com/Geal/nom) is a parser combinator library in Rust, described in ["Writing parsers like it is -2017"](http://spw17.langsec.org/papers/chifflier-parsing-in-2017.pdf). Puffs -differs from nom by itself, in that Puffs is an end to end implementation, not +2017"](http://spw17.langsec.org/papers/chifflier-parsing-in-2017.pdf). Wuffs +differs from nom by itself, in that Wuffs is an end to end implementation, not limited to that part of a file format that is easily expressible as a formal grammar. In particular, it also handles entropy encodings such as LZW (for -GIF), ZLIB (for PNG) and Huffman (for JPEG, TODO). Puffs differs from nom +GIF), ZLIB (for PNG) and Huffman (for JPEG, TODO). Wuffs differs from nom combined with other Rust code (e.g. a Rust LZW implementation) in that bounds and overflow checks not just ubiquitous but also completely eliminated at compile time. [Kaitai Struct](http://kaitai.io/) is in a similar space, generating safe parsers for multiple target programming languages from one declarative -specification. Again, Puffs differs in that it is a complete (and performant) +specification. Again, Wuffs differs in that it is a complete (and performant) end to end implementation, not just for the structured parts of a file format. Repeating a point in the previous paragraph, the difficulty in decoding the GIF format isn't in the regularly-expressible part of the format, it's in the LZW diff --git a/doc/wuffs-the-language.md b/doc/wuffs-the-language.md index edffe1e6..6446e40d 100644 --- a/doc/wuffs-the-language.md +++ b/doc/wuffs-the-language.md @@ -1,12 +1,12 @@ -# Puffs the Language +# Wuffs the Language -Puffs is an imperative, C-like language and much of it should look familiar to +Wuffs is an imperative, C-like language and much of it should look familiar to somebody versed in any one of C, C++, D, Go, Java, JavaScript, Objective-C, Rust, Swift, etc. Accordingly, this section is not an exhaustive specification, -merely a rough guide of where Puffs differs. +merely a rough guide of where Wuffs differs. Like Go, [semi-colons can be omitted](https://golang.org/ref/spec#Semicolons), -and `puffsfmt` will remove them. Similarly, the body of an `if` or `while` must +and `wuffsfmt` will remove them. Similarly, the body of an `if` or `while` must be enclosed by curly `{}`s. There is no 'dangling else' ambiguity. @@ -68,7 +68,7 @@ associative: `(a + b) + c` and `a + (b + c)` are equivalent, and can be written as `a + b + c`. The logical operators, `&&` and `||` and `!` in C, are written as `and` and -`or` and `not` in Puffs. +`or` and `not` in Wuffs. TODO: ignore-overflow ops, equivalent to Swift's `&+`. @@ -84,7 +84,7 @@ array of unsigned 32-bit integers. `ptr` here means a non-null pointer. Use Integer types can also be refined: `var x u32[10..20]` defines a variable x that is stored as 4 bytes (32 bits) and can be combined arithmetically (e.g. added, compared) with other `u32`s, but whose value must be between 10 and 20 -inclusive. The syntax is reminiscent of Pascal's subranges, but in Puffs, a +inclusive. The syntax is reminiscent of Pascal's subranges, but in Wuffs, a subsequent assignment like `x = 21` or even `x = y + z` is a compile time error unless the right hand side can be proven to be within range. @@ -97,7 +97,7 @@ i8[-7..]` means that `y` is between -7 and +127. `var z u32[..]` is equivalent to `var z u32`. Refinement bounds must be constant expressions. `var x u32[..2+3]` is valid, -but `var x u32; var y u32[..x]` is not. Puffs does not have dependent types. +but `var x u32; var y u32[..x]` is not. Wuffs does not have dependent types. Relationships such as `y <= x` are expressible as assertions (see below), but not by the type system. @@ -114,7 +114,7 @@ its methods may be coroutines. (See below). Function signatures read from left to right: `func max(x i32, y i32)(z i32)` is a function that takes two `i32`s and returns one `i32`. Two pairs of parentheses are required: a function that in other languages would return -`void` in Puffs returns the empty struct `()`, also known as the unit type. +`void` in Wuffs returns the empty struct `()`, also known as the unit type. When calling a function, each argument must be named. It is `m = max(x:10, y:20)` and not `m = max(10, 20)`. @@ -208,7 +208,7 @@ facts that include both the explicit assertion `x < y` and the implicit assertion `z == 3`, if none of `x`, `y` and `z` alias another (e.g. they are all local variables). -Puffs has two forms of non-sequential control flow: `if` branches (including +Wuffs has two forms of non-sequential control flow: `if` branches (including `if`, `else if`, `else if` chains) and `while` loops. For an `if` statement, such as `if b { etc0 } else { etc1 }`, the condition `b` @@ -237,14 +237,14 @@ and assumed after it. ## Proofs -Puffs' assertion and bounds checking system is a proof checker, not a full SMT +Wuffs' assertion and bounds checking system is a proof checker, not a full SMT solver or automated theorem prover. It verifies explicit annotations instead of the more open-ended task of searching for implicit proof steps. This involves -more explicit work by the programmer, but compile times matter, so Puffs is +more explicit work by the programmer, but compile times matter, so Wuffs is fast (and dumb) instead of smart (and slow). -Nonetheless, the Puffs syntax is regular (and unlike C++, does not require a -symbol table to parse), so it should be straightforward to transform Puffs code +Nonetheless, the Wuffs syntax is regular (and unlike C++, does not require a +symbol table to parse), so it should be straightforward to transform Wuffs code to and from file formats used by more sophisticated proof engines. Some rules are applied automatically by the proof checker. For example, if `x @@ -253,15 +253,15 @@ type system) or dynamic constraint (an asserted fact), then the checker knows that the expression `x + y` is bounded above by `10 + 5` and therefore will not overflow a `u8` (but would overflow a `u8[..12]`). -TODO: rigorously specify these automatic rules, when we have written more Puffs +TODO: rigorously specify these automatic rules, when we have written more Wuffs code and thus have more experience on what rules are needed to implement multiple, real world image codecs. Other rules are built in to the proof checker but are not applied automatically (see "fast... instead of smart" above). Such rules have double-quote enclosed names that look a little like mathematical statements. They are axiomatic, in -that these rules are assumed, not proved, by the Puffs toolchain. They are -typically at a higher level than e.g. Peano axioms, as Puffs emphasizes +that these rules are assumed, not proved, by the Wuffs toolchain. They are +typically at a higher level than e.g. Peano axioms, as Wuffs emphasizes practicality over theoretical minimalism. As they are axiomatic, they endeavour to only encode 'obvious' mathematical rules. For example, the rule named `"a < b: a < c; c <= b"` is one expression of transitivity: the assertion `a < b` is @@ -283,7 +283,7 @@ TODO: specify these built-in `via` rules, again after more experience. Labeled `break` and `continue` statements enable jumping out of loops that aren't the most deeply nested. The syntax for labels is `while:label` instead -of Java's `label:while`, as the former is slightly easier to parse, and Puffs +of Java's `label:while`, as the former is slightly easier to parse, and Wuffs does not otherwise use labels for switch cases or goto targets. TODO: describe the built in `buf1` and `buf2` types: 1- and 2-dimensional