Compare commits

...
5 Commits
7 changed files with 168 additions and 70 deletions

No files matched your search

+31 -14
View File
@@ -1,17 +1,20 @@
![WUFFS Logo](./doc/logo/wuffs-acronym-logo-1536x1024.png)
Wuffs (Wrangling Untrusted File Formats Safely) is [formerly known as
Puffs](https://groups.google.com/d/topic/puffslang/ZX-ymyf8xh0/discussion)
(Parsing Untrusted File Formats Safely).
Wuffs is a **memory-safe programming language** (and a **standard library**
written in that language) for wrangling **untrusted file formats** safely.
written in that language) for **Wrangling Untrusted File Formats Safely**.
Wrangling includes parsing, decoding and encoding. Example file formats include
images, audio, video, fonts and compressed archives.
It is also **fast**. On many of its GIF decoding
[benchmarks](/doc/benchmarks.md), Wuffs measures **2x faster than "giflib" (C),
3x faster than "image/gif" (Go) and 7x faster than "gif" (Rust)**.
It is **fast**. Per its [benchmarks](/doc/benchmarks.md) and other docs:
- It can decode bzip2 **[1.3x faster than `/usr/bin/bzcat`
(C)](https://nigeltao.github.io/blog/2022/wuffs-bzip2-decoder.html)**.
- It can decode deflate up to **1.4x faster [than zlib-the-library (C)**.
- It can decode GIF **2x-6x faster than "giflib" (C), "image/gif" (Go) and
"gif" (Rust)**.
- It can decode PNG **[1.2x-2.7x faster than "libpng"
(C)](https://nigeltao.github.io/blog/2021/fastest-safest-png-decoder.html),
"image/png" (Go) and "png" (Rust)**.
## Goals and Non-Goals
@@ -44,11 +47,21 @@ the increment would otherwise overflow. Similarly, an integer arithmetic
expression like `x / y` is a compile time error unless the compiler can also
prove that `y` is not zero.
## Hermeticity
Wuffs is not a general purpose programming language. **It is for writing
libraries, not programs**. The idea isn't to write your whole program in Wuffs,
only the **parts that are both performance-conscious and security-conscious**.
For example, while technically possible, it is unlikely that a Wuffs compiler
would be worth writing entirely in Wuffs.
libraries, not programs**. Wuffs code is [hermetic](/doc/note/hermeticity.md)
and can only compute (e.g. convert "compressed bytes" to "decompressed bytes").
**It cannot make any syscalls** (e.g. it has no ambient authority to read your
files), implying that it cannot allocate or free memory (and is therefore
trivially safe against things like memory leaks, use-after-frees and
double-frees).
The idea isn't to write your whole program in Wuffs, **only the parts that are
both performance-conscious and security-conscious**. For example, while
technically possible, it is unlikely that a Wuffs compiler would be worth
writing entirely in Wuffs.
## What Does Wuffs Code Look Like?
@@ -187,12 +200,16 @@ The [Note](/doc/note) directory also contains various short articles.
# Status
Version 0.2. The API and ABI aren't stabilized yet. The compiler undoubtedly
Version 0.3. The API and ABI aren't stabilized yet. The compiler undoubtedly
has bugs. Assertion checking needs more rigor, especially around side effects
and aliasing, and being sufficiently well specified to allow alternative
implementations. Lots of detail needs work, but the broad brushstrokes are
there.
Nonetheless, Wuffs' GIF decoder has shipped in the Google Chrome web browser
[since June
2021](https://chromium-review.googlesource.com/c/chromium/src/+/2940044).
# Discussion
@@ -223,4 +240,4 @@ owned by Google.
---
Updated on December 2019.
Updated on January 2023.
+107 -47
View File
@@ -3,68 +3,128 @@
Binary size measurements of `wuffs genlib` libraries on x86\_64 are below.
Lower is better.
clang8-dynamic:
-rwxr-xr-x 1 tao tao 90488 Oct 19 05:34 libwuffs.so
-rw-r--r-- 1 tao tao 7096 Oct 19 05:34 wuffs-base.lo
-rw-r--r-- 1 tao tao 2192 Oct 19 05:34 wuffs-std-adler32.lo
-rw-r--r-- 1 tao tao 19296 Oct 19 05:34 wuffs-std-crc32.lo
-rw-r--r-- 1 tao tao 20568 Oct 19 05:34 wuffs-std-deflate.lo
-rw-r--r-- 1 tao tao 26720 Oct 19 05:34 wuffs-std-gif.lo
-rw-r--r-- 1 tao tao 6592 Oct 19 05:34 wuffs-std-gzip.lo
-rw-r--r-- 1 tao tao 5808 Oct 19 05:34 wuffs-std-lzw.lo
-rw-r--r-- 1 tao tao 6584 Oct 19 05:34 wuffs-std-zlib.lo
clang11-dynamic:
-rwxr-xr-x 1 tao tao 322264 Jan 26 15:33 libwuffs.so
-rw-r--r-- 1 tao tao 5232 Jan 26 15:32 wuffs-base-core.lo
-rw-r--r-- 1 tao tao 26976 Jan 26 15:32 wuffs-base-floatconv.lo
-rw-r--r-- 1 tao tao 9312 Jan 26 15:32 wuffs-base-intconv.lo
-rw-r--r-- 1 tao tao 7920 Jan 26 15:33 wuffs-base-interfaces.lo
-rw-r--r-- 1 tao tao 3216 Jan 26 15:33 wuffs-base-magic.lo
-rw-r--r-- 1 tao tao 56192 Jan 26 15:33 wuffs-base-pixconv.lo
-rw-r--r-- 1 tao tao 3080 Jan 26 15:33 wuffs-base-utf8.lo
-rw-r--r-- 1 tao tao 4056 Jan 26 15:33 wuffs-std-adler32.lo
-rw-r--r-- 1 tao tao 28600 Jan 26 15:33 wuffs-std-bmp.lo
-rw-r--r-- 1 tao tao 17304 Jan 26 15:33 wuffs-std-bzip2.lo
-rw-r--r-- 1 tao tao 10168 Jan 26 15:33 wuffs-std-cbor.lo
-rw-r--r-- 1 tao tao 22336 Jan 26 15:33 wuffs-std-crc32.lo
-rw-r--r-- 1 tao tao 24816 Jan 26 15:33 wuffs-std-deflate.lo
-rw-r--r-- 1 tao tao 25088 Jan 26 15:33 wuffs-std-gif.lo
-rw-r--r-- 1 tao tao 7408 Jan 26 15:33 wuffs-std-gzip.lo
-rw-r--r-- 1 tao tao 20136 Jan 26 15:33 wuffs-std-json.lo
-rw-r--r-- 1 tao tao 6344 Jan 26 15:33 wuffs-std-lzw.lo
-rw-r--r-- 1 tao tao 9928 Jan 26 15:33 wuffs-std-nie.lo
-rw-r--r-- 1 tao tao 54760 Jan 26 15:33 wuffs-std-png.lo
-rw-r--r-- 1 tao tao 14552 Jan 26 15:33 wuffs-std-tga.lo
-rw-r--r-- 1 tao tao 9080 Jan 26 15:33 wuffs-std-wbmp.lo
-rw-r--r-- 1 tao tao 7224 Jan 26 15:33 wuffs-std-zlib.lo
clang8-static:
-rw-r--r-- 1 tao tao 101560 Oct 19 05:34 libwuffs.a
-rw-r--r-- 1 tao tao 7176 Oct 19 05:34 wuffs-base.o
-rw-r--r-- 1 tao tao 2176 Oct 19 05:34 wuffs-std-adler32.o
-rw-r--r-- 1 tao tao 20296 Oct 19 05:34 wuffs-std-crc32.o
-rw-r--r-- 1 tao tao 20432 Oct 19 05:34 wuffs-std-deflate.o
-rw-r--r-- 1 tao tao 26856 Oct 19 05:34 wuffs-std-gif.o
-rw-r--r-- 1 tao tao 6840 Oct 19 05:34 wuffs-std-gzip.o
-rw-r--r-- 1 tao tao 5776 Oct 19 05:34 wuffs-std-lzw.o
-rw-r--r-- 1 tao tao 6608 Oct 19 05:34 wuffs-std-zlib.o
clang11-static:
-rw-r--r-- 1 tao tao 398014 Jan 26 15:32 libwuffs.a
-rw-r--r-- 1 tao tao 5232 Jan 26 15:32 wuffs-base-core.o
-rw-r--r-- 1 tao tao 27136 Jan 26 15:32 wuffs-base-floatconv.o
-rw-r--r-- 1 tao tao 9912 Jan 26 15:32 wuffs-base-intconv.o
-rw-r--r-- 1 tao tao 7856 Jan 26 15:32 wuffs-base-interfaces.o
-rw-r--r-- 1 tao tao 3112 Jan 26 15:32 wuffs-base-magic.o
-rw-r--r-- 1 tao tao 59000 Jan 26 15:32 wuffs-base-pixconv.o
-rw-r--r-- 1 tao tao 3080 Jan 26 15:32 wuffs-base-utf8.o
-rw-r--r-- 1 tao tao 4088 Jan 26 15:32 wuffs-std-adler32.o
-rw-r--r-- 1 tao tao 29384 Jan 26 15:32 wuffs-std-bmp.o
-rw-r--r-- 1 tao tao 16784 Jan 26 15:32 wuffs-std-bzip2.o
-rw-r--r-- 1 tao tao 10304 Jan 26 15:32 wuffs-std-cbor.o
-rw-r--r-- 1 tao tao 23352 Jan 26 15:32 wuffs-std-crc32.o
-rw-r--r-- 1 tao tao 25344 Jan 26 15:32 wuffs-std-deflate.o
-rw-r--r-- 1 tao tao 25608 Jan 26 15:32 wuffs-std-gif.o
-rw-r--r-- 1 tao tao 7000 Jan 26 15:32 wuffs-std-gzip.o
-rw-r--r-- 1 tao tao 21000 Jan 26 15:32 wuffs-std-json.o
-rw-r--r-- 1 tao tao 6272 Jan 26 15:32 wuffs-std-lzw.o
-rw-r--r-- 1 tao tao 9840 Jan 26 15:32 wuffs-std-nie.o
-rw-r--r-- 1 tao tao 56560 Jan 26 15:32 wuffs-std-png.o
-rw-r--r-- 1 tao tao 14720 Jan 26 15:32 wuffs-std-tga.o
-rw-r--r-- 1 tao tao 8984 Jan 26 15:32 wuffs-std-wbmp.o
-rw-r--r-- 1 tao tao 7064 Jan 26 15:32 wuffs-std-zlib.o
gcc9-dynamic:
-rwxr-xr-x 1 tao tao 94464 Oct 19 05:34 libwuffs.so
-rw-r--r-- 1 tao tao 9784 Oct 19 05:34 wuffs-base.lo
-rw-r--r-- 1 tao tao 2512 Oct 19 05:34 wuffs-std-adler32.lo
-rw-r--r-- 1 tao tao 19568 Oct 19 05:34 wuffs-std-crc32.lo
-rw-r--r-- 1 tao tao 20752 Oct 19 05:34 wuffs-std-deflate.lo
-rw-r--r-- 1 tao tao 27936 Oct 19 05:34 wuffs-std-gif.lo
-rw-r--r-- 1 tao tao 7232 Oct 19 05:34 wuffs-std-gzip.lo
-rw-r--r-- 1 tao tao 6552 Oct 19 05:34 wuffs-std-lzw.lo
-rw-r--r-- 1 tao tao 7360 Oct 19 05:34 wuffs-std-zlib.lo
gcc10-dynamic:
-rwxr-xr-x 1 tao tao 388712 Jan 26 15:33 libwuffs.so
-rw-r--r-- 1 tao tao 5632 Jan 26 15:33 wuffs-base-core.lo
-rw-r--r-- 1 tao tao 26904 Jan 26 15:33 wuffs-base-floatconv.lo
-rw-r--r-- 1 tao tao 8584 Jan 26 15:33 wuffs-base-intconv.lo
-rw-r--r-- 1 tao tao 7664 Jan 26 15:33 wuffs-base-interfaces.lo
-rw-r--r-- 1 tao tao 4448 Jan 26 15:33 wuffs-base-magic.lo
-rw-r--r-- 1 tao tao 109488 Jan 26 15:33 wuffs-base-pixconv.lo
-rw-r--r-- 1 tao tao 3248 Jan 26 15:33 wuffs-base-utf8.lo
-rw-r--r-- 1 tao tao 4624 Jan 26 15:33 wuffs-std-adler32.lo
-rw-r--r-- 1 tao tao 28984 Jan 26 15:33 wuffs-std-bmp.lo
-rw-r--r-- 1 tao tao 18920 Jan 26 15:33 wuffs-std-bzip2.lo
-rw-r--r-- 1 tao tao 10040 Jan 26 15:33 wuffs-std-cbor.lo
-rw-r--r-- 1 tao tao 23056 Jan 26 15:33 wuffs-std-crc32.lo
-rw-r--r-- 1 tao tao 24264 Jan 26 15:33 wuffs-std-deflate.lo
-rw-r--r-- 1 tao tao 29808 Jan 26 15:33 wuffs-std-gif.lo
-rw-r--r-- 1 tao tao 8328 Jan 26 15:33 wuffs-std-gzip.lo
-rw-r--r-- 1 tao tao 20128 Jan 26 15:33 wuffs-std-json.lo
-rw-r--r-- 1 tao tao 7128 Jan 26 15:33 wuffs-std-lzw.lo
-rw-r--r-- 1 tao tao 11960 Jan 26 15:33 wuffs-std-nie.lo
-rw-r--r-- 1 tao tao 68440 Jan 26 15:33 wuffs-std-png.lo
-rw-r--r-- 1 tao tao 15736 Jan 26 15:33 wuffs-std-tga.lo
-rw-r--r-- 1 tao tao 10440 Jan 26 15:33 wuffs-std-wbmp.lo
-rw-r--r-- 1 tao tao 7744 Jan 26 15:33 wuffs-std-zlib.lo
gcc9-static:
-rw-r--r-- 1 tao tao 106120 Oct 19 05:34 libwuffs.a
-rw-r--r-- 1 tao tao 9784 Oct 19 05:34 wuffs-base.o
-rw-r--r-- 1 tao tao 2448 Oct 19 05:34 wuffs-std-adler32.o
-rw-r--r-- 1 tao tao 19488 Oct 19 05:34 wuffs-std-crc32.o
-rw-r--r-- 1 tao tao 20632 Oct 19 05:34 wuffs-std-deflate.o
-rw-r--r-- 1 tao tao 27528 Oct 19 05:34 wuffs-std-gif.o
-rw-r--r-- 1 tao tao 7144 Oct 19 05:34 wuffs-std-gzip.o
-rw-r--r-- 1 tao tao 6448 Oct 19 05:34 wuffs-std-lzw.o
-rw-r--r-- 1 tao tao 7248 Oct 19 05:34 wuffs-std-zlib.o
gcc10-static:
-rw-r--r-- 1 tao tao 480206 Jan 26 15:33 libwuffs.a
-rw-r--r-- 1 tao tao 5632 Jan 26 15:33 wuffs-base-core.o
-rw-r--r-- 1 tao tao 27008 Jan 26 15:33 wuffs-base-floatconv.o
-rw-r--r-- 1 tao tao 8768 Jan 26 15:33 wuffs-base-intconv.o
-rw-r--r-- 1 tao tao 7560 Jan 26 15:33 wuffs-base-interfaces.o
-rw-r--r-- 1 tao tao 4448 Jan 26 15:33 wuffs-base-magic.o
-rw-r--r-- 1 tao tao 109408 Jan 26 15:33 wuffs-base-pixconv.o
-rw-r--r-- 1 tao tao 3824 Jan 26 15:33 wuffs-base-utf8.o
-rw-r--r-- 1 tao tao 4680 Jan 26 15:33 wuffs-std-adler32.o
-rw-r--r-- 1 tao tao 29512 Jan 26 15:33 wuffs-std-bmp.o
-rw-r--r-- 1 tao tao 19000 Jan 26 15:33 wuffs-std-bzip2.o
-rw-r--r-- 1 tao tao 10400 Jan 26 15:33 wuffs-std-cbor.o
-rw-r--r-- 1 tao tao 23112 Jan 26 15:33 wuffs-std-crc32.o
-rw-r--r-- 1 tao tao 24936 Jan 26 15:33 wuffs-std-deflate.o
-rw-r--r-- 1 tao tao 30232 Jan 26 15:33 wuffs-std-gif.o
-rw-r--r-- 1 tao tao 8424 Jan 26 15:33 wuffs-std-gzip.o
-rw-r--r-- 1 tao tao 21456 Jan 26 15:33 wuffs-std-json.o
-rw-r--r-- 1 tao tao 7144 Jan 26 15:33 wuffs-std-lzw.o
-rw-r--r-- 1 tao tao 12168 Jan 26 15:33 wuffs-std-nie.o
-rw-r--r-- 1 tao tao 71504 Jan 26 15:33 wuffs-std-png.o
-rw-r--r-- 1 tao tao 16184 Jan 26 15:33 wuffs-std-tga.o
-rw-r--r-- 1 tao tao 10520 Jan 26 15:33 wuffs-std-wbmp.o
-rw-r--r-- 1 tao tao 7904 Jan 26 15:33 wuffs-std-zlib.o
## Comparison
Below are some standard C libraries shipped as part of Debian Testing as of
October 2019. The numbers aren't directly comparable, as these libraries have a
Below are some standard C libraries shipped as part of Debian 11 Bullseye as of
January 2023. The numbers aren't directly comparable, as these libraries have a
richer API, especially in providing an encoder and not just a decoder. Still,
it is a reference point for e.g. Wuffs (adler32 + crc32 + deflate + gzip +
zlib) vs libz and Wuffs (gif + lzw) vs libgif.
dynamic:
-rw-r--r-- 1 root root 113088 Aug 6 16:44 /lib/x86_64-linux-gnu/libz.so.1.2.11
-rw-r--r-- 1 root root 34656 Jun 5 2018 /usr/lib/x86_64-linux-gnu/libgif.so.7.0.0
-rw-r--r-- 1 root root 74688 Jul 20 2020 /lib/x86_64-linux-gnu/libbz2.so.1.0.4
-rw-r--r-- 1 root root 38752 Dec 21 2020 /lib/x86_64-linux-gnu/libgif.so.7.1.0
-rw-r--r-- 1 root root 231344 Sep 16 2020 /lib/x86_64-linux-gnu/libpng16.so.16.37.0
-rw-r--r-- 1 root root 113088 Aug 24 04:54 /lib/x86_64-linux-gnu/libz.so.1.2.11
static:
-rw-r--r-- 1 root root 50800 Jun 5 2018 /usr/lib/x86_64-linux-gnu/libgif.a
-rw-r--r-- 1 root root 143250 Aug 6 16:44 /usr/lib/x86_64-linux-gnu/libz.a
-rw-r--r-- 1 root root 80690 Jul 20 2020 /lib/x86_64-linux-gnu/libbz2.a
-rw-r--r-- 1 root root 51940 Dec 21 2020 /lib/x86_64-linux-gnu/libgif.a
-rw-r--r-- 1 root root 374338 Sep 16 2020 /lib/x86_64-linux-gnu/libpng16.a
-rw-r--r-- 1 root root 144866 Aug 24 04:54 /lib/x86_64-linux-gnu/libz.a
---
Updated on October 2019.
Updated on January 2023.
+12 -3
View File
@@ -1,7 +1,16 @@
# Changelog
## Work In Progress
## 2023-01-26 version 0.3.0
The headline feature is that we have a production quality PNG decoder. It's
also [the fastest, safest PNG decoder in the
world](https://nigeltao.github.io/blog/2021/fastest-safest-png-decoder.html).
There's also a [memory-safe, zero-allocation JSON
decoder](https://nigeltao.github.io/blog/2020/jsonptr.html).
The dot points below probably aren't of interest unless you're upgrading from
Wuffs version 0.2.
- Added `0b` prefixed binary numbers.
- Added `WUFFS_BASE__PIXEL_BLEND__SRC_OVER`.
@@ -14,6 +23,7 @@
- Added `cpu_arch`.
- Added `doc/logo`.
- Added `endwhile` syntax.
- Added `example/bzcat`.
- Added `example/cbor-to-json`.
- Added `example/convert-to-nia`.
- Added `example/imageviewer`.
@@ -37,7 +47,6 @@
- Added double-curly blocks.
- Added interfaces.
- Added iterate advance parameter.
- Added preprocessor.
- Added single-quoted strings.
- Added slice `uintptr_low_12_bits` method.
- Added tokens.
@@ -156,4 +165,4 @@ Wuffs version 0.1.
---
Updated on January 2020.
Updated on January 2023.
+4
View File
@@ -786,6 +786,10 @@ compute_exit_code(std::string status_msg) {
// (other non-zero exit codes). Specifically, exit code 2 for internal
// invariant violation, exit code 139 (which is 128 + SIGSEGV on x86_64
// linux) for a segmentation fault (e.g. null pointer dereference).
size_t i = status_msg.find('=');
if (i != std::string::npos) {
status_msg = status_msg.substr(0, i);
}
return (status_msg.find("internal error:") != std::string::npos) ? 2 : 1;
}
+4
View File
@@ -381,6 +381,10 @@ compute_exit_code(std::string status_msg) {
// (other non-zero exit codes). Specifically, exit code 2 for internal
// invariant violation, exit code 139 (which is 128 + SIGSEGV on x86_64
// linux) for a segmentation fault (e.g. null pointer dereference).
size_t i = status_msg.find('=');
if (i != std::string::npos) {
status_msg = status_msg.substr(0, i);
}
return (status_msg.find("internal error:") != std::string::npos) ? 2 : 1;
}
+5 -1
View File
@@ -479,7 +479,7 @@ class Callbacks : public wuffs_aux::DecodeJsonCallbacks {
top.has_map_key = false;
auto iter = jmap.find(top.map_key);
if (iter != jmap.end()) {
return "main: duplicate key: " + top.map_key;
return "main: duplicate key: key=" + top.map_key;
}
jmap.insert(iter, JsonMap::value_type(std::move(top.map_key),
std::move(jvalue)));
@@ -593,6 +593,10 @@ compute_exit_code(std::string status_msg) {
// (other non-zero exit codes). Specifically, exit code 2 for internal
// invariant violation, exit code 139 (which is 128 + SIGSEGV on x86_64
// linux) for a segmentation fault (e.g. null pointer dereference).
size_t i = status_msg.find('=');
if (i != std::string::npos) {
status_msg = status_msg.substr(0, i);
}
return (status_msg.find("internal error:") != std::string::npos) ? 2 : 1;
}
+5 -5
View File
@@ -85,15 +85,15 @@ extern "C" {
// each major.minor branch, the commit count should increase monotonically.
//
// WUFFS_VERSION was overridden by "wuffs gen -version" based on revision
// c27250f52fcfb936d4f0410d1199fe24c04b9713 committed on 2022-10-28.
// c698d81b4d6868f3c5e26d2d7c3b39fe97209ec0 committed on 2023-01-26.
#define WUFFS_VERSION 0x000030000
#define WUFFS_VERSION_MAJOR 0
#define WUFFS_VERSION_MINOR 3
#define WUFFS_VERSION_PATCH 0
#define WUFFS_VERSION_PRE_RELEASE_LABEL "rc.3"
#define WUFFS_VERSION_BUILD_METADATA_COMMIT_COUNT 3369
#define WUFFS_VERSION_BUILD_METADATA_COMMIT_DATE 20221028
#define WUFFS_VERSION_STRING "0.3.0-rc.3+3369.20221028"
#define WUFFS_VERSION_PRE_RELEASE_LABEL ""
#define WUFFS_VERSION_BUILD_METADATA_COMMIT_COUNT 3374
#define WUFFS_VERSION_BUILD_METADATA_COMMIT_DATE 20230126
#define WUFFS_VERSION_STRING "0.3.0+3374.20230126"
// ---------------- Configuration