php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login

go to bug id or search bugs for

Request #77281 zstandard compression support
Submitted: 2018-12-11 05:57 UTC Modified: 2018-12-11 14:39 UTC
From: rh_whatwg at skuldwyrm dot no Assigned:
Status: Suspended Package: *Compression related
PHP Version: Next Major Version OS:
Private report: No CVE-ID: None
View Add Comment Developer Edit
Welcome! If you don't have a Git account, you can't do anything here.
You can add a comment by following this link or if you reported this bug, you can edit this bug over here.
(description)
Block user comment
Status: Assign to:
Package:
Bug Type:
Summary:
From: rh_whatwg at skuldwyrm dot no
New email:
PHP Version: OS:

 

 [2018-12-11 05:57 UTC] rh_whatwg at skuldwyrm dot no 
Description:
------------
zstandard provides better compression at it's lowest level than deflate (zlib / gzip) on it's highest level.
zstandard is also faster that deflate on both compression and decompression.

https://facebook.github.io/zstd/

By the looks of it zstandard can become the longterm replacement/successor for both gzip and bzip.

The webserver ngninx has a feature request for zstandard and support should be added soon (I believe nginx master has it by now).
Weget2 supports it as well.

IANA and RFCs exists that define it properly so it should only be a matte of time before it joins brotli and gzip as part of the compression methods browsers support.

PHP can help speed up adoption by adding support for it in the next major (8.0) or if possible minor (7.4) release.

zstandard license is the 3 clause BSD, the previous controversy that facebook initially added added a "+PATENTS" clause has been removed, so the license is now the normal 3 clause BSD.

Speeds varies and there are a lot of benchmarks available now if you search the net (so you don't have to take my word for it). Speed can be on par with lz4 while compression can be on par with bzip and lzma. There are levels of compression from 1 to 19 with extra ultra levels from 20 to 22.

The reference implementation includes multithread support which roughly doubles the compression speed with no major impact on compression ratio, this "should" work fine on non-threadsafe PHP builds as well.

Patches

Add a Patch

Pull Requests

Add a Pull Request

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2018-12-11 14:39 UTC] cmb@php.net
-Status: Open +Status: Suspended
 [2018-12-11 14:39 UTC] cmb@php.net 
For reference:

  <https://tools.ietf.org/html/rfc8478>
  <https://github.com/kjdev/php-ext-zstd>

Anyhow, this topic certainly requires discussion on internals@,
and likely the RFC process[1].  Feel free to start it!  For the
time being, I'm suspending this ticket.

[1] <http://wiki.php.net/rfc/howto>
 [2024-04-06 11:13 UTC] rh_whatwg at skuldwyrm dot no 
I think it's time to dust this off again.

In Chromium v123 (latest public version of Chrome 123, and also Edge 123, and Opera and Vivaldi soon I assume)
zstd is now a content encoding. Both browsers state in accept-encoding that zstd is supported.
https://github.com/facebook/zstd/releases/tag/v1.5.6

There is a zstd PECL/PEAR package.
But with zstd being finally standard in browsers now perhaps a ext/php_zstd.so and ext\php_zstd.dll being bundled with PHP makes sense.

As an example in my case I pre-compress files and use Apache (or PHP) to serve zstd compressed content to browsers that support it.

So I have:
index.html
index.html.zst
index.html.gz

My CMS (which obviously is created in PHP) let me pre-compress .gz, and right now I have to use a "hack" and use the windows executable and sort of do thing manually (the webhost does not let me install PECL directly).

If you check phpinfo() you'll find that the zip extension actually exposes a "ZSTD compression	No"


The main benefit of zst (zstd) over gz (zlib/deflate) is that zstd has the same or better compression ration compared to gz but zstd is much faster in both compression and decompression.

zstd also supports "Training" which is basically creating a dictionary from several files, useful for small files with common headers is it let the compressor start with a filled window with things to lookup.

And new with v1.5.6 is that you can split up the stream into "blocks" which will allow very low latency from server to client for longer data (so the browser can start working on things without waiting for the entire compressed stream to be downloaded).

The PECL php extension is at https://github.com/kjdev/php-ext-zstd
and is still only at v1.5.5 of zstd.
 
PHP Copyright © 2001-2024 The PHP Group
All rights reserved. 		Last updated: Wed Apr 24 11:01:30 2024 UTC