php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Bug #75910 convert.base64-encode omits padding bytes
Submitted: 2018-02-02 22:09 UTC Modified: 2021-07-27 15:05 UTC
Votes:9
Avg. Score:4.3 ± 1.1
Reproduced:8 of 8 (100.0%)
Same Version:2 (25.0%)
Same OS:2 (25.0%)
From: me at levisflorian dot name Assigned: cmb (profile)
Status: Duplicate Package: Streams related
PHP Version: 7.2.2 OS: macOS 10.13.3 (via brew)
Private report: No CVE-ID: None
Welcome back! If you're the original bug submitter, here's where you can edit the bug or add additional notes.
If you forgot your password, you can retrieve your password here.
Password:
Status:
Package:
Bug Type:
Summary:
From: me at levisflorian dot name
New email:
PHP Version: OS:

 

 [2018-02-02 22:09 UTC] me at levisflorian dot name
Description:
------------
When using temp stream, filter convert.base64-encode omits padding bytes.
Actually on php 7.2.1 (instead of 7.2.2 selected above) - multiple php versions installed with brew.sh

Found a similar bug on old php version #68532 (https://bugs.php.net/bug.php?id=68532)

Test script:
---------------
Please find test script (based on #68532) here: https://gist.github.com/Gounlaf/c54dce4f843e89d4f5814eb7790cdcd3

Expected result:
----------------
encode - low-level - memory = dGVzdA==
decode - low-level - memory = test
encode - low-level - temp = dGVzdA==
decode - low-level - temp = test


Actual result:
--------------
encode - low-level - memory = dGVzdA==
decode - low-level - memory = test
encode - low-level - temp = dGVz
decode - low-level - temp = test


Patches

filter-flag-on-eof (last revision 2018-09-20 05:07 UTC by tenzzor at gmail dot com)

Pull Requests

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2018-02-02 22:19 UTC] me at levisflorian dot name
-Package: Filter related +Package: Streams related
 [2018-02-02 22:19 UTC] me at levisflorian dot name
Change "package" from "Filter related" to "Streams related"
 [2018-07-08 18:01 UTC] cmb@php.net
-Status: Open +Status: Verified -Assigned To: +Assigned To: pollita
 [2018-07-08 18:01 UTC] cmb@php.net
Confirmed: <https://3v4l.org/HFW1e>.

Apparently, this regression has been introduced by commit
0a45e8f[1].  Sara, could you please have a look at this?

[1] <https://github.com/php/php-src/commit/0a45e8f096a04464bda6277c6f3d0b5461737a27>
 [2018-09-20 05:13 UTC] tenzzor at gmail dot com
Hi guys. Found the same issue.
I added a patch, which I think might be more generic solution. 
In case of filtered stream, while loops until eof on stream. However, when we successfully read from stream and reached EOF on stream, flags are still set to NORMAL, next iteration won't happen, because there is EOF and filter is never notified of EOF.

This does not happen on php://memory stream right now, because on read it returns all data up to EOF, but does not mark stream as EOF. Then on next read it figures there is no more data in buffer, marks stream as eof and returns 0 read.
 [2019-03-14 12:09 UTC] alessandro dot lai85 at gmail dot com
It seems that this bug is not limited to macOS, and it's affecting PHP 7.2.0 up to the latest 7.3 (which is 7.3.3 RN).

Proof: https://3v4l.org/NnFV2
 [2019-03-14 15:24 UTC] pollita@php.net
Looking at this, I think your expected is backwards.

You *should* expect:

encode - low-level - memory = dGVz
decode - low-level - memory = test
encode - low-level - temp = dGVz
decode - low-level - temp = test

Which is still not what you get, of course, but before any fixing work is done, we should agree on what's broken.

In the encode cases (let's ignore decode for the moment, but there may be a hidden issue there too) a partial block has been written to the stream (base64 uses 3-byte input blocks), meaning that on read, we don't know if the string is done (and we should nil bad) or if more data will eventually come (and we should block).  The fact that the stream hasn't been closed means that we should be assuming that additional data will come, ergo we should block, and we should NOT get a second block of encoded data out of the filter.

Decode may have a similar issue if we fed it non-block-aligned input (not a multiple of 4-chars), but I haven't verified that either way yet.
 [2019-03-22 19:19 UTC] me at levisflorian dot name
Hi,

what pollita explained - if I understand correctly - make senses.

I have some questions:
- is there something in the stream telling it's the end?
- is there a way to do the padding "manually"?
 [2020-04-24 01:44 UTC] kazkiti0702 at gmail dot com
It doesn't seem to be fixed even in the current latest version.

The result of my experiment
[Behavior by version]
7.4.5  dGVz
7.3.15 dGVz
7.2.30 dGVz
7.2.28 dGVz
7.2.26 dGVz
7.2.20 dGVz
7.2.1  dGVz
7.2.0  dGVz
7.1.33 dGVzdA==
7.0.33 dGVzdA==
5.6.40 dGVzdA==
5.5.38 dGVzdA==
5.5.30 dGVzdA==
5.4.45 dGVz
5.3.29 dGVz
 [2021-07-27 15:05 UTC] cmb@php.net
-Status: Verified +Status: Duplicate -Assigned To: pollita +Assigned To: cmb
 [2021-07-27 15:05 UTC] cmb@php.net
> In the encode cases (let's ignore decode for the moment, but
> there may be a hidden issue there too) a partial block has been
> written to the stream (base64 uses 3-byte input blocks), meaning
> that on read, we don't know if the string is done (and we should
> nil bad) or if more data will eventually come (and we should
> block).

Note that the stream is rewound, before it is read.  This requires
a flush before the rewind, so the encoding *needs* to be finished.

Anyhow, this is actually a duplicate of bug #77069 (fixed as of
PHP 7.4.14 and 8.0.1, respectively).
 
PHP Copyright © 2001-2024 The PHP Group
All rights reserved.
Last updated: Thu Nov 21 14:01:29 2024 UTC