php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Doc Bug #71749 file_cache_consistency_checks has no effect
Submitted: 2016-03-09 01:47 UTC Modified: 2021-07-08 11:56 UTC
Votes:4
Avg. Score:4.2 ± 0.8
Reproduced:2 of 2 (100.0%)
Same Version:0 (0.0%)
Same OS:0 (0.0%)
From: iquito at gmx dot net Assigned:
Status: Verified Package: opcache
PHP Version: 7.0.4 OS:
Private report: No CVE-ID: None
 [2016-03-09 01:47 UTC] iquito at gmx dot net
Description:
------------
I did a lot of tests with the new opcache.file_cache possibilities and think that these new features are great. But as far as I can tell, "opcache.file_cache_consistency_checks=1" has no effect - there are no checks if files have changed or if the checksum is still the same, at least I could not make out any noticeable effects.

This is a problem because it is now a bit confusing and hard to clear the file cache - the only sure way to clear the cache is to delete the cached .bin file. If "opcache.validate_timestamps" is set to 1, then that also seems to work for the file_cache - an existing file_cache .bin file is compared with the timestamp of the original .php file.

But in production, "opcache.validate_timestamps" is better set to zero, to avoid file system requests. Until now, when deploying a new version of an application, I just called "opcache_reset();" to reload the application, which worked perfectly: in an instant, the old code is gone and the new code is loaded.

With validate_timestamps set to 0 and with the new file_cache, the file is instead immediately re-cached from the file system and not checked against the original file - meaning a newly deployed application is not loaded, the previous state is loaded from the file system. I thought "opcache.file_cache_consistency_checks" was to specifically address this issue - check if the file_cache version is still the same as the original file. But this does not seem to work, in all my tests.

Test script:
---------------
Use the following settings:

opcache.enable=1
opcache.enable_cli=1
opcache.validate_timestamps=0
opcache.file_cache="/srv/php-file-cache"
opcache.file_cache_only=1
opcache.file_cache_consistency_checks=1

Then call a simple script from PHP CLI (opcache enabled), like "<?php echo '5';" - this file will now be cached. If you change the file to "<?php echo '7';", this change will never be displayed until you delete the cached file from the file system.


Patches

Pull Requests

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2016-03-09 11:19 UTC] inefedor at gmail dot com
Looking at the code it seems you misunderstood the purpose of file_cache_consistency_checks directive. If I'm not mistaken it's only needed for checking if *.bin files are not damaged (i.e. their checksum is the same).
 [2016-03-09 13:50 UTC] iquito at gmx dot net
That is possible - the only documentation so far for this feature (that I could find) is in php.ini, stating:

"Enables or disables checksum validation when script loaded from file cache."

What is validated how is not explained, I thought it would compare the checksum of the binary file with the original file. If I am misinterpreting this feature, then I think there is at least one important feature missing for file_cache, something like

opcache.file_cache_validate_timestamps
(and maybe also something similar to opcache.revalidate_freq for file_cache)

Because otherwise you would need to enable opcache.validate_timestamps at all times when using file_cache, which is counterproductive in PHP-FPM, where you want to check if a file from file_cache is still up-to-date (because file_cache is the fallback cache) after an "opcache_reset();" or a restart, but when it is loaded to shared memory it should not be checked anymore.

There could also be something like a "opcache_file_reset();", after which PHP-FPM/PHP-CLI checks the next loading of a file_cache entry only once, which will benefit CLI - so CLI does not have to check every cached file all the time. This would be really great, but probably also difficult to implement, because that information would need to be persisted somewhere for both CLI and FPM to access it.

As it is now, I think file_cache is not easily usable in PHP-FPM, because no option to only recheck the cached files exist and many people use "opcache_reset();" with "opcache.validate_timestamps=0", which is perfect for deploying applications without downtime and errors (exchanging versions in an instant), but is also not compatible with the new file_cache.
 [2016-03-10 03:48 UTC] rasmus@php.net
I think you are overestimating the performance impact of opcache.validate_timestamps, Set your frequency high enough and I challenge you to measure any impact. And when it comes to deploying new code, the fact that you don't need to clear your cache means a deploy will not impact site performance. Assuming your deploy system is done right, of course. See https://codeascraft.com/2013/07/01/atomic-deploys-at-etsy/
 [2016-03-10 04:25 UTC] iquito at gmx dot net
I actually measured the impact of opcache.validate_timestamps - each time it triggers, my application is about 10ms slower, which in my application means between 35% and 100% slower than usual. That is a big impact, and I feel it is unnecessary. Me telling PHP when something has changed seems much more sensible than PHP checking itself if something has changed, because I can tell PHP exactly when it should check for changes.

I am sure there are better deployment models than mine - but the one I use actually works really well for me: If the whole application is in the opcache, I can exchange the whole application while PHP-FPM continues to run, and only when I tell PHP to reload the cache the whole application is replaced. Even with 100+ requests per second and a lot of concurrency I never had a single error/failure/problem in years with this kind of deployment, so it seems sound to me.

Giving a developer the chance to decide for himself when a cache should be rechecked, and having different options to do so, would just lead to more use cases where something like a file_cache can be used and can lead to new solutions. I am a big fan of opcache, and I think the file_cache is a great idea, but without more options to influence how memory opcache and file_cache interact and when file_cache should be "reset"/rechecked some solutions to existing problems are just not possible - for example my usage, but maybe also others. My suggestions are maybe not even really good, I am sure there are many possibilities for sensible and flexible settings.
 [2016-03-10 05:21 UTC] rasmus@php.net
10ms in stats on a single request? You either have a seriously slow OS/filesystem combination or you are measuring something incorrectly. Linux caches inode entries so you never actually touch the disk since presumably repeated stat'ing every 10-20 seconds is going to have an extremely high cache hit rate.

On my production servers, a stat takes approximately 0.00034ms which means that in order for that to add 10ms to a request it would take on the order of 30k stats on a single request. That is a lot of includes!
 [2016-03-10 15:28 UTC] iquito at gmx dot net
My tests only went on for a few minutes, and it was always between 7ms and 10ms slower when the request hit opcache.validate_timestamps. Why it takes so long - I have no idea. But the numbers are certainly correct.

Still, the "pull-only-mentality" of opcache.validate_timestamps does not seem sensible to me - the application should be able to choose when to do that. If I could choose new features for opcache, the following would seem perfect:

An additional function in PHP like opcache_revalidate_timestamps() which would check all timestamps within opcache once instead of resetting, and would recache changed files (if they still exist) or maybe just remove them from opcache if they are changed, both would be possible. Even if 20'000 files need to be checked, doing this once when specifically asked would be more manageable/certainly faster than PHP doing it in regular intervals. Because PHP is already doing this internally at fixed intervals when opcache.validate_timestamps is set, such a function should be possible.

This would then be an interesting alterative to opcache_reset(), because most of the opcache will stay intact, and file_cache could have an equivalent like opcache_revalidate_timestamps_file_cache() which would do the same for all .bin files. That way all opcache data can be checked with two functions when needed, and the cache does not need to be reset very often.
 [2017-03-07 07:41 UTC] persiantools at yahoo dot com
The OP has reported a genuine bug in my opinion, but he steered the discussion towards consistency checks. I have the exact same problem and expect that opcache_reset will really reset opcache whether it be in-memory cache or file chaches.

However, It is not the case when you have file caches enabled and set the validate_timestamps to false. In this scenario, calling opcache_reset or even a web server restart has no effect.
 [2019-08-17 01:20 UTC] rolmos at endertech dot com
Experiencing the same issue. What's the official method to clear the memory & file cache so that changes are detected by opcache, without a race condition of manually deleting cache files?
 [2021-07-08 11:54 UTC] cmb@php.net
-Type: Bug +Type: Documentation Problem
 [2021-07-08 11:54 UTC] cmb@php.net
> The OP has reported a genuine bug in my opinion, but he steered
> the discussion towards consistency checks.

This ticket is explicitely about .file_cache_consistency_checks,
and that documentation should be improved.  If you think there is
another bug, and this is still relvant for any of the actively
supported PHP versions[1], please file a new bug report.

> If I'm not mistaken it's only needed for checking if *.bin files
> are not damaged (i.e. their checksum is the same).

Right.  This should be documented.

> What's the official method to clear the memory & file cache so
> that changes are detected by opcache, without a race condition of
> manually deleting cache files?

See <https://codeascraft.com/2013/07/01/atomic-deploys-at-etsy/>
and <http://blog.jpauli.tech/2015-03-05-opcache-html/>.

[1] <https://www.php.net/supported-versions.php>
 [2021-07-08 11:56 UTC] cmb@php.net
-Status: Open +Status: Verified
 [2023-05-20 06:02 UTC] wirednewsfeed at gmail dot com
Weird News Feed Blog are sharing latest news about auto, tech, travel, home improvement, health, business, finance, loan, investment etc. More info to visit: (https://weirdnewsfeed.com)github.com
 
PHP Copyright © 2001-2024 The PHP Group
All rights reserved.
Last updated: Thu Nov 21 14:01:29 2024 UTC