-
Notifications
You must be signed in to change notification settings - Fork 9.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
PageCache: async rendering of blocks can corrupt layout cache #8554
Comments
Ok, I did some debugging. It seems to only affect CMS pages and is related to the way how
This is done in
However, because of the way how
I think the layout is not loaded because our area in this case is Now, the bad thing happens. This structure gets stored into the cache, if there is none. This is done in
The For normal requests on CMS pages, this is not happening, because the controller sets the I would really appreciate any advice on how to fix this. |
@bka Thanks for sharing your findings. I'm curious whether you spent time to find a solution to this issue? I'm experiencing a very similar issue with an empty homepage (where the body tag is empty). Until I read your issue description, I wasn't able to consistently reproduce the issue, but I'm now able to reproduce it if I follow these steps:
If I reverse steps 2 and 3, the homepage content will contain content like expected, and the curl request for the menu/navigation content will also contain content. On a side note, I initially suspected Varnish was at fault with this issue as I know that an improper Varnish configuration can cause the catalog navigation to go missing. I've been experiencing issues with both a blank homepage and then sometimes the navigation is missing (but the rest of the homepage content is loaded). I was able to figure out the I was curious to see what combination of handles were successful vs not, so I ran Broken requests:
Working requests:
|
Hi @erikhansen, thanks for your feedback. I have some additional insights. This issue relates to usage of Having a block defined with isScopePrivate will trigger async loading of blocks also known as Varnish Hole Punching. Sad thing is, if this Block is present on the homepage, indicated by handles Related source files are:
I fixed this by patching
|
@bka - Thanks for your feedback. While your workaround might fix your specific manifestation of this issue, it doesn't fix mine. And I don't think my manifestation of the issue has anything to do with @magento - I was able to easily reproduce my issue on a vanilla Magento 2 install using the steps I outlined in my comment above. |
@bka - I have a solution to the issue. You can reference the fix in this commit: 99bb170 Try applying this patch to your environment and see if the issue goes away: diff --git a/vendor/magento/module-ui/view/base/layout/default.xml b/vendor/magento/module-ui/view/base/layout/default.xml
index 7f2efbd..64d5f14 100644
--- a/vendor/magento/module-ui/view/base/layout/default.xml
+++ b/vendor/magento/module-ui/view/base/layout/default.xml
@@ -5,7 +5,7 @@
* See COPYING.txt for license details.
*/
-->
-<page xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" layout="admin-1column" xsi:noNamespaceSchemaLocation="urn:magento:framework:View/Layout/etc/page_configuration.xsd">
+<page xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework:View/Layout/etc/page_configuration.xsd">
<body>
<referenceContainer name="after.body.start">
<block class="Magento\Ui\Block\Logger" name="logger" template="Magento_Ui::logger.phtml"/>
If the issue does go away, I'd recommend closing this issue. |
@bka Have you had a chance to test my suggested patch yet? |
@bka Any update on this? |
The root cause of this issue is that the "pageLayout" parameter for the layout builder is missing. However before starting blocks and XML generation Magento adds additional blocks to the list based on selected pageLayout: 1column, 2columns or empty. By default for the most of the pages the layout is: 1column. As the result: almost all default pages are working as expected. But once you customize your theme a little bit, using "move" actions, the full page cache becomes broken. The solution is pretty complex: I have overridden Magento\Framework\View\Page\Builder, Magento\Framework\View\Layout, Magento\PageCache\Observer\ProcessLayoutRenderElement and Magento\PageCache\Controller\Block\Esi, by adding a new parameter to the URL and to the controller, named: "page_layout" and then I set it into the view in the Esi controller. Works for me. |
@andreymoskvenkov Can you provide a patch so we can verify your solution? |
Sorry, but I can provide only a tip, which I have already done. Tips are free :) |
@andreymoskvenkov C'mon man, this is opensource. Take a little, give a little ;) All kidding aside, I appreciate your tip, but I'm sure your patch would be appreciated even more and it could be validated. If you can't share it for whatever reason, I understand. |
@cherreman Did you happen to try the patch I included in my comment on February 28th? |
Yeah, there are many reasons of not sharing the code, one of them is - NDA of the company, where I am working for. We have fixed a lot of core bugs of M2 internally and this is one of the most difficult to find. And why have I posted a tip here is - because this issue number #8554 has also given me a tip about layout type such as "1column", "empty", etc. I was debugging it almost whole day and was starting to become a crazy, because I could not understand: why in the Layout object's "_blocks" property the list of generated blocks are only a few of them in page_cache controller, however in CMS_index controller the list of blocks in "_blocks" property is big: about 79 blocks there. And at the same time the final XML in both actions is almost the same! The cause of that was: because Magento adds some blocks to the "_blocks" property manually in a separate method depending on what "pageLayout" is set. As the result: in page_cache action no page layout was passed and no page layout was set. |
…ento#8554 magento#9050 Adapted page cache tests
…ento#8554 magento#9050 Update additional pagecache layout handle name
#9560 seems to fix this issue, can you try that fix and report back? I'd like to merge that PR but I will need some help from you 😄 |
:-D |
@adrian-martinez-interactiv4 can you chime in here? |
@andreymoskvenkov If I have understood well, you are proposing to include the additional handle in the ajax request, instead of putting it within the block page cache controller. I don't know if I'm missing something, could you clarify a bit your previous explanation pls? Thanks in advance. |
No, I propose to add an additional parameter to the PageCache's URL: 'page_layout', where to put what Layout type was used on the current page and then to add a processing of this parameter in the ESI controller, which should just take the value from that parameter from the request and to set it into Layout object. Then everything will work correctly. And this is not easy btw, because of crazy Magento 2 architecture (from what I see the core developers instead of thinking more deeply, they just split responsibilities between classes randomly and apply "protected", "public" and "private" keywords also randomly :) ). It is not a "handler", it is an additional characteristics (or parameter, or property) of Layout entity. |
Well, I'm been doing some research about this issue. I'll be explaining each point later, but long story short:
About point 1: And this one shows the empty body: Same steps reproduced at 2.2.0-dev, first shows non-empty body, as expected: But the same steps that resulted in an empty body in CE 2.1.6, return non-empty body in 2.2.0-dev: Without going too deep in what commits are involved in solving this issue, what seems clear is that this issue, as long as #9050 , that I consider a duplicate of this one, are already solved. About point 2: About point 3: As this issue seems to be solved, I'll continue giving feedback about #9560 in the thread of that PR. |
:-D |
@andreymoskvenkov can you clarify what "we have an internal patch" means? If you have a patch for this or other issues, would you kind sharing them for others to benefit? Thanks! |
It meas, that I have fixed this issue and wrapped it up as a separate module. However as I said earlier, I won't share it. Magento Community has helped me to find the root cause of the issue and that's why I was able to fix it. That's why I want to thank the community for this and share the root cause of it. However I will not share the fix, because if after my notes you cannot fix the issue like I fixed it, then you are useless as Magento specialists and will not be able to provide high quality assistance to your clients on Magento 2 platform. So the goal of my posts here is: to introduce an exercise, which is based on one of real problems, which faces a Magento developer almost every day. I want to make the community stronger. Just providing a final fix will not improve the level of community developers. |
That is not how any of this works. |
if one could see and read the fix, one can learn from it, trying to understand what the problem was and how the solution was build. For me thats a giant part of OSS. If you have a patch and don't want to share it, okay, go your way, but please don't brag (it's nothing more to me) about it. (And yes, it is about you, since you were implying, that you are good and if one can't do one thing you can, he or she is useless) |
Internal ticket to track issue progress: MAGETWO-69018 |
…ento#8554 magento#9050 magento#9560 Create cache key object to be injected separately with its own interface
…ento#8554 magento#9050 magento#9560 Renamed interface, LayoutCacheKeyInterface made optional in constructor, injected via di.xml, some other little fixes
…ento#8554 magento#9050 magento#9560 Adapted PageCache and Framework tests
…ento#8554 magento#9050 magento#9560 Adapted PageCache and Framework tests
…ento#8554 magento#9050 magento#9560 Adapted PageCache and Framework tests
…ento#8554 magento#9050 magento#9560 Adapted PageCache and Framework tests
…ento#8554 magento#9050 magento#9560 Adapted PageCache and Framework tests
@bka, thank you for your report. |
…ayout cache #8554 #9050 #11174 - Merge Pull Request #11174 from adrian-martinez-interactiv4/magento2:FR#FIX-PAGECACHE-LAYOUT-CACHE-KEY - Merged commits: 1. 3bdfa1b 2. 5a6a000 3. bf7df0d 4. 7feba6c 5. 1bb25ed 6. 0e830c1 7. 8a92e46 8. 0a69b5b 9. bd6ed7c 10. c6e2b39 11. 3cc51dc 12. 2ae9c26 13. 516b529 14. efa3edb 15. 3f60168
Internal ticket to track issue progress: MAGETWO-80647 |
Hi folks,
I disovered a potential bug regarding the layout cache which leads to a blank page. If this happens, customers may see a white page without any hmtl inside the body tags. It might be related to #6942, although there is no php-fpm or libxml in the game here.
Preconditions
Steps to reproduce
bin/magento cache:clean
curl -gI "http://magento2.local/page_cache/block/render/?ajax=1&blocks=[%22header.login%22]&handles=[%22default%22%2C%22cms_index_index%22%2C%22cms_page_view%22%2C%22cms_index_index_id_home%22]&originalRequest=a&version="
curl -s http://magento2.local | grep "<body" -A 3 -B 3
Expected result
Actual result
Discussion
This ajax call seems to destroy the layout cache. If it is not present, structure which gets loaded and unserialized in lib/internal/Magento/Framework/View/Layout.php:generateElements from cache does not contain any root element anymore:
Cleaning the cache again and loading the front page normally generates a valid layout cache, where root is present:
I think it has something do to with the used handles
[default, cms_index_index, cms_page_view, cms_index_index_id_home]
which areused for generating the cache key in
getCacheId
. If there is no cache present, the page cache block just writes its content into the layout cache although there is no root layout used.I need to dive deeper into this issue. It normally should not happen, because the DOM should get loaded first. And subsequent ajax calls should always have a layout cache present. But if cache gets cleared manually and there is already an ajax call on its way it can lead to a broken layout cache.
The text was updated successfully, but these errors were encountered: