Run al-folio without external ressources/cdns | Privacy / Offline Version #1181
Comments
|
It is a very good idea. Not only for privacy, it also reduces external calls and that usually improves performance (depending on where the web is hosted). I don't know how it could be done automatically, but I think that by writing a plugin in Ruby you could parse the external dependencies, save them in a folder and modify the urls to point to the new location. |
|
I manually integrated the assets on my site. I used the following wget --quiet --recursive --level=2 --no-parent --no-host-directories --include-directories=/ \
--span-hosts --domains localhost,cdn.jsdelivr.net \
--accept=.js,.map,.css,.woff,.woff2,.ttf \
--execute=robots=off \
--directory-prefix assets/thirdparty/ \
http://localhost:4000Then, I added new config variables for each host in each file, such as <link rel="stylesheet" href="{{ site.cdn.jsdelivr }} ..
and in cdn:
# jsdelivr: "https://cdn.jsdelivr.net"
jsdelivr: "/assets/thirdparty"This is far away from being a good solution. Regarding page speed: |
|
Thanks, @CheariX. I think getting assets locally can indeed improve the loading speed. I found a related reference from the chirpy theme (similar to your solution), maybe it's also suitable for the al-folio: |
|
You can manually integrate these files on your site, this can probably improve page speed a little, but not sure about huge bumps of speed. Note that the current speed has improved a lot, and since the addition of purgecss in #1562 we changed some libraries to be used locally. One of the benefits of using a CDN is the simpler update process of the libraries (e.g.: changing a library version in config vs downloading the library and putting in the right directory, renaming and so on). |
Please note that this issue is not about speed, but about privacy. |
|
It might be possible to write a plugin for this, it is worth the investigation. But I don't know if I will have the time for this in the next couple of weeks. |
Implementing a Privacy-Friendly, Offline Version of al-folio Theme with Local Resource HostingIn response to privacy concerns and the necessity for an offline-capable version of the al-folio theme, I've crafted a method to locally host all external resources. This approach not only mitigates privacy issues but also improves website loading speed by reducing external dependencies. The ChallengeAl-folio's default setup heavily relies on external resources for fonts, stylesheets, and scripts. Although convenient, this setup poses privacy issues and may degrade performance due to external requests. A discussion in GitHub issue #1181 highlighted the community's interest in a privacy-respecting, offline version. My SolutionTo align al-folio with privacy preferences and offline functionality, I developed a script that automates the downloading and local hosting of all necessary external resources. This script utilizes wget to fetch resources, arranging them into the appropriate directories within the project. The process includes:
I have hosted my shell script on GitHub Gist. Here is the link to it: This method ensures that all resources are available locally, enhancing privacy, reducing load times, and allowing the site to function entirely offline. |
|
I am developing a plug-in to download the files to allow offline usage. It is almost complete, but I am still working on the Google fonts part. As soon as it is finished I will point to it here then some of you could test it out and check if everything is working as expected. |
|
So, I created a plugin that downloads all javascript and css files defined in As I was trying to handle this, I found out that Google delivers fonts in a format appropriate to the requesting user-agent, so it is returning |
|
Please give #2312 a try to check if it fulfills your needs (note that tikzjax doesn't work this way). |
|
@george-gca Thank you for your work. Suggestion to Google Fonts Download: In my own script which I described in my previous comment I also did not get the download consistently, but I implemented "offline" Google Fonts the following way:
Edit 1: As I've just implemented my own Version of offline font's, I'm not going to try out your version (yet?). But great work! Thanks! Cheers, |
|
Please give my code a try. I implemented Google fonts download, the only thing that I didn't implement was downloading some stuff for Note that my code only does the modifications to the built site, not in the template part. This means that you'll only see the modifications on the generated |
Not yet (if at all). As I've painstakingly cobbled together my Script and manually set local paths. Sorry. But don't get me wrong: Great work! |
|
Have you managed to make tikzjax work offline? |
Nope! I think tikzjax and dimensions.ai did not work |
|
Thanks for your incredible work, @george-gca To use the PR, I needed to use I enabled the offline mode by setting In addition, I had to disable: This is totally fine, as they would need to request APIs from third-party servers anyway in order to work properly. The same holds true for the GitHub Repository plugin. I also found that emojies are causing third party request, such as in the File However, this might be out of scope, similarly to altmetric, dimension, and scholar.
It seems that some Google Fonts were not downloaded. Just found this in the logs. I could not find any visual issues.
In summary, I am more than happy with this PR. It does exactly what I wanted, and I would love to see it getting merged. |
|
Thanks for the testing. Can you please try it again to see if the plug-in downloads correctly the Google fonts? Maybe (just maybe) was a hickup in the internet that prevented the download, since it failed for only 3 files and not all of them. I will give jemoji a look to check what external links it access, but I agree with the rest, they are not related and must be dealt with the way you did. As for the video part, what would you suggest? Instead of embedding, just linking to the video (to be open in a new window)? |
I have attempted to recreate the problem, but unfortunately, I was unable to replicate the error on a new, clean run. This leads me to believe that the problem may not be as critical, particularly since I can't consistently reproduce the issue.
The video embedding is a bit strange for me.
|
|
Ah now I see, you are talking about videos in the bibliography part. I will give it a thought and update it here. |
|
I believe the current solution (displaying the video below in a small collapsible area) is not worth it. It is not even working properly, also the video is too small to be useful. I think I will change it to just pop in another window. So I think this solves this issue. Meanwhile, about the iframe solution, check the blog post about video and audio, which also use the same component. With this part:
You mean to have a flag in the config that, if set to true (or false) instead of displaying the video inline just redirects to the video in a new window, right? |
|
I think I will already merge #2312 and then do any other needed changes in other PRs. |
As discussed in #1181, I suggest to make embedding videos an optional feature. This behavior aligns well with recently merged PR #2312. Open questions: 1. I added a youtube link to `papers.bib`. Is this link okay? 2. I set `enable_video_embedding: false` as the default. I argue that privacy settings should be the default. Also, the current implementation of `video.liquid` only works for some very specific video URLs. For example, to embed youtube, specialized links must be used to avoid `X-Frame-Option` issues. This behavior can lead to a broken embedding, which would not look very nice. Feedback welcome.
As discussed in alshedivat#1181, I suggest to make embedding videos an optional feature. This behavior aligns well with recently merged PR alshedivat#2312. Open questions: 1. I added a youtube link to `papers.bib`. Is this link okay? 2. I set `enable_video_embedding: false` as the default. I argue that privacy settings should be the default. Also, the current implementation of `video.liquid` only works for some very specific video URLs. For example, to embed youtube, specialized links must be used to avoid `X-Frame-Option` issues. This behavior can lead to a broken embedding, which would not look very nice. Feedback welcome.
Created a plugin to tackle alshedivat#1181. Currently have an issue with tikzjax since it imports some wasm file from its javascript. The rest should work as expected. --------- Signed-off-by: George Araujo <george.gcac@gmail.com>
As discussed in alshedivat#1181, I suggest to make embedding videos an optional feature. This behavior aligns well with recently merged PR alshedivat#2312. Open questions: 1. I added a youtube link to `papers.bib`. Is this link okay? 2. I set `enable_video_embedding: false` as the default. I argue that privacy settings should be the default. Also, the current implementation of `video.liquid` only works for some very specific video URLs. For example, to embed youtube, specialized links must be used to avoid `X-Frame-Option` issues. This behavior can lead to a broken embedding, which would not look very nice. Feedback welcome.
|
I am also very interested in the offline mode and have just tested it. In general, it works well and is really easy to activate! Thanks for that! I have noticed problems with maps embedded via Leaflet. On the one hand, the integrity keys no longer fit because the files are changed. On the other hand, graphics expected from the library under /assets/libs/images/ are missing. Unfortunately, I don't have enough experience with the project here to be able to say whether this is a general problem or whether something needs to be adjusted specifically for Leaflet. |
|
If Leaflet expects some files to be available locally and they are not, this means that they should also be downloaded. But we should check if this is feasible, or if is something more like is the case for tikzjax, which could not be automated and it support must be dropped for the offline version or it should be done manually. Also the integrity keys should work since they are the same file, shouldn't they? Can you open a new issue for this? |
Created a plugin to tackle alshedivat#1181. Currently have an issue with tikzjax since it imports some wasm file from its javascript. The rest should work as expected. --------- Signed-off-by: George Araujo <george.gcac@gmail.com>
As discussed in alshedivat#1181, I suggest to make embedding videos an optional feature. This behavior aligns well with recently merged PR alshedivat#2312. Open questions: 1. I added a youtube link to `papers.bib`. Is this link okay? 2. I set `enable_video_embedding: false` as the default. I argue that privacy settings should be the default. Also, the current implementation of `video.liquid` only works for some very specific video URLs. For example, to embed youtube, specialized links must be used to avoid `X-Frame-Option` issues. This behavior can lead to a broken embedding, which would not look very nice. Feedback welcome.
Created a plugin to tackle alshedivat#1181. Currently have an issue with tikzjax since it imports some wasm file from its javascript. The rest should work as expected. --------- Signed-off-by: George Araujo <george.gcac@gmail.com>
As discussed in alshedivat#1181, I suggest to make embedding videos an optional feature. This behavior aligns well with recently merged PR alshedivat#2312. Open questions: 1. I added a youtube link to `papers.bib`. Is this link okay? 2. I set `enable_video_embedding: false` as the default. I argue that privacy settings should be the default. Also, the current implementation of `video.liquid` only works for some very specific video URLs. For example, to embed youtube, specialized links must be used to avoid `X-Frame-Option` issues. This behavior can lead to a broken embedding, which would not look very nice. Feedback welcome.
Created a plugin to tackle alshedivat#1181. Currently have an issue with tikzjax since it imports some wasm file from its javascript. The rest should work as expected. --------- Signed-off-by: George Araujo <george.gcac@gmail.com>
As discussed in alshedivat#1181, I suggest to make embedding videos an optional feature. This behavior aligns well with recently merged PR alshedivat#2312. Open questions: 1. I added a youtube link to `papers.bib`. Is this link okay? 2. I set `enable_video_embedding: false` as the default. I argue that privacy settings should be the default. Also, the current implementation of `video.liquid` only works for some very specific video URLs. For example, to embed youtube, specialized links must be used to avoid `X-Frame-Option` issues. This behavior can lead to a broken embedding, which would not look very nice. Feedback welcome.
Created a plugin to tackle alshedivat#1181. Currently have an issue with tikzjax since it imports some wasm file from its javascript. The rest should work as expected. --------- Signed-off-by: George Araujo <george.gcac@gmail.com>
As discussed in alshedivat#1181, I suggest to make embedding videos an optional feature. This behavior aligns well with recently merged PR alshedivat#2312. Open questions: 1. I added a youtube link to `papers.bib`. Is this link okay? 2. I set `enable_video_embedding: false` as the default. I argue that privacy settings should be the default. Also, the current implementation of `video.liquid` only works for some very specific video URLs. For example, to embed youtube, specialized links must be used to avoid `X-Frame-Option` issues. This behavior can lead to a broken embedding, which would not look very nice. Feedback welcome.
Created a plugin to tackle alshedivat#1181. Currently have an issue with tikzjax since it imports some wasm file from its javascript. The rest should work as expected. --------- Signed-off-by: George Araujo <george.gcac@gmail.com>
As discussed in alshedivat#1181, I suggest to make embedding videos an optional feature. This behavior aligns well with recently merged PR alshedivat#2312. Open questions: 1. I added a youtube link to `papers.bib`. Is this link okay? 2. I set `enable_video_embedding: false` as the default. I argue that privacy settings should be the default. Also, the current implementation of `video.liquid` only works for some very specific video URLs. For example, to embed youtube, specialized links must be used to avoid `X-Frame-Option` issues. This behavior can lead to a broken embedding, which would not look very nice. Feedback welcome.
Is your feature request related to a problem? Please describe.
In its default configuration, al-folio uses lots of externally hosted resources.
For example, it uses JavaScript libraries like
bootstrap,imagesloaded,jquery,masonry-layout,mdbootstrap,medium-zoomthat are hosted oncdn.jsdelivr.net.Also, Google Font
fonts.googleapis.comis used per default.They can be found in header.html.
Some features, like
altmetric_badgeanddimensions_badge, can be disabled simply in_config.yml. By their nature, these features cannot work on a privacy oriented website since they must request their respective API services.Describe the solution you'd like
I understand the benefits of using CDNs, but I would love to have the possibility to run the website with respect to privacy.
The best option IMHO would be a configuration option like
privacy_mode. It could also be calledoffline_modesince, since it provides the advantage to run the site while being offline.When activated, this option should automatically download all required libraries (e.g., those that are defined in
_config.yml) and put them in an appropriate assets folder.header.htmlshould then use these files instead.Describe alternatives you've considered
I looked into two different directions:
I looked into existing jekyll modules if anything for this purpose exists. I could not really find anything, but I have to say, that I'm very new to jekyll. So I might overlooked something.
I could simply download the files manually, place them in my
assets/jsfolder and editheader.html. I'm not sure if this is the expected way to do so. Future updates of al-folio are likely to cause trouble.The text was updated successfully, but these errors were encountered: