Pictures in Twitter feeds are not shown in TinyTinyRSS #957

ontheair81 · 2018-12-07T10:46:24Z

I am using RSS-Bridge for generating user-feeds from Twitter. These feeds are imported in TinyTinyRSS.

The problem is, that pictures in these feeds are not displayed at all in TinyTinyRSS. So I contacted the developer of TinyTinyRSS. In short he said that the feed, generated by RSS-Bridge, is bad:

in your feed enclosures both have wrong content type (application/octet-stream) and lead to a retarded URI which fails the check because of the suffix. so, in short, that feed is bad and whoever produced it should feel bad. my advice would be telling people at “rss bridge” to fix it.

The full conversation can be found here. There are also examples and more informations.

I did some workarounds in TwitterBridge.php and I can live with the result for now. But anyway, this is just quick-and-dirty as I am not a PHP-developer.

Maybe you can have a look at the problem? This would be very appreciated! Thank you in advance!

em92 · 2018-12-07T12:03:45Z

I did some workarounds in TwitterBridge.php
this is just quick-and-dirty as I am not a PHP-developer.

Just post these workarounds. If you resolved issue, share it in this thread for us to fix it. Nobody will throw rocks to you.

ontheair81 · 2018-12-07T14:04:34Z

Basically it was all about removing the :thumb or :orig, additionally to *.jpg.

I changed in line 243:
$item['enclosures'] = array($image . ':orig');
to
$item['enclosures'] = array($image);

Changes in line 243:
<a href="{$image}:orig">
to
<a href="{$image}">

Changes in line 249:
src="{$image}:thumb" />
to
src="{$image}" />

The same changes have to be done after line 280 again ("add embedded image to content").

So all the images are named as *.jpg now, not named as *.jpg:orig or *.jpg:thumb.
TinyTinyRSS is working with these changes; pictures in the feeds are not broken anymore (at least when using the Android-App, which is a frontend to the server-based TinyTinyRSS).

As I said: This is very quick-and-dirty, I am not a PHP-developer and for sure this looks really bad and ugly to you. Sorry for this. Anyway, I needed a quick solution and at least I get things working with these changes ;-)

Obviously this can never be the final solution to this problem ;-)

logmanoriginal · 2018-12-07T18:18:18Z

Cross post: #812 (comment)
My answer: #812 (comment)

Basically it was all about removing the :thumb or :orig, additionally to *.jpg.

Those parameters are specified by Twitter in order to retrieve different sizes of the same image. Embedding the originally sized image in the feed looks garbage in some feed readers (i.e. they don't scale well).

The problem is, that pictures in these feeds are not displayed at all in TinyTinyRSS.

I'm not sure why this is a problem on your feeds. It works fine on my machines (using Android and iOS Apps as well as Firefox). That being said, my server runs an older release of tt-rss (commit 0cd4a88dc2e0dc0e9a155ad8e7714f9fc7d4a481)

Edit:

in your feed enclosures both have wrong content type (application/octet-stream) and lead to a retarded URI which fails the check because of the suffix.

-- https://discourse.tt-rss.org/t/pictures-in-twitter-feeds-generated-by-rss-bridge/1722/4

He is right about the content type, yet I'm not sure if he actually looked at the URIs you posted, clearly :thumb and :original has nothing to do with trying to be smart, but rather are legit URIs of Twitter which point to different sizes of the same image. From your example:

https://pbs.twimg.com/media/DteaZZxWwAAo_fh.jpg:orig

https://pbs.twimg.com/media/DteaZZxWwAAo_fh.jpg:thumb

I'll check if updating tt-rss breaks it for me as well.

logmanoriginal · 2018-12-07T18:50:01Z

So, I updated my instance to current master and it looks fine to me. What am I missing?

@ontheair81 Can you share a screen shot of what is wrong with your feed?

ontheair81 · 2018-12-08T06:21:37Z

First of all: Thank you very much for your effort!

I dont think that this problem is related to my browser, as the pictures are not shown in the browser AND the app of TT-RSS.

Maybe there is something wrong with my installation of TT-RSS. But this would be very weird as I am using it for many years, and there are hundreds of feeds without any problem.

Sure, I can do a screenshot. But first I have to revert my workaround so you can see the original behavior. This will take some time as I have to leave for work now...
I try to do that later today or tomorrow and will come back to you!

Thank you for your help!

ontheair81 · 2018-12-08T07:48:21Z

To simplify things, I am not using my "workarounded" version of RSS-Bridge, but https://bridge.suumitsu.eu/ instead.

As an example, I will generate a feed from Twitter from the user "memes". Please note: I am using the TwitterBridge with "by username", not by keyword, hashtag or list.
So I am getting this feed URL: https://bridge.suumitsu.eu/?action=display&bridge=Twitter&u=memes&format=Atom

My browser shows the feed correct; the picture shown is *.jpg:thumb, linking to the full-res image *.jpg:orig:

In TT-RSS the feed is shown like this with an broken *.jpg:thumb:

In the App, no pictures at all are shown in the article list. All other feeds (when not using RSS-Bridge) are showing pictures in the article list:

When clicking to a tweet, then - and only then - the *.jpg:thumb is shown:

Now lets use my "workarounded" version of TwitterBridge.php. The pictures will be shown much bigger due to the changes:

The view in web-based TT-RSS is still the same (broken like above), but now the images are shown correctly in the list view of the app (which is most important to me):

This is exactly like it should be (in my opinion), so I am happy for now with my quick-and-dirty workaround ;-) Obviously its not nice that the images are still broken in the web-based view in TT-RSS.

I really dont have a clue, why this is working fine for you but not for me. Here are the activated (user-)plugins of TT-RSS; maybe there is a difference to your TT-RSS?

logmanoriginal · 2018-12-08T14:26:23Z

Thanks for sharing your data!

I used your example on my instance for comparison:
?action=display&bridge=Twitter&u=memes&format=Atom

Browser view

Subscription

Feed (tt-rss in browser view)

And this is even working with Privacy Badger enabled (this is an add-on to prevent being tracked on the Internet which may cause issues with tt-rss).

Privacy Badger

Of course, if I block contents from pbs.twimg.com (as it would do automatically after some time) the results look different:

Privacy Badger (pbs.twimg.com blocked)

Feed (tt-rss in browser view - with images blocked)

Then again, my mobile phone has no issues whatsoever.

Feed (tt-rss on mobile phone - using Android)

I don't think your issue is related to add-ons like Privacy Badger, as it would still block contents, even if :thumb and :orig were missing. The only clue left is the difference in how your server handles images. Maybe one of your plugins has an effect on the images. Here is my configuration for comparison:

Plugins

I actually checked if enabling af_tumblr_1280 had any effects, but it doesn't. Also the source code looks good to me. tumblr_gdpr is a bit more complex, but as far as I can tell makes no differences here.

So the last thing I've checked is whether media caching makes any difference (generally disabled on my feeds):

Media Caching enabled

But it doesn't

Feed (tt-rss in browser view)

This is really strange...

EDIT

One more clue, do you have tracking protection enabled on your instance of tt-rss by any chance?

Tracking Protection (in Firefox)

https://support.mozilla.org/en-US/kb/tracking-protection#w_turn-off-tracking-protection-for-individual-sites

ontheair81 · 2018-12-10T10:36:47Z

Thank you for the feedback and the detailed informations!

One more clue, do you have tracking protection enabled on your instance of tt-rss by any chance?

Thats it! Disabling tracking protection for tt-rss fixed things for the browser view. Thats great!
So there is just one problem left: The view in tt-rss via the app in Android.

Then again, my mobile phone has no issues whatsoever.

Your picture shows the content when clicking on an article, right? Could you please show a screenshot from the list view?

As you can see here, pictures are shown correct when clicking the article:

But still there are no images shown in the list view of the Twitter-feed (working fine in all other feeds):

How does it looks like in your list?

logmanoriginal · 2018-12-10T20:50:18Z

How do you get previews in the App?
This is my list (different feed, but they all look the same anyway):

ontheair81 · 2018-12-11T05:38:44Z

How do you get previews in the App?

I am using the original App of tt-rss from the same developer (https://play.google.com/store/apps/details?id=org.fox.ttrss). On Android 7.1.2 with latest security patches.

There are picture previews in the list view of every single feed I am using, except in the Twitter-feed generated by RSS-Bridge. For example, here are some feeds with previews in list view:

All feeds from flickr.com
All feeds from tumblr.com
androidauthority.com
tagesschau.de (german news)
heise.de (german IT news)
golem.de (german IT news)
rfa.org (radio free asia)
and many more

This is my list (different feed, but they all look the same anyway)

Your screenshot looks very different. Maybe you are not using the original tt-rss-App by Andrew Dolgov but a clone or third-party-App instead?

logmanoriginal · 2018-12-11T18:45:42Z

Your screenshot looks very different. Maybe you are not using the original tt-rss-App by Andrew Dolgov but a clone or third-party-App instead?

Yes, I'm using https://play.google.com/store/apps/details?id=org.ttrssreader
which is a fork developed on GitHub: https://github.com/nilsbraden/ttrss-reader-fork

There are picture previews in the list view of every single feed I am using, except in the Twitter-feed generated by RSS-Bridge.

I see. Since the Twitter URIs are valid RFC 2396 and RFC 3986 URIs, something must be wrong with either caching or loading image files.

You also said that the images are displayed correctly in the article, just not in the preview. That is a strong indicator for some mistake in the App preview.

So...

Since I was interested in looking at an Android app anyway, here are my findings:

Here is the code they use to figure out media elements in an article: https://git.tt-rss.org/fox/tt-rss-android/src/master/org.fox.ttrss/src/main/java/org/fox/ttrss/types/Article.java#L69-L158

They also consider images with incorrect content type, which is why your article view works.

The image caching also looks good to me. Actually, images are stored on disk in encoded (MD5) form, so colons are no problem, even on Windows machines: https://git.tt-rss.org/fox/tt-rss-android/src/master/org.fox.ttrss/src/main/java/org/fox/ttrss/util/ImageCacheService.java#L91-L109

If I'm correct, the HeadlineFragment is responsible for displaying your missing images, but it looks fine to me (at least the small section of this long file): https://git.tt-rss.org/fox/tt-rss-android/src/master/org.fox.ttrss/src/main/java/org/fox/ttrss/HeadlinesFragment.java#L1133-L1416

The only thing I could come up with, is the possibility that Glide doesn't handle colons correctly (which I highly doubt, as it is a core library for Android Apps - and a quick search was not sucessful): https://git.tt-rss.org/fox/tt-rss-android/src/master/org.fox.ttrss/src/main/java/org/fox/ttrss/HeadlinesFragment.java#L1241-L1242

I suppose articles are working because they are embedded as HTML, which uses a different method for loading images (possibly): https://git.tt-rss.org/fox/tt-rss-android/src/master/org.fox.ttrss/src/main/java/org/fox/ttrss/ArticleFragment.java#L427-L550

Anyway, if the developers have no solution for this and we don't want to embed large images in all Twitter feeds, we could always add an option to disable image scaling in the feed. It is not a fix because it requires manual labor (activate a checkbox) on your end, but it's still better than broken feeds, right?

ontheair81 · 2018-12-12T14:32:13Z

I am impressed by the lot work you are doing by checking this App! Thats highly appreciated.

we could always add an option to disable image scaling in the feed. It is not a fix because it requires manual labor (activate a checkbox) on your end, but it's still better than broken feeds, right?

This sounds like a good solution. It is better than broken feeds and better than my workaround ;-)

Images in Twitter feeds are currently being scaled by adding ':orig' (original image) and ':thumb' (thumbnail) to image URIs in the feed. This can cause issues with feed readers that don't handle colons in URIs correctly. Image scaling can now be disabled by adding '&noimgscaling=on' to the query. This parameter is optional to stay compatible to existing feeds. References #957

logmanoriginal · 2018-12-12T16:01:55Z

Fix added to master (see above commit). Let me know if it works for you.

ontheair81 · 2018-12-12T16:49:03Z

Just tried and: It works perfect! Thank you very much for your effort!

logmanoriginal · 2018-12-26T18:59:01Z

Thanks for the feedback!

Images in Twitter feeds are currently being scaled by adding ':orig' (original image) and ':thumb' (thumbnail) to image URIs in the feed. This can cause issues with feed readers that don't handle colons in URIs correctly. Image scaling can now be disabled by adding '&noimgscaling=on' to the query. This parameter is optional to stay compatible to existing feeds. References RSS-Bridge#957

logmanoriginal closed this as completed Dec 26, 2018

ontheair81 mentioned this issue Feb 13, 2022

Image scaling in Twitter is not working anymore #2449

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Pictures in Twitter feeds are not shown in TinyTinyRSS #957

Pictures in Twitter feeds are not shown in TinyTinyRSS #957

ontheair81 commented Dec 7, 2018 •

edited

Loading

em92 commented Dec 7, 2018

ontheair81 commented Dec 7, 2018 •

edited

Loading

logmanoriginal commented Dec 7, 2018 •

edited

Loading

logmanoriginal commented Dec 7, 2018

ontheair81 commented Dec 8, 2018

ontheair81 commented Dec 8, 2018 •

edited

Loading

logmanoriginal commented Dec 8, 2018 •

edited

Loading

ontheair81 commented Dec 10, 2018

logmanoriginal commented Dec 10, 2018

ontheair81 commented Dec 11, 2018

logmanoriginal commented Dec 11, 2018

ontheair81 commented Dec 12, 2018

logmanoriginal commented Dec 12, 2018

ontheair81 commented Dec 12, 2018

logmanoriginal commented Dec 26, 2018

Pictures in Twitter feeds are not shown in TinyTinyRSS #957

Pictures in Twitter feeds are not shown in TinyTinyRSS #957

Comments

ontheair81 commented Dec 7, 2018 • edited Loading

em92 commented Dec 7, 2018

ontheair81 commented Dec 7, 2018 • edited Loading

logmanoriginal commented Dec 7, 2018 • edited Loading

logmanoriginal commented Dec 7, 2018

ontheair81 commented Dec 8, 2018

ontheair81 commented Dec 8, 2018 • edited Loading

logmanoriginal commented Dec 8, 2018 • edited Loading

ontheair81 commented Dec 10, 2018

logmanoriginal commented Dec 10, 2018

ontheair81 commented Dec 11, 2018

logmanoriginal commented Dec 11, 2018

ontheair81 commented Dec 12, 2018

logmanoriginal commented Dec 12, 2018

ontheair81 commented Dec 12, 2018

logmanoriginal commented Dec 26, 2018

ontheair81 commented Dec 7, 2018 •

edited

Loading

ontheair81 commented Dec 7, 2018 •

edited

Loading

logmanoriginal commented Dec 7, 2018 •

edited

Loading

ontheair81 commented Dec 8, 2018 •

edited

Loading

logmanoriginal commented Dec 8, 2018 •

edited

Loading