Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

URL previewing support #688

Merged
merged 44 commits into from
Apr 11, 2016
Merged

URL previewing support #688

merged 44 commits into from
Apr 11, 2016

Commits on Jan 24, 2016

  1. initial WIP of a tentative preview_url endpoint - incomplete, unteste… 

    …d, experimental, etc. just putting it here for safekeeping for now
    @ara4n
    ara4n committed Jan 24, 2016
    Configuration menu
    Copy the full SHA
    7dd0c17 View commit details
    Browse the repository at this point in the history

Commits on Mar 25, 2016

  1. typo

    @ara4n
    ara4n committed Mar 25, 2016
    Configuration menu
    Copy the full SHA
    adafa24 View commit details
    Browse the repository at this point in the history

Commits on Mar 27, 2016

  1. Merge branch 'develop' into matthew/preview_urls

    @ara4n
    ara4n committed Mar 27, 2016
    Configuration menu
    Copy the full SHA
    d9d48aa View commit details
    Browse the repository at this point in the history

Commits on Mar 29, 2016

  1. typo

    @ara4n
    ara4n committed Mar 29, 2016
    Configuration menu
    Copy the full SHA
    ec0cf99 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'develop' into matthew/preview_urls

    @ara4n
    ara4n committed Mar 29, 2016
    Configuration menu
    Copy the full SHA
    e0c2490 View commit details
    Browse the repository at this point in the history

  3. make it build

    @ara4n
    ara4n committed Mar 29, 2016
    Configuration menu
    Copy the full SHA
    dd4287c View commit details
    Browse the repository at this point in the history

  4. make it work

    @ara4n
    ara4n committed Mar 29, 2016
    Configuration menu
    Copy the full SHA
    64b4aea View commit details
    Browse the repository at this point in the history

  5. debug

    @ara4n
    ara4n committed Mar 29, 2016
    Configuration menu
    Copy the full SHA
    1903858 View commit details
    Browse the repository at this point in the history

  6. implement redirects

    @ara4n
    ara4n committed Mar 29, 2016
    Configuration menu
    Copy the full SHA
    721b2bf View commit details
    Browse the repository at this point in the history

  7. fix bugs

    @ara4n
    ara4n committed Mar 29, 2016
    Configuration menu
    Copy the full SHA
    ae5831d View commit details
    Browse the repository at this point in the history

Commits on Mar 30, 2016

  1. spell out more packages

    @ara4n
    ara4n committed Mar 30, 2016
    Configuration menu
    Copy the full SHA
    7178ab7 View commit details
    Browse the repository at this point in the history

Commits on Mar 31, 2016

  1. handle requests with missing content-length headers (e.g. YouTube)

    @ara4n
    ara4n committed Mar 31, 2016
    Configuration menu
    Copy the full SHA
    a8a5dd3 View commit details
    Browse the repository at this point in the history

  2. sync in changes from matrixfederationclient

    @ara4n
    ara4n committed Mar 31, 2016
    Configuration menu
    Copy the full SHA
    0d3d7de View commit details
    Browse the repository at this point in the history

  3. synthesise basig OG metadata from pages lacking it

    @ara4n
    ara4n committed Mar 31, 2016
    Configuration menu
    Copy the full SHA
    bb9a2ca View commit details
    Browse the repository at this point in the history

  4. prevent choking on invalid utf-8, and handle image thumbnailing smarter

    @ara4n
    ara4n committed Mar 31, 2016
    Configuration menu
    Copy the full SHA
    72550c3 View commit details
    Browse the repository at this point in the history

  5. handle spidered relative images correctly

    @ara4n
    ara4n committed Mar 31, 2016
    Configuration menu
    Copy the full SHA
    683e564 View commit details
    Browse the repository at this point in the history

Commits on Apr 1, 2016

  1. fix assorted redirect, unicode and screenscraping bugs

    @ara4n
    ara4n committed Apr 1, 2016
    Configuration menu
    Copy the full SHA
    c60b751 View commit details
    Browse the repository at this point in the history

  2. refactor calc_og; spider image URLs; fix xpath; add a (broken) expiri… 

    …ngcache; loads of other fixes
    @ara4n
    ara4n committed Apr 1, 2016
    Configuration menu
    Copy the full SHA
    5fd07da View commit details
    Browse the repository at this point in the history

Commits on Apr 2, 2016

  1. make meta comparisons case insensitive

    @ara4n
    ara4n committed Apr 2, 2016
    Configuration menu
    Copy the full SHA
    b26e860 View commit details
    Browse the repository at this point in the history

  2. handle missing dimensions without crashing

    @ara4n
    ara4n committed Apr 2, 2016
    Configuration menu
    Copy the full SHA
    5037ee0 View commit details
    Browse the repository at this point in the history

  3. pass back SVGs as their own thumbnails

    @ara4n
    ara4n committed Apr 2, 2016
    Configuration menu
    Copy the full SHA
    2c838f6 View commit details
    Browse the repository at this point in the history

  4. how was _respond_default_thumbnail ever meant to work?

    @ara4n
    ara4n committed Apr 2, 2016
    Configuration menu
    Copy the full SHA
    9377157 View commit details
    Browse the repository at this point in the history

  5. support gzip compression, and don't pass through error msgs

    @ara4n
    ara4n committed Apr 2, 2016
    Configuration menu
    Copy the full SHA
    d1b154a View commit details
    Browse the repository at this point in the history

  6. add a persistent cache of URL lookups, and fix up the in-memory one t… 

    …o work
    @ara4n
    ara4n committed Apr 2, 2016
    Configuration menu
    Copy the full SHA
    7426c86 View commit details
    Browse the repository at this point in the history

  7. Ensure only one download for a given URL is active at a time

    @ara4n
    ara4n committed Apr 2, 2016
    Configuration menu
    Copy the full SHA
    b09e29a View commit details
    Browse the repository at this point in the history

  8. remove stale todo

    @ara4n
    ara4n committed Apr 2, 2016
    Configuration menu
    Copy the full SHA
    110780b View commit details
    Browse the repository at this point in the history

Commits on Apr 3, 2016

  1. rebase all image URLs

    @ara4n
    ara4n committed Apr 3, 2016
    Configuration menu
    Copy the full SHA
    c391646 View commit details
    Browse the repository at this point in the history

  2. fix etag typing error. fix timestamp typing error

    @ara4n
    ara4n committed Apr 3, 2016
    Configuration menu
    Copy the full SHA
    eab4d46 View commit details
    Browse the repository at this point in the history

  3. pep8

    @ara4n
    ara4n committed Apr 3, 2016
    Configuration menu
    Copy the full SHA
    8b98a7e View commit details
    Browse the repository at this point in the history

  4. char encoding

    @ara4n
    ara4n committed Apr 3, 2016
    Configuration menu
    Copy the full SHA
    0834b15 View commit details
    Browse the repository at this point in the history

  5. report image size (bytewise) in OG meta

    @ara4n
    ara4n committed Apr 3, 2016
    Configuration menu
    Copy the full SHA
    cf51c41 View commit details
    Browse the repository at this point in the history

  6. Merge branch 'develop' into matthew/preview_urls

    @ara4n
    ara4n committed Apr 3, 2016
    Configuration menu
    Copy the full SHA
    9f7dc2b View commit details
    Browse the repository at this point in the history

Commits on Apr 7, 2016

  1. Merge branch 'develop' into matthew/preview_urls

    @ara4n
    ara4n committed Apr 7, 2016
    Configuration menu
    Copy the full SHA
    d6e7333 View commit details
    Browse the repository at this point in the history

Commits on Apr 8, 2016

  1. Add url_preview_enabled config option to turn on/off preview_url endp… 

    …oint. defaults to off.

Add url_preview_ip_range_blacklist to let admins specify internal IP ranges that must not be spidered.
Add url_preview_url_blacklist to let admins specify URL patterns that must not be spidered.
Implement a custom SpiderEndpoint and associated support classes to implement url_preview_ip_range_blacklist
Add commentary and generally address PR feedback
    @ara4n
    ara4n committed Apr 8, 2016
    Configuration menu
    Copy the full SHA
    dafef5a View commit details
    Browse the repository at this point in the history

  2. Add doc

    @ara4n
    ara4n committed Apr 8, 2016
    Configuration menu
    Copy the full SHA
    ec9331f View commit details
    Browse the repository at this point in the history

  3. Add more doc

    @ara4n
    ara4n committed Apr 8, 2016
    Configuration menu
    Copy the full SHA
    b04f812 View commit details
    Browse the repository at this point in the history

  4. fix SQL based on PR feedback

    @ara4n
    ara4n committed Apr 8, 2016
    Configuration menu
    Copy the full SHA
    fb83f6a View commit details
    Browse the repository at this point in the history

  5. more PR feedback

    @ara4n
    ara4n committed Apr 8, 2016
    Configuration menu
    Copy the full SHA
    1ccabe2 View commit details
    Browse the repository at this point in the history

  6. fix error checking for new SQL

    @ara4n
    ara4n committed Apr 8, 2016
    Configuration menu
    Copy the full SHA
    2460d90 View commit details
    Browse the repository at this point in the history

  7. fix typo

    @ara4n
    ara4n committed Apr 8, 2016
    Configuration menu
    Copy the full SHA
    af582b6 View commit details
    Browse the repository at this point in the history

  8. move local_media_repository_url_cache.sql to schema v31

    @ara4n
    ara4n committed Apr 8, 2016
    Configuration menu
    Copy the full SHA
    6ff7a79 View commit details
    Browse the repository at this point in the history

  9. Fix pep8 warning

    Mark Haines committed Apr 8, 2016
    Configuration menu
    Copy the full SHA
    b36270b View commit details
    Browse the repository at this point in the history

  10. actually throw meaningful errors

    @ara4n
    ara4n committed Apr 8, 2016
    Configuration menu
    Copy the full SHA
    83b2f83 View commit details
    Browse the repository at this point in the history

Commits on Apr 11, 2016

  1. fix typos and needless try/except from PR review

    @ara4n
    ara4n committed Apr 11, 2016
    Configuration menu
    Copy the full SHA
    5ffacc5 View commit details
    Browse the repository at this point in the history