Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

3722 embeds vizjson in redis #3733

Merged
merged 13 commits into from
May 22, 2015
70 changes: 52 additions & 18 deletions app/controllers/admin/visualizations_controller.rb
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
require_dependency '../../lib/resque/user_jobs'
require_relative '../carto/admin/user_table_public_map_adapter'
require_relative '../carto/admin/visualization_public_map_adapter'
require_relative '../../helpers/embed_redis_cache'

class Admin::VisualizationsController < ApplicationController

Expand All @@ -22,8 +23,9 @@ class Admin::VisualizationsController < ApplicationController

before_filter :resolve_visualization_and_table, only: [:show, :public_table, :public_map,
:show_organization_public_map, :show_organization_embed_map,
:show_protected_public_map, :show_protected_embed_map,
:embed_map ]
:show_protected_public_map, :show_protected_embed_map]

before_filter :resolve_visualization_and_table_if_not_cached, only: [:embed_map]

skip_before_filter :browser_is_html5_compliant?, only: [:public_map, :embed_map, :track_embed,
:show_protected_embed_map, :show_protected_public_map]
Expand Down Expand Up @@ -362,24 +364,24 @@ def show_protected_embed_map
end

def embed_map
return(embed_forbidden) if @visualization.private?
return(embed_protected) if @visualization.password_protected?
return(show_organization_embed_map) if org_user_has_map_permissions?(current_user, @visualization)

response.headers['X-Cache-Channel'] = "#{@visualization.varnish_key}:vizjson"
response.headers['Surrogate-Key'] = "#{CartoDB::SURROGATE_NAMESPACE_PUBLIC_PAGES} #{@visualization.surrogate_key}"
response.headers['Cache-Control'] = "no-cache,max-age=86400,must-revalidate, public"

# We need to know if visualization logo is visible or not
@hide_logo = is_logo_hidden(@visualization, params)
if request.format == 'text/javascript'
error_message = "/* Javascript embeds are deprecated, please use the html iframe instead */"
return render inline: error_message, status: 400
end

respond_to do |format|
format.html { render layout: 'application_public_visualization_layout' }
format.js { render 'embed_map', content_type: 'application/javascript' }
if @cached_embed
response.headers.merge! @cached_embed[:headers].stringify_keys
respond_to do |format|
format.html { render inline: @cached_embed[:body] }
end
else
resp = embed_map_actual
if response.ok? && (@visualization.public? || @visualization.public_with_link?)
#cache response
embed_redis_cache.set(@visualization.id, response.headers, response.body)
end
resp
end
rescue => e
Rollbar.report_exception(e)
embed_forbidden
end

# Renders input password view
Expand Down Expand Up @@ -447,6 +449,14 @@ def resolve_visualization_and_table
render_pretty_404 if disallowed_type?(@visualization)
end

def resolve_visualization_and_table_if_not_cached
# TODO review the naming confusion about viz and tables, I suspect templates also need review
@cached_embed = embed_redis_cache.get(@table_id)
if !@cached_embed
resolve_visualization_and_table
end
end

# If user A shares to user B a table link (being both from same org), attept to rewrite the url to the correct format
# Messing with sessions is bad so just redirect to newly formed url and let new request handle permissions/access
def get_corrected_url_if_proceeds(for_table=true)
Expand Down Expand Up @@ -582,4 +592,28 @@ def sql_api_url(query, user)
"#{ ApplicationHelper.sql_api_template("public").gsub! '{user}', user.username }#{ Cartodb.config[:sql_api]['public']['endpoint'] }?q=#{ URI::encode query }"
end

def embed_map_actual
return(embed_forbidden) if @visualization.private?
return(embed_protected) if @visualization.password_protected?
return(show_organization_embed_map) if org_user_has_map_permissions?(current_user, @visualization)

response.headers['X-Cache-Channel'] = "#{@visualization.varnish_key}:vizjson"
response.headers['Surrogate-Key'] = "#{CartoDB::SURROGATE_NAMESPACE_PUBLIC_PAGES} #{@visualization.surrogate_key}"
response.headers['Cache-Control'] = "no-cache,max-age=86400,must-revalidate, public"

# We need to know if visualization logo is visible or not
@hide_logo = is_logo_hidden(@visualization, params)

respond_to do |format|
format.html { render layout: 'application_public_visualization_layout' }
end
rescue => e
Rollbar.report_exception(e)
embed_forbidden
end

def embed_redis_cache
@embed_redis_cache ||= EmbedRedisCache.new($tables_metadata)
end

end
55 changes: 55 additions & 0 deletions app/helpers/embed_redis_cache.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# encoding: utf-8

class EmbedRedisCache

# This needs to be changed whenever there're changes in the code that require invalidation of old keys
VERSION = '1'


def initialize(redis_cache = $tables_metadata)
@redis = redis_cache
end

def get(visualization_id)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe I'm over-thinking, but after a second thought, if Redis went down or there were a error or connection issue this would fail and it wouldn't fallback to normal rendering, maybe this class should have error handling: reporting it but not propagating the exception.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • varnish down => ok, serve through redis cache
  • varnish and redis down => we're f**ked
  • redis down => sets and invalidations would fail

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also needed for vizjson cache, maybe I will make a separate branch for that that will conflict a little with the branch refactoring it out of old and new models :S

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if redis go down, don't worry, everything will go down

key = key(visualization_id)
value = redis.get(key)
if value.present?
return JSON.parse(value, symbolize_names: true)
else
return nil
end
rescue Redis::BaseError => exception
Rollbar.report_exception(exception)
nil
end

# Only public and public with link
def set(visualization_id, response_headers, response_body)
serialized = JSON.generate({headers: response_headers,
body: response_body
})
redis.setex(key(visualization_id), 24.hours.to_i, serialized)
rescue Redis::BaseError => exception
Rollbar.report_exception(exception)
nil
end

def invalidate(visualization_id)
redis.del key(visualization_id)
rescue Redis::BaseError => exception
Rollbar.report_exception(exception)
nil
end

def key(visualization_id)
"visualization:#{visualization_id}:embed:#{VERSION}"
end


private

def redis
@redis
end

end
7 changes: 7 additions & 0 deletions app/models/visualization/member.rb
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
require_relative '../table/privacy_manager'
require_relative '../../../services/minimal-validation/validator'
require_relative '../../../services/named-maps-api-wrapper/lib/named_maps_wrapper'
require_relative '../../helpers/embed_redis_cache'

# Every table has always at least one visualization (the "canonical visualization"), of type 'table',
# which shares the same privacy options as the table and gets synced.
Expand Down Expand Up @@ -610,18 +611,24 @@ def qualified_name(viewer_user=nil)
def invalidate_redis_cache
self.class.redis_cache.del(redis_vizjson_key)
self.class.redis_cache.del(redis_vizjson_key(true))
embed_redis_cache.invalidate(self.id)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you checked this is called on permission change? I've taken a quick look through Visualization::Member code regarding caches invalidation but it's messy.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a bug prior to these changes on permission changes, it is quite easy to reproduce #2407 I'll fix it afterwards (it must also affect varnish btw)

end

def self.redis_cache
@@redis_cache ||= $tables_metadata
end



private

attr_reader :repository, :name_checker, :validator
attr_accessor :privacy_changed, :name_changed, :old_name, :permission_change_valid, :dirty

def embed_redis_cache
@embed_redis_cache ||= EmbedRedisCache.new($tables_metadata)
end

def calculate_vizjson(options={})
vizjson_options = {
full: false,
Expand Down
180 changes: 0 additions & 180 deletions app/views/admin/visualizations/embed_map.js.erb

This file was deleted.

Loading