Fix ActionView::Template::Error (invalid byte sequence in UTF-8) Exception

[ericdagenais]

My nxlog instance running on Windows 7 is sending event viewer entries to Graylog2 via syslog. Certain messages contain invalid UTF-8 characters which get stored by Graylog2 and cause the web interface to throw a 500 error each time the message is viewed or appears in search results. The exception from production.log is as follows:

MONGODB graylog2['$cmd'].find({"count"=>"streams", "query"=>{:_id=>{"$in"=>[]}}, "fields"=>nil})
  Rendered messages/_sidebar_index.html.erb (17.6ms)
  Rendered messages/_result_graph.html.erb (0.6ms)
MONGODB graylog2['settings'].find({:user_id=>BSON::ObjectId('5116403e6f9efb65bf000002'), :setting_type=>7}).sort([[:_id, :asc]])
MONGODB graylog2['settings'].find({:user_id=>BSON::ObjectId('5116403e6f9efb65bf000002'), :setting_type=>1}).sort([[:_id, :asc]])
MONGODB graylog2['filtered_terms'].find({})
  Rendered messages/_table.html.erb (19.1ms)
  Rendered messages/universalsearch.html.erb within layouts/application (62.3ms)
Completed 500 Internal Server Error in 240ms

ActionView::Template::Error (invalid byte sequence in UTF-8):
    55:         <% end %>
    56:         <td colspan="2">
    57:           <% short_message = message.message rescue "Invalid UTF-8" %>
    58:           <%= short_message %>
    59:           <%=raw (message.message.length > message_length) && !defined?(dont_show_links) ? "<span class='light'>...</span>" : nil %>
    60:         </td>
    61:       </tr>
  app/views/messages/_table.html.erb:58:in `block in _app_views_messages__table_html_erb___2392979296799035691_43196660'
  app/views/messages/_table.html.erb:46:in `each'
  app/views/messages/_table.html.erb:46:in `_app_views_messages__table_html_erb___2392979296799035691_43196660'
  app/views/messages/universalsearch.html.erb:39:in `_app_views_messages_universalsearch_html_erb__3237003117546474902_23791820'

I've resolved the issue by replacing invalid characters with a question mark.

[lennartkoopmann]

Hey Eric,

thank you very much! I think this should be fixed in the server code: No invalid UTF-8 should be written at all.

Graylog2/graylog2-server#116

Cheers,
Lennart