Ruby 1.9 + Sinatra Incompatible Character Encodings: Ascii-8Bit and Utf-8

ruby 1.9 + sinatra incompatible character encodings: ASCII-8BIT and UTF-8

I'm not familiar with the specifics of your situation, but this kind of error has come up in Ruby 1.9 when there's an attempt to concatenate a string in the source code (typically encoded in UTF-8) with a string from outside of the system, e.g., input from an HTML form or data from a database.

ASCII-8BIT is basically a synonym for binary. It suggests that the input string was not tagged with the actual encoding that has been used (for example, UTF-8 or ISO-8859-1).

My understanding is that exception messages are not seen in Ruby 1.8 because it treats strings as binary and silently concatenates strings of different encodings. For subtle reasons, this often isn't a problem.

I ran into a similar error yesterday and found this excellent overview.

• http://yehudakatz.com/2010/05/05/ruby-1-9-encodings-a-primer-and-the-solution-for-rails/

One option to get your error message to go away is to use force_encoding('UTF-8') (or some other encoding) on the string coming from the external source. This is not to be done lightly, and you'll want to have a sense of the implications.

Rails 2.3.5, Ruby 1.9, SQLite 3 incompatible character encodings: UTF-8 and ASCII-8BIT

Re the reason this was happening, here is a link to a similar question that was asked more recently than yours together with a response:

• ruby 1.9 + sinatra incompatible character encodings: ASCII-8BIT and UTF-8

Check out this excellent blog post for an overview of the underlying problem and some possible solutions:

• http://yehudakatz.com/2010/05/05/ruby-1-9-encodings-a-primer-and-the-solution-for-rails/

Incompatible character encoding in simple Sinatra app

This is an issue in Tilt, the templating system that Sinatra uses (and is being considered for Rails). Have a look at issues #75 and #107.

The problem is basically down to how Tilt reads template files from the disk - it uses binread. This means that the source string that is handed to the actual template engine has an associated encoding of ASCII-8BIT, which is basically saying that it’s unknown.

RDiscount has code to set the encoding of the output to match the input, but this isn’t much help when the input encoding is ASCII-8BIT; the result is given the same encoding. The same thing (or something similar) happens with Kramdown, so simply switching won’t solve this.

This causes problems when the template has non-ascii characters (i.e. £) and you try to combine the result with other utf-8 encoded strings. If the template only contains only ascii characters, it is compatible with utf-8 and Ruby can combine the two strings. If not, you get the CompatibilityError that you see.

A possible workaround is to read the template files yourself, and pass in the resulting string with the correct encoding to Tilt:

<%= markdown File.read './views/pound.md' %>
£

By reading the file yourself with read instead of binread, you can ensure it has the right encoding and so is compatible with the rest of the erb file. You may want to read the file in once and cache the contents somewhere if you try this.

An alternative workaround would be to capture the output of the markdown method and use force_encoding on it:

<%= markdown(:pound).force_encoding('utf-8') %>
£

This is possible because although the encoding is ASCII-8BIT, you know that the bytes in the string really are utf-8 encoded, so you can just change the encoding.

Action Controller: incompatible character encodings: ASCII-8BIT and UTF-8?

It appears that ruby thinks that the encoding of the uploaded file is ascii-8bit (that is to say binary).

If you know the encoding of the file, you can use force_encoding! to change the encoding of the string (without transcoding). If you're not always going to be sure of the encoding of the file, the charguess gem can be used to guess it.

gsub codification issues with UTF-8

I finally used transliterate from Rails ActiveSupport:

require 'active_support/all'
v = ActiveSupport::Inflector.transliterate v.downcase
v.gsub(/[^a-z1-9]+/, '-').chomp('-')

Works fine.

incompatible character encodings: UTF-8 and ASCII-8BIT on some redmine pages

Check your linux locales. Run a

dpkg-reconfigure locales

Check your locale

Generating locales (this might take a while)...
fr_FR.UTF-8... done

Delete your redmine cache

mv /path/to/redmine/tmp  /path/to/redmine/tmp_OLD
mkdir /path/to/redmine/tmp
chown youruser /path/to/redmine/tmp

Restart remine (and apache if using it)

---

Related Topics