How to Unescape C-Style Escape Sequences from Ruby

How do I unescape c-style escape sequences from ruby?

Okay, if you don't like eval solution, I've hacked a simple state machine in Ruby to parse simple "\n" and "\t" in strings correctly, including pre-escaping of backslash itself. Here it is:

BACKSLASH = "\\"

def unescape_c_string(s)
    state = 0
    res = ''
    s.each_char { |c|
        case state
        when 0
            case c
            when BACKSLASH then state = 1
            else res << c
            end
        when 1
            case c
            when 'n' then res << "\n"; state = 0
            when 't' then res << "\t"; state = 0
            when BACKSLASH then res << BACKSLASH; state = 0
            else res << BACKSLASH; res << c; state = 0
            end
        end
    }
    return res
end

This one can be easily extended to support more characters, including multi-character entities, like \123. Test unit to prove that it works:

require 'test/unit'

class TestEscapeCString < Test::Unit::TestCase
    def test_1
        assert_equal("abc\nasd", unescape_c_string('abc\nasd'))
    end
    def test_2
        assert_equal("abc\tasd", unescape_c_string('abc\tasd'))
    end
    def test_3
        assert_equal("abc\\asd", unescape_c_string('abc' + BACKSLASH * 2 + 'asd'))
    end
    def test_4
        assert_equal("abc\\nasd", unescape_c_string('abc' + BACKSLASH * 2 + 'nasd'))
    end
    def test_5
        assert_equal("abc\\\nasd", unescape_c_string('abc' + BACKSLASH * 3 + 'nasd'))
    end
    def test_6
        assert_equal("abc\\\\nasd", unescape_c_string('abc' + BACKSLASH * 4 + 'nasd'))
    end
end

Best way to escape and unescape strings in Ruby?

Ruby 2.5 added String#undump as a complement to String#dump:

$ irb
irb(main):001:0> dumped_newline = "\n".dump
=> "\"\\n\""
irb(main):002:0> undumped_newline = dumped_newline.undump
=> "\n"

With it:

def escape(s)
  s.dump[1..-2]
end

def unescape(s)
  "\"#{s}\"".undump
end

$irb
irb(main):001:0> escape("\n \" \\")
=> "\\n \\\" \\\\"
irb(main):002:0> unescape("\\n \\\" \\\\")
=> "\n \" \\"

Is this the best way to unescape unicode escape sequences in Ruby?

Your regex, /\u(....)/, has some problems.

First of all, \u doesn't work the way you think it does, in 1.9 you'll get an error and in 1.8 it will just match a single u rather than the \u pair that you're looking for; you should use /\\u/ to find the literal \u that you want.

Secondly, your (....) group is much too permissive, that will allow any four characters through and that's not what you want. In 1.9, you want (\h{4}) (four hexadecimal digits) but in 1.8 you'd need ([\da-fA-F]{4}) as \h is a new thing.

So if you want your regex to work in both 1.8 and 1.9, you should use /\\u([\da-fA-F]{4})/. This gives you the following in 1.8 and 1.9:

>> s = 'Where is \u03bc pancakes \u03BD house? And u1123!'
=> "Where is \\u03bc pancakes \\u03BD house? And u1123!"
>> s.gsub(/\\u([\da-fA-F]{4})/) {|m| [$1].pack("H*").unpack("n*").pack("U*")}
=> "Where is μ pancakes ν house? And u1123!"

Using pack and unpack to mangle the hex number into a Unicode character is probably good enough but there may be better ways.

Ruby -- looking for some sort of Regexp unescape method

If your strings give you the correct output when you print them then they are already escaped correctly. The extra backslashes you see are probably because you are displaying them in the interactive interpreter which adds extra backslashes for you when you display variables to make them less ambiguous.

> x
=> "\\"
> puts x
\
=> nil
> x.length
=> 1

Note that even though it looks like x contains two backslashes, the length of the string is one. The extra backslash is added by the interpreter and is not really part of the string.

If you still think there's a problem, please be more specific about how you are displaying the strings that you mentioned in your question.

---

Edit: In your example the only thing that need unescaping are octal escape codes. You could try this:

x = x.gsub(/\\[0-2][0-7]{2}/){ |c| c[1,3].to_i(8).chr }

How to un-escape a backslash-escaped string?

>>> print '"Hello,\\nworld!"'.decode('string_escape')
"Hello,
world!"

Ruby (Rails) unescape a string -- undo Array.to_s

I've edited my answer due to your edited question:

I still can't duplicate your results!

>> x = ['a']
=> ["a"]
>> x.to_s
=> "a"

But when I change the last call to this:

>> x.inspect
=> "[\"a\"]"

So I'll assume that's what you're doing?

it's not necessarily escaping the values - per se. It's storing the string like this:

%{["a"]}

or rather:

'["a"]'

In any case. This should work to un-stringify it:

>> x = ['a']
=> ["a"]
>> y = x.inspect
=> "[\"a\"]"
>> z = Array.class_eval(y)
=> ["a"]
>> x == z
=> true

I'm skeptical about the safe-ness of using class_eval though, be wary of user inputs because it may produce un-intended side effects (and by that I mean code injection attacks) unless you're very sure you know where the original data came from, or what was allowed through to it.

Removing backslash (escape character) from a string

When you write:

input = "{ \"foo\": \"bar\", \"num\": 3}"

The actual string stored in input is:

{ "foo": "bar", "num": 3}

The escape \" here is interpreted by Ruby parser, so that it can distinguish between the boundary of a string (the left most and the right most "), and a normal character " in a string (the escaped ones).

String#delete deletes a character set specified the first parameter, rather than a pattern. All characters that is in the first parameter will be removed. So by writing

input.delete('\\"')

You got a string with all \ and " removed from input, rather than a string with all \" sequence removed from input. This is wrong for your case. It may cause unexpected behavior some time later.

String#gsub, however, substitute a pattern (either regular expression or plain string).

input.gsub('\\"', '')

means find all \" (two characters in a sequence) and replace them with empty string. Since there isn't \ in input, nothing got replaced. What you need is actually:

input.gsub('"', '')

How do I escape a Unicode string with Ruby?

In Ruby 1.8.x, String#inspect may be what you are looking for, e.g.

>> multi_byte_str = "hello\330\271!"
=> "hello\330\271!"

>> multi_byte_str.inspect
=> "\"hello\\330\\271!\""

>> puts multi_byte_str.inspect
"hello\330\271!"
=> nil

In Ruby 1.9 if you want multi-byte characters to have their component bytes escaped, you might want to say something like:

>> multi_byte_str.bytes.to_a.map(&:chr).join.inspect
=> "\"hello\\xD8\\xB9!\""

In both Ruby 1.8 and 1.9 if you are instead interested in the (escaped) unicode code points, you could do this (though it escapes printable stuff too):

>> multi_byte_str.unpack('U*').map{ |i| "\\u" + i.to_s(16).rjust(4, '0') }.join
=> "\\u0068\\u0065\\u006c\\u006c\\u006f\\u0639\\u0021"

---

Related Topics