Unicodedecodeerror: 'Ascii' Codec Can't Decode Byte 0Xef in Position 1

UnicodeDecodeError: 'ascii' codec can't decode byte 0xef in position 1

This is to do with the encoding of your terminal not being set to UTF-8. Here is my terminal

$ echo $LANG
en_GB.UTF-8
$ python
Python 2.7.3 (default, Apr 20 2012, 22:39:59) 
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> s = '(\xef\xbd\xa1\xef\xbd\xa5\xcf\x89\xef\xbd\xa5\xef\xbd\xa1)\xef\xbe\x89'
>>> s1 = s.decode('utf-8')
>>> print s1
(｡･ω･｡)ﾉ
>>> 

On my terminal the example works with the above, but if I get rid of the LANG setting then it won't work

$ unset LANG
$ python
Python 2.7.3 (default, Apr 20 2012, 22:39:59) 
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> s = '(\xef\xbd\xa1\xef\xbd\xa5\xcf\x89\xef\xbd\xa5\xef\xbd\xa1)\xef\xbe\x89'
>>> s1 = s.decode('utf-8')
>>> print s1
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeEncodeError: 'ascii' codec can't encode characters in position 1-5: ordinal not in range(128)
>>> 

Consult the docs for your linux variant to discover how to make this change permanent.

Python: UnicodeDecodeError: 'ascii' codec can't decode byte 0xef in position 0: ordinal not in range(128)

codecs module is just what you need. detail here

import codecs
def replace_line(file_name, line_num, text):
    f = codecs.open(file_name, 'r', encoding='utf-8')
    lines = f.readlines()
    lines[line_num] = text
    f.close()
    w = codecs.open(file_name, 'w', encoding='utf-8')
    w.writelines(lines)
    w.close()

UnicodeDecodeError: 'ascii' codec can't decode byte 0xef in position 1: ordinal not in range(128)

You can use b'\xef or some string here'.decode("utf-8", "ignore") to simply ignore such an error. Another way of doing it is to use a try-catch block.

And either way, you'd probably need to examine Python Docs on Unicode.

Rodeo UnicodeDecodeError: 'ascii' codec can't decode byte 0xef in position 0: ordinal not in range(128)

Try specifying the encoding in the open function call.

fp = open('Proteomics_Data.csv', encoding='utf-8')

UnicodeDecodeError: 'ascii' codec can't decode byte 0xef in position 118374: ordinal not in range(128)

If you're using Python 3, try replacing

open('*******/electronics/positive.review')

with

open('*******/electronics/positive.review', encoding='utf-8')

MySQL Error: Unhandled exception: 'ascii' codec can't decode byte 0xef in position 0: ordinal not in range(128)

The 0xEF value is the first byte of the byte order mark in the UTF-8 encoded CSV file you exported.

Specify UTF-8 text encoding instead of ASCII encoding when you import the file.

If you cannot specify which text encoding to use, convert the file to ASCII before importing. You can do that with most modern text editors, e.g., Notepad++. See its documentation regarding New document and Encoding menu.

---

Related Topics