How to Write Unicode Characters to the Console

How to write Unicode characters to the console?

It's likely that your output encoding is set to ASCII. Try using this before sending output:

Console.OutputEncoding = System.Text.Encoding.UTF8;

(MSDN link to supporting documentation.)

And here's a little console test app you may find handy:

C#

using System;
using System.Text;

public static class ConsoleOutputTest {
public static void Main() {
Console.OutputEncoding = System.Text.Encoding.UTF8;
for (var i = 0; i <= 1000; i++) {
Console.Write(Strings.ChrW(i));
if (i % 50 == 0) { // break every 50 chars
Console.WriteLine();
}
}
Console.ReadKey();
}
}

VB.NET

imports Microsoft.VisualBasic
imports System

public module ConsoleOutputTest
Sub Main()
Console.OutputEncoding = System.Text.Encoding.UTF8
dim i as integer
for i = 0 to 1000
Console.Write(ChrW(i))
if i mod 50 = 0 'break every 50 chars
Console.WriteLine()
end if
next
Console.ReadKey()
End Sub
end module

It's also possible that your choice of Console font does not support that particular character. Click on the Windows Tool-bar Menu (icon like C:.) and select Properties -> Font. Try some other fonts to see if they display your character properly:

picture of console font edit

Set C# console application to Unicode output

It turns out that there are multiple things you need to set up in order to make the console display Unicode characters.

  1. Set the console to a Unicode supported font. To do this, run your C# console application once with Console.ReadKey(); so the window stays open. Right click on the title bar of the window and select Properties. These options will persist when debugging through Visual Studio. You might need to use the Default menu instead for persisting the options throughout the system. In the Fonts tab, you need to set the font to Lucida Console. This font supports Unicode characters. The related post can be found here.
  2. Set the console's code page to UTF-8. This one is a bit tricky. Because, you have to execute a command in the console window to change the code page. For whatever reason, this option is not available as a console preference. To do this, you'll need to make a separate cmd.exe process, and use this instead of the normal console provided.

    var cmd = new Process
    {
    StartInfo =
    {
    FileName = "cmd.exe",
    RedirectStandardInput = true,
    RedirectStandardOutput = true,
    CreateNoWindow = true,
    UseShellExecute = false
    }
    };
    cmd.Start();

    cmd.StandardInput.WriteLine("chcp 65001");
    cmd.StandardInput.Flush();
    cmd.StandardInput.Close();

    The first part of the code above will create a new cmd.exe process. The settings given to StartInfo will make sure that Console is redirected to this new process. The second part of the code sends a command to this console window and runs it. That command, chcp 65001, sets the console's code page to UTF-8. Related posts can be found here and here.

  3. Set the OutputEncoding to UTF-8. This is the only way that Console.WriteLine will actually output Unicode characters. Setting this is very simple.

    Console.OutputEncoding = Encoding.UTF8;

    Now, any output from Console will be in Unicode. The related post can be found here.

So, that's it! I hope this information helps someone. :-)

How to log unicode characters to the console in JavaScript?

I found out that if you set the proper charset in a <meta> tag in the <head> it will work:

<meta charset="UTF-8">

c# how to output Unicode characters?

The console is probably not using a Unicode or Japanese encoding, and/or the font used does not contain the required character glyphs.

Have a look at Console.OutputEncoding for more information, and the Unicode Support for the Console section.

How to use unicode characters in Windows command line?

My background: I use Unicode input/output in a console for years (and do it a lot daily. Moreover, I develop support tools for exactly this task). There are very few problems, as far as you understand the following facts/limitations:

  • CMD and “console” are unrelated factors. CMD.exe is a just one of programs which are ready to “work inside” a console (“console applications”).
  • AFAIK, CMD has perfect support for Unicode; you can enter/output all Unicode chars when any codepage is active.
  • Windows’ console has A LOT of support for Unicode — but it is not perfect (just “good enough”; see below).
  • chcp 65001 is very dangerous. Unless a program was specially designed to work around defects in the Windows’ API (or uses a C runtime library which has these workarounds), it would not work reliably. Win8 fixes ½ of these problems with cp65001, but the rest is still applicable to Win10.
  • I work in cp1252. As I already said: To input/output Unicode in a console, one does not need to set the codepage.

The details

  • To read/write Unicode to a console, an application (or its C runtime library) should be smart enough to use not File-I/O API, but Console-I/O API. (For an example, see how Python does it.)
  • Likewise, to read Unicode command-line arguments, an application (or its C runtime library) should be smart enough to use the corresponding API.
  • Console font rendering supports only Unicode characters in BMP (in other words: below U+10000). Only simple text rendering is supported (so European — and some East Asian — languages should work fine — as far as one uses precomposed forms). [There is a minor fine print here for East Asian and for characters U+0000, U+0001, U+30FB.]

Practical considerations

  • The defaults on Window are not very helpful. For best experience, one should tune up 3 pieces of configuration:

    • For output: a comprehensive console font. For best results, I recommend my builds. (The installation instructions are present there — and also listed in other answers on this page.)
    • For input: a capable keyboard layout. For best results, I recommend my layouts.
    • For input: allow HEX input of Unicode.
  • One more gotcha with “Pasting” into a console application (very technical):

    • HEX input delivers a character on KeyUp of Alt; all the other ways to deliver a character happen on KeyDown; so many applications are not ready to see a character on KeyUp. (Only applicable to applications using Console-I/O API.)
    • Conclusion: many application would not react on HEX input events.
    • Moreover, what happens with a “Pasted” character depends on the current keyboard layout: if the character can be typed without using prefix keys (but with arbitrary complicated combination of modifiers, as in Ctrl-Alt-AltGr-Kana-Shift-Gray*) then it is delivered on an emulated keypress. This is what any application expects — so pasting anything which contains only such characters is fine.
    • However, the “other” characters are delivered by emulating HEX input.

    Conclusion: unless your keyboard layout supports input of A LOT of characters without prefix keys, some buggy applications may skip characters when you Paste via Console’s UI: Alt-Space E P. (This is why I recommend using my keyboard layouts!)

One should also keep in mind that the “alternative, ‘more capable’ consoles” for Windows are not consoles at all. They do not support Console-I/O APIs, so the programs which rely on these APIs to work would not function. (The programs which use only “File-I/O APIs to the console filehandles” would work fine, though.)

One example of such non-console is a part of MicroSoft’s Powershell. I do not use it; to experiment, press and release WinKey, then type powershell.


(On the other hand, there are programs such as ConEmu or ANSICON which try to do more: they “attempt” to intercept Console-I/O APIs to make “true console applications” work too. This definitely works for toy example programs; in real life, this may or may not solve your particular problems. Experiment.)

Summary

  • set font, keyboard layout (and optionally, allow HEX input).

  • use only programs which go through Console-I/O APIs, and accept Unicode command-line arguments. For example, any cygwin-compiled program should be fine. As I already said, CMD is fine too.

UPD: Initially, for a bug in cp65001, I was mixing up Kernel and CRTL layers (UPD²: and Windows user-mode API!). Also: Win8 fixes one half of this bug; I clarified the section about “better console” application, and added a reference to how Python does it.

How do I print Unicode to the output console in C with Visual Studio?

This is code that works for me (VS2017) - project with Unicode enabled

#include <stdio.h>
#include <io.h>
#include <fcntl.h>

int main()
{
_setmode(_fileno(stdout), _O_U16TEXT);
wchar_t * test = L"the 来. Testing unicode -- English -- Ελληνικά -- Español." ;

wprintf(L"%s\n", test);
}

This is console

output

After copying it to the Notepad++ I see the proper string

the 来. Testing unicode -- English -- Ελληνικά -- Español.

OS - Windows 7 English, Console font - Lucida Console

Edits based on comments

I tried to fix the above code to work with VS2019 on Windows 10 and best I could come up with is this

#include <stdio.h>
int main()
{
const auto* test = L"the 来. Testing unicode -- English -- Ελληνικά -- Español.";

wprintf(L"%s\n", test);
}

When run it "as is" I see
Default console settings

When it is run with console set to Lucida Console fond and UTF-8 encoding I see
Console switched to UTF-8

As the answer to 来 character shown as empty rectangle - I suppose is the limitation of the font which does not contain all the Unicode gliphs

When text is copied from the last console to Notepad++ all characters are shown correctly

Proper way to print unicode characters to the console in Python when using inline scripts

The problem isn't printing to the console, the problem is interpreting the -c argument from the command line:

$ python -c "print repr('é')"
'\xc3\xa9' # OK, expected byte string
$ python -c "print repr('é'.decode('utf-8'))"
u'\xe9' # OK, byte string decoded explicitly
$ python -c "print repr(u'é')"
u'\xc3\xa9' # bad, decoded implicitly as iso-8859-1

Seems the problem is Python doesn't know what encoding command line arguments are using, so you get the same kind of problem as if a source code file had the wrong encoding. In that case you would tell Python what encoding the source used with a coding comment, and you can do that here too:

$ python -c "# coding=utf-8
print repr(u'é')"
u'\xe9'

Generally I'd try to avoid Unicode on the command line though, especially if you might ever have to run on Windows where the story is much worse.

Trouble getting Unicode character to display when printing to console (using a double buffer)

Here are links that helped me solve this issue!

  • https://social.msdn.microsoft.com/Forums/vstudio/en-US/9e979f3a-9bb2-461a-b654-7d88432012ee/unicode-characters-in-console-via-windows-api
  • Pinvoke. Speed up Console.Write();

In addition, the Microsoft Dev Center has some great resources for questions like this as well.

OK! I found the solution! It is actually quite a simple one. First off, I need to get rid of "CharUnion" and just put that in the "CharInfo" (not sure why, but it works - If you know how to fix CharUnion, please post!). In addition, I need to declare CharSet = CharSet.Auto, else it defaults to ascii. (Shout out to whoever posted their answer but deleted it - you had the right idea!). -- new struct CharInfo (replaces struct CharUnion):

[StructLayout(LayoutKind.Explicit, CharSet = CharSet.Auto)]
public struct CharInfo
{
[FieldOffset(0)]
public char UnicodeChar;
[FieldOffset(0)]
public byte bAsciiChar;
[FieldOffset(2)]
public short Attributes;
}

Next, not that I think it changes anything, but instead of creating a new file, I now get a handle on the current output buffer. This removes the need for SafeFileHandle CreateFile.

public const Int32 STD_OUTPUT_HANDLE = -11;

[DllImportAttribute("kernel32.dll", SetLastError = true, CharSet = CharSet.Auto)]
public static extern ConsoleHandle GetStdHandle(Int32 nStdHandle);

For ease of use, for any future readers, this is the code I am currently using. Note, there may be some random snippets that have no use to the buffer working, I took this directly out of the project I am working in. You should get the point though.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using System.Runtime.InteropServices;
using Microsoft.Win32.SafeHandles;
using System.IO;
using System.Text;

namespace DoubleBuffer
{
/*
* Copyright [2012] [Jeff R Baker]

* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
* v 1.2.0
*/
///<summary>
///This class allows for a double buffer in Visual C# cmd promt.
///The buffer is persistent between frames.
///</summary>
class buffer
{
private int width;
private int height;
private int windowWidth;
private int windowHeight;
private ConsoleHandle h;
private CharInfo[] buf;
private SmallRect rect;

public const Int32 STD_OUTPUT_HANDLE = -11;

[DllImportAttribute("kernel32.dll", SetLastError = true, CharSet = CharSet.Auto)]
public static extern ConsoleHandle GetStdHandle(Int32 nStdHandle);

[DllImport("kernel32.dll", SetLastError = true, CharSet = CharSet.Auto)]
static extern bool WriteConsoleOutput(
ConsoleHandle hConsoleOutput,
CharInfo[] lpBuffer,
Coord dwBufferSize,
Coord dwBufferCoord,
ref SmallRect lpWriteRegion);

[StructLayout(LayoutKind.Sequential)]
public struct Coord
{
private short X;
private short Y;

public Coord(short X, short Y)
{
this.X = X;
this.Y = Y;
}
};



[StructLayout(LayoutKind.Explicit, CharSet = CharSet.Auto)]
public struct CharInfo
{
[FieldOffset(0)]
public char UnicodeChar;
[FieldOffset(0)]
public byte bAsciiChar;
[FieldOffset(2)]
public short Attributes;
}


[StructLayout(LayoutKind.Sequential)]
public struct SmallRect
{
private short Left;
private short Top;
private short Right;
private short Bottom;
public void setDrawCord(short l, short t)
{
Left = l;
Top = t;
}
public short DrawCordX()
{
return Left;
}
public short DrawCordY()
{
return Top;
}
public void setWindowSize(short r, short b)
{
Right = r;
Bottom = b;
}
}

/// <summary>
/// Consctructor class for the buffer. Pass in the width and height you want the buffer to be.
/// </summary>
/// <param name="Width"></param>
/// <param name="Height"></param>
public buffer(int Width, int Height, int wWidth, int wHeight) // Create and fill in a multideminsional list with blank spaces.
{
if (Width > wWidth || Height > wHeight)
{
throw new System.ArgumentException("The buffer width and height can not be greater than the window width and height.");
}
h = GetStdHandle(STD_OUTPUT_HANDLE);
width = Width;
height = Height;
windowWidth = wWidth;
windowHeight = wHeight;
buf = new CharInfo[width * height];
rect = new SmallRect();
rect.setDrawCord(0, 0);
rect.setWindowSize((short)windowWidth, (short)windowHeight);


Console.OutputEncoding = System.Text.Encoding.Unicode;
Clear();

}

/// <summary>
/// This method draws any text to the buffer with given color.
/// To chance the color, pass in a value above 0. (0 being black text, 15 being white text).
/// Put in the starting width and height you want the input string to be.
/// </summary>
/// <param name="str"></param>
/// <param name="x"></param>
/// <param name="y"></param>
/// <param name="attribute"></param>
public void Draw(String str, int x, int y, short attribute) //Draws the image to the buffer
{

if (x > windowWidth - 1 || y > windowHeight - 1)
{
throw new System.ArgumentOutOfRangeException();
}
if (str != null)
{
Char[] temp = str.ToCharArray();

int tc = 0;
foreach (Char le in temp)
{
buf[(x + tc) + (y * width)].UnicodeChar = le; //Height * width is to get to the correct spot (since this array is not two dimensions).

if (attribute != 0)
buf[(x + tc) + (y * width)].Attributes = attribute;
tc++;


}
}


}
/// <summary>
/// Prints the buffer to the screen.
/// </summary>
public void Print() //Paint the image
{
if (!h.IsInvalid)
{

bool b = WriteConsoleOutput(h, buf, new Coord((short)width, (short)height), new Coord((short)0, (short)0), ref rect);
}
}
/// <summary>
/// Clears the buffer and resets all character values back to 32, and attribute values to 1.
/// </summary>
public void Clear()
{
for (int i = 0; i < buf.Length; i++)
{
buf[i].Attributes = 1;
buf[i].UnicodeChar = '\u0020';
}
}
/// <summary>
/// Pass in a buffer to change the current buffer.
/// </summary>
/// <param name="b"></param>
public void setBuf(CharInfo[] b)
{
if (b == null)
{
throw new System.ArgumentNullException();
}

buf = b;
}

/// <summary>
/// Set the x and y cordnants where you wish to draw your buffered image.
/// </summary>
/// <param name="x"></param>
/// <param name="y"></param>
public void setDrawCord(short x, short y)
{
rect.setDrawCord(x, y);
}

/// <summary>
/// Clear the designated row and make all attribues = 1.
/// </summary>
/// <param name="row"></param>
public void clearRow(int row)
{
for (int i = (row * width); i < ((row * width + width)); i++)
{
if (row > windowHeight - 1)
{
throw new System.ArgumentOutOfRangeException();
}
buf[i].Attributes = 0;
buf[i].UnicodeChar = '\u0020';
}
}

/// <summary>
/// Clear the designated column and make all attribues = 1.
/// </summary>
/// <param name="col"></param>
public void clearColumn(int col)
{
if (col > windowWidth - 1)
{
throw new System.ArgumentOutOfRangeException();
}
for (int i = col; i < windowHeight * windowWidth; i += windowWidth)
{
buf[i].Attributes = 0;
buf[i].UnicodeChar = '\u0020';
}
}

/// <summary>
/// This function return the character and attribute at given location.
/// </summary>
/// <param name="x"></param>
/// <param name="y"></param>
/// <returns>
/// byte character
/// byte attribute
/// </returns>
public KeyValuePair<byte, byte> getCharAt(int x, int y)
{
if (x > windowWidth || y > windowHeight)
{
throw new System.ArgumentOutOfRangeException();
}
return new KeyValuePair<byte, byte>((byte)buf[((y * width + x))].UnicodeChar, (byte)buf[((y * width + x))].Attributes);
}

public class ConsoleHandle : SafeHandleMinusOneIsInvalid
{
public ConsoleHandle() : base(false) { }

protected override bool ReleaseHandle()
{
return true; //releasing console handle is not our business
}
}

public int X
{
get { return width; }
}

public int Y
{
get { return height; }
}

public int dX
{
get { return rect.DrawCordX(); }
}

public int dY
{
get { return rect.DrawCordY(); }
}
}
}

Any changes to this may be updated on the "Console Double Buffer C#" on MSDN.

Can I use Unicode in a C# Console Application?

yes you need to convert it:
yes you need to convert it:

you need to make a "\" that will show the program, that you are not just simply type any characters.

you can find the Unicode on many websites for example here

than you will get your Smiley, this dose not work with every Unicode Character!
Converted Picture

kindly regards Juan ☺



Related Topics



Leave a reply



Submit