Unicode HOWTO: UTF-8 locales, file names and text on Linux
Every current Linux distribution uses UTF-8 by default, so most of the work is checking that your locale says UTF-8, then fixing the odd old file or file name that does not. This page shows how to check and set locales, convert text and file names from older encodings, and type characters that are not on your keyboard.
When the classic Unicode HOWTO was written in 2000, switching a Linux system to UTF-8 meant patching the C library and hunting for fonts. Today the hard parts are done. What is left is mostly old data and odd settings.
Unicode and UTF-8 in two minutes
Unicode gives every character a number, called a code point, written like U+00E9 (é) or U+20AC (€). Code points run from U+0000 to U+10FFFF.
UTF-8 is the way those numbers are stored as bytes. It uses one byte for plain ASCII (A–Z, digits, punctuation), so old ASCII files are already valid UTF-8. Other characters take two to four bytes: é is two bytes (c3 a9), € is three, and most emoji are four. Other encodings exist (UTF-16 on Windows internally, and old 8-bit sets like ISO-8859-1 and Windows-1252), but on Linux files, terminals and file names use UTF-8.
- 18.0latest Unicode version as of November 2026 (released September 2026)
- 172,808characters encoded in Unicode 18.0
- 1–4bytes per character in UTF-8
printf 'é€\n' | od -An -tx1 # shows: c3 a9 e2 82 ac 0a
Check your locale
A locale tells programs your language, country and character encoding. It is set with environment variables:
locale # what the current shell uses
localectl status # the system-wide default
locale -a # locales installed on this machine
Good output looks like LANG=en_US.UTF-8, with every LC_ line also ending in UTF-8. The rules for which variable wins are simple:
LC_ALL, if set, overrides everything. Use it only for one command, never in your profile.- Each
LC_category (LC_TIME,LC_COLLATE,LC_CTYPEand so on) overridesLANGfor that one area. LANGis the default for everything else.
So LANG=en_US.UTF-8 with LC_TIME=en_GB.UTF-8 gives English messages with a day-month-year date format. LC_CTYPE is the category that controls the character encoding.
If locale prints "Cannot set LC_CTYPE to default locale", the locale named in your settings is not installed. This often happens over SSH, because your client sends its own LANG to the server. Install that locale on the server, or change the client setting.
Install and set a locale
Debian and Ubuntu
Locales are compiled from a list in /etc/locale.gen. The easiest way is the menu:
sudo dpkg-reconfigure locales
Or uncomment a line such as de_DE.UTF-8 UTF-8 in /etc/locale.gen and run sudo locale-gen.
Fedora, RHEL, Alma, Rocky
Locales come in language packs:
dnf list 'glibc-langpack*'
sudo dnf install glibc-langpack-de
Minimal installs may only have C.UTF-8 until you add a langpack.
Arch Linux
As on Debian: uncomment the locale in /etc/locale.gen and run sudo locale-gen.
Set the system default
sudo localectl set-locale LANG=en_US.UTF-8
sudo localectl set-locale LANG=en_US.UTF-8 LC_TIME=en_GB.UTF-8 # with an override
localectl writes /etc/locale.conf. On Debian 13, /etc/default/locale is a symbolic link to that file, so tools that use either name see the same setting. Older Debian and Ubuntu releases kept it in /etc/default/locale alone. Log out and back in for it to take effect. Desktop users can also choose a language per user in GNOME or KDE settings.
C.UTF-8 is a neutral locale built into current distributions: English messages, plain sorting, but full UTF-8. It is a good choice for servers and containers.
Terminals and the console
Modern terminal emulators (GNOME Terminal/Console, Konsole, foot, kitty, Alacritty, xterm) work in UTF-8 when your locale is UTF-8. If you see question marks or boxes, the character is usually fine but the font lacks it. See Font HOWTO.
The text console (the screen before a desktop starts) can only show a few hundred glyphs at a time. It handles Latin, Greek and Cyrillic, but not Chinese, Japanese or emoji. Choose a console font with FONT= in /etc/vconsole.conf, or try one with setfont.
In tmux and screen, start them from a shell that already has a UTF-8 locale. Over SSH, the remote side needs a UTF-8 locale installed too.
File names
To the Linux kernel, a file name is just a sequence of bytes. It does not check the encoding. Files copied from very old systems, ZIP files made on old Windows versions, or old backups can have names in ISO-8859-1 or a Windows code page. They show up as Gr?n.txt or with odd symbols.
convmv renames them. It only shows what it would do until you add --notest:
sudo apt install convmv # or: sudo dnf install convmv
convmv -f iso-8859-1 -t utf-8 -r ~/old-files # dry run
convmv -f iso-8859-1 -t utf-8 -r --notest ~/old-files # really rename
A second trap is normalization. "é" can be stored as one code point (U+00E9, form NFC) or as "e" plus a combining accent (U+0065 U+0301, form NFD). They look the same but are different bytes, so you can end up with two files that seem to have the same name. Linux programs normally use NFC, while macOS has used NFD. convmv -f utf-8 -t utf-8 --nfc -r --notest DIR converts names to NFC.
Convert text files
First find out what a file is:
file --mime-encoding notes.txt # e.g. notes.txt: iso-8859-1
uchardet notes.txt # a second opinion (package uchardet)
Detection is a guess, especially for short files. Check the result by looking at the converted text. Then convert with iconv:
iconv -f ISO-8859-15 -t UTF-8 notes.txt > notes-utf8.txt
iconv -f WINDOWS-1252 -t UTF-8 old.csv > new.csv
iconv -f UTF-16 -t UTF-8 export.txt > export-utf8.txt
iconv -l # list known encodings
Going the other way, to a smaller character set, some characters may not fit. Add //TRANSLIT to replace them with close matches (€ becomes EUR, ß becomes ss). Characters with no match become ?:
echo 'Straße 5 €' | iconv -f UTF-8 -t ASCII//TRANSLIT # Strasse 5 EUR
Mojibake: "café" instead of "café"
This garbage appears when UTF-8 bytes are read as Latin-1 or Windows-1252. If a file was converted twice by mistake, one reverse conversion often repairs it:
iconv -f UTF-8 -t ISO-8859-1 broken.txt > fixed.txt
Keep the original until you have checked the result.
Other details to know:
- BOM: some Windows tools put the bytes
ef bb bfat the start of UTF-8 files. Linux tools do not need it, and it breaks shell scripts that start with#!. - Line endings: Windows files end lines with CR LF.
dos2unixfixes that. It is a separate problem from the encoding. - Counting:
wc -ccounts bytes andwc -mcounts characters. With a UTF-8 locale, "é" is 2 bytes but 1 character. - Sorting:
sortfollows the locale's rules. UseLC_ALL=C sortfor plain byte order in scripts.
Typing special characters
- By code point: in GNOME and most GTK and IBus-based apps, press Ctrl+Shift+U, type the hex code (for example
20ac), then press Space or Enter to get €. - Compose key: set a key (often Right Alt) as Compose in your desktop's keyboard settings. Then Compose,
',egives é, and Compose,o,cgives ©. - Emoji: in GNOME apps, press Ctrl+; (semicolon) and search. The Characters app (GNOME) and KCharSelect (KDE) let you browse every character.
- Input methods for Chinese, Japanese, Korean, Vietnamese and other languages are IBus (built into GNOME; add one under Settings, Keyboard, Input Sources) and Fcitx 5 (common on KDE Plasma). On Plasma with Wayland, pick Fcitx 5 under System Settings, Virtual Keyboard, and do not set
GTK_IM_MODULEorQT_IM_MODULE.
For console keymaps and desktop keyboard layouts, see Keyboard and console.
The classic Unicode HOWTO is kept by the Linux Documentation Project at tldp.org. It records how the switch to UTF-8 was made around 2000.
Sources
- Unicode 18.0.0, Unicode Consortium and enumerated versions
- UTF-8, UTF-16, UTF-32 and BOM FAQ, Unicode Consortium
- Normalization FAQ, Unicode Consortium
- locale(7), man7.org
- localectl(1), man7.org
- locale-gen(8), Debian
- Locale, Debian Wiki
- Configuring system language (locale) and glibc langpacks, Oracle Linux 9 documentation (RHEL-compatible)
- iconv(1), man7.org
- convmv(1), Debian
- file(1) and uchardet(1), Debian
- Enter special characters, GNOME Help
- Using Fcitx 5 on Wayland, Fcitx project