Tools / Unicode inspector

Unicode inspector: code points, invisible characters, escapes

Paste text that behaves strangely: a string that does not compare equal, an identifier that fails to match, a line that renders in the wrong order. Every code point is listed, and the characters you cannot see are marked.

Text Up to 5,000 code points are analysed

Code points (first 500)
Code point	Shown	Name and category	UTF-8	UTF-16

Escape the text

Format

Escaped

Normalise

Normalised text

Examples:

What gets flagged

Kind	Examples	Why it matters
invisible	`U+200B` zero width space, `U+FEFF` byte order mark, `U+00AD` soft hyphen	two strings look identical and do not compare equal; a BOM at the start of a CSV breaks the first column name
bidirectional control	`U+202E` right-to-left override, `U+2066`-`U+2069` isolates	text is displayed in a different order than it is stored, as in the "Trojan Source" attacks on source code (CVE-2021-42574)
unusual space	`U+00A0` no-break space, `U+202F`, `U+3000`	not matched by a plain space in searches and splits; `\s` in JavaScript does match them
control character	`U+0000`-`U+001F` except tab and line breaks	often a sign of binary data or a broken export
mixed scripts	Latin "paypal" with a Cyrillic `а` (U+0430)	look-alike letters in names, domains and identifiers
replacement character	`U+FFFD`	bytes were decoded with the wrong encoding somewhere upstream

Names are shown for the characters that are hard to tell apart by eye: spaces, invisible and bidirectional characters, dashes, quotes, variation selectors and a few others. For every other character the general category is shown, from the Unicode tables of your browser.

Normalisation

é can be stored as one code point (U+00E9) or as e followed by a combining accent (U+0301). The two look the same and compare unequal. NFC composes such pairs and NFD splits them. NFKC and NFKD also replace compatibility characters: ﬁ becomes fi and ① becomes 1. Normalise to NFC before comparing or storing user input, and use NFKC only where losing those distinctions is acceptable, such as identifiers and search keys.

Counting

A thumbs-up with a skin tone is one character on screen, two code points, four UTF-16 code units (what JavaScript's length counts) and eight UTF-8 bytes. Database column limits, HTTP body limits and many APIs count bytes; JavaScript counts UTF-16 units; people count what they see. "Characters as seen" uses Intl.Segmenter, where the browser supports it.

Escapes

Every escaped form reads back to the same code points in its language. Two cannot express everything: HTML turns a reference to U+0000 into U+FFFD and references to U+0080-U+009F into Windows-1252 characters, and a lone surrogate has no UTF-8 form, so the page warns instead of giving a wrong result.

What was checked

Categories were compared with Python's unicodedata for every code point of the Basic Multilingual Plane and 30,000 others; the only differences are characters whose category Unicode itself changed after the version Python ships. The name table, the UTF-8 and UTF-16 forms, the escapes and the four normalisation forms were checked the same way.