Many years ago, domains could only consist of the Latin letters A to Z, digits, and a few other characters. Internationalized Domain Names (IDNs) were created to better support non-Latin alphabets for web users around the globe.
Different characters from different (or even the same!) languages can look very similar. We’ve seen reports of proof-of-concept attacks. These are called homograph attacks. For example, the Latin “a” looks a lot like the Cyrillic “а”, so someone could register http://ebаy.com (using Cyrillic “а”), which could be confused for http://ebay.com. This is a limitation of how URLs are displayed in browsers in general, not a specific bug in Chrome.
In a perfect world, domain registrars would not allow these confusable domain names to be registered. Some domain registrars do exactly that, mostly by restricting the characters allowed, but many do not. To better protect against these attacks, browsers display some domains in punycode (looks like xn--...) instead of the original IDN, according to their own IDN policies.
This is a challenging problem space. Chrome has a global user base of billions of people around the world, many of whom are not viewing URLs with Latin letters. We want to prevent confusion, while ensuring that users across languages have a great experience in Chrome. Displaying either punycode or a visible security warning on too wide of a set of URLs would hurt web usability for people around the world.
Chrome and other browsers try to balance these needs by implementing IDN policies in a way that allows IDN to be shown for valid domains, but protects against confusable homograph attacks.
Chrome's IDN policy is one of several tools that aim to protect users. Google Safe Browsing continues to help protect over two billion devices every day by showing warnings to users when they attempt to navigate to dangerous or deceptive sites or download dangerous files. Password managers continue to remember which domain password logins are for, and won’t automatically fill a password into a domain that is not the exactly correct one.
IDNs were devised to support arbitrary Unicode characters in hostnames in a backward-compatible way. This works by having user agents transform hostnames containing non-ASCII Unicode characters into an ASCII-only hostname, which can then be sent on to DNS servers. This is done by encoding each domain label into its punycode representation. This representation includes a four-character prefix (xn--) and then the unicode translated to ASCII Compatible Encoding (ACE). For example, http://öbb.at is transformed to http://xn--bb-eka.at.
Since Chrome 51, Chrome uses an IDN display policy that does not take into account the language settings (the Accept-Language list) of the browser. A similar strategy is used by Firefox.
Google Chrome decides if it should show Unicode or punycode for each domain label (component) of a hostname separately. To decide if a component should be shown in Unicode, Google Chrome uses the following algorithm:
Convert each component stored in the ACE to Unicode per UTS 46 transitional processing (ToUnicode).
If there is an error in ToUnicode conversion (e.g. contains disallowed characters, starts with a combining mark, or violates BiDi rules), show punycode.
If there is a character in a label not belonging to Characters allowed in identifiers per Unicode Technical Standard 39 (UTS 39), show punycode.
If any character in a label belongs to the disallowed list, show punycode.
If the component uses characters drawn from multiple scripts, it is subject to a script mixing check based on “Highly Restrictive” profile of UTS 39 with an additional restriction on Latin. If the component fails the check, show the component in punycode.
If two or more numbering systems (e.g. European digits + Bengali digits) are mixed, show punycode.
If there are any invisible characters (e.g. a sequence of the same combining mark or a sequence of Kana combining marks), show punycode.
If there are any characters used in an unusual way, show punycode. E.g. LATIN MIDDLE DOT (·) used outside ela geminada.
Test the label for mixed script confusable per UTS 39. If mixed script confusable is detected, show punycode.
Test the label for whole script confusables: If all the letters in a given label belong to a set of whole-script-confusable letters in one of the whole-script-confusable scripts and if the hostname doesn't have a corresponding allowed top-level-domain for that script, show punycode. Example for Cyrillic: The first label in hostname аррӏе.com (xn--80ak6aa92e.com) is all Cyrillic letters that look like Latin letters AND the TLD (com) is not Cyrillic AND the TLD is not one of the TLDs known to host a large number of Cyrillic domains (e.g. ru, su, pyc, ua). Show it in punycode.
If the label contains only digits and digit spoofs, show punycode.
If the label matches a dangerous pattern, show punycode.
If the skeleton of the registrable part of a hostname is identical to one of the top domains after removing diacritic marks and mapping each character to its spoofing skeleton (e.g. www.googlé.com with é in place of e), show punycode.
Otherwise, show Unicode.
This is implemented by IDNToUnicodeOneComponent() and IsIDNComponentSafe() in components/url_formatter/url_formatter.cc and IDNSpoofChecker class in components/url_formatter/spoof_checks/idn_spoof_checker.cc.
In addition to the spoof checks above, Chrome also implements a full page security warning to protect against lookalike URLs. You can find an example of this warning at chrome://interstitials/lookalike. This warning blocks main frame navigations that involve lookalike URLs, either as a direct navigation or as part of a redirect.
The algorithm to show this warning is as follows:
If the scheme of the navigation is not http or https, allow the navigation.
If the navigation is a redirect, check the redirect chain. If the redirect chain is safe, allow the navigation. (See Defensive Registrations section for details).
If the hostname of the navigation has at least a medium site engagement score, allow the navigation. Site engagement score is assigned to sites by the Site Engagement Service.
If the hostname of the navigation is in domains.list, allow the navigation.
If the user previously allowed the hostname of the navigation by clicking “Ignore” in the warning, allow the navigation. Currently, user decisions are stored per tab, so navigating to the same site in a new tab may show the warning.
If the hostname has the same skeleton as a recently engaged site or a top 500 domain, block the navigation and show the warning.
All of these checks are done locally on the client side.
Domain owners can sometimes register multiple versions of their domains, such as the ASCII and IDN versions, to improve user experience and prevent potential spoofs. We call these supplementary domains defensive registrations.
In some cases, Chrome's lookalike warning may flag and block navigations to these domains:
domains.list but the other isn't, the latter will be blocked.Domain owners can avoid the “Did you mean” warning by redirecting their defensive registrations to their canonical domain.
Example: If you own both example.com and éxample.com and the majority of your traffic is to example.com, you can fix the warning by redirecting éxample.com to example.com. The lookalike warning logic considers this a safe redirect and allows the navigation. If you must also redirect http navigations to https, do this in a single redirect such as http://éxample.com -> https://example.com. Use HTTP 301 or HTTP 302 redirects, the lookalike warning ignores meta redirects.
We reward certain cases of IDN spoofs according to Chrome's Vulnerability Reward Program policies. Please see this document before reporting a security bug.