Author Topic: Backup font for missing characters  (Read 7170 times)

Hakazaba

  • Newbie
  • *
  • Posts: 20
Backup font for missing characters
« on: December 23, 2015, 10:38:43 PM »
There are many fonts out there that lack characters. I think it would be much better to have it so that rather than rendering a red X when a font lacks a character we instead have a secondary font that will be used for missing characters.

It should be a rather simple feature.

Stephan B.

  • Administrator
  • Hero Member
  • *****
  • Posts: 5687
Re: Backup font for missing characters
« Reply #1 on: December 24, 2015, 01:11:57 AM »
There are many fonts out there that lack characters. I think it would be much better to have it so that rather than rendering a red X when a font lacks a character we instead have a secondary font that will be used for missing characters.

It should be a rather simple feature.

This is commonly referred to as a Fallback font.

This is actually pretty complex to implement as this fallback font uses a different font atlas texture and metrics. So you have to either add the missing glyph into the texture of the current font at run time which assuming there is enough room in the current font atlas texture can also result in a significant performance overhead. Alternatively, you could create separate geometry for those missing characters and interleave those in the current font's geometry. Since two materials would be used in this case, you would end up with an additional draw call.

Having said that, this feature is actually part of the Multi Font & Material support which I am currently working on. If you do a search on Multi Font & Material or Fallback you'll find those threads with additional information.

Here is a preview of Multi Fonts & Materials as well as Sprites.

« Last Edit: December 24, 2015, 01:15:21 AM by Stephan B. »

Hakazaba

  • Newbie
  • *
  • Posts: 20
Re: Backup font for missing characters
« Reply #2 on: December 24, 2015, 12:13:06 PM »
The multi font support demo on youtube is actual the entire reason i purchased text mesh pro. So im looking forwards to that a lot. So yes, if you are planning to impliment a fallback font at the same time already then great!

Multi font support is critical for the game im making. So information on when that will be available would be greatly appreciated.

Hakazaba

  • Newbie
  • *
  • Posts: 20
Re: Backup font for missing characters
« Reply #3 on: December 27, 2015, 12:10:39 AM »
I have noticed another thing, some fonts dont have any capital letters, and at the moment, it just leaves a space. It would be nice if TMP detected if there was no captial letters, and just used a lowercase capital in this case.

Stephan B.

  • Administrator
  • Hero Member
  • *****
  • Posts: 5687
Re: Backup font for missing characters
« Reply #4 on: December 27, 2015, 01:58:08 AM »
I have noticed another thing, some fonts dont have any capital letters, and at the moment, it just leaves a space. It would be nice if TMP detected if there was no captial letters, and just used a lowercase capital in this case.

That should already be working unless there is a new bug in there. I'll check in the morning to make sure this is still working as expected.

Stephan B.

  • Administrator
  • Hero Member
  • *****
  • Posts: 5687
Re: Backup font for missing characters
« Reply #5 on: January 09, 2016, 02:08:02 AM »
Here are the first screenshots of a text object using multiple fonts and sprites along with a fallback font asset.

[ Guests cannot view attachments ]

In this example, ARIAL SDF is the font asset assigned to the text object. Using the UTF-16 support I am typing the alpha and beta letters which are not available in the ARIAL SDF Font Asset. As per the image below, you will notice that I have assigned the ARIAL Greek SDF font asset to the list of Fallback fonts. This ARIAL Greek SDF font asset only contains the Greek alphabet. So when these glyph are not found in the ARIAL SDF font asset, TextMeshPro looks through the list of fallback fonts to locate those glyphs.

Here is an image showing the revised TMP Settings panel where the fallback fonts can be defined.

[ Guests cannot view attachments ]

In this next image, I added the FontAwesome SDF Font Asset to the fallback list and using the UTF-32 support \U0000F179 to add the apple glyph from that font.

[ Guests cannot view attachments ]
« Last Edit: January 09, 2016, 05:10:24 AM by Stephan B. »

AurochChris

  • Newbie
  • *
  • Posts: 32
Re: Backup font for missing characters
« Reply #6 on: February 01, 2016, 04:27:04 AM »
Just wondering; would it be hard to define a character range for which the fallback automatically triggers; regardless of whether characters exist in the main font or not?

The use case is an example is in one of our current projects (not using TMPro) we are using Helvetica Neue, which came with the Arial characters for CJK languages merged in. Our Chinese player base has requested us to use Source Hans instead (apparently Arial CJK is really unpleasant to read!); so having a parameter that would switch out the font automatically would be amazing.

But don't worry about it if it's a significant amount of work, just curious :)

Stephan B.

  • Administrator
  • Hero Member
  • *****
  • Posts: 5687
Re: Backup font for missing characters
« Reply #7 on: February 01, 2016, 11:30:39 AM »
Just wondering; would it be hard to define a character range for which the fallback automatically triggers; regardless of whether characters exist in the main font or not?

The use case is an example is in one of our current projects (not using TMPro) we are using Helvetica Neue, which came with the Arial characters for CJK languages merged in. Our Chinese player base has requested us to use Source Hans instead (apparently Arial CJK is really unpleasant to read!); so having a parameter that would switch out the font automatically would be amazing.

But don't worry about it if it's a significant amount of work, just curious :)

I'll have to think about this one.

It might be easier to remove the Arial CJK characters from that font and then assign whatever font asset you wish to use as a fallback for CJK.

AurochChris

  • Newbie
  • *
  • Posts: 32
Re: Backup font for missing characters
« Reply #8 on: February 08, 2016, 09:21:26 AM »
That's a completely fair point. I've been messing around with this for a short time; and my main concern with editing an existing font with CJK is missing something - considering how many different sets there are to edit.

I'll try and find an automated way of deleting a certain range of chars in font forge or similar program. :)

AurochChris

  • Newbie
  • *
  • Posts: 32
Re: Backup font for missing characters
« Reply #9 on: February 29, 2016, 06:24:21 AM »
After playing with fall-back fonts for a day or two; I can see two useful features. (No clue about the feasibility of these!)

1) Set specific font-specific fallbacks. For example, I might want "Helvetica Bold" to fall-back to "Helvetica Cyrillic Bold" first, while in other places I might need "Linux Biolinum" to go to "Linux Biolinum Cyrillic".

2) If a fall-back font is used; have an option to replace the existing characters with the ones on the fall-back font if they're available. Again mostly useful for Cyrillic, but let's say I have to replace "Helvetica" for Cyrillic with "Arial" - I will have some Helvetica letters (a, p, n) and some "Arial" Cyrillic letters in the same word. If I had an option to toggle then I'd just design once , and then I'd never have to look again. :)

Racking my brain a bit I don't see any horrible consequences; but could be wrong. Let me know what you think!

Stephan B.

  • Administrator
  • Hero Member
  • *****
  • Posts: 5687
Re: Backup font for missing characters
« Reply #10 on: February 29, 2016, 06:36:49 AM »
After playing with fall-back fonts for a day or two; I can see two useful features. (No clue about the feasibility of these!)

1) Set specific font-specific fallbacks. For example, I might want "Helvetica Bold" to fall-back to "Helvetica Cyrillic Bold" first, while in other places I might need "Linux Biolinum" to go to "Linux Biolinum Cyrillic".

As part of the ability to use multiple fonts in the same object, I started implementing support for font-weight where you will be able to assign to a font the various weights like bold and the italic counter part. See the following thread http://digitalnativestudios.com/forum/index.php?topic=762.msg6861#msg6861

Quote
2) If a fall-back font is used; have an option to replace the existing characters with the ones on the fall-back font if they're available. Again mostly useful for Cyrillic, but let's say I have to replace "Helvetica" for Cyrillic with "Arial" - I will have some Helvetica letters (a, p, n) and some "Arial" Cyrillic letters in the same word. If I had an option to toggle then I'd just design once , and then I'd never have to look again. :)

Racking my brain a bit I don't see any horrible consequences; but could be wrong. Let me know what you think!

I am not sure I follow the second example. If the characters are contained in both fonts, why would it use the fallback?

AurochChris

  • Newbie
  • *
  • Posts: 32
Re: Backup font for missing characters
« Reply #11 on: March 01, 2016, 05:05:55 AM »
Apologies - I don't think I explained myself well on the first point. I've included a screenshot of the behaviour I am currently seeing, and the behaviour I'd ideally like to see as the UI artist. In the screenshot I haven't included the different font weights and styles; but the 'problem' would be exacerbated with that. (But please note - it's not a deal-breaker or anything and I am very happy we get to show missing chars!)

For point two, I actually made a bunch of assumptions that are false:

a) I assume cyrillic uses the latin letters "A a" - it actually uses a cyrillic unicode instead, so you'll never mix the Cyrillic and English alphabet in the same phrase.
b) I assumed players using CJK languages might want to type out foreign names with latin script - this is also false, as the most common usage is to translate them phonetically into CJK.
c) Plenty of Chinese players will want to communicate in a foreign language still; but if they start typing in English and then add a Chinese character, my proposed feature would visibly change the font which is ugly.

Given those reasons, I think it's better if I ask one our programmers to write a little script which detects the language the game is set to, which then switches out the fonts on TMPro objects for the appropriate CJK font.

« Last Edit: March 01, 2016, 12:05:44 PM by AurochChris »

Stephan B.

  • Administrator
  • Hero Member
  • *****
  • Posts: 5687
Re: Backup font for missing characters
« Reply #12 on: March 02, 2016, 02:50:06 AM »
Apologies - I don't think I explained myself well on the first point. I've included a screenshot of the behaviour I am currently seeing, and the behaviour I'd ideally like to see as the UI artist. In the screenshot I haven't included the different font weights and styles; but the 'problem' would be exacerbated with that. (But please note - it's not a deal-breaker or anything and I am very happy we get to show missing chars!)

Once the font weight is implemented and you have the ability to assign to each font in the font editor panel the appropriate font asset for each weight and the italic version as well, that should cover the first item.

Quote
For point two, I actually made a bunch of assumptions that are false:

a) I assume cyrillic uses the latin letters "A a" - it actually uses a cyrillic unicode instead, so you'll never mix the Cyrillic and English alphabet in the same phrase.
b) I assumed players using CJK languages might want to type out foreign names with latin script - this is also false, as the most common usage is to translate them phonetically into CJK.
c) Plenty of Chinese players will want to communicate in a foreign language still; but if they start typing in English and then add a Chinese character, my proposed feature would visibly change the font which is ugly.

Given those reasons, I think it's better if I ask one our programmers to write a little script which detects the language the game is set to, which then switches out the fonts on TMPro objects for the appropriate CJK font.

Like the old days of desktop publishing where people could created Ransom Notes where each character used a different font and size, i guess my job is to make it possible although it might break all the rules of typography. As they say, I just make the tool. I don't take responsibility for how people use it :) 

Stephan B.

  • Administrator
  • Hero Member
  • *****
  • Posts: 5687
Re: Backup font for missing characters
« Reply #13 on: March 04, 2016, 04:09:18 PM »
I made further improvements to the Font Fallback system where you will now be able to assign Fallback Font Assets for individual Font Assets in the Font Asset Editor Panel.

Here is an example where I have a text object using two different fonts which are ARIAL SDF and IMPACT SDF. Neither of these font assets include the Greek character for Omega "Ω" or "\u03a9". So without the Fallback Font Asset support, we would be getting something like this.

[ Guests cannot view attachments ]

Now by creating another Font Asset for ARIAL which only includes the Greek characters which are from U+0370 to U+3FF Unicode range and assigning this new Font Asset as a Fallback to our main ARIAL SDF Font Asset in the Font Asset Editor Panel (as seen below) the missing glyph (square) is now replaced by the Greek Omega character from this new Fallback Font Asset.

[ Guests cannot view attachments ]

[ Guests cannot view attachments ]

Now in the current release where the Fallback Font Assets are only defined in the TMP Settings file, TextMesh Pro would search thru this list of Font Assets and use the first Font Asset which includes this missing character. So as per our example here, we would end up with the ARIAL version of the Omega character although using the <font> our text is now using IMPACT SDF. With this new change which will be included in the next release, TextMesh Pro will now search thru the Fallback Font Assets of the currently used Font Asset and then if it is unable to locate the missing glyph, look through the list of Fallback Font Assets defined in the TMP Settings file. Below is the result using the new implementation where you can see the IMPACT version of the Omega character is used.

[ Guests cannot view attachments ]

This new feature should prove useful in handling multiple languages as users will be able to create specific Fallback Font Assets for different characters sets on a per font asset basis.



« Last Edit: March 04, 2016, 04:52:59 PM by Stephan B. »

AurochChris

  • Newbie
  • *
  • Posts: 32
Re: Backup font for missing characters
« Reply #14 on: March 07, 2016, 02:51:58 AM »
Awesome Stephan! Much appreciated!