New bug in TeXLive 18.
Missing characters with XeTeX and wrong characters with LuaTeX.
Also:
* Remove spurious (Latin) characters from uk/Intro.lyx
* "wrong-output" tag for Cyrillic documents with XeTeX and TeX fonts.
* Do not change font choices when testing with non-TeX fonts.
* Add "nonstandard" tag for documents with requirements outside TeXLive.
* Ignore missing character in legacy Basque document.
If Document>Settings>Language>Encoding is set to any value except "auto" or "default", we
expect the whole document to use this encoding. Wiht encodings from the CJK package, this means
one big "CJK" environment and no encoding switches.
Characters that are not handled by the CJK package need to be "forced" in lib/unicodesymbols.
This is completed for "euc-cn", the others will follow.
utf8-plain (Unicode (utf8 XeTeX)) is a power-user setting
for the input encoding with two use cases:
a) setup of system fonts or
b) setup of input encoding supportuser preamble
in the document class or user preamble.
The test file is an example for use case b.
We don't invert unreliable tests for the same reason they are
inverted but, e.g., a nonstandard test that fails for some reason even with the
additional requirements installed or a test that shows wrong output
but also an error.
This encoding (modified Mac Cyrillic for Asian languages) is rarely used and not supported by Gnu iconv.
Update comments in lib/encodings.
Update ctests: Gnu iconv only supports cp858, if configured with "--enable-extra-encodings".
Test unicodesymbols for most supported input encodings with Kornel's addition to ctests.
Add required "forces" to unicodesymbols:
* utf8x does not support all characters supported by LyX
* several 8-bit encodings map characters to math-mode commands - force replacement in text-mode so that LyX can wrap them in \\ensuremath.
Fix a misalignment (wrong replacements) in the Cyrillic Unicode block.
Use \\mathscr for Mathematical Script characters in Mathematical Alphanumeric Characters (in line with the characters in other unicode blocks.
First run of Kornels patch for tests with all input encodings in lib/encodings.
Remove redundant sample files - keep only one sample and change the input encoding in the test script.
Put remaining failing test in "unreliableTests" for later sorting...
* custom non-tex fonts with all required characters
* use 2.2 fileformat (easier backporting)
* test all export formats
Also, fix pattern for "mixing_inTitle_layouts" in unreliableTests.
europeCV and modernCV examples can now be exported to PDF using
LuaTeX. For the specific output that was fixed, look at the diff and
see the description in suspiciousTests that was removed by this
commit. The output was checked manually and appears fine. These
tests are thus "uninverted".
Exporting those examples to DVI with LuaTeX does not exit with
error, but the output drops characters with accents. Thus, these
tests are now marked with the "wrong_output" label. I reported this
issue on the LuaTeX mailing list at [1], but since DVI export is not
given high priority, I don't expect much action.
Note that these changes reflect an updated TeX Live 2016
installation.
[1]
https://www.mail-archive.com/search?l=mid&q=20160831134006.4fewxothddqfeyw4%40steph
As of 0b1cf133 we now warn in the GUI of this issue, but there is a
discussion about whether we should change our LaTeX output and allow
for the workflow of mixing inTitle layouts. For more information,
see #10347.
XeTeX with TeX fonts is only safe with ASCII input encoding (see #9740)
and we therefore force "ascii" when exporting with XeTeX and 8-bit TeX-fonts.
However, "utf8-plain" is a "power-user" option, which allows to switch off LyX's
encoding of the LaTeX file:
keep this also for "XeTeX with TeX fonts".
The user is responsible to ensure all characters can be processed and are
correctly shown in the output. The provided test sample shows the problems
with this encoding without special measures (like loading fontspec in the
user-preamble or a document class).
Simplify the logic for language package selection and make it more consistent:
Use polyglossia with non-TeX fonts (system fonts/Unicode fonts) for all
export flavours (XeTeX, LuaTeX, DVI-LuaTeX), if the language package setting
is "auto" and there is no language not supported by Babel and no package
providing Babel.
This solves some Babel-related autotest cases and leads to some new failures
due to the polyglossia language nesting problem.
Comment by Günter Milde:
Actually, *all* Spanish manuals either fail or have wrong output with
Unicode TeX engines and 8-bit fonts. The reason is known: a bug in Babel
that uses utf8 strings whenever Xe- or LuaTeX is detected.
