Commit Graph

136 Commits

Author SHA1 Message Date
Günter Milde
40240ef6a0 Improve error message and ctests for input encoding cp858.
Encoding cp858 supported by only some iconv variants.
Most users will want to change their "encoding" setting instead
of installing/recompiling "iconv" to support this legacy encoding.

ctests are likely will fail with either "vanilla" or "enhanced"
iconv and test a situation that is unlikely to change generally,
so we ignore this test now by default.
2019-05-09 09:28:37 +02:00
Günter Milde
df1f0d82a2 ctests: dedicated tests for languages supported by polyglossia/babel 2019-05-08 12:07:20 +02:00
Günter Milde
cb39220930 Update tests and documentation for supported languages. 2019-05-03 14:04:11 +02:00
Günter Milde
74b1b9d86f ctest update
Separate xetex-inputenc test sample in working and non-working parts.

Sort HTML-only tests.

Update tagging and ignore-rules.

Change inputencoding to utf8 in dedicated tests (get pdf4_texF working).
2019-04-29 21:13:33 +02:00
Günter Milde
9fdae0d84c Use utf8 inputencoding for Russian documentation.
Solves errors with luatex and enables compilation with xetex.
Add dedicated test document for the errors with auto-legacy and xe/luatex.
2019-04-28 22:45:51 +02:00
Günter Milde
e8e3dafd8f ctests: Sort dedicated test documents for language support. 2019-04-28 19:34:37 +02:00
Günter Milde
7a87599a3e Load inputenc after babel with Thai.
"thai.ldf" breaks "inputenc" by changing catcodes.
Loading "inputenc" after "babel" allows a workaround.
2019-04-27 17:09:29 +02:00
Günter Milde
c6f64bc734 ctest update
invert tests failing in TL19,
fix sorting in "supported-languages.lyx".
2019-04-25 14:02:58 +02:00
Günter Milde
5c0ea92f80 ctests: add some dedicated test samples. 2019-04-20 21:53:16 +02:00
Günter Milde
15b7ea4828 ctest and documentation update
Thai works fine with LuaTeX, TeX-fonts and auto-legacy input encoding.

Remove obsolete preamble code,
we now load "fontenc" with Japanese documents by default.
2019-04-20 21:39:49 +02:00
Günter Milde
29265ce702 language support update (tests and workaround for Thai). 2019-04-16 22:05:58 +02:00
Günter Milde
62f8b4fac1 Document languages with new polyglossia support. 2019-04-12 18:34:06 +02:00
Günter Milde
e5435cd61e Update language support documentation and comments. 2019-04-12 18:08:17 +02:00
Günter Milde
9dd7ffff7b ctest update: Japanese documents (should) work with non-TeX fonts.
* do not ignore Japanese (platex) with system fonts.

* CJK can be used with XeTeX and TeX-fonts if the input encoding is utf8.
  do not ignore.

* TODO: set non-TeX fonts and uninvert where possible.
2019-04-12 18:08:17 +02:00
Günter Milde
da23637d7f Do not switch the input encoding inside documents using platex.
Fixes wrong and missing characters in text parts in other languages
(platex does not support "inputenc").

Fixes compilation errors due to desynchronized encoding switches.
2019-04-12 18:08:17 +02:00
Günter Milde
230cc9bc5b ctest: update tagging scripts. 2019-04-10 21:02:10 +02:00
Günter Milde
b32cf2a4c0 unicodesymbols: support Thai characters.
Also update the ctests.
2019-04-09 23:07:05 +02:00
Günter Milde
2efe99c33d Improve and document use of Thai language. 2019-04-09 17:13:44 +02:00
Günter Milde
f9486e8cf3 ctests: update names of included files in input encoding samples for Xe/LuaTeX. 2019-04-08 22:08:49 +02:00
Günter Milde
895fa9b3c6 ctest fixes.
Hebrew with "nikud" requires HE8 font encoding.
All tests requiring Hebrew 8-bit fonts are "nonstandard".
2019-04-03 08:36:46 +02:00
Günter Milde
2e854cd36c ctest update.
* some Japanese (platex) documents fail with inputenc "utf8-platex"
  (missing characters in non-Japanese text parts), because the
  Unicodechar definitions from "inputenc" are not used.

* some Japanes (platex) documents show wrong output with "auto",
  because platex ignores the encoding switch for text parts
  in other languages.

* Japanese Beamer documents must set default output to "pdf",
  because dvipdfm(x) produces wrong output with document class "Beamer".

* update tagging/inverting rules.

* use HE8 font encoding for Hebrew in language test.
2019-04-02 16:46:26 +02:00
Günter Milde
b724b4f32f Do not force the use of the HE8 font encoding for Hebrew.
While HE8 provides more characters and prevents use of bitmap fonts,
forcing its use may break older installations.

The dedicated test file 012_hebrew_he_HE8.lyx provides an
example for use of HE8 encoded fonts with babel-hebrew.
2019-04-02 13:21:38 +02:00
Günter Milde
d4ade2bc1c unicodesymbols: add Hebrew nikud symbols and presentation forms.
The "nikud" (vowel) signs, shindot, and shindot  are combining Unicode
characters. However, LaTeX-Hebrew expects them as postfix characters, not
accent macros (cf. www.cs.tau.ac.il/~stoledo/Bib/Pubs/vowels.pdf).
2019-04-01 19:12:36 +02:00
Günter Milde
e5567a1377 ctests: sample for Arabic as secondary language. 2019-03-29 10:44:21 +01:00
Günter Milde
ffc682424e Update ctest tagging and documentation/sample organisation. 2019-03-28 13:30:22 +01:00
Günter Milde
f0b326a9f4 Comprehensive language test document and some more fixes.
Work around or document language clashes and limitations.
2019-03-27 17:49:08 +01:00
Günter Milde
32b47e975e Fix settings for Hebrew with 8-bit TeX fonts. 2019-03-27 17:09:12 +01:00
Günter Milde
b1b8071168 ctest update.
#9681 is fixed in master.
Hebrew with 8-bit fonts fails with vanilla TeXLive.
Add link to ticket in dedicated test document.
2019-03-27 16:20:17 +01:00
Günter Milde
48b92cf63c Test for #11532 "inputencoding desynchronisation". 2019-03-27 15:57:38 +01:00
Günter Milde
f7fbd8c1ab Hebrew shindot and shindot but work fine as postfix. 2019-03-27 14:39:08 +01:00
Günter Milde
80b6a0038a Add lib/unicodesymbols support for Hebrew.
Hebrew language must be set (otherwise character macros are not defined)
but this is normally no problem because it is also neede for RTL support.
2019-03-22 23:55:00 +01:00
Günter Milde
1933102593 ctest update and small documentation fixes. 2019-03-19 22:17:52 +01:00
Günter Milde
dae90c6161 Warn user, if input encodings of master and child document differ. 2019-03-18 18:17:39 +01:00
Günter Milde
bda8acee67 Amend d53a13a: Bug 11519 only manifests if inputenc != "auto". 2019-03-17 11:20:58 +01:00
Günter Milde
d53a13a475 ctest for bug #11519 2019-03-16 17:55:44 +01:00
Günter Milde
2b61033324 ctests: add tests for #10600 (forced inputenc "ascii" with XeTeX and TeX-fonts). 2019-03-16 13:39:56 +01:00
Günter Milde
ddf947b8fd ctests: dedicated test for problem with textgreek and textcyrillic. 2019-03-12 22:44:13 +01:00
Günter Milde
cc84fa6173 unicodesymbols: Use \r{A} instead of \AA for "Latin letter A with ring" and "Ångström sign".
Xe/LuaTeX convert \AA to the deprecated character u212B (which is missing
in the default LatinModern font) instead of the recommended u00C5.

Also fix some of the "missing character" errors in Math.lyx if compiled with
Xe/LuaTeX which were caused by the replacement of \AA with literal u212B characters
in math-insets due to the old definitions in unicodesymbols.

Update the minimal example for failures of Math.lyx with system fonts.
2019-03-08 18:36:56 +01:00
Günter Milde
f674f563da ctest update: compilation errors due to #6463 are fixed. 2019-03-05 10:00:46 +01:00
Günter Milde
ff8cec5ea5 ctests: Problem with auto-date for Ukrainian with Xe/LuaTeX and TeX fonts.
New bug in TeXLive 18.
Missing characters with XeTeX and wrong characters with LuaTeX.

Also:
* Remove spurious (Latin) characters from uk/Intro.lyx
* "wrong-output" tag for Cyrillic documents with XeTeX and TeX fonts.
2019-03-04 16:22:00 +01:00
Günter Milde
fd106637be ctest and documentation update
The special test "arabic_simple.lyx" is no longer required,
"examples/ar/splash.lyx" works with 8-bit fonts now.

Update tagging patterns and comments.
2019-03-04 00:04:04 +01:00
Günter Milde
7a17610a00 Load "textcomp" and "pmboxdraw" before "(lua)inputenc" to fix #11454.
Prevents wrong or missing characters with LuaTeX and 8-bit fonts.

Also "uninvert" the corresponding test case and two other
no longer failing "unicodesymbols" exports.
2019-02-12 09:22:55 +01:00
Günter Milde
f8d42aed39 ctests: update Korean tests, add "language default" test including Chinese.
Set suitable non-TeX fonts in the test document.
Use "kotex" language package.
2019-02-05 11:24:53 +01:00
Günter Milde
1c9d3da938 Force "unicodesymbols" conversion for legacy CJK encodings where required. 2019-01-31 18:29:04 +01:00
Günter Milde
7e120159fc ctests: Update Chinese report example. 2019-01-30 15:54:30 +01:00
Günter Milde
319a986fc0 Do not use \inputencoding when the buffer input encoding is one of the "legacy" CJK encodings.
If Document>Settings>Language>Encoding is set to any value except "auto" or "default", we
expect the whole document to use this encoding. Wiht encodings from the CJK package, this means
one big "CJK" environment and no encoding switches.

Characters that are not handled by the CJK package need to be "forced" in lib/unicodesymbols.
This is completed for "euc-cn", the others will follow.
2019-01-30 15:14:45 +01:00
Günter Milde
b01eb6c6a4 ctests: fix/update CJK tests. 2019-01-30 00:53:31 +01:00
Günter Milde
34f5ba3c54 Fix for CJK-documents with input-encoding "utf8". 2019-01-28 17:49:58 +01:00
Günter Milde
964c15f80b ctests: update CJK export tests. 2019-01-28 17:24:39 +01:00
Günter Milde
5ebc0b3f43 Fix second part of #5386.
A \clearpage command issued right before \end{CJK} is recommended by the
package author to prevent any un-processed CJK chars outside the
\begin{CJK} and \end{CJK} scope. Otherwise, TOC, header, footer,
and may contain CJK chars but get processed outside the CJK environment scope.

Tha new dedicated export test fails without the fix.
2019-01-24 14:40:19 +01:00
Günter Milde
197735747e Support for documents in other scripts with parts in CJK scripts.
Fix CJK support for documents using input-encoding "utf8-cjk"
when the main language is not Chinese, Korean, or Japanese.
2019-01-24 00:40:51 +01:00
Günter Milde
e8c497e33e ctests: export tests for CJK in multilingual documents. 2019-01-23 00:57:20 +01:00
Günter Milde
968a3d06a4 Test for bug #6197.
With language Polish, "amssymb" must be loaded before "babel" to
avoid a name clash for command "\lll".
2019-01-22 23:59:11 +01:00
Günter Milde
f82030d211 Farsi document with custom font encoding OT1 fails.
Set font encoding in examples to "auto" and add minimal test sample.
2019-01-22 12:02:57 +01:00
Günter Milde
12c301d1ca update latex export tests, add test for bug #9871 2019-01-20 17:23:41 +01:00
Günter Milde
b9833f180a ctests: update inputenc tests.
Use \input in luainputenc-utf8 instead of copying the
content of test documents in latex/unicodesymbols/.
2019-01-20 17:23:41 +01:00
Günter Milde
b9041328a0 unicodesymbols Force substitution of NNBSP in utf8.
NNBSP (narrow no break space) is supported by LyX's unicodesymbols
but missing in LaTeX inputenc *.dfu definitions. (TeXLive 2017)
2019-01-16 18:34:35 +01:00
Günter Milde
cc0ef52a37 Update some special latex export tests. 2019-01-14 22:48:38 +01:00
Günter Milde
4ebb415773 Update unicodesymbols replacements
"textcomp" supports some more characters in its ts1enc.dfu Unicode definition file.
2019-01-14 22:48:38 +01:00
Günter Milde
0594a4fe72 ctests: update test tagging/categorization and export test for accents on Cyrillic letters. 2019-01-14 12:21:34 +01:00
Günter Milde
0f569e961d ctests: Update/clean up latex export tests after fix of #9681. 2019-01-10 17:21:01 +01:00
Günter Milde
251aaf102f unicodesymbols: support for LOW ASTERISK, use textcomp for (non-combining) Breve and Caron
The textcomp Unicode support file "ts1enc.dfu" defines 0x204E Low Asterisk
as \textasteriskcentered. LyX should follow suit.
The ASTERISK OPERATOR (correctly) maps to the same macro,
the "deprecated" tag marks the upstream mapping as preferred choice.
2019-01-09 14:59:22 +01:00
Günter Milde
23f7431cfa ctests: Dedicated LaTeX export tests for problematic input encodings. 2019-01-09 14:10:40 +01:00
Günter Milde
0cfc50883c Remove omikron from math tests.
The letter Omikron is not available in TeX math mode because
it looks like the latin O and hence not used as mathematical symbol.
2019-01-09 10:39:40 +01:00
Günter Milde
41055bab88 Fix handling of Thai tis620-0 input encoding.
The Thai tis620-0 input encoding is supported via the inputenc "plug in"
(data) file tis620.def from https://ctan.org/pkg/babel-thai.

We can handle it like the other contributed input encodings, e.g.,
Greek (ISO 8859-7) and the several Cyrillic encodings from
http://www.ctan.org/pkg/latex-cyrillic.

Under TeXLive 2018, the input encoding defaults to utf8, if there is no call to
inputenc. The added test file fails without the patch but compiles fine, if the
file "tis620.def" is present in the TEXPATH.
2019-01-09 09:23:16 +01:00
Günter Milde
81066e2894 ctests: utf8-plain now works with all export variants.
utf8-plain (Unicode (utf8 XeTeX)) is a power-user setting
for the input encoding with two use cases:

a) setup of system fonts or

b) setup of input encoding supportuser preamble

in the document class or user preamble.

The test file is an example for use case b.
2019-01-07 23:21:04 +01:00
Kornel Benko
620efb3540 Amend 88e5e5f: Wrap rotated cells with linebreaks in varwidth
Added testcase from #8085
2018-06-30 17:00:56 +02:00
Kornel Benko
0add7120ba ctests: Test dedicated LaTeX test samples with LaTeX export only.
A fixed size table cell which contains multiple paragraphs in other language
2018-04-25 12:14:32 +02:00
Günter Milde
275bb44158 ctests: move issue with nomenclature insets to a dedicated test case.
iconv fails, if a nomenclature inset contains an uncodable character
This led to failure of the indonesian UserGuide in the attic.
Fix it there and add a minimal, specific test sample instead.
2017-05-04 18:05:18 +02:00
Günter Milde
5637c9b321 ctests: Add test file for "exotic" inputencoding cp858.
Encoding cp858 is only supported by some iconv variants
Gnu iconv only supports it, if configured with "--enable-extra-encodings"
(see https://www.gnu.org/software/libiconv/)

Maybe drop support or add a configuration check?
2017-03-01 23:16:59 +01:00
Günter Milde
7b5de36a75 Use xfrac package for "fraction characters".
The xfrac package is the "state of the art" for "split-level" (nice) fractions.
Character replacements look consistent, scale properly and fit in the line.

Fixes #5220.
2016-12-18 23:42:15 +01:00
Scott Kostyshak
7cbbf00eaf Improve ctesting of Arabic documents
- invert the currently failing tests
- add a new test document that shows LyX's support of pdf2

For more details, see the discussion here:
https://www.mail-archive.com/search?l=mid&q=20161114035123.s5lt4ib4x4obtptp%40steph
2016-11-24 23:10:16 -05:00
Günter Milde
15fd79201e lib/unicodesymbols tests and fixes.
Test unicodesymbols for most supported input encodings with Kornel's addition to ctests.

Add required "forces" to unicodesymbols:
* utf8x does not support all characters supported by LyX
* several 8-bit encodings map characters to math-mode commands - force replacement in text-mode so that LyX can wrap them in \\ensuremath.

Fix a misalignment (wrong replacements) in the Cyrillic Unicode block.

Use \\mathscr for Mathematical Script characters in Mathematical Alphanumeric Characters (in line with the characters in other unicode blocks.
2016-11-15 23:50:41 +01:00
Günter Milde
1ff4aeaf1f Complete the renaming of a ctest sample directory.
Rename the directory for test samples  "export/latex/Unicode-characters" to "export/latex/unicodesymbols". This matches the purpose to test the lib/unicodesymbols file.
2016-11-14 14:32:46 +01:00
Günter Milde
6674a91573 lib/unicodesymbols test update and fixes.
First run of Kornels patch for tests with all input encodings in lib/encodings.

Remove redundant sample files - keep only one sample and change the input encoding in the test script.

Put remaining failing test in "unreliableTests" for later sorting...
2016-11-09 21:15:14 +01:00
Günter Milde
120e84af3a ctests: explore export failure of Math.lyx with Unicode fonts. 2016-10-17 10:36:45 +02:00
Günter Milde
971025b80d custom ctest sample: Accented Cyrillic characters fail if the input encoding is ASCII. 2016-10-11 00:38:17 +02:00
Günter Milde
9c0f829d12 lib/unicodesymbols fixes and tests
Unicode blocks 125, 152, and 246
2016-10-11 00:38:17 +02:00
Günter Milde
52fbe6ea26 lib/unicodesymbols: fix test sample
The package "dictsym"provides symbols for maps and dictionaries.
Should it be used in unicodefonts or is it "too exotic"?
2016-10-08 22:48:13 +02:00
Günter Milde
72ddc8fffa fix and test lib/unicodesymbols: Blocks 87 - 92 2016-09-30 15:07:25 +02:00
Günter Milde
30c9c8e81a lib/unicodesymbols tests: add comments and remove spurious test samples. 2016-09-22 12:10:39 +02:00
Günter Milde
559c895130 lib/unicodesymbols: tests and fixes, Blocks 87 - 91
Also add more "stmaryrd" symbols.
2016-09-20 08:16:39 +02:00
Günter Milde
5156416279 lib/unicodesymbols tests and fixes: Block Dingbats. 2016-09-18 17:08:56 +02:00
Günter Milde
f957d32ed3 lib/unicodesymbols tests and fixes: Block Miscellaneous Symbols.
* add \Coffeecup and \Wheelchair

* add "force=utf8" where required

* uninvert tests: ctest works again after Kornel fixed the problem in 9f1b80a2af
2016-09-16 12:10:48 +02:00
Günter Milde
9c8f815b37 lib/unicodesymbols tests: split 2-page test and fix missing command. 2016-09-15 18:15:07 +02:00
Günter Milde
d1a41a9bdc ctests: Test dedicated LaTeX test samples with LaTeX export only.
Move them to a subdir, ignore this subdir for other tests.

Dedicated test samples for LaTeX-specific problems don't give additional value if tested for loading, conversion, or other exports.
2016-09-15 17:18:47 +02:00