Commit Graph

193 Commits

Author SHA1 Message Date
Günter Milde
b32cf2a4c0 unicodesymbols: support Thai characters.
Also update the ctests.
2019-04-09 23:07:05 +02:00
Günter Milde
d4ade2bc1c unicodesymbols: add Hebrew nikud symbols and presentation forms.
The "nikud" (vowel) signs, shindot, and shindot  are combining Unicode
characters. However, LaTeX-Hebrew expects them as postfix characters, not
accent macros (cf. www.cs.tau.ac.il/~stoledo/Bib/Pubs/vowels.pdf).
2019-04-01 19:12:36 +02:00
Günter Milde
df79f329e6 Unicodesymbols: add force flags. 2019-03-28 13:30:22 +01:00
Günter Milde
f7fbd8c1ab Hebrew shindot and shindot but work fine as postfix. 2019-03-27 14:39:08 +01:00
Günter Milde
80b6a0038a Add lib/unicodesymbols support for Hebrew.
Hebrew language must be set (otherwise character macros are not defined)
but this is normally no problem because it is also neede for RTL support.
2019-03-22 23:55:00 +01:00
Günter Milde
d6d98ffbe4 Fix remainder of #9681 (textcyrillic and textgreek handling).
* Fix macro termination if \textcyrillic or \textgreek is not required
  for Greek or Cyrillic letter.

* Replace "writeScriptChars" with conditionals in the character-output loop in
  "Paragraph::latex" (solves "FIXME: modifying i here is not very nice...").

The font changing commands \textcyrillic and \textgreek are no longer
part of the textcommand in "lib/unicodesymbols" but added when required
in Paragraph::Private::latexSpecialChar.
2019-03-19 22:46:33 +01:00
Günter Milde
7f86f93810 unicodesymbols: Add force flags for Arabic and Hebrew input encodings. 2019-03-17 11:31:07 +01:00
Günter Milde
cc84fa6173 unicodesymbols: Use \r{A} instead of \AA for "Latin letter A with ring" and "Ångström sign".
Xe/LuaTeX convert \AA to the deprecated character u212B (which is missing
in the default LatinModern font) instead of the recommended u00C5.

Also fix some of the "missing character" errors in Math.lyx if compiled with
Xe/LuaTeX which were caused by the replacement of \AA with literal u212B characters
in math-insets due to the old definitions in unicodesymbols.

Update the minimal example for failures of Math.lyx with system fonts.
2019-03-08 18:36:56 +01:00
Kornel Benko
d50923d48a Typo 2019-03-02 21:59:29 +01:00
Günter Milde
9ad8c25d7b unicodesymbols: Add force flags for arabic codepage. 2019-03-01 18:52:12 +01:00
Günter Milde
b0d42b73f1 unicodesymbols: even more "force" flags. 2019-02-11 18:34:08 +01:00
Günter Milde
624b32616c unicodesymbols: one more missing "force" flag. 2019-02-10 16:54:34 +01:00
Günter Milde
ec86638d23 Amend 1e3e30da15 (unicodesymbols) and uninvert tests that now compile. 2019-02-09 20:44:29 +01:00
Günter Milde
1e3e30da15 Force "unicodesymbols" conversions for Hebrew and Arabic encodings.
Fixes wrong output with these encodings for Latin Unicode blocks.
Other Unicode blocks still need verification for correct output.
2019-02-06 16:50:49 +01:00
Günter Milde
321fbb8610 Amend 1c9d3da938 again: some more forced "unicodesymbols" conversions. 2019-02-05 20:29:10 +01:00
Günter Milde
9bca147d8b Amend 1c9d3da938: some more forced "unicodesymbols" conversions. 2019-02-05 10:22:53 +01:00
Günter Milde
1c9d3da938 Force "unicodesymbols" conversion for legacy CJK encodings where required. 2019-01-31 18:29:04 +01:00
Günter Milde
319a986fc0 Do not use \inputencoding when the buffer input encoding is one of the "legacy" CJK encodings.
If Document>Settings>Language>Encoding is set to any value except "auto" or "default", we
expect the whole document to use this encoding. Wiht encodings from the CJK package, this means
one big "CJK" environment and no encoding switches.

Characters that are not handled by the CJK package need to be "forced" in lib/unicodesymbols.
This is completed for "euc-cn", the others will follow.
2019-01-30 15:14:45 +01:00
Günter Milde
b9041328a0 unicodesymbols Force substitution of NNBSP in utf8.
NNBSP (narrow no break space) is supported by LyX's unicodesymbols
but missing in LaTeX inputenc *.dfu definitions. (TeXLive 2017)
2019-01-16 18:34:35 +01:00
Günter Milde
8b46630034 Fix unicodesymbols.
Remove 4 spurious tags that slipped in in [4ebb415/lyxgit].
2019-01-15 16:35:34 +01:00
Günter Milde
4ebb415773 Update unicodesymbols replacements
"textcomp" supports some more characters in its ts1enc.dfu Unicode definition file.
2019-01-14 22:48:38 +01:00
Juergen Spitzmueller
ead498f0a6 Add support for U+2693 (ANCHOR)
Fixes: #10706
2019-01-11 16:42:08 +01:00
Günter Milde
c2d20645be Force more symbols with utf8-cjk 2019-01-10 18:46:47 +01:00
Juergen Spitzmueller
81af82d1e8 Force more symbols with utf8-cjk 2019-01-10 09:56:11 +01:00
Juergen Spitzmueller
4786c59a8c force some symbols as macros with utf8-cjk
Probably more needed.
2019-01-10 09:43:40 +01:00
Günter Milde
251aaf102f unicodesymbols: support for LOW ASTERISK, use textcomp for (non-combining) Breve and Caron
The textcomp Unicode support file "ts1enc.dfu" defines 0x204E Low Asterisk
as \textasteriskcentered. LyX should follow suit.
The ASTERISK OPERATOR (correctly) maps to the same macro,
the "deprecated" tag marks the upstream mapping as preferred choice.
2019-01-09 14:59:22 +01:00
Günter Milde
f85dc3fbd4 Fix Ticket #9904 ("strange dots" with input encoding iso8859-7).
Added "force=iso8859-7" for some characters:

The iso8859-7.def file for the Greek 8-bit input encoding
used \textbullet as placeholder for non-defined characters.
This is fixed in v1.7 2019/01/08.

Once the fixed version is in common use, all "force=iso8859-7"
tags can be removed.
2019-01-09 09:23:16 +01:00
Juergen Spitzmueller
fd7490762d Support key symbols via menukeys package 2018-08-09 15:05:19 +02:00
Juergen Spitzmueller
09b2e337f7 Make textcommabelow feature conditional on encoding 2018-05-05 15:43:54 +02:00
Günter Milde
fa35f552cd Drop "force" from replacements for letters with comma-below accent. 2018-05-04 15:29:12 +02:00
Juergen Spitzmueller
b68bbe8c2e 0x2dd is not covered by utf8 inputenc yet (TL 2018) 2018-04-30 14:50:19 +02:00
Juergen Spitzmueller
0b2fae66e3 unicodesymbols: add general way to require a feature only for specific encodings
A feature can now be required only for specific input or font encodings:
- <feature>=enc1;enc2...  Require the feature <feature> only if the
                          character is used in one if the specified font
                          or input encodings.
- <feature>!=enc1;enc2... Require the feature <feature> only if the
                          character is used in a font or input encoding
                          that is not among the specified.
2018-04-28 13:31:29 +02:00
Juergen Spitzmueller
02d37932d0 \textcyr -> \textcyrillic
Use the command as defined by Babel. This allows us to use the (more
advanced) Babel command if provided instead of rolling our own.

I add a dummy file format change in case it turns out we need to
do something here for old documents (e.g. with user preamble definitions)
2018-04-21 16:28:15 +02:00
Günter Milde
ed3f6801bd Remove spurious no-break space. 2018-01-31 11:01:11 +01:00
Günter Milde
0d9708ef04 Use LaTeX macros instead of Latin transcription for Greek characters.
Use the LaTeX internal character representation (LICR) macros
provided by lgrenc.def (since version 0.8 from 2013-05-13)
in lib/unicodesymbols. This fixes the PDF bookmarks (except for the
legacy input encoding iso-8859-7) and solves the problem of a missing
"v" character in Libertine LGR fonts (see lyx-users from 2018-01-29).

The ctest unicodesymbols/008-greek-and-coptic_iso8859-7_pdf2" now fails
(due to #9681). This is not a regression, as it is already
"unreliable" (wrong output, Latin character instead of Greek).

Drop compatibility definition of \~ as perispomeni accent
(that was required with lgrenc.def < 0.8).
2018-01-30 22:31:08 +01:00
Günter Milde
24021a425e Force "textdegree" for encoding koi8-u. 2017-05-01 10:35:20 +02:00
Günter Milde
58a4cd3082 Remove workaround for bug in TeXLive2005. 2017-04-25 18:45:51 +02:00
Günter Milde
7b5de36a75 Use xfrac package for "fraction characters".
The xfrac package is the "state of the art" for "split-level" (nice) fractions.
Character replacements look consistent, scale properly and fit in the line.

Fixes #5220.
2016-12-18 23:42:15 +01:00
Günter Milde
15fd79201e lib/unicodesymbols tests and fixes.
Test unicodesymbols for most supported input encodings with Kornel's addition to ctests.

Add required "forces" to unicodesymbols:
* utf8x does not support all characters supported by LyX
* several 8-bit encodings map characters to math-mode commands - force replacement in text-mode so that LyX can wrap them in \\ensuremath.

Fix a misalignment (wrong replacements) in the Cyrillic Unicode block.

Use \\mathscr for Mathematical Script characters in Mathematical Alphanumeric Characters (in line with the characters in other unicode blocks.
2016-11-15 23:50:41 +01:00
Günter Milde
1805b41564 Fallback definitions for unicode paragraph/line break.
Convert characters 0x2028  LINE SEPARATOR and
0x2029 PARAGRAPH SEPARATOR to LaTeX equivalent.
(Just in case they slipped somehow into the source.)
2016-11-10 12:45:27 +01:00
Günter Milde
6674a91573 lib/unicodesymbols test update and fixes.
First run of Kornels patch for tests with all input encodings in lib/encodings.

Remove redundant sample files - keep only one sample and change the input encoding in the test script.

Put remaining failing test in "unreliableTests" for later sorting...
2016-11-09 21:15:14 +01:00
Günter Milde
6ff89c4b95 Do not convert \\\\textbackslash to 0x29f5.
Do not use REVERSE SOLIDUS OPERATOR for backwards conversion of
\\\\textbackslash in LyX and tex2lyx.

Both, \\\\ (005C REVERSE SOLIDUS = backslash) and 0x29f5 map to
\\\\textbackslash but 005c is the preferred back-transformation.

Otherwise, using \\\\ in "mathematical text" leads to literal 0x29f5 in the LyX
source which leads to "missing character" errors with non-TeX fonts.
2016-10-17 09:04:56 +02:00
Günter Milde
9c0f829d12 lib/unicodesymbols fixes and tests
Unicode blocks 125, 152, and 246
2016-10-11 00:38:17 +02:00
Günter Milde
52fbe6ea26 lib/unicodesymbols: fix test sample
The package "dictsym"provides symbols for maps and dictionaries.
Should it be used in unicodefonts or is it "too exotic"?
2016-10-08 22:48:13 +02:00
Günter Milde
efa844702c unicodesymbols fixes.
force=utf8 is required for most characters provided by add-on packgages
and (almost) all mathematical characters, because these are not
set up for inputencs utf8

unicodesymbols.py failed here (python 2.7 under Linux) before the simple fix
included in this commit.
2016-10-08 16:57:52 +02:00
Günter Milde
30c9c8e81a lib/unicodesymbols tests: add comments and remove spurious test samples. 2016-09-22 12:10:39 +02:00
Günter Milde
559c895130 lib/unicodesymbols: tests and fixes, Blocks 87 - 91
Also add more "stmaryrd" symbols.
2016-09-20 08:16:39 +02:00
Günter Milde
e82e48c44d lib/unicodesymbols: replace TABs. 2016-09-20 08:16:21 +02:00
Günter Milde
5156416279 lib/unicodesymbols tests and fixes: Block Dingbats. 2016-09-18 17:08:56 +02:00
Günter Milde
f957d32ed3 lib/unicodesymbols tests and fixes: Block Miscellaneous Symbols.
* add \Coffeecup and \Wheelchair

* add "force=utf8" where required

* uninvert tests: ctest works again after Kornel fixed the problem in 9f1b80a2af
2016-09-16 12:10:48 +02:00