Move them to a subdir, ignore this subdir for other tests.
Dedicated test samples for LaTeX-specific problems don't give additional value if tested for loading, conversion, or other exports.
This led to errors when compiling with polyglossia (and non-TeX fonts).
A minimal (currently non-compiling) test sample is kept in autotests/export/
and inverted in suspiciousTests.
europeCV and modernCV examples can now be exported to PDF using
LuaTeX. For the specific output that was fixed, look at the diff and
see the description in suspiciousTests that was removed by this
commit. The output was checked manually and appears fine. These
tests are thus "uninverted".
Exporting those examples to DVI with LuaTeX does not exit with
error, but the output drops characters with accents. Thus, these
tests are now marked with the "wrong_output" label. I reported this
issue on the LuaTeX mailing list at [1], but since DVI export is not
given high priority, I don't expect much action.
Note that these changes reflect an updated TeX Live 2016
installation.
[1]
https://www.mail-archive.com/search?l=mid&q=20160831134006.4fewxothddqfeyw4%40steph
As of 0b1cf133 we now warn in the GUI of this issue, but there is a
discussion about whether we should change our LaTeX output and allow
for the workflow of mixing inTitle layouts. For more information,
see #10347.
The keytests were previously enabled by default if the necessary
dependencies were found. They require a GUI and mouse so can
sometimes be annoying. Further, they are not currently reliable.
They are thus now disabled by default.
XeTeX with TeX fonts is only safe with ASCII input encoding (see #9740)
and we therefore force "ascii" when exporting with XeTeX and 8-bit TeX-fonts.
However, "utf8-plain" is a "power-user" option, which allows to switch off LyX's
encoding of the LaTeX file:
keep this also for "XeTeX with TeX fonts".
The user is responsible to ensure all characters can be processed and are
correctly shown in the output. The provided test sample shows the problems
with this encoding without special measures (like loading fontspec in the
user-preamble or a document class).
In collaboration with Günter Milde:
1.) Allow char ':' be part of a ctest-label
2.) Eliminate redundant label naming and directory names
(The testnames should not repeat the directory name)
When updating to the latest TeX Live revision, these exports now
succeed.
Inspecting the differences between the "good" PDF and the "bad" PDF
(where the test failed) there are three differences, which can be
found on the printed page numbers 81 and 82 (PDF page numbers 91 and
92). The accented i in "lím" was printed incorrectly (and the
missing glyph was correctly detected). After the update the
character is printed correctly and there is no longer an error.
The package that caused the change is likely babel or babel-spanish.
You can see the log from the TeX Live update that caused the fix by
seeing the attachment to the message here:
https://www.mail-archive.com/search?l=mid&q=20160313015902.lny3g5aujh4c4aps%40scott-Za1510
It could happen that the variable was set in creating a previous test-case.
Some combinations in the controlling files (suspiciousTests, unrelibleTests, ...)
did not set this variable.
* sort and comment remaining unsorted filter patterns in suspiciousTests
(mostly just move to TODO and state what needs to be done).
* comment the patterns for achemso and Math,
(LuTeX incompatibility fixed in TeXLive 2016-01-07).
Simplify the logic for language package selection and make it more consistent:
Use polyglossia with non-TeX fonts (system fonts/Unicode fonts) for all
export flavours (XeTeX, LuaTeX, DVI-LuaTeX), if the language package setting
is "auto" and there is no language not supported by Babel and no package
providing Babel.
This solves some Babel-related autotest cases and leads to some new failures
due to the polyglossia language nesting problem.
* missing characters in linguistics example with system fonts
* language nesting problem in fr/linguistics example
* 110f505b63 solved one failure (Basque example file)
Works with system-fonts due to font substitution in the test-script.
TODO: set system fonts that work in the lyx source(s) to allow compilation
"by hand".
testcases_speed_lyx21 added to suspiciousTests
Handling unknown body token: `\begin_manifest'at line ...
Literate_lyx16 and noweb2lyx_lyx16 added to ignoredTests
Plenty of warnings (~ 390)
New sublabels LyXBug, ERT.
Renamed sublabel TeXLimit to TeXissue.
More sorting and unification of regular expressions.
Removing double entries.
Fixes for some expressions (eu_UserGuide_pdf, examples/ja/(knitr|lilypond|sweave))
Since the stop condition is that the last two consecutive created
files are identical, we do not need to check the last file.
Also checking for load of the created files is not needed, because
exporting the previous file implicitly loads too.
Exporting to some previous lyx format:
Inside the loop
Stop at export with errors/warnings
Stop if exported file not loadable
Stop if exported file identical to previous exported file (OK case)
This is done through creating MD5SUM
We test here, if any \end_{somethig} matches \begin_{something}.
Exeptions are \end_index and \end_branch ATM, they should
match \index and \branch respectively.
Also added new testfile.
Remove patterns matching tests now ignored (CJK/Korean, latex8).
Move patterns matching documents in the attic under new sublabel.
Add more verbose comments.
Comment by Günter Milde:
Actually, *all* Spanish manuals either fail or have wrong output with
Unicode TeX engines and 8-bit fonts. The reason is known: a bug in Babel
that uses utf8 strings whenever Xe- or LuaTeX is detected.
Always-Babel now set in the example document.
Adapted the autotest categorization:
* fails for some developers (why?)
* wrong output with pdflatex/LuaTeX and DVI (missing landscape slides).
Previously all labels got the depth '7' while processing 'suspiciousTestss'.
The depth is used to sort how our labels are used to build a test-label.
Say a test gets label a, b, and c, with depth 3, 8 and 2.
The constructed test-label will be "b🅰️c"
For new (to be implemented) 'reason-labels' it is more convenient to assign them higher values.
suspiciousTests: Remove Tutorial from the regex
suspendedTests: Explicit list of languages needed
ignoredTests: Language-string is separated from other strings only by '/'. '|', '_', '(' and ')'
LyXMacros.cmake: Overseen the macro 'setmarkedtestlabel()' which added it automatically
if the test was to invert the test result.
ExportTests.cmake: Correct label handling
This works around a limitation of the test machinery, which never switches
TeX fonts on for format that need that, it only switches TeX fonts off for
formats needing it.
Thanks to Kornel we do now have the infrastructure for running dedicated
export tests. This is the first one, showing a language nesting bug which is
already in 2.1. It is inverted for now, but this will hopefully change soon.
and introduce sublabels
Sublabels section in *.Tests starts with 'Sublabel: name'
and is valid until start of a new sublabel.
'name' contains only ascii characters [a-z]+
To test all export use 'ctest -L export'. This is unchanged.
The following lists directories and assigned sub-label
lib/doc: manuals
lib/templates: templates
lib/examples: examples
development/mathmacro: mathmacros
autotests: autotests
Now
'ctest -L export' should be without errors
If there _are_ errors, the appropriate test should go to nonstandardTests
'ctest -L reverted' should be without errors
If there _are_ errors, the appropriate test should go to nonstandardTests
'ctest -L nonstandard'
Tests here may, or may not fail. Depends of installed tex extensions
All comments in revertedTests comes from Günter Milde
Non standard test is new. It should collect all tests
which may not work because of some missing non-standard
tex package or some exotic system font.
All dvi_texF and pdf5_texF testcases are nov candidates for being suspended.
After commit 279d084, now export/.*/Math_(dvi3|pdf5)_texF fails,
therefore the tests are also inverted.
This file holds regular expression to select which of the inverted
export test cases will be suspended.
These tests will not be executed with the call 'ctest -L export'.
These exports correctly fail now that we've switched to polyglossia.
Although they compiled without error with babel, the resulting PDFs
had gibberish.
I believe these tests were fixed by TeX Live updates.
Comparing the log files for a system where the tests fail with a
system where the tests pass, below are some of the differences
between the "good" and "bad" logs:
bad:
LaTeX2e <2015/01/01>
Babel <3.9l>
Package: fontspec 2015/03/14 v2.4c
Package: expl3 2015/03/01 v5547 L3 programming layer
good:
LaTeX2e <2015/01/01> patch level 2
Babel <3.9m>
Package: fontspec 2015/07/22 v2.4d
Package: expl3 2015/07/30 v5724 L3 programming layer
Many of our documents have babel-specific preamble code. By putting
this code in a \@ifpackageloaded{babel}{}{} conditional, XeTeX and
LuaTeX compilation with polyglossia now works. This fixes some
LuaTeX tests that were broken by edd37de8 and also allows us to
uninvert some XeTeX tests.
Note that in some of the files although the preambles were fixed to
allow for polyglossia, they still do not compile without errors:
es/Math.lyx
es/Customization.lyx
de/Customization.lyx
Similar fixes might be desired in other manuals but these at least
fix regressions in the tests.
PolyglossiaOpts are case-sensitive so "latin" must be changed to
"Latin". Without this change, compiling examples/sr/Braille.lyx
with LuaTeX and system fonts gives the following error:
Package Polyglossia Error: Unknown script `latin' for Serbian
language
Many of our Spanish documents use babel-specific features in the
documents, e.g. to write "sin" in Spanish ("sen"). Because babel
seems to have good support for Spanish, I am setting the "Always
babel" for the manuals.
This fixes several LuaTeX tests with non-TeX fonts. A XeTeX test is
also reverted accordingly.
We do not currently have any Arabic documents, but when we do, these
fonts will be used for the tests that use system fonts.
They can be installed on Debian-type systems from the
'fonts-sil-scheherazade' package.
The bug that caused these tests to enter into infinite loops
(and have a memory leak) has been fixed. These tests now
pass on an updated TeX Live 2014 as well as the current TeX
Live 2015 pretest.
One of the tests is also disabled for es/Math.lyx. However,
the other test passes for es/Math.lyx. To reproduce,
open fr/Math.lyx, click "Use non-TeX fonts", and choose
e.g. "FreeSans" for the three fonts. Then view as PDF (LuaTeX).
I put a note to look into why this one fails.
According to fix#9432, Scott found a new crash
1. start a new LyX document, english GUI
2. alt+m f to create a fraction
3. alt+m r to insert a root
4. undo
This should prevent possible errors for tests running in parallel.
We now also check for created export file.
complainer/feature requester: Scott Kostyshak
If the LyX window does not appear after a certain (conservative)
time, the keytest now fails. One reason that caused the LyX window
to never show up was when DISPLAY was not set. This particular case
has been fixed in <commit1hash>. This commit is meant for the more general
scenario.
(Collaboration with Scott Kostyshak)
