The tests are now more robust if ctest uses '-j' (number of threads) parameter.
a.) keytests are running in sequence and in one thread only, no other thread running
b.) tex2lyx tests are locking in respect to each other
c.) Test gets one or more labels, so that we can select
ctest -L url
to run tests labeled url
d.) New macro settestlabel() to add labels to a test
Files, which contain names of ctest-tests controling
whether they should be inverted, or ignored.
We have two such files ATM: revertedTests + ignoredTests
These tests load the .lyx files to see if there are
any parsing errors or other LyX errors/warnings. The
tests fail if LyX writes a line to STDERR that does not
match any of the regexes stored in filterCheckWarnings.
Thanks to Kornel for the CMake expertise and corrections.
This patch was written jointly with him.
a) test export/templates/ectaart_pdf5 removed, because
of incompatibility between ectart.cls and luatex.sty
b) renamed some tests to reflect their purpose
tex2lyx2lyx/roundtrip/ => tex2lyx/cmplyx/
# the tests are comparing files written by lyx (not by tex2lyx)
autotests/ => keytest/
# we are testing reactions of lyx to some key strokes
c) add tests for lyx2lyx. This is rather rudimental, in that it only
checks for created output, error codes and error output.
d) use correct environment variable to set up the userdir for testing
Some export test are known to fail. Checking them every time is cumbersome.
Now, the tests marked in file revertedTests will invert the success condition
and mark the test as INVERTED_SEE-README.ctest.
If some of the tests in revertedTests fails, this is good, and the testname
can be removed from it.
If no libraries are found, XVFBDLIBS is not initialized and the command "list(REMOVE_DUPLICATES XVFBDLIBS)" fails. So, initialize XVFBDLIBS beforehand.
The xhtml format is always tested, regardless of the default format.
Without this, if the default format is xhtml, CMake gives an error
when trying to add the xhtml test because it was already added.
Now all export tests can be run with:
ctest -R "export"
Only export tests for manuals can be run with:
ctest -R "export/doc"
And only export tests for examples can be run with:
ctest -R "export/examples"
Instead of 'autotests/export/doc/' export tests are prefixed
with 'export/doc/'. This has the advantage of the name better
reflecting the test (export tests are not autotests), but has a
disadvantage in that the old name gave a better idea of where the
export test scripts are located (in the autotests directory).
keytest.py now only prints one time the command that it is executing for
an assert. The command is already printed in the function responsible
for executing it, so it is not necessary to print the command before
calling that function.
This test should fail because #7673 is not yet fixed in trunk.
Note that it is fixed in branch.
This is the first autotest that uses an existing .lyx file,
bug-7673.lyx.
If the test opens an existing .lyx file and crashes, it will leave
around a .emergency file. If a test with that same name is run again,
LyX will try to recover the .emergency file, which could throw
off the test.
This is implemented for both CMake and autotools.
To reproduce some bugs, tests need to open an existing LyX file. This
is now possible with CMake and is consistent with testing under
autotools in the sense that the .lyx file is referred to in the test
by being in the parent directory.
If the test is named bug-XXXX-in.txt and there exists a file
bug-XXXX.lyx, that file will be copied to the parent directory of
out-home in the build directory.
