lyx_mirror/development/autotests
Scott Kostyshak a175caf256 ctests: uninvert 2 LuaTeX tests fixed on TL21
These exports now pass, and a quick check of the output looks good.

These might have been fixed (although it is just a guess) due to the
following entry in the changelog of luaotfload v3.19:

  A new algorithm for selecting fonts based on font family names
  allows to more reliably load fonts based on their family name.

This commit reverts 2a665dcc.
2022-03-06 09:20:12 -05:00
..
xvkbd
beginEndStructureCheck.pl
bug-7673-in.txt
bug-7673.lyx
bug-7673b-in.txt
bug-8370-in.txt
bug-8370.lyx
bug-8482-in.txt
bug-8540-in.txt
bug-8684-in.txt
bug-8684.lyx
bug-10636-in.txt Modify 2 tests to reflect actual GUI 2020-05-30 11:14:26 +02:00
bug-10636.lyx
bug-export-latex-in.txt Modify 2 tests to reflect actual GUI 2020-05-30 11:14:26 +02:00
bug-math-undo-in.txt
check_load.cmake
CheckLoadErrors.cmake
CMakeLists.txt Cmake tests: Save the number of ignored tests in file for later review 2020-11-27 00:51:36 +01:00
ctests-costs-benefits.txt ctests: log fixed upstream Polyglossia issues 2022-02-16 22:07:05 -05:00
examineXmllintOutput.pl Cmake export xhtml tests: load dtd if checking ouput with xmllint 2020-09-20 11:47:04 +02:00
export-in.sh
export.cmake Revert "Revert "Cmake export tests: Add epub-format to be tested"" 2021-02-17 22:41:40 -05:00
ExportTests.cmake Revert "Revert "Cmake export tests: Add epub-format to be tested"" 2021-02-17 22:41:40 -05:00
filterCheckWarnings Some more tests which are not failin anymore 2019-03-25 19:32:59 +01:00
filterXml4Sax.pl Fix filterXml4Sax.pl to work under Windows 2020-10-13 21:38:38 +02:00
findadv-01-in.txt
findadv-02-in.txt
findadv-03-in.txt
findadv-04-in.txt
findadv-05-in.txt Make some findadv-tests working again, more to follow 2021-01-15 17:13:07 +01:00
findadv-06-in.txt Make some findadv-tests working again, more to follow 2021-01-15 17:13:07 +01:00
findadv-07-in.txt Cmake tests: findadv-01 - findadv-21 working 2021-01-17 14:30:34 +01:00
findadv-08-in.txt
findadv-09-in.txt Cmake key-tests: These are very touchy in respect to timings 2021-01-29 16:11:16 +01:00
findadv-10-in.txt FindAdv: Correct some testcases 2020-05-29 14:22:34 +02:00
findadv-11-in.txt Cmake tests: findadv-01 - findadv-21 working 2021-01-17 14:30:34 +01:00
findadv-12-in.txt
findadv-13-in.txt
findadv-14-in.txt FindAdv: Correct some testcases 2020-05-29 14:22:34 +02:00
findadv-15-in.txt FindAdv: Correct some testcases 2020-05-29 14:22:34 +02:00
findadv-16-in.txt FindAdv: Correct next test (keytest/findadv-16) 2020-05-29 20:04:57 +02:00
findadv-17-in.txt FindAdv: next 3 tests working 2020-05-29 20:18:45 +02:00
findadv-18-in.txt Cmake key-tests: These are very touchy in respect to timings 2021-01-29 16:11:16 +01:00
findadv-19-in.txt Cmake key-tests: These are very touchy in respect to timings 2021-01-29 16:11:16 +01:00
findadv-20-in.txt Cmake key-tests: These are very touchy in respect to timings 2021-01-29 16:11:16 +01:00
findadv-21-in.txt Cmake key-tests: These are very touchy in respect to timings 2021-01-29 16:11:16 +01:00
findadv-22-in.txt FindAdv: next 3 tests working 2020-05-29 20:18:45 +02:00
findadv-combined-in.txt
findadv-crash-in.txt
findadv-logo-in.txt FindAdv: Remaining tests working now again 2020-05-29 20:56:38 +02:00
findadv-re-01-in.txt Cmake key-tests: These are very touchy in respect to timings 2021-01-29 16:11:16 +01:00
findadv-re-02-in.txt Cmake key-tests: These are very touchy in respect to timings 2021-01-29 16:11:16 +01:00
findadv-re-03-in.txt Cmake key-tests: These are very touchy in respect to timings 2021-01-29 16:11:16 +01:00
findadv-re-04-in.txt Cmake key-tests: These are very touchy in respect to timings 2021-01-29 16:11:16 +01:00
findadv-re-05-in.txt Cmake key-tests: These are very touchy in respect to timings 2021-01-29 16:11:16 +01:00
findadv-re-06-in.txt Cmake key-tests: These are very touchy in respect to timings 2021-01-29 16:11:16 +01:00
first-time-in.txt
getTempDir.pl
hello-world-in.txt
ignoredTests Amend c21cd7dc: Ignore epub tests for now 2021-02-18 09:34:43 +01:00
ignoreLatexErrorsTests Revert "Ignore missing glyphs for fr/Customization.lyx" 2020-08-29 15:20:49 +02:00
invertedTests ctests: uninvert 2 LuaTeX tests fixed on TL21 2022-03-06 09:20:12 -05:00
keytest.py
lyx2lyxtest.cmake
lyxStatus.pm Cmake docbook5 tests: Use absolute path-hints for 'bibfiles' entries in lyx-file 2020-11-29 08:48:16 +01:00
Makefile
README
run-tests.sh
searchPatterns.pl
single-test.cmake
single-test.sh
stop_autotests.tcl
stop.gif
suspendedTests DocBook: deploy fix for acknowledgements. 2020-11-27 02:12:29 +01:00
tabular-footnote-in.txt
unreliableTests ctests: some compilation errors fixed with last TeXLive update. 2020-11-22 22:16:57 +01:00
useSystemFonts.pl Revert "Revert "Cmake export tests: Add epub-format to be tested"" 2021-02-17 22:41:40 -05:00
xmlParser.pl Cmake docbook tests: Check for correct use of namespaces 2020-08-22 17:40:34 +02:00

Automated tests based on the "MonKey Testing" keytest program.
======================================================================

WARNING!!
----------------------------------------------------------------------
Running the tests in this folder may cause abrupt kill of existing
LyX instances on your system. In order to avoid any possible data
loss, quit any running instance of LyX in which you're carrying out
real work. Also, although precautions have been taken in order to not
allow the key presses to reach non-LyX windows, this may sometimes
happen anyway. Please, close any running application that is not
strictly needed.


DEPENDENCIES
----------------------------------------------------------------------
The following programs are needed by the testing framework:
- xvkbd: a modified copy of xvkbd is included in this folder, and should
         be built before running the tests
- wmctrl
- pcregrep
- Tcl/Tk (wish8.5)


USAGE
----------------------------------------------------------------------
In order to launch all the tests available in this folder, just run

  make

and do not interact with your computer while the tests are running.

If you wish to launch one or more specific tests, then provide them
as argument to the run-tests.sh script, e.g.:

  ./run-tests.sh <whatever>*-in.txt

For failed tests you get a folder with the logs that help you identify
the problem.

Despite the efforts to avoid it, sometimes a test fails simply because
the key presses provided to the process are someway lost due to slow
hardware, slow window manager, too beautiful and animated desktop,...
When this happens, the tests should be run with a higher delay among
key presses. The default of 100 milliseconds can be changed by setting
the XVKBD_DELAY environment variable.

The standard xvkbd available on your system may be used to run the
tests. However, a custom xvkbd may optionally be used that forbids the
testing framework to type into non-LyX windows and may mitigate the
issue described right above. This option can be activated by defining
the XVKBD_HACKED environment variable to any value.


SYNTAX
----------------------------------------------------------------------
Each test-case script should be named as xxx-in.txt. The syntax of the
script is described in detail in the sample test-case script
hello-world-in.txt.

Alternatively, a test-case can be an arbitrary executable script with
name ending in "-in.sh", that is launched inside a dedicated folder,
and to which the variables LYX_ROOT and LYX_EXE are exported, so that
you can easily reference the root LyX sources folder and the program
executable. The test script needs to complete with a no-error return
value (zero), otherwise a failure is reported (see export-in.sh for
an example).

Whenever LyX is launched, in both cases, the LYX_USERDIR variable is
set to autotests/out-home, in order to rely on clean preferences
settings (use make clean or make without arguments in order to let
that folder be created again).


TODO
----------------------------------------------------------------------
Speed-up the execution of the tests as much as possible (i.e., get rid
of heuristic waits when present and replace them with exact check of
the condition we're waiting for).

Allow for tests with multiple LyX instances, e.g., for copy&paste
across LyX instances, or multiple applications, i.e., for testing
the interaction between LyX and other applications.

The modified xvkbd forces the focus on the specified target window
(for safety reasons).  Allow for a temporary disable of this within
the scripts, whenever we need to interact with a pop-up dialog.