lyx_mirror/development/autotests
Scott Kostyshak 7ea9210853 Uninvert Japanese tests after fix to #8823
The export ja/lilypond_pdf fails because ps2pdf gives an error. It
is thus still inverted, under the category 'externalissues'. As
Jürgen discovered, ps2pdf succeeds if the -dNOSAFER flag is used.

Note that Kornel is seeing strange behavior with the sweave test,
and thus the label of that test might be changed soon (e.g. to
"unreliable"). For discussion, see:

https://www.mail-archive.com/search?l=mid&q=20171001032524.fr5xfngylththwv2%40steph
(cherry picked from commit 0c7e5c115d)
2017-10-07 22:40:05 -04:00
..
xvkbd Cmake build key tests: Omit the tests and creation of xvkbd command, 2013-07-13 22:44:56 +02:00
beginEndStructureCheck.pl Cmake export tests: Add structue test for exported lyx files 2015-12-27 14:33:11 +01:00
bug-7673-in.txt keytests: rewritten testcases to use the new searching engine searchPatterns.pl 2017-05-24 16:29:32 +02:00
bug-7673.lyx Add an autotest for #7673 2013-01-29 16:18:08 -05:00
bug-7673b-in.txt keytests: rewritten testcases to use the new searching engine searchPatterns.pl 2017-05-24 16:29:32 +02:00
bug-8370-in.txt keytests: rewritten testcases to use the new searching engine searchPatterns.pl 2017-05-24 16:29:32 +02:00
bug-8370.lyx Add test for #8370 (not yet fixed) 2013-04-04 06:31:00 -04:00
bug-8482-in.txt keytests: Add a small delay to each special key 2017-05-27 11:18:58 +02:00
bug-8540-in.txt keytests: rewritten testcases to use the new searching engine searchPatterns.pl 2017-05-24 16:29:32 +02:00
bug-8684-in.txt keytests: rewritten testcases to use the new searching engine searchPatterns.pl 2017-05-24 16:29:32 +02:00
bug-8684.lyx Fix EOLs 2013-10-06 05:57:53 +02:00
bug-10636-in.txt keytests: Do not save the session info in bug-10636-test 2017-05-29 13:53:39 +02:00
bug-10636.lyx Add test for #10636 (fixed in master) 2017-05-27 19:54:59 +02:00
bug-export-latex-in.txt keytests: rewritten testcases to use the new searching engine searchPatterns.pl 2017-05-24 16:29:32 +02:00
bug-math-undo-in.txt keytests: rewritten testcases to use the new searching engine searchPatterns.pl 2017-05-24 16:29:32 +02:00
check_load.cmake Cmake check load tests: Allow tests to use the filter .*Tests 2015-12-27 18:56:36 +01:00
CheckLoadErrors.cmake Cmake export tests: Handle some more warnings 2015-12-25 20:03:35 +01:00
CMakeLists.txt Remove xvkbd from cmake build. 2017-04-28 11:42:50 +02:00
examineXmllintOutput.pl Cmake xhtml export tests: Missing script for commit 4162424762 2016-05-30 10:22:59 +02:00
export-in.sh Add templates to export tests (CMake & autotools) 2013-02-21 08:25:13 -05:00
export.cmake cmake tests: Try to use valid locales for testing 2017-10-03 22:09:06 +02:00
ExportTests.cmake ctests: test unicodesymbols also with utf8. 2017-04-12 12:42:26 +02:00
filterCheckWarnings Cmake export tests: Handle some more warnings 2015-12-25 20:03:35 +01:00
findadv-01-in.txt keytests: rewritten testcases to use the new searching engine searchPatterns.pl 2017-05-24 16:29:32 +02:00
findadv-02-in.txt keytests: rewritten testcases to use the new searching engine searchPatterns.pl 2017-05-24 16:29:32 +02:00
findadv-03-in.txt keytests: rewritten testcases to use the new searching engine searchPatterns.pl 2017-05-24 16:29:32 +02:00
findadv-04-in.txt keytests: rewritten testcases to use the new searching engine searchPatterns.pl 2017-05-24 16:29:32 +02:00
findadv-05-in.txt keytests: rewritten testcases to use the new searching engine searchPatterns.pl 2017-05-24 16:29:32 +02:00
findadv-06-in.txt keytests: rewritten testcases to use the new searching engine searchPatterns.pl 2017-05-24 16:29:32 +02:00
findadv-07-in.txt keytests: rewritten testcases to use the new searching engine searchPatterns.pl 2017-05-24 16:29:32 +02:00
findadv-08-in.txt keytests: Add a small delay to each special key 2017-05-27 11:18:58 +02:00
findadv-09-in.txt keytests: rewritten testcases to use the new searching engine searchPatterns.pl 2017-05-24 16:29:32 +02:00
findadv-10-in.txt keytests: rewritten testcases to use the new searching engine searchPatterns.pl 2017-05-24 16:29:32 +02:00
findadv-11-in.txt keytests: Add a small delay to each special key 2017-05-27 11:18:58 +02:00
findadv-12-in.txt keytests: rewritten testcases to use the new searching engine searchPatterns.pl 2017-05-24 16:29:32 +02:00
findadv-13-in.txt keytests: rewritten testcases to use the new searching engine searchPatterns.pl 2017-05-24 16:29:32 +02:00
findadv-14-in.txt keytests: rewritten testcases to use the new searching engine searchPatterns.pl 2017-05-24 16:29:32 +02:00
findadv-15-in.txt keytests: rewritten testcases to use the new searching engine searchPatterns.pl 2017-05-24 16:29:32 +02:00
findadv-16-in.txt keytests: rewritten testcases to use the new searching engine searchPatterns.pl 2017-05-24 16:29:32 +02:00
findadv-17-in.txt keytests: Add a small delay to each special key 2017-05-27 11:18:58 +02:00
findadv-18-in.txt keytests: rewritten testcases to use the new searching engine searchPatterns.pl 2017-05-24 16:29:32 +02:00
findadv-19-in.txt keytests: rewritten testcases to use the new searching engine searchPatterns.pl 2017-05-24 16:29:32 +02:00
findadv-20-in.txt keytests: rewritten testcases to use the new searching engine searchPatterns.pl 2017-05-24 16:29:32 +02:00
findadv-21-in.txt keytests: rewritten testcases to use the new searching engine searchPatterns.pl 2017-05-24 16:29:32 +02:00
findadv-combined-in.txt keytests: rewritten testcases to use the new searching engine searchPatterns.pl 2017-05-24 16:29:32 +02:00
findadv-crash-in.txt keytests: rewritten testcases to use the new searching engine searchPatterns.pl 2017-05-24 16:29:32 +02:00
findadv-logo-in.txt keytests: rewritten testcases to use the new searching engine searchPatterns.pl 2017-05-24 16:29:32 +02:00
findadv-re-01-in.txt keytests: rewritten testcases to use the new searching engine searchPatterns.pl 2017-05-24 16:29:32 +02:00
findadv-re-02-in.txt keytests: rewritten testcases to use the new searching engine searchPatterns.pl 2017-05-24 16:29:32 +02:00
findadv-re-03-in.txt keytests: rewritten testcases to use the new searching engine searchPatterns.pl 2017-05-24 16:29:32 +02:00
findadv-re-04-in.txt keytests: rewritten testcases to use the new searching engine searchPatterns.pl 2017-05-24 16:29:32 +02:00
findadv-re-05-in.txt keytests: rewritten testcases to use the new searching engine searchPatterns.pl 2017-05-24 16:29:32 +02:00
findadv-re-06-in.txt keytests: rewritten testcases to use the new searching engine searchPatterns.pl 2017-05-24 16:29:32 +02:00
first-time-in.txt keytests: Added debug 'key' to be able to analyse test results. 2017-05-09 21:08:52 +02:00
getTempDir.pl Cmake tests: make parallel export tests safer. 2015-03-28 21:21:16 +01:00
hello-world-in.txt keytests: amend 8f74cac381 2017-05-19 12:56:43 +02:00
ignoredTests Cmake export tests: Ignore some acmart tests for now 2017-06-01 11:07:22 +02:00
ignoreLatexErrorsTests Cmake export tests: Ignore missing glyphs also for doc/(fr|es)/UserGuide 2017-08-12 14:08:12 +02:00
invertedTests Uninvert Japanese tests after fix to #8823 2017-10-07 22:40:05 -04:00
keytest.py keytests: Allow for checking multiple log files 2017-08-04 11:56:19 +02:00
lyx2lyxtest.cmake Cmake lyx2lyx tests: Ignore warning about "no conversion needed" 2015-11-29 15:26:56 +01:00
lyxStatus.pm Cmake export tests: Adapt to minted listings 2017-06-16 11:39:26 +02:00
Makefile
README Added the possibility to build arbitrary scripts (*-in.sh) as tests. They are all launched 2011-07-03 17:00:18 +00:00
run-tests.sh findadv: almost all test cases, including \regex[{}, back to working with std::regex. 2017-04-17 00:15:26 +02:00
searchPatterns.pl keytests: rewritten testcases to use the new searching engine searchPatterns.pl 2017-05-24 16:29:32 +02:00
single-test.cmake keytests: New script to replace 'pcregrep' in more complex cases. 2017-05-14 18:58:55 +02:00
single-test.sh keytests: New script to replace 'pcregrep' in more complex cases. 2017-05-14 18:58:55 +02:00
stop_autotests.tcl Recovered working autotests for findadv on Linux/Ubuntu (after a long no-use time). 2017-04-15 23:38:41 +02:00
stop.gif
suspendedTests ctests: Test dedicated LaTeX test samples with LaTeX export only. 2016-09-15 17:18:47 +02:00
tabular-footnote-in.txt keytests: rewritten testcases to use the new searching engine searchPatterns.pl 2017-05-24 16:29:32 +02:00
unreliableTests ctests: adapt patterns to file move 2017-05-14 04:24:17 -04:00
useSystemFonts.pl Cmake export tests: Check exports also for different possible input encodigs 2016-11-10 00:56:49 +01:00

Automated tests based on the "MonKey Testing" keytest program.
======================================================================

WARNING!!
----------------------------------------------------------------------
Running the tests in this folder may cause abrupt kill of existing
LyX instances on your system. In order to avoid any possible data
loss, quit any running instance of LyX in which you're carrying out
real work. Also, although precautions have been taken in order to not
allow the key presses to reach non-LyX windows, this may sometimes
happen anyway. Please, close any running application that is not
strictly needed.


DEPENDENCIES
----------------------------------------------------------------------
The following programs are needed by the testing framework:
- xvkbd: a modified copy of xvkbd is included in this folder, and should
         be built before running the tests
- wmctrl
- pcregrep
- Tcl/Tk (wish8.5)


USAGE
----------------------------------------------------------------------
In order to launch all the tests available in this folder, just run

  make

and do not interact with your computer while the tests are running.

If you wish to launch one or more specific tests, then provide them
as argument to the run-tests.sh script, e.g.:

  ./run-tests.sh <whatever>*-in.txt

For failed tests you get a folder with the logs that help you identify
the problem.

Despite the efforts to avoid it, sometimes a test fails simply because
the key presses provided to the process are someway lost due to slow
hardware, slow window manager, too beautiful and animated desktop,...
When this happens, the tests should be run with a higher delay among
key presses. The default of 100 milliseconds can be changed by setting
the XVKBD_DELAY environment variable.

The standard xvkbd available on your system may be used to run the
tests. However, a custom xvkbd may optionally be used that forbids the
testing framework to type into non-LyX windows and may mitigate the
issue described right above. This option can be activated by defining
the XVKBD_HACKED environment variable to any value.


SYNTAX
----------------------------------------------------------------------
Each test-case script should be named as xxx-in.txt. The syntax of the
script is described in detail in the sample test-case script
hello-world-in.txt.

Alternatively, a test-case can be an arbitrary executable script with
name ending in "-in.sh", that is launched inside a dedicated folder,
and to which the variables LYX_ROOT and LYX_EXE are exported, so that
you can easily reference the root LyX sources folder and the program
executable. The test script needs to complete with a no-error return
value (zero), otherwise a failure is reported (see export-in.sh for
an example).

Whenever LyX is launched, in both cases, the LYX_USERDIR variable is
set to autotests/out-home, in order to rely on clean preferences
settings (use make clean or make without arguments in order to let
that folder be created again).


TODO
----------------------------------------------------------------------
Speed-up the execution of the tests as much as possible (i.e., get rid
of heuristic waits when present and replace them with exact check of
the condition we're waiting for).

Allow for tests with multiple LyX instances, e.g., for copy&paste
across LyX instances, or multiple applications, i.e., for testing
the interaction between LyX and other applications.

The modified xvkbd forces the focus on the specified target window
(for safety reasons).  Allow for a temporary disable of this within
the scripts, whenever we need to interact with a pop-up dialog.