lyx_mirror/development/autotests
Kornel Benko fc33092fd8 Cmake build: modify some tests
a) test export/templates/ectaart_pdf5 removed, because
    of incompatibility between ectart.cls and luatex.sty
b) renamed some tests to reflect their purpose
    tex2lyx2lyx/roundtrip/ => tex2lyx/cmplyx/
          # the tests are comparing files written by lyx (not by tex2lyx)
    autotests/ => keytest/
          # we are testing reactions of lyx to some key strokes
c) add tests for lyx2lyx. This is rather rudimental, in that it only
    checks for created output, error codes and error output.
d) use correct environment variable to set up the userdir for testing
2013-08-21 13:34:26 +02:00
..
xvkbd Cmake build key tests: Omit the tests and creation of xvkbd command, 2013-07-13 22:44:56 +02:00
bug-7673-in.txt Add an autotest for #7673 2013-01-29 16:18:08 -05:00
bug-7673.lyx Add an autotest for #7673 2013-01-29 16:18:08 -05:00
bug-7673b-in.txt Add another test for #7673 (currently fails) 2013-04-04 06:58:42 -04:00
bug-8370-in.txt Add test for #8370 (not yet fixed) 2013-04-04 06:31:00 -04:00
bug-8370.lyx Add test for #8370 (not yet fixed) 2013-04-04 06:31:00 -04:00
bug-8482-in.txt Add an autotest for a trunk regression (#8482) 2013-01-05 04:44:14 -05:00
bug-8540-in.txt Add an autotest for #8540 2013-02-07 03:42:14 -05:00
bug-8684-in.txt Cmake tests: use multiple log files for multiple test-cases 2013-05-21 19:54:20 +02:00
bug-8684.lyx Add autotests for #8684 2013-05-20 18:52:16 -04:00
CMakeLists.txt Cmake build: modify some tests 2013-08-21 13:34:26 +02:00
export-in.sh Add templates to export tests (CMake & autotools) 2013-02-21 08:25:13 -05:00
export.cmake Cmake build: modify some tests 2013-08-21 13:34:26 +02:00
findadv-01-in.txt Saving a doc is faster and more reliable by Ctrl-s, rather than activating a menu (thx Pavel). 2011-02-03 09:13:19 +00:00
findadv-02-in.txt More regular expression testing. 2011-02-06 22:48:54 +00:00
findadv-03-in.txt fix findadv-03-in.txt 2011-02-03 00:01:53 +00:00
findadv-04-in.txt Saving a doc is faster and more reliable by Ctrl-s, rather than activating a menu (thx Pavel). 2011-02-03 09:13:19 +00:00
findadv-05-in.txt More regular expression testing. 2011-02-06 22:48:54 +00:00
findadv-06-in.txt More regular expression testing. 2011-02-06 22:48:54 +00:00
findadv-07-in.txt More regular expression testing. 2011-02-06 22:48:54 +00:00
findadv-08-in.txt Cmake build autotests: 2012-12-15 12:58:31 +01:00
findadv-09-in.txt Replaced the Ctrl+Shft+r sequences for entering regexp-mode with [Alt+x]regexp-mode[Ret], 2011-02-14 20:41:46 +00:00
findadv-10-in.txt Added test-case for finding (properly) LyX & LaTeX (special phrases exported 2011-02-12 11:06:09 +00:00
findadv-11-in.txt Revert wrong previous commit. 2011-03-19 10:34:59 +00:00
findadv-12-in.txt Regression test for #7442. 2011-04-14 22:21:18 +00:00
findadv-13-in.txt Bugfix in how the search buffer was exported for the Advanced Find & Replace. 2011-05-26 17:08:48 +00:00
findadv-14-in.txt Fixed Advanced F&R issue in matching things at environment borders with ignore-format off. 2011-05-27 20:39:58 +00:00
findadv-15-in.txt Fixed Advanced F&R issue in matching things at environment borders with ignore-format off. 2011-05-27 20:39:58 +00:00
findadv-16-in.txt Fixed bug in matching at borders within *-environments with ignore-format off. 2011-05-27 21:14:36 +00:00
findadv-17-in.txt Avoid extra ' ' in text-only export of consecutive macros in math mode, for Advanced F&R matching, with ignore-format off. 2011-05-29 14:08:29 +00:00
findadv-18-in.txt Updated UserGuide after removal of macro insertion while in regexp mode. 2011-06-12 01:25:26 +00:00
findadv-19-in.txt Added test-case for regular expressions using "\" for backwards references (e.g., \1). 2011-06-12 01:23:07 +00:00
findadv-20-in.txt Added to the test-case the case that was actually failing (replacing with multi-cells inside math). 2011-07-12 15:55:32 +00:00
findadv-re-01-in.txt Now Adv. F&R correctly interprets escape sequences within regexp insets, e.g., \\ or \(, for finding characters with a special meaning in regular expression context (\^()[]). 2011-06-19 16:41:23 +00:00
findadv-re-02-in.txt Now Adv. F&R correctly interprets escape sequences within regexp insets, e.g., \\ or \(, for finding characters with a special meaning in regular expression context (\^()[]). 2011-06-19 16:41:23 +00:00
findadv-re-03-in.txt Fixed longstanding bug in Advanced Find&Replace, when dealing with documents containing braces. 2011-08-25 19:16:14 +00:00
findadv-re-04-in.txt Fixed bug in escape sequence, showing up with ignore format off (addressing #8380). 2012-10-22 01:05:29 +01:00
findadv-re-05-in.txt Added regression test for simple use-case documented in Advanced F&R manual. 2013-04-29 00:48:43 +01:00
findadv-re-06-in.txt Added regression test for simple use-case documented in Advanced F&R manual. 2013-04-29 00:48:43 +01:00
first-time-in.txt Added the creation of a custom .lyx folder inside pwd/home/.lyx at 2011-02-01 22:08:07 +00:00
hello-world-in.txt Add description of Sleep (== pause) command in keutests 2013-01-24 08:48:18 +01:00
ignoredTests Cmake build: modify some tests 2013-08-21 13:34:26 +02:00
keytest.py I seem to need this to let the 'Lang' keyword work, within *-in.txt files. 2013-07-23 02:10:48 +01:00
lyx2lyxtest.cmake Cmake build: modify some tests 2013-08-21 13:34:26 +02:00
Makefile Added version suffix to symlinks. Tested with --enable-version-suffix. 2011-02-11 23:27:34 +00:00
README Added the possibility to build arbitrary scripts (*-in.sh) as tests. They are all launched 2011-07-03 17:00:18 +00:00
revertedTests The cmake-test 'export/examples/sterrors_pdf' is now expected to pass 2013-08-16 18:23:46 +02:00
run-tests.sh Allow for keeping succesful tests output directories if the V env variable is non-null. 2012-12-15 12:07:31 +00:00
single-test.cmake Cmake build: modify some tests 2013-08-21 13:34:26 +02:00
single-test.sh autotests: clean up .emergency file of input .lyx 2013-01-29 16:18:08 -05:00
stop_autotests.tcl Now the framework works running LyX in the language required by the test 2011-02-10 22:33:23 +00:00
stop.gif Added emergency STOP button in tcl/tk window for immediate & painless termination of the tests. 2011-02-07 23:32:27 +00:00
tabular-footnote-in.txt Shave 3 seconds off an autotest 2013-01-05 04:39:49 -05:00

Automated tests based on the "MonKey Testing" keytest program.
======================================================================

WARNING!!
----------------------------------------------------------------------
Running the tests in this folder may cause abrupt kill of existing
LyX instances on your system. In order to avoid any possible data
loss, quit any running instance of LyX in which you're carrying out
real work. Also, although precautions have been taken in order to not
allow the key presses to reach non-LyX windows, this may sometimes
happen anyway. Please, close any running application that is not
strictly needed.


DEPENDENCIES
----------------------------------------------------------------------
The following programs are needed by the testing framework:
- xvkbd: a modified copy of xvkbd is included in this folder, and should
         be built before running the tests
- wmctrl
- pcregrep
- Tcl/Tk (wish8.5)


USAGE
----------------------------------------------------------------------
In order to launch all the tests available in this folder, just run

  make

and do not interact with your computer while the tests are running.

If you wish to launch one or more specific tests, then provide them
as argument to the run-tests.sh script, e.g.:

  ./run-tests.sh <whatever>*-in.txt

For failed tests you get a folder with the logs that help you identify
the problem.

Despite the efforts to avoid it, sometimes a test fails simply because
the key presses provided to the process are someway lost due to slow
hardware, slow window manager, too beautiful and animated desktop,...
When this happens, the tests should be run with a higher delay among
key presses. The default of 100 milliseconds can be changed by setting
the XVKBD_DELAY environment variable.

The standard xvkbd available on your system may be used to run the
tests. However, a custom xvkbd may optionally be used that forbids the
testing framework to type into non-LyX windows and may mitigate the
issue described right above. This option can be activated by defining
the XVKBD_HACKED environment variable to any value.


SYNTAX
----------------------------------------------------------------------
Each test-case script should be named as xxx-in.txt. The syntax of the
script is described in detail in the sample test-case script
hello-world-in.txt.

Alternatively, a test-case can be an arbitrary executable script with
name ending in "-in.sh", that is launched inside a dedicated folder,
and to which the variables LYX_ROOT and LYX_EXE are exported, so that
you can easily reference the root LyX sources folder and the program
executable. The test script needs to complete with a no-error return
value (zero), otherwise a failure is reported (see export-in.sh for
an example).

Whenever LyX is launched, in both cases, the LYX_USERDIR variable is
set to autotests/out-home, in order to rely on clean preferences
settings (use make clean or make without arguments in order to let
that folder be created again).


TODO
----------------------------------------------------------------------
Speed-up the execution of the tests as much as possible (i.e., get rid
of heuristic waits when present and replace them with exact check of
the condition we're waiting for).

Allow for tests with multiple LyX instances, e.g., for copy&paste
across LyX instances, or multiple applications, i.e., for testing
the interaction between LyX and other applications.

The modified xvkbd forces the focus on the specified target window
(for safety reasons).  Allow for a temporary disable of this within
the scripts, whenever we need to interact with a pop-up dialog.