Remove patterns matching tests now ignored (CJK/Korean, latex8).
Move patterns matching documents in the attic under new sublabel.
Add more verbose comments.
Comment by Günter Milde:
Actually, *all* Spanish manuals either fail or have wrong output with
Unicode TeX engines and 8-bit fonts. The reason is known: a bug in Babel
that uses utf8 strings whenever Xe- or LuaTeX is detected.
Always-Babel now set in the example document.
Adapted the autotest categorization:
* fails for some developers (why?)
* wrong output with pdflatex/LuaTeX and DVI (missing landscape slides).
he.po in branch contains a number of translations where the translation is
identical with the original text (e.g. for encoding names) which look suspicios
to me. Therefore I did not want to merge these and leave the decision to the
translator. No information is lost, since these "translations" can easily be
recreated from the original texts by copy-paste.
Previously all labels got the depth '7' while processing 'suspiciousTestss'.
The depth is used to sort how our labels are used to build a test-label.
Say a test gets label a, b, and c, with depth 3, 8 and 2.
The constructed test-label will be "b🅰️c"
For new (to be implemented) 'reason-labels' it is more convenient to assign them higher values.
suspiciousTests: Remove Tutorial from the regex
suspendedTests: Explicit list of languages needed
ignoredTests: Language-string is separated from other strings only by '/'. '|', '_', '(' and ')'
This is useful if you know that the translation of a particular language has
not been updated at all in master, but the stable version has been worked on.
Some characters (like ") are escaped in .po files. These are handled correctly
by polib, but the self written parser did not yet take them into account.
We need to remove the #~ prefix of each line, since it is re-added after
wrapping, and because otherwise the comparison with entries from polib
does not work.
Now it is possible to specify the target directory, and the command line syntax
follows the standard rules: It uses options for optional arguments, and one
positional argument for the single required argument.
LyXMacros.cmake: Overseen the macro 'setmarkedtestlabel()' which added it automatically
if the test was to invert the test result.
ExportTests.cmake: Correct label handling
Surprisingly I could not find a tool that merges updated translations into a
.po file in a way that
a) merges only missing translations (does not overwrite existing ones)
b) produces a minimal diff so that there is a chance to manually check the result
Therefore I wrote my own. You can use it for a single language
python development/tools/mergepo.py ../lyx-2.1/po fr
or for all languages:
python development/tools/mergepo.py ../lyx-2.1/po
The python code is not the most elegant one, but it works. What is missing is
a command line switch to merge changed translations as well. This is useful
for languages that have not yet received any translation update in the
development branch, only in the stable branch: In this case we know that
translations that are not identical in both branches should be overtaken from
the stabkle one.
This works around a limitation of the test machinery, which never switches
TeX fonts on for format that need that, it only switches TeX fonts off for
formats needing it.
Thanks to Kornel we do now have the infrastructure for running dedicated
export tests. This is the first one, showing a language nesting bug which is
already in 2.1. It is inverted for now, but this will hopefully change soon.
- also remove support metafile2eps because:
- it is outdated
- unnecessary third-party program since ImageMagick can do this too
- needs unnecessarily installation time
and introduce sublabels
Sublabels section in *.Tests starts with 'Sublabel: name'
and is valid until start of a new sublabel.
'name' contains only ascii characters [a-z]+
The files will be locatet in the build-directory "Testing/Temporary" as
LastFailedAccessibleURLS.log
LastFailedInaccessibleURLS.log
LastFailedKnownInvalidURLS.log
To test all export use 'ctest -L export'. This is unchanged.
The following lists directories and assigned sub-label
lib/doc: manuals
lib/templates: templates
lib/examples: examples
development/mathmacro: mathmacros
autotests: autotests
Now
'ctest -L export' should be without errors
If there _are_ errors, the appropriate test should go to nonstandardTests
'ctest -L reverted' should be without errors
If there _are_ errors, the appropriate test should go to nonstandardTests
'ctest -L nonstandard'
Tests here may, or may not fail. Depends of installed tex extensions
All comments in revertedTests comes from Günter Milde
Non standard test is new. It should collect all tests
which may not work because of some missing non-standard
tex package or some exotic system font.
texindy requires some more Perl and unfortunately the path tho the perl.exe in Windows' PATh environment variable.
The new EnvVarUpdate.nsh contains a function to modify Windows environment variables easily.
Thanks Scott for the idea to modify the document. This seems to work, but I am
not surer whether it is safe in all cases, so better warn if this is used.
Also MacOSX ReadMe files. Note that a few lib/doc files are
also "updated" because trailing spaces are removed, but their file
formats are the same because they were recently updated at 83672113.
I did "git checkout LFUNs.lyx" because this file is generated
automatically and has a special header.
gen_lfuns.py does now produce the current file format, and and LFUNs.lyx was
re-created with the updated script.
There is one difference if you compare this version of LFUNs.lyx with the old
version updated by lyx2lyx: All occurences of LyX, TeX etc. in the lfun
descriptions are no longer output as logos. I do consider this as a feature,
since the old version did also output the TeX part of BibTeX as a logo, as well
as places where the names were part of some syntax, e.g. lyx::LyXRC::LyXRCTags.
lyx2lyx did not yet know about /systemlyxdir/ and set \origin to the path
where my git tree lives instead. This path is not usable except on my machine,
so better write something more usable instead.
This is a special command line switch of lyx2lyx, so it does not interfere
with normal usage. I did not try to deduce the systemlyxdir from lyx2lyx to
be on the safe side.
This is one part of bug 9744: If you toggle between TeX fonts and non-TeX
fonts, the settings of the other choice are no longer thrown away, but stored
and re-activated if you switch back. Most parts of the patch are purely
mechanical (duplicating some BufferParams members), the only non-mechanical
change is in the GUI logic.
The file was updated to format 14 in fc22ba16 but the format line
was missing. A missing format line is interpreted as format 0, I
believe, which could lead to incorrect conversion to later formats.
All dvi_texF and pdf5_texF testcases are nov candidates for being suspended.
After commit 279d084, now export/.*/Math_(dvi3|pdf5)_texF fails,
therefore the tests are also inverted.
This file holds regular expression to select which of the inverted
export test cases will be suspended.
These tests will not be executed with the call 'ctest -L export'.
These exports correctly fail now that we've switched to polyglossia.
Although they compiled without error with babel, the resulting PDFs
had gibberish.
I believe these tests were fixed by TeX Live updates.
Comparing the log files for a system where the tests fail with a
system where the tests pass, below are some of the differences
between the "good" and "bad" logs:
bad:
LaTeX2e <2015/01/01>
Babel <3.9l>
Package: fontspec 2015/03/14 v2.4c
Package: expl3 2015/03/01 v5547 L3 programming layer
good:
LaTeX2e <2015/01/01> patch level 2
Babel <3.9m>
Package: fontspec 2015/07/22 v2.4d
Package: expl3 2015/07/30 v5724 L3 programming layer
Many of our documents have babel-specific preamble code. By putting
this code in a \@ifpackageloaded{babel}{}{} conditional, XeTeX and
LuaTeX compilation with polyglossia now works. This fixes some
LuaTeX tests that were broken by edd37de8 and also allows us to
uninvert some XeTeX tests.
Note that in some of the files although the preambles were fixed to
allow for polyglossia, they still do not compile without errors:
es/Math.lyx
es/Customization.lyx
de/Customization.lyx
Similar fixes might be desired in other manuals but these at least
fix regressions in the tests.
PolyglossiaOpts are case-sensitive so "latin" must be changed to
"Latin". Without this change, compiling examples/sr/Braille.lyx
with LuaTeX and system fonts gives the following error:
Package Polyglossia Error: Unknown script `latin' for Serbian
language
Many of our Spanish documents use babel-specific features in the
documents, e.g. to write "sin" in Spanish ("sen"). Because babel
seems to have good support for Spanish, I am setting the "Always
babel" for the manuals.
This fixes several LuaTeX tests with non-TeX fonts. A XeTeX test is
also reverted accordingly.
We already have a CoordCache of insets dimensions. It is not necessary
to store the same information in two places.
Give a name to CoordCache tables types to improve code readability.
Remove ParagraphMetrics::singleWidth, which is not used anymore.
- there are some new packages required by LyX's supported document classes and example files
(this list is only used when LyX is installed the first time on a PC together with MiKTeX to shorten the installation time)
This is a work in progress intended to start collective work towards improving the performance of our painting process.
The intent is to make it a living document that is updated as code evolves.
- the long awaited MiKTeX installer is available, this fixes many installation errors in LyX is installed for the first time (I will provide a new installer build as soon as I find more time)
Being able to compile document with zipped .eps files was a useful feature of
the graphicxs package 20 years ago, but the LyX support is no longer relevant:
- The flag is ignored if preview is on
- If pdflatex is used then uncompressing happens during the compilation anyway
- If set, the flag prevents LyX from issuing proper error messages if
something with the image is wrong
- For hard disk capacities from 20 years ago not uncompressing is a useful
feature, but for current hard disk capacities it does not matter
- The external inset does not have it, and if we want to merge both insets
one day we would need to implement it there, which is even more difficult
than in InsetGraphics
We do not currently have any Arabic documents, but when we do, these
fonts will be used for the tests that use system fonts.
They can be installed on Debian-type systems from the
'fonts-sil-scheherazade' package.
Since lyX 2.0 we support to use the in_preamble tag. This allows to support more commands. For an unknown reason I forgot the jss.layout when I updated the other layouts once.
Fileformat change
The general problem is that Acrobat opens PDF files with write access so that PDF files cannot be modified externally by LyX/LaTeX.
The old pdfview used the programs pdfopen and pdfclose to open and close PDF files in Acrobat and it additionally closed and opened the PDF whenever it was changed (to simulate LyX's update button).
Since Acrobat DC (aka Acrobat 15) it is no longer possible to close PDF files in Acrobat. Due to changes in the Acrobat API also the pdfopen program can no longer be used.
Besides this the approach of updating the PDF whenever it changed was always a hack that often killed Acrobat. The reason is that one cannot immediately update the PDF view if the PDF is changed because depending on the size LaTeX needs up to several seconds to generate the PDF. If the PDF is opened while TeX is still creating is one get error messages from Acrobat Acrobat must be restarted - very annoying. During the years I experimented with a wait time to minimize these problems but of course it can never be solved.
The new pdfview version uses a completely different approach:
- it checks if the PDF is opened by Acrobat by reading the window name. If it is found this program windows will be closed. So Acrobat is not ended and other opened PDF files stay opened.
- to update the PDF view from within LyX one has to press the view button, not the update button. Only this way we know that TeX is ready and the PDF can be opened by Acrobat with write permissions.
The math parser could not handle multicolumn grids. This is a problem because
there is no true ERT in math (everything is parsed).
Now multicolumn cells are parsed correctly. The display is also somewhat OK,
but apart from that any multicolumn related UI is missing. Since the file
format change is now done the UI can be added at any later point. The most
important part of bug 396 is now fixed: tex2lyx does not create invalid .lyx
files anymore for formulas containing \multicolumn.
I updated the tex2lyx test cases that produce correct output. tex2lyx does
still produce invalid output for the test cases which are not updated because
of the previous format change.
The "save-as" part of the bug is fixed by extending the \textclass tag
such that, if a local layout file is used, its path relative to the
document directory is now stored together with the name. If a relative
path cannot be used, an absolute one is used but, in this case, the
document is not usable on a different platform.
The "copy" part is fixed by introducing a new \origin tag, which is
written when the file is saved. This tag stores the absolute path of
the document directory. If the document is manually copied to a
different location, the local layout file is retrivied by using
\origin (which is only updated on save).
This new tag may prove useful also for locating other files when the
document is manually moved to a different directory.
As in the original implementation the files needed for the layout
(for example, a latex class) had to be in the same directory as the
layout file, this directory has also to be added to TEXINPUTS.
The GNU libstdc++ that ships witch gcc 5 can be used with the same ABI as
older versions, or with a new ABI which is conformant to the C++11 standard.
LyX did not build if the latter was used:
https://kojipkgs.fedoraproject.org//work/tasks/1267/9651267/build.log
This is now fixed by detecting the ABI version and disabling the wrong forward
declarations. At the same time, STD_STRING_USES_COW is switched off for the
C++11 ABI version, because the std::basic_string implementation is now C++11
conformant. Since the GNU libstdc++ can also used by other compilers such as
clang, we must not test for the compiler version.
The bug that caused these tests to enter into infinite loops
(and have a memory leak) has been fixed. These tests now
pass on an updated TeX Live 2014 as well as the current TeX
Live 2015 pretest.
One of the tests is also disabled for es/Math.lyx. However,
the other test passes for es/Math.lyx. To reproduce,
open fr/Math.lyx, click "Use non-TeX fonts", and choose
e.g. "FreeSans" for the three fonts. Then view as PDF (LuaTeX).
I put a note to look into why this one fails.
This is the rersult of a discussion on the list. Now all special characters
have meaningful names, and it is clear that the LyX file syntax is not LaTeX.
According to fix#9432, Scott found a new crash
1. start a new LyX document, english GUI
2. alt+m f to create a fraction
3. alt+m r to insert a root
4. undo
Recent versions of the cygwin X11 server come with a startup script
that explicitly uses '-nolisten tcp' for improved security. This means
that mentioning a host part in the DISPLAY variable precludes correct
operation. So, leave blank the host part such that only local connections
are attempted. Even if a user can override this setting in the own
~/.lyxprofile, novice users (and even experienced ones, at first) would be
probably confused by the "Error: Can't open display: localhost:0" message
and thus it is better to make this work out of the box.
Previously, LyX did replace some words with typeset logos, and there was no
way to prvent this except putting them, in ERT (bug #4752). Now we have
special insets for these words, and standard text is left alone.
Previously, consecutive dashes in .lyx files were combined to endash and emdash
in some cases, and in other cases they were output as is. This made the code
complicated, and resulted in inconsitencies ((bug #3647).
Now, a dash in a .lyx file is always a dash in the output, for all flavours.
The special handling is moved to the input side, so that you still get an
endash if you type two hyphens. If needed, this can be changed or made
customizable without the need to update the file format again. Many thanks
for the fruitful mailing list dicsussion, which contributed significantly to
the final version.
- the installer now re-synchronize MiKTeX's package database before installing or updating LaTeX-packages. This assures that new packages and renamed packages can automatically be installed when they are missing (fixes bug #9370)
- update the list of LaTeX packages that have to be installed when LyX is installed via the bundle installer for the first time on a PC
- add support for Turkish spell-checking
As discussed on the list. We don't need it anymore, either we have a modern
compiler that supports C++11, or we fall back to boost. I kept and adjusted
the regex #define, since we cannot use std regex completely yet.
The old detection did only work if CFLAGS contained -std=c++11, since ciso646
was only included for __cplusplus > 199711.
Thanks to Koernel for the cmake part.
This makes the script usable on windows and speeds it up by an order of
magnitude, since no new process needs to be forked for each layout file.
It also does not conevrt .old files again.
1.) The only file which needs this define is src/version.cpp. Our use in config.h
resulted in recompiling most of our sources.
With this change, only version.cpp needs to be recompiled.
2.) Without reconfiguring, the old commit hash was still valid. Now the
detection of commit hash is done at compile time.
This should prevent possible errors for tests running in parallel.
We now also check for created export file.
complainer/feature requester: Scott Kostyshak
If the LyX window does not appear after a certain (conservative)
time, the keytest now fails. One reason that caused the LyX window
to never show up was when DISPLAY was not set. This particular case
has been fixed in <commit1hash>. This commit is meant for the more general
scenario.
(Collaboration with Scott Kostyshak)
Before, the keytests would run infinitely if the
tests were run, e.g. through SSH without X forwarding.
To reproduce that problem, run
DISPLAY='' ctest -R hello
This commit causes the test to correctly fail in such situations.
(Collaboration with Scott Kostyshak)
The option --enable-qt5 allows configuring for Qt5. The default is Qt4.
Nothing special is done with respect to Qt4, apart from pulling in the
correct libraries. Indeed, other than the core and gui libraries, now
also the concurrent and widgets libraries are needed.
Forthcoming versions of cygwin will use a different mechanism for
obtaining passwd/group information based on /etc/nsswitch.conf.
Thus, it will not be guaranteed that the files /etc/passwd and
/etc/group even exist. The recommended way for obtaining those
info is by using the getent command, which already works in
current versions.
The configuration time suffers mostly on checking, which of the export tests
is to be reverted.
1.) There is a new configuration flag now, "LYX_ENABLE_EXOPRT_TESTS.
If not set (default) no export tests are created.
2.) If set, then the optimization halves the time needed for creation of tests.
The effect on my side:
a.) Until now the time was: ~ 26 seconds
b.) The optimized time is now: ~ 16 seconds
c.) With not enabled export tests: ~ 5 seconds
This is a follow-up of bug #8967. The implementation is self-explaining, the
only part which needs a comment is lyx2lyx: Since a 100% correct solution is
not possible, it has been decided not to switch amsmath off in the forward
conversion if no other ams command than \smash[t] and \smash[b] is used, but
to consider it a bug that older versions do not load amsmath automatically for
these commands. In the backward direction it is easy to keep the document
compilable, so just do that.
In collaboration with Scott Kostyshak:
With Qt 4 we could use Q_WS_X11, defined by FindQt4.cmake. In Qt 5,
there is no FindQt5.cmake. Instead we now define our own variable,
QT_USES_X11 by using class QX11Info available only on X11.
(http://qt-project.org/doc/qt-5/QX11Info.html)
The main consequence of this improved support is that now the keytests
can be run (ctest -R "keytest") when LyX is compiled with Qt 5.
Before, with Qt 5 we did not know if X11 was available, which is
needed by xvkbd, so the tests were not enabled. Note, however, that
many tests fail with Qt version 5.2.1 because there was a change in
the event handling mechanism in Qt that causes xvkbd to be unable to
pass capital letters (so case sensitive greps in the tests fail). This
needs to be investigated and reported.
The algorithm used for breaking a paragraph in LaTeX export is changed
for avoiding spurious blank lines causing too much vertical space.
This change is tied to the introduction of a new inset (with two
different specializations) helping in either outputing LaTeX paragraph
breaks or separating environments in LyX. Both of the above goals were
previously achieved by the ---Separator--- layout and can now be
accomplished by the new inset in a more natural way. As an example,
after leaving an environment by hitting the Return key for two times,
a third return automatically inserts a parbreak inset, which is
equivalent to the old separator layout, i.e., it also introduces a
blank line in the output. If this blank line is not wanted, the
parbreak separator can be changed to a plain separator by a right
click of the mouse. Of course, an environment can still be separated
by the following one by using the Alt+P+Return shortcut (or the
corresponding menu key), but now the plain separator inset is used
instead of the old separator layout, such that no blank line occurs in
the LaTeX output.
Old documents are converted such that the LaTeX output remains unchanged.
As a result of this conversion, the old separator layout is replaced by
the new parbreak inset, which may also appear in places where the old
algorithm was introducing blank lines while the new one is not.
Note that not all blank lines were actually affecting the LaTeX output,
because a blank line is simply ignored by the TeX engine when it occurs
in the so called "vertical mode" (e.g., after an alignment environment).
The old ---Separator--- layout is now gone and old layout files using it
are also automatically converted.
Round trip conversions between old and new format should leave a document
unchanged. This means that the new behavior about paragraph breaking is
not "carried back" to the old format. Indeed, this would need introducing
special LaTeX commands in ERT that would accumulate in roundtrip
conversions, horribly cluttering the document. So, when converting a
modified document to old formats, the LaTeX output may slightly differ in
vertical spacing if the document is processed by an old version of LyX.
In other words, forward compatibility is guaranteed, but not backwards.
This patch improves the cmake bundling process:
- reworked bundle handling for OS X: only when installing resources are copied, otherwise
only the smallest bundle is built
- on OS X, the utility programs (tex2lyx) are now installed in the right location
- it removes some unneeded BUNDLE DESTINATION
- it provides a basic support for QT plugins inclusion
- it properly builds a disk image on OS X (only the background image is missing)
- it fixes the library paths for all executables (not only LyX)
- Use the COPYING file for cmake install license
These started failing after we implemented tests for formats
that are in the complement set to the default format (7ecbb068).
It might be worth it in the future to take a look at each individually
and see whether they are supposed to fail or if there is something LyX
can do to add support for exporting them.
As discussed with Vincent I added 3 new features:
* LyX can now also be installed over an existing installation of the same LyX version if the users really wants this.
(A message box appears recommending not to do so for test releases and when there are problems with an existing installation of the same release.)
* LyX can now also be installed if the uninstaller executable of a former LyX was accidentally deleted.
(That this case was not handled was more a bug. Thanks Vicent for the pointer.)
* If a user set MiKTeX's automatic package installation to "No" the installer will.
(As discussed with Vincent, if a user has disabled this MiKTeX feature he most probably know the consequences. So respect his decision.)
respect that and not re-enable it.
I would be happy if anybody can spend a translation of the changed installer message.
- known problem: if LyX 2.0.x is installed and you install 2.1rc1, the .lyx file associations will not be updated but if LyX 2.1 is installed on a new PC the file associations work (work is in progress)
Define new policy of what export formats to check.
Depending on default_output_format
default or unset xhtml, dvi*, pdf*
xhtml xhtml
any of pfd variants xhtml, pdf*
any of dvi variants xhtml, dvi*
each other (like eps3) as specified
- it turned out that the hack to install a stripped-down Perl directly to MiKTEX is not only error-prone, requires admin permission but that it is also not safe because MiKTeX changes the folder structure from time to time. With the new solution Perl is installed directly for LyX like we do for ImageMagick, Python etc.
- for pdfcrop support there is a bug in pdfcriop that the requirement is hardcoded to Ghostscript 32bit - I contacted the pdfcrop author to fix this
This test was broken after wrap-around search was implemented. From
what I understand, the point of the test is that search should _not_
match in this case because "ignore format" is checked by default.
Thus, a new way of testing that there was no match is to check whether
the wrap around message shows up, because the cursor is initially at
the beginning so that would mean that the entire buffer was checked.
I confirmed that the test now works by checking that it passes and
checking that it fails if "ignore format" is unchecked. Although this
test is still fragile, it is fragile in a conservative way (i.e. it
will give false failures rather than false passes). It will fail if
the Italian translation changes, if "ignore format" is unchecked by
default, or if the wrap-around mechanism changes (e.g. does not ask if
the cursor started at the beginning of the buffer).
