there were 2 issues:
- the body was not output to the Preamble
- the theorem module already defines a theorem. It must not be output to the preamble to avoid a redefinition error of LaTeX
update the test files accordingly
\framebox{} is equal to \fbox{} and \makebox{} is equal to \mbox{}
When I once wrote this code LyX did not support \fbbox and \mbox and therefore had to use a workaround.
The math parser could not handle multicolumn grids. This is a problem because
there is no true ERT in math (everything is parsed).
Now multicolumn cells are parsed correctly. The display is also somewhat OK,
but apart from that any multicolumn related UI is missing. Since the file
format change is now done the UI can be added at any later point. The most
important part of bug 396 is now fixed: tex2lyx does not create invalid .lyx
files anymore for formulas containing \multicolumn.
I updated the tex2lyx test cases that produce correct output. tex2lyx does
still produce invalid output for the test cases which are not updated because
of the previous format change.
Revert "tex2lyx testfiles: update to latest fileformat"
This reverts commit afdfc3cf43.
The tex2lyx test vrefernces must be produced by tex2lyx, otherwise the tests
would still fail.
This is the rersult of a discussion on the list. Now all special characters
have meaningful names, and it is clear that the LyX file syntax is not LaTeX.
tex2lyx did not use the InsetLayout for the script insets, so it did not know
that it had to use Plain Layout. However, there is still a possible pitfall:
InsetScript uses a hard coded plain layout for the InsetText constructor, so
if anybody would remove ForcePlain 1 and MultiPar false from the InsetLayout
then LyX would still write script insets with plain layout, but tex2lyx would
output standard layout again.
Previously, LyX did replace some words with typeset logos, and there was no
way to prvent this except putting them, in ERT (bug #4752). Now we have
special insets for these words, and standard text is left alone.
Previously, consecutive dashes in .lyx files were combined to endash and emdash
in some cases, and in other cases they were output as is. This made the code
complicated, and resulted in inconsitencies ((bug #3647).
Now, a dash in a .lyx file is always a dash in the output, for all flavours.
The special handling is moved to the input side, so that you still get an
endash if you type two hyphens. If needed, this can be changed or made
customizable without the need to update the file format again. Many thanks
for the fruitful mailing list dicsussion, which contributed significantly to
the final version.
The textcyr test did not work anymore, since the output of
"CYRILLIC SMALL LETTER VE" changed from \textcyr{\char226} to \textcyr{\cyrv}.
The the old version is now converted to ERT. In theory it could also be
converted to "CYRILLIC SMALL LETTER VE", but this would mean to maintain an
extra list, which is not worth the effort.
This is the first part of bug #8553. \textgreek was interpreted as the
polyglossia version even if polyglossia was not used. Now it is not recognized
at all and converted to ERT, but this ensures at least correct LaTeX output.
Previously tex2lyx did ignore table columns if the column specifier was
unknown. This can lead to data loss, and now it does not ignore these columns,
but assumes that the column specifiers use only once character. This can still
lead to data loss, but less often.
Now we can handle all types of captions defined in layout files or modules.
This does also mean that \captionabove and \captionbelow are only recognized
in KOMA-classes and not always anymore, but this is correct, since these
captions would not work in other document classes anyway.
If we are testing tex2lyx the first time, then the userdir
is empty. We call now configure.py to initalize in case it is needed.
The algorithm to detect if reconfigure is needed mimics the lyx behaviour.
This is a follow-up of bug #8967. The implementation is self-explaining, the
only part which needs a comment is lyx2lyx: Since a 100% correct solution is
not possible, it has been decided not to switch amsmath off in the forward
conversion if no other ams command than \smash[t] and \smash[b] is used, but
to consider it a bug that older versions do not load amsmath automatically for
these commands. In the backward direction it is easy to keep the document
compilable, so just do that.
writer2latex surrounds quotes by braces which we skip to avoid useless ERT.
I broke this in 25fe87e5 which made Parser::next_next_token() not recognize
that it needed to parse one more token. If we had a unit test for the Parser
class it would probably have detected that. Now we have at least a test for
the special quote.
The tests are now more robust if ctest uses '-j' (number of threads) parameter.
a.) keytests are running in sequence and in one thread only, no other thread running
b.) tex2lyx tests are locking in respect to each other
c.) Test gets one or more labels, so that we can select
ctest -L url
to run tests labeled url
d.) New macro settestlabel() to add labels to a test
When comparing the contents of two lyx-files, we should not care about the specific version. Later, this can be extended to also not care whether the file is produced by tex2lyx, lyx2lyx, or LyX.
Modified-By: Vincent van Ravesteijn <vfr@lyx.org>
LyX, lyx2lyx and tex2lyx produce now all the same version indicator consisting
only of the major and minor version. It is not decided yet whether future
development versions will add a -dev suffix, but for 2.1.0 this change fixes
the inconsistencies.
Unfortunately automake does not support file names with spaces for files in
EXTRA_DIST: These file names are put into a shell list, and it is impossible
quote the name in a way that works both with the shell and with make.
Therefore I removed the corresponding test file.
In roundtrip mode, tex2lyx produces documents with extension lyx.lyx, so that
the original files are not overwritten on re-export. This was not done for
included documents which were converted, and this broke 'make dist'.
These should be used if any new style needs to be introduced in the stable
2.1 series: If the ForceLocal flag of the style is set, it will always be
written to the document header, so that even older 2.1 versions can read
and correctly output the document.
This fixes bug #8554 and some recently introduced busg:
- Encodings::fromLaTeXCommand() can now handle all combining characters,
not only the one letter ones
- The remainder returned from Encodings::fromLaTeXCommand() must never be
thrown away in tex2lyx, but output as ERT
- No special case for combining diacritical marks needed anymore in parse_text()
- No special cases for accents and IPA combining diacritical marks needed
anymore in parse_text()
- special tipa short cuts may only be recognized if the tipa package is loaded
- Use requirements returned by Encodings::fromLaTeXCommand() instead of
hardcoded registering of tipa and tipax
- Get rid of the name2 variable in parse_text(): We must use name, otherwise
the extra stuff that might have been put into name vanishes
tex2lyx skips LyX preamble code only if it thinks that the file was created
by LyX (i.e. special comments are found). I don't like that, but this is
how it works and therefore the special comment neds to be added if the
theorem commands are to be skipped. This header makes also the temporary
setting of in_lyx_preamble obsolete, which caused skipping of user preamble
commands, which were not re-added by LyX.
- Convert prettyref to the autopackage mechanism
- Do not load refstyle automatically if some refstyle preamble code of LyX
is found, since LyX will only load the package if an actual reference
command is used. This is needed for mixed refstyle/prettyref documents.
- Only recognize refstyle commands if refstyle was detected in the preamble
- Only recognize prettyref commands if prettyref was detected in the preamble
- Add a mixed refstyle/prettyref test case
- support for Cyrillic characters
- support for \textifsymbol and \ascii (fixes bug #8556)
- support for \ding
- tex2lyx/text.cpp: correct an indentation and use "name2" because "name" is already defined in this clause
The LyX file format only supports one CJK font mapping per document.
Therefore all CJK environments which use a different mapping than the main
one are output in ERT. THis is now explained in the test case, and also the
converion to ERT is improved (although it is still wrong e.g. for paragraphs).
- support for \textdoublevertline, \textvertline, \textglobfall
- support \!o and \!b and \!d and \!g and \!G and \!j
- support and \*k and \*r and \*t and \*w
- register the package "tipa" if \textipa is detected in an equation
- add a \textipa equation to the testfile
(I failed to implement support for the TIPA character "\t*{ }" and the command "\=*".)
- support for the combining diacritical marks
- support for \texttoptiebar and \textbottomtiebar
- test-insets.lyx: add complete testcase
- TODO.txt: update what still needs to be done
This is an addendum to [72a44b3c/lyxgit] because depending on the environment, LyX adds a \phantomsection before \addcontentsline.
- also update the test file
This is an important part of the tests. If updating the test cases is really a
problem there are two better solutions than not testing the format: Convert
the references with lyx2lyx on the fly, or remove the hard coupling of tex2lyx
and LyX versions again. Both increase additional possible error sources, but
these errors could at least be detected. If the test machinery is made blind
for versions, file format errors are impossible to detect.
These encodings were not defined, since they must not be used as document
encodings (the characters {, } and \ may appear in high bytes, and latex
would be confused). However, they are supported by CJK.sty (which uses a
preprocessor to circumvent the limitations of the latex executable). These
encodings are now defined, but used for import in tex2lyx only.
The test case CJK.tex contained fake tests for shift-jis and big5 (the
japanese and chinese characters were entered using the utf8 encoding), and
therefore the wrong interpretation of these encoding looked as if it worked.
The comments about missing iconv support of shift-jis and big5 were wrong as
well (otherwise shift-jis-plain would not work either).
Actually, the test case showed several problems:
- ERT insets did use layout "Standard", not "Plain Layout"
- The font scale was read correctly, but tex2lyx claimed that it did ignore
the option "scaled=0.95"
- If a third argument of the CJK environment was given, it caused the whole
environment to be put in ERT with a broken encoding. This is now fixed for
the bug test case by using the \font_cjk header variable, but the encoding
problem still exists for unsupported encodings. I'll file a separate bug
for that.
- The CJKutf8 package was not handled in the preamble parsing. Therefore the
chinese comment in the preamble was read with a wrong encoding, and guessing
the document language did not work.
The new file CJKutf8.tex was created by copying and modifying CJK.tex, but
unfortunately it is impossible to tell git to inherit the history of CJK.tex
for the new file (search the web for git svn copy if you want to know details).
Special characters as created by latex_path() where not converted corectly
from LaTeX macros by tex2lyx. Now this is done, even for file names containing
double quotes which are not used for quoting spaces. These file names are not
legal on windows, and will causes probelms in DVI files, but if they occur
tex2lyx does not produce invalid .lyx files.
The fix is basically mechanical, the additional code for fraction like insets
with three arguments was stolen from \unitfrac. As any math package,
stackrel.sty needs a buffer parameter to switch it off.
I also added the two stackrel flavours to the toolbar.
The stmaryrd package adds support for lots of math symbols, using a font
designed to accompany the computer modern fonts. The changes in detail:
- Fix generate_symbols_list.py to work with stmaryrd.sty. It loooks like it
was automatically translated from a perl version and never used.
- Generate the new symbols in lib/symbols using generate_symbols_list.py and
add some manual adjustments
- Generate stmary10.ttf by a simple ttf export from stmary10.sfd with fontforge
- Add license info for stmary10.ttf
- Create a test file with all symbols from stmaryrd.sty. Actually it would be
nice to have this for the other fonts as well.
- The mechanics: lyx2lyx, tex2lyx, font machinery etc.
Actually tex2lyx can handle modules since some time (#5702), but not
theorems (#5776). Now the following issues are fixed:
- Modules that depend on other modules can be loaded, since the dependencies
are loaded first
- Default moduls of the text class are loaded correctly
- \newtheorem is recognized as a command that defines new environments and
treated similar to \newenvironment
The listings inset does automatically load the color package if any parameter
contains \color. As mentioned in bug #8066 tex2lyx needs to be aware of this,
so the \usepckage{color} is automatically skipped in these cases.
s is a valid horizontal position for framebox (as well as for makebox, which
was already parsed correctly). There was even a test case, but with a wrong
reference.
The previous output was read correctly by LyX as well, but the new version is
more similar to the files written by LyX. Also update TODO with recent file
format changes.
The old fix was incomplete (\verb~\~ was translated to \verb~~ in roundtrip).
The real cause for this bug (and also the mistranslation of \href{...}{\}})
was the misbehaviour of Token::character() (see comment in Parser.h): This
method even returns a character if the category is catEscape, and this is not
wanted in most (all?) cases.
It is invoked by 'make check' (automake only, it would be nice if someone
could add it to cmake as well), or by calling
python src/tex2lyx/test/runtests.py <path to tex2lyx binary>
by hand. Currently, it does not compare the output (this comes later).
The added .lyx files are from tex2lyx around mid of april, so that you
can see the regressions of the current version if you run the test
yourself (simply run git diff afterwards).
The home made test runner is quite stupid, but better than nothing.
Feel free to improve it or replace it with something better, as long
as running it stays as simple as now.
- tex2lyx/text.cpp:
- fix bug that swallowed valid braces
- the encoding is "Bg5" not "BIG5"
- add and update comments
- CJK.tex: add an example for the Bg5 encoding
Preamble.cpp:
- support the font scaling
- improve coding style
text.cpp: - improve logic/coding style
XeTeX-polyglossia.tex:
- add scaling
- change document so that it is compilable using the Libertine font
- bugfix for the parsing of polyglossia's paragraph environments
- new support for polyglossia's language change commands \textxxx
- XeTeX-polyglossia.tex: add more testcases and add a missing \end
This is a part of bug #8081: Translate cells in
\begin{sideways}...\end{sideways} to native syntax. The turn environment
comes in a later commit, since this one os suitable for backporting.
This is not needed, since LyX supports comments in math. Data loss with math
comments containing a backslash in LyX has been fixed as well.
The test case was found in bug #8104.
I introduced this bug in r40091 where support for empty dates was added.
I also noticed that the title_layout_found flag was not properly passed down
to all variants of parse_text*. Since this is rather a global variable I
placed it into the preamble class.
The test case of bug #8104 shows other problems as well, but those are no
regressions and will be fixed separately.
OK for branch?
- Parser.cpp: - new function to parse verbatim environments
- test/test-structure.tex: updated example
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@40850 a592a061-630c-0410-9148-cb99ea01b6c8
A complete fix is currently impossible because LyX assumes that all longtable
captions are in the first head.
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@40361 a592a061-630c-0410-9148-cb99ea01b6c8
