We open the input file now twice: The first time in latin1 encoding to read
the document encoding from the preamble. This does always work, since
traditional TeX does not allow non-ASCII contents without an encoding changing
command (except for comments, but we do not need them, and using latin1 rather
than utf8 ensures that they do not produce an iconv exception, but are simply
recored with wrong characters), and we do detect the utf8 based TeX engines
XeTeX and LuaTeX as well. The second time we open the file directly with the
document encoding.
This fixes a few tex2lyx tests on OS X, since changing the encoding of an
open file steam does not work with clang on OS X. Files using more than one
encoding are still broken, but all single-encoding files are fixed now.
This does not make any difference currently, since the only instance of the
Preamble class is a global one, but it is cleaner and will be needed for
parsing the encoding with a second Preamble instance.
Increment LyX format to 504.
With this new parameter, the user can indicate that some other parameters that
are frequently switched must not be recorded in the file (as if they were a
setting specific to the user or transient, rather than a document setting). This
will play nicer with version control systems.
See the discussion, e.g.:
http://thread.gmane.org/gmane.editors.lyx.devel/157824/focus=157993 (third
solution mentioned)
TODO:
* The interface remains to be set up. We cannot change this setting from LyX
for now.
* If save_transient_properties is false, we should read the user setting as a
per-user-per-document (session) setting (e.g. like the cursor position).
* Once the above is done, we can treat \justification the same way (but it would
be even better if \justification was moved to lyxrc).
The lib/unicodesymbols part is based on work by Günter Milde:
Both, \r{A} and \AA (rsp. \r{a} and \aa) are equivalent standard LICR macros
for Aring/aring as well as the deprecated "angstrom sign" character (212B).
However, with \AA for 212B and \r{A} for 00C5, tex2lyx converts \AA to the
deprecated "angstrom sign" which is missing in many fonts including the
Unicode version of Latin Modern.
I added the normalize_c() calls so that tex2lyx prefers the precomposed forms
(these are better editable in LyX) and the deprecated flag.
The conversion is not completely correct yet, but adding the test before the
fix has the advantage that the fix can easily be documented by changing the
test reference.
When importing TeX code like
%comment
TEXT
the LyX document currently becomes
[ERT %comment] TEXT
so that TEXT is now part of the comment.
Now output_comment adds a trailing newline if the token after the
comment is not a newline. Note that the newline that marks the end of
the comment has already been parsed at this point.
tex2lyx tests have been checked manually and updated.
Fixes ticket #9551.
This is one part of bug 9744: If you toggle between TeX fonts and non-TeX
fonts, the settings of the other choice are no longer thrown away, but stored
and re-activated if you switch back. Most parts of the patch are purely
mechanical (duplicating some BufferParams members), the only non-mechanical
change is in the GUI logic.
PDF properties (Author, Title...) need to be quoted properly when writing (normal operation and tex2lyx). Also, reading them requires to use Lexer::getString() directly, because >> uses next(), which does not handle escapes.
Fixes bug #9830.
The execution took some time and the messages were delayed
until all test files were processed. Now as each testfile
is processed, the appropriate messages are printed.
There is also a new target: cleanupdatetex2lyxtests
It is the same as updatetex2lyxtests, but removes also
some extra created files from the source directory.
Cmake versions prior to 2.8.11 don't know this command, but
our minimal requirenment is 2.6.4
Thank to Vincent, it is replaced by using 'if(file1 IS_NEWER_THAN file2)' comparision.
The horizontal alignment changes caused a regression for documents in older
formats. This could have been fixed by adding the following lines to the
format entry for 489 in development/FORMAT:
Previously, the horizontal position was ignored except for the
following parameter combinations:
- fixed width and type Boxed and without inner box
- fixed width and any type and with inner box and with makebox
Now, it is also used for the parameter combination below:
- makebox is not used and not (type Boxed and without inner box)
and a corresponding conversion to convert_BoxFeatures() in
lib/lyx2lyx/lyx_2_2.py. However, it was decided to revert the box alignment
changes instead: The box alignment can contradict the paragraph alignment. A
better way to avoid the additional space that can be created by paragraph
alignment (which was the motivation for implementing box alignment) is the
same as for table cells (see noTrivlistCentering() in src/Paragraph.cpp).
This would be a file format change and it is too late now for 2.2.0.
See http://www.mail-archive.com/lyx-devel@lists.lyx.org/msg188147.html for the
complete discussion (subject "Regression in lyx2lyx box alignment").
The correct mappings break the tex2lyx roundtrip tests because both,
the tests and tex2lyx use the wrong mapping (\b for "combining minus below").
Fix tex2lyx and the test files so that round-trip tests pass again.
These were found by cppcheck:
Member variable 'x' is not initialized in the constructor.
The crash #9788 would not have happened if this had been done earlier.
There are still a few warnings of the kind
(style) Variable 'x' is assigned a value that is never used.
since I did not touch code where I was not sure whether there might be a real
bug, and I kept some for symmetry reasons as well.
These were all flagged by "(style) The scope of the variable 'x' can be reduced."
Narowing the scope improves readability, and if it is in a loop then the
compiler will be clever enough to produce efficient code, we do not need
manual optimization for POD types.
As Günter found out running the tex2lyx tests overwrites the test references
if the build directory is identical to the source directory. Therefore the
tests would always pass, but git diff would show a non-empty diff if the tests
should have failed. Since it is better anyway to build in a separate directory
we simply do not support srcdir = builddir for the tests and abort with an
error.
This brings the external inset on par with the graphics insets as far as the
clipping option is concerned. The graphicxs package supports both: A bounding
box without units (which means that bp ia assumed), and a bounding box with
units, so we can simply output the values including the units.
Being able to compile document with zipped .eps files was a useful feature of
the graphicxs package 20 years ago, but the LyX support is no longer relevant:
- The flag is ignored if preview is on
- If pdflatex is used then uncompressing happens during the compilation anyway
- If set, the flag prevents LyX from issuing proper error messages if
something with the image is wrong
- For hard disk capacities from 20 years ago not uncompressing is a useful
feature, but for current hard disk capacities it does not matter
- The external inset does not have it, and if we want to merge both insets
one day we would need to implement it there, which is even more difficult
than in InsetGraphics
Unfortunately I overlooked in 44f73b0650 that the first three whitespace
changes in box-color-size-space-align.lyx.lyx were actually correct, so they
should not have been reverted. In detail:
1), 2): The space after \raggedleft must not be part of the ERT inset, but it
is ouput by check_space() as part of the standard text which follows.
3): The space in front of www is caused by the fact that there is a
newline between the opening brace of the parbox and the \centering
command, so this space is not the one after \centering (which is
correctly swallowed). This additional space is in fact not needed,
and the contents would look better in LyX without it, but since it is
not caused by special code I'll put it back in the refernce for now.
We can still improve this in the future if anybody has a good idea.
The remaining whitespace issues are all fixed by a simple change in
parse_text(): Instead of always eating whitespace after detecting \centering
et al, and always output a space as part of the ERT if these commands need an
ERT, let the standard space handling mechanism kick in: skip whitespace if
no ERT is used (in this case LyX will always output the needed space), and
do not touch whitespace if an ERT is used.
These should not have been done without discussion.
- Removal of the dcolumn table in 1a8b74f5e1. Even if LyX does not support
dcolumn anymore, it is still a useful test whether tex2lyx imports it
correctly.
- Removal of the first "%% LyX" line. A long time ago it was decided (after
long discussion between at least Jean-Marc, Uwe and me), that this line is
interpreted by tex2lyx, and used to remove some LyX-generated preamble code.
These lines in the current tests exist on purpose (one can see in the diff
how the removal added unwanted stuff). I do not really like the
interpretation of the "%% LyX" line, but if this behaviour is to be changed
then this needs discussion first.
- Changed comment of \date. The comment was put there on purpose, and the
warning which was "fixed" by the change hints at a limitation in LyX, not a
tex2lyx problem (LyX does not know that a comment inset between some title
insets is OK). The roundtrip .tex output was OK with the old version.
- Change of \verbatiminput{foo}. This was supposed to test whether a
verbatim inset is correctly created even if the included file does not
exist.
- Removal of \lyxlines. Although these tests test input of files created by
old LyX versions, they are useful.
- Change of the lemma in test-modules.tex. The old version was put there on
purpose, and the file itself explains why it is translated to ERT.
There is a general problem of tex2lyx handling theorems. it is not sufficient to cure only one instance as I did. For more info see bug #9561.
Update the references accordingly.
With the now removed command we said that this is a file created by LyX but this is not the case
- also replace a comment to avoid LaTeX warnings about mixing title and non-title stuff
before LyX 2.1 was released the dcolumn support was dropped and another method was used to align at the decimal point. Nevertheless the old LaTeX table was even wrong in terms of dcolumn
Now a table is used that uses the decimal alignment as it is supported by LyX
The idea is to get a compilable file that does not require programs that are only available on certain platforms.
For example on Windows there is no Gnumeric available (only a very outdated and unsupported version with bugs).
there were 2 issues:
- the body was not output to the Preamble
- the theorem module already defines a theorem. It must not be output to the preamble to avoid a redefinition error of LaTeX
update the test files accordingly
\framebox{} is equal to \fbox{} and \makebox{} is equal to \mbox{}
When I once wrote this code LyX did not support \fbbox and \mbox and therefore had to use a workaround.
The math parser could not handle multicolumn grids. This is a problem because
there is no true ERT in math (everything is parsed).
Now multicolumn cells are parsed correctly. The display is also somewhat OK,
but apart from that any multicolumn related UI is missing. Since the file
format change is now done the UI can be added at any later point. The most
important part of bug 396 is now fixed: tex2lyx does not create invalid .lyx
files anymore for formulas containing \multicolumn.
I updated the tex2lyx test cases that produce correct output. tex2lyx does
still produce invalid output for the test cases which are not updated because
of the previous format change.
Revert "tex2lyx testfiles: update to latest fileformat"
This reverts commit afdfc3cf43.
The tex2lyx test vrefernces must be produced by tex2lyx, otherwise the tests
would still fail.
The FontMetrics dummy class was in the wrong namespace. Linking on linux
did work because the GNU linker is more clever than the MSVC linker for this
particular example and throws out more unused code, so that theFontMetrics was
not referenced at all.
The computation of length on screen depend in particular of the computation of the size of an em. Many places of the code used to rely on the width of the M character, which is not really correct:
http://en.wikipedia.org/wiki/Em_%28typography%29
In digital typography, the best value to use is the point size of the font.
* Implement FontMetrics::em(), which returns the value in pixels of the EM unit.
Convert code to use it.
* Introduce Length::inPixel(MetricsBase const &), which takes the textwidth and em information from the MetricsBase object. Convert code to use it.
* Fix several places where Length::inPixel is used without a proper em value.
* add mathed_font_em() helper function. It should eventually be removed like some other functions in MathSupport.
* Add dummy implementation of FontMetrics to tex2lyx for linking purposes.
This is the rersult of a discussion on the list. Now all special characters
have meaningful names, and it is clear that the LyX file syntax is not LaTeX.
tex2lyx did not use the InsetLayout for the script insets, so it did not know
that it had to use Plain Layout. However, there is still a possible pitfall:
InsetScript uses a hard coded plain layout for the InsetText constructor, so
if anybody would remove ForcePlain 1 and MultiPar false from the InsetLayout
then LyX would still write script insets with plain layout, but tex2lyx would
output standard layout again.
Previously, LyX did replace some words with typeset logos, and there was no
way to prvent this except putting them, in ERT (bug #4752). Now we have
special insets for these words, and standard text is left alone.
Previously, tex2lyx did not remove the \protect which is output by LyX in
front of \nobreakdash- if needed. Now tex2lyx removes it unconditionally (like
it does elsewhere), because LyX will add it if needed.
Previously, consecutive dashes in .lyx files were combined to endash and emdash
in some cases, and in other cases they were output as is. This made the code
complicated, and resulted in inconsitencies ((bug #3647).
Now, a dash in a .lyx file is always a dash in the output, for all flavours.
The special handling is moved to the input side, so that you still get an
endash if you type two hyphens. If needed, this can be changed or made
customizable without the need to update the file format again. Many thanks
for the fruitful mailing list dicsussion, which contributed significantly to
the final version.
The textcyr test did not work anymore, since the output of
"CYRILLIC SMALL LETTER VE" changed from \textcyr{\char226} to \textcyr{\cyrv}.
The the old version is now converted to ERT. In theory it could also be
converted to "CYRILLIC SMALL LETTER VE", but this would mean to maintain an
extra list, which is not worth the effort.
This is needed since src/support calls lots of qt code, and some parts of it
(e.g. QFileInfo) require a running event loop. This fixes bug #4917 which is
a symptom of the problem.
The fix is to create a QCoreApplication for tex2lyx, lyxclient and LyX running
without GUI. In the future this could be extended, i.e. to split up the
frontend Application class in a GUI and a console class, and having the
console class use LyXConsoleApp in the same way as Application now uses
GuiApplication. If that is done, one could also think of moving
support/ConsoleApplication to frontend/console/ConsoleApplication.
This is the first part of bug #8553. \textgreek was interpreted as the
polyglossia version even if polyglossia was not used. Now it is not recognized
at all and converted to ERT, but this ensures at least correct LaTeX output.
Previously tex2lyx did ignore table columns if the column specifier was
unknown. This can lead to data loss, and now it does not ignore these columns,
but assumes that the column specifiers use only once character. This can still
lead to data loss, but less often.
If the user gave the -fixednec argument to tex2lyx then set that encoding as
LaTeX input encoding in the generated LyX file. Otherwise, included .bib files
could be interpreted using a wrong encoding.
Now we can handle all types of captions defined in layout files or modules.
This does also mean that \captionabove and \captionbelow are only recognized
in KOMA-classes and not always anymore, but this is correct, since these
captions would not work in other document classes anyway.
This fixes the tex2lyx test test-refstyle-theorems.tex. It seems that the
intent of the fix was to remove a \protect in front of a \caption,
\captionabove or \captionbelow, but the implementation did not really do that.
Furthermore, it is not clear in which cases a \protect in front of a caption
needs to removed, and in which cases it needs to be kept: After looking at the
LyX sources I could not see that caprions are always output with \protect.
These are now in version.cpp. The build machinery should therefore make sure
that version.cpp is recompiled at every compilation.
These variables are now referred to by the other places that made use of __DATE__ and __TIME__.
If we are testing tex2lyx the first time, then the userdir
is empty. We call now configure.py to initalize in case it is needed.
The algorithm to detect if reconfigure is needed mimics the lyx behaviour.
The option --enable-qt5 allows configuring for Qt5. The default is Qt4.
Nothing special is done with respect to Qt4, apart from pulling in the
correct libraries. Indeed, other than the core and gui libraries, now
also the concurrent and widgets libraries are needed.
This is a follow-up of bug #8967. The implementation is self-explaining, the
only part which needs a comment is lyx2lyx: Since a 100% correct solution is
not possible, it has been decided not to switch amsmath off in the forward
conversion if no other ams command than \smash[t] and \smash[b] is used, but
to consider it a bug that older versions do not load amsmath automatically for
these commands. In the backward direction it is easy to keep the document
compilable, so just do that.
The algorithm used for breaking a paragraph in LaTeX export is changed
for avoiding spurious blank lines causing too much vertical space.
This change is tied to the introduction of a new inset (with two
different specializations) helping in either outputing LaTeX paragraph
breaks or separating environments in LyX. Both of the above goals were
previously achieved by the ---Separator--- layout and can now be
accomplished by the new inset in a more natural way. As an example,
after leaving an environment by hitting the Return key for two times,
a third return automatically inserts a parbreak inset, which is
equivalent to the old separator layout, i.e., it also introduces a
blank line in the output. If this blank line is not wanted, the
parbreak separator can be changed to a plain separator by a right
click of the mouse. Of course, an environment can still be separated
by the following one by using the Alt+P+Return shortcut (or the
corresponding menu key), but now the plain separator inset is used
instead of the old separator layout, such that no blank line occurs in
the LaTeX output.
Old documents are converted such that the LaTeX output remains unchanged.
As a result of this conversion, the old separator layout is replaced by
the new parbreak inset, which may also appear in places where the old
algorithm was introducing blank lines while the new one is not.
Note that not all blank lines were actually affecting the LaTeX output,
because a blank line is simply ignored by the TeX engine when it occurs
in the so called "vertical mode" (e.g., after an alignment environment).
The old ---Separator--- layout is now gone and old layout files using it
are also automatically converted.
Round trip conversions between old and new format should leave a document
unchanged. This means that the new behavior about paragraph breaking is
not "carried back" to the old format. Indeed, this would need introducing
special LaTeX commands in ERT that would accumulate in roundtrip
conversions, horribly cluttering the document. So, when converting a
modified document to old formats, the LaTeX output may slightly differ in
vertical spacing if the document is processed by an old version of LyX.
In other words, forward compatibility is guaranteed, but not backwards.
In C++98 std::istream does not use an operator bool(), but an operator
void*() instead, which prevents some unwanted conversions (this is one
possible implementation of the safe bool idiom).
In C++11 std::istream uses explicit operator bool, which prevents the unwanted
conversions using a new language feature.
This change does not have any effect on correct code, but prevents some
mistakes.
* remove unused class TexStream.
* remove unused virtual method Inset::cellXOffset
* remove second argument of FileDialog constructor, which was actually
not used
* remove some dead local code
* remove some unused private members of classes
* in InsetMathNest::updateBuffer, fix the logic of a test
This patch improves the cmake bundling process:
- reworked bundle handling for OS X: only when installing resources are copied, otherwise
only the smallest bundle is built
- on OS X, the utility programs (tex2lyx) are now installed in the right location
- it removes some unneeded BUNDLE DESTINATION
- it provides a basic support for QT plugins inclusion
- it properly builds a disk image on OS X (only the background image is missing)
- it fixes the library paths for all executables (not only LyX)
- Use the COPYING file for cmake install license
writer2latex surrounds quotes by braces which we skip to avoid useless ERT.
I broke this in 25fe87e5 which made Parser::next_next_token() not recognize
that it needed to parse one more token. If we had a unit test for the Parser
class it would probably have detected that. Now we have at least a test for
the special quote.
If we call tex2lyx on a temporary file created from the clipboard, the
file is always in utf8 encoding, without any temporary changes, even if it
contains encoding changing LaTeX commands. Therefore, we must tell tex2lyx
to use a fixed utf8 encoding for the whole file, and this is done using the
new latexclipboard format. Previously, tex2lyx thought the encoding was
latin1.
As a side effect, the -e option is now also documented in the man page.
The tests are now more robust if ctest uses '-j' (number of threads) parameter.
a.) keytests are running in sequence and in one thread only, no other thread running
b.) tex2lyx tests are locking in respect to each other
c.) Test gets one or more labels, so that we can select
ctest -L url
to run tests labeled url
d.) New macro settestlabel() to add labels to a test
When comparing the contents of two lyx-files, we should not care about the specific version. Later, this can be extended to also not care whether the file is produced by tex2lyx, lyx2lyx, or LyX.
Modified-By: Vincent van Ravesteijn <vfr@lyx.org>
LyX, lyx2lyx and tex2lyx produce now all the same version indicator consisting
only of the major and minor version. It is not decided yet whether future
development versions will add a -dev suffix, but for 2.1.0 this change fixes
the inconsistencies.
Unfortunately automake does not support file names with spaces for files in
EXTRA_DIST: These file names are put into a shell list, and it is impossible
quote the name in a way that works both with the shell and with make.
Therefore I removed the corresponding test file.
In roundtrip mode, tex2lyx produces documents with extension lyx.lyx, so that
the original files are not overwritten on re-export. This was not done for
included documents which were converted, and this broke 'make dist'.
* InsetBox and GuiBox: Use proper empty length instead of the broken -9.99col% trick
* some slight changes to the logic of GuiBox to make sure that values are set as needed.
* lengthToWidget(): handle properly the empty length case. All the other related Qt helpers did it already, it was probably an oversight. Also set the default_unit parameter as optional (not needed in this patch actually, but I got carried away :)
* allow generating LaTeX code for an empty length, since some broken code does that.
The default citation capability of LaTeX is not a true numerical
citation engine, rather it uses a mixture of labels/numbers. Thus
we now distinguish them: "numerical" always increments the bibitem
counter and uses its value as a numerical citation label, while
"default" only uses the bibitem counter when no label is provided.
LyX file format incremented to 471.
These should be used if any new style needs to be introduced in the stable
2.1 series: If the ForceLocal flag of the style is set, it will always be
written to the document header, so that even older 2.1 versions can read
and correctly output the document.
When running the test, you'll get the following messages:
Testing ../../src/../lib/layouts/siamltex.layout...
Layout.cpp (268): Cannot copy unknown style `Enumerate'
Testing ../../src/../lib/layouts/svglobal.layout...
Cannot delete style `Dedication'
Testing ../../src/../lib/layouts/svjog.layout...
Cannot delete style `Dedication'
Testing ../../src/../lib/layouts/svprobth.layout...
Cannot delete style `Dedication'
These are no errors of Layout::write(), but they indicate problems in the
definition of the layout files.
Kornel, it would be nice if you could do the cmake part.
- fileformat change
- it was a pity that LyX did not yet support a simple rectangular frame without a defined width but LyX did this for e.g. oval frames
- \fbox and \mbox often occur in TeX files and can now be imported
* Powerdot now also uses the native overlay item arguments
* a list option argument is finally available
* \pause natively supported (like in beamer)
* support for \onslide (via InsetFlex)
* support for \twocolumn
File format change.
With this commit, old beamer frames are converted to new ones. The old styles are removed (including the infamous \lyxframe).
This should be tested with as much beamer documents as possible (I have already done so), also, tex2lyx now probably produces invalid LyX files.
I added it because of a misunderstanding. If the encoding is really set too
late, to_utf8() in the Token constructor would probably fail, but a non-empty
putback buffer is no problem, since it was always created via to_utf8() ->
from_utf8().
This fixes bug #8554 and some recently introduced busg:
- Encodings::fromLaTeXCommand() can now handle all combining characters,
not only the one letter ones
- The remainder returned from Encodings::fromLaTeXCommand() must never be
thrown away in tex2lyx, but output as ERT
- No special case for combining diacritical marks needed anymore in parse_text()
- No special cases for accents and IPA combining diacritical marks needed
anymore in parse_text()
- special tipa short cuts may only be recognized if the tipa package is loaded
- Use requirements returned by Encodings::fromLaTeXCommand() instead of
hardcoded registering of tipa and tipax
- Get rid of the name2 variable in parse_text(): We must use name, otherwise
the extra stuff that might have been put into name vanishes
tex2lyx skips LyX preamble code only if it thinks that the file was created
by LyX (i.e. special comments are found). I don't like that, but this is
how it works and therefore the special comment neds to be added if the
theorem commands are to be skipped. This header makes also the temporary
setting of in_lyx_preamble obsolete, which caused skipping of user preamble
commands, which were not re-added by LyX.
- Convert prettyref to the autopackage mechanism
- Do not load refstyle automatically if some refstyle preamble code of LyX
is found, since LyX will only load the package if an actual reference
command is used. This is needed for mixed refstyle/prettyref documents.
- Only recognize refstyle commands if refstyle was detected in the preamble
- Only recognize prettyref commands if prettyref was detected in the preamble
- Add a mixed refstyle/prettyref test case
* use verbatimStuff for parsing chunks and make try to follow closely the sweave sytax constraints.
* merge the two cases for parsing << (noweb or quote)
* \verb|ff| requires that its parameter is on a single line.
- support for Cyrillic characters
- support for \textifsymbol and \ascii (fixes bug #8556)
- support for \ding
- tex2lyx/text.cpp: correct an indentation and use "name2" because "name" is already defined in this clause
- for all TIPA characters the tipa and tipx package must always be registered because these characters are also allowed outside a TIPA inset
- include \textvertline because we again reached the maximum number of if statements allowed by MSCV (so safe at least one for now)
The LyX file format only supports one CJK font mapping per document.
Therefore all CJK environments which use a different mapping than the main
one are output in ERT. THis is now explained in the test case, and also the
converion to ERT is improved (although it is still wrong e.g. for paragraphs).
