Prevent encoding changes whenever the TeX engine is XeTeX or LuaTeX,
as XeTeX/LuaTeX use only one encoding per document:
* with useNonTeXFonts: "utf8plain",
* with XeTeX and TeX fonts: "ascii" (inputenc fails),
* with LuaTeX and TeX fonts: only one encoding accepted by luainputenc.
+1 no needless encoding switches
+1 runparams.encoding matches the correct encoding at any time
+1 less complicated code.
-1 there may still be problems with CJK (possibly impossible to
solve for Xe/LuaTeX with TeX fonts).
For LuaTeX & TeX fonts, the complete document uses the encoding
of the global document language.
See also #9740.
Actually, the changed tests were used to prevent overwriting the encoding
changed in Buffer::writeLaTeX with a language-default encoding.
This is still required for XeTeX with TeX-fonts unless a proper solution is found.
Documents with more than one encoding and TeX-fonts fail with LuaTeX,
as "luainputenc" can only handle one encoding.
With inputenc == "auto" or "default", the encoding changes with
the language and must be reset after an eventual language switch in insets
or environments (see #6216).
However, whether we need to do this does not depend on 8-bit TeX vs. LuaTeX
but on the possible use of more than one encoding for the document.
With "nonTeXFonts", the encoding is utf8,
LuaTeX with TeX fonts requires encoding handling similar to 8-bit TeX.
(Additionally, the value of "params.inputenc" could be tested: if it is
not "auto" or "default", we have just one common encoding and could skip
the reset as well.) Not sure how much time this saves, though.
This is a tiny simplification that makes understanding the code more easy and
will help in fixing the remaining regressions. The logic remains the same, no
export test result is changed.
Fixes output for 3 of the 4 test lyx-files.
Includes "FIXME"s at places where further action is required to get the XeTeX
export right but I don't know how.
b1c68dccf8 and 46aed6d2b9 fixed some language nesting issues, but introduced
a regression for the case that there is a standard paragraph in a foreign
language, followed by a list (e.g. itemize) in the same language, followed
by the end of the document, as e.g. in lib/doc/de/Additional.lyx. The reason
for this was that not all language ending commands did reset
state->open_polyglossia_lang_ correctly.
I am sure that one can still construct broken corner cases, and I am also sure
that this was already possible before b1c68dccf8 and 46aed6d2b9. However,
this fix seems to fix the most important issues, and to get nesting completely
correct we would probably need some stack-like structure, for languages and
encodings, also for the CJK part (which is not touched at all by this commit).
The code that sets open_polyglossia_lang_ is not only executed for polyglossia,
but also for babel, so we have to use the correct language end command.
* TexRow now computes rows from a DocIterator. In practice, the cursor
highlighting is now correct inside insets, it is no longer restricted to the
topmost level. It certainly also makes forward-search more precise.
* Added the option to disable a texrow when not needed, for perf.
* Fixed a bug where the last paragraph was not properly highlighted.
Limitations:
* TexRow still does not handle: math (e.g. multi-cell), sub-captions, inset
arguments.
There are still a few warnings of the kind
(style) Variable 'x' is assigned a value that is never used.
since I did not touch code where I was not sure whether there might be a real
bug, and I kept some for symmetry reasons as well.
Newer boost versions use complicated type traits for boost::next and
boost::prior, which do not work with the RandomAccessList iterators.
The long term solution is to use std::next and std::prev, for now supply
simple replacements for compilers that do not support C++11 yet.
This is one of the more important threadsafety issues because of export in
thread and simultanous view source. The solution is ugly, but a better one
(see FIXME) would require major rework. These static variables should not
have been used in the first place IMHO.
When deciding whether a paragraph should be indented or not, LyX
only takes into account default layouts. This is wrong, because
an environment could be nested into another one and thus a following
paragraph would not be "default". With this patch all paragraphs
after an environment are correctly indented, independently of
whether their layouts are "default" or not.
The latex output (which was modeled following the previous wrong
assumption) is also correspondingly adapted.
This is necessary, for example, if a standard paragraph is nested
in an environment and the environment does not end immediately after.
To be strictly correct, the layout of the following paragraph should
be compared to the layout of the nesting environment, otherwise, if
they are not the same, an empty line is nevertheless output. However,
this is harmless because an "\end{layout}" tag immediately follows.
I am not sure I fully understand the pending_newline/unskip_newline
logic (which seems mainly related to rtl writing), so I prefer to
leave it alone, in the sense that now things go again as in 2.1
until the point where those booleans are used for producing output.
If it turns out that a spurious (and unwanted) empty line comes
from the previous code, it can be easily corrected later.
I am also reintroducing the check about a separator inset at the end
of the paragraph, because that is necessary for the plain version.
LyX assumes that a standard paragraph following an aligned one or
a layout with NextNoIndent==false has to be indented on screen.
This means that in the latex output a blank line has to follow.
In this case there should be no problem as regards extra vertical
space and it simpler and more elegant to simply uncheck "Indent
Paragraph" in the Paragraph settings pane rather than changing
the current logic and allowing to insert a parbreak separator.
This is what LyX was previously doing. It has no effect on vertical
spacing but, for example, sectioning commands stand out on the output.
The parbreak is not output if an environment follows or the alignment
of the current or next paragraph is changed.
Also remove some superfluous code.
The algorithm used for breaking a paragraph in LaTeX export is changed
for avoiding spurious blank lines causing too much vertical space.
This change is tied to the introduction of a new inset (with two
different specializations) helping in either outputing LaTeX paragraph
breaks or separating environments in LyX. Both of the above goals were
previously achieved by the ---Separator--- layout and can now be
accomplished by the new inset in a more natural way. As an example,
after leaving an environment by hitting the Return key for two times,
a third return automatically inserts a parbreak inset, which is
equivalent to the old separator layout, i.e., it also introduces a
blank line in the output. If this blank line is not wanted, the
parbreak separator can be changed to a plain separator by a right
click of the mouse. Of course, an environment can still be separated
by the following one by using the Alt+P+Return shortcut (or the
corresponding menu key), but now the plain separator inset is used
instead of the old separator layout, such that no blank line occurs in
the LaTeX output.
Old documents are converted such that the LaTeX output remains unchanged.
As a result of this conversion, the old separator layout is replaced by
the new parbreak inset, which may also appear in places where the old
algorithm was introducing blank lines while the new one is not.
Note that not all blank lines were actually affecting the LaTeX output,
because a blank line is simply ignored by the TeX engine when it occurs
in the so called "vertical mode" (e.g., after an alignment environment).
The old ---Separator--- layout is now gone and old layout files using it
are also automatically converted.
Round trip conversions between old and new format should leave a document
unchanged. This means that the new behavior about paragraph breaking is
not "carried back" to the old format. Indeed, this would need introducing
special LaTeX commands in ERT that would accumulate in roundtrip
conversions, horribly cluttering the document. So, when converting a
modified document to old formats, the LaTeX output may slightly differ in
vertical spacing if the document is processed by an old version of LyX.
In other words, forward compatibility is guaranteed, but not backwards.
for possible thread conflicts, of the sort Georg resolved at
6a30211f. I have made static variables const where possible,
and marked cases that looked potentially problematic with the
comment:
// FIXME THREAD
Many of these definitely are vulnerable to concurrent access, such
as the static variables declared at the start of output_latex.cpp.
Suppose, e.g., we were outputting latex and also displaying the
source of a different document.
I'd appreciate it if others could grep for "FIXME THREAD" and see
if some of these are harmless, or what.
each failure.
There are several places I was not sure what to do. These are marked
by comments beginning "LASSERT:" so they can be found easily. At the
moment, they are at:
Author.cpp:105: // LASSERT: What should we do here?
Author.cpp:121: // LASSERT: What should we do here?
Buffer.cpp:4525: // LASSERT: Is it safe to continue here, or should we just return?
Cursor.cpp:345: // LASSERT: Is it safe to continue here, or should we return?
Cursor.cpp:403: // LASSERT: Is it safe to continue here, or should we return?
Cursor.cpp:1143: // LASSERT: There have been several bugs around this code, that seem
CursorSlice.cpp:83: // LASSERT: This should only ever be called from an InsetMath.
CursorSlice.cpp:92: // LASSERT: This should only ever be called from an InsetMath.
LayoutFile.cpp:303: // LASSERT: Why would this fail?
Text.cpp:995: // LASSERT: Is it safe to continue here?
This default argument is inserted iff no inset argument is present. This is useful particularly for mandatory arguments that need to have a sensible default value.
Actually, the test case showed several problems:
- ERT insets did use layout "Standard", not "Plain Layout"
- The font scale was read correctly, but tex2lyx claimed that it did ignore
the option "scaled=0.95"
- If a third argument of the CJK environment was given, it caused the whole
environment to be put in ERT with a broken encoding. This is now fixed for
the bug test case by using the \font_cjk header variable, but the encoding
problem still exists for unsupported encodings. I'll file a separate bug
for that.
- The CJKutf8 package was not handled in the preamble parsing. Therefore the
chinese comment in the preamble was read with a wrong encoding, and guessing
the document language did not work.
The new file CJKutf8.tex was created by copying and modifying CJK.tex, but
unfortunately it is impossible to tell git to inherit the history of CJK.tex
for the new file (search the web for git svn copy if you want to know details).
While cppcheck did not turn out any suspicious error messages, using
the "performance" flag highlighted several nitpicks in three categories
* do not use it++ for iterators, ++it is better
* do not use size() to test for emptyness, empty() is here
* do not use "const T" as a function parameter, "const & T" is better
I doubt that any of these is a real performance problem, but the code is cleaner anyway.
- Interpret argument of LFUN_SPACE_INSERT correctly
- Use InsetMathSpace instead of InsetMathSpecialChar for "\ " (bug # 7728)
- Use InsetMathSpace instead of InsetMathChar for ~ (bug # 7728).
This fixes also the display in LyX (previously a literal ~ was displayed).
Using InsetMathSpace enables also the "Insert Formatting" menu entries.
No file format change is needed, since the LaTeX export is unchanged.
Note that there are still some bugs related to spaces in math:
#7746, #7747, #7749, #7842
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@39947 a592a061-630c-0410-9148-cb99ea01b6c8
counting when exporting to latex. This is done for the code comprised
between \begin{document} and \end{document}, while the preamble code
still needs manual calls to TexRow::newline() for registering new lines.
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@37584 a592a061-630c-0410-9148-cb99ea01b6c8
blank lines may be inadvertently output. This is achieved by using two
special iomanip-like variables (breakln and safebreakln) in the lyx::
namespace. When they are inserted in the stream, a newline is output
only if not already at the beginning of a line. The difference between
breakln and safebreakln is that, if needed, the former outputs '\n'
and the latter "%\n".
In future, the new class will also be used for counting the number of
newlines issued. Even if the infractrure for doing that is already in
place, the counting is essentially still done the old way.
There are still places in the code where the functionality of the
class could be used, most probably. ATM, it is used for InsetTabular,
InsetListings, InsetFloat, and InsetText.
The Comment and GreyedOut insets required a special treatment and a
new InsetLayout parameter (Display) has been introduced. The default
for Display is "true", meaning that the corresponding latex
environment is of "display" type, i.e., it stands on its own, whereas
"false" means that the contents appear inline with the text. The
latter is the case for both Comment and GreyedOut insets.
Mostly, the only visible effects on latex exports should be the
disappearing of some redundant % chars and the appearing/disappearing
of null {} latex groups after a comment or lyxgreyedout environments
(they are related to the presence or absence of a space immediately
after those environments), as well as the fact that math environments
are now started on their own lines.
As a last thing, only the latex code between \begin{document} and
\end{document} goes through the new class, the preamble being directly
output through odocstream, as usual.
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@37360 a592a061-630c-0410-9148-cb99ea01b6c8
* only one for loop
* uses runparams.par_end
InsetInclude and InsetIext: initialize runparams.par_begin and runparams.par_end
Except for a few additonal empty lines the LateX exports are identical for all help files. The reason for the additional empty lines is the more consistent TeXEnvironment() call.
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@36934 a592a061-630c-0410-9148-cb99ea01b6c8
This introduces an isFullUnicode() function to OutputParams that currently only contains XETEX, but later can also contain other fully unicode-aware backends (LUATEX for instance).
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@36445 a592a061-630c-0410-9148-cb99ea01b6c8
In particular, it makes paragraph breaks generate single \n in latex output
when ParbreakIsNewline is true
This means that it is not necessary anymore to use newlines to break lines.
Plain paragraph breaks can be used instead, like is done now in ERT/Listings.
This is mainly aimed at sweave support.
lyx2lyx support courtesy of Richard Heck
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@36163 a592a061-630c-0410-9148-cb99ea01b6c8
The goal is first to remove explicit testing against ERT_CODE,
and second to allow to define layouts with the same properties as ERT insets.
This commit does not change behaviour
* rename OutputParams::verbatim to pass_thru
* disable encoding switch inside pass_thru layouts
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@35091 a592a061-630c-0410-9148-cb99ea01b6c8
the OptionalArgs tag does and is implemented by the now misnamed
InsetOptArgs, except that its content gets wrapped in "{}" rather than
"[]". Required arguments do not actually have to be provided, but they
are always output.
This will allow e.g. beamer's Block environment to be implemented
without ERT.
Documentation to follow.
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@34591 a592a061-630c-0410-9148-cb99ea01b6c8
When deciding whether to switch encoding or not, the "fragileness" of
the enclosing environment (rather than that of the actual one) should
be taken into account, as we already account for the current "fragility"
by outputting the encoding switch just before switching environment.
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@33882 a592a061-630c-0410-9148-cb99ea01b6c8
Fixes bug #5690: Hebrew file cannot be exported to dvi or pdf.
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@30848 a592a061-630c-0410-9148-cb99ea01b6c8
Essentially, non-default aligned paragraphs should be treated as nested ones.
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@30766 a592a061-630c-0410-9148-cb99ea01b6c8
- slightly change FIXME comment. This only applies to the newly added condition.
The other conditions should be left untouched, even if we find a way to access the TeXRow.
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@27237 a592a061-630c-0410-9148-cb99ea01b6c8
(All of these commits, by the way, are just stuff I'm stumbling
across while trying to deal with the newline bug.)
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@27216 a592a061-630c-0410-9148-cb99ea01b6c8
I tested it carefully but in case this makes problems, please revert this immediately.
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@27205 a592a061-630c-0410-9148-cb99ea01b6c8
for entering math accents, such that an umlauted A is translated as \ddot{A},
for example.
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@26577 a592a061-630c-0410-9148-cb99ea01b6c8
unicodesymbols file. Also take into account that a unicode symbol
may appear in both mathmode and textmode and thus both the mathmode
and textmode features may be required. As a side effect, the problem
that the \lyxmathsym definition is added to the preamble even when
a unicode symbol appears only in textmode is solved.
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@26548 a592a061-630c-0410-9148-cb99ea01b6c8
* src/OutputParams.{cpp,h}:
- new param isLastPar, indicating if the current paragraph is the last in a given inset/buffer
* src/output_latex.cpp:
- set isLastPar
* src/Paragraph.cpp:
- noTrivlistCentering also in tabular cells
- in the last par, use alignment switch instead of environment
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@26443 a592a061-630c-0410-9148-cb99ea01b6c8
* src/Encoding.h:
- new member hasFixedWidth()
* src/output_latex.{cpp,h} (switchEncoding):
- add option to force encoding switch
* src/insets/InsetListing.cpp (latex):
- force encoding switch only in multibyte context, switch the encoding properly,
and add some error messages.
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@25730 a592a061-630c-0410-9148-cb99ea01b6c8
* lib/encodings:
* src/support/docstream.h:
- JIS is not a valid iconv name. Rename to ISO-2022-JP
* src/output_latex.cpp:
- fix wrong encoding switch.
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@25510 a592a061-630c-0410-9148-cb99ea01b6c8
if (!par->forceEmptyLayout())
but in fact things can be simplified more.
Cosmetics will follow. This patch makes the action more obvious.
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@24453 a592a061-630c-0410-9148-cb99ea01b6c8
The need for a LayoutPtr arises from the fact that (a) we do not want to give our clients a Layout &, since we do not want them to be able to change our Layout; but (b) we also need to be able to change which layout is ours. So we cannot store a Layout const &. Or so it seems to the compiler.
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@23522 a592a061-630c-0410-9148-cb99ea01b6c8
Sorry, Andre, but this broke not only the modules stuff but the general handling of TextClasses. I'm not opposed to doing this sort of thing, but it's going to be a little more complicated. I'll do it when I get a bit of time, or I can explain what the issue is here if you want to do it.
I'll separately re-commit some of the cleanup here.
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@23189 a592a061-630c-0410-9148-cb99ea01b6c8
Also a few cosmetical changes where I xould not resist.
Would be nice if the 'modules people' could verify that their baby
is strill alive...
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@23154 a592a061-630c-0410-9148-cb99ea01b6c8
from the one posted to the list.
The basic idea has two parts. First, we hard code an "empty layout"
(called PlainLayout, for want of a better name) in TextClass and read it
before doing anything else. It can therefore be customized by classes,
if they want---say, to make it left-aligned. Second, InsetText's are
divided into three types: (i) normal ones, that use the "default" layout
defined by the text class; (ii) highly restrictive ones, such as ERT and
(not quite an inset) table cells, which demand the empty layout; (iii)
middling ones, which default to an empty layout and use the empty layout
in place of the default. (This is so we don't get the same problem we
had with ERT in e.g. footnotes.) The type of inset is signaled by new
methods InsetText::forceEmptyLayout() and InsetText::useEmptyLayout().
(The latter might better be called: useEmptyLayoutInsteadOfDefault(),
but that's silly.) The old InsetText::forceDefaultParagraphs() has been
split into these, plus a new method InsetText::allowParagraphCustomization().
A lot of the changes just adapt to this change.
The other big change is in GuiToolbar: We want to show LyXDefault and
the "default" layout only when they're active.
There are a handful of places where I'm not entirely sure whether we
should be using forceEmptyLayout or !allowParagraphCustomization() or
both. The InsetCaption is one of these. These places, and some others,
are marked with FIXMEs, so I'd appreciate it if people would search
through the patch and let me know whether these need changing. If they
don't, the FIXMEs can be deleted.
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@22966 a592a061-630c-0410-9148-cb99ea01b6c8
* src/output_latex.{cpp,h}:
- (latexParagraphs, TeXEnvironment, TeXOnePar, TeXDeeper): pass Text element
instead of ParagraphList. This is necessary to detect whether we are in the main text.
- check if we are in the main text for language switches, if needed.
* src/Buffer.cpp:
* src/insets/InsetEnvironment.cpp:
* src/insets/InsetText.cpp:
* src/mathed/InsetMathMBox.cpp:
- adapt to new latexParagraphs definition.
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@22666 a592a061-630c-0410-9148-cb99ea01b6c8
directing to use memoir's capitalized latex macros.
* src/LaTeXFeatures.cpp (getPackages): act on feature "SetSpace".
* src/output_latex.cpp (TeXOnePar): ditto
* lib/layouts/memoir.layout: provide feature SetSpace, indicating that
memoir provides the setspace functionality, but with a different syntax.
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@22551 a592a061-630c-0410-9148-cb99ea01b6c8
* src/output_latex.{cpp, h}:
- Many changes in order to allow CJK in a mulitlingual context.
* src/Paragraph.cpp:
- Fix file encoding switch and language nesting with CJK.
* src/Buffer.cpp:
- Move the opening and closing CJK and lanaguage tags to ouput_latex.cpp.
* src/Font.cpp:
- switchEncoding call now takes runparams as argument.
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@22011 a592a061-630c-0410-9148-cb99ea01b6c8
Now support/* should have no dependencies on src/* anymore.
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@21851 a592a061-630c-0410-9148-cb99ea01b6c8
- prevent wrong whitespace by trailing '%' on line endings.
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@21742 a592a061-630c-0410-9148-cb99ea01b6c8
* Font::FontBits -> FontInfo
* Font::FONT_XXX -> all enums transfered to FontEnums.h and renamed to FontXxx
I've replaced Font uses with FontInfo were the language() member was not needed, basically all draw() and metrics methods. There's one problematic cases with InsetQuotes which I solved by taking the Buffer main language.
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@21240 a592a061-630c-0410-9148-cb99ea01b6c8
- simpleTeXOnePar(): renamed to latex(), beginning of de-spaghettization by truly splitting inset and character cases.
- appendChar(), appendString(): new method for buffer reading.
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@21117 a592a061-630c-0410-9148-cb99ea01b6c8
- Move Changes.h out of Paragraph.h
- pimpl the inset list.
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@21033 a592a061-630c-0410-9148-cb99ea01b6c8
(maily move layout related enums into a header of there own to remov
include dependencies, alos rename a few things)
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@20598 a592a061-630c-0410-9148-cb99ea01b6c8
- Modifier on return type ignored
- Ambigous else
- Ambigous logical operators
- Semi-colon instead of empty block
- Missing header files
- Missing using std::XX
- Initialization out of order
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@20304 a592a061-630c-0410-9148-cb99ea01b6c8
insets. Up until now this has not been 100% correct, especially in the case
of RTL languages (but not only those).
This fixes bug 1820 (http://bugzilla.lyx.org/show_bug.cgi?id=1820)
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@19150 a592a061-630c-0410-9148-cb99ea01b6c8
of \inputencoding commands, which are not needed in some cases (e.g., Hebrew
with ivritex).
This fixes some of the remaining problems (but not all) from bug 3613 (namely, http://bugzilla.lyx.org/show_bug.cgi?id=3613#c9).
See also some example files fixed by this in http://permalink.gmane.org/gmane.editors.lyx.devel/88805 .
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@18994 a592a061-630c-0410-9148-cb99ea01b6c8
Introduces new encodings for the CJK-languages.
The CJK-languages can now be selected from LyX's menu.
For remaining small problems, see the thread "Help needed for bug 3043" on the devel-list.
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@18216 a592a061-630c-0410-9148-cb99ea01b6c8
