Commit Graph

45 Commits

Author SHA1 Message Date
Georg Baum
de3e5280f6 Open tex2lyx input files with correct encoding
We open the input file now twice: The first time in latin1 encoding to read
the document encoding from the preamble. This does always work, since
traditional TeX does not allow non-ASCII contents without an encoding changing
command (except for comments, but we do not need them, and using latin1 rather
than utf8 ensures that they do not produce an iconv exception, but are simply
recored with wrong characters), and we do detect the utf8 based TeX engines
XeTeX and LuaTeX as well. The second time we open the file directly with the
document encoding.

This fixes a few tex2lyx tests on OS X, since changing the encoding of an
open file steam does not work with clang on OS X. Files using more than one
encoding are still broken, but all single-encoding files are fixed now.
2016-01-31 12:54:59 +01:00
Georg Baum
92d0835e14 Avoid encoding changes of open streams if possible
Changing the codecvt_facet of a file stream after the file has been opened
does not work with clang on OS X. Therefore we avoid it if possible (i. e. the
new encoding is the same as the old one).
2016-01-31 12:49:17 +01:00
Vincent van Ravesteijn
990446c0e1 msvc: Move implementation of numpunct<> functions to cpp file
These functions use lyx::from_ascii which means they need to include
docstring.h, but that results in a circular include.
2015-10-20 21:28:40 +02:00
Georg Baum
7b0485dbb2 Remove gcc 3 traces
We claim that gcc 4.x is needed in INSTALL, so it does not make sense to keep
this old stuff. Instead, I made configure output an error if gcc is too old.
2015-05-10 10:01:33 +02:00
Georg Baum
8f2828197c Move otexstream out of support
This resolves a dependency of src/support/docstream.cpp on src/TexRow.cpp,
which is the wrong way round. This fixes the linking
src/tests/check_ExternalTransforms with MSVC where the linker is not clever
enough to detect that the whole otexstream class is unused.
2015-01-09 21:07:59 +01:00
Enrico Forestieri
ee9ff6cb0c Extend the otexstream class to also report about paragraph breaks.
The new method afterParbreak() returns true if a blank line was just
output and we are at the beginning of the next line, false otherwise.
2014-05-18 22:46:33 +02:00
Enrico Forestieri
8a4b25875a Thinking about it, it is better to always convert to docstrings all
utf8 strings, and not only if they contain encoding changes. This is
because if the output encoding was previously changed and an utf8
string is inserted in the stream, the encoding changes cannot occur.
This was not a problem until now because normal strings could not be
inserted in a odocstream, as them would have been exchanged with encoding
changes. Indeed, the SetEnc struct has only a std::string member and
outputting a std::string would be interpreted by the compiler the same
as inserting setEncoding(std::string). However, a std::string can be
inserted in an otexstream and it is better to account for this.
I wonder whether trying "os << std::string", where os is an odocstream,
should produce an error instead of actually trying to change the stream
output encoding, but this has not been a problem until now...


git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@39944 a592a061-630c-0410-9148-cb99ea01b6c8
2011-10-23 14:24:17 +00:00
Enrico Forestieri
7fec736162 As stated in r39928, conversion on the fly only works for docstrings
and not for utf8 ones. But the simple solution is to convert to a
docstring before outputting to the underlying docstream.


git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@39930 a592a061-630c-0410-9148-cb99ea01b6c8
2011-10-23 03:04:09 +00:00
Enrico Forestieri
bbca759363 Restore r39921, as a simple solution exists.
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@39929 a592a061-630c-0410-9148-cb99ea01b6c8
2011-10-23 03:02:02 +00:00
Enrico Forestieri
936ae8020f Revert r39921. This would not work because proper facets for converting
on the fly are only available for ucs4 encoded strings and not for utf8
ones. So, it does only work with docstrings.


git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@39928 a592a061-630c-0410-9148-cb99ea01b6c8
2011-10-23 02:33:38 +00:00
Enrico Forestieri
5a69c96ccd Fix typo.
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@39922 a592a061-630c-0410-9148-cb99ea01b6c8
2011-10-22 18:29:11 +00:00
Enrico Forestieri
0cac034d10 Also account for possible encoding changes embedded in utf8 strings
other than docstrings.


git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@39921 a592a061-630c-0410-9148-cb99ea01b6c8
2011-10-22 18:27:56 +00:00
Enrico Forestieri
d703d90574 Fix bug #7835 (Relative paths are not supported by the TEXINPUTS prefix)
This restores \input@path handling, which turns out to be necessary, as
the TEXINPUTS mechanism is not used with relative paths. It turns out
that both methods must be used, because \input@path does not work in all
cases (most notably with tikz).


git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@39918 a592a061-630c-0410-9148-cb99ea01b6c8
2011-10-22 15:10:43 +00:00
Enrico Forestieri
6c85f8338f No need for the more expensive to_utf8() here.
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@39821 a592a061-630c-0410-9148-cb99ea01b6c8
2011-10-08 22:21:01 +00:00
Enrico Forestieri
33c5f6d628 Fix bug #7800 (Lyx cannot create dvi with Russian)
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@39820 a592a061-630c-0410-9148-cb99ea01b6c8
2011-10-08 22:13:38 +00:00
Enrico Forestieri
2de1163efc Fix bug #253 (Incorrect protection of closing quotation marks)
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@39420 a592a061-630c-0410-9148-cb99ea01b6c8
2011-08-05 11:23:00 +00:00
Enrico Forestieri
05491c58cc Also catch the endl manipulator for automatic line counting.
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@37657 a592a061-630c-0410-9148-cb99ea01b6c8
2011-02-14 17:09:39 +00:00
Enrico Forestieri
d866717ef7 Integrate texrow with otexstream in order to perform automatic line
counting when exporting to latex. This is done for the code comprised
between \begin{document} and \end{document}, while the preamble code
still needs manual calls to TexRow::newline() for registering new lines.


git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@37584 a592a061-630c-0410-9148-cb99ea01b6c8
2011-02-10 20:02:48 +00:00
Enrico Forestieri
ab9548d06d In order to avoid code replication, use a template-based solution for
all types requiring the exact same code, as suggested by Tommaso.
This allows to simply add a single-line instantiation declaration
if there's the need of adding another type requiring the same code.


git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@37366 a592a061-630c-0410-9148-cb99ea01b6c8
2011-01-29 19:44:53 +00:00
Enrico Forestieri
1ef605f625 Introduce a wrapper class for odocstream to help ensuring that no
blank lines may be inadvertently output. This is achieved by using two
special iomanip-like variables (breakln and safebreakln) in the lyx::
namespace. When they are inserted in the stream, a newline is output
only if not already at the beginning of a line. The difference between
breakln and safebreakln is that, if needed, the former outputs '\n'
and the latter "%\n".
In future, the new class will also be used for counting the number of
newlines issued. Even if the infractrure for doing that is already in
place, the counting is essentially still done the old way.
There are still places in the code where the functionality of the
class could be used, most probably. ATM, it is used for InsetTabular,
InsetListings, InsetFloat, and InsetText.
The Comment and GreyedOut insets required a special treatment and a
new InsetLayout parameter (Display) has been introduced. The default
for Display is "true", meaning that the corresponding latex
environment is of "display" type, i.e., it stands on its own, whereas
"false" means that the contents appear inline with the text. The
latter is the case for both Comment and GreyedOut insets.
Mostly, the only visible effects on latex exports should be the
disappearing of some redundant % chars and the appearing/disappearing
of null {} latex groups after a comment or lyxgreyedout environments
(they are related to the presence or absence of a space immediately
after those environments), as well as the fact that math environments
are now started on their own lines.
As a last thing, only the latex code between \begin{document} and
\end{document} goes through the new class, the preamble being directly
output through odocstream, as usual.


git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@37360 a592a061-630c-0410-9148-cb99ea01b6c8
2011-01-29 02:41:13 +00:00
Peter Kümmel
e948caf637 build with msvc10. Seems there is a bug in their STL code:
// std::numpunct has a hardcoded dllimport in definition, but we wanna it with 32 bit 
// so we can't import it and must define it but then the compiler complains.


git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@34262 a592a061-630c-0410-9148-cb99ea01b6c8
2010-04-22 14:28:52 +00:00
Jürgen Spitzmüller
d5a41946a6 * calculate the size of the iconv output buffer (bug 5951).
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@29735 a592a061-630c-0410-9148-cb99ea01b6c8
2009-05-19 15:27:54 +00:00
Jean-Marc Lasgouttes
fcc4a0f794 Sometimes, iconv on windows returns -1, but sets errno to 0. I do not
know what this means, but tex2lyx did not like it at all.


git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@29392 a592a061-630c-0410-9148-cb99ea01b6c8
2009-04-24 12:28:00 +00:00
Jean-Marc Lasgouttes
f3038bed06 Remove bogus ifdocstream constructor argument, which _may_ be the cause of
tex2lyx woes under windows. It may also help chktex under windows
(does it works currently?)


git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@29361 a592a061-630c-0410-9148-cb99ea01b6c8
2009-04-20 15:40:01 +00:00
Enrico Forestieri
bd8788f377 Fix bug 5802 (http://bugzilla.lyx.org/show_bug.cgi?id=5802)
Japanese char between math insets becomes wrong in TeX output


git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@28732 a592a061-630c-0410-9148-cb99ea01b6c8
2009-03-09 06:47:26 +00:00
Enrico Forestieri
0a2b62f468 According to http://en.wikipedia.org/wiki/ISO-2022-JP, a character may be
encoded using 1 or 2 bytes, but it may be prefixed by 3-byte escape sequence.
So, a single ucs4 char could need a maximum of 5 bytes. I think that it
is better to be safe than sorry...


git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@27645 a592a061-630c-0410-9148-cb99ea01b6c8
2008-11-20 19:32:22 +00:00
Enrico Forestieri
487af713fb Hrmpf.
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@27644 a592a061-630c-0410-9148-cb99ea01b6c8
2008-11-20 19:13:41 +00:00
Enrico Forestieri
b48de25bb1 Was forgetting this.
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@27643 a592a061-630c-0410-9148-cb99ea01b6c8
2008-11-20 19:05:47 +00:00
Enrico Forestieri
01c6c4879e After a better understanding of the problem, restrict the fix for
bugs 5216, 5280, and 5489 to the ISO-2022-JP encoding.



git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@27642 a592a061-630c-0410-9148-cb99ea01b6c8
2008-11-20 18:48:56 +00:00
Enrico Forestieri
07c0a6e496 Fix bug http://bugzilla.lyx.org/show_bug.cgi?id=5489 and at the same time
properly fix bugs 5216 and 5280. The best thing to do would be recognizing
at configure time a buggy iconv and #defining WORKAROUND_ICONV_BUG in
config.h, but I don't know how that could be done.


git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@27618 a592a061-630c-0410-9148-cb99ea01b6c8
2008-11-18 10:40:22 +00:00
Jean-Marc Lasgouttes
40f5f4e12f add support for changing encoding on idocstream
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@27579 a592a061-630c-0410-9148-cb99ea01b6c8
2008-11-16 19:25:26 +00:00
Jean-Marc Lasgouttes
0778007050 rename [io]docfstream to [io]fdocstream
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@27530 a592a061-630c-0410-9148-cb99ea01b6c8
2008-11-16 12:21:29 +00:00
Jürgen Spitzmüller
612e52b093 Back to CJK sanity I: Fix bug 4698 and 4696.
* lib/encodings:
* src/support/docstream.h:
	- JIS is not a valid iconv name. Rename to ISO-2022-JP

* src/output_latex.cpp:
	- fix wrong encoding switch.

git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@25510 a592a061-630c-0410-9148-cb99ea01b6c8
2008-07-09 07:21:02 +00:00
José Matox
03fb3b0aec Patch to please gcc 4.3. Less bloat in the include headers means that
we need to include explicitly some header files now.



git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@22846 a592a061-630c-0410-9148-cb99ea01b6c8
2008-02-07 17:04:06 +00:00
Abdelrazak Younes
58729e1117 Following rev 21967: final touch to odocfstream:
- odocfstream: properly fix plaintext output by getting rid of ctor with std::string argument as this can be mixed up with std::ofstream(std::string) ctor where the argument is a file. UTF8 is now the default encoding.
- PreviewLoader::Impl::startLoading(): catch another potential iconv exception.


git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@22332 a592a061-630c-0410-9148-cb99ea01b6c8
2007-12-29 18:04:43 +00:00
Abdelrazak Younes
9abb7db468 getting rid of superfluous std:: statements.
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@22104 a592a061-630c-0410-9148-cb99ea01b6c8
2007-12-12 19:28:07 +00:00
Abdelrazak Younes
9383f4c3c6 'using namespace std' instead of 'using std::xxx'
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@22097 a592a061-630c-0410-9148-cb99ea01b6c8
2007-12-12 10:16:00 +00:00
Abdelrazak Younes
974f44ea79 * docstream: factorize out some code and introduce odocfstream::reset()
* Buffer::makeLaTeXFile(): try to catch one more iconv exception. Fix bug 4385 (http://bugzilla.lyx.org/show_bug.cgi?id=4385)




git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@21967 a592a061-630c-0410-9148-cb99ea01b6c8
2007-12-05 13:56:53 +00:00
André Pönitz
9d0ea8aeff Move debug.{cpp,h}, Messages.{cpp,h} and gettext.{cpp,h} to support/.
Now support/* should have no dependencies on src/* anymore.



git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@21851 a592a061-630c-0410-9148-cb99ea01b6c8
2007-11-29 07:04:28 +00:00
André Pönitz
8d06b3049b cosmetics
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@21544 a592a061-630c-0410-9148-cb99ea01b6c8
2007-11-10 13:44:50 +00:00
Enrico Forestieri
4a35906da7 Fix problems with odocstream on "exotic" systems caused by the strfwd gimmick.
* src/support/docstream.{cpp,h}:
	Move insertion operator for char types from docstream.h to
	docstream.cpp and compile it only when USE_WCHAR_T is not defined.

	* src/support/strfwd.h:
	Implement forward declarations in standard C++ way.


git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@21513 a592a061-630c-0410-9148-cb99ea01b6c8
2007-11-08 06:05:19 +00:00
Jürgen Spitzmüller
402a13ba60 * src/support/docstream.cpp (iconv_codecvt_facet::do_max_length()):
- consider that not only utf8, but also most cjk encodings, are multibyte encodings
	  (fixes bug 4012)

git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@19076 a592a061-630c-0410-9148-cb99ea01b6c8
2007-07-14 13:00:24 +00:00
José Matox
5cbe5cf5ad FreeBSD libc uses UCS4, but libstdc++ has no proper wchar_t support compiled in (Jean-Marc)
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@18913 a592a061-630c-0410-9148-cb99ea01b6c8
2007-06-27 18:02:54 +00:00
Lars Gullik Bjønnes
897436efbb Whitespace cleanup
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@18550 a592a061-630c-0410-9148-cb99ea01b6c8
2007-05-28 22:27:45 +00:00
Bo Peng
f212b48335 Rename .C ==> .cpp for files in src/support, part two
git-svn-id: svn://svn.lyx.org/lyx/lyx-devel/trunk@18025 a592a061-630c-0410-9148-cb99ea01b6c8
2007-04-26 05:13:44 +00:00