mirror of
https://git.lyx.org/repos/lyx.git
synced 2024-09-19 14:30:32 +00:00
132 lines
5.0 KiB
Plaintext
132 lines
5.0 KiB
Plaintext
The ctests are not convenient tests. In this file we can attempt to log the
|
|
benefits and costs to using them so we can periodically evaluate which tests we
|
|
should keep and which we should get rid of.
|
|
|
|
Below are some objective costs:
|
|
|
|
- The time it takes to run the ctests.
|
|
- The email discussion back-and-forth.
|
|
|
|
There are also some non-objective costs. The ctests are annoying and not fun to
|
|
maintain, discuss, and fix. Although it's hard to measure this "annoyance"
|
|
cost, it should be taken into account when we review.
|
|
|
|
Below we can try to keep a log of which bugs are found with which ctests. This
|
|
way we can see which tests are the most useful. It is difficult to objectively
|
|
differentiate between minor bugs and important bugs.
|
|
|
|
The "Benefit" row gives the amount of days in-between when an issue is found
|
|
because of the ctests and when an issue is reported by someone not using the
|
|
ctests.
|
|
|
|
The "ctest" row gives a ctest pattern for the tests that were affected by the
|
|
regression.
|
|
|
|
Examples where convergence tests took time and annoyance to figure out that
|
|
they were actually "false positives" (i.e., no bug that is worth the time or
|
|
complexity to fix):
|
|
|
|
Report : https://www.mail-archive.com/search?l=mid&q=20210125200655.e27xaaoy2belyxhw%40tallinn
|
|
Report : https://www.mail-archive.com/lyx-devel@lists.lyx.org/msg218248.html
|
|
|
|
Testing compilation after lyx2lyx roundtrip was not helpful in this case:
|
|
report : https://www.mail-archive.com/lyx-devel@lists.lyx.org/msg218232.html
|
|
ctests : de/UserGuide_lyx(22|23)
|
|
|
|
|
|
Examples where an upstream issue was reported and ended up wasting time:
|
|
|
|
Report : https://github.com/latex3/latex2e/issues/836#issuecomment-1159826566
|
|
ctests : export/templates/Obsolete/.*American_Geophysical_Union.*_(lyx22|lyx23|pdf|dvi).*
|
|
|
|
|
|
Examples where bugs were fixed (more quickly) because of ctests:
|
|
|
|
Report : https://www.mail-archive.com/search?l=mid&q=20210126190837.pwuxhrhrikzszfrh%40tallinn
|
|
Fix : 92f8612a
|
|
|
|
Report : https://www.mail-archive.com/search?l=mid&q=b0a33192bc5635bd1cb5596e346f9c34615ffeb4.camel%40lyx.org
|
|
Fix : ab1d418f
|
|
|
|
Fix : ef2b2827
|
|
|
|
Fix : 1ddc1c6b
|
|
|
|
Report : https://www.mail-archive.com/search?l=mid&q=20210218051838.jbwnemfv5ilhlsls%40tallinn
|
|
Fix : 358e4ace
|
|
|
|
Fix : 3df42924
|
|
|
|
Fix : 67d0b80f
|
|
|
|
Fix : 8ce6c815
|
|
|
|
Report : https://www.mail-archive.com/search?l=mid&q=20210523014926.xewdr5x62br2okvy%40tallinn
|
|
Fix : 39a9e370
|
|
|
|
Report : https://www.mail-archive.com/search?l=mid&q=20210528001413.56phju7dvfn4dexs%40tallinn
|
|
Fix : ffa6a724
|
|
|
|
Report : https://www.mail-archive.com/search?l=mid&q=20210529174524.idnmt5w4tvhuvyy6%40tallinn
|
|
Note1 : We are still not sure what to do regarding how to make it more user-friendly
|
|
for users to realize that "-dALLOWPSTRANSPARENCY" is needed for the ps2pdf converter.
|
|
Benefit: 14 days (https://www.lyx.org/trac/ticket/12303)
|
|
|
|
Report : https://www.mail-archive.com/search?l=mid&q=20211205190238.edrtqmg3fkdr7gbo%40poole
|
|
Fix : 489bf46a, e3489d54
|
|
|
|
Report : https://www.mail-archive.com/search?l=mid&q=20220205184128.4clckanaufwmx5z4%40gary
|
|
Fix : d66c27de
|
|
Report : https://www.mail-archive.com/search?l=mid&q=20220206142559.rpgwki7wruhtasxp%40gary
|
|
Fix : f71f71d0
|
|
|
|
Report : https://www.mail-archive.com/search?l=mid&q=20220310131706.qdthdjblfpme2wew%40gary
|
|
Fix : dc502c82
|
|
ctests : examples/Welcome_pdf2
|
|
|
|
report : https://www.mail-archive.com/lyx-devel@lists.lyx.org/msg218232.html
|
|
Fix : b8502a3e, e316e21a
|
|
ctests : de/UserGuide_lyx(16|20|21)
|
|
|
|
report : https://www.mail-archive.com/search?l=mid&q=20221218210348.gkwloeu74zjewqsu%40gary
|
|
Fix : c2a324e24
|
|
ctests : DEFAULTOUTPUT_export/export/latex/CJK/zh_CN-en-de-el-ru_language-default_pdf2
|
|
|
|
|
|
|
|
Examples where upstream (e.g., LaTeX packages) bugs were fixed (more quickly)
|
|
because of ctests:
|
|
|
|
Fix : https://github.com/nrnrnr/noweb/pull/12#event-4892099481
|
|
|
|
Report : (private message to Jürgen: Message-ID:<20211205192054.zbxij5butj3wdyvf@poole>)
|
|
Fix : some of the polyglossia commits in the range 1c36dcdc^...bf69288c
|
|
Benefit: Not clear. The regressions were in the development version so they
|
|
could have easily been caught before release.
|
|
|
|
Report : https://www.mail-archive.com/search?l=mid&q=20211207200100.o27ojaftehrjfcza%40poole
|
|
Fix : 2eaf30c5
|
|
|
|
|
|
Report : (private message to Jürgen: LyXMessage-ID:<20220216020703.la2fd5gu7yz7y7dc@gary>)
|
|
Fix : https://github.com/reutenauer/polyglossia/commit/c7489ebaf8c66a7e4fe2529de7068ebb79c68812
|
|
Benefit: Not clear. The regressions were in the development version so they
|
|
could have easily been caught before release.
|
|
ctests : export/export/lyx2lyx/revert-languages-polyglossia-primary_lyx22
|
|
|
|
|
|
Report : https://github.com/latex3/latex2e/issues/836
|
|
Fix : https://github.com/latex3/latex2e/commit/878a4715df024fc3425d2bfd5d3d138b9395c178
|
|
ctests : doc/EmbeddedObjects
|
|
|
|
|
|
Report : https://github.com/gsilano/EuropeCV/issues/29
|
|
Fix : none yet
|
|
ctests : Modern_CV
|
|
|
|
|
|
Report : https://github.com/schlcht/microtype/issues/21
|
|
upstream: https://github.com/latex3/latex2e/issues/880
|
|
Fix : none yet
|
|
ctests : export/templates/Articles/Hebrew_Article_.*(dvi|pdf|pdf2|pdf3)
|