mirror of
https://git.lyx.org/repos/lyx.git
synced 2024-11-07 12:32:26 +00:00
2dd9e1a338
The "benefit" row gives the amount of days in-between when an issue is found because of the ctests and when an issue is reported by someone not using the ctests. In the example of the ps2pdf flag, the issue was reported just a couple of weeks after we found it with the ctests, indicating that the ctests were not that helpful in this case.
66 lines
2.3 KiB
Plaintext
66 lines
2.3 KiB
Plaintext
The ctests are not convenient tests. In this file we can attempt to log the
|
|
benefits and costs to using them so we can periodically evaluate which tests we
|
|
should keep and which we should get rid of.
|
|
|
|
Below are some objective costs:
|
|
|
|
- The time it takes to run the ctests.
|
|
- The email discussion back-and-forth.
|
|
|
|
There are also some non-objective costs. The ctests are annoying and not fun to
|
|
maintain, discuss, and fix. Although it's hard to measure this "annoyance"
|
|
cost, it should be taken into account when we review.
|
|
|
|
Below we can try to keep a log of which bugs are found with which ctests. This
|
|
way we can see which tests are the most useful. It is difficult to objectively
|
|
differentiate between minor bugs and important bugs.
|
|
|
|
The "Benefit" row gives the amount of days in-between when an issue is found
|
|
because of the ctests and when an issue is reported by someone not using the
|
|
ctests.
|
|
|
|
Examples where convergence tests took time and annoyance to figure out that
|
|
they were actually "false positives" (i.e., no bug that is worth the time or
|
|
complexity to fix):
|
|
|
|
https://www.mail-archive.com/search?l=mid&q=20210125200655.e27xaaoy2belyxhw%40tallinn
|
|
|
|
|
|
Examples where bugs were fixed (more quickly) because of ctests:
|
|
|
|
Report : https://www.mail-archive.com/search?l=mid&q=20210126190837.pwuxhrhrikzszfrh%40tallinn
|
|
Fix : 92f8612a
|
|
|
|
Report : https://www.mail-archive.com/search?l=mid&q=b0a33192bc5635bd1cb5596e346f9c34615ffeb4.camel%40lyx.org
|
|
Fix : ab1d418f
|
|
|
|
Fix : ef2b2827
|
|
|
|
Fix : 1ddc1c6b
|
|
|
|
Report : https://www.mail-archive.com/search?l=mid&q=20210218051838.jbwnemfv5ilhlsls%40tallinn
|
|
Fix : 358e4ace
|
|
|
|
Fix : 3df42924
|
|
|
|
Fix : 67d0b80f
|
|
|
|
Fix : 8ce6c815
|
|
|
|
Report : https://www.mail-archive.com/search?l=mid&q=20210523014926.xewdr5x62br2okvy%40tallinn
|
|
Fix : 39a9e370
|
|
|
|
Report : https://www.mail-archive.com/search?l=mid&q=20210528001413.56phju7dvfn4dexs%40tallinn
|
|
Fix : ffa6a724
|
|
|
|
Report : https://www.mail-archive.com/search?l=mid&q=20210529174524.idnmt5w4tvhuvyy6%40tallinn
|
|
Note1 : We are still not sure what to do regarding how to make it more user-friendly
|
|
for users to realize that "-dALLOWPSTRANSPARENCY" is needed for the ps2pdf converter.
|
|
Benefit: 14 days (https://www.lyx.org/trac/ticket/12303)
|
|
|
|
|
|
Examples where upstream (e.g., LaTeX packages) bugs were fixed (more quickly)
|
|
because of ctests:
|
|
|
|
Fix : https://github.com/nrnrnr/noweb/pull/12#event-4892099481
|