From 2dd9e1a338bd76589fffef2db586574ab88f7b06 Mon Sep 17 00:00:00 2001 From: Scott Kostyshak Date: Mon, 21 Jun 2021 12:57:38 -0400 Subject: [PATCH] ctests-costs-benefits: add a "benefit" row The "benefit" row gives the amount of days in-between when an issue is found because of the ctests and when an issue is reported by someone not using the ctests. In the example of the ps2pdf flag, the issue was reported just a couple of weeks after we found it with the ctests, indicating that the ctests were not that helpful in this case. --- development/autotests/ctests-costs-benefits.txt | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/development/autotests/ctests-costs-benefits.txt b/development/autotests/ctests-costs-benefits.txt index fc39060ffa..3865ee1024 100644 --- a/development/autotests/ctests-costs-benefits.txt +++ b/development/autotests/ctests-costs-benefits.txt @@ -15,6 +15,10 @@ Below we can try to keep a log of which bugs are found with which ctests. This way we can see which tests are the most useful. It is difficult to objectively differentiate between minor bugs and important bugs. +The "Benefit" row gives the amount of days in-between when an issue is found +because of the ctests and when an issue is reported by someone not using the +ctests. + Examples where convergence tests took time and annoyance to figure out that they were actually "false positives" (i.e., no bug that is worth the time or complexity to fix): @@ -52,6 +56,7 @@ Fix : ffa6a724 Report : https://www.mail-archive.com/search?l=mid&q=20210529174524.idnmt5w4tvhuvyy6%40tallinn Note1 : We are still not sure what to do regarding how to make it more user-friendly for users to realize that "-dALLOWPSTRANSPARENCY" is needed for the ps2pdf converter. +Benefit: 14 days (https://www.lyx.org/trac/ticket/12303) Examples where upstream (e.g., LaTeX packages) bugs were fixed (more quickly)