Merge pull request #24 from mageroni/main

Integrating Matt's update for survey questions

README.md

@@ -4,11 +4,11 @@
 
 ## App Objective
 
-As more companies adopt GitHub Copilot, it becomes increasingly important to measure the benefits it brings to the organization. While quantitative analysis is valuable for measuring usage patterns and adoption rates, it is equally essential to consider qualitative analysis. By combining both quantitative and qualitative approaches, we can gain a comprehensive understanding of the developer experience when using this tool.
+As more companies adopt GitHub Copilot, it becomes increasingly important to measure the benefits it brings to the organization. This survey is an effort to combine both quantitative and qualitative data. To improve validity of the quantitative responses, Developers are asked to document their rationale for the time-savings percentage they choose. 
 
-Quantitative analysis provides valuable insights, but it alone cannot capture the full story of how developers truly feel about Copilot. This is particularly important because Copilot offers various ways for developers to interact with it and derive value, some of which may not be captured by current Key Performance Indicators (KPIs).
+Quantitative feedback from the Developer at the time of creating a PR provides valuable insights on the time savings experienced by the Developer. Time savings is needed first before other downstream impacts (like velocity increases, or other improvements can happen. The level of granularity provides multiple feedback opportunities for Developers and can capture a variety of PRs so we can understand adoption challenges and improvement opportunities. If helpful, the Survey results may also be combined with Key Performance Indicators (KPIs) that the product provides to further contextualize the survey responses.
 
-To gain a deeper understanding of user satisfaction, we have designed this app a part of the DevOps lifecycle to facilitate qualitative surveys. The survey responses are stored in your private Azure SQL database to provide insights into how developers are using the tool, the value they perceive, and the challenges they encounter.
+The survey responses are stored in your private Azure SQL database to provide insights into how developers are using the tool, the value they report, and the challenges they encounter.
 
 We hope that this project provides value to your organization, and we encourage you to contribute and build upon it. Your contributions can help further enhance the survey capabilities and provide even greater insights into the developer experience with Copilot.
 
@@ -23,8 +23,37 @@ When a pull request is closed, the app automatically creates an issue that promp
 Note: *If the env file does not contain a Language API Key or Endpoint, the analysis will be skipped and the default language will always be English.*
 
 ### Sample screenshot of a survey 
+### Copilot Usage Survey
 
-![Sample screenshot of a survey](image.png)
+1. ***Did you use Copilot in developing this PR? (If you select No, just answer question 5***
+- [ ] No
+- [ ] Yes
+
+2. Compared to your previous experience coding WITHOUT using Copilot, 
+
+   ***How much less time did the coding take during this PR with Copilot?***
+
+   (Example: The PR would normally take 5 days, but only took 4 days with Copilot then the answer is 20%)
+- [ ] 0%
+- [ ] > 0% but < 10%
+- [ ] > 11% but < 20%
+- [ ] > 21% but < 30%
+- [ ] ≥ 31% but < 40%
+- [ ] ≥ 41%
+
+3. ***Describe your thought process for calculating (or estimating) the time saved in Question 2***
+
+ - [ replace this line with your answer. ]
+
+4. ***How often did you use Copilot in this PR?***
+- [ ] All or most of the time
+- [ ] About Half of the time
+- [ ] Some of the time
+- [ ] Not very much
+
+5. ***What other information can you share about Copilot's ability to save you time coding?*** 
+
+ - [ replace this line with your answer. ]
 
 ### Where does the app store surveys?
 
@@ -89,4 +118,4 @@ For more, check out the [Contributing Guide](CONTRIBUTING.md).
 
 ## License
 
-[ISC](LICENSE) © 2023 Mabel Geronimo
+[ISC](LICENSE) © 2023 Mabel Geronimo

issue_template/copilot-usage-en.md

@@ -2,23 +2,32 @@
 
 For Pull Request XXX:
 
-1. Did you use Copilot in developing this PR? (If you select No, just answer question 4)
+1. ***Did you use Copilot in developing this PR? (If you select No, just answer question 5)***
 - [ ] No
 - [ ] Yes
 
-2. What percentage of time would you estimate Copilot saved you on this PR?
+2. Compared to your previous experience coding WITHOUT using Copilot (This number represents 100%)
+
+***How much less time did the coding take during this PR with Copilot?***
+
+(Example: The PR would normally take 5 days, but only took 4 days with Copilot then the answer is 20%)
 - [ ] 0%
-- [ ] > 0% but < 5%
-- [ ] ≥ 5% but < 10%
-- [ ] ≥ 10% but < 20%
-- [ ] ≥ 20% but < 30%
-- [ ] ≥ 30%
+- [ ] > 0% but < 10%
+- [ ] > 11% but < 20%
+- [ ] > 21% but < 30%
+- [ ] ≥ 31% but < 40%
+- [ ] ≥ 41%
+
+3. ***Describe your thought process for calculating (or estimating) the time saved in Question 2***
 
-3. How often did you use Copilot in this PR?
+- (Please tell us in a comment)
+
+4. ***How often did you use Copilot in this PR?***
 - [ ] All or most of the time
-- [ ] Half of the time or slightly more
+- [ ] About Half of the time
 - [ ] Some of the time
-- [ ] Little of the time
-- [ ] None of the time
+- [ ] Not very much
+
+5. ***What other information can you share about Copilot's ability to save you time coding?*** 
 
-4. What other feedback do you have for us? (Please tell us in a comment)
+- (Please tell us in a comment)

issue_template/copilot-usage-es.md

@@ -2,23 +2,32 @@
 
 Para el Pull Request XXX:
 
-1. Usaste Copilot en el desarrollo de este PR? (Si selecciona No, solo responda la pregunta 4)
+1. *** ¿Utilizó Copilot para desarrollar este PR? (Si selecciona No, simplemente responda la pregunta 5)***
 - [ ] No
 - [ ] Si
 
-2. Qué porcentaje de tiempo estimas que Copilot te ahorró en este PR?
+2. En comparación con su experiencia anterior codificando sin usar Copilot (este número representa el 100%)
+
+***¿Cuánto menos tiempo tomó la codificación durante este PR con Copilot?***
+
+(Ejemplo: el PR normalmente tomaría 5 días, pero solo tomó 4 días con Copilot, entonces la respuesta es 20%)
 - [ ] 0%
-- [ ] > 0% pero < 5%
-- [ ] ≥ 5% pero < 10%
-- [ ] ≥ 10% pero < 20%
-- [ ] ≥ 20% pero < 30%
-- [ ] ≥ 30%
+- [ ] > 0% pero < 10%
+- [ ] > 11% pero < 20%
+- [ ] > 21% pero < 30%
+- [ ] ≥ 31% pero < 40%
+- [ ] ≥ 41%
+
+3. ***Describe tu proceso de pensamiento para calcular (o estimar) el tiempo ahorrado en la Pregunta 2***
 
-3. Qué tan frecuente usaste Copilot en este PR?
+- (Por favor díganos en un comentario)
+
+4. ***¿Con qué frecuencia usaste Copilot en este PR?***
 - [ ] Todo o la mayor parte del tiempo
-- [ ] La mitad del tiempo o un poco más
-- [ ] Algunas veces
-- [ ] Pocas veces
-- [ ] En ningún momento
+- [] Aproximadamente la mitad del tiempo
+- [ ] Algo de tiempo
+- [ ] No mucho
+
+5. ***¿Qué otra información puedes compartir sobre la capacidad de Copilot para ahorrarte tiempo codificando?***
 
-4. Qué otro mensaje tienes para nosotros? (Por favor cuéntanos en un comentario)
+- (Por favor díganos en un comentario)

issue_template/copilot-usage-fr.md

@@ -2,23 +2,32 @@
 
 Pour le Pull Request XXX :
 
-1. Avez-vous utilisé Copilot pour développer ce PR ? (Si vous sélectionnez Non, répondez simplement à la question 4)
+1. ***Avez-vous utilisé Copilot pour développer ce PR ? (Si vous sélectionnez Non, répondez simplement à la question 5)***
 - [ ] Non
 - [ ] Oui
 
-2. Quel pourcentage de temps estimeriez-vous que Copilot vous a fait gagner sur ce PR ?
+2. Par rapport à votre expérience précédente de codage SANS utiliser Copilot (ce nombre représente 100 %)
+
+***Combien de temps le codage a-t-il pris pendant ce PR avec Copilot ?***
+
+(Exemple : le PR prendrait normalement 5 jours, mais n'a pris que 4 jours avec Copilot alors la réponse est 20 %)
 - [ ] 0%
-- [ ] > 0% mais < 5%
-- [ ] ≥ 5% mais < 10%
-- [ ] ≥ 10 % mais < 20 %
-- [ ] ≥ 20% mais < 30%
-- [ ] ≥ 30%
+- [ ] > 0 % mais < 10 %
+- [ ] > 11 % mais < 20 %
+- [ ] > 21 % mais < 30 %
+- [ ] ≥ 31 % mais < 40 %
+- [ ] ≥ 41%
+
+3. ***Décrivez votre processus de réflexion pour calculer (ou estimer) le temps gagné à la question 2***
 
-3. À quelle fréquence avez-vous utilisé Copilot dans ce PR ?
+- (Merci de nous le dire dans un commentaire)
+
+4. ***À quelle fréquence avez-vous utilisé Copilot dans ce PR ?***
 - [ ] Tout le temps ou la plupart du temps
-- [ ] La moitié du temps ou un peu plus
+- [ ] Environ la moitié du temps
 - [ ] Une partie du temps
-- [ ] Peu de temps
-- [ ] A aucun moment
+- [ ] Pas beaucoup
+
+5. ***Quelles autres informations pouvez-vous partager sur la capacité de Copilot à vous faire gagner du temps en matière de codage ?***
 
-4. Quels autres commentaires avez-vous pour nous ? (Merci de nous le dire en commentaire)
+- (Merci de nous le dire dans un commentaire)

issue_template/copilot-usage-pt.md

@@ -2,23 +2,32 @@
 
 Para o Pull Request XXX:
 
-1. Você usou o Copilot no desenvolvimento deste PR? (Se você selecionar Não, apenas responda à pergunta 4)
+1. ***Você usou o Copilot no desenvolvimento deste PR? (Se você selecionar Não, basta responder à pergunta 5)***
 - [ ] Não
 - [ ] Sim
 
-2. Qual a porcentagem de tempo que você estima que o Copilot economizou neste PR?
+2. Em comparação com sua experiência anterior de codificação SEM usar o Copilot (este número representa 100%)
+
+***Quanto menos tempo demorou a codificação durante este PR com o Copilot?***
+
+(Exemplo: o PR normalmente levaria 5 dias, mas demorou apenas 4 dias com o Copilot, então a resposta é 20%)
 - [ ] 0%
-- [ ] > 0% mas < 5%
-- [ ] ≥ 5% mas < 10%
-- [ ] ≥ 10% mas < 20%
-- [ ] ≥ 20% mas < 30%
-- [ ] ≥ 30%
+- [ ] > 0% mas < 10%
+- [ ] > 11% mas < 20%
+- [ ] > 21% mas < 30%
+- [ ] ≥ 31% mas < 40%
+- [ ] ≥ 41%
+
+3. ***Descreva seu processo de cálculo para calcular (ou estimar) o tempo economizado na Pergunta 2***
 
-3. Com que frequência você usou o Copilot neste PR?
+- (Por favor, conte-nos em um comentário)
+
+4. ***Com que frequência você usou o Copilot neste PR?***
 - [ ] Todo ou a maior parte do tempo
-- [ ] Metade do tempo ou um pouco mais
-- [ ] Algumas vezes
-- [ ] Pouco tempo
-- [ ] Em nenhum momento
+- [ ] Cerca de metade das vezes
+- [] Algumas vezes
+- [ ] Não muito
+
+5. ***Que outras informações você pode compartilhar sobre a capacidade do Copilot de economizar seu tempo de codificação?***
 
-4. Que outro feedback você tem para nós? (Por favor, diga-nos em um comentário)
+- (Por favor, conte-nos em um comentário)

test/fixtures/issue_body.md

@@ -2,23 +2,32 @@
 
 For Pull Request #44:
 
-1. Did you use Copilot in developing this PR? (If you select No, just answer question 4)
+1. ***Did you use Copilot in developing this PR? (If you select No, just answer question 5)***
 - [ ] No
 - [ ] Yes
 
-2. What percentage of time would you estimate Copilot saved you on this PR?
+2. Compared to your previous experience coding WITHOUT using Copilot (This number represents 100%)
+
+***How much less time did the coding take during this PR with Copilot?***
+
+(Example: The PR would normally take 5 days, but only took 4 days with Copilot then the answer is 20%)
 - [ ] 0%
-- [ ] > 0% but < 5%
-- [ ] ≥ 5% but < 10%
-- [ ] ≥ 10% but < 20%
-- [ ] ≥ 20% but < 30%
-- [ ] ≥ 30%
+- [ ] > 0% but < 10%
+- [ ] > 11% but < 20%
+- [ ] > 21% but < 30%
+- [ ] ≥ 31% but < 40%
+- [ ] ≥ 41%
+
+3. ***Describe your thought process for calculating (or estimating) the time saved in Question 2***
 
-3. How often did you use Copilot in this PR?
+- (Please tell us in a comment)
+
+4. ***How often did you use Copilot in this PR?***
 - [ ] All or most of the time
-- [ ] Half of the time or slightly more
+- [ ] About Half of the time
 - [ ] Some of the time
-- [ ] Little of the time
-- [ ] None of the time
+- [ ] Not very much
+
+5. ***What other information can you share about Copilot's ability to save you time coding?*** 
 
-4. What other feedback do you have for us? (Please tell us in a comment)
+- (Please tell us in a comment)

test/fixtures/issue_comment.created.json

@@ -49,7 +49,7 @@
       "assignee": null,
       "assignees": [],
       "author_association": "NONE",
-      "body": "### Copilot Usage Survey\n\nFor Pull Request #44:\n\n1. Did you use Copilot in developing this PR? (If you select No, just answer question 4)\n- [x] No\n- [ ] Yes\n\n2. What percentage of time would you estimate Copilot saved you on this PR?\n- [ ] 0%\n- [ ] > 0% but < 5%\n- [ ] ≥ 5% but < 10%\n- [ ] ≥ 10% but < 20%\n- [ ] ≥ 20% but < 30%\n- [ ] ≥ 30%\n\n3. How often did you use Copilot in this PR?\n- [ ] All or most of the time\n- [ ] Half of the time or slightly more\n- [ ] Some of the time\n- [ ] Little of the time\n- [ ] None of the time\n\n4. What other feedback do you have for us? (Please tell us in a comment)",
+      "body": "### Copilot Usage Survey\n\nFor Pull Request #44:\n\n1. ***Did you use Copilot in developing this PR? (If you select No, just answer question 5)***\n- [ ] No\n- [x] Yes\n\n2. Compared to your previous experience coding WITHOUT using Copilot (This number represents 100%)\n\n   ***How much less time did the coding take during this PR with Copilot?***\n   \n   (Example: The PR would normally take 5 days, but only took 4 days with Copilot then the answer is 20%)\n- [ ] 0%\n- [ ] > 0% but < 10%\n- [ ] > 11% but < 20%\n- [x] > 21% but < 30%\n- [ ] ≥ 31% but < 40%\n- [ ] ≥ 41%\n\n3. ***Describe your thought process for calculating (or estimating) the time saved in Question 2***\n    \n - (Please tell us in a comment)\n\n4. ***How often did you use Copilot in this PR?***\n- [x] All or most of the time\n- [ ] About Half of the time\n- [ ] Some of the time\n- [ ] Not very much\n\n5. ***What other information can you share about Copilot's ability to save you time coding?*** \n\n - (Please tell us in a comment)",
       "closed_at": null,
       "comments": 1,
       "comments_url": "https://api.github.com/repos/mageroni/TestRepo/issues/60/comments",

test/fixtures/issues.edited.json

@@ -10,7 +10,7 @@
       "assignee": null,
       "assignees": [],
       "author_association": "NONE",
-      "body": "### Copilot Usage Survey\n\nFor Pull Request #44:\n\n1. Did you use Copilot in developing this PR? (If you select No, just answer question 4)\n- [ ] No\n- [x] Yes\n\n2. What percentage of time would you estimate Copilot saved you on this PR?\n- [ ] 0%\n- [ ] > 0% but < 5%\n- [x] ≥ 5% but < 10%\n- [ ] ≥ 10% but < 20%\n- [ ] ≥ 20% but < 30%\n- [ ] ≥ 30%\n\n3. How often did you use Copilot in this PR?\n- [x] All or most of the time\n- [ ] Half of the time or slightly more\n- [ ] Some of the time\n- [ ] Little of the time\n- [ ] None of the time\n\n4. What other feedback do you have for us? (Please tell us in a comment)",
+      "body": "### Copilot Usage Survey\n\nFor Pull Request #44:\n\n1. ***Did you use Copilot in developing this PR? (If you select No, just answer question 5)***\n- [ ] No\n- [x] Yes\n\n2. Compared to your previous experience coding WITHOUT using Copilot (This number represents 100%)\n\n   ***How much less time did the coding take during this PR with Copilot?***\n   \n   (Example: The PR would normally take 5 days, but only took 4 days with Copilot then the answer is 20%)\n- [ ] 0%\n- [ ] > 0% but < 10%\n- [ ] > 11% but < 20%\n- [x] > 21% but < 30%\n- [ ] ≥ 31% but < 40%\n- [ ] ≥ 41%\n\n3. ***Describe your thought process for calculating (or estimating) the time saved in Question 2***\n    \n - (Please tell us in a comment)\n\n4. ***How often did you use Copilot in this PR?***\n- [x] All or most of the time\n- [ ] About Half of the time\n- [ ] Some of the time\n- [ ] Not very much\n\n5. ***What other information can you share about Copilot's ability to save you time coding?*** \n\n - (Please tell us in a comment)",
       "closed_at": "2023-03-19T01:12:10Z",
       "comments": 0,
       "comments_url": "https://api.github.com/repos/mageroni/TestRepo/issues/62/comments",

test/index.test.js

@@ -18,6 +18,7 @@ const issue_body = fs.readFileSync(
 const expected_issue = {
   title: "Copilot Usage - PR#44",
   body: issue_body,
+  assignee: "mageroni"
 }
 
 const privateKey = fs.readFileSync(