Probot feedback from GitHub Ecosystem Hackathon

[lee-dohm]

Kudos

I wrote an App! In an afternoon! That does a thing (auto-deleting branches after PRs are merged)! In a grand total of 22 lines (7 of which are debugging statements)! That's pretty sweet, and the create-probot-app template-thingy is super awesome.

General

• The documentation about setting the webhook url was confusing (wasn't clear what the path of the URL should be)
• The remix example uses a pretty outdated version of probot. (ref: Cleanup & Updates hello-world#2)
• Because we were using an outdated version of probot, there wasn't any request logging which made debugging the incorrect webhook URL pretty hard. We first tried turning on the TRACE environment variable but eventually just upgraded the NPM packages and got better logging.
• We wanted to send a graphql query but it’s not built into probot (ref: Add GraphQL support probot#472)
• Inconsistent terminology. We add a label with labels but remove a label with name.
• Unhelpful/missing error message: the simulator was working, smee.io says 200, deliveries say 200, but app was crashing. It turned out the developer secret was set locally, but not configured in the app. Error message was just the crash trace. (ref: Better error messages probot#434)

create-probot-app

Overall the guide for Probot + create-probot-app is really awesome and helped me get going pretty quickly.

• Why doesn't this app I've created with create-probot-app include some code!!
  I want this to do something off the bat!!

  • There's a Hello World page
    in the docs with an example of using a webhook, but it tells me

  "Integrate Probot into your awesome Node application"

  ... and I have no idea how to use Node. The template should
  include this code or something else that does something, rather than just
  your code here.

  • The Hello World page in this guide feels really out of order - it feels
    like it has a lot of information but I haven't actually set up my app yet
    so I can't use it yet, and the next page in the guide - Developing an App -
    seems to have a lot of the prerequisites for this page? And the
    Interacting with GitHub page seems like it should come earlier/has
    information that was included on some of the earlier pages.

• I've tried issues.opened... but I don't think I'm getting anything because
  maybe I don't have the right permissions?? Or is it because my webhook setup
  isn't correct?

  • I wish there was a better way to test that webhooks are actually set up
    right

• ERROR probot: No X-Hub-Signature found on request

  • Apparently means webhook secret is not correct - googled and found
    Unable to test plugins locally: auth error? probot#147 (comment)
  • This error message is really not obvious
  • I wish the "Create a GitHub App" page would test out that the webhook info you
    provided actually works

• Can't see debug or trace output from my Probot app in iTerm2 with solarized Dark

  • Bunyan-format: https://github.com/thlorenz/bunyan-format/blob/master/index.js#L38
  • Issue for Bunyan: Bright black "hides" some bunyan CLI styled text when using Solarized theme trentm/node-bunyan#160
  • Fix for Bunyan: trentm/node-bunyan@26a8bb0
  • Workaround - bump colors contrast in iTerm to fix issue
    • Color fix needed?  altercation/solarized#220 (comment)

[tcbyrd]

I've linked to a few in progress PRs in the OP that are related to this feedback. Still some follow-up needed on create-probot-app. I think we can make the template a little more useful. Ref: probot/template#34 for prior art.

[stale]

Is this still causing friction? If so, please comment with any updates or addition details.