Plugin API

The idea of this page is to brainstorm a standard Plugin API structure.

Thoughts on updating API format

Below is the response structure we expect to craft from any plugin response. This structure is used to make the issue page on the frontend. We have crafted our basic plugins to confront already. This structure is based on ZAP's output, by the way.

{
  'Summary': "XSS discovered.",
  'Description': "Bad things happen when you XSS!",
  'Severity': 'High',
  'URLs": [
      {
        'URL': 'http://foobar.com/foo',
        'Extra': 'More information goes to here."
      },
      ...
  ],
  'FurtherInfo': [ 
      {
        'URL': 'https://mdn.mozilla.com',
        'Title': 'Mozilla Developer Network'
      },
      ...
  ]

Before we move on, we have to understand how current issue page is implemented.

First, when we finished a ZAP scan, we first group all issues with the same "Summary" title in a single issue. Each issue in this group issue has a URL and information such as attack, attack parameter and attack evidene. We put them into the URLs. i.e., from ZAP code:

                if issue['Summary'] not in issues_by_summary:
                    issues_by_summary[issue['Summary']] = issue
                else:
                    issues_by_summary[issue['Summary']]['URLs'] += issue['URLs']

So next time when you click on an issue raised by ZAP, you might find 10 URLs under SQL injection.

Building richer frontend

There are two schools of thoughts:

1. only tell developers what's wrong
2. tell developers both what's wrong and output good stuff

Sometimes, it is much more helpful to do #2 and sometimes #1 is more feasiable (ZAP, for example).

Sometimes we have a difficult time making our issue page richer because different plugins have different, additional fields await to be exposed. But we can't always add to the frontend:

1. we end up with a poor separation of display (frontend) and plugin logic,
2. we just can't edit the HTML to fit every single useful (really useful one) by including them in the frontend.

Two options are possible, from what I understand:

1. Each plugin is responsible for providing the backend a template and provide the data. The backend will have access to the template already (by looking up the plugin registr, i.e. we don't save the template into the DB just to save the paranoid here...), and when frontend asks to render we give that to the frontend. This is okay for a few plugins, but it becomes inefficient: we need to do a round trip to ask the backend / frontend to get the template, and there seems to be little use of angular.js if we go with the usual Django-template powered idea. I guess that's the right term to say it.

2. Each plugin will continue to obey a format, and the report emits from each plugin will tell the frontend what header to expect, what data fall under X,Y,Z headers. Let me show you my idea. Take the soon-to-be-rewritten-and-released-CSP plugin (I have rewritten the whole thing and is just in need to craft the output):

My ideal display for the not-so-intelligent CSP looks like this:

I also want a page like this (even more advance, but the picture has fewer stuff shown.. just saying)

As you can see, this is not possible unless we craft our own HTML entirely for each plugin. I propose the following structure loaded from plugin, and save this entire structure into the database:

{
    "Summary": "script-srcallowsinlineJavascriptcode",
    "Description": "InlineJavascriptcodecanmakeXSSpossible..........",
    "Severity": "High",
    "Report": [
        {
            "issues": [
                {
                    "Summary": "default-src 'self';",
                    "Description": "The default-src directive whitelists ...",
                    "indicators": [
                        {
                            "parsed": "TRUE",
                            "safe": "TRUE"
                        }
                    ],
                    "solutions": "NULL"
                },
                {
                    "Summary": "script-src 'self' 'unsafe-inline';",
                    "Description": "Enabling inline Javascript can make XSS possible.... ",
                    "indicators": [
                        {
                            "parsed": "TRUE",
                            "safe": "FALSE"
                        }
                    ],
                    "solutions": [
                        "GetridofinlineJScodebro"
                    ]
                }
            ]
        }
    ],
    "Solutions": [
        "soln1",
        "soln2"
    ],
    "Artifacts": []
}

This is a lot to digest. So bear with me. The idea is not so complex:

We assume the issue page is designed in certain way: It has a report section that was supposed to be our current's URLs section. This section is used to tell developer which set of urls belong to this particular issue (see above when we discussed how issue page is built.)

Okay, so much distraction. So we have an issue page. It has a section called Report. Then we assume under this Report section, we have a vanilla <ul> like structure, exactly like the first picture I have drawn.

This works for many plugins, I can easily see what's wrong, right? Some issues are GOOD, so the solution field is NULL and we will skip that when we render it. The top-level Solutions can be NULL, because, in the case of CSP, a solution is usually specific to the issue the plugin is raising.

What if I want the second picture? That's a little nasty. The best I can come up with is by adding a level of dictionary to issues: [ {' header_name1': [ {'summary', 'description', 'indicator' ... } ] } ]. Okay you are lost.. basically, we add an additional level to specify the sub header name. So if I want this list of issues to be rendered under a particular sub-header, I will place it as the value of that key....

So to generalize, we are effectively asking the plugin to render in certain way, but it will provide us the field names and details. What is not solved is the plugin cannot specify style, cannot say I want this to be in this div... that's close to NP because then we end up just sending a template!

So to generalize, we can run the first picture using the additional level as in the second-picutre by saying:

If a plugin does not care any sub header, they can place a _top key name and it will render the first.

I haven't introduced much, but I just want to show that we can continue to restrict how the frontend supposes to render (we say it has to be in this way), but we can instruct the plugin author to supply such structure.

Yes, I still break the principle of data separation from frontend logic, but we are just violating a little.

Authentication

Plan: Expose as many options as possible and have user to indicate the type of authentication. Users generally pick the options needed for a particular authentication (we should document the required and optional options for each authentication method for each plugin).

ZAP allows the followings:

• username
• password
• relam
• Enable (I believe this enables authentication module)
• Hide Password (not sure if this only applies to the GUI, or actually allows ZAP to hide password from logging as well)
• session (basically add relevant session token names and values)

Skipfish allows the followings:

• auth_form
• auth_user
• auth_pass
• auth_verify_url
• auth_user_field
• auth_pass_field
• auth_form_target (specify the url where the form should be submitted to)
• -C (cookie)
• -X (url blacklisted)
• -A (basic auth user:pass)

[:yeukhon] A basic API for authentication might be

{ 
  'auth': {
      'type': 'basic_auth' / 'cookie' / 'session'
      ....
  }
}

The idea is pass whichever authentication method the plugin supports, and pass this to the plugin, and have the plugin converts this JSON into the correct key:value and format.

And we can keep a record of the type of authentication was used, minus sensitive information such as password and even cookie. We can probably allow minion to keep cookie and session as artifact as long as we can secure our authorization and data protection, right?

We might want to add a method to our plugin class?

Configuration Examples

Basic Auth

{ "target": "http://some.site.tld",
  "auth": { "type": "basic",
            "username": "bob",
            "password": "secret",
            "realm": "Secret Realm" } }

Form Based

{ "target": "http://some.site.tld",
  "auth": { "type": "form",
            "action": "http://some.site.tld/login",
            "method": "POST",
            "encoding": "application/x-www-form-urlencoded",
            "fields": { "username": "bob",
                        "password": "secret" } }

Persona

{ "target": "http://some.site.tld",
  "auth": { "type": "persona",
            "username": "bob@site.tld",
            "password": "secret" } }