hook.io-sdk-python

python implementation of bigcompany/hook.io-sdk

Status

Misc status:

Actual WIP

• Writing tests:
  • tests for CLI subcommands (second target - coverage 98% for every file)
• Stoppers:
  • hook.io#240
  • hook.io#245
  • hook.io#251

Misc status:

Roadmap

• 0.0.* - implement hook.io-sdk replacement + (debug) additions (like account.login) + CLI + tests + examples
• 0.1.* - implement Hook replacement (python-style layer) + compilehook
• 0.2.* - python-style layer on client
• 0.* - attempt to implement all [plans](#Current plans)
• 0.9.* - documentation (at least auto-documentation) must be ready before 1.0

Available Endpoints

• Hook
  • run
  • create
  • update
  • destroy
  • resource
  • source
  • package
  • fork
  • view
  • presenter
  • refresh
  • all (/<user> or /services) blocked by bigcompany/hook.io#251
• Datastore
  • get
  • set
  • del
  • recent
• Logs
  • read
  • stream
  • flush
  • write inside hook (=sys.stderr.write)
  • write outside hook
• Events
  • get
  • stream
  • write
• Keys
  • checkAccess
  • create
  • destroy
  • all
• Files
  • readFile
  • writeFile
  • removeFile
  • readdir
  • stat
  • createReadStream
  • createWriteStream
  • download
  • upload
• Env
  • get
  • set
• Metrics
  • hits
• Domains
  • all
  • create
  • destroy
  • find
  • get
  • update
• Account
  • name blocked by #251 (comment)
    • Can be implemented by:
      • listing keys (needs keys::read)
      • creating temporary hook (needs hook::create and hook::destroy)
      • creating temporary file (readdir shows owner) (needs files::writeFile, files::readdir and files::removeFile)
      • Using Hook variable if we are inside hook.io server
  • signup
  • login (user/password session instead of API key)
  • services (list hooks - maybe should go to hook.all) see stackvana/hook.io#251
• server
  • languages
  • package managers for language (https://hook.io/packages)
  • installed packages for PM (package manager)
  • queued packages for PM
  • failed packages for PM
  • list available themes

Endpoints not existing in original SDK

• hook.resource
• hook.source
• logs.flush
• logs.write
• keys.info
• metrics.hits
• files.createReadStream
• files.createWriteStream
• files.download
• files.upload
• accounts.login
• accounts.services

Points in question

• In original SDK TO-DO section there are "Better ENV exports configuration" entry. Maybe "support" or "import"?
• In original SDK TO-DO section there are "Add all hook.io API Methods" entry. Where one can get list of all methods?

Current plans

• Implement current state/plans of js hook.io-sdk:
  • Basic client creation and configuration
  • Minimal pipeable CLI tool
  • Most hook.io API Methods
  • Ability to pass command line arguments
  • Ability to pipe arbitrary code snippets
  • Add ws:// protocol for Websockets API (https://hook.io/websockets)
  • More error-waiting tests (like in hook.io-test)
• Test automation
  • Misc tests
    • List hookio.keys.roles up to date with https://hook.io/roles
    • Test server-side operation
    • Import https://github.com/bigcompany/hook.io-test/tree/master/tests/client
    • Add tests for rich unicode support
  • Tests for basic API
    • Client object creation
    • Server-side object creation
    • Hook run+metadata+creation
    • Hook modification
    • Metrics
    • Datastore
    • Logs
    • Events
    • Keys
    • Files
    • Env
    • Domains
  • Tests for CLI
    • Basic parsing (test_cli.py)
    • Hook run
    • Hook modification
    • Metrics
    • Datastore
    • Logs
    • Events
    • Keys
    • Files
    • Env
    • Domains
  • Test env matrix (tox)
  • Code beauty (flake8)
  • Coverage
  • Setup test CI
    • travis
    • appveyor
• Auto-test available changes in hook.io-sdk since last sync (helpers/jssdksyncmon.py)
• Auto-generate docs
• CLI improvements:
  • Hook download with creation repository package.json and so on
  • Hook creation from hooks repository package.json
    • Add hookio package bundle to top of hook source (helpers/compilehook.py)
    • Add package bundle to account files and some loader to top of hook source
    • Rely on availability of hookio package on server (sometime it will happens)
  • Hook running via gateway-*s (https://hook.io/gateways)
  • Parse logging.LogRecord dict-style entries in hookio logs subcommands
  • Mass operations (i.e. mass hookio hook destroy)
  • Pseudo-FS (fuse, pyfilesystems, ...):
    • Access to vfs (sdk.files)
    • hooks (with resources and logs)
    • datastore maybe
• Server-side operations (use sdk inside hook):
  • Hook logging to write hookio.logs.write (maybe as a separate code in run-hook-python - see stackvana/hook.io#236)
  • All payload mentioned in https://github.com/bigcompany/hook.io/blob/master/bin/run-hook
    • addTrailers
    • removeHeader
    • setHeader
    • setTimeout
    • sendDate
    • statusMessage
    • statusCode
    • writeContinue
    • writeHead
    • error
    • log
  • Implement WSGI
    • https://github.com/bninja/wsgim-rip
• Rethink support for async operations (depends on async support for requests)
• Second-layer over API with more python-friendly interfaces:
  • Interface to fork a thread/threads to join sdk.logs.stream to logging
    • Converter to make logging.LogRecord from parsed log dict
    • async access for logs.stream with ability to add joined streams (depends on async feature)
  • Wrapper interface to sdk.files API (Hook.os maybe?)
  • __call__ interface to sdk.hook.run(hookio.any.<owner>.<name>() for anonymous calls?)
  • sdk.environ: Cache?

Decisions

• def createClient has js-like interface (gets dict as only argument - you can pass config to it).
• def createClient reads environment variables for absent parameters (i.e. $hookAccessKey).
• class Client has some sane defaults, but don't reads any config.
• CLI argument parsing is separated from library API. I.e. argparse specifics should be kept in runclient.py.
• helpers/compilehook.py can concat library with hook that uses it.

Documentation

Documentation Status

There are no documentation yet - it's just several lists (plans, implemented parts, decisions). If someone want to add important documentation parts here - help will be appreciated. Also you are free to play in project wiki.

Handout

• You must create Client instance by Client() or createClient() call.
• If you are on server side (i.e. inside hook) you can create if using install_hook_sdk(). This will also attempt to replace __main__.Hook variable with hookio.utils.Namespace instance with additional field interface and __main__.Hook.sdk field referring Client instance.
• Client instance has lazy-import fields associated with modules - methods of these fields are public API.

Revised API

• Most of Client public API corresponds to call to one URL. Such methods has raw parameter. Default value of this parameter defers by methods - usually False, but can be True (i.e. for streaming methods). If raw=True method will return requests.Response instance. If raw=False method will return some object representation - usually result of JSON parse. Exception is streaming interface.
• Some methods of public API supports streaming interface. You can pass callable object as streaming parameter and it will be called for each line or object (depending on raw). You can pass streaming=True and use iter_lines() or iter_content() methods of requests.Response instance. Precise meaning of combinations of streaming, raw and other parameters may differ for each methods - look documentation and source.