WhatWhat

whatwhat is an OCaml command-line tool to help monitor project status and allocations in the Research Engineering Group, aka Hut23. It is the successor to NowWhat (F#) and WhatNow (Racket); the latter contains an illuminating backbrief describing the history of project allocations in REG.

Brew install WhatWhat

The simplest way to use whatwhat is to brew install it. It is hosted in our local Hut23 tap. You will need to run the following commands:

brew update
brew tap alan-turing-institute/hut23
brew install alan-turing-institute/hut23/whatwhat

This is likely to install whatwhat from the pre-compiled Homebrew bottle (associated with a GitHub release), which will make the installation quicker. However, if there is not a bottle that matches your computer build it will automatically build from the source.

Once you have installed it, you will need to create a secrets.json file to store your personal access tokens, which you can do by running whatwhat init then populating the file with your github and forecast tokens.

These tokens can be found here:

• githubToken here refers to a "classic" personal access token, which you can generate at https://github.com/settings/tokens. The token will need to have the permissions: repo (top level), read:user and user:email (both within user).
• forecastToken can be obtained from https://id.getharvest.com/oauth2/access_tokens/new (log in if necessary).

whatwhat should now work, the following examples can get you started.

For extensive usage options, do whatwhat --help, or whatwhat <COMMAND> --help for the subcommands.

• whatwhat: Report errors for projects on the four main columns of the issue tracker. Print output to terminal.
• whatwhat project [NUM|NAME]: Print an overview of a project, as specified by its GitHub issue number, or a (sub)string of its title.
• whatwhat person [NAME]: Print an overview of a person, as specified by their name or GitHub username (a substring is fine).
• whatwhat --notify github: Same as whatwhat, but additionally post GitHub comments on all of those issues.
• whatwhat export-{project,team}: Create Forecast project or team export CSV files. Useful for other reporting purposes.
• whatwhat open [NUM]: Open a GitHub issue in a browser (macOS only, as this uses open(1).

Lookup table for GitHub usernames

whatwhat attempts to match people's full names on Forecast with GitHub usernames. It is quite successful at doing this, but there are a few edge cases where people's GitHub profiles do not have enough data. To get around this, you can manually add a mapping inside the file ~/.config/whatwhat/users. Each line in this file should look like:

{Full name on Forecast}:{GitHub username}

(without the curly braces). If whatwhat's automatic username detection is turning up false positives for a person, you can also override it by entering their info in this file.

Developers Contents

1. Setting up OCaml on macOS
2. Installation and configuration
3. Usage
4. Resources for getting started with OCaml
5. New Tags and Releases

Setting up OCaml on macOS

opam is the OCaml package manager, which can be installed using Homebrew. It installs all packages into ~/.opam. Most online instructions suggest that you need to run eval $(opam env), but with the invocation of opam init below you should not need to.

brew install opam
opam init --shell-setup -y
opam install dune utop odig ocaml-lsp-server ocamlformat -y

Setup for some editors (others can be set up in an analogous way):

• (Emacs) opam install tuareg
• (Neovim) The easiest way is to use lspconfig, and add something like require'lspconfig'.ocamllsp.setup{} to your config.
• (VSCode) There is a VS Code extension for OCaml called OCaml Platform. This includes features for syntax highlighting, editing, navigation and snippets.

Installation and configuration

First, check that you have the dune executable available on your $PATH:

which dune

dune is the 'official' OCaml build system, and is used throughout this README.

git clone git@github.com:alan-turing-institute/whatwhat.git
cd whatwhat
make install-deps
dune build

The make install-deps command takes care of installing project dependencies. See the wiki for more explanation of what goes on inside there.

dune places all its build artifacts in the _build folder. If you got to this point without errors, you should be able to run (the local version of) whatwhat:

dune exec -- whatwhat

It will most likely complain about a missing configuration or missing secret. To get around this, you will have to set up two files, one containing secrets and one containing other non-sensitive info.

Secrets

Put this in ~/.config/whatwhat/secrets.json (and make sure that it is not stored in source control, e.g. in your personal dotfiles / config!):

{
    "githubToken"    : "<yours here>",
    "githubBotToken" : "<get from the Shared Drive>",
    "forecastToken"  : "<yours here>",
    "slackToken"     : "<get from the Shared Drive>"
}

• githubToken here refers to a "classic" personal access token, which you can generate at https://github.com/settings/tokens. The token will need to have the permissions: read:user, repo, and user:email.
• forecastToken can be obtained from https://id.getharvest.com/oauth2/access_tokens/new.
• To obtain githubBotToken and slackToken, you need to be added to the hut23-1206-nowwhat@turing.ac.uk group (ask someone else on the whatwhat developer team to add you, e.g. the person who most recently committed to main). However, for initial development purposes, it suffices to use an empty string, because these tokens are only used if you wish to actually post to GitHub or Slack respectively.

Configuration

Separately, place this in ~/.config/whatwhat/config.json:

{
    "githubProjectName"    : "Project Tracker",
    "githubProjectColumns" : ["Active", "Awaiting start", "Finding people", "Awaiting go/no-go"],
    "forecastId"           : "<yours here>",
    "githubRepoName"       : "Hut23",
    "githubRepoOwner"      : "alan-turing-institute"
}

• forecastID can be obtained by opening Forecast. It will redirect you to another URL that contains your ID, which is a series of several digits. Paste the ID in as a string.

You shouldn't need to change any of the other settings here.

Lookup table for GitHub usernames

whatwhat attempts to match people's full names on Forecast with GitHub usernames. It is quite successful at doing this, but there are a few edge cases where people's GitHub profiles do not have enough data. To get around this, you can manually add a mapping inside the file ~/.config/whatwhat/users. Each line in this file should look like:

{Full name on Forecast}:{GitHub username}

(without the curly braces). If whatwhat's automatic username detection is turning up false positives for a person, you can also override it by entering their info in this file.

Usage

Note that, to compile and execute the source code in your working directory, whatwhat should always be run using dune exec -- whatwhat [options]. Just running whatwhat [options] will use the installed version inside ~/.opam (if it exists). (You can create a shell alias if you're lazy: something like alias dew='dune exec -- whatwhat').

The following examples can get you started. For extensive usage options, do dune exec -- whatwhat --help, or dune exec -- whatwhat <COMMAND> --help for the subcommands.

• dune exec -- whatwhat: Report errors for projects on the four main columns of the issue tracker. Print output to terminal.
• dune exec -- whatwhat init: Sets up the secrets and config files from a template.
• dune exec -- whatwhat --notify github: Same as above, but additionally post GitHub comments on all of those issues.
• dune exec -- whatwhat project [NUM|NAME]: Print an overview of a project, as specified by its GitHub issue number, or a (sub)string of its title.
• dune exec -- whatwhat person [NAME]: Print an overview of a person, as specified by their name or GitHub username (a substring is fine).
• dune exec -- whatwhat export-{project,team}: Create Forecast project or team export CSV files. Useful for other reporting purposes.
• dune exec -- whatwhat open [NUM]: Open a GitHub issue in a browser (macOS only, as this uses open(1).

Documentation

Online documentation for the main branch of whatwhat can be viewed at:

https://alan-turing-institute.github.io/whatwhat/whatwhat/index.html

Documentation is written using odoc syntax. To build documentation locally, there are a couple of options:

1. make dunedoc

   Uses dune to build docs based on the development version of whatwhat (i.e. the actual source code on your computer). This documentation does not contain links to types, functions, etc. from other modules, and the build process can emit a few spurious warnings (see #64).

2. make odigdoc

   Uses odig to build docs based on the installed version of whatwhat (i.e. the version inside ~/.opam). This documentation does link to that of other modules. However, you do have to install whatwhat first before this can be done; the Makefile takes care of this automatically.

See the Documentation wiki page for more description of what's going on with these commands.

Resources for getting started with OCaml

Official

• Main website: https://ocaml.org/
• Installation instructions from the official website

Learning and getting started

• Merlin: set up for Emacs and VIM
• OCamlverse is a community wiki
• OCaml Discourse
• OCaml PRO

Books and documentation

• OCaml Programming: Correct + Efficient + Beautiful (A nice textbook with accompanying lecture series.)
• Real World OCaml (Jane Street) (Goes faster and is more in-depth, so quite good if you have some experience with functional programming. However, note that it relies on Jane Street's alternative standard library, which whatwhat doesn't use.)
• Developing Applications with Objective Caml (Book)

New Tags and Releases

When you make changes to the whatwhat code, the Homebrew tap of whatwhat needs to be updated. To do this, run dune exec update_whatwhat from the top level of the git repository. It takes a few minutes to run.

If it fails for any reason, fix the underlying error, then just rerun it and increment the version number one more time.

The update process used to be manual. It involved the following steps, which are saved here for posterity:

Steps for manually updating Homebrew release

1. Update the version in the dune-project file and commit that.
2. Create a new git tag using git tag -a vX.Y.Z -m vX.Y.Z (replace X, Y, Z with the appropriate numbers.
3. To push the tag to the repo, run git push followed by git push --tags
4. Edit the whatwhat.rb file in the homebrew-hut23 repo. In this instance, you only need to edit the line beginning url "...", tag "vX.Y.Z", revision "...". Change the tag string to the new version number, and replace the revision string with the full git commit SHA corresponding to the tag you just added. You can obtain this from the output of git log.
   This single step used to be done via a GitHub Action. The action yaml has since been removed from the repository, but you can see the file in a previous commit here.
5. In the terminal navigate to a custom directory. You now need to type brew update to make sure you have the latest whatwhat formula. If you already have whatwhat brew installed, you'll need to uninstall it brew uninstall whatwhat. Then type

   brew install --build-bottle --verbose whatwhat
   brew bottle whatwhat --no-rebuild

   This will create a bottle (a prebuilt binary) in that directory and will print a new bottle do block to your terminal
6. Copy that code snippet, go to the ruby file in the homebrew-hut23 and paste it to overwrite the previous bottle do instructions.
7. Commit the changes to main
8. Rename the bottle file to remove one of the dashes (i.e. from whatwhat-- to whatwhat-)
9. Going back to the whatwhat repo go to releases -> tags tab and click on the latest tag.
10. Click create release from tag
11. Upload the binary from your computer
12. Click 'publish release'
13. Copy the url of the release and paste this as the first line within bottle do in the ruby file, in the form of root_url "https://github.com/alan-turing-institute/whatwhat/releases/download/<VERSION>". Note this should be the same as the url, but with 'download' rather than 'tag'. Make sure you commit this change.
14. This should now work - users will now be able to brew install the latest version of Whatwhat.