Skip to content

Commit

Permalink
Merge pull request #39 from reload/chore/update-readme
Browse files Browse the repository at this point in the history
  • Loading branch information
lasseborly authored May 4, 2020
2 parents cf2503b + 470b615 commit 812e4c3
Show file tree
Hide file tree
Showing 16 changed files with 982 additions and 198 deletions.
10 changes: 0 additions & 10 deletions .travis.yml

This file was deleted.

19 changes: 0 additions & 19 deletions LICENSE

This file was deleted.

12 changes: 12 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
up:
docker-compose run server composer install && \
docker-compose up

create-config:
cp docker-compose.override.yml-example docker-compose.override.yml

fetch:
docker-compose exec server app/console harvester:fetch

create-admin:
docker-compose exec server sh -c 'app/console harvester:user --admin="yes" --active="yes" --password harvest@reload.dk'
141 changes: 104 additions & 37 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,75 +1,142 @@
![https://i.imgur.com/jdTgyyI.png](https://i.imgur.com/jdTgyyI.png)

# Harvester
![Docker Badge](https://img.shields.io/docker/automated/reload/harvester.svg) ![Docker Badge](https://img.shields.io/docker/build/reload/harvester.svg)

Harvester
=========
Welcome to Harvester, a service which fetch data from HarvestApp and have a simple API to get data out.


## Get started

Welcome to Harvester, a service which fetch data from HarvestApp and have a simple API to get data out. This is so much
faster than the actual HarvestApp API since everything is stored locally.
#### 1. Create the config:

```
make create-config
```

## Installing
#### 2. Fill in envs in `docker-compose.override.yml`:

When it comes to installing Harvester you will need to clone the repository from GitHub.
```
HARVESTER_HARVEST_USER: harvest_account_email@email.com
HARVESTER_HARVEST_PASSWORD: secretpassword
# only use the actual name, e.g. "reload". *Not* the full url (reload.harvestapp.com)
HARVESTER_HARVEST_ACCOUNT: harvestapp_account_name
```

git clone https://github.com/reload/harvester.git
The credentials can be found in LastPass. Search for __"Harvester API"__.

And install Harvester with composer:
#### 3. Get everything up and running. This takes ~30 minutes:

cd harvester/
```
make up
```

docker-compose run composer install
When your teminal looks something likes this:

Composer will install Harvester and all its dependencies. And you will need to configure database and the system.
```
server_1 | *** Running /etc/my_init.d/00_regen_ssh_host_keys.sh...
server_1 | *** Running /etc/my_init.d/harvester...
server_1 | *** Running /etc/rc.local...
server_1 | *** Booting runit daemon...
server_1 | *** Runit started as PID 10
server_1 | tail: unrecognized file system type 0x794c7630 for ‘/var/log/apache2/error.log’. please report this to bug-coreutils@gnu.org. reverting to polling
server_1 | tail: unrecognized file system type 0x794c7630 for ‘/var/log/syslog’. please report this to bug-coreutils@gnu.org. reverting to polling
server_1 | AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.28.0.2. Set the 'ServerName' directive globally to suppress this message
server_1 | AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.28.0.2. Set the 'ServerName' directive globally to suppress this message
server_1 | [Fri Apr 17 08:35:03.303829 2020] [mpm_prefork:notice] [pid 21] AH00163: Apache/2.4.7 (Ubuntu) PHP/5.5.9-1ubuntu4.29 configured -- resuming normal operations
server_1 | [Fri Apr 17 08:35:03.303888 2020] [core:notice] [pid 21] AH00094: Command line: 'apache2 -D FOREGROUND'
server_1 | Apr 17 08:35:03 d37b70b08786 syslog-ng[18]: syslog-ng starting up; version='3.5.3'
```

Install the database
__The wait is over!__

docker-compose exec harvester app/console doctrine:database:create
#### 4. Create an admin user:

Import the database schema
```
make create-admin
```

docker-compose exec harvester app/console doctrine:schema:create
Fill in your password of choice and now you are ready to login with the user __harvest@reload.dk__ and your new password at [http://harvester.docker/login](http://harvester.docker/login).

Setup your apache/nginx virtualhost, and a few urls is now available
#### 5. Harvester is now accessible at:

[http://yourho.st/api/doc]()
- Login: [http://harvester.docker/login](http://harvester.docker/login)
- Admin: [http://harvester.docker/admin](http://harvester.docker/admin)
- Docs: [http://harvester.docker/api/doc](http://harvester.docker/api/doc)

[http://yourho.st/admin]()

## CLI

## Configuration
For fetching data from the HarvestApp API and mannaging users.

For the HarvestApp API fetcher to work you need to configure the API info via environment variables in the container:
### User mannagement

cp docker-compose.override.yml-example docker-compose.override.yml
Print the available configuration for the user command.

Then edit the variables in docker-compose.override.yml, which correspond to those in `parameters.yml`. You will need to restart the container if it is already running.
```
docker-compose exec server app/console help harvester:user
```


## Usage
### Data fetching

There is a commmand line command available for fetching data from the HarvestApp API.
This could be executed from crontab
<details>
<summary>
Print the available configuration for the fetch command.
</summary>

To see the command
```
docker-compose exec server app/console help harvester:fetch
```
</details>

docker-compose exec harvester app/console help harvester:fetch
<details>
<summary>
Run the fetch command without parameters. Will fetch everything from HarvestApp (~30 minutes).
</summary>

To run the command without parameters, will fetch everything from Harvest (could tak several minutes)
```
docker-compose exec server app/console harvester:fetch
```
</details>

docker-compose exec harvester app/console harvester:fetch
<details>
<summary>
Data from this month
</summary>

To fetch updated data from this month
```
docker-compose exec server app/console harvester:fetch `date "+%Y%m01"`
```
</details>

docker-compose exec harvester app/console harvester:fetch `date "+%Y%m01"`

To fetch that has been updated since yesterday
<details>
<summary>
Since yesterday
</summary>

docker-compose exec harvester app/console harvester:fetch --updated-yesterday
```
docker-compose exec server app/console harvester:fetch --updated-yesterday
```
</details>

To delete entries and repopulate within a given timespan
<details>
<summary>
Delete entries and repopulate within a given timespan
</summary>

docker-compose exec harvester app/console harvester:refresh `date "+%Y%m01"`
```
docker-compose exec server app/console harvester:refresh `date "+%Y%m01"`
```
</details>

To delete entries and repopulate within a static amount of days
<details>
<summary>
Delete entries and repopulate within a static amount of days
</summary>

docker-compose exec harvester app/console harvester:refresh --days=30
```
docker-compose exec server app/console harvester:refresh --days=30
```
</details>
3 changes: 3 additions & 0 deletions app/config/config_dev.yml
Original file line number Diff line number Diff line change
Expand Up @@ -35,3 +35,6 @@ assetic:
swiftmailer:
disable_delivery: true

# Twig Configuration
twig:
cache: false
2 changes: 1 addition & 1 deletion composer.json
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
},
"require": {
"php": ">=5.3.3",
"symfony/symfony": "~2.7",
"symfony/symfony": "~2.8",
"doctrine/orm": "~2.2,>=2.2.3",
"doctrine/doctrine-bundle": "~1.2",
"twig/extensions": "~1.0",
Expand Down
Loading

0 comments on commit 812e4c3

Please sign in to comment.