2
2
3
3
Simple service for indexing RapidPro contacts into ElasticSearch.
4
4
5
- This service can run in two modes:
5
+ # Deploying
6
+
7
+ As Indexer is a go application, it compiles to a binary and that binary along with the config file is all
8
+ you need to run it on your server. You can find bundles for each platform in the
9
+ [ releases directory] ( https://github.com/nyaruka/rp-indexer/releases ) . You should only run a single indexer
10
+ instance for a deployment.
11
+
12
+ Indexer can run in two modes:
6
13
7
14
1 ) the default mode, which simply queries the ElasticSearch database, finds the most recently
8
15
modified contact, then on a schedule queries the ` contacts_contact ` table on the RapidPro
@@ -14,44 +21,90 @@ all contacts on RapidPro. Once complete, this switches out the alias for the con
14
21
with the newly build index. This can be run on a cron (in parallel with the mode above) to rebuild
15
22
your index occasionally to get rid of bloat.
16
23
17
- ## Usage
24
+ # Configuration
25
+
26
+ Indexer uses a tiered configuration system, each option takes precendence over the ones above it:
27
+ 1 . The configuration file
28
+ 2 . Environment variables starting with ` INDEXER_ `
29
+ 3 . Command line parameters
30
+
31
+ We recommend running Indexer with no changes to the configuration and no parameters, using only
32
+ environment variables to configure it. You can use ` % rp-indexer --help ` to see a list of the
33
+ environment variables and parameters and for more details on each option.
18
34
19
- It is recommended to run the service with two environment variables set:
35
+ ## RapidPro Configuration
36
+
37
+ For use with RapidPro, you will want to configure these settings:
20
38
21
39
* ` INDEXER_DB ` : a URL connection string for your RapidPro database
22
40
* ` INDEXER_ELASTIC_URL ` : the URL for your ElasticSearch endpoint
41
+
42
+ Recommended settings for error reporting:
43
+
44
+ * ` INDEXER_SENTRY_DSN ` : The DSN to use when logging errors to Sentry
45
+
46
+ # Development
47
+
48
+ Install Indexer source in your workspace with:
49
+
50
+ ```
51
+ go get github.com/nyaruka/rp-indexer
52
+ ```
53
+
54
+ Build Indexer with:
55
+
56
+ ```
57
+ go build github.com/nyaruka/rp-indexer/cmd/rp-indexer
58
+ ```
59
+
60
+ This will create a new executable in your current directory ` rp-indexer `
61
+
62
+ To run the tests you need to create the test database:
63
+
64
+ ```
65
+ $ createdb elastic_test
66
+ ```
67
+
68
+ To run all of the tests:
69
+
70
+ ```
71
+ go test github.com/nyaruka/rp-indexer/... -p=1
72
+ ```
73
+
74
+ # Usage
23
75
24
76
```
25
77
Indexes RapidPro contacts to ElasticSearch
26
78
27
79
Usage of indexer:
80
+ -cleanup
81
+ whether to remove old indexes after a rebuild
28
82
-db string
29
- the connection string for our database (default "postgres://localhost/rapidpro")
83
+ the connection string for our database (default "postgres://localhost/rapidpro?sslmode=disable ")
30
84
-debug-conf
31
- print where config values are coming from
85
+ print where config values are coming from
32
86
-elastic-url string
33
- the url for our elastic search instance (default "http://localhost:9200")
87
+ the url for our elastic search instance (default "http://localhost:9200")
34
88
-help
35
- print usage information
89
+ print usage information
36
90
-index string
37
- the alias for our contact index (default "contacts")
91
+ the alias for our contact index (default "contacts")
92
+ -log-level string
93
+ the log level, one of error, warn, info, debug (default "info")
38
94
-poll int
39
- the number of seconds to wait between checking for updated contacts (default 5)
95
+ the number of seconds to wait between checking for updated contacts (default 5)
40
96
-rebuild
41
- whether to rebuild the index, swapping it when complete, then exiting (default false)
97
+ whether to rebuild the index, swapping it when complete, then exiting (default false)
98
+ -sentry-dsn string
99
+ the sentry configuration to log errors to, if any
42
100
43
101
Environment variables:
102
+ INDEXER_CLEANUP - bool
44
103
INDEXER_DB - string
45
104
INDEXER_ELASTIC_URL - string
46
105
INDEXER_INDEX - string
106
+ INDEXER_LOG_LEVEL - string
47
107
INDEXER_POLL - int
48
108
INDEXER_REBUILD - bool
49
- ```
50
-
51
- ## Development
52
-
53
- To generate a local build for a linux amd64 architecture, use the following:
54
-
55
- ``` shell
56
- GOOS=linux GOARCH=amd64 go build github.com/nyaruka/rp-indexer/cmd/rp-indexer
57
- ```
109
+ INDEXER_SENTRY_DSN - string
110
+ ```
0 commit comments