Skip to content

Commit

Permalink
Readme updates
Browse files Browse the repository at this point in the history
  • Loading branch information
KH committed Mar 8, 2024
1 parent 9b6cea2 commit 3892b54
Show file tree
Hide file tree
Showing 4 changed files with 25 additions and 14 deletions.
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ Capillaries is a data processing framework that:

| | BEFORE | AFTER |
| ----------- | ------ |------ |
| Cloud-friendly | depends | Can be deployed to the cloud within minutes, Docker-ready |
| Data aggregation | SQL joins | Capillaries [lookups](doc/glossary.md#lookup) in Cassandra + [Go expressions](doc/glossary.md#go-expressions) (scalability, parallel execution) |
| Data filtering | SQL queries, custom code | [Go expressions](doc/glossary.md#go-expressions) (scalability, maintainability) |
| Data transform | SQL expressions, custom code | [Go expressions](doc/glossary.md#go-expressions), Python [formulas](doc/glossary.md#py_calc-processor) (parallel execution, maintainability) |
Expand Down
Binary file modified doc/beforeafter.png
100644 → 100755
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
34 changes: 22 additions & 12 deletions doc/beforeafter.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
4 changes: 2 additions & 2 deletions doc/qna.md
Original file line number Diff line number Diff line change
Expand Up @@ -105,11 +105,11 @@ A. Here are some, in no particular order:

1. Performance enhancements, espcecially those related to the efficient use of Cassandra.

2. Read/write from/to other file formats, maybe databases.
2. Read/write from/to other file formats, maybe databases. Update 2023: Apache Parquet support was added.

3. Creating node configuration is a tedious job. Consider adding a toolbelt command that takes a CSV file as an input and generates JSON for a corresponding file_table/table_file node. Update 2023: done, see [proto_file_reader_creator test](../test/code/proto_file_reader_creator/README.md).

4. Is the lack of NULL support a deal-breaker?
4. Is the lack of NULL vsalues support a deal-breaker? Update March 2024: support for *_if aggregate functions was added, it should help mitigate the lack of NULL support.

5. Need a strategy to mitigate potential security threats introduced by py_calc. SELinux/AppArmor?

Expand Down

0 comments on commit 3892b54

Please sign in to comment.