Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release v1.12.0 #392

Merged
merged 179 commits into from
Sep 14, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
179 commits
Select commit Hold shift + click to select a range
563add8
Improved the symbols rule factory to generate only the offending word…
tznind May 29, 2020
e63dee8
IsIdentifiable Reviewer 'Symbols' rule factory now supports digits on…
tznind May 29, 2020
14f3daf
Bump Microsoft.CodeAnalysis.CSharp.Scripting from 3.6.0 to 3.7.0
dependabot-preview[bot] Aug 6, 2020
4f929be
Make Mongo show status output instead of dumping it
jas88 Aug 6, 2020
be78847
Change Mongo quoting
jas88 Aug 6, 2020
5916fce
Re-enable appveyor now Mongo is fixed
jas88 Aug 6, 2020
6490891
Merge branch 'master' into develop
rkm Aug 6, 2020
aad39f2
Bump Microsoft.NET.Test.Sdk from 16.6.1 to 16.7.0
dependabot-preview[bot] Aug 6, 2020
2b964db
Bump fo-dicom.NetCore from 4.0.5 to 4.0.6
dependabot-preview[bot] Aug 7, 2020
67b7157
Bump fo-dicom.Drawing from 4.0.5 to 4.0.6
dependabot-preview[bot] Aug 7, 2020
1741075
Bump Microsoft.Extensions.Caching.Memory from 3.1.6 to 3.1.7
dependabot-preview[bot] Aug 11, 2020
a3687a9
Bump maven-resources-plugin from 3.1.0 to 3.2.0
dependabot-preview[bot] Aug 12, 2020
2319800
Disable publishtrimmed since it breaks things
jas88 Aug 12, 2020
84d68f6
Merge branch 'master' into develop
rkm Aug 12, 2020
2f60fea
Bump mockito-core in /src/common/com.smi.microservices.parent
dependabot-preview[bot] Aug 17, 2020
4290b86
Bump mockito-core from 3.4.6 to 3.5.0
jas88 Aug 17, 2020
3cb0cb7
Bump HIC.RDMP.Dicom from 2.1.6 to 2.1.7 (#352)
dependabot-preview[bot] Aug 17, 2020
8826801
Bump HIC.RDMP.Plugin from 4.1.7 to 4.1.8
dependabot-preview[bot] Aug 17, 2020
b956580
Bump HIC.RDMP.Plugin.Test from 4.1.7 to 4.1.8
dependabot-preview[bot] Aug 17, 2020
7e66293
Bump HIC.DicomTypeTranslation from 2.3.0 to 2.3.1
dependabot-preview[bot] Aug 17, 2020
f918b2b
exclude fo-dicom from dependabot updates
rkm Aug 18, 2020
a2c944d
Update Dependabot config file (#356)
dependabot-preview[bot] Aug 18, 2020
7099bdb
Bump amqp-client in /src/common/com.smi.microservices.parent
dependabot[bot] Aug 18, 2020
8d97c0e
Bump jackson-databind in /src/common/com.smi.microservices.parent (#357)
dependabot[bot] Aug 18, 2020
57842bd
Merge pull request #358 from SMI/dependabot/maven/src/common/com.smi.…
jas88 Aug 18, 2020
2ae6e5a
Bump HIC.BadMedicine.Dicom from 0.0.6 to 0.0.7
dependabot[bot] Aug 18, 2020
4f1fb5f
Merge pull request #360 from SMI/dependabot/nuget/develop/HIC.BadMedi…
dependabot[bot] Aug 18, 2020
ab4d864
Bump mockito-core in /src/common/com.smi.microservices.parent
dependabot[bot] Aug 19, 2020
b4293ba
Bump mockito-core from 3.5.0 to 3.5.2
jas88 Aug 19, 2020
0ab4944
Bump Microsoft.NET.Test.Sdk from 16.7.0 to 16.7.1
dependabot[bot] Aug 21, 2020
a750933
Bump Microsoft.NET.Test.Sdk from 16.7.0 to 16.7.1
jas88 Aug 21, 2020
bee6cc5
Bump mockito-core in /src/common/com.smi.microservices.parent
dependabot[bot] Aug 24, 2020
95185ca
Merge pull request #365 from SMI/dependabot/maven/src/common/com.smi.…
jas88 Aug 24, 2020
cf39a28
Feature/security code scan (#366)
jas88 Aug 24, 2020
97ad6a0
Bump mockito-core in /src/common/com.smi.microservices.parent
dependabot[bot] Aug 25, 2020
31d2d31
Bump Terminal.Gui from 0.81.0 to 0.89.4
dependabot[bot] Aug 25, 2020
62cd838
Update PACKAGES.md
rkm Aug 25, 2020
f68387b
Merge pull request #367 from SMI/dependabot/maven/src/common/com.smi.…
dependabot[bot] Aug 25, 2020
682609e
Updated to match new API
tznind Aug 25, 2020
a828fd3
Set Toggled to = in both cases
tznind Aug 25, 2020
75eb5a4
Add debug message
rkm Aug 25, 2020
371a631
Improve extraction report sections
rkm Aug 25, 2020
9be29af
Extraction report: Group PixelData separately and sort by length
rkm Aug 25, 2020
bf152e2
Merge pull request #369 from SMI/feature/extraction-report-pixel-data
rkm Aug 25, 2020
d5e685f
Reference CommandLineParser centrally
rkm Aug 25, 2020
c271af2
Add FileCopier base
rkm Aug 25, 2020
350a542
Implement file copier
rkm Aug 25, 2020
b1e9d9f
Add FileCopier test proj
rkm Aug 25, 2020
d6168fc
Update TODOs
rkm Aug 25, 2020
e8c9730
Add initial tests for FileCopier
rkm Aug 25, 2020
f3b192d
Fix SmiServices.sln
rkm Aug 25, 2020
dcc7fcd
Bump mockito-core in /src/common/com.smi.microservices.parent
dependabot[bot] Aug 26, 2020
e3fbf82
Merge pull request #370 from SMI/dependabot/maven/src/common/com.smi.…
jas88 Aug 26, 2020
de3f5a9
Merge pull request #368 from SMI/dependabot/nuget/develop/Terminal.Gu…
jas88 Aug 26, 2020
d7d7941
Bump HIC.RDMP.Dicom from 2.1.7 to 2.1.8 (#372)
dependabot[bot] Aug 26, 2020
c37cd3f
Bump MongoDB.Driver from 2.11.0 to 2.11.1 (#371)
dependabot[bot] Aug 26, 2020
b35d1b5
update RMQ extraction config
rkm Aug 26, 2020
cfada0f
update default config and queue names
rkm Aug 26, 2020
7b11aac
Make routing keys configurable via yaml
rkm Aug 26, 2020
b17a81e
Add IsIdentifiableExtraction to CohortExtractor
rkm Aug 26, 2020
7aa46ae
Add new routing keys to CTP
rkm Aug 26, 2020
1d20cf3
Make Java ExtractFileStatus match c#
rkm Aug 26, 2020
9946576
Add IsIdentifiableExtraction to extractorCli
rkm Aug 26, 2020
2df8b89
CTP should throw on messages with IsIdentifiableExtraction set
rkm Aug 26, 2020
12b1645
FileCopier should throw when IsIdentifiableExtraction not set
rkm Aug 26, 2020
536598d
Fix missing arg
rkm Aug 26, 2020
d2107f4
Fix missing arg
rkm Aug 26, 2020
481cf4a
Merge branch 'feature/identifiable-extraction' of github.com:SMI/SmiS…
rkm Aug 26, 2020
2ba8ed1
fix Java tests
rkm Aug 26, 2020
63a8af3
Fix missing exchange
rkm Aug 26, 2020
adc8756
Allow routing keys to be injected to MicroserviceTesters
rkm Aug 27, 2020
91c22e2
fix extraction output directory
rkm Aug 27, 2020
442b017
Add IsIdentifiableExtraction to MongoDB extraction metadata
rkm Aug 27, 2020
30b5153
Rename message and property to match new format
rkm Aug 27, 2020
1211606
Support identifiable extractions in CohortPackager
rkm Aug 27, 2020
f8321f2
Ensure RejectReason specified if Reject=true
rkm Aug 27, 2020
5cb561a
Merge branch 'develop' into feature/identifiable-extraction
rkm Aug 27, 2020
dfec42d
Fix Java test
rkm Aug 27, 2020
04ac3d1
Tidy
rkm Aug 27, 2020
2921177
Add tests for FileCopierHost and CohortExtractor consumer
rkm Aug 28, 2020
cc395be
Implement FileCopyQueueConsumer tests
rkm Aug 28, 2020
b953ca4
Bump HIC.RDMP.Dicom from 2.1.8 to 2.1.9 (#374)
dependabot[bot] Aug 28, 2020
633f165
Handle old format in MongoDB extraction documents
rkm Aug 28, 2020
dc6ecc2
Update CHANGELOG
rkm Aug 28, 2020
7c9ea52
Add IsNoFilterExtraction to extraction metadata
rkm Aug 28, 2020
e127637
Skip rejectors in FromCataloguesExtractionRequestFulfiller when required
rkm Aug 28, 2020
a9a6f66
Add IsNoFilterExtraction to CohortPackager store & report
rkm Aug 28, 2020
ed7f502
Update CHANGELOG
rkm Aug 28, 2020
da3be79
Changed mocks for testing/null classes
tznind Aug 31, 2020
0b5db87
Update extraction diagram
rkm Sep 1, 2020
ac2e67d
Rename ExtractedFileStatus Unused -> None
rkm Sep 1, 2020
0211eae
Clarify test method purpose
rkm Sep 1, 2020
23d7f4d
Use TestTimelineAwaiter instead of sleep
rkm Sep 1, 2020
4707199
Update Fatal logging in Consumer and refactor sleep out from tests
rkm Sep 1, 2020
90f97fd
Merge branch 'feature/identifiable-extraction' into feature/no-filter…
rkm Sep 1, 2020
52a2ac9
Fix FileCopierHostTest
rkm Sep 1, 2020
04119f0
Merge branch 'feature/identifiable-extraction' into feature/no-filter…
rkm Sep 1, 2020
8d07a64
Keep LGTM happy
rkm Sep 1, 2020
b2cab30
Merge pull request #373 from SMI/feature/identifiable-extraction
rkm Sep 1, 2020
c23bcac
Fixup java tests
rkm Sep 1, 2020
daa8f22
Merge branch 'develop' into feature/no-filters-extraction
rkm Sep 1, 2020
29c53a1
Merge pull request #375 from SMI/feature/no-filters-extraction
rkm Sep 1, 2020
e4bb0ac
Bump mockito-core in /src/common/com.smi.microservices.parent
dependabot[bot] Sep 2, 2020
7c3ba30
Merge pull request #376 from SMI/dependabot/maven/src/common/com.smi.…
jas88 Sep 2, 2020
f64f229
Bump HIC.RDMP.Dicom from 2.1.9 to 2.1.10
dependabot[bot] Sep 2, 2020
7389645
Merge branch 'develop' into feature/reviewer-summaryscreen
tznind Sep 3, 2020
eb008e1
Class comments for reviewer
tznind Sep 3, 2020
b33e917
Fixed changelog
tznind Sep 3, 2020
30ada92
Fixed other bit of changelog
tznind Sep 3, 2020
ad64826
Bump mockito-core in /src/common/com.smi.microservices.parent
dependabot[bot] Sep 4, 2020
44cfb82
Merge pull request #379 from SMI/dependabot/maven/src/common/com.smi.…
jas88 Sep 4, 2020
aa2309a
Merge branch 'develop' into feature/extraction-fixes
rkm Sep 4, 2020
c3d100e
Tidy
rkm Sep 4, 2020
055dfa3
Cache values passed to the Validate method to speed up relooks
tznind Sep 7, 2020
d715247
Added maximum cache size
tznind Sep 7, 2020
663cad0
updated changelog
tznind Sep 7, 2020
f87b587
Fix for null values in dictionary cache
tznind Sep 7, 2020
c0573c9
Fix for null values in dictionary cache (oops)
tznind Sep 7, 2020
f208324
Fixed tests to properly pass a field name
tznind Sep 7, 2020
30fabc3
Changed to MemoryCache
tznind Sep 7, 2020
3d69e32
Added tests and fixed MemoryCache adding
tznind Sep 7, 2020
3364be5
Exposed MaxValidationCacheSize as CLI arg
tznind Sep 7, 2020
41d9cae
Added tracking of cache hit/miss rate
tznind Sep 7, 2020
4962ffc
Added logging of cache hit/miss rate on exit
tznind Sep 7, 2020
6c5996f
Merge pull request #380 from SMI/feature/isident-cache
tznind Sep 7, 2020
636f426
Fixed class diagrams
tznind Sep 8, 2020
1078534
Bump Microsoft.Extensions.Caching.Memory from 3.1.7 to 3.1.8
dependabot[bot] Sep 9, 2020
ab060d2
Removed version and empty end column from Packages.md
tznind Sep 9, 2020
dd55ba1
Bump Microsoft.Extensions.Caching.Memory from 3.1.7 to 3.1.8
jas88 Sep 9, 2020
da8b7df
Moved logs root to GlobalOptions
tznind Sep 9, 2020
96942b9
Changed usages to use globals.LogsRoot
tznind Sep 9, 2020
2772be7
Removed pointless count variable
tznind Sep 9, 2020
4a5eeed
Small tidyup, remove disused count of regex matches in line
jas88 Sep 9, 2020
551fd5c
Merge branch 'feature/packagesmd' of https://github.com/SMI/SmiServic…
jas88 Sep 9, 2020
3e87608
Moved mongodb password to config
tznind Sep 9, 2020
82ec6aa
Removed ISIDENTIFIABLE_NUMTHREADS
tznind Sep 9, 2020
459969d
Merge branch 'develop' into feature/reviewer-summaryscreen
jas88 Sep 9, 2020
85b9d5d
Switched from static to factory/decorator pattern
tznind Sep 9, 2020
3397e59
Implemented decorator pattern for environment variables
tznind Sep 9, 2020
7914d16
Fixed producer not being IOptions
tznind Sep 9, 2020
f2adc8f
Have Appveyor quit as early as possible if not needed
jas88 Sep 9, 2020
de6f01f
Fixed missing interface
tznind Sep 9, 2020
9c6d7d5
Added new rejector that queries a column for patients to reject
tznind Sep 9, 2020
64fc453
Added tests
tznind Sep 9, 2020
c079c82
Merge pull request #384 from SMI/feature/packagesmd
tznind Sep 9, 2020
3485029
Merge branch 'develop' into dependabot/nuget/develop/Microsoft.Extens…
jas88 Sep 9, 2020
42a3fb0
Merge pull request #381 from SMI/feature/fix-class-diagrams
tznind Sep 9, 2020
adeaf88
Merge branch 'develop' into feature/config-injection
tznind Sep 9, 2020
d61de5e
Fixed not opening connection and PatientId field
tznind Sep 9, 2020
4624c65
Merge pull request #378 from SMI/feature/reviewer-summaryscreen
tznind Sep 9, 2020
8c6de70
Merge pull request #383 from SMI/dependabot/nuget/develop/Microsoft.E…
dependabot[bot] Sep 9, 2020
1dcd89d
Tidy a bit, add a new test to get coverage over the exception case
rkm Sep 9, 2020
7d48a80
Renamed and split up patient rejector
tznind Sep 10, 2020
8a3a24f
Merge pull request #386 from SMI/feature/patient-rejector
tznind Sep 10, 2020
4d348e6
add password entries to default config
rkm Sep 10, 2020
fd42ce7
Allow decorators to be injected to GlobalOptionsFactory
rkm Sep 10, 2020
52fe40a
Clarify comment
rkm Sep 10, 2020
21c68dd
Bump MongoDB.Driver from 2.11.1 to 2.11.2
dependabot[bot] Sep 11, 2020
ac8253f
changed `RejectPatientsIn` to `RejectColumnInfos`
tznind Sep 11, 2020
60253f0
Updated yaml comments
tznind Sep 11, 2020
fbad604
updated changelog
tznind Sep 11, 2020
227725d
Merge pull request #387 from SMI/dependabot/nuget/develop/MongoDB.Dri…
jas88 Sep 11, 2020
dc72af0
Removed uneeded static keyword
tznind Sep 11, 2020
80e2a49
Merge pull request #388 from SMI/feature/reject-multiple
tznind Sep 11, 2020
bc1c427
Merge pull request #385 from SMI/feature/config-injection
tznind Sep 11, 2020
0131046
Bump System.IO.Abstractions.TestingHelpers from 12.1.1 to 12.1.9
dependabot[bot] Sep 14, 2020
d0edba0
Bump System.IO.Abstractions from 12.1.1 to 12.1.9
dependabot[bot] Sep 14, 2020
6d39a4f
Merge pull request #390 from SMI/dependabot/nuget/develop/System.IO.A…
jas88 Sep 14, 2020
003b150
Merge pull request #389 from SMI/dependabot/nuget/develop/System.IO.A…
jas88 Sep 14, 2020
8d34844
Merge branch 'develop' into feature/extraction-fixes
rkm Sep 14, 2020
5b8a64d
update changelog
rkm Sep 14, 2020
e9dcb5c
Add test for QueryToExecuteResult
rkm Sep 14, 2020
6af90b8
Merge pull request #391 from SMI/feature/extraction-fixes
2bPro Sep 14, 2020
fb3c864
add releasing notes
rkm Sep 14, 2020
a43671e
Start release branch for v1.12.0
rkm Sep 14, 2020
b5bf725
update RELEASING
rkm Sep 14, 2020
0ad1649
sort changelog
rkm Sep 14, 2020
b67721d
Merge branch 'master' into release/v1.12.0
rkm Sep 14, 2020
ee4765f
add dependencies to changelog
rkm Sep 14, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 48 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,52 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),

## [Unreleased]

## [1.12.0] - 2020-09-14

### Added

- [breaking] Add identifiable extraction support
- New service "FileCopier" which sits in place of CTP for identifiable extractions and copies source files to their output dirs
- Changes to MongoDB extraction schema, but backwards compatibility has been tested
- RabbitMQ extraction config has been refactored. Queues and service config files need to be updated
- Add [SecurityCodeScan](https://security-code-scan.github.io/) tool to build chain for .NET code
- Add "no filters" extraction support. If specified when running ExtractorCLI, no file rejection filters will be applied by CohortExtractor. True by default for identifiable extractions
- Added caching of values looked up in NLP/rulesbase for IsIdentifiable tool
- Added new rejector that throws out values (e.g. patient IDs) whose IDs are stored in a database table. Set `RejectColumnInfos` option in yaml to enable this
- Added a check to QueryToExecuteResult for RejectReason being null when Reject is true.

### Changed

- [breaking] Environment variables are no longer required. Previous settings now appear in configuration file
- Environment variable `SMI_LOGS_ROOT` is now `GlobalOptions.LogsRoot`
- Environment variable `MONGO_SERVICE_PASSWORD` is now `MongoDbOptions.Password`
- Removed `ISIDENTIFIABLE_NUMTHREADS` as it didn't work correctly anyway
- Extraction report: Group PixelData separately and sort by length
- IsIdentifiable Reviewer 'Symbols' rule factory now supports digits only or characters only mode (e.g. use `\d` for digits but leave characters verbatim)
- IsIdentifiable Reviewer 'symbols' option when building Regex now builds capture groups and matches only the failing parts of the input string not the full ProblemValue. For example `MR Head 12-11-20` would return `(\d\d-\d\d-\d\d)$`

### Fixed

- Fix the extraction output directory to be `<projId>/extractions/<extractname>`

### Dependencies

- Bump fo-dicom.Drawing from 4.0.5 to 4.0.6
- Bump fo-dicom.NetCore from 4.0.5 to 4.0.6
- Bump HIC.BadMedicine.Dicom from 0.0.6 to 0.0.7
- Bump HIC.DicomTypeTranslation from 2.3.0 to 2.3.1
- Bump HIC.FAnsiSql from 1.0.2 to 1.0.5
- Bump HIC.RDMP.Dicom from 2.1.6 to 2.1.10
- Bump HIC.RDMP.Plugin from 4.1.6 to 4.1.8
- Bump HIC.RDMP.Plugin.Test from 4.1.6 to 4.1.8
- Bump Microsoft.CodeAnalysis.CSharp.Scripting from 3.6.0 to 3.7.0
- Bump Microsoft.Extensions.Caching.Memory from 3.1.6 to 3.1.8
- Bump Microsoft.NET.Test.Sdk from 16.6.1 to 16.7.1
- Bump MongoDB.Driver from 2.11.0 to 2.11.2
- Bump System.IO.Abstractions from 12.1.1 to 12.1.9
- Bump System.IO.Abstractions.TestingHelpers from 12.1.1 to 12.1.9
- Bump Terminal.Gui from 0.81.0 to 0.89.4

## [1.11.1] - 2020-08-12

- Set PublishTrimmed to false to fix bug with missing assemblies in prod.
Expand Down Expand Up @@ -366,7 +412,8 @@ First stable release after importing the repository from the private [SMIPlugin]
- Anonymous `MappingTableName` must now be fully specified to pass validation (e.g. `mydb.mytbl`). Previously skipping database portion was supported.


[Unreleased]: https://github.com/SMI/SmiServices/compare/v1.11.1...develop
[Unreleased]: https://github.com/SMI/SmiServices/compare/v1.12.0...develop
[1.12.0]: https://github.com/SMI/SmiServices/compare/v1.11.1...v1.12.0
[1.11.1]: https://github.com/SMI/SmiServices/compare/v1.11.0...v1.11.1
[1.11.0]: https://github.com/SMI/SmiServices/compare/v1.10.0...v1.11.0
[1.10.0]: https://github.com/SMI/SmiServices/compare/v1.9.0...v1.10.0
Expand Down
59 changes: 30 additions & 29 deletions PACKAGES.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,32 +7,33 @@
2. This package is widely used and is actively maintained.
3. It is open source.

| Package | Source Code | Version | License | Purpose | Additional Risk Assessment |
| ------- | ------------| --------| ------- | ------- | -------------------------- |
| CommandLineParser | [GitHub](https://github.com/commandlineparser/commandline) | [2.8.0](https://www.nuget.org/packages/CommandLineParser/2.8.0) | [MIT](https://opensource.org/licenses/MIT)| Command line argument parsing | |
| CsvHelper | [GitHub](https://github.com/JoshClose/CsvHelper) | [15.0.5](https://www.nuget.org/packages/CsvHelper/15.0.5) | [MS-PL and Apache 2.0](https://github.com/JoshClose/CsvHelper/blob/master/LICENSE.txt)| Writting reports out to CSV reports | |
| fo-dicom.NetCore | [GitHub](https://github.com/fo-dicom/fo-dicom) | [4.0.5](https://www.nuget.org/packages/fo-dicom.NetCore/4.0.5) | [MS-PL](https://opensource.org/licenses/MS-PL) | | |
| HIC.DicomTypeTranslation | [GitHub](https://github.com/HicServices/DicomTypeTranslation) | [2.3.0](https://www.nuget.org/packages/HIC.DicomTypeTranslation/2.3.0) | [GPL 3.0](https://www.gnu.org/licenses/gpl-3.0.html) | Translate dicom types into C# / database types | |
| HIC.FAnsiSql | [GitHub](https://github.com/HicServices/FansiSql) | [1.0.2](https://www.nuget.org/packages/HIC.FAnsiSql/1.0.2) | [GPL 3.0](https://www.gnu.org/licenses/gpl-3.0.html) | Database abstraction layer | |
| HIC.RDMP.Dicom | [GitHub](https://github.com/HicServices/RdmpDicom) | [2.1.6](https://www.nuget.org/packages/HIC.RDMP.Dicom/2.1.6) | [GPL 3.0](https://www.gnu.org/licenses/gpl-3.0.html) | RDMP Plugin containing data load / pipeline components for imaging, reading dicom files etc | |
| HIC.RDMP.Plugin | [GitHub](https://github.com/HicServices/RDMP) | [4.1.6](https://www.nuget.org/packages/HIC.RDMP.Plugin/4.1.6) | [GPL 3.0](https://www.gnu.org/licenses/gpl-3.0.html) | Interact with RDMP objects, base classes for plugin components etc | |
| JetBrains.Annotations | | [2020.1.0](https://www.nuget.org/packages/JetBrains.Annotations/2020.1.0) |[MIT](https://opensource.org/licenses/MIT) | Static analysis tool | |
| Magick.NET-Q16-AnyCPU | [GitHub](https://github.com/dlemstra/Magick.NET) | [7.21.1](https://www.nuget.org/packages/Magick.NET-Q16-AnyCPU/7.21.1) | [Apache License v2](https://github.com/dlemstra/Magick.NET/blob/master/License.txt) | The .NET library for [ImageMagick](https://imagemagick.org/index.php) | |
| Microsoft.CodeAnalysis.CSharp.Scripting | [GitHub](https://github.com/dotnet/roslyn) | [3.6.0](https://www.nuget.org/packages/Microsoft.CodeAnalysis.CSharp.Scripting/3.6.0) | [MIT](https://opensource.org/licenses/MIT) | Supports dynamic rules for cohort extraction logic | |
| Microsoft.Extensions.Caching.Memory | [GitHub](https://github.com/dotnet/extensions) | [3.1.6](https://www.nuget.org/packages/Microsoft.Extensions.Caching.Memory/3.1.6) | [Apache 2.0](https://www.nuget.org/packages/Microsoft.Extensions.Caching.Memory/3.1.6/License) | Caching ID mappings retrieved from Redis/MySQL |
| MongoDB.Driver | [GitHub](https://github.com/mongodb/mongo-csharp-driver) |[2.11.0](https://www.nuget.org/packages/MongoDB.Driver/2.11.0)| [Apache 2.0](https://www.nuget.org/packages/MongoDB.Driver/2.11.0/License) | For writting/reading dicom tags into MongoDb databases|
| NLog | [GitHub](https://github.com/NLog/NLog) | [4.6.4](https://www.nuget.org/packages/NLog/4.6.4) | [BSD 3-Clause](https://github.com/NLog/NLog/blob/dev/LICENSE.txt) | Flexible user configurable logging | |
| Newtonsoft.Json | [GitHub](https://github.com/JamesNK/Newtonsoft.Json) | [12.0.3](https://www.nuget.org/packages/Newtonsoft.Json/12.0.3) | [MIT](https://opensource.org/licenses/MIT) | Serialization of objects for sharing/transmission |
| RabbitMQ.Client | [GitHub](https://github.com/rabbitmq/rabbitmq-dotnet-client) | [5.1.2](https://www.nuget.org/packages/RabbitMQ.Client/5.1.2) | [Apache License v2 / MPL 1.1](https://github.com/rabbitmq/rabbitmq-dotnet-client/blob/master/LICENSE) | Handles messaging between microservices | |
| StackExchange.Redis | [GitHub](https://github.com/StackExchange/StackExchange.Redis) | [2.1.58](https://www.nuget.org/packages/StackExchange.Redis/2.1.58) |[MIT](https://opensource.org/licenses/MIT) | Required for RedisSwapper | |
| Stanford.NLP.CoreNLP | [GitHub Pages](https://sergey-tihon.github.io/Stanford.NLP.NET/) | [3.9.2](https://www.nuget.org/packages/Stanford.NLP.CoreNLP/3.9.2) | [GNU v2](https://github.com/sergey-tihon/Stanford.NLP.NET/blob/master/LICENSE.txt)| Name / Organisation detection in text | |
| System.Drawing.Common | [GitHub](https://github.com/dotnet/corefx) | [4.7.0](https://www.nuget.org/packages/System.Drawing.Common/4.7.0) | [MIT](https://opensource.org/licenses/MIT) | Supports reading pixel data | |
| System.IO.Abstractions | [GitHub](https://github.com/System-IO-Abstractions/System.IO.Abstractions) | [12.1.1](https://www.nuget.org/packages/System.IO.Abstractions/12.1.1) | [MIT](https://opensource.org/licenses/MIT) | Makes file system injectable in tests | |
| System.IO.FileSystem | [GitHub](https://github.com/dotnet/corefx) | [4.3.0](https://www.nuget.org/packages/System.IO.FileSystem/4.3.0) |[MIT](https://opensource.org/licenses/MIT) | File I/O | |
| System.Security.AccessControl | [GitHub](https://github.com/dotnet/corefx) | [4.7.0](https://www.nuget.org/packages/System.Security.AccessControl/4.7.0) |[MIT](https://opensource.org/licenses/MIT) | File access perimssions| |
| Terminal.Gui | [GitHub](https://github.com/migueldeicaza/gui.cs/) | [0.81.0](https://www.nuget.org/packages/Terminal.Gui/0.81.0) |[MIT](https://opensource.org/licenses/MIT) | Console GUI library | |
| Tesseract | [GitHub](https://github.com/charlesw/tesseract/) | [4.1.0-beta1](https://www.nuget.org/packages/Tesseract/4.1.0-beta1) |[Apache License v2](https://github.com/charlesw/tesseract/blob/master/LICENSE.txt) | Optical Character Recognition in Dicom Pixel data| |
| YamlDotNet | [GitHub](https://github.com/aaubry/YamlDotNet) | [8.1.2](https://www.nuget.org/packages/YamlDotNet/8.1.2) | [MIT](https://opensource.org/licenses/MIT) |Loading configuration files|
| fo-dicom.Drawing | [GitHub](https://github.com/fo-dicom/fo-dicom) | [4.0.5](https://www.nuget.org/packages/fo-Dicom.Drawing/4.0.5) | [MS-PL](https://opensource.org/licenses/MS-PL)| Support library for reading DICOM pixel data | |
| coveralls.io | [GitHub](https://github.com/coveralls-net/coveralls.net) | [1.4.2](https://www.nuget.org/packages/coveralls.io/1.4.2) | [GNU](https://github.com/coveralls-net/coveralls.net#license)| Uploader for dot net coverage reports to Coveralls.io | |
| OpenCover | [GitHub](https://github.com/OpenCover/opencover) | [4.7.922](https://www.nuget.org/packages/OpenCover/4.7.922) |[MIT Compatible](https://github.com/OpenCover/opencover/blob/master/LICENSE) | Calculates code coverage for tests| |
| Package | Source Code | License | Purpose
| ------- | ------------| ------- | ------- |
| CommandLineParser | [GitHub](https://github.com/commandlineparser/commandline) | [MIT](https://opensource.org/licenses/MIT)| Command line argument parsing |
| CsvHelper | [GitHub](https://github.com/JoshClose/CsvHelper) | [MS-PL and Apache 2.0](https://github.com/JoshClose/CsvHelper/blob/master/LICENSE.txt)| Writting reports out to CSV reports |
| fo-dicom.NetCore | [GitHub](https://github.com/fo-dicom/fo-dicom) | [MS-PL](https://opensource.org/licenses/MS-PL) | |
| HIC.DicomTypeTranslation | [GitHub](https://github.com/HicServices/DicomTypeTranslation) | [GPL 3.0](https://www.gnu.org/licenses/gpl-3.0.html) | Translate dicom types into C# / database types |
| HIC.FAnsiSql | [GitHub](https://github.com/HicServices/FansiSql) | [GPL 3.0](https://www.gnu.org/licenses/gpl-3.0.html) | Database abstraction layer |
| HIC.RDMP.Dicom | [GitHub](https://github.com/HicServices/RdmpDicom) | [GPL 3.0](https://www.gnu.org/licenses/gpl-3.0.html) | RDMP Plugin containing data load / pipeline components for imaging, reading dicom files etc |
| HIC.RDMP.Plugin | [GitHub](https://github.com/HicServices/RDMP) | [GPL 3.0](https://www.gnu.org/licenses/gpl-3.0.html) | Interact with RDMP objects, base classes for plugin components etc |
| JetBrains.Annotations | |[MIT](https://opensource.org/licenses/MIT) | Static analysis tool |
| Magick.NET-Q16-AnyCPU | [GitHub](https://github.com/dlemstra/Magick.NET) | [Apache License v2](https://github.com/dlemstra/Magick.NET/blob/master/License.txt) | The .NET library for [ImageMagick](https://imagemagick.org/index.php) |
| Microsoft.CodeAnalysis.CSharp.Scripting | [GitHub](https://github.com/dotnet/roslyn) | [MIT](https://opensource.org/licenses/MIT) | Supports dynamic rules for cohort extraction logic |
| Microsoft.Extensions.Caching.Memory | [GitHub](https://github.com/dotnet/extensions) | [Apache 2.0](https://www.nuget.org/packages/Microsoft.Extensions.Caching.Memory/3.1.7/License) | Caching ID mappings retrieved from Redis/MySQL
| MongoDB.Driver | [GitHub](https://github.com/mongodb/mongo-csharp-driver) | [Apache 2.0](https://www.nuget.org/packages/MongoDB.Driver/2.11.1/License) | For writting/reading dicom tags into MongoDb databases
| NLog | [GitHub](https://github.com/NLog/NLog) | [BSD 3-Clause](https://github.com/NLog/NLog/blob/dev/LICENSE.txt) | Flexible user configurable logging |
| Newtonsoft.Json | [GitHub](https://github.com/JamesNK/Newtonsoft.Json) | [MIT](https://opensource.org/licenses/MIT) | Serialization of objects for sharing/transmission
| RabbitMQ.Client | [GitHub](https://github.com/rabbitmq/rabbitmq-dotnet-client) | [Apache License v2 / MPL 1.1](https://github.com/rabbitmq/rabbitmq-dotnet-client/blob/master/LICENSE) | Handles messaging between microservices |
| SecurityCodeScan | [GitHub](https://security-code-scan.github.io/) | [LGPL 3.0](https://opensource.org/licenses/lgpl-3.0.html) | Scans code for security issues during build |
| StackExchange.Redis | [GitHub](https://github.com/StackExchange/StackExchange.Redis) |[MIT](https://opensource.org/licenses/MIT) | Required for RedisSwapper |
| Stanford.NLP.CoreNLP | [GitHub Pages](https://sergey-tihon.github.io/Stanford.NLP.NET/) | [GNU v2](https://github.com/sergey-tihon/Stanford.NLP.NET/blob/master/LICENSE.txt)| Name / Organisation detection in text |
| System.Drawing.Common | [GitHub](https://github.com/dotnet/corefx) | [MIT](https://opensource.org/licenses/MIT) | Supports reading pixel data |
| System.IO.Abstractions | [GitHub](https://github.com/System-IO-Abstractions/System.IO.Abstractions) | [MIT](https://opensource.org/licenses/MIT) | Makes file system injectable in tests |
| System.IO.FileSystem | [GitHub](https://github.com/dotnet/corefx) |[MIT](https://opensource.org/licenses/MIT) | File I/O |
| System.Security.AccessControl | [GitHub](https://github.com/dotnet/corefx) |[MIT](https://opensource.org/licenses/MIT) | File access perimssions|
| Terminal.Gui | [GitHub](https://github.com/migueldeicaza/gui.cs/) |[MIT](https://opensource.org/licenses/MIT) | Console GUI library |
| Tesseract | [GitHub](https://github.com/charlesw/tesseract/) |[Apache License v2](https://github.com/charlesw/tesseract/blob/master/LICENSE.txt) | Optical Character Recognition in Dicom Pixel data|
| YamlDotNet | [GitHub](https://github.com/aaubry/YamlDotNet) | [MIT](https://opensource.org/licenses/MIT) |Loading configuration files
| fo-dicom.Drawing | [GitHub](https://github.com/fo-dicom/fo-dicom) | [MS-PL](https://opensource.org/licenses/MS-PL)| Support library for reading DICOM pixel data |
| coveralls.io | [GitHub](https://github.com/coveralls-net/coveralls.net) | [GNU](https://github.com/coveralls-net/coveralls.net#license)| Uploader for dot net coverage reports to Coveralls.io |
| OpenCover | [GitHub](https://github.com/OpenCover/opencover) |[MIT Compatible](https://github.com/OpenCover/opencover/blob/master/LICENSE) | Calculates code coverage for tests|
16 changes: 1 addition & 15 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
[![Total alerts](https://img.shields.io/lgtm/alerts/g/SMI/SmiServices.svg?logo=lgtm&logoWidth=18)](https://lgtm.com/projects/g/SMI/SmiServices/alerts/)
[![Coverage Status](https://coveralls.io/repos/github/SMI/SmiServices/badge.svg)](https://coveralls.io/github/SMI/SmiServices)

Version: `1.11.1`
Version: `1.12.0`

# SMI Services

Expand All @@ -21,7 +21,6 @@ The latest binaries can be downloaded from the [releases section](https://github
1. [Microservices](#microservices)
1. [Data Load Microservices](#data-load-microservices)
2. [Image Extraction Microservices](#image-extraction-microservices)
1. [Environment Variables](#environment-variables)
2. [Solution Overivew](#solution-overview)
3. [Building](#building)
4. [Testing](#testing)
Expand Down Expand Up @@ -74,19 +73,6 @@ A control queue is provided for controlling Microservices during runtime. It su
| Fatal Error Logging | All Microservices that crash or log a fatal error are shut down and log a message to the Fatal Error Logging Exchange. TODO: Nobody listens to this currently.|
| Quarantine | TODO: Doesn't exist yet.|

## Environment Variables

The following environment variables are expected by the program:

>TODO can we move `ISIDENTIFIABLE_NUMTHREADS` to config yaml/CLI?

| Name | Purpose | Example |
|------|---------|---------|
| SMI_LOGS_ROOT | Required to be set if `ForceSmiLogsRoot` is set to `true` in the service config. Determines where log files are written to | `/var/log/smi` |
| MONGO_SERVICE_PASSWORD | Password that should be used to access the MongoDb database used by ETL pipeline services | MyPassword|
| ISIDENTIFIABLE_NUMTHREADS | Optional (defaults to 1). The number of threads to use when looking for identifiable data in extraction pipeline | 1|
| CI | When running tests in a CI, this ensures that integration tests are failed instead of skipped | 1|

## Solution Overview

Appart from the Microservices (documented above) the following library classes are also included in the solution:
Expand Down
16 changes: 15 additions & 1 deletion SmiServices.sln
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@


Microsoft Visual Studio Solution File, Format Version 12.00
# Visual Studio Version 16
VisualStudioVersion = 16.0.29411.108
Expand Down Expand Up @@ -75,6 +75,10 @@ Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Microservices.IsIdentifiabl
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "IsIdentifiableReviewer", "src\applications\IsIdentifiableReviewer\IsIdentifiableReviewer.csproj", "{C2031E86-81B4-405A-A923-9B82E0CE196F}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Microservices.FileCopier", "src\microservices\Microservices.FileCopier\Microservices.FileCopier.csproj", "{D4E52707-FFF7-41E6-8057-C6DB344B8CD7}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Microservices.FileCopier.Tests", "tests\microservices\Microservices.FileCopier.Tests\Microservices.FileCopier.Tests.csproj", "{D61F6BF9-E857-457C-B745-40489A8CFE65}"
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution
Debug|x64 = Debug|x64
Expand Down Expand Up @@ -181,6 +185,14 @@ Global
{C2031E86-81B4-405A-A923-9B82E0CE196F}.Debug|x64.Build.0 = Debug|x64
{C2031E86-81B4-405A-A923-9B82E0CE196F}.Release|x64.ActiveCfg = Release|x64
{C2031E86-81B4-405A-A923-9B82E0CE196F}.Release|x64.Build.0 = Release|x64
{D4E52707-FFF7-41E6-8057-C6DB344B8CD7}.Debug|x64.ActiveCfg = Debug|x64
{D4E52707-FFF7-41E6-8057-C6DB344B8CD7}.Debug|x64.Build.0 = Debug|x64
{D4E52707-FFF7-41E6-8057-C6DB344B8CD7}.Release|x64.ActiveCfg = Release|x64
{D4E52707-FFF7-41E6-8057-C6DB344B8CD7}.Release|x64.Build.0 = Release|x64
{D61F6BF9-E857-457C-B745-40489A8CFE65}.Debug|x64.ActiveCfg = Debug|x64
{D61F6BF9-E857-457C-B745-40489A8CFE65}.Debug|x64.Build.0 = Debug|x64
{D61F6BF9-E857-457C-B745-40489A8CFE65}.Release|x64.ActiveCfg = Release|x64
{D61F6BF9-E857-457C-B745-40489A8CFE65}.Release|x64.Build.0 = Release|x64
EndGlobalSection
GlobalSection(SolutionProperties) = preSolution
HideSolutionNode = FALSE
Expand Down Expand Up @@ -211,6 +223,8 @@ Global
{1A27E9E8-F16E-43F9-927E-5FE92E2F97D8} = {421CCD37-3817-4748-B184-A134E19DD75C}
{E632E673-0766-4A4D-ABE3-4B6D4F5BEFE2} = {421CCD37-3817-4748-B184-A134E19DD75C}
{C2031E86-81B4-405A-A923-9B82E0CE196F} = {8B943F2C-835B-484A-86D2-3F1462970605}
{D4E52707-FFF7-41E6-8057-C6DB344B8CD7} = {421CCD37-3817-4748-B184-A134E19DD75C}
{D61F6BF9-E857-457C-B745-40489A8CFE65} = {421CCD37-3817-4748-B184-A134E19DD75C}
EndGlobalSection
GlobalSection(ExtensibilityGlobals) = postSolution
SolutionGuid = {11CDEA53-71E8-4A9B-BC0D-74F4EB54F740}
Expand Down
Loading