ForeSITE Alerting Report

ForeSITE Alerting Report is a an open-source C# WPF application designed for time-series epidemiological data analysis and automated alerting reports. It enables healthcare and public health professionals to transform raw surveillance data into actionable insights with minimal setup.

This application integrates with a Python Flask localhost web server to process data using the epySurv module, leveraging CDC public surveillance data for demonstration and allowing users to import custom datasets for analysis. The application supports creating PDF reports with visualizations, managing data sources, scheduling automated report generation, and emailing reports via Gmail/Outlook. Built with C#, XAML, QuestPDF, epySurv, matplotlib, rpy2 and a Flask backend, it provides a user-friendly interface for analysts and data professionals to create and share insightful reports.

✨ Features

📊 Time-Series Data Integration

Import and process surveillance datasets (CSV, JSON, Web APIs)
Seamlessly interchange datasets between R and Python for flexible analysis.
Prebuilt models (Farrington, Bayes, Boda, CDC, EARS-C1, etc.) for anomaly detection in clinical/public health data.

🔔 Alerting & Report Generation

Automated PDF report generation with customizable titles and data-driven plots.
Configurable parameters: data source, time range, thresholds, and models.
Monthly scheduled reports with Windows Task Scheduler and automatic email delivery (Gmail).

📈 Visualization

Generate rich plots from CDC public surveillance data or user-imported datasets.
Flexible configuration for baselines, thresholds, and model comparisons.

🔧 Data Source Management

Add, edit, or delete data sources with full API integration.
Supports both internal datasets and external provider APIs.

💻 Interactive Analysis

Notebook-like interface for advanced users, supporting Python and R cells for runtime analysis.
Dynamically add, execute, delete, save, and load analysis cells.

🎨 User-Friendly Customization

Rich text editing for report titles (bold, italic, etc.).
Robust font fallback (Arial, Helvetica, Times New Roman) ensuring PDF compatibility across systems.

⚙️ Backend Processing

Local Flask server (http://127.0.0.1:5001/epyapi) for time-series processing and plot generation.

✨ Changelog

Version 0.9

Added Model Management to support user's Python package and R package installation
Prebuilt models (Farrington, Bayes, Boda, CDC, EARS-C1, etc.) for anomaly detection in clinical/public health data.

Version 0.8

We’ve added support for Windows scheduled tasks to automate report delivery. Users can now configure a schedule (daily, weekly, or monthly) directly within the application. Once set, the system will automatically:

Generate updated reports based on the latest data
Export them to PDF (change PdfSharpCore to QuestPDF)
Send them to the specified recipients via email

This means reports are sent hands-free and on time, according to your preferred schedule—no manual steps required.

Version 0.7

Added R support in Jupyter-like notebook to enable seamless data exchnage between R and Python
Updated log system for users to better understand the output

Version 0.6

Added a Jupyter-like notebook for advanced users to perform data analysis at runtime.
Users can now add, run, delete, save, and load code cells in a non-modal window.
Enhanced integration with Python environment for code execution.
Minor bug fixes and performance improvements in report rendering.

Version 0.5

Supported four CDC data sources, including:
COVID-19 Testing （only 2020 CDC Covid Testing data;)
Real-time Deaths (COVID-19, Pneumonia, and Influenza)
Allowed users to build surveillance models with configurable sliding window parameters, and generate dynamic, customized alert reports.

✨ Prerequisites

Operating System: Windows 10 or later
.NET Framework: .NET 8.0 or higher
Python: Python 3.7 or later (for Flask server and epySurv package)
Dependencies:
- WPF Application:
  - QuestPDFfor PDF generation
  - Newtonsoft.Json for JSON parsing
  - Google SMTP for automated email delivery
- Flask Server:
  - Flask
  - epySurv (surveillance data processing module, assumed to be a custom or external library)
  - Other Python dependencies (e.g., pandas, matplotlib for data processing and plotting)
Tools:
- Visual Studio 2022 or later (with WPF development workload)
- PowerShell (for scheduled task execution)
- Python environment (e.g., Anaconda, virtualenv)
Administrative Privileges: Required for creating scheduled tasks

Installation From Source Code

1. Clone the Repository

git clone https://github.com/yourusername/ForeSITE-Alerting-Report.git
cd ForeSITE-Alerting-Report

2. Set Up the WPF Application

Open in Visual Studio:
- Open ForeSITETestApp.sln in Visual Studio.
- Restore NuGet packages:
```
dotnet restore
```
Build the Solution:
- Build in Visual Studio (Build > Build Solution) or via CLI:
```
dotnet build
```

3. Set Up the epySurv environment (optional)

epySurv for epySurv Setup

4. Set Up the Flask Server (optional)

Install Dependencies:
- Install Flask :
```
pip install flask pandas matplotlib 
```
Just for test: Run the Flask Server:
- Start the server at http://127.0.0.1:5001:
```
python app.py
```
  Our Application doesn't need to call the Flask Server manually. it will be called by app into the background.

Step 3 and 4 are optional User can download epySurv environment from Release 0.6

5. Run the Application

Run the WPF application as administrator to enable task scheduling:
```
dotnet run --project ForeSITETestApp
```

Installation From Release Version

This release includes the ForeSITE Alerting Report application, featuring integrated Python-based surveillance data processing envrionment and .Net 8 runtime. Key components:

Python Environment: Includes the epysurv-env virtual environment with all required dependencies (e.g., rpy2, pandas, Flask) pre-configured for seamless operation. .NET Runtime: Bundled with the .NET 8 runtime to support the WPF application. If the runtime fails to work on your system, please install the .NET 8 runtime manually from the official .NET website.

Setup

Download and extract the release archive.
Run ForeSITEAlertingReport.exe from the extracted folder.
Logs and generated files will be saved to C:\Users<YourUsername>\Documents\ForeSITEAlertingReportFiles.

Usage

Creating a Report

Launch the Application:
- Start the application and navigate to the "Reporter" tab.
Add a Title:
- Click "Add Title" to create a customizable report title using the rich text editor.
Add Plots:
- Click "Add Plot" and configure:
  - Plot title
  - Data source (CDC public data or custom imported data)
  - Model (e.g., Farrington)
  - Time range, frequency (e.g., weekly), and threshold
- Plots are fetched via the Flask API (http://127.0.0.1:5001/epyapi) using epySurv.
Save as PDF:
- Click "Save" to generate a PDF report (Report.pdf) via a file dialog.

Importing Custom Data

Navigate to the "Data Source Manager" tab.
Add a new data source:
- Enter name, app token, and API URL (if connecting to a custom server).
- For local data, upload CSV/JSON files (assumed to be handled by the Flask server).
Use the imported data in the "Add Plot" dialog by selecting the custom data source.

Managing Data Sources

Add a new data source:
- Enter name, app token, and API URL.
- Click "Save" to add to the list.
Delete data sources:
- Select sources and click "Delete".

Scheduling Automated Reports

Navigate to the "Report" tab.
Click "Scheduling" to create a task template JSON file.
Navigae to the "Scheduler" tab:
- Id: We can see our task file path and we can edit/input email addresses to receive our report
- Schedule: we can see the start date and freq options for our task
- Start/End: Runs to setup our task on/off in windows task list.
Ensure the application is run as administrator.

Flask Server

The Flask server (epyflaServer.py) handles:

API Endpoint: http://127.0.0.1:5001/epyapi for plot generation.
epySurv Integration: Processes CDC public surveillance data or user-imported data using the epySurv module.
Data Processing: Generates plot images returned to the WPF application.

Example Flask Setup

from flask import Flask, request, jsonify
import epySurv  # Assumed module for surveillance data processing

app = Flask(__name__)

@app.route('/epyapi', methods=['POST'])
def generate_plot():
    data = request.get_json()
    # Process data with epySurv (e.g., CDC data or user-imported CSV)
    plot_file = epySurv.generate_plot(data['graph'])
    return jsonify({'status': 'ready', 'file': plot_file})

if __name__ == '__main__':
    app.run(host='127.0.0.1', port=5001)

Ensure epySurv is installed or included in the repository.
Update requirements.txt with all dependencies.

Project Structure

ForeSITE-Alerting-Report/
├── ForeSITETestApp/
│   ├── Dashboard.xaml         # Main UI layout
│   ├── Dashboard.xaml.cs      # Main logic (report generation, scheduling)
│   ├── ReportElement.cs       # Report element model
│   ├── PlotTitleDialog.xaml   # Dialog for plot titles
│   └── ForeSITETestApp.sln    # Solution file
├── Server/
│   ├── epyflaServer.py        # Flask server for epySurv integration
│   └── requirements.txt       # Python dependencies
│   
├── README.md                  # This file
└── LICENSE                    # License file (add your preferred license)

Dependencies

C# WPF Application:
- QuestDF: For PDF generation
- Newtonsoft.Json: For API response parsing
- MailKit: For Email notification
Flask Server:
- Flask: Web server framework
- epySurv: Surveillance data processing module
- pandas, matplotlib: Common data processing and plotting libraries (adjust based on epySurv requirements)

Animation Demo

The application uses CDC public surveillance data for demonstration purposes, processed via the epySurv module through the Flask server. Users can:

Select CDC data as a data source in the "Add Plot" dialog.

When the user selects Abnormal Report Only, the system will review data from the most recent three days or weeks to validate whether n_case exceeds the threshold. If the value is below the threshold, the plot will not be included in the report.

This rule helps limit unnecessary email notifications.

Scheduler Start Screen

Start a windows task for our application

Advanced Data Analysis with Notebook

Advanced users can leverage the "Data Analysis" feature to open a Jupyter-like Notebook window for runtime data analysis. This non-modal window allows users to:

Input Python Code: Utilize a code editor with Python syntax highlighting to write and edit Python scripts.
Input R Code: Utilize a code editor with R code to write and edit R scripts.
Dynamic Cell Management: Add new Python/R code cells with the "Python" or "R" button, delete cells with the "Delete" button, and execute code using the "Run" button (currently a placeholder, with plans for integration with the Python environment).
Save and Load: Persist notebook content to Documents\notebook.json using the "Save" button and reload it with the "Load" button, enabling session continuity.
Real-Time Analysis: Perform data analysis on CDC public surveillance data or imported custom datasets, enhancing exploratory data analysis capabilities.

This feature is designed for users with Python expertise, providing a flexible environment to experiment with data processing and visualization alongside the application's core reporting functionality. (version 0.6)

R Language cell support (version 0.7)

Import customized data set

Customized Data: One column with name "date" or "dates" or "start_date", and another column with name "cases" or "n_cases" or "count" or "deaths" or "case_count"

Screenshots

(version 0.9.1)

Troubleshooting

Font Error in PDF:
- If Arial is unavailable, the application falls back to Helvetica or Times New Roman (see GetSafeFont in Dashboard.xaml.cs).
- Install ttf-mscorefonts-installer on non-Windows systems or embed fonts.
Flask Server Not Running:
- Ensure the server is running at http://127.0.0.1:5001 before adding plots.
- Check Python dependencies: pip list and install missing packages.
Plots Not Generated:
- Verify the Year Back parameter. if user doesn't have 5 years history data, please set to 3
- Verify the Flask server is running and accessible.
- Ensure data source configuration (CDC or custom) is correct.
- Check epySurv module documentation for specific requirements.
- Check if we generated time-series data from Data Sources

Contributing

Contributions are welcome! Please follow these steps:

Fork the repository.
Create a feature branch (git checkout -b feature/your-feature).
Commit changes (git commit -m "Add your feature").
Push to the branch (git push origin feature/your-feature).
Open a pull request.

License

MIT License

Contact

For issues or feature requests, please open an issue on GitHub or contact Dr. Makoto Jones and Tao He.

Name		Name	Last commit message	Last commit date
Latest commit History 71 Commits
images		images
python		python
src/Project		src/Project
.gitattributes		.gitattributes
README.markdown		README.markdown

EpiForeSITE/ForeSITE_Alerting_Report

Folders and files

Latest commit

History

Repository files navigation