Initial upload

.gitignore

@@ -150,3 +150,8 @@ cython_debug/
 #  and can be added to the global gitignore or merged into this file.  For a more nuclear
 #  option (not recommended) you can uncomment the following to ignore the entire idea folder.
 #.idea/
+
+*/RESTIC_SOURCE_FILES/restic*
+_private*
+*/BUILD-PRIVATE
+*.conf

CHANGELOG

@@ -0,0 +1,60 @@
+## v2.0.0
+
+  - First open public release
+  - Refactor GUI so actions became non-blocking
+    - Added loader animation for gui events
+    - Added backup window with current state
+    - Added internationalization support (currently en and fr)
+  - Refactor restic.ls function to return a generator since restic ls command gives chunks of json, which is bad memory wise when parsing each chunk into a big list
+  - On Windows, "The cloud operation is not supported on a read-only volume." and "The media is write protected." errors will not make backup fail, see #4155
+    - This fix can be removed once #4155 is resolved, see # TEMP-FIX-4155 comment in restic_wrapper package
+  - Added Additional backup parameter option
+  - Added optional backup tags
+  - Added optional prometheus group label and additional labels entries
+  - Make sure backend connections number parameter works with all backends
+  - Added priority configuration for restic operations, defaults to low priority to not disturb operations
+  - Added WindowsApps folder (reparse points) to exclusion list (fix "The file cannot be accessed by the system." error)
+  - Fixed backup did use verbose option twice
+  - Updated restic binary to restic 0.15.0 compiled with go1.19.5 on windows/amd64
+  - Made theme configurable
+  - Make sure we exit all subprocesses when exiting npbackup
+  - Improve config GUI for supporting elder config files properly
+  - Fix restic metrics when 0 bytes of data have been added to the repo
+  - Fix restoration must ignore case on Windows
+  - Fix backup_job variable substitution in config file
+  - Fix for ruamel.yaml global objects being modified out of current scope encrypting current config while saving
+  - Fix npbackup could not start when no config file was present
+  - Drastically improve ls operation speed by changing command_runner method from poller to monitor (no live output) except for backup operation
+  - Compiler:
+    - Compile python 3.7 and 3.10 targets so we also get to use Windows 7 which can't run python > 3.7
+  - Installer:
+    - Moved task creation process to npbackup so we can recreate tasks on systems where npbackup is already installed
+
+## v1.8.0
+
+  - Implement VSS support for Windows
+  - Do not require UAC elevation for npbackup.exe (Still highly recommended, but at least we allow non admin users to backup their data (without VSS))
+  - Search restic executable path when not found locally for both Windows and Linux platforms
+  - Make exclusion lists always ignore case on Windows systems
+  - Highly improve exclusion lists
+  - Keep structure and comments in yaml configuration files
+  - Simplify configuration file data encryption
+  - Fix limit-download parameter not working
+  - Fix error when rest_connections was set to 0
+  - Add some config gui logging
+  - Cosmetic gui fixes
+  - Fix never enforce execution timeout on backup operation
+  - Fix bogus filename encodings while showing backup contents / usernames
+  - Added more restic metrics
+  - Fixed bogus restric metrics
+  - Allow setting additional prometheus labels
+  - Fix gui snapshot contents don't show duplicate parents on Windows when backing up 'c:\foo' and 'C:\bar'
+  - Internal code cleanup
+  - Installer:
+    - Make sure we don't overwrite destination configuration file if existing
+
+## v1.5.3
+  - First closed public release
+
+## v0.1.0 - 1.5.2
+  - Internal development and tests

README.md

@@ -0,0 +1,122 @@
+# NPBackup
+
+A one fits all backup solution that solves modern problems with modern solutions
+
+## Features
+
+- Data deduplication and fast zstd compression
+- Client side data encryption
+- Wide storage backend support
+  - local files
+  - SFTP
+  - High performance HTTP REST server
+  - Amazon S3/Minio/Wasabi
+  - Blackblaze B2
+  - Microsoft Azure Blob Storage
+  - Google Cloud Storage
+  - OpenStack Swift
+  - Alibaba Cloud (Aliyun) Object Storage System (OSS)
+- Full CLI interface for scheduled task usage
+  - Checks for recent backups before launching a backup
+- Optional end user GUI
+  - Backup content view and restore
+  - Configuration interface
+  - Internationalization support (en, fr as of jan 2023)
+- Performance
+  - Backup process and IO priority settings
+  - Upload / download speed limits
+  - Concurrency settings
+- Comes with complete exclusion lists for Linux and Windows files and folders
+- First class prometheus support
+  - Grafana dashboard included
+  - node_exporter file collector support
+  - Optional push gateway metrics uploading
+- First class Windows support
+  - VSS snapshots
+  - Cloud file exclusions (reparse points)
+  - Windows pre-built executables
+  - Windows installer
+- Additional security
+  - repository uri / password and http metrics identification is encrypted
+- yaml file configuration (or gui configuration)
+
+## About
+
+So, a new backup solution out of nowhere, packed with too much features for it's own good ? Not really !
+
+NPBackup relies on the well known [restic](https://restic.net) backup program, which has been battle proven for years.
+While restic is a fanstastic program, NPBackup tries to complete restic in order to offer a broader user experience.
+
+## Quickstart
+
+On Windows, use `NPBackupInstaller.exe` to install NPBackup into program files.
+On Linux, just copy `npbackup` to `/usr/local/bin`
+
+Copy the example config from model `examples/npbackup.conf.dist` into the directory where npbackup is installed.
+
+You can adjust the parameters directly in the file, or via a config GUI by launching `npbackup --config-file=npbackup.conf --config-gui`
+
+Once configured, you can launch manual backups via `npbackup --backup`. Those can be scheduled.
+Windows schedule is created automatically by the installer program. On Linux, you'll have to create a cronjob or a systemd timer.
+
+Since NPBackup is configured to only proceed with backups when no recent backups are detected, you should consider scheduling npbackup executions quite often.
+The default schedule should be somewhere around 15 minutes.
+
+You can use `npbackup --list` or the GUI to list backups.
+
+The GUI allows an end user to check current backups & restore files.rom backups:
+
+The YAML configuration file encrypts sensible data so the end user doesn't have to know repository URI or password.
+
+### The difficulty of laptop backups
+
+As a matter of fact, laptop backups are the hardest. No one can predict when a laptop is on, and if it has access to internet.
+Creating a backup strategy in those cases isn't a simple task.
+
+NPBackup solves this by checking every 15 minutes if a backup newer than 24h exists.
+If so, nothing is done. In the case no recent backup exists, a backup is immediately launched.
+OF course, both time options are configurable.
+In order to avoid sluggish user experience while backing up, process and io priority can be lowered.
+Once done, NPBackup can send backup results in Prometheus format directly to a push gateway, for prometheus to catch them.
+
+### A good server backup solution
+
+Server backups can be achieved by setting up a scheduled task / cron job.
+
+Of course, since NPBackup supports pre-exec and post-exec commands, it can be used to backup various services like virtual hosts or databases where snapshot/dump operations are required.
+When run on a server, prometheus support can be shifted from a push gateway to a file, which will be picked up by node_exporter file collector.
+
+### End user expericence
+
+While admin user experience is important, NPBackup also offers a GUI for end user experience, allowing to list all backup contents, navigate and restore files, without the need of an admin. The end user can also check if they have a recent backup completed, and launch backups manually if needed.
+
+### Security
+
+NPBackup inherits all security measures of restic (AES-256 client side encryption including metadata), append only mode REST server backend.
+
+On top of those, NPBackup itself encrypts sensible information like the repo uri and password, as well as the metrics http username and password.
+This ensures that end users can restore data without the need to know any password, without compromising a secret. Note that in order to use this function, one needs to use the compiled version of NPBackup, so AES-256 keys are never exposed. Internally, NPBackup never directly uses the AES-256 key, so even a memory dump won't be enough to get the key.
+
+## Compilation
+
+In order to fully protect the AES key that is needed to support NPBackup, one can compile the program with Nuitka.
+Compiling needs restic binary for the target platform in `RESTIC_SOURCE_FILES` folder, files must be named `restic_{version}_{platform}_{arch}[.extension]` like provided by restic.net or [github](github.com/restic/restic)
+Compile options are available in `compile.py`. Nevertheless, you should probably go for the official binaries.
+Also, We maintain a special 32 bit binary for Windows 7 which allows to backup those old machines until they get replaced.
+
+## Smart shield, antivirus and reputation
+
+Official binaries for Windows provided by NetInvent are signed with a certificate, allowing to gain trust and reputation in antivirus analysis.
+Also, official binaries are compiled using Nuitka Commercial grade, which is more secure in storing secrets.
+
+## Misc
+
+NPBackup supports internationalization and automatically detects system's locale.
+Still, locale can be overrided via an environment variable, eg on Linux:
+```
+export NPBACKUP_LOCALE=en
+```
+On Windows:
+```
+set NPBACKUP_LOCALE=en
+```

TODO.md

@@ -0,0 +1,27 @@
+## List of features that would be nice to have
+
+- Cube qemu plugin
+- Fallback server when primary repo is not available
+  - Shall we also include the recent backup job verification ?
+    - Example of a bad remote repo path:
+
+      Fatal: unable to open config file: Head "https:/user:***@bad.example.tld/user/config": dial tcp: lookup bad.example.tld: no such host
+
+    - Example of a bad auth:
+
+      Fatal: unable to open config file: unexpected HTTP response (401): 401 Unauthorized
+Is there a repository at the following location?
+
+    - Example of a good path, good auth but no repo initialized:
+
+      Fatal: unable to open config file: <config/> does not exist
+Is there a repository at the following location?
+
+    - Example: bad password
+      
+      Fatal: wrong password or no key found
+
+
+- Linux installer script
+- Windowss installer GUI
+