TO DO

Deduposaur is a command-line program to help you add files to your personal archive.

Functions:

It checks the integrity of your archive.
It helps you process files before adding them to your archive.
- Renames files that you previously processed and decided not to add, adding DELETED to the filename.
- Renames files that already exist in your archive, adding DUPE to the filename.
- Renames files that already exist in your archive but their contents are different, adding CHANGED to the filename.
- Renames files that already exist in your archive, but their names or dates are different, adding METADATA to the filename.
- Leaves new files untouched.
- Remembers files that you delete.

Install

$ cargo install deduposaur

Create a New Archive

First create an empty deduposaur.archive_metadata.json file:

$ cd /my_archive
$ ls
1.jpg 2.jpg 3.jpg 4.jpg 5.jpg
$ touch deduposaur.archive_metadata.json

Then run deduposaur:

$ deduposaur --archive=.
1.jpg is new
2.jpg is new
3.jpg is new
4.jpg is new
5.jpg is new
Verified /my_archive
$

Check Your Archive

To check your archive, simply run deduposaur again:

$ deduposaur --archive=/my_archive
Verified /my_archive
$

Update your Archive

After updating your archive, run deduposaur again and respond to the prompts:

$ cd /my_archive
$ ls
deduposaur.archive_metadata.json 1.jpg 2.jpg 3.jpg 4.jpg 5.jpg
$ mv 2.jpg 2.hawaii.jpg
$ rm 3.jpg
$ echo 'corrupted' > 4.jpg
$ touch 5.jpg
$ deduposaur --archive=.
WARNING 2.jpg is renamed to 2.hawaii.jpg
Accept (y/n) or revert (r)? y
WARNING 3.jpg is deleted
Accept change? (y/n) y
WARNING 4.jpg is changed
Accept change? (y/n) n
WARNING 5.jpg mtime changed 2021-07-10T12:30:00-0700 -> 2021-07-20T15:11:03-0700
Accept (y/n) or revert (r)? r
$ cp /another_backup/4.jpg .
$ deduposaur --archive=.
Verified .

Add Files to Your Archive

First, run deduposaur and it will record metadata of new files and rename known files:

$ cd /new_files
$ ls
1.jpg 2.jpg 3.jpg 4.jpg 5.jpg 6.jpg 7.jpg
$ deduposaur --archive=/my_archive --process=.
Verified /my_archive
Created deduposaur.process_metadata.json
Renamed DUPE.1.jpg - /my_archive/1.jpg
Renamed DUPE.2.jpg - /my_archive/2.hawaii.jpg
Renamed DELETED.3.jpg
Renamed CHANGED.4.jpg - /my_archive/4.jpg
Renamed METADATA.5.jpg - /my_archive/5.jpg
$

Second, go through through the files. Move some to your archive and delete the rest.

$ rm DUPE.1.jpg
$ rm DUPE.2.jpg
$ rm DELETED.3.jpg
$ mv CHANGED.4.jpg /my_archive/4.jpg
$ rm METADATA.5.jpg
$ mv 6.jpg /my_archive/
$ rm 7.jpg
$ ls
deduposaur.process_metadata.json
$

Finally, run deduposaur again to update your archive and remember the deleted files.

$ deduposaur --archive=/my_archive --process=.
/my_archive/4.jpg is replaced by 4.jpg
/my_archive/6.jpg is new
Verified /my_archive
METADATA.5.jpg was deleted
7.jpg was deleted
Deleting deduposaur.process_metadata.json since it is now empty.
$ ls
$

TO DO

Find a way to make it run faster. Re-validating the archive takes a long time. Consider skipping validating the archive when the user specifies --process.
Report duplicates in archive and process dir
DONE - Integration tests
DONE - Make tests pass.
Switch away from libraries with unsafe code:
- structopt (Why does command-line processing need unsafe code for?)
- serde_json
- sha2

Cargo Geiger Safety Report


Metric output format: x/y
    x = unsafe code used by the build
    y = total unsafe code found in the crate

Symbols: 
    🔒  = No `unsafe` usage found, declares #![forbid(unsafe_code)]
    ❓  = No `unsafe` usage found, missing #![forbid(unsafe_code)]
    ☢️  = `unsafe` usage found

Functions  Expressions  Impls  Traits  Methods  Dependency

0/0        0/0          0/0    0/0     0/0      ❓  deduposaur 0.1.0
1/1        44/90        2/2    0/0     0/0      ☢️  ├── chrono 0.4.19
1/20       10/365       0/2    0/0     5/40     ☢️  │   ├── libc 0.2.124
0/0        0/0          0/0    0/0     0/0      ❓  │   ├── num-integer 0.1.44
0/0        4/10         0/0    0/0     0/0      ☢️  │   │   └── num-traits 0.2.14
                                                       │   │       [build-dependencies]
0/0        0/0          0/0    0/0     0/0      ❓  │   │       └── autocfg 1.1.0
                                                       │   │   [build-dependencies]
0/0        0/0          0/0    0/0     0/0      ❓  │   │   └── autocfg 1.1.0
0/0        4/10         0/0    0/0     0/0      ☢️  │   ├── num-traits 0.2.14
0/0        5/5          0/0    0/0     0/0      ☢️  │   ├── serde 1.0.136
0/0        0/0          0/0    0/0     0/0      ❓  │   │   └── serde_derive 1.0.136
0/0        12/12        0/0    0/0     3/3      ☢️  │   │       ├── proc-macro2 1.0.37
0/0        0/0          0/0    0/0     0/0      🔒  │   │       │   └── unicode-xid 0.2.2
0/0        0/0          0/0    0/0     0/0      ❓  │   │       ├── quote 1.0.18
0/0        12/12        0/0    0/0     3/3      ☢️  │   │       │   └── proc-macro2 1.0.37
0/0        47/47        3/3    0/0     2/2      ☢️  │   │       └── syn 1.0.91
0/0        12/12        0/0    0/0     3/3      ☢️  │   │           ├── proc-macro2 1.0.37
0/0        0/0          0/0    0/0     0/0      ❓  │   │           ├── quote 1.0.18
0/0        0/0          0/0    0/0     0/0      🔒  │   │           └── unicode-xid 0.2.2
1/1        218/218      0/0    0/0     0/0      ☢️  │   └── time 0.1.44
1/20       10/365       0/2    0/0     5/40     ☢️  │       └── libc 0.2.124
0/0        35/78        0/0    0/0     0/0      ☢️  ├── filetime 0.2.16
0/0        0/0          0/0    0/0     0/0      ❓  │   ├── cfg-if 1.0.0
1/20       10/365       0/2    0/0     5/40     ☢️  │   └── libc 0.2.124
0/0        0/0          0/0    0/0     0/0      ❓  ├── hex 0.4.3
0/0        5/5          0/0    0/0     0/0      ☢️  │   └── serde 1.0.136
0/0        5/5          0/0    0/0     0/0      ☢️  ├── serde 1.0.136
0/0        4/7          0/0    0/0     0/0      ☢️  ├── serde_json 1.0.79
0/0        7/7          0/0    0/0     0/0      ☢️  │   ├── itoa 1.0.1
7/9        587/723      0/0    0/0     2/2      ☢️  │   ├── ryu 1.0.9
0/0        5/5          0/0    0/0     0/0      ☢️  │   └── serde 1.0.136
0/0        6/6          0/0    0/0     0/0      ☢️  ├── serde_with 1.13.0
1/1        44/90        2/2    0/0     0/0      ☢️  │   ├── chrono 0.4.19
0/0        0/0          0/0    0/0     0/0      ❓  │   ├── doc-comment 0.3.3
0/0        0/0          0/0    0/0     0/0      ❓  │   ├── hex 0.4.3
0/1        0/1          0/0    0/0     0/0      ❓  │   ├── rustversion 1.0.6
0/0        5/5          0/0    0/0     0/0      ☢️  │   ├── serde 1.0.136
0/0        4/7          0/0    0/0     0/0      ☢️  │   ├── serde_json 1.0.79
0/0        0/0          0/0    0/0     0/0      🔒  │   └── serde_with_macros 1.5.2
0/0        0/0          0/0    0/0     0/0      ❓  │       ├── darling 0.13.4
0/0        0/0          0/0    0/0     0/0      ❓  │       │   ├── darling_core 0.13.4
0/0        0/0          0/0    0/0     0/0      ❓  │       │   │   ├── fnv 1.0.7
0/0        0/0          0/0    0/0     0/0      ❓  │       │   │   ├── ident_case 1.0.1
0/0        12/12        0/0    0/0     3/3      ☢️  │       │   │   ├── proc-macro2 1.0.37
0/0        0/0          0/0    0/0     0/0      ❓  │       │   │   ├── quote 1.0.18
0/0        0/0          0/0    0/0     0/0      🔒  │       │   │   ├── strsim 0.10.0
0/0        47/47        3/3    0/0     2/2      ☢️  │       │   │   └── syn 1.0.91
0/0        0/0          0/0    0/0     0/0      ❓  │       │   └── darling_macro 0.13.4
0/0        0/0          0/0    0/0     0/0      ❓  │       │       ├── darling_core 0.13.4
0/0        0/0          0/0    0/0     0/0      ❓  │       │       ├── quote 1.0.18
0/0        47/47        3/3    0/0     2/2      ☢️  │       │       └── syn 1.0.91
0/0        12/12        0/0    0/0     3/3      ☢️  │       ├── proc-macro2 1.0.37
0/0        0/0          0/0    0/0     0/0      ❓  │       ├── quote 1.0.18
0/0        47/47        3/3    0/0     2/2      ☢️  │       └── syn 1.0.91
8/8        202/202      0/0    0/0     0/0      ☢️  ├── sha2 0.9.9
0/0        6/6          0/0    0/0     0/0      ☢️  │   ├── block-buffer 0.9.0
1/1        292/292      20/20  8/8     5/5      ☢️  │   │   └── generic-array 0.14.5
0/0        5/5          0/0    0/0     0/0      ☢️  │   │       ├── serde 1.0.136
0/0        0/0          0/0    0/0     0/0      🔒  │   │       └── typenum 1.15.0
                                                       │   │       [build-dependencies]
0/0        0/0          0/0    0/0     0/0      ❓  │   │       └── version_check 0.9.4
0/0        0/0          0/0    0/0     0/0      ❓  │   ├── cfg-if 1.0.0
0/1        0/14         0/0    0/0     0/0      ❓  │   ├── cpufeatures 0.2.2
0/0        0/0          0/0    0/0     0/0      🔒  │   ├── digest 0.9.0
1/1        292/292      20/20  8/8     5/5      ☢️  │   │   └── generic-array 0.14.5
0/0        0/0          0/0    0/0     0/0      ❓  │   └── opaque-debug 0.3.0
0/0        0/0          0/0    0/0     0/0      🔒  └── structopt 0.3.26
0/0        1/1          0/0    0/0     0/0      ☢️      ├── clap 2.34.0
0/0        32/32        0/0    0/0     0/0      ☢️      │   ├── ansi_term 0.12.1
0/0        5/5          0/0    0/0     0/0      ☢️      │   │   └── serde 1.0.136
2/2        45/45        0/0    0/0     0/0      ☢️      │   ├── atty 0.2.14
1/20       10/365       0/2    0/0     5/40     ☢️      │   │   └── libc 0.2.124
0/0        0/0          0/0    0/0     0/0      ❓      │   ├── bitflags 1.3.2
0/0        0/0          0/0    0/0     0/0      ❓      │   ├── strsim 0.8.0
0/0        0/0          0/0    0/0     0/0      ❓      │   ├── textwrap 0.11.0
0/0        0/0          0/0    0/0     0/0      ❓      │   │   └── unicode-width 0.1.9
0/0        0/0          0/0    0/0     0/0      ❓      │   ├── unicode-width 0.1.9
0/0        0/0          0/0    0/0     0/0      ❓      │   └── vec_map 0.8.2
0/0        5/5          0/0    0/0     0/0      ☢️      │       └── serde 1.0.136
0/0        7/7          1/1    0/0     0/0      ☢️      ├── lazy_static 1.4.0
0/0        0/0          0/0    0/0     0/0      🔒      └── structopt-derive 0.4.18
0/0        0/0          0/0    0/0     0/0      ❓          ├── heck 0.3.3
0/0        0/0          0/0    0/0     0/0      ❓          │   └── unicode-segmentation 1.9.0
0/0        0/0          0/0    0/0     0/0      🔒          ├── proc-macro-error 1.0.4
0/0        0/0          0/0    0/0     0/0      ❓          │   ├── proc-macro-error-attr 1.0.4
                                                               │   │   [build-dependencies]
0/0        0/0          0/0    0/0     0/0      ❓          │   │   └── version_check 0.9.4
0/0        12/12        0/0    0/0     3/3      ☢️          │   │   ├── proc-macro2 1.0.37
0/0        0/0          0/0    0/0     0/0      ❓          │   │   └── quote 1.0.18
0/0        12/12        0/0    0/0     3/3      ☢️          │   ├── proc-macro2 1.0.37
0/0        0/0          0/0    0/0     0/0      ❓          │   ├── quote 1.0.18
0/0        47/47        3/3    0/0     2/2      ☢️          │   └── syn 1.0.91
                                                               │   [build-dependencies]
0/0        0/0          0/0    0/0     0/0      ❓          │   └── version_check 0.9.4
0/0        12/12        0/0    0/0     3/3      ☢️          ├── proc-macro2 1.0.37
0/0        0/0          0/0    0/0     0/0      ❓          ├── quote 1.0.18
0/0        47/47        3/3    0/0     2/2      ☢️          └── syn 1.0.91

21/44      1564/2168    26/28  8/8     17/52

License

License: Apache-2.0

Name		Name	Last commit message	Last commit date
Latest commit History 37 Commits
.github/workflows		.github/workflows
src		src
tests		tests
.gitignore		.gitignore
Cargo.lock		Cargo.lock
Cargo.toml		Cargo.toml
LICENSE		LICENSE
Readme.md		Readme.md
check-readme.sh		check-readme.sh
check.sh		check.sh
dewailly.py		dewailly.py
license-apache-2.0.svg		license-apache-2.0.svg
release.sh		release.sh
update-readme.sh		update-readme.sh

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

Install

Create a New Archive

Check Your Archive

Update your Archive

Add Files to Your Archive

TO DO

Cargo Geiger Safety Report

License

About

Releases

Packages

Languages

License

mleonhard/deduposaur

Folders and files

Latest commit

History

Repository files navigation

Install

Create a New Archive

Check Your Archive

Update your Archive

Add Files to Your Archive

TO DO

Cargo Geiger Safety Report

License

About

Resources

License

Stars

Watchers

Forks

Releases

Packages 0

Languages

Packages