-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathnotes
76 lines (62 loc) · 2.64 KB
/
notes
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
Work:
-----
1. validate_db_pairs: make sure corresponding photo is really in flickr. Delete from db.
2. Add 'reset dirs' command to reset DB for a new computer
3. Smart start:
One starting point - one job
1. Get last run status from a json in the DB
2. Fix all indices - (quickly, with many threads) if this wasn't done in the last 12 hours
3. Validate DB pairs: report duplicates
4. Upload DB to github once index is fixed
5. Upload photos
6. Uplaod DB to github once done uploading
Wishlist
--------
Optional menu for opts:
- all
- upload
- sync with github (plus a BD merge function)
- fix index
Fix for 'no-date'
- Check photo name and look for sqeuntial photos with data
- Check for similar tags (if first check failed)
Params for fix index:
- ignore existing tags
- ignore existing DB entries
Remove duplicates (generally we should now be duplicates free, but in case we're not)
- read all tags from flickr, compare sums and look for duplicates
- if found, fetch photos and present links (user will delete manually)
- auto delete option
Main run point:
- opt to login with a new user
- opt to delete cached users
General:
- pull/commit DB to git automatically
- move all initiations to main and use a constructor
- provide the DB handle from the @conf in main
-------------------
Database
--------
Key-Value store. Daybreak: http://propublica.github.io/daybreak/
DB File per account.
data:
1. Hash sum of the photo: string (MD5 Hex Digest. 32 hex chars)
2. Flickr photo id: string
3. Dir path (relative to mount point) 'drive_name/photos/dir/subdir/subdir2' where 'drive_name' can be within /mnt/PC/drive_name/photos/...
So later in time, you can mount the same drive to another mount point and not need to re-scan it all.
4. Last modification date of Dir
Schema:
-- key ---- | -- value -- | -- comment --
hash sum | flickr id | Used for checking local photos we have
flickr id | hash sum | Used for checking photos in flickr
* both are filled up when we (1) successfully upload a photo
(2) update the hash sum of a photo from flickr
The double entry is because I don't know how to index values in this db and this seems efficient enough
path/to/dir | dir_mod | relative to mount point which is configureable, epoch time
1. When uploading, we first successfully upload the photo
and only then store the photo id and hash.
2. When checking all photos in flickr, we get the photo id
and then look for it in the db.
3. When a photo is deleted from flicker we won't be uploading
it again since it's already in db. This is the desired behavior.
We don't delete the photo id from flickr, it will never be used again.