Bug: Ignore ".folder_name" but sync "folder_name" #2796

cyb3rko · 2024-09-02T20:56:26Z

Describe the bug

Hi Alex, first of all thanks for this awesome piece of software, I've been using it for years now <3

I mainly sync my Android project folders between by devices. Each project folder has one .gradle and one gradle folder.
Now I want to ignore the .gradle folder and not sync it while keeping gradle in sync.

Project 1/
├── .gradle/
├── gradle/
└── etc/
Project 2/
├── .gradle/
├── gradle/
└── etc/
Project 3/
└── ...

Here's my sync_list:

!build/kotlin/*
!.kotlin/*
!venv/*
!.venv/*
!.gradle/*
!.idea/libraries/*
!__pycache__/*
!node_modules/*
!.next/*
!/Programming/Projects/Android/**/build/*
!/Programming/Projects/Android/**/.cxx/*
!/Programming/Projects/Web/**/build/*

/Programming

For .gradle it's working as expected:
Skipping item - excluded by sync_list config: Programming/Projects/Android/FlashDimCompose/.gradle/file-system.probe

But for some reason the sync_list above does also exclude my gradle folder as shown by one of the log entries:
Skipping item - excluded by sync_list config: Programming/Projects/Android/FlashDimCompose/gradle/libs.versions.toml

Am I misunderstanding something here?
Thanks for your help!

Operating System Details

Linux niko-fedo 6.10.6-200.fc40.x86_64 #1 SMP PREEMPT_DYNAMIC Mon Aug 19 14:09:30 UTC 2024 x86_64 GNU/Linux

Fedora Linux 40 (KDE Plasma)

Client Installation Method

From Distribution Package

OneDrive Account Type

Personal

What is your OneDrive Application Version

v2.4.25

What is your OneDrive Application Configuration

onedrive version                             = v2.4.25
Config path                                  = /home/niko/.config/onedrive
Config file found in config path             = true
Config option 'sync_dir'                     = /home/niko/OneDrive
Config option 'enable_logging'               = true
Config option 'log_dir'                      = /var/log/onedrive/
Config option 'disable_notifications'        = false
Config option 'min_notify_changes'           = 5
Config option 'skip_dir'                     = 
Config option 'skip_dir_strict_match'        = false
Config option 'skip_file'                    = ~*|.~*|*.tmp
Config option 'skip_dotfiles'                = false
Config option 'skip_symlinks'                = false
Config option 'monitor_interval'             = 300
Config option 'monitor_log_frequency'        = 6
Config option 'monitor_fullscan_frequency'   = 12
Config option 'read_only_auth_scope'         = false
Config option 'dry_run'                      = false
Config option 'upload_only'                  = false
Config option 'download_only'                = false
Config option 'local_first'                  = false
Config option 'check_nosync'                 = false
Config option 'check_nomount'                = false
Config option 'resync'                       = false
Config option 'resync_auth'                  = false
Config option 'cleanup_local_files'          = false
Config option 'classify_as_big_delete'       = 1000
Config option 'disable_upload_validation'    = false
Config option 'bypass_data_preservation'     = false
Config option 'no_remote_delete'             = false
Config option 'remove_source_files'          = false
Config option 'sync_dir_permissions'         = 700
Config option 'sync_file_permissions'        = 600
Config option 'space_reservation'            = 52428800
Config option 'application_id'               = 
Config option 'azure_ad_endpoint'            = 
Config option 'azure_tenant_id'              = common
Config option 'user_agent'                   = 
Config option 'force_http_11'                = false
Config option 'debug_https'                  = false
Config option 'rate_limit'                   = 0
Config option 'operation_timeout'            = 3600
Config option 'dns_timeout'                  = 60
Config option 'connect_timeout'              = 10
Config option 'data_timeout'                 = 600
Config option 'ip_protocol_version'          = 0
Config option 'sync_root_files'              = false
Selective sync 'sync_list' configured        = true
sync_list contents:
!build/kotlin/*
!.kotlin/*
!venv/*
!.venv/*
!.gradle/*
!.idea/libraries/*
!__pycache__/*
!node_modules/*
!.next/*
!/Programming/Projects/Android/**/build/*
!/Programming/Projects/Android/**/.cxx/*
!/Programming/Projects/Web/**/build/*

/Programming
Config option 'sync_business_shared_folders' = false
Business Shared Folders configured           = false
Config option 'webhook_enabled'              = false

What is your 'curl' version

8.6.0

Where is your 'sync_dir' located

Local

What are all your system 'mount points'

/

What are all your local file system partition types

sdb                                                                                                                                  
├─sdb1  vfat        FAT32  BBD4-0741  579,8M  3%  /boot/efi
├─sdb2  ext4        1.0    f6337a4..  539M    38% /boot
└─sdb3  crypto_LUKS 2      2f4ba1d..                
  └─luks-2f4ba1d1.. btrfs  fedora_niko-fedo 4422bda7..  386,5G  13%  /home

How do you use 'onedrive'

/

Steps to reproduce the behaviour

Create a folder gradle
Create a folder .gradle
Ignore .gradle via sync_list
Sync

Complete Verbose Log Output

Using 'user' Config Dir: /home/niko/.config/onedrive
Using 'system' Config Dir: /etc/onedrive
Configuration file successfully loaded

The use of --resync will remove your local 'onedrive' client state, thus no record will exist regarding your current 'sync status'
This has the potential to overwrite local versions of files with potentially older versions downloaded from OneDrive which can lead to data loss
If in-doubt, backup your local data first before proceeding with --resync

Are you sure you wish to proceed with --resync? [Y/N] y

Using logfile dir: /var/log/onedrive/
Deleting the saved application sync status ...
Checking Application Version ...
Initializing the OneDrive API ...
Configuring Global Azure AD Endpoints
Using Curl defaults for all HTTP operations
Opening the item database ...
All operations will be performed in: /home/niko/OneDrive
Application version: v2.4.25
Account Type: personal
Default Drive ID: 4066fec86c8fc596
Default Root ID: 4066FEC86C8FC596!103
Remaining Free Space: 1005988714905
Fetching details for OneDrive Root
OneDrive Root does not exist in the database. We need to add it.
Added OneDrive Root to the local database
Initializing the Synchronization Engine ...
Syncing changes and items from OneDrive ...
Applying changes of Path ID: 4066FEC86C8FC596!103
Updated Remaining Free Space: 1005988714905
Processing 275 OneDrive items to ensure consistent local state due to sync_list being used
Skipping item - excluded by sync_list config: Programming/Projects/Android/FlashDimCompose/.gradle/config.properties
Skipping item - excluded by sync_list config: Programming/Projects/Android/FlashDimCompose/.gradle/8.5
Skipping item - excluded by sync_list config: Programming/Projects/Android/FlashDimCompose/.gradle/vcs-1
Skipping item - excluded by sync_list config: Programming/Projects/Android/FlashDimCompose/.gradle/file-system.probe
Skipping item - excluded by sync_list config: Programming/Projects/Android/FlashDimCompose/.gradle/buildOutputCleanup
Skipping item - excluded by sync_list config: Programming/Projects/Android/FlashDimCompose/gradle/libs.versions.toml
Skipping item - excluded by sync_list config: Programming/Projects/Android/FlashDimCompose/gradle/wrapper  
...

Screenshots

No response

Other Log Information or Details

No response

Additional context

No response

The text was updated successfully, but these errors were encountered:

abraunegg · 2024-09-02T21:08:08Z

@cyb3rko
Unfortunately where the development cycle is at with v2.5.0 this will not be looked at. v2.5.0 is due for release 16th September 2024.

Please can you upgrade your client to v2.5.0 RC3 and retest your issue. Please read #2770 for details on how to build RC3.

cyb3rko · 2024-09-02T21:10:58Z

@abraunegg
I understand that, then I will just wait until 16th September and test it out on the new release if that's okay.
Otherwise I would have to upgrade my other clients to RC3 as well because of incompatibility, right?

abraunegg · 2024-09-02T21:26:01Z

@abraunegg I understand that, then I will just wait until 16th September and test it out on the new release if that's okay. Otherwise I would have to upgrade my other clients to RC3 as well because of incompatibility, right?

I would rather you find a way to test this before release, so that a fix can be made if required.

The v2.5.0 client is not backwards compatible - many things internally have changed how sync's occur, how integrity is checked - so as an abundance of caution is needed. Please read the client architecture documentation: https://github.com/abraunegg/onedrive/blob/onedrive-v2.5.0-release-candidate-3/docs/client-architecture.md

Please find a way to test this issue with RC3 before the 16th.

cyb3rko · 2024-09-02T23:48:04Z

Tested with the newest version 'onedrive v2.5.0-rc3-58-g0d3d416'.
The issue persists:

Skipping file - excluded by sync_list config: ./Programming/Projects/Android/FlashDimCompose/.gradle/config.properties
Skipping path - excluded by sync_list config: ./Programming/Projects/Android/FlashDimCompose/.gradle/8.5
Skipping path - excluded by sync_list config: ./Programming/Projects/Android/FlashDimCompose/.gradle/vcs-1
Skipping file - excluded by sync_list config: ./Programming/Projects/Android/FlashDimCompose/.gradle/file-system.probe
Skipping path - excluded by sync_list config: ./Programming/Projects/Android/FlashDimCompose/.gradle/buildOutputCleanup
Skipping file - excluded by sync_list config: ./Programming/Projects/Android/FlashDimCompose/gradle/libs.versions.toml
Skipping path - excluded by sync_list config: ./Programming/Projects/Android/FlashDimCompose/gradle/wrapper

abraunegg · 2024-09-03T00:05:30Z

Thanks - will look at this with priority this week.

abraunegg · 2024-09-09T08:46:36Z

@cyb3rko

Regarding this issue, I believe I have this sorted via PR #2797

From this PR, this is the logging output:

Skipping path - excluded by sync_list config: Issue_2796/Programming
Skipping path - excluded by sync_list config: Issue_2796/Programming/Projects
Skipping path - excluded by sync_list config: Issue_2796/Programming/Projects/Android
Skipping path - excluded by sync_list config: Issue_2796/Programming/Projects/Web
Including path - included by sync_list config: Issue_2796/Programming/Projects/Web/folder_a
Including path - included by sync_list config: Issue_2796/Programming/Projects/Android/Web
Including path - included by sync_list config: Issue_2796/Programming/Projects/Android/App1
Including path - included by sync_list config: Issue_2796/Programming/Projects/Android/App2
Including path - included by sync_list config: Issue_2796/Programming/Projects/Web/folder_b
Including path - included by sync_list config: Issue_2796/Programming/Projects/Android/App2/gradle
Skipping path - excluded by sync_list config: Issue_2796/Programming/Projects/Android/Web/.gradle
Including path - included by sync_list config: Issue_2796/Programming/Projects/Android/Web/gradle
Skipping path - excluded by sync_list config: Issue_2796/Programming/Projects/Android/App2/.gradle
Including path - included by sync_list config: Issue_2796/Programming/Projects/Android/App1/gradle
Skipping path - excluded by sync_list config: Issue_2796/Programming/Projects/Android/App1/.gradle
Skipping path - excluded by sync_list config: Issue_2796/Programming/Projects/Android/Web/build
Skipping path - excluded by sync_list config: Issue_2796/Programming/Projects/Android/Web/.cxx
Skipping path - excluded by sync_list config: Issue_2796/Programming/Projects/Android/App1/.cxx
Skipping path - excluded by sync_list config: Issue_2796/Programming/Projects/Android/App1/build
Skipping path - excluded by sync_list config: Issue_2796/Programming/Projects/Android/App2/.cxx
Skipping path - excluded by sync_list config: Issue_2796/Programming/Projects/Android/App2/build
Including path - included by sync_list config: Issue_2796/Programming/Projects/Web/folder_a/sub_folder_a
Including path - included by sync_list config: Issue_2796/Programming/Projects/Web/folder_a/subfolder_b
Including path - included by sync_list config: Issue_2796/Programming/Projects/Android/Web/must_include
Including path - included by sync_list config: Issue_2796/Programming/Projects/Android/App1/must_include
Including path - included by sync_list config: Issue_2796/Programming/Projects/Android/App2/must_include
Including path - included by sync_list config: Issue_2796/Programming/Projects/Web/folder_b/tier_1
Including file - included by sync_list config: Issue_2796/Programming/Projects/Android/App2/gradle/libs.versions.toml
Parental Path structure needs to be created to support included file: Issue_2796/Programming/Projects/Android/App2/gradle
Including file - included by sync_list config: Issue_2796/Programming/Projects/Android/Web/gradle/libs.versions.toml
Parental Path structure needs to be created to support included file: Issue_2796/Programming/Projects/Android/Web/gradle
Including file - included by sync_list config: Issue_2796/Programming/Projects/Android/App1/gradle/libs.versions.toml
Parental Path structure needs to be created to support included file: Issue_2796/Programming/Projects/Android/App1/gradle
Skipping path - excluded by sync_list config: Issue_2796/Programming/Projects/Web/folder_a/sub_folder_a/build
Skipping path - excluded by sync_list config: Issue_2796/Programming/Projects/Web/folder_a/subfolder_b/build
Including path - included by sync_list config: Issue_2796/Programming/Projects/Web/folder_a/sub_folder_a/.cxx
Including path - included by sync_list config: Issue_2796/Programming/Projects/Web/folder_a/subfolder_b/.cxx
Including file - included by sync_list config: Issue_2796/Programming/Projects/Android/Web/must_include/data_to_include.data
Parental Path structure needs to be created to support included file: Issue_2796/Programming/Projects/Android/Web/must_include
Including file - included by sync_list config: Issue_2796/Programming/Projects/Android/App1/must_include/data_to_include.data
Parental Path structure needs to be created to support included file: Issue_2796/Programming/Projects/Android/App1/must_include
Including file - included by sync_list config: Issue_2796/Programming/Projects/Android/App2/must_include/data_to_include.data
Parental Path structure needs to be created to support included file: Issue_2796/Programming/Projects/Android/App2/must_include
Including path - included by sync_list config: Issue_2796/Programming/Projects/Web/folder_b/tier_1/tier_2
Including file - included by sync_list config: Issue_2796/Programming/Projects/Web/folder_a/sub_folder_a/.cxx/build.junk
Parental Path structure needs to be created to support included file: Issue_2796/Programming/Projects/Web/folder_a/sub_folder_a/.cxx
Including file - included by sync_list config: Issue_2796/Programming/Projects/Web/folder_a/subfolder_b/.cxx/build.junk
Parental Path structure needs to be created to support included file: Issue_2796/Programming/Projects/Web/folder_a/subfolder_b/.cxx
Including path - included by sync_list config: Issue_2796/Programming/Projects/Web/folder_b/tier_1/tier_2/.cxx
Including path - included by sync_list config: Issue_2796/Programming/Projects/Web/folder_b/tier_1/tier_2/tier_3
Including file - included by sync_list config: Issue_2796/Programming/Projects/Web/folder_b/tier_1/tier_2/.cxx/build.junk
Parental Path structure needs to be created to support included file: Issue_2796/Programming/Projects/Web/folder_b/tier_1/tier_2/.cxx
Including path - included by sync_list config: Issue_2796/Programming/Projects/Web/folder_b/tier_1/tier_2/tier_3/must_include
Skipping path - excluded by sync_list config: Issue_2796/Programming/Projects/Web/folder_b/tier_1/tier_2/tier_3/build
Including file - included by sync_list config: Issue_2796/Programming/Projects/Web/folder_b/tier_1/tier_2/tier_3/must_include/data_to_include.data
Parental Path structure needs to be created to support included file: Issue_2796/Programming/Projects/Web/folder_b/tier_1/tier_2/tier_3/must_include
Skipping path - excluded by sync_list config: Issue_2796/Programming/Projects/Web/folder_b/tier_1/tier_2/tier_3/build/include_under_build

The application config was:

Selective sync 'sync_list' configured        = true
sync_list config option 'sync_root_files'    = false
sync_list contents:
# EXCLUDE
!.gradle/*
!/Issue_2796/Programming/Projects/Android/**/build/*
!/Issue_2796/Programming/Projects/Android/**/.cxx/*
!/Issue_2796/Programming/Projects/Web/**/build/*

# INCLUDE
/Issue_2796/Programming/*

# INCLUDE ANY PDF FILE ANYWHERE
*.pdf

# INCLUDE ALL TEXT FILES IN SPECIFID DIRECTORY
/new_local_dir/*.txt

Please can you URGENTLY test this.

To test this PR:

git clone https://github.com/abraunegg/onedrive.git
cd onedrive
git fetch origin pull/2797/head:pr2797
git checkout pr2797

To run the PR, you need to run the client from the PR build directory:

./onedrive <any options needed>

To install the PR, you will need to perform sudo make install to install the PR version.

When running the PR, your version should be: onedrive v2.5.0-rc3-62-gc82a285 or greater

cyb3rko · 2024-09-09T09:57:45Z

Thanks for the fix. I will test it out when I come home from work.

cyb3rko · 2024-09-09T18:26:00Z

@abraunegg I've just tested it and it works as expected, thank you a lot! 🚀

abraunegg · 2024-09-09T19:13:51Z

@cyb3rko
Thanks for the confirmation of the issue being resolved.

I need to run that urgently through all my existing 'sync_list' test cases before merging into v2.5.0 RC3.

* Fix issue #2796 before v2.5.0 release * Validate against other 'sync_list' use cases with verification * Update Client Side Filtering documentation

abraunegg · 2024-09-13T02:45:01Z

PR merged into RC3

* Add #2796 details

abraunegg · 2024-09-21T01:49:41Z

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

cyb3rko added the Bug Something isn't working label Sep 2, 2024

abraunegg added Will Not Fix - v2.4.x Waiting on Response labels Sep 2, 2024

abraunegg added In Progress Currently being worked on and removed Waiting on Response labels Sep 4, 2024

abraunegg changed the title ~~Bug: Ignore "folder_name" but sync ".folder_name"~~ Bug: Ignore ".folder_name" but sync "folder_name" Sep 7, 2024

abraunegg added PR Provided - Needs Testing Waiting on Response and removed In Progress Currently being worked on labels Sep 9, 2024

abraunegg added this to the v2.5.0 milestone Sep 9, 2024

abraunegg added Fixed and removed PR Provided - Needs Testing Waiting on Response labels Sep 9, 2024

abraunegg linked a pull request Sep 9, 2024 that will close this issue

Fix issue #2796 #2797

Merged

abraunegg added a commit that referenced this issue Sep 13, 2024

Fix issue #2796 (#2797)

ef6a431

* Fix issue #2796 before v2.5.0 release * Validate against other 'sync_list' use cases with verification * Update Client Side Filtering documentation

abraunegg closed this as completed Sep 13, 2024

abraunegg added a commit that referenced this issue Sep 16, 2024

Update changelog.md

4c84623

* Add #2796 details

BrewTestBot mentioned this issue Sep 16, 2024

onedrive 2.5.0 Homebrew/homebrew-core#190777

Merged

Repository owner locked as resolved and limited conversation to collaborators Sep 21, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bug: Ignore ".folder_name" but sync "folder_name" #2796

Bug: Ignore ".folder_name" but sync "folder_name" #2796

cyb3rko commented Sep 2, 2024 •

edited

Loading

abraunegg commented Sep 2, 2024

cyb3rko commented Sep 2, 2024

abraunegg commented Sep 2, 2024

cyb3rko commented Sep 2, 2024

abraunegg commented Sep 3, 2024

abraunegg commented Sep 9, 2024

cyb3rko commented Sep 9, 2024

cyb3rko commented Sep 9, 2024 •

edited

Loading

abraunegg commented Sep 9, 2024

abraunegg commented Sep 13, 2024

abraunegg commented Sep 21, 2024

Bug: Ignore ".folder_name" but sync "folder_name" #2796

Bug: Ignore ".folder_name" but sync "folder_name" #2796

Comments

cyb3rko commented Sep 2, 2024 • edited Loading

Describe the bug

Operating System Details

Client Installation Method

OneDrive Account Type

What is your OneDrive Application Version

What is your OneDrive Application Configuration

What is your 'curl' version

Where is your 'sync_dir' located

What are all your system 'mount points'

What are all your local file system partition types

How do you use 'onedrive'

Steps to reproduce the behaviour

Complete Verbose Log Output

Screenshots

Other Log Information or Details

Additional context

abraunegg commented Sep 2, 2024

cyb3rko commented Sep 2, 2024

abraunegg commented Sep 2, 2024

cyb3rko commented Sep 2, 2024

abraunegg commented Sep 3, 2024

abraunegg commented Sep 9, 2024

cyb3rko commented Sep 9, 2024

cyb3rko commented Sep 9, 2024 • edited Loading

abraunegg commented Sep 9, 2024

abraunegg commented Sep 13, 2024

abraunegg commented Sep 21, 2024

cyb3rko commented Sep 2, 2024 •

edited

Loading

cyb3rko commented Sep 9, 2024 •

edited

Loading