Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LDAP error when indexing many files #175

Closed
linuxrrze opened this issue May 5, 2017 · 5 comments
Closed

LDAP error when indexing many files #175

linuxrrze opened this issue May 5, 2017 · 5 comments

Comments

@linuxrrze
Copy link

While trying to index quite some files (~145k) with nextant, I always get the following error after all files were processed (which takes > 20h).

It looks like the (time consuming) data crawling part is finished when this happens, however the status seems to be not committed, as on restart the whole indexing process starts anew from the beginning.

The bug is obviously LDAP related, so I tried to increase the LDAP timeout limit (from 15s default to 120s) but this did not help either.

My setup:

  • nextcloud 11.0.3
  • nextant 1.0.8
  • user_ldap

LDAP server is Samba 4.6.2 (domain controller mode).

# sudo -u www-data php occ nextant:index
nextant v1.0.8

* Extracting files:

/Administrator/files            12/    12 [============================] 100%
/dani/files                      7/     7 [============================] 100%
/flo/files                       7/     7 [============================] 100%
/frank/files                     7/     7 [============================] 100%
/ijp/files                       7/     7 [============================] 100%
/marcel/files               145150/145150 [============================] 100%


                                    
  [OC\ServerNotAvailableException]  
  Lost connection to LDAP server.   
                                    

nextant:index [--output [OUTPUT]] [--debug] [--debugall [DEBUGALL]] [-k|--unlock] [-f|--force] [-u|--user [USER]] [-c|--background] [-b|--bookmarks] [-i|--files] [-x|--files_extract] [-r|--files_update]

@JasperZ
Copy link

JasperZ commented May 14, 2017

I get exactly the same error message with the following setup:

  • Nextcloud(11.0.3.2)
  • Nextant(1.0.8)
  • LDAP user and group backend(1.1.2)

This is the output I'm getting:

# sudo -u apache ./occ nextant:index
nextant v1.0.8

* Extracting files:

/0912c10a-b43111e6-8376fd05-fab6b4e8     0/     0 [>---------------------------]   0% 
/17c2d982-d66711e6-8e6cfd05-fab6b4e8/files     8/     8 [============================] 100% 
/2d947306-be0b11e6-8cbcfd05-fab6b4e8    68/    68 [============================] 100% 
/32164103-931e11e6-8376fd05-fab6b4e8     0/     0 [>---------------------------]   0% 
/69DDFAC9-5B90-44EF-A396-8D5DBFAE0216/files  2158/  2158 [============================] 100% 
/77c12304-b33a11e6-8376fd05-fab6b4e8     0/     0 [>---------------------------]   0% 
/B08AB0AA-1934-4D24-A705-4BB3CBA7DBF5/files   741/   741 [============================] 100% 
/B883F0E4-313B-4D67-9D86-60DCA5FC43F4/files  1600/  1600 [============================] 100% 
/admin/files                     1/     1 [============================] 100% 
/cddf4d04-909111e6-8376fd05-fab6b4e8/files 37377/ 37377 [============================] 100% 
/fc96920c-d5b211e6-8e6cfd05-fab6b4e8     0/     0 [>---------------------------]   0% 
                                                   
  1 file(s) processed ; 0 orphan(s) removed
  5760 documents indexed ; 5760 fully extracted

* Updating files:



                                    
  [OC\ServerNotAvailableException]  
  Lost connection to LDAP server.   
                                    

nextant:index [--output [OUTPUT]] [--debug] [--debugall [DEBUGALL]] [-k|--unlock] [-f|--force] [-u|--user [USER]] [-c|--background] [-b|--bookmarks] [-i|--files] [-x|--files_extract] [-r|--files_update]

Segmentation fault

EDIT: My Nextcloud instance connects to two different ldap servers(Active Directory and FreeIPA), if i disable one of them(doesn't matter which one) the index gets created without any problems. But when I try to create the index with both ldap connections enabled i get the error message from above.

@ArtificialOwl
Copy link
Member

Can you try this patch for NC11 ?
nextcloud/server#5128

@JasperZ
Copy link

JasperZ commented May 26, 2017

Since I already updated my nextcloud to version 12 I can't apply this patch anymore. But with the update the issue disappeared and the index creation finishes successfully!

@ArtificialOwl
Copy link
Member

I linked the backport to NC11.

This is the patch for NC12: nextcloud/server#5104
Before 12.0.1, if you experience some issue, please try this patch

@linuxrrze
Copy link
Author

Tested patch with nextcloud 11.0.3:

No more exception -> works.

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants