LDAP search failed: Result Code 4 "Size Limit Exceeded"

[csquire]

Describe the bug
LDAP queries that return a large number of results are still erroring with the fix for #4162 . When an LDAP server is setup to only return 1000 results a group query which returns more than that will fail.

To Reproduce
Steps to reproduce the behavior:

1. Setup LDAP auth with a group filter that will return more than 1000 groups
2. Attempt to login
3. Login fails with LDAP search failed: LDAP Result Code 4 "Size Limit Exceeded"

Expected behavior
The login should be successful and all groups should be returned in the query response so authentication will succeed.

Environment:

• Vault Server Version (retrieve with vault status): 1.2.3
• Vault CLI Version (retrieve with vault version): 1.2.3
• Server Operating System/Architecture: Linux

Vault server configuration file(s):

vault write auth/ldap/config \
  url="ldaps://ldap.example.com:636" \
  userdn="OU=Users,OU=Org,OU=BusinessUnits,DC=example,DC=com" \
  groupdn="OU=AccountGroups,OU=Groups,OU=Org,OU=BusinessUnits,DC=example,DC=com" \
  groupfilter="(&(objectClass=group))" \
  groupattr="memberOf" \
  upndomain="example.com" \
  insecure_tls=false \
  starttls=false

vault write auth/ldap/groups/developers policies=dev

Additional context
To see the problem, the LDAP server must be setup to return fewer groups than the number of groups which match the query.

This search appears to need updated: https://github.com/hashicorp/vault/blob/master/sdk/helper/ldaputil/client.go#L120-L125

Should use something like this:

result, err := conn.SearchWithPaging(&ldap.SearchRequest{
  BaseDN:    cfg.UserDN,
  Scope:     ldap.ScopeWholeSubtree,
  Filter:    filter,
  SizeLimit: math.MaxInt32,
}, math.MaxInt32)

Example using go-ldap. Given the proper group query, the first search will fail and the second will succeed:

package main

import (
	"fmt"
	"log"
	"math"

	"github.com/go-ldap/ldap"
)

func main() {
	l, err := ldap.Dial("tcp", "ldap.example.com:389")
	if err != nil {
		log.Fatal(err)
	}
	defer l.Close()
	l.Bind("<binduser>", "<bindpw>")
	request := &ldap.SearchRequest{
		BaseDN: "OU=AccountGroups,OU=Groups,OU=Org,OU=BusinessUnits,DC=example,DC=com",
		Scope:  ldap.ScopeWholeSubtree,
		Filter: "(&(objectClass=group))",
		Attributes: []string{
			"member:1.2.840.113556.1.4.1941:",
		},
		SizeLimit: math.MaxInt32,
	}

	//ERROR: SIZE LIMIT EXCEEDED
	sr, err := l.Search(request)
	if err != nil {
		log.Print(err)
	} else {
		for _, entry := range sr.Entries {
			fmt.Printf("%s: %v\n", entry.DN, entry.GetAttributeValue("cn"))
		}
	}

	//WORKS
	sr, err = l.SearchWithPaging(request, math.MaxInt32)
	if err != nil {
		log.Fatal(err)
	}

	for _, entry := range sr.Entries {
		fmt.Printf("%s: %v\n", entry.DN, entry.GetAttributeValue("cn"))
	}
}

[jefferai]

Any chance you're willing to PR that? :-)

[csquire]

Sure, I can try to make some time

[csquire]

We were able to work around this for now by changing our group filter.