for loop invoked from generic procedure defined in another module cant find items iterator

[SolitudeSF]

Template with for loop invoked from generic procedure defined in another cant find items iterator

Example

a.nim

iterator items(a: int): int = discard

template t(a: int): untyped =
  for _ in a:
    discard

proc p*(a: int | uint) =
  t a

test.nim

import a

p 1

Current Output

a.nim(1, 10) Hint: 'items' is declared but not used [XDeclaredButNotUsed]
test.nim(3, 3) template/generic instantiation of `p` from here
a.nim(8, 5) Error: type mismatch: got <int>
but expected one of: 
iterator items[T](a: seq[T]): T
iterator items(a: string): char
iterator items[IX, T](a: array[IX, T]): T
iterator items[T](a: set[T]): T
iterator items(a: cstring): char
iterator items[T](a: openArray[T]): T
iterator items[T](s: HSlice[T, T]): T
iterator items(E: typedesc[enum]): E:type

Expected Output

Should compile and run without errors.

Additional Information

Removing generic component from p or running p from the same module removes the error.

$ nim -v
Nim Compiler Version 0.19.9 [Linux: amd64]
Compiled at 2019-05-03
git hash: 515ab81477c1c3e4811c4fbf43a3ff81b87be970

links

• similar (but different) issue: in inside a template not lowered to contains if called from another module #18150

[nc-x]

It also works if items is exported or using for _ in a.items().

[krux02]

I looks like semTemplate does not introduce the implicit items call for the iterator, and maybe it should not. If it should not, this is not a bug, but it is certainly something that should be documented.

[timotheecour]

looks like template is irrelevant, see reduced case below
@SolitudeSF maybe change the title to:

for loop invoked from generic procedure defined in another module cant find `items` iterator

?

reduced case: replace a.nim with:

  iterator items(a: int): int = discard
  proc p*(a: int | uint) =
    for _ in a:
      discard

[timotheecour]

workaround

use for _ in items(a) instead of for _ in a makes it work, both in original post as well as in reduced case i wrote above #11167 (comment)

(works both with or without mixin items)

potential solution

maybe semcheck of generic should early expand the for loop with items / pairs transformations, but needs to work with https://nim-lang.org/docs/manual.html#generics-symbol-lookup-in-generics in mind

EDIT

related (perhaps even duplicate) of #4773; see for example: #4773 (comment)
ideally though, instead of patching stdlib to use items/pairs everywhere, this should just work

[pmetras]

Another simple case where the problem occurs in nim 1.0.4 and #head:

First module ee.nim:

import sets

proc initH*[V]: HashSet[V] =
  result = initHashSet[V]()

proc foo*[V](h: var HashSet[V], c: seq[V]) =
  h = h + c.toHashSet()

Second module ff.nim

import hashes

import ee

type
  Choice = object
    i: int

proc hash(c: Choice): Hash =
  result = Hash(c.i)

var h = initH[Choice]()
let c = @[Choice(i: 1)]

foo(h, c)

Compilation of ff.nim shows the error:

$ nim c ff
Hint: used config file '/home/pierre/.choosenim/toolchains/nim-#devel/config/nim.cfg' [Conf]
Hint: used config file '/home/pierre/.choosenim/toolchains/nim-#devel/config/config.nims' [Conf]
Hint: system [Processing]
Hint: widestrs [Processing]
Hint: io [Processing]
Hint: ff [Processing]
Hint: hashes [Processing]
Hint: ee [Processing]
Hint: sets [Processing]
Hint: math [Processing]
Hint: bitops [Processing]
Hint: macros [Processing]
/home/pierre/alternaCloud/Temp/sdk/ff.nim(17, 4) template/generic instantiation of `foo` from here
/home/pierre/alternaCloud/Temp/sdk/ee.nim(9, 9) template/generic instantiation of `+` from here
/home/pierre/.choosenim/toolchains/nim-#devel/lib/pure/collections/sets.nim(487, 17) template/generic instantiation of `union` from here
/home/pierre/.choosenim/toolchains/nim-#devel/lib/pure/collections/sets.nim(411, 7) template/generic instantiation of `incl` from here
/home/pierre/.choosenim/toolchains/nim-#devel/lib/pure/collections/sets.nim(211, 15) Error: type mismatch: got <HashSet[ff.Choice]>
but expected one of: 
iterator items[T](a: seq[T]): T
  first type mismatch at position: 1
  required type for a: seq[T]
  but expression 'other' is of type: HashSet[ff.Choice]
iterator items[T](s: HSlice[T, T]): T
  first type mismatch at position: 1
  required type for s: HSlice[items.T, items.T]
  but expression 'other' is of type: HashSet[ff.Choice]
iterator items(a: string): char
  first type mismatch at position: 1
  required type for a: string
  but expression 'other' is of type: HashSet[ff.Choice]
iterator items[IX, T](a: array[IX, T]): T
  first type mismatch at position: 1
  required type for a: array[IX, T]
  but expression 'other' is of type: HashSet[ff.Choice]
iterator items[T](a: set[T]): T
  first type mismatch at position: 1
  required type for a: set[T]
  but expression 'other' is of type: HashSet[ff.Choice]
iterator items(a: cstring): char
  first type mismatch at position: 1
  required type for a: cstring
  but expression 'other' is of type: HashSet[ff.Choice]
iterator items[T](a: openArray[T]): T
  first type mismatch at position: 1
  required type for a: openArray[T]
  but expression 'other' is of type: HashSet[ff.Choice]
iterator items(E: typedesc[enum]): E:type
  first type mismatch at position: 1
  required type for E: type enum
  but expression 'other' is of type: HashSet[ff.Choice]

expression: items(other)

If both files are merged, the error disappear. Or if sets is imported in ff.nim and initH is not used.

[pmetras]

Simply adding import sets in the second module ff.nim makes the error disappear.

[timotheecour]

bumping priority; keeps happening, see #16060

[oakes]

I noticed that with the latest devel, sets.len also must be exported. I made foo.nim:

import sets, sequtils

proc dedupe*[T](arr: openArray[T]): seq[T] =
  arr.toHashSet.toSeq

export sets.items

...and bar.nim:

import foo

echo dedupe([1, 2, 3])

Running nim c -r bar.nim gives me:

C:\Users\sekao\Documents\Nim\lib\pure\collections\sets.nim(241, 17) Error: type mismatch: got <HashSet[system.int]>
but expected one of:
func len(x: (type array) | array): int
  first type mismatch at position: 1
  required type for x: typedesc[array] or array
  but expression 's' is of type: HashSet[system.int]
func len(x: string): int
func len[TOpenArray: openArray | varargs](x: TOpenArray): int                                                             first type mismatch at position: 1                                                                                      required type for x: TOpenArray: openArray or varargs                                                                   but expression 's' is of type: HashSet[system.int]
...

Exporting sets.len as well fixes it.

nim -v
Nim Compiler Version 1.5.1 [Windows: amd64]
Compiled at 2021-02-21
Copyright (c) 2006-2021 by Andreas Rumpf

git hash: 70ec17eede74f4216aee3b0bfb11e789bcbf0e54
active boot switches: -d:release

[timotheecour]

@oakes you can send a PR to fix it similar to https://github.com/nim-lang/Nim/pull/16060/files

[timotheecour]

candidate places to look at for a fix:

• implicitIterator
• binding mechanism in generics that should transform for x in b by for x in items(b) (ditto with pairs) before binding is done; but possibly with mixing items ?

[Araq]

binding mechanism in generics that should transform for x in b by for x in items(b) (ditto with pairs) before binding is done; but possibly with mixing items ?

Yeah... But it changes the AST and isn't correct for when b ends up being an iterator...