Cannot compute centralizers in matrix group

[hungaborhorvath]

This affects current master with packages. This has been revealed already in #973, but that issue will be fixed by #985 using the suggestions from @hulpke. However, the following is still a problem in current master, so I decided to report it again without all the other problems going on in #973.

 ┌───────┐   GAP v4.8.6-925-g8efd681 of 2016-12-06 08:13:49 (CET)
 │  GAP  │   http://www.gap-system.org
 └───────┘   Architecture: x86_64-pc-linux-gnu-gcc-default64
 Libs used:  gmp, readline
 Loading the library and packages ...
 Components: trans 1.0, prim 3.0, small* 1.0, id* 1.0
 Packages:   AClib 1.2, Alnuth 3.0.0, AtlasRep 1.5.1, AutPGrp 1.6, 
             Browse 1.8.6, CRISP 1.4.4, Cryst 4.1.12, CrystCat 1.1.6, 
             CTblLib 1.2.2, FactInt 1.5.3, FGA 1.3.1, GAPDoc 1.5.1, IO 4.4.6, 
             IRREDSOL 1.3.1, LAGUNA 3.7.0, Polenta 1.3.7, Polycyclic 2.11, 
             RadiRoot 2.7, ResClasses 4.5.0, Sophus 1.23, SpinSym 1.5, 
             TomLib 1.2.6, Utils 0.43
 Try '??help' for help. See also '?copyright', '?cite' and '?authors'
gap> testG :=
>    function ( a, b )
>      local  M1;
>       M1 := [ [ [      0, -E(a)^-1 ], [ -E(a),       0 ] ],
>               [ [      0,       -1 ], [     1,       0 ] ],
>               [ [ E(4*b),        0 ], [     0, -E(4*b) ] ],
>               [ [     -1,        0 ], [     0,      -1 ] ]];
>       return (Group(M1));
>    end;;
gap> G := testG(8,2);;
gap> for N in NormalSubgroups(G) do for H in ComplementClassesRepresentatives(G, N) do Centralizer(H, N); od; od;
Error, AppendList: <list2> must be a small list (not a boolean or fail) in
  Append( resrow, Coefficients( B, entry * b ) 
 ); at /home/ghorvath/work/gap/lib/matrix.gi:3082 called from 
BlownUpMat( Basis( iso ), mat 
 ) at /home/ghorvath/work/gap/lib/grpmat.gi:993 called from
ImagesRepresentative( nice, elm 
 ) at /home/ghorvath/work/gap/lib/grpnice.gi:224 called from
gen in G at /home/ghorvath/work/gap/lib/grp.gi:3303 called from
func( elm ) at /home/ghorvath/work/gap/lib/coll.gi:1613 called from
ForAll( GeneratorsOfGroup( H ), function ( gen )
      return gen in G;
  end ) at /home/ghorvath/work/gap/lib/grp.gi:3303 called from
...  at *stdin*:11
you can replace <list2> via 'return <list2>;'
brk> 

I would think this is a bug, but maybe I am wrong here?

Note, that current master without packages is not affected. There the calculation runs through, albeit very slowly:

 ┌───────┐   GAP v4.8.6-925-g8efd681 of 2016-12-06 08:13:49 (CET)
 │  GAP  │   http://www.gap-system.org
 └───────┘   Architecture: x86_64-pc-linux-gnu-gcc-default64
 Libs used:  gmp, readline
 Loading the library and packages ...
 Components: trans 1.0, prim 3.0, small* 1.0, id* 1.0
 Packages:   GAPDoc 1.5.1
 Try '??help' for help. See also '?copyright', '?cite' and '?authors'
gap> testG :=
>    function ( a, b )
>      local  M1;
>       M1 := [ [ [      0, -E(a)^-1 ], [ -E(a),       0 ] ],
>               [ [      0,       -1 ], [     1,       0 ] ],
>               [ [ E(4*b),        0 ], [     0, -E(4*b) ] ],
>               [ [     -1,        0 ], [     0,      -1 ] ]];
>       return (Group(M1));
>    end;;
gap> G := testG(8,2);;
gap> for N in NormalSubgroups(G) do for H in ComplementClassesRepresentatives(G, N) do Centralizer(H, N); od; od;
gap> 

Stable 4.8.6 (downloaded from website) also quickly breaks down with the usual packages:

 ┌───────┐   GAP 4.8.6, 12-Nov-2016, build of 2016-11-14 16:42:56 (CET)
 │  GAP  │   http://www.gap-system.org
 └───────┘   Architecture: x86_64-pc-linux-gnu-gcc-default64
 Libs used:  gmp, readline
 Loading the library and packages ...
 Components: trans 1.0, prim 2.1, small* 1.0, id* 1.0
 Packages:   AClib 1.2, Alnuth 3.0.0, AtlasRep 1.5.1, AutPGrp 1.6, 
             Browse 1.8.6, CRISP 1.4.4, Cryst 4.1.12, CrystCat 1.1.6, 
             CTblLib 1.2.2, FactInt 1.5.3, FGA 1.3.1, GAPDoc 1.5.1, IO 4.4.6, 
             IRREDSOL 1.3.1, LAGUNA 3.7.0, Polenta 1.3.7, Polycyclic 2.11, 
             RadiRoot 2.7, ResClasses 4.5.0, Sophus 1.23, SpinSym 1.5, 
             TomLib 1.2.6, Utils 0.43
 Try '??help' for help. See also '?copyright', '?cite' and '?authors'
gap> testG :=
>    function ( a, b )
>      local  M1;
>       M1 := [ [ [      0, -E(a)^-1 ], [ -E(a),       0 ] ],
>               [ [      0,       -1 ], [     1,       0 ] ],
>               [ [ E(4*b),        0 ], [     0, -E(4*b) ] ],
>               [ [     -1,        0 ], [     0,      -1 ] ]];
>       return (Group(M1));
>    end;;
gap> G := testG(8,2);;
gap> for N in NormalSubgroups(G) do for H in ComplementClassesRepresentatives(G, N) do Centralizer(H, N); od; od;
Error, AppendList: <list2> must be a small list (not a boolean or fail) in
  Append( resrow, Coefficients( B, entry * b ) 
 ); at /home/ghorvath/work/gap4r8/lib/matrix.gi:3082 called from 
BlownUpMat( Basis( iso ), mat 
 ) at /home/ghorvath/work/gap4r8/lib/grpmat.gi:993 called from
ImagesRepresentative( nice, elm 
 ) at /home/ghorvath/work/gap4r8/lib/grpnice.gi:224 called from
gen in G at /home/ghorvath/work/gap4r8/lib/grp.gi:2846 called from
func( elm ) at /home/ghorvath/work/gap4r8/lib/coll.gi:1613 called from
ForAll( GeneratorsOfGroup( H ), function ( gen )
      return gen in G;
  end ) at /home/ghorvath/work/gap4r8/lib/grp.gi:2846 called from
...  at line 11 of *stdin*
you can replace <list2> via 'return <list2>;'
brk> 

[hulpke]

The problem here is triggered by the fact that at some point the subgroup H get created anew and does not inherit the nice monomorphism of G but its own (which is defined over a smaller field). When trying to apply it to N it runs into trouble.

This is a more fundamental question -- how robust against elements from left field do nice monomorphisms have to be? Resolving this (without performance penalty) is likely to be messy, and I have absolutely no interest in working on this for what is a rather esoteric situation that can be easily resolved by moving to an isomorphic permutation representation.

In the long run the nice monomorphisms should be replaced by recognition, and there a similar issue will come up -- what if elements do not preserve the geometric structure? I think the cleanest way of dealing with this without performance penalty is (similarly as Magma does) actually to start associating with matrices a ``largest group'' (which has been recognized, and in which all calculations take place), so we can quickly verify that many of the matrices will be good to go from start. But that is really another problem.