Notes: QO calculated from LCAO at some kpoint has very bad condition number

[kirk0830]

Details

Just a note leave here, for some k-point, QO has very bad condition number, these will bring about problem when doing kpoint extrapolation.
For many of kpoints, QO can reproduce band structures well (by selecting 4-th to 8-th bands to reproduce):

--------------------------------------------------
For k-point No. 709
Calculate QO.
QO Normalization: after, norm of the  0 -th QO is: 1.00000000
QO Normalization: after, norm of the  1 -th QO is: 1.00000000
QO Normalization: after, norm of the  2 -th QO is: 1.00000000
QO Normalization: after, norm of the  3 -th QO is: 1.00000000
QO Normalization: after, norm of the  4 -th QO is: 1.00000000
QO Normalization: after, norm of the  5 -th QO is: 1.00000000
QO Normalization: after, norm of the  6 -th QO is: 1.00000000
QO Normalization: after, norm of the  7 -th QO is: 1.00000000
Calculate hamiltonian matrix in QO basis in k-space.
Eigenvalues of overlap of in basis Sqo(k) are:
 [1.37591919e-05 1.91976105e-05 4.89133630e-05 6.45648954e-05
 1.67252582e-03 6.38869074e-03 1.26278042e-02 7.97916454e+00]
Eigenvalues of Hamiltonian in QO basis Hqo(k) are:
 [0.34982477 0.69600839 0.7633131  0.88039793 0.89976318 2.4037417
 5.36962957 8.45016526]
Eigenvalues of Hamiltonian in NAO basis H(k) are:
 [-0.28226041  0.07554561  0.24690713  0.34982477  0.69600839  0.7633131
  0.88039793  0.89976318  1.1614511   1.33479144  1.4129278   1.62183167
  1.80650309  1.86654435  2.04743874  2.11125568  2.16577733  2.34028265
  2.51328901  2.97026378  3.65447573  4.06235185  6.11166278  7.80674016
  8.47872031 10.37336728]
--------------------------------------------------
For k-point No. 710
Calculate QO.
QO Normalization: after, norm of the  0 -th QO is: 1.00000000
QO Normalization: after, norm of the  1 -th QO is: 1.00000000
QO Normalization: after, norm of the  2 -th QO is: 1.00000000
QO Normalization: after, norm of the  3 -th QO is: 1.00000000
QO Normalization: after, norm of the  4 -th QO is: 1.00000000
QO Normalization: after, norm of the  5 -th QO is: 1.00000000
QO Normalization: after, norm of the  6 -th QO is: 1.00000000
QO Normalization: after, norm of the  7 -th QO is: 1.00000000
Calculate hamiltonian matrix in QO basis in k-space.
Eigenvalues of overlap of in basis Sqo(k) are: 
 [2.55515636e-05 3.58777609e-05 5.60451697e-05 7.40148454e-05
 3.80573917e-03 2.29179947e-02 6.52376137e-02 7.90784716e+00]
Eigenvalues of Hamiltonian in QO basis Hqo(k) are:
 [0.31231487 0.62105173 0.82877096 0.88208311 0.91599265 5.84995075
 7.80215918 9.25252437]
Eigenvalues of Hamiltonian in NAO basis H(k) are:
 [-0.25054789  0.03528504  0.24191266  0.31231487  0.62105173  0.82877096
  0.88208311  0.91599265  1.33781311  1.37202076  1.40322039  1.44691195
  1.8415802   1.91742587  2.00216059  2.05701202  2.23696654  2.39167068
  2.39880304  2.93749718  3.21040033  3.72204254  6.58992061  7.92319413
  8.66731331 10.10129888]
--------------------------------------------------
For k-point No. 711
Calculate QO.
QO Normalization: after, norm of the  0 -th QO is: 1.00000000
QO Normalization: after, norm of the  1 -th QO is: 1.00000000
QO Normalization: after, norm of the  2 -th QO is: 1.00000000
QO Normalization: after, norm of the  3 -th QO is: 1.00000000
QO Normalization: after, norm of the  4 -th QO is: 1.00000000
QO Normalization: after, norm of the  5 -th QO is: 1.00000000
QO Normalization: after, norm of the  6 -th QO is: 1.00000000
QO Normalization: after, norm of the  7 -th QO is: 1.00000000
Calculate hamiltonian matrix in QO basis in k-space.
Eigenvalues of overlap of in basis Sqo(k) are:
 [3.40606669e-06 8.59305425e-06 5.61631694e-05 1.08563096e-04
 1.41437203e-03 2.97532058e-02 1.20561485e+00 6.76304085e+00]
Eigenvalues of Hamiltonian in QO basis Hqo(k) are:
 [0.3262208  0.70283112 0.7457545  0.89506182 1.00581617 5.91399415
 7.93434594 9.90148176]
Eigenvalues of Hamiltonian in NAO basis H(k) are:
 [-0.2240069  -0.01928574  0.23216319  0.3262208   0.70283112  0.7457545
  0.89506182  1.00581617  1.17183244  1.29356157  1.48507097  1.58852431
  1.75348654  1.9581841   2.02390915  2.11029909  2.1258369   2.25668741
  2.48324856  2.88243831  3.71398143  3.81608756  6.48049499  7.00525089
  8.29873407 10.39827421]
--------------------------------------------------
For k-point No. 712
Calculate QO.
QO Normalization: after, norm of the  0 -th QO is: 1.00000000
QO Normalization: after, norm of the  1 -th QO is: 1.00000000
QO Normalization: after, norm of the  2 -th QO is: 1.00000000
QO Normalization: after, norm of the  3 -th QO is: 1.00000000
QO Normalization: after, norm of the  4 -th QO is: 1.00000000
QO Normalization: after, norm of the  5 -th QO is: 1.00000000
QO Normalization: after, norm of the  6 -th QO is: 1.00000000
QO Normalization: after, norm of the  7 -th QO is: 1.00000000
Calculate hamiltonian matrix in QO basis in k-space.
Eigenvalues of overlap of in basis Sqo(k) are:
 [4.86063550e-06 4.28594442e-05 1.15326897e-04 1.95414585e-04
 5.65420952e-03 4.13453022e-01 2.63942490e+00 4.94110940e+00]
Eigenvalues of Hamiltonian in QO basis Hqo(k) are:
 [0.2772957  0.68912551 0.83425731 0.95768298 1.00643448 7.50524433
 8.41977832 9.24465157]
Eigenvalues of Hamiltonian in NAO basis H(k) are:
 [-0.17420403 -0.03780389  0.16037801  0.2772957   0.68912551  0.83425731
  0.95768298  1.00643448  1.05228007  1.28094533  1.53146432  1.76248423
  1.83009217  1.84672455  2.00801446  2.02147789  2.05210757  2.17525211
  2.48118134  2.70711254  3.56917076  3.78714936  7.47595241  7.88252627
  8.71764412  9.78042546]
-------------------------------------------------- 
For k-point No. 713
Calculate QO.
QO Normalization: after, norm of the  0 -th QO is: 1.00000000
QO Normalization: after, norm of the  1 -th QO is: 1.00000000
QO Normalization: after, norm of the  2 -th QO is: 1.00000000
QO Normalization: after, norm of the  3 -th QO is: 1.00000000
QO Normalization: after, norm of the  4 -th QO is: 1.00000000
QO Normalization: after, norm of the  5 -th QO is: 1.00000000
QO Normalization: after, norm of the  6 -th QO is: 1.00000000
QO Normalization: after, norm of the  7 -th QO is: 1.00000000
Calculate hamiltonian matrix in QO basis in k-space.
Eigenvalues of overlap of in basis Sqo(k) are:
 [2.56424091e-05 1.01803587e-04 1.82552238e-04 3.38342177e-04
 2.20167242e-02 2.10139040e+00 2.35984746e+00 3.51609708e+00]
Eigenvalues of Hamiltonian in QO basis Hqo(k) are:
 [0.24090821 0.65063716 0.77293776 0.99820207 1.02695925 8.33044921
 8.64439726 8.9239683 ]
Eigenvalues of Hamiltonian in NAO basis H(k) are:
 [-0.12394661 -0.08005865  0.16371306  0.24090821  0.65063716  0.77293776
  0.99820207  1.02695925  1.19707558  1.28836898  1.53553813  1.64774402
  1.7815941   1.84203562  2.02441282  2.03316324  2.11364098  2.19767274
  2.47143849  2.58820847  3.37930345  3.6623095   8.02954105  8.56570043
  9.19631302  9.51782525]
--------------------------------------------------
For k-point No. 714
Calculate QO.
QO Normalization: after, norm of the  0 -th QO is: 1.00000000
QO Normalization: after, norm of the  1 -th QO is: 1.00000000
QO Normalization: after, norm of the  2 -th QO is: 1.00000000
QO Normalization: after, norm of the  3 -th QO is: 1.00000000
QO Normalization: after, norm of the  4 -th QO is: 1.00000000
QO Normalization: after, norm of the  5 -th QO is: 1.00000000
QO Normalization: after, norm of the  6 -th QO is: 1.00000000
QO Normalization: after, norm of the  7 -th QO is: 1.00000000
Calculate hamiltonian matrix in QO basis in k-space.
Eigenvalues of overlap of in basis Sqo(k) are:
 [2.24435696e-05 1.87135595e-04 2.11001458e-04 4.88190425e-04
 4.42370185e-02 6.57863073e-01 2.25377331e+00 5.04321783e+00]
Eigenvalues of Hamiltonian in QO basis Hqo(k) are:
 [0.23446706 0.61736052 0.65203888 1.03862458 1.16872639 7.82172055
 8.24280988 9.49507204]
Eigenvalues of Hamiltonian in NAO basis H(k) are:
 [-0.16212153 -0.04638285  0.21853842  0.23446706  0.61736052  0.65203888
  1.03862458  1.16872639  1.22651329  1.35275419  1.47547462  1.60889536
  1.62288553  1.78973169  2.07319422  2.11804647  2.20514678  2.29780496
  2.4607455   2.55132756  3.22857104  3.30794628  8.20318512  8.33271175
  8.69395121 10.17917967]
--------------------------------------------------
For k-point No. 715
Calculate QO.
QO Normalization: after, norm of the  0 -th QO is: 1.00000000
QO Normalization: after, norm of the  1 -th QO is: 1.00000000
QO Normalization: after, norm of the  2 -th QO is: 1.00000000
QO Normalization: after, norm of the  3 -th QO is: 1.00000000
QO Normalization: after, norm of the  4 -th QO is: 1.00000000
QO Normalization: after, norm of the  5 -th QO is: 1.00000000
QO Normalization: after, norm of the  6 -th QO is: 1.00000000
QO Normalization: after, norm of the  7 -th QO is: 1.00000000
Calculate hamiltonian matrix in QO basis in k-space.
Eigenvalues of overlap of in basis Sqo(k) are:
 [9.03275309e-05 1.22297723e-04 1.90042734e-04 3.24004760e-04
 2.85643381e-02 1.04254456e-01 5.02472751e-01 7.36398178e+00]
Eigenvalues of Hamiltonian in QO basis Hqo(k) are:
 [0.27822259 0.56635153 0.66405417 1.05251601 1.05984917 6.93093617
 8.06223153 9.35997322]
Eigenvalues of Hamiltonian in NAO basis H(k) are:
 [-0.21991531  0.01800269  0.23536461  0.27822259  0.56635153  0.66405417
  1.05251601  1.05984917  1.36055404  1.40882631  1.42768068  1.42901799
  1.69713843  1.82310113  2.05374275  2.12637425  2.29565142  2.35764977
  2.43766914  2.75848382  2.90251389  3.28720859  7.62405519  8.38795304
  8.59804795 10.14421769]
--------------------------------------------------
For k-point No. 716
Calculate QO.
QO Normalization: after, norm of the  0 -th QO is: 1.00000000
QO Normalization: after, norm of the  1 -th QO is: 1.00000000
QO Normalization: after, norm of the  2 -th QO is: 1.00000000
QO Normalization: after, norm of the  3 -th QO is: 1.00000000
QO Normalization: after, norm of the  4 -th QO is: 1.00000000
QO Normalization: after, norm of the  5 -th QO is: 1.00000000
QO Normalization: after, norm of the  6 -th QO is: 1.00000000
QO Normalization: after, norm of the  7 -th QO is: 1.00000000
Calculate hamiltonian matrix in QO basis in k-space.
Eigenvalues of overlap of in basis Sqo(k) are:
 [4.31514777e-05 5.51351427e-05 9.01073806e-05 1.86133616e-04
 2.23429263e-02 1.09232696e-01 1.84807538e+00 6.01997447e+00]
Eigenvalues of Hamiltonian in QO basis Hqo(k) are:
 [0.29179575 0.62434642 0.73511123 0.89905188 1.01541175 6.06096708
 7.38688387 8.7887353 ]
Eigenvalues of Hamiltonian in NAO basis H(k) are: 
 [-0.26128768  0.07281316  0.239687    0.29179575  0.62434642  0.73511123
  0.89905188  1.01541175  1.24618429  1.36380464  1.41297865  1.58147806
  1.72455975  1.95727903  1.97788789  2.10903773  2.17736432  2.35650731
  2.48403827  2.79274172  3.31280196  3.54805045  6.80407535  8.65702897
  8.8031135   9.85705379]
--------------------------------------------------
For k-point No. 717
Calculate QO.
QO Normalization: after, norm of the  0 -th QO is: 1.00000000
QO Normalization: after, norm of the  1 -th QO is: 1.00000000
QO Normalization: after, norm of the  2 -th QO is: 1.00000000
QO Normalization: after, norm of the  3 -th QO is: 1.00000000
QO Normalization: after, norm of the  4 -th QO is: 1.00000000
QO Normalization: after, norm of the  5 -th QO is: 1.00000000
QO Normalization: after, norm of the  6 -th QO is: 1.00000000
QO Normalization: after, norm of the  7 -th QO is: 1.00000000
Calculate hamiltonian matrix in QO basis in k-space.
Eigenvalues of overlap of in basis Sqo(k) are:
 [1.37591919e-05 1.91976105e-05 4.89133630e-05 6.45648954e-05
 1.67252582e-03 6.38869074e-03 1.26278042e-02 7.97916454e+00]
Eigenvalues of Hamiltonian in QO basis Hqo(k) are:
 [0.34982477 0.69600839 0.7633131  0.88039793 0.89976318 2.4037417
 5.36962957 8.45016526]
Eigenvalues of Hamiltonian in NAO basis H(k) are:
 [-0.28226041  0.07554561  0.24690713  0.34982477  0.69600839  0.7633131
  0.88039793  0.89976318  1.1614511   1.33479144  1.4129278   1.62183167
  1.80650309  1.86654435  2.04743874  2.11125568  2.16577733  2.34028265
  2.51328901  2.97026378  3.65447573  4.06235185  6.11166278  7.80674016
  8.47872031 10.37336728]
--------------------------------------------------

But for other kpoints, condition numbers are very bad:

For k-point No. 718
Calculate QO.
QO Normalization: after, norm of the  0 -th QO is: 1.00000000
QO Normalization: after, norm of the  1 -th QO is: 1.00000000
QO Normalization: after, norm of the  2 -th QO is: 1.00000000
QO Normalization: after, norm of the  3 -th QO is: 1.00000000
QO Normalization: after, norm of the  4 -th QO is: 1.00000000
QO Normalization: after, norm of the  5 -th QO is: 1.00000000
QO Normalization: after, norm of the  6 -th QO is: 1.00000000
QO Normalization: after, norm of the  7 -th QO is: 1.00000000
Calculate hamiltonian matrix in QO basis in k-space.
Eigenvalues of overlap of in basis Sqo(k) are:
 [-8.57065013e-14  1.05051173e-05  5.63411027e-05  7.24538495e-05
  2.24425610e-04  2.70097358e-03  4.29519162e-02  7.95398338e+00]
WARNING: BAD CONDITION NUMBER OF OVERLAP MATRIX IN QO BASIS, THIS WILL CAUSE PROBLEMS IN DIAGONALIZATION.
...
For k-point No. 720
Calculate QO.
QO Normalization: after, norm of the  0 -th QO is: 1.00000000
QO Normalization: after, norm of the  1 -th QO is: 1.00000000
QO Normalization: after, norm of the  2 -th QO is: 1.00000000
QO Normalization: after, norm of the  3 -th QO is: 1.00000000
QO Normalization: after, norm of the  4 -th QO is: 1.00000000
QO Normalization: after, norm of the  5 -th QO is: 1.00000000
QO Normalization: after, norm of the  6 -th QO is: 1.00000000
QO Normalization: after, norm of the  7 -th QO is: 1.00000000
Calculate hamiltonian matrix in QO basis in k-space.
Eigenvalues of overlap of in basis Sqo(k) are:
 [7.78591963e-07 5.45634820e-06 5.49614825e-06 1.39923108e-04
 9.38162572e-04 1.34990031e-02 2.27331328e+00 5.71209790e+00]
WARNING: BAD CONDITION NUMBER OF OVERLAP MATRIX IN QO BASIS, THIS WILL CAUSE PROBLEMS IN DIAGONALIZATION.

Detailed information

qo.log

[kirk0830]

The bad condition number of QO in present implementation may result from wrong selection of subspace to project AO, I mean:

$\langle Q_\mu(\mathbf{k})|\hat{S}|Q_\nu(\mathbf{k})\rangle$
, is the overlap matrix ill-conditioned, or say QO overlap in subspace representation. To see this, in QO paper the QO is defined as:

$|Q_{\nu}(\mathbf{k})\rangle =\sum_{\beta}{|\varphi_{\beta}(\mathbf{k})\rangle\langle\varphi_{\beta}(\mathbf{k})|A_{\nu}(\mathbf{k})\rangle}$

Thus the overlap is:

$\langle Q_{\mu}(\mathbf{k})|\hat{S}|Q_{\nu}(\mathbf{k})\rangle =\sum_{\alpha\beta}{\langle A_{\mu}(\mathbf{k})|\varphi_{\alpha}(\mathbf{k})\rangle\langle \varphi_{\alpha}(\mathbf{k})|\hat{S}|\varphi_{\beta}(\mathbf{k})\rangle\langle \varphi_{\beta}(\mathbf{k})|A_{\nu}(\mathbf{k})\rangle}$

With orthonormality $\langle \varphi_{\alpha}(\mathbf{k})|\hat{S}|\varphi_{\beta}(\mathbf{k})\rangle =\delta_{\alpha\beta}$, we have:

$\langle Q_{\mu}(\mathbf{k})|\hat{S}|Q_{\nu}(\mathbf{k})\rangle =\sum_{\alpha}{\langle A_{\mu}(\mathbf{k})|\varphi_{\alpha}(\mathbf{k})\rangle\langle\varphi_{\alpha}(\mathbf{k})|A_{\nu}(\mathbf{k})\rangle}$

One can find projection operator $\sum_\alpha|\varphi_{\alpha}(\mathbf{k})\rangle\langle \varphi_{\alpha}(\mathbf{k})|$, therefore QO's overlap depends on selection of subspace to project on.