[refactor] Towards a non-driver focused future

[mrossinek]

What should we add?

This Epic outlines the plan for the upcoming restructuring of the entire Qiskit Nature code base.
Separate issues will be created outlining the refactoring plan for each existing module in more
detail. (Some modules/files will not be migrated. See at the bottom of this description.)

Vision

We are undertaking a step away from the existing control flow in which Qiskit Nature uses
drivers to handle the electronic structure integral generation by calling other classical codes.
Instead, the plan is to design a code- and language-agnostic description (implemented with HDF5) of
a problem for Qiskit Nature to solve, which the classical code can generate directly.
In this way, the responsibility over the control flow of a calculation is handed over to the
classical codes rather than remaining within Qiskit Nature.

This has multiple benefits, a major one being the immediate availability of iterative embedding
techniques already implemented in classical codes (e.g. CASSCF). In Qiskit Nature's current design,
only CASCI calculations are possible and an implementation of CASSCF would require a custom driver
class for each classical code, which is not scalable for the future.

Another issue will be opened in the near future outlining this vision in more detail and discussing
the planned HDF5 file structure.

Code Structure

With such a drastic change in design, we are taking this opportunity to restructure the code base.
The following structure roughly outlines the intended new structure:

qiskit_nature/
    second_quantization/
        algorithms/
            excited_state_solvers/
            ground_state_solvers/
            initial_points/
        circuit/
            library/
                ansatzes/
                initial_states/
        operators/
        mappers/
        hamiltonians/
        properties/
        problems/
        transformers/
        drivers/

A more detailed overview including the main classes is given by the following UML:

Workflow

Someone already familiar with Qiskit Nature, should not have much trouble understanding the
relations outlined by the following class diagram:

Together with this pseudo code, this should roughly outline the intended workflow:

from qiskit_nature.second_quantization.algorithms.ground_state_solvers import (
    GroundStateEigensolver,
    VQEUCCFactory,
)
from qiskit_nature.second_quantization.mappers import JordanWignerMapper, QubitConverter
from qiskit_nature.second_quantization.drivers import HDF5Driver

problem = HDF5Driver("problem_description.hdf5").run()

solver = GroundStateEigensolver(
    VQEUCCFactory(),
    QubitConverter(JordanWignerMapper()),
)

result = solver.solve(problem)

More details on the various modules will be provided by the respective issues.

Unaffected modules

• constants.py
• deprecation.py
• exceptions.py
• hdf5.py
• logging.py
• optionals.py
• runtime
• settings.py
• utils
• version.py
• VERSION.txt

[kevinsung]

When doing these kinds of refactorings it's good to try as much as possible to preserve the version control history of each file (so git blame is still useful afterwards).

[mrossinek]

When doing these kinds of refactorings it's good to try as much as possible to preserve the version control history of each file (so git blame is still useful afterwards).

I agree, that would be ideal. However, to some degree this might not be possible because we need to actually copy instead of move the code, in order to abide by our deprecation policies.
There are ways to copy files within a git repository and include the history but I'm not sure how feasible this will be on this large scale.

[kevinsung]

However, to some degree this might not be possible because we need to actually copy instead of move the code, in order to abide by our deprecation policies.

Perhaps we could have one commit that moves the code to the new location, followed by another commit that copies it back to the old.

[ElePT]

The idea behind this comment is to explain the change in direction that you will see in my refactoring PR (#722). As discussed with @mrossinek and @woodsp-ibm, we will change this proposed structure in favor of a flatter and hopefully more intuitive one, where:

• the operator_factories module will be split into
  • hamiltonians
  • properties
• we will keep the original drivers module for the time being. In future refactoring steps this could turn into an api module or similar.
• mappers and transformers will be exposed as their own modules, instead of being nested in operators and problems
• not all mappers work with all operators. Instead of creating submodules (fermonic, vibrational, spin), the corresponding operator for each mapper will be indicated via type hinting (For example, JordanWignerMapper will inherit from FermionicMapper)
• similarly, not all transformers work with all problems. Instead of creating submodules (electronic), the corresponding problem/use for each transformer could be indicated via type hinting (inheriting from ElectronicBaseTransformer).

So, some modules will still follow the structure shown above:

qiskit_nature/
    second_q/
        algorithms/
            excited_state_solvers/
            ground_state_solvers/
            initial_points/
        circuit/
            library/
                ansatzes/
                initial_states/

But we will slightly modify the rest of the modules to follow:

qiskit_nature/
    second_q/
        mappers/
            # all mappers here
            JordanWignerMapper
            LinearMapper
            ...
            QubitMapper
            QubitConverter # included with mappers
      
        operators/
             FermionicOp
             SpinOp
             VibrationalOp
         
         hamiltonians/
                QuadraticHamiltonian
                ElectronicEnergy
                VibrationalEnergy
                LatticeModel
                ...
                lattices/ # used to construct lattice models, right place?
                      Line
                      Square
                      ...       
                      
         properties/  # these are properties of the hamiltonians              
                 AngularMomentum
                 DipoleMoment
                 ...
                 OccupiedModals
                 ...
          
          transformers/
                ActiveSpaceTransformer
                BasisTransformer
                FreezeCoreTransformer
           
           problems/
                ElectronicStructureProblem
                ElectronicStructureResult
                LatticeModelProblem
                LatticeModelResult
                VibrationalStructureProblem
                VibrationalStructureResult
            
            drivers/
                # keep legacy drivers as part of the new structure

Did I overlook anything?

[mrossinek]

Thanks for the summary, @ElePT! Looks like you got everything right 👍

EDIT: You may want to indicate that ElectronicStructureResult (and similar classes) will become part of the problems module

[mrossinek]

I am reopening this Epic. I faultily linked it to the PR which does the code restructuring, but more upcoming refactorings are related to this.

[mrossinek]

Minor update: I updated the issue description, including the UML graphics, to reflect the design changes as also discussed by @ElePT further up