This PR introduces a type distinction between different kinds of operators and migrates the quantum operator infrastructure in Python to be bindings instead of Python code. This fixes previous performance issues after the initial introduction of the general quantum operator API in Python with 0.9.0.

The following types are introduced:

• BosonOperator, BosonOperatorTerm, BosonOperatorElement
• FermionOperator, FermionOperatorTerm, FermionOperatorElement
• SpinOperator, SpinOperatorTerm, SpinOperatorElement
• MatrixOperator, MatrixOperatorTerm, MatrixOperatorElement

The OperatorSum class is now a union of BosonOperator | FermionOperator | SpinOperator | MatrixOperator.
The ProductOperator class is now a union of BosonOperatorTerm | FermionOperatorTerm | SpinOperatorTerm | MatrixOperatorTerm.
The ElementaryOperator class is now a union of BosonOperatorElement | FermionOperatorElement | SpinOperatorElement | MatrixOperatorElement.

Operators for bosons, fermions and spins leverage a dedicated internal representation that allows for significant performance improvements compared to general matrix operators. When operators of different kinds (types) are combined, the resulting operator will always be a matrix operator.

Breaking changes:

• Each operator now keeps track of exactly what degrees it is acting on (no implicit injection of identities).
  • num_qubits reflects only the degrees the operator explicitly acts upon; e.g. spin_op::i(2) act on a single degree
  • to_matrix reflects only the degrees the operator explicitly acts upon; e.g. spin_op::x(1) returns a 2x2 matrix (previously a 4x4 matrix)
  • useful functions to replicate the previous behavior where an operator would always act on all consecutive degrees from 0 to max_degree are: a constructor SpinOperatorTerm(first_degree, last_degree), the member function op.canonicalize(Iterable[int]), and of course explicit multiplication with the identity(target).
• We introduce a type distinction between a SpinOperator (a sum of SpinOperatorTerms) and a SpinOperatorTerm (a product of Pauli operators). This requires a couple of breaking changes:
  • The default constructor (SpinOperator()) will no longer create a product term with value 1. Instead, it will create a default instantiated sum, which will take the value of the first operator it is multiplied with or added to. For clarity and robustness, we recommend using an explicit cudaq.spin.empty() to create a sum with no terms, or cudaq.spin.identity() to create a product with value 1 as the previous constructor did.
  • If an operator is created as a product (e.g. op = spin.idenity()), it is not possible to += other terms, since this requires op to be a SpinOperator rather than a SpinOperatorTerm. You can ensure op is of the appropriate type by either starting out with an empty sum (op = spin.empty()), or by explicitly converting it to the correct type (op = SpinOperator(spin.identity())).
  • spin.empty() now creates a spin operator with no terms; to determine whether a variable op of type SpinOperator is empty (has no terms), use op.num_terms == 0.
  • The constructor SpinOperator(num_qubits: int) no longer exists; the constructor SpinOperatorTerm(first_degree: int, last_degree: int) can be used to create a product term that applies the identity to all targets in the open range [first_degree, last_degree). The constructor SpinOperator(size: int) instantiates a sum without any terms, reserving space for the given size (estimated number of terms).
• The serialization format for SpinOperators changed;
  • serialization methods (op.serialize() and op.to_json()) are supported on both SpinOperator and SpinOperatorTerm, with matching constructors that take an Iterable[float] and methods SpinOperator.from_json and SpinOperatorTerm.from_json.
  • the constructor SpinOperator(filename: str) expects the file to contain the new serialization format; please use SpinOperator(filename: str, legacy: bool) with legacy = True to load existing files using the format/representation
• The module cudaq.operator has been restructured;
  • all operators and classes previously available under cudaq.operator.operators are now available under cudaq.operators
  • cudaq.operators contains the submodules spin, boson, and fermion (aliased also directly under cudaq)
  • the evolve related functionality (e.g. integrators) has been moved into a separate cudaq.dynamics folder; this should be largely non-breaking since the public classes and functions were and continue to be aliased directly under cudaq
• ElementaryOperator.define no longer exists; it is replaced by cudaq.operators.define, which defines a MatrixOperatorElement
• ElementaryOperator(op_id: str, degrees: Iterable[int]) no longer exists; it is replaced by cudaq.operators.instantiate(op_id, degrees), which produces a MatrixOperatorTerm (product operator)
• The class methods ElementaryOperator.zero and ElementaryOperator.identity no longer exist; they are replaced by operators.zero and operators.identity, which produce a MatrixOperatorTerm (product operator)
• all class methods for operator instantiation (such as e.g. operators.identity or boson.identity) will always produce a product operator as well; ElementaryOperators are no longer intended to be constructed directly but merely encountered when iterator over a ProductOperator
• ProductOperator and OperatorSum constructors that take a vector of operators/terms are no longer available; products and sums of operators should be created by using +/-/* instead
• Setting the generator of a ScalarOperator after construction is no longer supported (necessary change due to C++ binding)
• ScalarOperators who are not constant are usually not equal even if they are defined with the same Python function (necessary consequence of the migration to C++ bindings)

Deprecations:

• operators.create and operators.annihilate are deprecated; please use boson.create/annihilate or fermion.create/annihilate instead.
• While the other operators (such as, e.g., operators.number) continue to be supported, we highly recommend replacing them with the appropriate boson.number or fermion.number instead to greatly improve performance.
• get_coefficient is deprecated; use evaluate_coefficient on a/each SpinOperatorTerm instead.
• get_term_count is deprecated; use the property term_count instead. Both are only available on SpinOperator, not on SpinOperatorTerm.
• get_qubit_count is deprecated; use the property qubit_count instead.
• get_raw_data is deprecated; raw data access will no longer be supported in future releases. Coefficients can be obtained via the coefficient property on a SpinOperatorTerm, and the binary symplectic form via the method get_binary_symplectic_form() on SpinOperatorTerm.
• is_identity will no longer be supported on SpinOperator in future releases, but will continue to be supported on SpinOperatorTerm
• the constructor SpinOperator(data: Iterable[double], num_qubits: int) used to load the old deserialization format will be removed in future releases; it is replaced by the constructor SpinOperator(data: Iterable[double]) using the new format
• op.to_string(print_coefficient: bool) is deprecated and will be removed in future releases; please use str(op) instead. The Pauli word representation for a SpinOperatorTerm can be obtained with op.get_pauli_word(pad_identities: int = 0) instead.
• for_each_term and for_each_pauli are deprecated and replaced by the standard iteration over the sum, and over each term in the sum instead.

See also the description in #2710 to understand the broader changes across the CUDA-Q codebase that are now reflected also in the Python API.

Potential follow ups that are not part of this PR:

• Expose commutation_behavior in Python as well
• Should we migrate RydbergHamiltonian to C++ bindings as well?
• Implement OperatorEvaluation in C++ instead?
• Add bindings and tests for to_sparse_matrix as soon as Efficient matrix evaluation for bosons and fermions #2764 is merged
• Add detection for conflicting commutation relation when a sum is evaluated
• Allow create a new operator by defining a mapping of degrees
• Support partial evaluation of operators (fixing some parameters but not others)