Probabilistic circuits are tractable probabilistic models that represent complex probability distributions while allowing many probabilistic queries to be evaluated exactly and efficiently. Most existing probabilistic circuits rely on non-negative scalar parameters and consequently inherit structural and expressive limitations from classical probabilistic models. This project investigates Positive Unital Circuits (PUnCs), a recently proposed family of non-monotonic probabilistic circuits based on positive-semidefinite operator-valued representations. The project studies how operator-valued parameters and quantum-inspired linear-algebraic operations can extend the expressive power of probabilistic circuits while retaining efficient probabilistic inference. The research has both theoretical and computational components. The theoretical work studies the expressive power of PUnCs and related extensions, their relationship to conventional probabilistic circuits and non-monotonic sum-product models, and the structural conditions required for tractable inference. The computational work develops implementations of these models and evaluates their numerical behaviour, scalability, expressiveness, and training efficiency. Experiments will involve circuits with varying structures, operator dimensions, circuit widths and depths, as well as comparisons with conventional probabilistic circuits and related tractable models. A significant part of the computation consists of batched tensor operations, Kronecker products, matrix products, eigendecompositions and automatic differentiation through positive-semidefinite matrix parameterizations. These operations are well suited for GPU acceleration. The computational resources will primarily be used for developing and benchmarking JAX implementations of PUnCs, running controlled expressiveness experiments, performing hyperparameter and architecture sweeps, and training models on standard probabilistic modelling datasets. The goal is to understand when richer operator-valued representations provide computational or representational advantages and to develop practical implementations that exploit modern GPU hardware. This project forms part of my doctoral research in Computer Science at Örebro University.