Quantum Reinforcement Learning via Parameterized Quantum Walks

Quantum Reinforcement Learning via Parameterized Quantum Walks

Abstract:

Random walks find application in various domains of research such as computer science, psychology, finance or mathematics, as they are a fundamental concept in probability theory and stochastics. But conventional computers quickly reach their limits regarding computational complexity, so other ways of efficiently solving complex problems like Quantum Computing are needed. Quantum walks, the quantum equivalent of classical random walks, use quantum effects such as superposition and entanglement to be more efficient than their classical counterparts. Nevertheless, running programs on quantum devices at near-term intermediate scale quantum devices presents some challenges due to high error rates, noise, and the number of available qubits. For a large number of graph problems, Gray Code Directed Edges (GCDE) encoding counteracts these problems by reducing the required number of qubits through an efficient representation of bipartite graphs using gray code.

This work investigates random walks in grid worlds and glued trees using classical reinforcement learning strategies such as Proximal Policy Optimization or Deep Q-learning Networks. In a next step, these environments are re-built using efficient GCDE encoding. The environments are translated into parameterized quantum circuits whose parameters are optimized and learned by the walker. The contribution of this work contains the application of efficient GCDE encoding in quantum environments and a comparison between a quantum and a random walker regarding training times and target distances. Furthermore, the effects of different start positions during training and evaluation are
considered.

Author:

Sabrina Egger

Advisors:

Jonas Stein, Michael Kölle, Claudia Linnhoff-Popien

Student Thesis | Published October 2024 | Copyright © QAR-Lab
Direct Inquiries to this work to the Advisors