Path-Connectedness of the Boundary between Features that Are Labeled Differently by a Single Layer Perceptron

Path-Connectedness of the Boundary between Features that Are Labeled Differently by a Single Layer Perceptron

Abstract:

Due to the remarkable advancements in high-performance computing, machines can process an increasingly high amount of data to adjust numerous parameters in a Machine Learning model (ML model). In this way, the machine recognizes and learns patterns and might come to good and fast decisions. Though, the success of an ML model does not just depend on the performance of the computer where it is deployed on that assures the capability of processing huge databases. Mostly, a high amount of data is helpful, but not the key to obtain a reliable model itself. Already models with just a few trainable parameters, where smaller data sets are sufficient for the training, can produce stunning outputs if the basic model is chosen adequately and fits to the data and to the task.

From an abstract point of view, ML models are parameterized functions, where the parameters are optimized during the learning process. To examine if a certain ML model qualitatively fits, we can set up requirements in a mathematical way. Here, we discuss specifications that do not consider a concrete assignment of the parameters but expect a certain behavior of the to a model corresponding function for arbitrary parameters. Subsequently, we can prove that a certain model fulfills them, or give a more specific counter-example, which yields that a certain mathematical property does not hold, in general, for the regarded model.

In this thesis, we consider a Single Layer Perceptron (SLP), the root of Deep Neural Networks, that categorizes features between two different labels. We show that under certain preconditions the boundary between the two categories within the feature space is path-connected. This indicates the SLP being a proper choice if we have pre-knowledge about the features: If we know that the boundary between the two categories is path-connected in reality, we can exclude such models that generate a boundary with gaps.

Author:

Advisors:

Maximilian Balthasar Mansky, Thomas Gabor, Claudia Linnhoff-Popien

Student Thesis | Published December 2023 | Copyright © QAR-Lab
Direct Inquiries to this work to the Advisors