Back to search

Boston University

Soft-SVM regression for binary classification and its extensions

Abstract

dc:description.abstract

My dissertation comprises three interconnected projects (Chapters 2 to 4), focused on developing a novel binary classification approach that blends elements of logistic regression and Support Vector Machines (SVMs), and extends this approach to other application tools such as a Bayesian and multi-class classifier. This innovative method is termed 'Soft-SVM' regression. Chapter 1 introduces the background information and the motivation for conducting research on this topic. It briefly summarizes the algorithms of logistic regression and SVM, as well as their corresponding strengths and limitations in performing binary classification. This discussion leads to our initial motivation for pursuing this research topic. Chapter 2 lays the groundwork by detailing the motivation and methodology behind Soft-SVM. Recognizing the similarities in the loss function shapes of logistic regression and SVM, we approach the problem through the lens of cumulant functions in the generalized linear model (GLM) framework. Soft-SVM is designed with shifted soft plus functions, allowing for a tailored balance between logistic regression and SVM characteristics. We employ Fisher scoring with active set method for constrained optimization to train parameters. The efficacy of Soft-SVM is demonstrated using simulated datasets and case studies, where its performance is benchmarked against traditional logistic regression and SVM. As we show, our proposed methodology offers better, more robust, and more computationally efficient performance. In Chapter 3, the narrative shifts to a Bayesian perspective on the same regression problem. Here, the regression coefficients and softness parameter are treated as random, while other aspects remain consistent with the developments in Chapter 2. We adopt the Riemannian manifold Hamiltonian Monte Carlo (RMHMC) method for parameter estimation, crafting a sampler for the posterior distribution of model parameters. Chapter 4 expands the scope of Soft-SVM by adapting it for multi-class classification scenarios. This adaptation opens up new potential for SVM in multi-class settings, representing a breakthrough that hasn't been previously studied.

Degree

thesis:*
Name thesis:degree_name
Doctor of Philosophy
Level thesis:degree_level
doctoral
Discipline thesis:degree_discipline
Mathematics & Statistics
Grantor
Boston University
Year dc:date.issued
2024

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Huang, Man
Advisor dc:contributor.advisor
  • Carvalho, Luis
ORCID dc:identifier.orcid

Subjects

dc:subject × 3

Rights

Language dc:language.iso
en_US

Identifiers

dc:identifier.*
Handle dc:identifier.uri
https://hdl.handle.net/2144/49968

Chain of custody

source
Harvested from
Boston University
Base URL
open.bu.edu/oai/request
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
citation

Huang, Man. Soft-SVM regression for binary classification and its extensions. doctoral thesis, Boston University, 2024. https://hdl.handle.net/2144/49968