DOCTOR OF PHILOSOPHY (2019) McMaster University, Hamilton, Ontario (Medical Sciences) TITLE: Serotonergic antagonists affect the activity of breast tumor initiating cells in human and mouse models of breast cancer. AUTHOR: William D. Gwynne, BSc SUPERVISOR: Dr. John A. Hassell NUMBER OF PAGES: XXI; 255 / Breast cancer is the most frequently diagnosed cancer and the leading cause of cancer-related death amongst women worldwide. The relatively unchanging breast cancer-associated mortality rate is in part due to the existence of rare tumor cells (breast tumor initiating cells; BTIC) that possess stem-like properties permitting them to survive therapy and initiate disease recurrence. Hence, identifying agents capable of eradicating these cells would be a favourable therapeutic strategy to improve the durability of breast cancer remissions. To achieve the latter objective our lab screened over 35,000 small molecules for their capacity to inhibit the viability of BTIC-enriched mouse tumor cells. Unexpectedly, several antagonists of the serotonin (5-hydroxytryptamine; 5-HT) transporter and select receptors were among the hit compounds identified in the screen. This thesis aims to establish a connection between serotonergic activity and BTIC. We accomplished the latter by assessing whether components of the 5-HT signaling system are expressed in mouse and human breast tumor cells and whether inhibition of their activity affects BTIC frequency using multiple orthogonal assays. Our data suggest that breast tumor cells of both mouse and human origin express the components necessary for 5-HT synthesis, activity and metabolism and that inhibition of these proteins with selective antagonists reduces the capacity of these cells to form tumorspheres. We demonstrate that highly selective antagonists of SERT and HTR5A target BTIC as established ex vivo cell transplantation assays. We also discovered that these agents synergize with chemotherapy in vivo to affect the growth of mouse breast tumor allografts and human breast tumor xenografts. To validate the molecular targets of these agents, we attempted to phenocopy their effects in functional assays by knocking out their respective genes using CRISPR-Cas9 technology. Collectively, this thesis contributes to an understanding of how 5-HT signaling affects BTIC and identifies serotonergic antagonists as novel anticancer agents. / Dissertation / Doctor of Philosophy (PhD) / Despite improvements in screening technologies and the development of targeted therapies breast cancer remains the second leading cause of cancer-related death among Canadian women. Whereas the current standard of care is effective at treating the majority of patients diagnosed with breast cancer, there remains a substantial proportion of patients that experience relapse after undergoing therapy. Recurrence is due in part to the existence of rare, stem-like tumor cells, termed breast tumor-initiating cells (BTIC) that are insensitive to existing anticancer agents. Hence, identifying drugs capable of targeting these cells is a desirable goal. To pursue the latter, our lab screened approximately 35,000 compounds for their capacity to affect the growth of BTIC-enriched tumor cell populations. Among the hit compounds were antagonists of the serotonin transporter and serotonin receptors, including FDA-approved psychiatric medications. Here, we explore a connection between serotonin-related proteins and BTIC activity with the aim of identifying novel therapeutic agents.

The high rate of relapse often seen in breast cancer patients has been suggested to be the result of a small subset of chemotherapy-resistant cancer stem cells (CSCs), believed to be responsible for initiating tumor formation. These CSCs possess the capability to self-renew and give rise to a hierarchy of cells which makes up the bulk of a tumor. Neurotransmitters have been suggested to influence CSC self-renewal and proliferation capabilities, and antagonists of neurotransmission pathways have been implicated as possible treatment methods for chemo-resistant tumors. Using nicotinic acetylcholine receptor (nAChR) antagonists in sphere-forming assays, we have identified a very promising candidate compound: MG624. We found this compound to have a high selectivity for sphere-forming cells over non-sphere-forming cells in vitro, in a dose-dependent relationship, across a panel of cell lines as well as in patient-derived xenograft cells. This was validated in two ex vivo assays, where tumor formation was significantly delayed in mice injected with MG624-treated HCC1954 cells at both the IC50 and IC90 of the compound, indicating that MG624 does indeed target functional BTICs. MG624 was also found to synergize with both taxotere and doxorubicin chemotherapies in vitro, and shrink tumors in NOD/SCID mice when combined with taxotere in vivo. MG624 in combination with taxotere was found to induce apoptosis, and prevent cells from entering into the M-phase of the cell cycle. Interestingly, MG624 was found to eliminate intratumoral fibroblasts in combination with taxotere, despite taxotere being found to recruit fibroblasts to the tumor site when used on its own. Most importantly, the combination of MG624 and taxotere was found to significantly delay tumor progression/relapse in mice, indicating that MG624 may be an excellent candidate compound to one day be combined with chemotherapy to provide durable remission to breast cancer patients. / Thesis / Master of Science (MSc)

Breast cancer therapy resistance and disease recurrence are driven by an infrequent population of stem-like tumor cells, termed breast cancer stem cells or tumor-initiating cells (BTIC). Whereas drugs that target BTIC could be combined with conventional therapies to provide durable remissions, identifying such agents has been difficult. To achieve the latter, our lab screened more than 35,000 compounds for their capacity to reduce the activity of BTIC-enriched mouse mammary tumorspheres, wherein we identified numerous antagonists of multiple serotonin receptors (HTRs). The serotonergic antagonist that prevented sphere formation with the highest potency is a highly selective antagonist of HTR5A, SB-699551. We subsequently demonstrated that this agent affects BTIC activity in breast tumor cell lines representative of all clinical and molecular subtypes of breast cancer. Whereas the primary target of SB-699551 is known, the downstream signaling pathways responsible for its anti-BTIC effect remains enigmatic. The goal of this thesis work was to elucidate the signaling pathways downstream of HTR5A in human breast tumor cell lines. We used a phospho-proteomic approach to establish that treatment of human SB-699551 affects the phosphorylation of proteins involved in the Gi-coupled and the PI3K/AKT/mTOR signaling axes. Moreover, we demonstrated that selective antagonists of PI3K, AKT, and mTOR phenocopied the effect of SB-699551 in tumorsphere forming assays. Taken together, our data suggests that SB-699551 elicits its effect through the PI3K/AKT/mTOR signaling pathways downstream of HTR5A. / Thesis / Master of Health Sciences (MSc) / Accumulating data suggests that the progression of breast cancer is driven by a rare population of breast tumor-initiating cells (BTIC). BTIC lie dormant during conventional therapy and initiate recurrence after such therapies are withdrawn. Hence, there is an urgent need to develop drugs that target BTIC that can be combined with the current standard of care to improve the durability of remission. With the latter objective in mind, our lab previously determined that antagonists of serotonin signaling target BTIC. One of the agents that we identified in our screen inhibits the activity of serotonin receptor 5A (HTR5A). The exact signaling mechanism whereby inhibition of HTR5A leads to a loss in BTIC activity was enigmatic. Hence, this thesis aims to elucidate the signaling pathways downstream of HTR5A in breast cancer. Knowledge of the latter will help identify a plausible mechanism in addition to identifying biomarkers of therapy efficacy.