Financial trading data presents a unique challenge for anomaly detection due to its high dimensionality and often lack of labelled anomalous examples. Nevertheless, it is of great interest for financial institutions to gain insight into potential trading activities that might lead to financial losses and reputational damage. Given the complexity and unlabelled nature of this financial data, deep learning models such as the Transformer model are particularly suited for this task.   This work investigates the application of a Transformer-based autoencoder for anomaly detection in unlabelled financial transaction data with sequential characteristics. To assess the model's ability to detect anomalies and analyse the effects of class imbalance, synthetic anomalies are injected into the dataset. This creates a controlled environment where the model's performance can be evaluated but also the affects of imbalance can be investigated.    Two approaches are particularly explored for anomaly detection purposes: an unsupervised approach and a semi-supervised approach that explicitly leverages the presence of anomalies in the training data. Experiments suggest that while the unsupervised approach can detect anomalies with distinctive features, its performance suffers when anomalies are included in the training data since the model tends to reconstruct them. Conversely, the semi-supervised approach effectively addresses this limitation, demonstrating a clear advantage in the presence of class imbalance. While synthetic anomalies enable controlled evaluation and class imbalance analysis, generalizability to real-world financial data requires true anomalies.