Abstract
Aim/backround: Transformers were introduced mainly to solve the shortcomings of Recurrent Neural Networks (RNNs), namely their inability to handle long-range dependencies. Transformers overcome the drawbacks of RNN-based techniques through the use of self-attention mechanisms, positional encoding, and multi-head attention. This work aims to highlight the role of Transformers and compare it with RNNs for temporal processing, presenting the basic attributes of Transformers, such as their structure and functioning, followed by detailed procedures concerning their applications, developing two efficient implementations of Transformers, one for protein prediction and one for the classification of breast cancer ultrasound images
Methods: We use the Julia Flux framework, using two different datasets, the Breast Ultrasound Images Dataset (BUSI) and the BUS-UCLM-Breast Ultrasound Lession Segmentation Dataset (BUS-UCLM).
Results: Results indicate that Transformer succeeds in modeling the statistical properties of the aminoacid composition of proteins, while for the breast cancer classification problem accuracies of up to 99.62% and 100% are reported, for BUSI and BUS-UCLM datasets. These results, not only demonstrating the versatility of Transformers across domains but also highlight their potential as a unifying framework for complex biological and medical tasks, underscoring the broader relevance of Transformer architectures in advancing both computational biology and medical imaging.
Conclusion: The results obtained with the Gated Recurrent Unit (GRU) type of Recurrent Neural Network are similar in terms of performance and training times. Both models performed excellently at both the breast cancer classification and the protein prediction problems.

Key words: Recurrent Neural Networks, Long Short-Term Memory, Gated Recurrent Units, Transformers, Positional Encoding, Multi-Head Attention, Sequence Modeling.