References & Citations
Computer Science > Computer Vision and Pattern Recognition
Title: Transformer Architecture for NetsDB
(Submitted on 8 May 2024 (v1), last revised 9 May 2024 (this version, v2))
Abstract: Transformers models have become the backbone of the current state-of-the-art models in language, vision, and multimodal domains. These models, at their core, utilize multi-head self-attention to selectively aggregate context, generating dynamic contextual embeddings and modeling long-range dependencies for a clear contextual understanding. Lixi et al. \cite{zhou2022serving} proposed a method to use relational databases for deploying large-scale deep learning models and created an open-source implementation called NetsDB for the same. We build upon the previous work of these authors by creating an end-to-end implementation of the Encoder part of the transformer for model serving in NetsDB. Specifically, we construct a two-block encoder that includes Multi-Head Attention and its accompanying self-attention mechanism, Layer-Norm, Dropout, FeedForward Layers, and the necessary residual connections. We load out weights from our model for distributed processing, deployment, and efficient inferencing. To prove the efficacy of our implementation, we conduct a comprehensive performance analysis by comparing it with existing implementations in PyTorch, Tensorflow, Flax, and MxNet across key metrics such as inference time and model size.
Submission history
From: Kunal Kasodekar [view email][v1] Wed, 8 May 2024 04:38:36 GMT (2534kb,D)
[v2] Thu, 9 May 2024 12:02:22 GMT (2534kb,D)
Link back to: arXiv, form interface, contact.