Large Language Models (LLMs) have gained significant traction across critical domains owing to their impressive contextual understanding and generative capabilities. However, their increasing deployment in high stakes applications necessitates rigorous evaluation of embedded biases, particularly in culturally diverse contexts like India where existing embedding-based bias assessment methods often fall short in capturing nuanced stereotypes. We propose an evaluation framework based on an encoder trained using contrastive learning that captures fine-grained bias through embedding similarity. We also introduce a novel dataset - IndiCASA (IndiBias-based Contextually Aligned Stereotypes and Anti-stereotypes) comprising 2,575 human-validated sentences spanning five demographic axes: caste, gender, religion, disability, and socioeconomic status.
@inproceedings{santhosh2025indicasa,title={IndiCASA: A Dataset and Bias Evaluation Framework in LLMs Using Contrastive Embedding Similarity in the Indian Context},author={G S, Santhosh and Govind, Akshay S and Krishnan, Gokul S and Ravindran, Balaraman and Natarajan, Sriraam},booktitle={AAAI/ACM Conference on AI, Ethics, and Society (AIES)},year={2025},organization={AAAI/ACM},proceedings={https://ojs.aaai.org/index.php/AIES/article/view/36605},}
arXiv
AQUA: Attention via QUery mAgnitudes for Memory and Compute Efficient Inference in LLMs
Santhosh
G S, Saurav
Prakash, and Balaraman
Ravindran
The quadratic complexity of the attention mechanism remains a fundamental barrier to scaling Large Language Models (LLMs) to longer contexts, creating a critical bottleneck in both computation and memory. To address this, we introduce AQUA (Attention via QUery mAgnitudes) a novel and versatile approximation strategy that significantly reduces the cost of attention with a graceful performance trade-off. Our method operates in two phases: an efficient offline step where we compute a universal, language agnostic projection matrix via SVD on a calibration dataset, and an online inference step where we project query and key vectors and dynamically select a sparse subset of dimensions based on the query’s magnitude. We demonstrate that a 25% reduction in the attention dot-product computation can be achieved with statistically insignificant impact on performance across a wide range of benchmarks.
@article{santhosh2025aqua,title={AQUA: Attention via QUery mAgnitudes for Memory and Compute Efficient Inference in LLMs},author={G S, Santhosh and Prakash, Saurav and Ravindran, Balaraman},journal={arXiv preprint arXiv:2509.11155},year={2025},}
arXiv
SWAN: Sparse Winnowed Attention for Reduced Inference Memory via Decompression-Free KV-Cache Compression
Santhosh
G S, Saurav
Prakash, and Balaraman
Ravindran
Large Language Models (LLMs) face a significant bottleneck during autoregressive inference due to the massive memory footprint of the Key-Value (KV) cache. Existing compression techniques like token eviction, quantization, or other low-rank methods often risk information loss, have fixed limits, or introduce significant computational overhead from explicit decompression steps. In this work, we introduce SWAN, a novel, fine-tuning-free framework that eliminates this overhead. Our method uses an offline orthogonal matrix to rotate and prune the KV-cache, which is then used directly in the attention computation without any reconstruction. Our extensive experiments demonstrate that SWAN, augmented with a small dense buffer, offers a robust trade-off, maintaining performance close to the uncompressed baseline even at aggressive 50-60% memory savings per-token on KV-cache.
@article{santhosh2025swan,title={SWAN: Sparse Winnowed Attention for Reduced Inference Memory via Decompression-Free KV-Cache Compression},author={G S, Santhosh and Prakash, Saurav and Ravindran, Balaraman},journal={arXiv preprint arXiv:2511.18936},year={2025},}
2024
COMSNETS
A Quantum Approach to Solve N-Queens Problem
Santhosh
G S, P
Joshi, A
Barui, and
1 more author
In 2024 16th International Conference on COMmunication Systems & NETworkS (COMSNETS), 2024
@inproceedings{santhosh2024quantum,title={A Quantum Approach to Solve N-Queens Problem},author={G S, Santhosh and Joshi, P and Barui, A and Panigrahi, PK},booktitle={2024 16th International Conference on COMmunication Systems \& NETworkS (COMSNETS)},year={2024},organization={IEEE}}
ICEES
Energy Conservation in Educational Buildings using IOT
L
Raju, Prathyumnan
M, Santhosh
G S, and
2 more authors
In 2024 10th International Conference on Electrical Energy Systems (ICEES), 2024
@inproceedings{raju2024energy,title={Energy Conservation in Educational Buildings using IOT},author={Raju, L and M, Prathyumnan and G S, Santhosh and K S, Sasikaran and others},booktitle={2024 10th International Conference on Electrical Energy Systems (ICEES)},pages={1--6},year={2024},organization={IEEE}}
2023
ICEES
Advanced Automation and Energy Management using IOT
L
Raju, Santhosh
G S, and Sasikaran
K S
In 2023 9th International Conference on Electrical Energy Systems (ICEES), 2023
@inproceedings{raju2023automation,title={Advanced Automation and Energy Management using IOT},author={Raju, L and G S, Santhosh and K S, Sasikaran},booktitle={2023 9th International Conference on Electrical Energy Systems (ICEES)},pages={52--57},year={2023},organization={IEEE}}
