Md. Faiyaz Abdullah Publications
16 peer-reviewed papers — 3 journal articles, 8 conference papers and 5 workshop papers. Bold marks my name; * marks equal contribution.
Journal Articles (3)
-
[J1]IEEE Access
JailbreakTracer: Explainable Detection of Jailbreaking Prompts in LLMs Using Synthetic Data Generation
IEEE Access, 2025
The emergence of Large Language Models (LLMs) has revolutionized natural language processing (NLP), enabling remarkable advancements across various applications. However, these models remain susceptible to adversarial prompts, commonly referred to as jailbreaks, which exploit their vulnerabilities to bypass ethical and safety constraints. These prompts manipulate LLMs to produce harmful or forbidden outputs, posing serious ethical and security challenges. In this study, we propose JailbreakTracer, a novel framework leveraging synthetic data generation and Explainable AI (XAI) to detect and classify jailbreaking prompts. We first construct two comprehensive datasets: a Toxic Prompt Classification Dataset, combining real-world and synthetic jailbreak prompts, and a Forbidden Question Reasoning Dataset, categorizing forbidden queries into 13 distinct scenarios with clear reasoning labels. Synthetic toxic prompts are generated using a fine-tuned GPT model, achieving an attack success rate of 95.1%, effectively addressing the class imbalance. Using transformer-based architectures, we train classifiers that achieved 97.25% accuracy in detecting jailbreak prompts and 100% accuracy in categorizing forbidden questions. Our approach integrates XAI techniques, such as LIME, to ensure interpretability and transparency in the model’s predictions. Extensive evaluations demonstrate the efficacy of JailbreakTracer in detecting and reasoning about jailbreak prompts, providing a critical step toward enhancing the safety and accountability of LLMs.
-
[J2]Data in Brief
ElectroCom61: A Multiclass Dataset for Detection of Electronic Components
Data in Brief, Elsevier, 2024
In contemporary industrial, robotics, and technical education settings, the efficient detection and sorting of electronic components play a pivotal role in advancing automation and increasing efficiency in these sectors. To address this need, we present “ElectroCom61,” a comprehensive multi-class object detection dataset encompassing 61 commonly used electronic components. Our dataset, sourced from the electronic components collection at United International University (UIU) in Dhaka, Bangladesh, comprises 2121 meticulously annotated images. We ensured that these images reflect real-world conditions, incorporating varied lighting, backgrounds, distances, and camera angles to bolster the potential machine learning model's robustness. We also divided the dataset into training, validation, and test sets to facilitate deep learning model development. Additionally, we conducted minimal pre-processing to optimise model training and performance. “ElectroCom61” stands as a valuable asset for developing cutting-edge electronic component detection systems, with far-reaching applications in both education and industry. Its potential applications span from interactive educational tools to e-waste management systems and streamlined inventory management processes in electronic manufacturing and automation.
-
[J3]Applied Sciences
A Comparative Analysis for Optimizing Machine Learning Model Deployment in IoT Devices
Applied Sciences, MDPI, 2024
In the intersection of the Internet of Things (IoT) and Machine Learning (ML), the choice between high-level and low-level programming libraries presents a significant dilemma for developers, impacting not only the efficiency and effectiveness of ML models but also their environmental footprint. We have proposed a comprehensive framework to aid in this decision-making process, underpinned by a detailed comparative analysis of both types of libraries on one of the key IoT ML tasks: image classification. We have introduced a novel algorithm designed to calculate the green footprint of ML model training, factoring in execution time, memory utilization, power consumption, and CPU temperature, addressing the urgent need for sustainable ML practices. Through an empirical evaluation of popular libraries such as PyTorch for high-level and Libtorch for low-level development, we have assessed their performance, development efficiency, and hardware compatibility. The culmination of our research is a decision support system that synthesizes the experimental findings to guide developers toward choices that harmonize model performance with environmental sustainability.
Conference Papers (8)
-
[C1]EMNLP 2026
Many Dialects, Many Languages, One Cultural Lens: Evaluating Multilingual VLMs for Bengali Culture Understanding Across Historically Linked Languages and Regional Dialects
Findings of the Conference on Empirical Methods in Natural Language Processing (EMNLP 2026)
Bangla culture is richly expressed through region, dialect, history, food, politics, media, and everyday visual life, yet it remains underrepresented in multimodal evaluation. To address this gap, we introduce BanglaVerse, a culturally grounded benchmark for evaluating multilingual vision-language models (VLMs) on Bengali culture across historically linked languages and regional dialects. Built from 1,152 manually curated images across nine domains, the benchmark supports visual question answering and captioning, and is expanded into four languages and five Bangla dialects, yielding ~32.2K artifacts. Our experiments show that evaluating only standard Bangla overestimates true model capability: performance drops under dialectal variation, especially for caption generation, while historically linked languages such as Hindi and Urdu retain some cultural meaning but remain weaker for structured reasoning. Across domains, the main bottleneck is missing cultural knowledge rather than visual grounding alone, with knowledge-intensive categories. These findings position BanglaVerse as a more realistic test bed for measuring culturally grounded multimodal understanding under linguistic variation.
-
[C2]ACL 2026
Ready to Translate, Not to Represent? Bias and Performance Gaps in Multilingual LLMs Across Language Families and Domains
Findings of the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026)
The rise of Large Language Models (LLMs) has redefined Machine Translation (MT), enabling context-aware and fluent translations across hundreds of languages and textual domains. Despite their remarkable capabilities, LLMs often exhibit uneven performance across language families and specialized domains. Moreover, recent evidence reveals that these models can encode and amplify different biases present in their training data, posing serious concerns for fairness, especially in low-resource languages. To address these gaps, we introduce Translation Tangles, a unified framework and dataset for evaluating the translation quality and fairness of open-source LLMs. Our approach benchmarks 24 bidirectional language pairs across multiple domains using different metrics. We further propose a hybrid bias detection pipeline that integrates rule-based heuristics, semantic similarity filtering, and LLM-based validation. We also introduce a high-quality, bias-annotated dataset based on human evaluations of 1,439 translation-reference pairs.
-
[C3]EACL 2026
MathMist: A Parallel Multilingual Benchmark Dataset for Mathematical Problem Solving and Reasoning
Findings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (EACL 2026)
Mathematical reasoning remains one of the most challenging domains for large language models (LLMs), requiring not only linguistic understanding but also structured logical deduction and numerical precision. While recent LLMs demonstrate strong general-purpose reasoning abilities, their mathematical competence across diverse languages remains underexplored. Existing benchmarks primarily focus on English or a narrow subset of high-resource languages, leaving significant gaps in assessing multilingual and cross-lingual mathematical reasoning. To address this, we introduce MathMist, a parallel multilingual benchmark for mathematical problem solving and reasoning. MathMist encompasses 2,890 parallel Bangla-English gold standard artifacts, totaling ≈ 30K aligned question–answer pairs across thirteen languages, representing an extensive coverage of high-, medium-, and low-resource linguistic settings. The dataset captures linguistic variety, multiple types of problem settings, and solution synthesizing capabilities. We systematically evaluate a diverse suite of models, including open-source small and medium LLMs, proprietary systems, and multilingual-reasoning-focused models under zero-shot, chain-of-thought (CoT), perturbated reasoning, and code-switched reasoning paradigms. Our results reveal persistent deficiencies in LLMs’ ability to perform consistent and interpretable mathematical reasoning across languages, with pronounced degradation in low-resource settings.
-
[C4]*SEM 2026
Can LLMs Solve My Grandma’s Riddle? Evaluating Multilingual Large Language Models on Reasoning Traditional Bangla Tricky Riddles
Poster Presentation, 15th Joint Conference on Lexical and Computational Semantics (*SEM 2026), co-located with ACL 2026
Large Language Models (LLMs) show impressive performance on many NLP benchmarks, yet their ability to reason in figurative, culturally grounded, and low-resource settings remains underexplored. We address this gap for Bangla by introducing BanglaRiddleEval, a benchmark of 1,244 traditional Bangla riddles instantiated across four tasks (4,976 riddle-task artifacts in total). Using an LLM-based pipeline, we generate Chain-of-Thought explanations, semantically coherent distractors, and fine-grained ambiguity annotations, and evaluate a diverse suite of open-source and closed-source models under different prompting strategies. Models achieve moderate semantic overlap on generative QA but low correctness, MCQ accuracy peaks at only about 56% versus an 83.3% human baseline, and ambiguity resolution ranges from roughly 26% to 68%, with high-quality explanations confined to the strongest models. These results show that current LLMs capture some cues needed for Bangla riddle reasoning but remain far from human-level performance, establishing BanglaRiddleEval as a challenging new benchmark for low-resource figurative reasoning.
-
[C5]ICLR 2024
MosquitoFusion: A Multiclass Dataset for Real-Time Detection of Mosquitoes, Swarms, and Breeding Sites Using Deep Learning
Notable (Top 5%), Tiny Papers Track, Oral Presentation, The Twelfth International Conference on Learning Representations (ICLR 2024)
In this paper, we present an integrated approach to real-time mosquito detection using our multiclass dataset (MosquitoFusion) containing 1204 diverse images and leverage cutting-edge technologies, specifically computer vision, to automate the identification of Mosquitoes, Swarms, and Breeding Sites. The pre-trained YOLOv8 model, trained on this dataset, achieved a mean Average Precision (mAP@50) of 57.1%, with precision at 73.4% and recall at 50.5%. The integration of Geographic Information Systems (GIS) further enriches the depth of our analysis, providing valuable insights into spatial patterns.
-
[C6]TENSYMP 2024
MosquitoMiner: A Light Weight Rover for Detecting and Eliminating Mosquito Breeding Sites
Oral Presentation, IEEE Region 10 Symposium 2024 (TENSYMP 2024)
In this paper, we present a novel approach to the development and deployment of an autonomous mosquito breeding place detector rover with the object and obstacle detection capabilities to control mosquitoes. Mosquito-borne diseases continue to pose significant health threats globally, with conventional control methods proving slow and inefficient. Amidst rising concerns over the rapid spread of these diseases, there is an urgent need for innovative and efficient strategies to manage mosquito populations and prevent disease transmission. To mitigate the limitations of manual labor and traditional methods, our rover employs autonomous control strategies. Leveraging our own custom dataset, the rover can autonomously navigate along a pre-defined path, identifying and mitigating potential breeding grounds with precision. It then proceeds to eliminate these breeding grounds by spraying a chemical agent, effectively eradicating mosquito habitats. Our project demonstrates the effectiveness that is absent in traditional ways of controlling and safeguarding public health.
-
[C7]ICEEICT 2024
BrailleSense: Deep Learning for Braille Character Classification
Oral Presentation, 6th International Conference on Electrical Engineering and Information Communication Technology (ICEEICT 2024)
Individuals with visual impairments face challenges accessing written information. Braille is one of many solutions that only requires the reader to sense the depth of the paper with their hand to comprehend written information without the need to look at the text. However, learning Braille, especially for those losing sight later in life, presents difficulties. This research introduces the BrailleSense system, a technological solution designed to assist visually impaired individuals in learning and utilizing the Braille system effectively. The system features a virtual prototype of hand gloves equipped with a camera, aiming to alleviate challenges associated with Braille pattern memorization. Key contributions include the development of a custom lightweight Convolutional Neural Network (CNN) model for Braille pattern classification coined as the BrailleNet. This model is then deployed on a Raspberry Pi to investigate the feasibility of working with resource-limited portable devices, BrailleNet achieves an impressive accuracy of 97.44% under real-world constraints. The research outlines the conceptual design through a 3D model of the gloves, addressing spatial allocation. Acknowledging challenges in user comfort and alignment, BrailleSense presents a pioneering step towards empowering visually impaired individuals, enhancing literacy, and fostering independence.
-
[C8]InC4 2023
Fem-Scale: A Data-Driven Approach for Quantifying Degree of Individual Feminism Perspective
Oral Presentation, IEEE International Conference on Contemporary Computing and Communications (InC4 2023)
Abuse of women is widespread in underdeveloped nations like Bangladesh, where women are still afraid to assert their rights, and this trend persists. Even though significant changes are apparent, they have not entirely disappeared from society. On the other hand, there are some negative notions of feminism prevailing in our community. The lack of proper classification of the individual feminism perspective in relevant degrees is a significant concern while developing effective policies against this phenomenon. In this work, we develop a data-driven approach to quantify the degree of the individual feminist standpoint. We collected data from a selected crowd using survey questionnaires and analyzed the responses to segregate the participants into a Fem-Scale with five distinct classes: Sexist, Partial Sexist, Moderate, Feminist, and Radical feminist. We also validated the proposed Fem-Scale with real-life data generating an accuracy of 97.67% and highlighted the critical insights of the qualitative analysis of the collected data. To the best of the authors’ knowledge, this is the first attempt to introduce the degree of individual feminism standpoint. The proposed method can be beneficial in future research, including justifying the standpoint regarding feminism using public data from an individual’s social networking site (SNS).
Workshop Papers (5)
-
[W1]HCI+NLP 2025
Rethinking Search: A Study of University Students’ Perspectives on Using LLMs and Traditional Search Engines in Academic Problem Solving
Workshop on Bridging Human-Computer Interaction and Natural Language Processing (HCI+NLP), co-located with EMNLP 2025
With the increasing integration of Artificial Intelligence (AI) in academic problem solving, university students frequently alternate between traditional search engines like Google and large language models (LLMs) for information retrieval. This study explores students’ perceptions of both tools, emphasizing usability, efficiency, and their integration into academic workflows. Employing a mixed-methods approach, we surveyed 109 students from diverse disciplines and conducted in-depth interviews with 12 participants. Quantitative analyses, including ANOVA and chi-square tests, were used to assess differences in efficiency, satisfaction, and tool preference. Qualitative insights revealed that students commonly switch between GPT and Google: using Google for credible, multi-source information and GPT for summarization, explanation, and drafting. While neither tool proved sufficient on its own, there was a strong demand for a hybrid solution. In response, we developed a prototype, a chatbot embedded within the search interface, that combines GPT’s conversational capabilities with Google’s reliability to enhance academic research and reduce cognitive load.
-
[W2]MusIML 2024
MIMIC: Multimodal Islamophobic Meme Identification and Classification
Poster Presentation, Muslims in ML Workshop (MusIML), co-located with NeurIPS 2024
Anti-Muslim hate speech has emerged within memes, characterized by context-dependent and rhetorical messages using text and images that seemingly mimic humor but convey Islamophobic sentiments. This work presents a novel dataset and proposes a classifier based on the Vision-and-Language Transformer (ViLT) specifically tailored to identify anti-Muslim hate within memes by integrating both visual and textual representations. Our model leverages joint modal embeddings between meme images and incorporated text to capture nuanced Islamophobic narratives that are unique to meme culture, providing both high detection accuracy and interoperability.
-
[W3]EACL SRW 2026
Do Multi-Agents Solve Better Than Single? Evaluating Agentic Frameworks for Diagram-Grounded Geometry Problem Solving and Reasoning
EACL 2026 Student Research Workshop (SRW)
Diagram-grounded geometry problem solving is a critical benchmark for multimodal large language models (MLLMs), yet the benefits of multi-agent design over single-agent remain unclear. We systematically compare single-agent and multi-agent pipelines on four visual math benchmarks: Geometry3K, MathVerse, OlympiadBench, and We-Math. For open-source models, multi-agent consistently improves performance. For example, Qwen-2.5-VL (7B) gains +6.8 points and Qwen-2.5-VL (32B) gains +3.3 on Geometry3K, and both Qwen-2.5-VL variants see further gains on OlympiadBench and We-Math. In contrast, the closed-source Gemini-2.0-Flash generally performs better in single-agent mode on classic benchmarks, while multi-agent yields only modest improvements on the newer We-Math dataset. These findings show that multi-agent pipelines provide clear benefits for open-source models and can assist strong proprietary systems on newer, less familiar benchmarks, but agentic decomposition is not universally optimal.
-
[W4]CVPRW 2026
VisText-Mosquito: A Unified Multimodal Benchmark Dataset for Visual Detection, Segmentation, and Textual Reasoning on Mosquito Breeding Sites
International Workshop on Vision Intelligence for Real-world Challenges, IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPRW 2026)
Mosquito-borne diseases pose a major global health risk, requiring early detection and proactive control of breeding sites to prevent outbreaks. In this paper, we present VisText-Mosquito, a multimodal dataset that integrates visual and textual data to support automated detection, segmentation, and explanation for mosquito breeding site analysis. The dataset includes 1,828 annotated images for object detection, 142 images for water surface segmentation, and natural language explanation texts linked to each image. The YOLOv9s model achieves the highest precision of 0.92926 and mAP@50 of 0.92891 for object detection, while YOLOv11n-Seg reaches a segmentation precision of 0.91587 and mAP@50 of 0.79795. For textual explanation generation, we tested a range of large vision-language models (LVLMs) in both zero-shot and few-shot settings. Our fine-tuned Mosquito-LLaMA3-8B model achieved the best results, with a final loss of 0.0028, a BLEU score of 54.7, BERTScore of 0.91, and ROUGE-L of 0.85. This dataset and model framework emphasize the theme “Prevention is Better than Cure”, showcasing how AI-based detection can proactively address mosquito-borne disease risks.
-
[W5]MeLLM 2026
MultiHaluDet: Multilingual Hallucination Detection via LLM Hidden State Probing
ACL 2026 Workshop on Multilinguality in the Era of Large Language Models (MeLLM @ ACL 2026)
Hallucinations in Large Language Models (LLMs) represent a critical barrier to their reliable deployment, a vulnerability heavily exacerbated in non-English and resource-constrained contexts. Existing detection approaches that rely on output confidence heuristics or single-layer internal representations frequently fail to capture deep, complex factual inconsistencies across diverse languages. To address this, we introduce MultiHaluDet, a novel three-stage stacking framework that detects multilingual hallucinations by probing the full hidden state trajectories of frozen LLMs without requiring language-specific fine-tuning. Our method extracts sequential features across multiple layers and processes them via a hybrid architecture using multi-scale attention and self-attention pooling. By generating out-of-fold embeddings that feed into a calibrated classical classifier ensemble, MultiHaluDet captures both fine-grained and coarse-grained patterns of factual inconsistency. Extensive experiments demonstrate that our framework achieves state-of-the-art detection performance, reaching up to 98.55% AUROC on the English HaluEval and TriviaQA benchmarks using Mistral-7B and LLaMA2-7B architectures. Crucially, we rigorously evaluate our framework’s cross-lingual generalization across high (French), medium (Bangla), and low-resource (Amharic) languages. MultiHaluDet demonstrates exceptional representational robustness, consistently outperforming baselines and successfully transferring hallucination detection capabilities across typologically diverse linguistic tiers.
Under Review (8)
-
[U1]ACL ARR 2026
Time Present and Time Past: Benchmarking Large Language Models on Temporally Evolving Document Understanding
Under review — ACL ARR 2026, August Submission
-
[U2]ACL ARR 2026
To What Extent Do Large Language Models Understand Bangla Idioms?
Under review — ACL ARR 2026, August Submission
-
[U3]ACL ARR 2026
EGAL: Evidence-Grounded Multi-Agent Adversarial Learning at Inference Time for Research Ideation
Under review — ACL ARR 2026, August Submission
-
[U4]ACL ARR 2026
BanTMedQA: A Multimodal Benchmark for Open Ended Bangla Medical Question Answering
Under review — ACL ARR 2026, August Submission
-
[U5]WACV 2027
SafeProbe: Measuring Cross-Modal Validity of Latent Safety Monitors in Vision-Language Models
Under review — Winter Conference on Applications of Computer Vision (WACV 2027), Round 2 Submission
-
[U6]WACV 2027
Entropy-Gated Dynamic Rectification for Object Hallucination in Large Vision-Language Models
Under review — Winter Conference on Applications of Computer Vision (WACV 2027), Round 2 Submission
-
[U7]WACV 2027
TrajAD: Modeling Feature Trajectories for Training-Free Few-Shot Visual Anomaly Localization
Under review — Winter Conference on Applications of Computer Vision (WACV 2027), Round 2 Submission
-
[U8]AI Review
Therapist and Patient Simulation using Large Language Models in Mental Health: A Survey
Under review — Artificial Intelligence Review, Springer Nature
Contact
Happy to talk about research, collaborations, or anything in the papers above. Email is the fastest way to reach me.
Send a message
Contact details
- Email msayeedi212049@bscse.uiu.ac.bd
- Secondary email faiyazabdullah114708@gmail.com
- Phone +880 1704 054900
- Location Dhaka, Bangladesh