Research Assistant at Sharif University of Technology

Utilizing Large Language Models for Medical Question Answering in Persian and English

It introduces a model that utilizes PubMed QA and PubMed summarization datasets for answering questions and novel dataset including persian-english sentences with translated disease names. This model is equipped to respond to medical questions in both Persian and English by following these steps:

• A dataset is created using a Large Language Models (LLMs) to formulate sentences featuring disease names in both English and Persian.
• The small-100 translator model is fine-tuned with this dataset to accurately translate medical sentences from English to Persian and vice versa.
• The PubMed QA dataset is used to obtain embeddings via Bio-Bert.
• ElasticSearch is utilized for vector search, which produces a multitude of documents.
• Bio-Bert, T5, and Large Language Models (LLMs) are used to summarize the documents provided by ElasticSearch. The T5 and Bio-Bert models are fine-tuned using the PubMed summarization dataset.

Under the Supervision of Ehsaneddin Asgari at Computer Engineering Department.

July. 2023 – Present

Utilizing Large Language Models for Multilingual News Question Answering

It introduces a model that utilizes news dataset, generated by other students, for answering questions. This model is equipped to respond to news questions in all languages by following these steps:

• The small-100 translator model is evaluated for its ability to accurately translate sentences in different languages.
• Use tsdae-bert-base-dv-news-title in order to get embedding.
• ElasticSearch is utilized for vector search, which produces a multitude of news documents.
• T5 and Large Language Models (LLMs) are used to summarize the documents provided by ElasticSearch (The T5 model is fine-tuned).

Under the Supervision of Ehsaneddin Asgari at Computer Engineering Department.

August. 2023 – Present

Bug Issues Ranker Based on Frequently Used Programming Methods, Sentiment, and Reactions

The model being introduced sorts the issues based on their methods using a call graph and some other features. It calculates the frequency of method usage, analyzes sentiment, and reactions, giving more importance to issues linked to the most commonly used methods, those with the highest reaction rate, and those displaying the most negative sentiment. The model is also engineered to pinpoint methods related to each issue in large, high-quality GitHub projects. It performs in following the steps below:

• Crawl most important github projects’ issues and preprocess their methods.
• Build call graph by using the method calls.
• Compute sentiment of issues and extract statistics related to issue reactions
• Use FastText, TF-IDF, and transformer models in order to find related methods to each issues based on their priority.

Under the Supervision of Abbas Heydarnoori at Computer Engineering Department.

August. 2022 – Present

A Unique Approach for Node Identification, Weighted Tree for HTML Page, and Difference Calculation in DOM Trees (HDNA)

It identifies the differences by assigning a unique identifier, referred to as HDNA (HTML DNA), to each HTML page based on its structure and arrangement of tags. This identifier can be used to detect changes between two or more HTML pages. The approach is designed to efficiently capture structural changes in DOM trees, even with dynamically generated content, by analyzing hierarchical relationships, node attributes, and content variations. This could potentially improve website performance, enhance user experience, and increase security.

• Each tag (DOM node) is assigned an ordered triple (D,N,A)
  • The tag name is represented by A.
  • The count number, represented by N, is the numbering of nodes in the DOM tree from left to right at each level.
  • The number of descendants, represented by D, refers to the number of nodes in a tag’s sub-tree.
• A weighting mechanism is implemented to compute differences between multiple trees.
• The difference can be calculated in various ways depending on the problem, using the DNA and the weight for the trees.

Under the Supervision of Abbas Heydarnoori at Computer Engineering Department.

June. 2023 – Present

Research Assistant at University of New South Wales

Developing Automated Medical Report Generation for Fundus Fluorescein Angiography Images (A Novel Approach in Ophthalmology Research)

It introduces a model that uses Fundus Fluorescein Angiography Images and their corresponding reports, collectively known as FFA-IR datasets. The aim is to enhance the model’s reliance on medical information rather than solely on language metrics when generating reports. Some modifications have also been made to the Reinforcement Learning (RL) algorithm to improve its performance. The model follows these steps to generate a report for each case based on the patient’s images:

• A Convolutional Neural Network (CNN), acting as the Visual Extractor, extracts visual features from the medical images. The goal is to improve its understanding of medical features from images, rather than just using a standard CNN.
• Cross-modal Memory aligns the visual and textual features of a medical image and its report. Shared memory records the mappings between visual and textual information.
• The encoder-decoder in this model is built upon a standard Transformer with additional medical components to enhance its ability to generate medical reports. The decoder in such architectures typically takes the encoded representations of the input data and transforms them into the desired output format.
• A slightly modified version of the proposed reinforcement learning (RL) algorithm is applied. It leverages signals from natural language generation (NLG) metrics, such as BLEU, to guide the cross-modal mappings. This helps to better match features from images and texts, and provides a direct target of learning outcome for report generation. (This section is still under development)

Under the Supervision of Imran Razzak and Usman Naseem

June. 2023 – Present (Summer internship)