Research

➡️Inspired by the success of using transformers in NLP and CV, researchers have explored their potentiality in Graph Learning, leading to the development of advanced models like Graph Transformer, Graphormer, Gophormer, Graph-Bert, etc.

💡Incorporating graph topology information is crucial in adapting transformers for graph data. Without this, the application of transformers becomes akin to processing IID data, which fails to capture the essence of graph structures and becomes less meaningful.

🔍 Our research has identified a notable gap in this area, especially regarding 𝐇𝐲𝐩𝐞𝐫𝐠𝐫𝐚𝐩𝐡 𝐓𝐫𝐚𝐧𝐬𝐟𝐨𝐫𝐦𝐞𝐫𝐬. While working on this, we found only a few papers worked on the modeling transformers for hypergraphs (hypergraphs transformers), and even then, they often overlooked the vital aspect of hypergraph topology information. Moreover, they are highly limited for Heterogeneous hypergraphs, resulting in a lack of generalization.

➡ To address these challenges, we present a novel approach: a 𝑻𝒐𝒑𝒐𝒍𝒐𝒈𝒚-𝑮𝒖𝒊𝒅𝒆𝒅 𝑯𝒚𝒑𝒆𝒓𝒈𝒓𝒂𝒑𝒉 𝑻𝒓𝒂𝒏𝒔𝒇𝒐𝒓𝒎𝒆𝒓 𝑵𝒆𝒕𝒘𝒐𝒓𝒌 (𝑻𝑯𝑻𝑵) designed explicitly for hypergraph learning. This model is a significant stride in graph analysis🔥, incorporating different innovative 𝐬𝐭𝐫𝐮𝐜𝐭𝐮𝐫𝐞-𝐩𝐨𝐬𝐢𝐭𝐢𝐨𝐧 𝐞𝐧𝐜𝐨𝐝𝐞𝐫𝐬. These encoders are instrumental in embedding both structural and spatial information of hypernodes, ensuring the model's sensitivity to the unique characteristics of hypergraphs.

➡ Additionally, we have developed different unique 𝐥𝐨𝐜𝐚𝐥 𝐚𝐧𝐝 𝐠𝐥𝐨𝐛𝐚𝐥 𝐬𝐭𝐫𝐮𝐜𝐭𝐮𝐫𝐞 𝐥𝐞𝐚𝐫𝐧𝐢𝐧𝐠 𝐦𝐨𝐝𝐮𝐥𝐞𝐬 working as topological inductive bias. These are integrated within the self-attention networks of our model, playing a crucial role in discerning the structural importance of hypernodes for a hyperedge and hyperedges for hypernodes in a hypergraph.

🚀 Through these advancements, our research aims to provide a more robust and nuanced approach to hypergraph learning, leveraging the complexities of hypergraph structures to enhance the capabilities of transformer models for hypergraphs.

➡️HyGNN is an attention-based encoder-decoder architecture designed to predict drug-drug interactions (DDIs). HyGNN relies on the hypothesis that similar drugs behave similarly, are likely to interact with the same drugs, and two drugs are similar if they have similar substructures as functional groups in their SMILES strings. To properly depict the structural-based similarity between drugs, we present them in a novel hypergraph setting (Drug-Hypergraph), representing drugs as hyperedges connecting many substructures as nodes.

After constructing the hypergraph, we develop a Hypergraph Neural Network (HyGNN), a model that learns the DDIs by generating and using the representation of hyperedges as drugs. HyGNN proposes 𝐚 𝐧𝐨𝐯𝐞𝐥 𝐡𝐲𝐩𝐞𝐫𝐠𝐫𝐚𝐩𝐡 𝐞𝐝𝐠𝐞 𝐞𝐧𝐜𝐨𝐝𝐞𝐫 🛎️ consisting of two layers of attention (node and edge level) mechanism which can precisely 𝐝𝐢𝐬𝐜𝐨𝐯𝐞𝐫 𝐢𝐦𝐩𝐨𝐫𝐭𝐚𝐧𝐭 𝐬𝐮𝐛𝐬𝐭𝐫𝐮𝐜𝐭𝐮𝐫𝐞𝐬  🔎 from a chemical compound. Finally, a decoder is exploited to predict DDIs.

Extensive experiments are performed on the proposed methods and some baselines as well. Results show that the proposed HyGNN can accurately predict DDIs, and its performance significantly outperforms all the baselines 🔥, including two notable pioneer research works, CASTER and Decagon, even for 𝐛𝐫𝐚𝐧𝐝 𝐧𝐞𝐰 𝐝𝐫𝐮𝐠𝐬. 

➡️Drug-drug interaction (DDI) is the activity that occurs when the impact of one drug changes when combined with another. DDIs may obstruct, increase, or decrease the intended effect of either drug or, in the worst-case scenario, create adverse side effects. In this paper, we present a novel heterogeneous graph attention model, HAN-DDI, to predict drug-drug interactions. We create a heterogeneous network of drugs with different biological entities. Then, we develop a heterogeneous graph attention network to learn DDIs using the relations of drugs with other entities. It consists of an attention-based heterogeneous graph node encoder for obtaining drug node representations and a decoder for predicting drug-drug interactions. Further, we utilize comprehensive experiments to evaluate our model and to compare it with state-of-the-art models. Experimental results show that our proposed method, HAN-DDI, outperforms the baselines significantly and accurately predicts DDIs, even for new drugs.

➡️In this paper, we propose a novel method for predicting DDIs based on the vital chemical substructure of drugs extracted from their SMILES strings. We construct a graph that connects drugs based on their common functional chemical substructures. Furthermore, we apply different well-known graph neural network (GNN) methods to generate drug embeddings. Drug embeddings of individual drugs are concatenated to generate features of drug pairs. Finally, drug pair features are fed to different machine learning (ML) classifiers for DDI prediction. We evaluate our model on the DrugBank dataset. Our result shows promising results, and our model outperforms a baseline model based on different DDI representation creation methods.

➡️Extracting meaningful features from sequences and devising effective similarity measures are vital for sequence data mining tasks, particularly sequence classification. While Neural Network models are commonly used to learn features of sequence automatically, they are limited to capturing adjacent structural connection information and ignore global, higher-order information between the sequences. 

To address these challenges, we propose a novel Hypergraph Attention Network model, namely Seq-HyGAN, for sequence classification problems. To capture the complex structural similarity between sequence data, we create a novel hypergraph model by defining higher-order relations between subsequences (nodes) extracted from sequences (hyperedges). Subsequently, we introduce a Sequence Hypergraph Attention Network that learns sequence features by considering the significance of subsequences and sequences to one another. It consists of three levels of aggregation with attention that captures different levels of context. At the first level, it generates node embedding that incorporates global context by aggregating hyperedge embeddings. At the second level, the model refines node embeddings for each hyperedge. It captures local context by aggregating neighboring node embeddings in the same hyperedge and considering the subsequence position in a sequence. Finally, at the third level, it generates sequence embedding by aggregating node embeddings from both global (level 1) and local (level 2) perspectives, resulting in a comprehensive representation of the sequences 

Through extensive experiments, we demonstrate the effectiveness of our proposed Seq-HyGAN model in accurately classifying sequence data, outperforming several state-of-the-art methods by a significant margin. 

➡️The rate of non-medical use of opioid drugs has increased markedly since the early 2000s. Many studies have been done to detect Drug Abuse (DA) events from social media data using machine learning and deep learning concepts. Moreover, Graph Neural Networks (GNNs) have recently become popular in text classification tasks due to their high accuracy and capability to handle complex structures. In this work, we collect drugs-related Twitter data (tweets) and build text graphs (corpus-level and document-level) to capture word-word, document-word, and document-document relations. Then we apply different GNN models on those text graphs and thus turn the text classification task into a node classification (for corpus-level graph) and graph classification (for document-level graph) task to detect DA events. Finally, we compare our graph-based DA detection models with different types of baseline models, including rule-based, traditional machine learning, and deep learning models. Our result shows that graph-based models outperform traditional machine learning and deep learning-based models.

➡️Opioid Use Disorder (OUD) is one of the most severe healthcare problems in the USA. People addicted to opioids need various treatments, including Medication-Assisted Treatment (MAT), proper counseling, and behavioral therapies. However, during the peak time of the COVID-19 pandemic, the supply of emergency medications was seriously disrupted. Patients faced severe medical care scarcity since many pharmaceutical companies, drugstores, and local pharmacies were closed. Import-export was also canceled to consent to the government emergency law, i.e., lockdown, quarantine, and isolation. These circumstances and their negative effects on OUD patients’ psychology could have led them to drop out of MAT medications and be persuaded to resume illicit opioid use. This project involves collecting and analyzing a large volume of Twitter data related to MAT medications for OUD patients. We discover the Active MAT Medicine Users (AMMUs) on Twitter. For this, we build a seed dictionary of words related to OUD and MAT and apply association rules to expand it. Further, AMMUs’ tweet posts are studied ‘before the pandemic’ (BP) and ‘during the pandemic’ (DP) to understand how drug behaviors and habits have changed due to COVID-19. We also perform sentiment analysis on Tweets to determine the impact of the COVID-19 pandemic on the psychology of AMMUs. Our analysis shows that the use of MAT medications has decreased by around 30.54%, where the use of illicit drugs and other prescription opioids increased by 18.06% and 12.12%, respectively, based on AMMUs’ tweets posted during the lockdown compared with before the lockdown statistics. The COVID-19 pandemic and lockdown may result in the resumption of illegal and prescription opioid abuse by OUD patients. Necessary steps and precautions should be taken by health care providers to ensure the emergency supply of medicines and also psychological support and thus prevent patients from illicit opioid use.