Selected Projects

ScaLable Sequential Object-Oriented Representation Learning (SCALOR) [ICLR-2020]

SCALOR (SCALable sequential Object-oriented Representation learning) is the first completely unsupervised generative model capable of simultaneously tracking several tens of objects in scenes with dynamic background,

as well as in realistic natural scenes. It is also capable of future-time sequence generation.

  • SCALOR is a totally unsupervised generative model.

  • SCALOR significantly improves the tracking scalability (two orders of magnitude) compared to the state-of-the-art models.

  • It is not only applicable to setting containing nearly a hundred objects, but it is also more computationally efficient compared to SQAIR (which scales only to a few objects).

  • Propagation–discovery process is parallelized by introducing the propose–reject model, reducing the time complexity from O(N) to O(1).

  • SCALOR can model scenes with a complex dynamic background.

  • SCALOR is the first probabilistic model capable of handling natural images.

Domain Authoring Assistant for Intelligent Virtual Agent [AAMAS-2019]

Developing intelligent virtual characters has recently attracted a lot of attention. The process of creating such characters often involves a team of creative authors who describe different aspects of the characters in natural language, and another team planning experts that translate this description into a specific planning domain. This can be quite challenging and resource-demanding as the team of creative authors should diligently define every aspect of the character especially if it contains complex human-like behavior. The team of engineers has to manually translate the natural language description of a character’s personality into the planning domain knowledge afterward. 

The goal of this paper is to introduce an authoring assistant tool to automate the process of domain generation from natural language description of virtual characters, thus bridging between the creative authoring team and the planning domain experts. This can be roughly categorized under the tasks of semantic parsing of highly structured text. Moreover, our proposed model also identifies possible missing information in the domain description and iteratively makes suggestions to the author.  Our model is able to achieve reasonable accuracy on an annotated dataset authored by professional authors while simultaneously providing enjoyable user experience.

In this paper, we provide a tool that :

  • uses a natural language model to translate (parse semantically) a natural language text describing a  virtual character's world and set behavior and abilities into a runnable planning domain that can be directly used in a planner.

  • generates domain-specific environment and object properties as well as object affordances, given a natural language description of a character's behavior.

  • Automatically identify potentially missing aspects of the character's personality not mentioned by the author, and asks the author to provide them.

  • identifies inconsistent information in character description and suggests possible improvements.

  • uses deep sentence embeddings as well as a commonsense knowledge-base to suggest new possible       aspects of the character and the story it comes in.

  • assist the authors to develop virtual characters while having enjoyable user experiences.

Topic Spotting using Hierarchical Networks with Self Attention [NAACL-2019]

The success of deep learning techniques has renewed interest in the development of dialogue systems. However, current systems struggle to have consistent long term conversations with the users and fail to build rapport. Topic spotting, the task of automatically inferring the topic of a conversation, has been shown to be helpful in making a dialog system more engaging and efficient. We propose a hierarchical model with self-attention for topic spotting. Experiments on the Switchboard corpus show the superior performance of our model over previously proposed techniques for topic spotting as well as other SOTA baselines for text classification. Additionally, in contrast to the offline processing of dialog, we also analyze the performance of our model in a more realistic online scenario where the topic is identified in real-time as the dialog progresses. Results show that our model is able to generalize even with limited information in the online setting.

In this paper, we present a model that :

  • uses 2-layered deep Bi-LSTM network with self attention for the task of topic classification in dialogue systems containing a wide range of topics.

  • Our model is not only superior compared to the state of the art in the offline setting but also outperforms baselines in an online setting.

  • Our model is able to generalize better  when data is scarce.

  • Interestingly even when the model misclassifies topics, only relevant ones are confused.

Statistical Association Mapping of Population-Structured Genetic Data 

[IEEE Transaction on Computational Biology and Bioinformatics]

The purpose of Genome-Wide Association Studies (GWAS) is to infer statistical associations between specific regions
of DNA sequence with the causal factors underlying a specific disease or any other observable property of an organism. Although during recent years, GWAS methods have been successful in identifying many causal factors for different types of diseases, traditional methods in this area suffer from critical drawbacks.
First, most traditional GWAS frameworks consider DNA regions independent from each other thus neglecting their biochemical dependencies. This often leads to suboptimal results especially when multiple regions are involved in the formation of a complex disease such as cancer. Second, they assume genetic homogeneity for the population under study, which is usually not a plausible assumption.

In this paper, we propose a novel Bayesian statistical framework for association mapping (mapping disease types to underlying genes) to address the mentioned limitations. Our model works in the challenging scenario where our population consists of different latent subpopulations each with different genetic marks.  Our method employs Markov-Chain Moten-Carlo (MCMC) techniques such as Gibbs Sampling for association mapping.

In this paper, we introduce :

  • A novel Bayesian model based on Gibbs Sampling for association mapping in the presence of hidden population structures, where the population under study consists of numerous latent subpopulations with different genetic backgrounds. 

  • Our model not only identifies the latent population structure for each datapoint but is also able to identify hidden disease causal factors.

  • Our model outperforms state-of-the-art methods such as STRUCTURE and PLINK and is able to reach ~15% higher accuracy.

Other Projects







Crowd Behavior Modeling using Deep Neural Networks

To be completed!




A Novel Method for Fake News Classification

In this project, we provide a comprehensive review of the most recent and fundamental approaches for fake news classification. Furthermore, we introduce a generalized method based on multi-module deep neural nets to classify fake news not only based on content, but also style while proof checking it with public factual databases. Out model includes different BiLSTM  components for style extraction, content extraction as well as Information Retrieval. knowledge base using information

Our experiments conducted on fake news Challenge dataset and UW dataset show that for the given classification task our model performs comparably to state-of-the-art methods.



Classifying Motor Movements from EEG Data Using Spiking Neural Networks

The main goal of this project is to develop a SNN architecture to classify simple hand/leg movements in EEG data. This approach not only provides reasonable classification performance but also has biological plausibility. Currently, in the world of Artificial Intelligence (AI), the best classification systems are deep neural
networks that use artificial neurons. However, these systems employ learning techniques such as back-propagation that still have not been discovered in the brain. As such, spiking neural networks (SNNs) offer a more biologically plausible choice for such tasks. In this project, we develop a spiking neural net that aims to classify hand vs. leg movements from complex time-series EEG noisy data. This task is challenging in particular due to the complexity of EEG waves as well as the noise inherent in the datasets. Our obtained results are comparable to state-of-the art methods.


Report available upon request