Seminars

View all Seminars  |  Download ICal for this event

Temporal Point Processes for Forecasting Events in Higher-Order Networks

Series: Ph.D. Colloquium

Speaker: Tony Gracious, Ph.D (Engg.) student, Dept. of C.S.A

Date/Time: Jul 21 10:00:00

Location: CSA Seminar Hall (Room No. 254, First Floor)

Faculty Advisor: Prof. Ambedkar Dukkipati

Abstract:
Complex systems consisting of interacting entities can be effectively represented as time-evolving networks or graphs, where the entities are depicted as nodes, and the interactions between them are represented as instantaneous edges. Modeling the evolution of these systems and forecasting interaction events are of significant importance for many fields, such as e-commerce, financial markets, neuroscience, etc. This is achieved using the Temporal Point Process (TPP) framework, a stochastic process that models these interactions as discrete events occurring in continuous time. The existing works on interaction forecasting are applicable only to pair-wise edges.

However, real-world events involving interactions are much more complex than pair-wise interactions. It involves a group of entities interacting in a complex way rather than just two entities. This leads to the formation of time-evolving higher-order networks. There has not been much research to develop machine learning algorithms for event prediction in these types of networks. This thesis addresses this by providing solutions to the following problems: (i) How can we use TPP to forecast higher-order interaction events? (ii) Considering the number of possible events grows exponentially when problem setting changes from pair-wise to higher-order, can we forecast the next event in a scalable way? (iii) How can we incorporate relations and group structure within a higher-order interaction into the forecasting model?

The first contribution of this thesis is a model for forecasting higher-order interactions among a group of entities as instantaneous hyperedge events in a network. In this model, we introduce a TPP on each hyperedge, with the conditional intensity parameterized by a hyperedge-based decoder that uses node embeddings. To account for the temporal evolution of entities, we employ Temporal Graph Representation Learning (TGRL) techniques such as hypergraph convolution to learn dynamic node embeddings, and model parameters are learned by minimizing the negative log-likelihood of the TPP process. Furthermore, our model has been extended to accommodate bipartite interactions, where interactions occur between two distinct groups of entities of different types. To achieve this, we introduce a bipartite hyperedge-based decoder, which incorporates separate node embedding modules for each node type. Through comprehensive experiments, we demonstrate the effectiveness of our model by comparing it to previous works and baseline models. Moreover, through ablation studies, we highlight the superior performance of hyperedge-based models in capturing higher-order interactions compared to pairwise models.

Secondly, we introduce a model to forecast directed higher-order interactions occurring between two distinct groups of entities. Unlike the previous approach that focuses on representation learning from higher-order interactions, here we also introduce a strategy to forecast hyperedges in a scalable way. For that, we employ a multi-task framework for forecasting candidate hyperedges. This involves a TPP-based model to predict the time of events on each node, followed by pairwise neighborhood and hyperedge size prediction modules for generating candidate hyperedges. This will reduce the exponential search in forecasting future hyperedges in the previous models. Then a directed hyperedge link predictor is used to identify the true hyperedge from false ones. Further, we devise a TGRL framework that improves the models scalability when dealing with datasets containing many interactions using a memory network module.

In our final contribution, we extend the existing higher-order interaction forecasting approaches based on TPP to accommodate real-world interactions that involve internal group structures with nodes of different types. Each group is associated with a specific relation type, and to address this complexity, we introduce the concept of multi-relational recursive hyperedge formation events. In this framework, hyperedges can serve as nodes within other hyperedges, creating a hierarchical structure. Additionally, we extended the widely-used Temporal Knowledge Graphs (TKG) framework within this context by incorporating subject-specific and object-specific hyperedges.


In conclusion, this thesis emphasizes and demonstrates the importance of employing higher-order dynamic network models to forecast interactions in real-world complex systems effectively. As part of this, we have shown that hyperedges can represent higher-order interactions as they provide a natural framework for representing relations between more than two entities. Our first work uses it for higher-order interaction forecasting of homogeneous and bipartite nature. This is further extended to directed higher-order interactions in the second work, where we show the use of directed hyperedges. In this, we tackled the challenge of scalability in forecasting higher-order interactions by adopting a TPP-based multi-task approach. Through our final contribution, we showed the need to go beyond hyperedge by using multi-relation recursive hyperedges to incorporate the intricate relations of entities in a higher-order interaction.