publications | Anindya Sarkar

2025

AAAI

Active Geospatial Search for Efficient Tenant Eviction Outreach

Anindya Sarkar, Alex DiChristofano, Sanmay Das, Patrick Fowler, and 2 more authors

In the 39th AAAI Conference on Artificial Intelligence, 2025, Philadelphia, Feb 2025

Abs HTML PDF Code

Tenant evictions threaten housing stability and are a major concern for many cities. An open question concerns whether data-driven methods enhance outreach programs that target at-risk tenants to mitigate their risk of eviction. We propose a novel active geospatial search (AGS) modeling framework for this problem. AGS integrates property-level information in a search policy that identifies a sequence of rental units to canvas to both determine their eviction risk and provide support if needed. We propose a hierarchical reinforcement learning approach to learn a search policy for AGS that scales to large urban areas containing thousands of parcels, balancing exploration and exploitation and accounting for travel costs and a budget constraint. Crucially, the search policy adapts online to newly discovered information about evictions. Evaluation using eviction data for a large urban area demonstrates that the proposed framework and algorithmic approach are considerably more effective at sequentially identifying eviction cases than baseline methods.

2024

NeurIPS

GOMAA-Geo: GOal Modality Agnostic Active Geo-localization

Anindya Sarkar, Srikumar Sastry, Aleksis Pirinen, Chongjie Zhang, and 2 more authors

In the 38th Neural Information Processing Systems, 2024, Vancouver, Dec 2024

Abs HTML PDF Code

We consider the task of active geo-localization (AGL) in which an agent uses a sequence of visual cues observed during aerial navigation to find a target specified through multiple possible modalities. This could emulate a UAV involved in a search-and-rescue operation navigating through an area, observing a stream of aerial images as it goes. The AGL task is associated with two important challenges. Firstly, an agent must deal with a goal specification in one of multiple modalities (e.g., through a natural language description) while the search cues are provided in other modalities (aerial imagery). The second challenge is limited localization time (e.g., limited battery life, urgency) so that the goal must be localized as efficiently as possible, i.e. the agent must effectively leverage its sequentially observed aerial views when searching for the goal. To address these challenges, we propose GOMAA-Geo - a goal modality agnostic active geo-localization agent - for zero-shot generalization between different goal modalities. Our approach combines cross-modality contrastive learning to align representations across modalities with supervised foundation model pretraining and reinforcement learning to obtain highly effective navigation and localization policies. Through extensive evaluations, we show that GOMAA-Geo outperforms alternative learnable approaches and that it generalizes across datasets - e.g., to disaster-hit areas without seeing a single disaster scenario during training - and goal modalities - e.g., to ground-level imagery or textual descriptions, despite only being trained with goals specified as aerial views.
WACV

A Visual Active Search Framework for Geospatial Exploration

Anindya Sarkar, Michael Lanier, Alfeld Scott, Jiarui Feng, and 3 more authors

In the IEEE/CVF Winter Conference on Applications of Computer Vision, 2024, Hawaii, Jan 2024

Abs HTML PDF Code

Many problems can be viewed as forms of geospatial search aided by aerial imagery, with examples ranging from detecting poaching activity to human trafficking. We model this class of problems in a visual active search (VAS) framework, which has three key inputs: (1) an image of the entire search area, which is subdivided into regions, (2) a local search function, which determines whether a previously unseen object class is present in a given region, and (3) a fixed search budget, which limits the number of times the local search function can be evaluated. The goal is to maximize the number of objects found within the search budget. We propose a reinforcement learning approach for VAS that learns a meta-search policy from a collection of fully annotated search tasks. This meta-search policy is then used to dynamically search for a novel target-object class, leveraging the outcome of any previous queries to determine where to query next. Through extensive experiments on several large-scale satellite imagery datasets, we show that the proposed approach significantly outperforms several strong baselines. We also propose novel domain adaptation techniques that improve the policy at decision time when there is a significant domain gap with the training data. Code is publicly available.
AAMAS

Geospatial Active Search for Preventing Evictions

Anindya Sarkar, Alex Dichristofano, Sanmay Das, Patrick Fowler, and 2 more authors

In the 23rd International Conference on Autonomous Agents and Multiagent Systems, 2024, Auckland, May 2024

Abs HTML PDF Code

Evictions are a threat to housing stability and a major concern for many cities. An open question is whether data-driven methods can enhance door-to-door outreach programs to target at-risk tenants. We model this problem using a new framework we term geospatial active search. Geospatial Active Search integrates visual information such as satellite imagery along with tabular data such as property and neighborhood-level information to create an online exploration plan. We develop an approach for the implementation of Geospatial Active Search in St. Louis to find properties containing tenants who will have an eviction filed against them.
AAAI Workshop

Attacks on Node Attributes in Graph Neural Networks

Xu Ying, Michael Lanier, Anindya Sarkar, and Yevgeniy Vorobeychik

In the AAAI Workshop on "AI for Cyber Security", 2024, Vancouver, Mar 2024

Abs HTML PDF Code

Graphs are commonly used to model complex networks prevalent in modern social media and literacy applications. Our research investigates the vulnerability of these graphs through the application of feature based adversarial attacks, focusing on both decision-time attacks and poisoning attacks. In contrast to state-of-the-art models like Net Attack and Meta Attack, which target node attributes and graph structure, our study specifically targets node attributes. For our analysis, we utilized the text dataset Hellaswag and graph datasets Cora and CiteSeer, providing a diverse basis for evaluation. Our findings indicate that decision-time attacks using Projected Gradient Descent (PGD) are more potent compared to poisoning attacks that employ Mean Node Embeddings and Graph Contrastive Learning strategies. This provides insights for graph data security, pinpointing where graph-based models are most vulnerable and thereby informing the development of stronger defense mechanisms against such attacks.

2023

NeurIPS

A Partially-Supervised Reinforcement Learning Framework for Visual Active Search

Anindya Sarkar, Nathan Jacobs, and Yevgeniy Vorobeychik

In the 37th Neural Information Processing Systems, 2023, New Orleans, Dec 2023

Abs HTML PDF Code

Visual active search (VAS) has been proposed as a modeling framework in which visual cues are used to guide exploration, with the goal of identifying regions of interest in a large geospatial area. Its potential applications include identifying hotspots of rare wildlife poaching activity, search-and-rescue scenarios, identifying illegal trafficking of weapons, drugs, or people, and many others. State of the art approaches to VAS include applications of deep reinforcement learning (DRL), which yield end-to-end search policies, and traditional active search, which combines predictions with custom algorithmic approaches. While the DRL framework has been shown to greatly outperform traditional active search in such domains, its end-to-end nature does not make full use of supervised information attained either during training, or during actual search, a significant limitation if search tasks differ significantly from those in the training distribution. We propose an approach that combines the strength of both DRL and conventional active search by decomposing the search policy into a prediction module, which produces a geospatial distribution of regions of interest based on task embedding and search history, and a search module, which takes the predictions and search history as input and outputs the search distribution. We develop a novel meta-learning approach for jointly learning the resulting combined policy that can make effective use of supervised information obtained both at training and decision time. Our extensive experiments demonstrate that the proposed representation and meta-learning frameworks significantly outperform state of the art in visual active search on several problem domains.

2022

NeurIPS

How powerful are K-hop message passing graph neural networks

Jiarui Feng, Yixin Chen, Anindya Sarkar, Fuhai Li, and 1 more author

In the 36th Neural Information Processing Systems, 2022, New Orleans, Dec 2022

Abs HTML PDF Code

The most popular design paradigm for Graph Neural Networks (GNNs) is 1-hop message passing—aggregating information from 1-hop neighbors repeatedly. However, the expressive power of 1-hop message passing is bounded by the WeisfeilerLehman (1-WL) test. Recently, researchers extended 1-hop message passing to K-hop message passing by aggregating information from K-hop neighbors of nodes simultaneously. However, there is no work on analyzing the expressive power of K-hop message passing. In this work, we theoretically characterize the expressive power of K-hop message passing. Specifically, we first formally differentiate two different kernels of K-hop message passing which are often misused in previous works. We then characterize the expressive power of K-hop message passing by showing that it is more powerful than 1-WL and can distinguish almost all regular graphs. Despite the higher expressive power, we show that K-hop message passing still cannot distinguish some simple regular graphs and its expressive power is bounded by 3-WL. To further enhance its expressive power, we introduce a KP-GNN framework, which improves K-hop message passing by leveraging the peripheral subgraph information in each hop. We show that KP-GNN can distinguish many distance regular graphs which could not be distinguished by previous distance encoding or 3-WL methods. Experimental results verify the expressive power and effectiveness of KP-GNN. KP-GNN achieves competitive results across all benchmark datasets.
CVPR

A Framework for Learning Ante-hoc Explainable Models via Concepts

Anirban Sarkar, Deepak Vijaykeerthy, Anindya Sarkar, and Vineeth Balasubramanian

In the IEEE / CVF Computer Vision and Pattern Recognition Conference, 2022, New Orleans, Jun 2022

Abs HTML PDF Code

Self-explaining deep models are designed to learn the latent concept-based explanations implicitly during training, which eliminates the requirement of any post-hoc explanation generation technique. In this work, we propose one such model that appends an explanation generation module on top of any basic network and jointly trains the whole module that shows high predictive performance and generates meaningful explanations in terms of concepts. Our training strategy is suitable for unsupervised concept learning with much lesser parameter space requirements compared to baseline methods. Our proposed model also has provision for leveraging self-supervision on concepts to extract better explanations. However, with full concept supervision, we achieve the best predictive performance compared to recently proposed concept-based explainable models. We report both qualitative and quantitative results with our method, which shows better performance than recently proposed concept-based explainability methods. We reported exhaustive results with two datasets without ground truth concepts, i.e., CIFAR10, ImageNet, and two datasets with ground truth concepts, i.e., AwA2, CUB-200, to show the effectiveness of our method for both cases. To the best of our knowledge, we are the first ante-hoc explanation generation method to show results with a large-scale dataset such as ImageNet.
WACV

Leveraging Test-Time Consensus Prediction for Robustness against Unseen Noise

Anindya Sarkar, Anirban Sarkar, and Vineeth Balasubramanian

In the IEEE/CVF Winter Conference on Applications of Computer Vision, 2022, Hawaii, Oct 2022

Abs HTML PDF

We propose a method to improve DNN robustness against unseen noisy corruptions, such as Gaussian noise, Shot Noise, Impulse Noise, Speckle noise with different levels of severity by leveraging ensemble technique through a consensus based prediction method using self-supervised learning at inference time. We also propose to enhance the model training by considering other aspects of the issue i.e. noise in data and better representation learning which shows even better generalization performance with the consensus based prediction strategy. We report results of each noisy corruption on the standard CIFAR10-C and ImageNet-C benchmark which shows significant boost in performance over previous methods. We also introduce results for MNIST-C and TinyImagenet-C to show usefulness of our method across datasets of different complexities to provide robustness against unseen noise. We show results with different architectures to validate our method against other baseline methods, and also conduct experiments to show the usefulness of each part of our method.
GAMESEC

Reward Delay Attacks on Deep Reinforcement Learning

Anindya Sarkar, Jiarui Feng, Yevgeniy Vorobeychik, Christopher Gill, and 1 more author

In the 13th Conference on Decision and Game Theory for Security, 2022, Pittsburgh, Oct 2022

Abs HTML PDF Code

Most reinforcement learning algorithms implicitly assume strong synchrony. We present novel attacks targeting Q-learning that exploit a vulnerability entailed by this assumption by delaying the reward signal for a limited time period. We consider two types of attack goals: targeted attacks, which aim to cause a target policy to be learned, and untargeted attacks, which simply aim to induce a policy with a low reward. We evaluate the efficacy of the proposed attacks through a series of experiments. Our first observation is that reward-delay attacks are extremely effective when the goal is simply to minimize reward. Indeed, we find that even naive baseline reward-delay attacks are also highly successful in minimizing the reward. Targeted attacks, on the other hand, are more challenging, although we nevertheless demonstrate that the proposed approaches remain highly effective at achieving the attacker’s targets. In addition, we introduce a second threat model that captures a minimal mitigation that ensures that rewards cannot be used out of sequence. We find that this mitigation remains insufficient to ensure robustness to attacks that delay, but preserve the order, of rewards.
ECCV Workshop

Empowering a Robust Model with Stable and Object-Aligned Explanations

Sowrya Gali, Anindya Sarkar, and Vineeth Balasubramanian

In the European Conference on Computer Vision Workshop on "Adversarial Robustness In The Real World", 2022, Tel Aviv, Oct 2022

Abs HTML PDF

The current state-of-the-art adversarially robust models have more object-aligned attributions, and attributionally robust models have stabler attributions than adversarially trained models. However, these robust models’ attributions suffer from another stability-alignment tradeoff problem, i.e., these models’ attributions are either stable against explanation-based attacks like IFIA at the cost of relevance or have well object-aligned attributions but unstable against the explanation-based attacks. We propose a training strategy that enforces attribution alignment through teacher saliency within the robust attribution training framework to curb this tradeoff. Moreover, we also note that the current evaluation metrics for measuring the stability of the attribution maps do not consider the object alignment of the generated attribution map and propose new metrics that capture both facets of the attribution maps, i.e., stability and alignment.

2021

NeurIPS

Get Fooled for the Right Reason: Improving Adversarial Robustness through a Teacher-guided Curriculum Learning Approach

Anindya Sarkar, Anirban Sarkar, Sowrya Gali, and Vineeth Balasubramanian

In the 35th Neural Information Processing Systems, 2021, Vancouver, Dec 2021

Abs HTML PDF Code

Current SOTA adversarially robust models are mostly based on adversarial training (AT) and differ only by some regularizers either at inner maximization or outer minimization steps. Being repetitive in nature during the inner maximization step, they take a huge time to train. We propose a non-iterative method that enforces the following ideas during training. Attribution maps are more aligned to the actual object in the image for adversarially robust models compared to naturally trained models. Also, the allowed set of pixels to perturb an image (that changes model decision) should be restricted to the object pixels only, which reduces the attack strength by limiting the attack space. Our method achieves significant performance gains with a little extra effort (10-20%) over existing AT models and outperforms all other methods in terms of adversarial as well as natural accuracy. We have performed extensive experimentation with CIFAR-10, CIFAR-100, and TinyImageNet datasets and reported results against many popular strong adversarial attacks to prove the effectiveness of our method.
AAAI

Enhanced Regularizers for Attributional Robustness

Anindya Sarkar, Anirban Sarkar, and Vineeth Balasubramanian

In the 35th AAAI Conference on Artificial Intelligence, 2021, Vancouver, Feb 2021

Abs HTML PDF Code

Deep neural networks are the default choice of learning models for computer vision tasks. Extensive work has been carried out in recent years on explaining deep models for vision tasks such as classification. However, recent work has shown that it is possible for these models to produce substantially different attribution maps even when two very similar images are given to the network, raising serious questions about trustworthiness. To address this issue, we propose a robust attribution training strategy to improve attributional robustness of deep neural networks. Our method carefully analyzes the requirements for attributional robustness and introduces two new regularizers that preserve a model’s attribution map during attacks. Our method surpasses state-of-the-art attributional robustness methods by a margin of approximately 3% to 9% in terms of attribution robustness measures on several datasets including MNIST, FMNIST, Flower and GTSRB.

2020

ICANN

Enforcing Linearity in DNN succours Robustness and Adversarial Image Generation

Anindya Sarkar, and Raghu Iyengar

In the 29th International Conference on Artificial Neural Network, 2020, Slovakia, Sep 2020

Abs HTML PDF

Recent studies on the adversarial vulnerability of neural networks have shown that models trained with the objective of minimizing an upper bound on the worst-case loss over all possible adversarial perturbations improve robustness against adversarial attacks. Beside exploiting adversarial training framework, we show that by enforcing a Deep Neural Network (DNN) to be linear in transformed input and feature space improves robustness significantly. We also demonstrate that by augmenting the objective function with Local Lipschitz regularizer boost robustness of the model further. Our method outperforms most sophisticated adversarial training methods and achieves state of the art adversarial accuracy on MNIST, CIFAR10 and SVHN dataset. In this paper, we also propose a novel adversarial image generation method by leveraging Inverse Representation Learning and Linearity aspect of an adversarially trained deep neural network classifier.
ECCV Workshop

Inducing Semantic Grouping of Latent Concepts for Explanations: An Ante-Hoc Approach

Anirban Sarkar, Deepak Vijaykeerthy, Anindya Sarkar, and Vineeth Balasubramanian

In the ECCV Workshop, 2020, Glasgow, Sep 2020

Abs HTML PDF

Self-explainable deep models are devised to represent the hidden concepts in the dataset without requiring any posthoc explanation generation technique. We worked with one of such models motivated by explicitly representing the classifier function as a linear function and showed that by exploiting probabilistic latent and properly modifying different parts of the model can result better explanation as well as provide superior predictive performance. Apart from standard visualization techniques, we proposed a new technique which can strengthen human understanding towards hidden concepts. We also proposed a technique of using two different self-supervision techniques to extract meaningful concepts related to the type of self-supervision considered and achieved significant performance boost. The most important aspect of our method is that it works nicely in a low data regime and reach the desired accuracy in a few number of epochs. We reported exhaustive results with CIFAR10, CIFAR100 and AWA2 datasets to show effect of our method with moderate and relatively complex datasets
ICMLA

ODE guided Neural Data Augmentation Techniques for Time Series Data and its Benefits on Robustness

Anindya Sarkar, Anirudh Raj, and Raghu Iyengar

In the 19th International Conference on Machine Learning and Applications, 2020, Florida, Dec 2020

Abs HTML PDF

Exploring adversarial attack vectors and studying their effects on machine learning algorithms has been of interest to researchers. Deep neural networks working with time series data have received lesser interest compared to their image counterparts in this context. In a recent finding, it has been revealed that current state-of-the-art deep learning time series classifiers are vulnerable to adversarial attacks. In this paper, we introduce two local gradient based and one spectral density based time series data augmentation techniques. We show that a model trained with data obtained using our techniques obtains state-of-the-art classification accuracy on various time series benchmarks. In addition, it improves the robustness of the model against some of the most common corruption techniques,such as Fast Gradient Sign Method (FGSM) and Basic Iterative Method (BIM).

2019

NLPIR

Zero-Shot Multilingual Sentiment Analysis using Hierarchical Attentive Network and BERT

Anindya Sarkar, Sujeeth Reddy, and Raghu Iyengar

In the 3rd International Conference on Natural Language Processing and Information Retrieval, 2019, Tokushima, Jun 2019

Abs HTML PDF Code

Sentiment analysis is considered an important downstream task in language modelling. We propose Hierarchical Attentive Network using BERT for document sentiment classification. We further showed that importing representation from Multiplicative LSTM model in our architecture results in faster convergence. We then propose a method to build a sentiment classifier for a language in which we have no labelled sentiment data. We exploit the possible semantic invariance across languages in the context of sentiment to achieve this.
Neuroinformatics

Curated model development using NEUROiD: A web-based NEUROmotor integration and Design platform

Anindya Sarkar

In the Frontiers in Neuroinformatics, 2019, Jun 2019

Abs HTML PDF

The explosive growth in imaging technologies has been matched by an extraordinary increase in the number of investigations focusing on the structural and functional organization of the human spinal cord. The construction of comprehensive spinal cord atlases and databases of 3-dimensional spinal cord maps are very helpful in the field of neurological treatment and surgical analytics. Some of the notable ones among these Atlases are MRI Based Atlases,Cryoimaging Atlases,Multi-modality Atlases etc. But these Atlases are passive images that show the distribution of different anatomical or functional aspects. We aim to build a neurophysiology simulator of the relevant circuits in 3D anatomical coordinates that embody the biophysical knowledge. After superimposing this simulator on available Atlases or functional imagery like MRI.A clinician can ask questions on the functional implication of a procedure or the potential etiologies for a manifest condition. Further a systems analysis of the connectivities can provide clues to identification of network motifs responsible for oscillations and pattern generators. Towards this end we are making a realistic 3 dimensional model of a human spinal cord using NeuroConstruct and NEURON Simulation environment. The construction of such a 3D model involves identification of neuron biophysics, connectivities,cell types,synaptic Mechanisms, Channel Mechanisms,Morphology, Dendritic Architectures,locations.