2024

Rating Sentiment Analysis Systems for Bias Through a Causal Lens
Authors: Kausik Lakkaraju, Biplav Srivastava, Marco Valtorta
Summary:
Sentiment Analysis Systems (SASs) analyze text emotions but can inaccurately change ratings over minor input variations, showing potential bias towards attributes like gender or race. We propose a method to evaluate and rate SASs on their sensitivity to these attributes, aiming to help choose more fair and reliable systems and reduce bias-induced hate speech online.
Publication Type: Journal
Venue: IEEE Transactions on Technology and Society
Paper |Bibtex

2023

Trust and ethical considerations in a multi-modal, explainable AI-driven chatbot tutoring system: The case of collaboratively solving Rubik's Cube
Authors: Kausik Lakkaraju, Vedant Khandelwal, Biplav Srivastava, Forest Agostinelli, Hengtao Tang, Prathamjeet Singh, Dezhi Wu, Matt Irvin, Ashish Kundu
Summary:
AI can revolutionize education by analyzing vast data on student learning but faces unresolved ethical concerns, such as data privacy and fairness, especially in high school settings. This paper introduces the ALLURE chatbot, a platform designed to address these ethical issues, allowing students to collaboratively solve the Rubik's cube with AI. Key features include prioritizing informed consent for data use and ensuring safe interaction and language use to protect students. It also focuses on preventing information leakage between user groups as the system learns and improves.
Publication Type: Workshop
Venue: ICML Workshop on What’s left to TEACH (Trustworthy, Enhanced, Adaptable, Capable and Human-centric) chatbots?
Paper |Bibtex

Advances in Automatically Rating the Trustworthiness of Text Processing Services
Authors: Biplav Srivastava, Kausik Lakkaraju, Mariana Bernagozzi, Marco Valtorta
Summary:
In this symposium paper, we talked about the previous approaches that were used to rate the trustworthiness of AI systems and we also outlined the challenges and vision for a principled, causality-based, and multi-modal rating methodologies.
Publication Type: Journal, Symposium
Venue: AI and Ethics Journal; AAAI Spring Symposium
Paper |Bibtex

LLMs for Financial Advisement: A Fairness and Efficacy Study in Personal Decision Making
Authors: Kausik Lakkaraju, Sara E Jones, Sai Krishna Revanth Vuruma, Vishal Pallagani, Bharath C Muppasani, Biplav Srivastava
Summary:
We compared ChatGPT and Bard, LLM-based chatbots, with SafeFinance, a rule-based chatbot, in the personal finance domain. Our findings reveal that ChatGPT and Bard often provide inconsistent and unreliable financial advice, while SafeFinance, though simpler, offers dependable and accurate information. This study highlights the current limitations of LLM-based chatbots in handling financial advisement tasks effectively.
Publication Type: Conference
Venue: Proceedings of the Fourth ACM International Conference on AI in Finance
Paper |Bibtex

The Effect of Human v/s Synthetic Test Data and Round-tripping on Assessment of Sentiment Analysis Systems for Bias
Authors: Kausik Lakkaraju, Aniket Gupta, Biplav Srivastava, Marco Valtorta, Dezhi Wu
Summary:
Sentiment Analysis Systems (SASs), AI tools that analyze text sentiment, can show unstable and biased behavior, raising trust issues. A new method rates these systems for bias using synthetic data. We enhanced this by using real chatbot conversations and a technique that translates data through another language and back. This revealed more bias in real compared to synthetic data, but translating through Spanish or Danish reduced bias significantly in real data.
Publication Type: Conference
Venue: The Fifth IEEE International Conference on Trust, Privacy and Security in Intelligent Systems, and Applications
Paper |Bibtex

Evaluating Chatbots to Promote Users' Trust -- Practices and Open Problems
Authors: Biplav Srivastava, Kausik Lakkaraju, Tarmo Koppel, Vignesh Narayanan, Ashish Kundu, Sachindra Joshi
Summary:
Chatbots have gained widespread attention, especially with the advent of LLM-based systems like ChatGPT and Bard. As they become integral in business for engaging with customers, suppliers, and employees, ensuring their reliability through thorough testing is crucial. This paper examines how chatbots are currently tested, highlights the challenges in building user trust, and proposes directions for future research and development.
Publication Type: Unpublished Manuscript
Paper |Bibtex

Can LLMs be Good Financial Advisors?: An Initial Study in Personal Decision Making for Optimized Outcomes
Authors: Kausik Lakkaraju, Sai Krishna Revanth Vuruma, Vishal Pallagani, Bharath Muppasani, Biplav Srivastava
Summary:
We tested advanced chatbots like ChatGPT and Bard on personal finance advice, using 13 questions in different languages and dialects. Although the chatbots' answers sounded good, we found they often lacked accuracy and reliability in providing financial information.
Publication Type: Workshop
Venue: ICAPS Workshop on Planning for Financial Services
Paper |Bibtex

On Safe and Usable Chatbots for Promoting Voter Participation.
Authors: Bharath Muppasani, Vishal Pallagani, Kausik Lakkaraju, Shuge Lei, Biplav Srivastava, Brett Robertson, Andrea Hickerson, Vignesh Narayanan
Summary:
We created chatbots to help increase voting among seniors and first-time voters by giving them easy access to trusted election information tailored to their needs. Our system, built on the Rasa platform, ensures the information is reliable and allows for quick chatbot setup for any region. We've tested these chatbots in two US states where voting has been difficult, focusing on groups of senior citizens. This project aims to support voters and democracy by making accurate election information more accessible.
Publication Type: Workshop
Venue: AAAI Workshop on AI for Credible Elections
Paper |Bibtex

2022

Why is my System Biased?: Rating of AI Systems through a Causal Lens
Authors: Kausik Lakkaraju
Summary:
This is a student paper which formulates my PhD dissertation problem and gives an overview of the solution. Idea is to evaluate / rate AI systems for bias using causal analysis.
Publication Type: Doctoral Consortium
Venue: AIES
Paper |Bibtex

ALLURE: A Multi-Modal Guided Environment for Helping Children Learn to Solve a Rubik’s Cube with Automatic Solving and Interactive Explanations
Authors: Kausik Lakkaraju, Thahimum Hassan, Vedant Khandelwal, Prathamjeet Singh, Cassidy Bradley, Ronak Shah, Forest Agostinelli, Biplav Srivastava, Dezhi Wu
Summary:
ALLURE is a Deep Reinforcement Learning based, multi-modal, explainable chatbot which teaches children how to solve a Rubik’s Cube and allows the children to interact with the multi-modal chatbot while trying to solve the Cube.
Publication Type: Demonstration
Venue: AAAI
Paper |Bibtex |Video

Data-Based Insights for the Masses: Scaling Natural Language Querying to Middleware Data
Authors: Lakkaraju Kausik, Palaiya Vinamra, Paladi Sai Teja, Appajigowda Chinmayi, Srivastava Biplav, Johri Lokesh
Summary:
This is a demonstration paper which talks about a RASA-based chatbot that allows users to control their network usage and bandwith using smart routers in a household or office setting. We also demonstrated another chatbot in the same paper which helps users in monitoring the power usage in a house, office or university setting using smart sensors. These were deployed on Alexa device and Web for demonstration.
Publication Type: Demonstration
Venue: DASFAA
Paper |Bibtex |Video

A Rich Recipe Representation as Plan to Support Expressive Multi-Modal Queries on Recipe Content and Preparation Process.
Authors: Vishal Pallagani, Priyadharsini Ramamurthy, Vedant Khandelwal, Revathy Venkataramanan, Kausik Lakkaraju, Sathyanarayanan N Aakur, Biplav Srivastava
Summary:
In this paper, we discussed the construction of machine-understandable rich recipe representation (R3), in the form of plans, from the recipes available in natural language. R3 is infused with additional knowledge like allergens and possible failures at each cooking step.
Publication Type: Workshop
Venue: ICAPS Workshop on Knowledge Engineering for Planning and Scheduling
Paper |Bibtex |Video

Explainable Pathfinding for Inscrutable Planners with Inductive Logic Programming
Authors: Rojina Panta, Forest Agostinelli, Vedant Khandelwal, Biplav Srivastava, Bharath Chandra Muppasani, Kausik Lakkaraju, Dezhi Wu
Summary:
By combining inductive logic programming (ILP) with a given inscrutable planner, we constructed an explainable graph representing solutions to all states in the state space. This graph can then be summarized using a variety of methods such as hierarchical representations or simple if/else rules. We tested our approach on Towers of Hanoi.
Publication Type: Workshop
Venue: ICAPS Workshop on Explainable AI Planning
Paper |Bibtex |Video

ROSE: Tool and Data ResOurces to Explore the Instability of SEntiment Analysis Systems
Authors: Gaurav Mundada, Kausik Lakkaraju, Biplav Srivastava
Summary:
ROSE is a tool that helps examine gender bias in Sentiment Analysis Systems (SASs), which score text for sentiment and emotion. It offers a dataset of text inputs with their sentiment scores and a visualization tool for analyzing SAS behavior towards gender. Developed with d3.js, ROSE is freely accessible for public use.
Publication Type: Unpublished Manuscript
Paper |BibTex |Tool