ArgMap

On this page

  • Publications
  • Overall Research Questions
  • Methodology
  • Documents
  • View source

Argument Mapping to Guide Policy Decisions

How can LLMs enable us to ingest massive streams of unstructured information, incorporate diverse perspectives and distill them into actionable insights that align with public opinion?

This study focuses specifically on data from publicly hosted online discourse platforms such as Polis or Kialo that are geared towards specific topics or events.

Publications

  • Moderating Democratic Discourse with LLMs, SBP-BRiMS (September 2024)
  • Using LLMs to Structure and Visualize Policy Discourse, DemocrAI workshop at IJCAI 2024 (August 2024)
  • Advancing Policy Insights: Opinion Data Analysis and Discourse Structuring Using LLMs. Graduate Thesis and Dissertation 2023-2024. University of Central Florida. (May 2024)

Overall Research Questions

  • How effectively can language models structure and enable access to large amounts of opinion data?

This involves investigating how these technologies can be leveraged for topic modeling, measuring consensus or discord among opinions, and generating executive summaries and visualizations of the opinion landscape.

  • What metrics and insights can we generate from embeddings?

Text embeddings are numerical representations of text produced by transformer models that capture semantic meaning of textual data. This research leverages to cluster statements and opinions, identify outliers, and facilitate topic modeling.

  • What are the inherent risks associated with the deployment of language models?

This work aims to address concerns related to data bias, ethical implications, the potential for misinformation, and the overall integrity of AI-driven systems in shaping public policy. While language models offer transformative capabilities in analyzing vast, unstructured datasets, they also carry the potential to skew public discourse or influence policy decisions in unintended ways. Addressing these risks is crucial for the responsible and equitable use of AI in democratic deliberation.

Methodology

---
title: "Methodology: Content Processing Pipeline"
---
flowchart LR
    DataIngestion ==> Embeddings
    DataIngestion ==> CommentModeration
    Embeddings ==> TopicModeling
    TopicModeling ==> Labeling
    TopicModeling ==> Structure
    CommentModeration ==> TopicModeling
    Structure & Labeling ==> Tree
    Tree == Agreement Scoring ==> Insights

    DataIngestion[Data Ingestion]
    CommentModeration[Comment \n Moderation]
    Labeling[Topic Label \n Generation]
    TopicModeling[Topic Modeling]
    Structure[Insight \n Generation]
    Tree[Argument Mapping]
    Insights[Actionable Insights]

Documents

Data Ingestion
Read Polis data, calculate embeddings, and store in Polars Dataframe
Sonny Bhatia
Mar 1, 2024

Comment Moderation Experiment
Polis Comment Moderation using Language Models
Sonny Bhatia
Feb 29, 2024

Comment Moderation Analysis
Discussion of Experiment Results using Mixtral 8x7B v0.1
Sonny Bhatia
Mar 29, 2024

Topic Modeling
Use BERTopic to analyze comments, separate them into clusters and assign topic labels
Sonny Bhatia
Mar 12, 2024

Topic Titles using LLMs
Test various language models to generate appealing and representative topic titles
Sonny Bhatia
Feb 18, 2024

Generate Arguments
Turn each set of statements into actionable insights
Sonny Bhatia
Mar 16, 2024

Analyze Generated Arguments
Statistical analysis on generated arguments
Sonny Bhatia
Mar 17, 2024

Create Argument Maps
Generate argument maps for visual representation of generated insights
Sonny Bhatia
Mar 20, 2024

Advancing Policy Insights
Thesis Defense Presentation
Aaditya (Sonny) Bhatia
Apr 3, 2024

Using LLMs to Structure and Visualize Policy Discourse
IJCAI DemocrAI 2024 Presentation
Aaditya (Sonny) Bhatia
Aug 4, 2024

Moderating Democratic Discourse
SBP-BRiMS 2024 Presentation
Aaditya (Sonny) Bhatia
Sep 18, 2024
No matching items
 

© 2024 Aaditya Bhatia

  • View source