Assignments

Assignment 1: Detecting and Mitigating AI Bias

What do I do?

It's a two-part assignment. First, you will use our interactive platform to explore, implement, and inspect methods for detecting, scoring, and mitigating AI bias. Second, you will reflect on your activities through answering a series of discussion questions below.

Discussion Questions

Please answer the following questions after you complete the exploration and implementation through the platform above. Make sure to cite any external sources when you refer to examples, ideas, and quotes to support your arguments.

1. Explore the raw tweets before proceeding any further. Any notable, interesting, or worrisome observations?
2. Let's say we released our logistic regression model trained with the given dataset without any debiasing technique to perform content moderation on the live Twitter platform.
   (1) Do you think it is socially, culturally, and ethically acceptable? Would there be any potential problems?
   (2) For the problems you identified, what would be potential solutions to mitigate them? Don’t limit yourself to technical solutions.
3. The datasets are highly skewed. The dataset1 contains 77% of "Offensive" tweets, but the dataset2 contains 34% of "Hateful" tweets.
   (1) The way we construct a dataset significantly affects the distribution of labels. Why do you think such skew occurs in the datasets provided? You may want to refer to the data collection methods described in the paper (dataset1, dataset2).
   (2) How does the skewness need to be handled? Is it an ultimate goal to reduce such skew and balance all the labels? Then how can we determine whether the current dataset is balanced enough?
   (3) Let's say you are going to collect the dataset again to train the logistic regression model. How would you design the data collection pipeline? Explain the rationale.
4. Do you think the pipeline we used to train the model (i.e. data preprocessing and the model selection) contributed to the biased result? If so, why do you think so? Note that we have used n-grams (n <= 3) as the vocabulary and TF-IDF as the feature.
5. The trained logistic regression model performed pretty well with respect to the accuracy measure (weighted avg of F1-score), although the model gives a racially biased prediction result. One might say that model performance was enough to be deployed and conclude that AAE is more hateful and abusive than SAE. Do you think this is a reasonable conclusion? Please discuss your thoughts.
6. Have you observed anything on the relationship between the model performance and bias measure, even if it is not significant in our assignment? If so, share your opinions on why such a relationship could exist.
7. We explored two debiasing techniques, undersampling and post-processing.
   (1) Why do you think undersampling reduced a measure of bias? What are the cases it might not work well?
   (2) Why do you think post-processing reduces a measure of bias?What are the cases it might not work well?
8. The bias score on the "Hateful" label increased whereas the score on "Offensive" decreased. Why do you think it happened? What do you think needs to be done to achieve improvements on the measures on both labels?
9. Is debiasing always necessary? When do we need debiasing? Are there cases where debiasing might not be desired?

Any useful resources?

The following papers should be useful to get a sense of the background, methodology, and technical approaches we will be using.

• Davidson, Thomas, Debasmita Bhattacharya, and Ingmar Weber. "Racial bias in hate speech and abusive language detection datasets." arXiv preprint arXiv:1905.12516 (2019).
• Dixon, Lucas, et al. "Measuring and mitigating unintended bias in text classification." Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society. 2018.
• Sap, Maarten, et al. "The risk of racial bias in hate speech detection." Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019.

What do I submit?

You need to submit two things: (1) code with your implementation and (2) answers to the discussion questions. You do not need to explicitly submit your code as the server keeps track of your latest implementation. Answers to the discussion questions need to be written as a report in PDF. We highly encourage you to use resources from your Jupyter Notebook such as code, figures, and statistical results to support your arguments in the discussion.

Grading

• Code for bias detection (15%)
• Code for measure the bias score (15%)
• Code for bias mitigation (20%)
• Discussion (50%): further broken down to the following components
  • Completeness (10%): Are all the questions are answered with enough substance?
  • Depth (10%): Include thoughtful analysis beyond surface level observations.
  • Clarity (10%): The reader who sees the example for the first time should not struggle to understand your points.
  • Visual Communication (10%): Use various resources such as numbers, figures, charts, external data, and statistics from the interactive platform to effectively to communicate your idea.
  • Conciseness (10%): Avoid being verbose in the description.

Please make sure to read the academic integrity and collaboration policy of this course carefully.

How do I submit?

Make a single PDF file of your report and submit via KLMS. Code is automatically stored on the server with your latest version.

Assignment 2. Explaining Model Predictions to Users

Learning Objectives

Explainable AI helps users understand and interpret predictions produced by models. The objective of this assignment is for you to try existing off-the-shelf tools for explanations, think about strengths and weaknesses of the explanations they provide, and design your own user-centered explanations that can address such weaknesses.

Background

You will work with methods for explaining model predictions in image classification tasks. Such explanations help users resolve questions around what’s happening inside of the model and why. As users explore these explanations, they may come up with additional questions about the model, which possibly requires other kinds of explanations.

What should I do?

In this assignment, you are asked to (1) explore Google’s What-If Tool, a platform that helps users understand the performance of models, (2) build and run an algorithm based on LIME for presenting which parts of an image contribute to the prediction for better interpretation of classification results, and (3) design a UI that further helps users interpret the results especially when such explanation is not enough. For each of the stages, you are asked to discuss what can be explained with such tools/methods, and limitations of such explanations. For (2), we are going to use Google Colab, an environment for executing code with documentations, similar to Jupyter Notebook.

Instructions

1. Go to a What-If Tool demo about a smile classification task. You can explore the dataset, browse the model performance, and experiment with the model by asking what-if questions to the tool.
2. Answer questions in Discussion “Stage 1” below. The discussion contains a specific task that you need to perform on the What-If tool.
3. Go to the Colab notebook, which contains skeleton code that describes an algorithm that explains which parts of an image contribute to the prediction result. You need to fill out some blanks in the code with your implementation to make it work.
1. Go to “File → Save a copy in Drive” so that your code is maintained in your personal Google drive. You need to have a Google account for this.
2. Implement the empty parts in the skeleton code.
4. Answer questions in Discussion “Stage 2” below.
5. Create your own prototype of an interactive explainable UI [examples] to help users better understand the prediction results from Stage 2. Please follow the steps below:
   1. Prepare an example where the prediction result and LIME explanation from Stage 2 is hard to understand, e.g., “I cannot judge if referring to the shape of the ears would lead to a correct prediction, so I need to see more examples for comparison.”
   2. Come up with dimensions and functionalities that you think would be helpful to provide a better explanation in the case you identified in Step a, e.g., data points’ feature editor, feature statistics visualization, ROC curve visualization, a chatbot for explanations, etc.
   3. Use Figma to build a prototype of a web-based UI like Google’s What-If Tool that interactively shows all the information you listed in Step b. Make it interactive so that users can proactively seek satisfying explanations. For example, users may want to query additional information, request clarifications, edit data points to test a hypothesis, or give feedback to improve model performance. Your prototype doesn’t need to be fully interactive with every feature you add, e.g.,”login” button or “upload new image” button doesn’t need to be interactive. Please focus on the core interactive and explainable concept you’re introducing.
6. Answer questions in Discussion “Stage 3” below. You should include the URL of your prototype in the report.

Discussion

Stage 1. (What-If Tool)

What-If Tool consists of three tabs: Datapoint editor, Performance & Fairness, and Features. Each tab represents different aspects of the model and results.

Datapoint editor tab

• 1-1. Choose an image on the right plane, and modify a feature value on the left. What happens? How can the Datapoint editor be used as an explanation? Do you have any interesting observations?
• 1-2. You can set X-Axis, Y-Axis, Color By, and others on the top bar. Fix X-Axis to “Inference correct” and explore the visualization updates by changing other dimensions. What happens? How can this be used as an explanation? Do you have any interesting observations? You can try various combinations and explore how the visualization updates.

Performance & Fairness tab

• 1-3. It shows the overall performance of the model. How can this be used as an explanation? Do you have any interesting observations?
• 1-4. In the Configure tab on the left, fix “Ground Truth Feature” to “Smiling” and select a feature on “Slice By”. What happens? How can this be used as an explanation? Do you have any interesting observations?

Strength and Limitation of the tool

• 1-5. What are the strengths of this tool as an explanation?
• 1-6. What are the weaknesses of this tool as an explanation?

Stage 2. (LIME explainer)

• 2-1. Report the result of Activity 2, which asks you to compare explanations of multiple images. Attach code and images if necessary.
• 2-2. How does the generated explanation, which presents a set of important superpixels, help users understand the model predictions? Does it complement performance measures such as accuracy and F1-score? If so, how?
• 2-3. In Activity 2, did you find any image for which the algorithm does not give an explanation that is easy to understand for users? Why do you think the algorithm gives such an explanation?
• 2-4. Is our explanation sufficient to trust the model predictions? When is it sufficient? When is it not sufficient? What kinds of additional information would you include in your explanation?
• 2-5. Are explanations always necessary in order to understand the model performance? When do we need it? When do we not need it?

Stage 3. (Figma prototype)

• 3-1. List a few representative questions that your UI can possibly answer. Show us a walkthrough of how to resolve the questions. You may want to add screenshots of your UI.

Any useful resources?

The following resources should be useful for you in getting a sense of the background, methodology, and technical approaches we will be using.

1. What-If Tool: https://pair-code.github.io/what-if-tool/
2. Ribeiro et al., “"Why Should I Trust You?": Explaining the Predictions of Any Classifier”, Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. 2016.

What do I submit?

You need to submit two things: (1) your .ipynb file exported from the CoLab notebook and (2) .pdf file that answers to the discussion questions. Note that you need to explicitly submit your code at this time. We highly encourage you to use resources such as code, figures, and statistical results to support your arguments in the discussion.

Grading

• Stage 2 Implementation (50%)
  • Activity 1 / Task 1: Code for creating perturbed data (20%)
  • Activity 1 / Task 2: Code for fitting a linear model (15%)
  • Activity 1 / Task 3: Code for feature selection (15%)
• Discussion (50%): further broken down to the following components
  • Completeness (10%): Are all the questions are answered with enough substance?
  • Depth (10%): Include thoughtful analysis beyond surface level observations.
  • Clarity (10%): The reader who sees the example for the first time should not struggle to understand your points.
  • Visual Communication (10%): Use various resources such as numbers, figures, charts, external data, and statistics from the Google What-If Tool and the Colab code to effectively communicate your idea.
  • Conciseness (10%): Avoid being verbose in the description.

Please make sure to read the academic integrity and collaboration policy of this course carefully.

How do I submit?

Submit your code (.ipynb file) and report (a PDF file) via KLMS. Make sure that your report should include the URL of your prototype.