Value LABS#
Overview of the Value labs for the current semester for Foundations of Data Science Course at UVA.
Lab 1 - Guess Who#
DS 1001 - Spring 2023 - Professors Wright and Alonzi Due: End of lab period (or later that day) Submission format: Word doc or PDF summarizing your findings
Individual Assignment
General Descripition: This lab is design for you to reflect on the process used for winning the game Guess Who. Students should work in teams to play the game and take notes on the processes that work best for quickly guessing the opponents card. This will likely include the creation of a tree based decision diagram, a example will be provided in class.
Preparatory Assignments - None
Why am I doing this? In order to allow you to reflect on how data is used to generated decision/predictive algorithms and what issues could arise as a result. Work to incorporate the materials you’ve been exposed to up to this point as you work through the lab.
What am I going to do? You are working through a interactive process with your group to find the optimal list of yes/no questions for any given card. Play the game as many times as necessary to construct a list of the top five questions using a tree based diagram.
Answer these questions:
In this situation what is the dataset and what is the algorithm?
How did your approach to identifying cards change throughout the lab period?
Are there cards that are easier to identify, why?
This is just a game, but given the materials you’ve been exposed to what concerns do you have about this process and algorithmic decision making?
Tips for success:
Don’t worry about winning
Work as a team with your group
Try to document the process used right at the start of the lab, don’t wait till after you have played to start taking notes
How will I know I have succeeded
Specs Category |
Specs Details |
|---|---|
Formatting |
- Submit via Canvas |
Text |
- Goal: The questions are designed to be answered during or right after the lab period |
Acknowledgements: Special thanks for Jess Taggart from UVA CTE for coaching us. This structure is pulled directory from Steifer & Palmer (2020).
Lab 2 - AI Fairness 360#
DS 1001 - Spring 2023 - Professors Wright and Alonzi Due: End of lab period (or later that day) Submission format: Word doc or PDF summarizing your findings
Individual Assignment
General Descripition: This lab is designed for you to get exposure to AI fairness approaches in a no code environment on the website AI Fairness 360. You will be able to work through the various fairness methods at different stages of the pipeline and reflect on which methods seem to work the best on the given datasets.
Preparatory Assignments - None
Why am I doing this? In order to give you exposure to and practice with the various methods being developed and deployed in the ML fairness space. After completing the lab you’ll have a better sense of how these tools are used, when they are used and how the work.
What am I going to do? The AI Fairness 360 website has a demo module that includes three datasets. Work through the demo on all three datasets, trying all the methods provided, and answer the questions below.
Answer these questions:
For each protected class variable which evaluation methods showed bias to be present?
Note how each method preform at removing bias.
Was the accuracy of the model effected when using the various approaches, if so how?
Given the above what are some patterns you noticed, which methods seem to work the best, where in the data process are these methods located (pre/in/post).
Tips for success:
Take careful notes as you go through each method
Have fun
How will I know I have succeeded:
Specs Category |
Specs Details |
|---|---|
Formatting |
- Submit via Canvas |
Text |
- Goal: the questions are designed to be answered during or right after the lab period. |
Acknowledgements: Special thanks for Jess Taggart from UVA CTE for coaching us. This structure is pulled directory from Steifer & Palmer (2020).
Lab 3 - Explainable AI Interview#
DS 1001 - Spring 2023 - Professors Wright and Alonzi Due: End of lab period (or later that day) Submission format: Word doc or PDF summarizing your findings
Individual Assignment
General Descripition: This lab is designed for you to verbalize the concept of Explainable AI to someone that is unfamiliar with the concepts and track their responses.
Preparatory Assignments - None
Why am I doing this? This will allow you to work on communicating the concepts thus understanding them better and judging whether trust is increased with the knowledge that explainable approach to AI models are present.
What am I going to do? Interview three people that have somewhat limit knowledge of Machine Learning and AI. Ask the first two questions then provide a brief overview of Explainable AI and then ask the remaining three questions. After all three interviews are complete provide a brief summary of what patterns you notice (a paragraph or two is fine)
Answer these questions:
Do you believe that AI has a large influence over society currently?
On a scale from 1 to 10, with 10 being total trust and 1 being no trust: Do you believe that AI models can be trusted and used in a way that promotes positive outcomes for individuals and society?
Give a brief overview of the Data Ethic concepts with a emphasis on Explainable AI models.
Given what you just heard does that in anyway change your answer to question 2, such that, if explainable AI models were more commonly used would you be more likely to believe that they can promote positive outcomes for individuals and society? (scale of 1 to 10 for you belief the AI models can promote positive outcomes)
Are you interested in learning more about Explainable AI as a concept?
Tips for success:
Keep your over of data Ethics and Explainable AI at a high level and use the slides provided in class as needed.
Feel free to improvise and ask additional questions if you’d like or change the questions slightly.
How will I know I have succeeded:
Specs Category |
Specs Details |
|---|---|
Formatting |
- Submit via Canvas |
Text |
- Goal: the interview is designed to be fairly quick, 10 minutes or so for each. All together the assignment should take approximately one hour. |
Acknowledgements: Special thanks for Jess Taggart from UVA CTE for coaching us. This structure is pulled directory from Steifer & Palmer (2020).