Ethics & AI

00:00:00 good afternoon everyone welcome to the helix center's very first um zoom round table of course you all appreciate we've been forced uh online by the ongoing corona pandemic and we're looking forward soon to having ourselves back in our normal venue but until then we're we're being we're adapting as best we can and uh here we are with our first fall 2020 uh roundtable on ethics nai um i'm joined here with uh the helix

00:00:32 executive director ed nercessian i'm jerry hurwitz the associate director and today we have an esteemed panel of uh experts in uh and i'll read their bios to you in just a moment but uh we're looking forward to having a really wonderful robust conversation about this very important topic um let me just say a word about some of our participants uh the the uh our sort of acting moderator today is brandon fiddleson

00:01:04 and uh brandon is a distinguished professor of philosophy at northeastern university before teaching at northeastern brandon had held teaching positions at rutgers university of california berkeley san jose state and stanford and visiting positions at the munich center for mathematical philosophy gabrielle johnson is an assistant professor of philosophy at claremont mckenna college

00:01:35 before joining claremont mckenna she was a bursoff faculty fellow at nyu affiliated with the center for mind brain and consciousness she works primarily in philosophy of psychology philosophy of cognitive science philosophy of science and philosophy of technology her projects explore the nature and structure of social bias as it occurs in computational systems including the visual perceptual system sociocognitive systems scientific inference and predictive

00:02:05 models and machine learning programs rayed ghani is a distinguished career professor in the machine learning department and the hinds college of public policy at carnegie mellon university raiders of reformed computer scientist and a want to be social science scientist but most mostly just wants to increase the use of large-scale ai machine learning data science in collaboratively solving large public policies and social challenges in a fair

00:02:36 and equitable manner reid works with the government and non-profits across policy areas including health criminal justice education public safety economic development and urban infrastructure right is also passionate about teaching and practical data and science and started the data science for social good in fellowship tracy mears is the walton hale hamilton professor and a founding director of the justice collaborative at yale law school

00:03:06 before bringing the fact before joining the faculty at yale she was a professor at the university of chicago law school from 1995 to 2007 serving as max pam professor and director of the center for studies in criminal justice she was the first african-american woman to be granted tenure at both law schools professor miers is a nationally recognized expert on policing in urban communities for research focuses on understanding how members of the public think about their relationships with legal authorities such as police

00:03:38 prosecutors and judges tina eliasirad is an associate professor of computer science at northeastern university in boston massachusetts she is also a core faculty member at northeastern university's network science institute prior to joining northeastern tina was an associate professor of computer science at rutgers university and before that she was a member of the technical staff and principal

00:04:09 investigator at lawrence livermore livermore national laboratory tina earned her phd in computer science at the university of wisconsin-madison her research is rooted in data mining and machine learning and spans theory algorithms and applications of big data from networked representations of physical and social phenomena she's had over 100 peer review publications including a few best papers and best paper runner-up awards and has given over 200 invited talks and 14 tutorials so with all that

00:04:42 i'm handing it over now to brandon fiddleson who will get discussion underway thanks jerry i just want to start by saying thanks to both ed and jerry and to the helix center it's always a pleasure to do these round tables i hope we can get back to doing them in person in new york city next year and i want to welcome everybody both on the webinar here and also on youtube in the live stream i want to remind people that we you can start putting questions into the q a tab

00:05:13 and zoom if you're on zoom or uh you can write in some comments and questions in the live stream on on youtube and those are going to be compiled throughout the course for the first part of the the rounds table and then after we get done with this first part which i'll say a little bit more in a second we're going to have a q a at the end um so feel free to start entering questions and comments as you have them all right so the plan rough plan for today such as it is is roughly three parts about 30 minutes each where i'm going to

00:05:45 ask the panel about first problems and pitfalls pertaining to the current use of ai technology and some of the some of the moral problems that it currently faces in its applications now and then we'll move on to um potential solutions to some of those problems and finally i want to end on an up note i want to end on a positive note here if we can uh to say something about how we might harness these powerful technologies for good for morally good outcomes in the future

00:06:16 all right without further ado starting with pitfalls and problems um i'm going to start i'm going to ask tracy to go ahead and tell us some of her thoughts uh about pitfalls and problems that she's seen and in her her work she's addressing tracy good afternoon everybody um as i was listening to the bios of my co-panelists i thought wow one of these things is not like the other and that would be me most of my work has focused

00:06:46 on thinking about problems in the criminal legal system although lately this my center of the justice laboratory is branched out to think about problems of social media governance which one might think is more directly related to what we're doing but in the spirit of getting as many things on the table as possible and at the beginning especially in terms of articulating problems i guess i would say when i think about um ai

00:07:17 big data you know machine learning and automation um in the context of criminal legal processing we're faced with some serious issues about the difference between what we think about the fairness of how humans make decisions about outcomes especially penal outcomes for individuals and how we think about machines doing this there are many people who think that machines engaging in these processes

00:07:48 make it fairer because they think that machines themselves aren't impacted by bias that may or may not be true it depends on you know how you think about how machines learn but certainly the crudest form of how this works ultimately or initially depends on how the machines are programmed i'm not saying anything that anyone doesn't know already about thinking about risk

00:08:19 assessments and who has denied bail and the extent to which one can make those algorithms fair just one point on that then i'll say something about social media one point is to say to the extent that people point to all of those problems with machine learning learning again focused on you know the issues inherent in having a human uh program uh these machines um i think folks too often forget of

00:08:51 course that the human beings making these decisions um which would be the alternative it's always compared to what um of course have algorithms in their head um and they always do it it's always a when i'm at conferences like this not this but conferences discussing bail reform and risk assessment i will always say well what about the algorithm and the judge's head because the critic always seems to posit the human alternative

00:09:22 as better um now it might be that they both just to use my kids terms suck and so the question is which sucks the least um and how we think about which sucks the least which gets to my second point around um social media governance one way in which humans may suck less is the uh is the ways in which we uh can be forced in a sense to explain what we do there's ways in which machines and

00:09:53 algorithms in particular are especially inscrutable even if we can make them transparent they're still inscrutable in all the ways in which humans just have trouble at some level at the capacity making the assessments even if i can explain to you in words what the what the algorithm is doing and so in my social media work and then i'll end with this because brandon said i wasn't supposed to talk much more than about five minutes so it's table setting

00:10:23 one of the things we've tried to do at the collaboratory and the work on social media governance is to try to apply learnings from the social psychology of procedural justice and legitimacy in the real world mostly in the criminal legal system context to how platforms for online interaction manage problems disputes content moderation and so on and we've been relatively successful working with a number of platforms

00:10:53 to have them understand how they can use their data science to infuse that science with these ideas that um procedural justice um points to um most i'll i'll just say one more thing and we'll wait for the uh you know the next session but the key thing to understand about how procedural justice informs this is that it focuses much more on process than it poses focuses on outcome so to the extent that when people are

00:11:24 trying to make these outcomes fair through machine learning the procedural justice approach would say don't worry about the outcome as much focus on the process by which you reach those outcomes and then that the key would be is there a way in which we can have ai focus on these processes in ways that are transparent and sit transparent and salient to humans

00:11:56 thanks so much tracy um i think a natural this is a natural segue into asking gabby just because gabby's research it really is about the similarities and differences between uh biases that occur in human judgment and and uh automatic biases if you will so gabby take it away great thanks brandon yeah and thanks for the great intro tracy um so as brandon said i'm interested in cases of bias as they manifest both for human decision makers and for artificial decision makers um so

00:12:28 insofar as i'm highlighting various pitfalls and problems now i guess my hobby horses the biases that manifest in these decision-making procedures and more so than with the human decision-making procedures i think the issue is really bias under the guise of neutrality or scientific objectivity and so i think every computer scientist is taught this motto of garbage and garbage out the computer decision maker is only ever as good as the data going into it

00:12:59 and so i try to focus my area of research on two different focal points one is on the data going into the decision making procedure and how various systematic patterns of oppression could be encoded in that data um but the other decision or focal point for me is on the decision points of the algorithmic designer and i think both allow opportunities for bias to creep into what seem like objective or impartial decision making

00:13:29 patterns um so i think that uncovering the theory behind the data is something that philosophers of science have been worried about for a long time and that um various methodologies that are apparent in philosophy science could be useful in the domain of machine learning algorithms and so part of what i'm trying to do is to bring a better theory to understanding both the patterns that are encoded in the data but also the theory of how we take objective scientific inference to occur

00:14:00 more generally which is of course what we're trying to model in the case of algorithmic decision making um so just how human values get encoded in basically every single step of that procedure or that model is something that i think is that the four of those concerned with ethical ai um i also think as tracy said another issue is the lack of transparency um so we have a lack of transparency on two fronts one is that machine learning programs are proprietary as we all know and so because they're commercial

00:14:33 and um because they're commercial programs it seems like they are not going to be open to public scrutiny and even if they were the data that they're operating on would also cause issues of privacy concerns um so that their proprietary is one issue um but the second one and i think this is the one that tracy was intimating is that they likely manifest what we call black box algorithms that is even if we could as it were appear under the hood and take a closer look it's not obvious

00:15:05 that even a good computer scientist or philosopher would be able to understand exactly how the program is operating so the lack of transparency on those two fronts i think makes and compounds these issues of bias um and then the third thing that i'll just say uh as a a problem or pitfall is just the speed and ubiquit ubiquity with which um artificial intelligence and machine learning technology has proliferated uh way further than um

00:15:35 we anticipated at the rate that it has and so it has been allowed to grow basically um unchecked and unfettered uh throughout many aspects of human lives and so i think that the lack of accountability or mechanisms for ethical responsibility is a big issue for our current technology industry and so one of the major pitfalls that those of us interested in ethical ai need to approach and discuss

00:16:10 thanks gabby that that was great uh so having heard from the the legal and the philosophical sides of this um when it turned out to sort of the more practical the more more of the practitioner side um so right um maybe you could share some of your thoughts from the more practical side someone's sort of on the ground using the tools doing the stuff um yeah so i think before we sort of you know before at least we start talking about all the horrible things

00:16:41 you know i think the reason we're having this conversation i'm assuming is because we think there is some good that can be done right that that we have a lot of issues in this world and humans have been doing trying to fix these issues for a very long time but they've created these issues right it didn't it didn't just happen and and so i think if the premise is there is some potential in using computers and data and evidence to improve policies um and there have been a lot of evidence for that right we've done a lot of good things that

00:17:11 these are the risks that we're talking about and these risks are only worth talking about because there is that potential for good otherwise it would be very easy let's just stop not do any of these things um and and i think that comparison to sort of tracy's point is you know it has to always be compared to what right it's not compared to the perfection that we all desire but compared to what humans do today um and i think that's the point that i think we need to keep repeating which is why i'm repeating it because it gets lost in practice very often where the critics you know if you look

00:17:41 at any criticism of these types of tools it's often you know these machines they they're racist and and they don't justify what they do and you can replace the machines with humans and you it would be the same same paragraph uh so so i think but in my mind the bigger so given that the risks are pretty much the same as they have been with humans all this time any human decisions that are made have had these issues to me that the incremental risk that comes up is partially what sort of gabrielle talked about of how fast these tools are are spreading

00:18:13 but then also unfortunately the consistency in these tools right the the humans the hope is that you know yeah judges are a lot of them are racist but and they all make sort of horrible decisions generally but there's variance and each individual decisions combine you know it results in overall racism we've seen that but every single decision may not be racist but but if these systems let's say for criminal justice or allocating human services or health services if there are three such systems in the world that are being used

00:18:44 that variance goes away and if these three systems are bad then there is no hope for us to recover from that so i think that's for me it is an incremental risk apart from what everything we've talked about that that makes it worth really focusing on um and thinking through um second thing is again what what again both gabrielle and tracy mentioned was there is sort of this fake sense of trust as oh because it's based on data it must be right right and and yes you know the garbage

00:19:14 can garbage out garbage out but not garbage in doesn't mean not garbage out right you can put a lot of good data in um but the system can still result in horrible um outputs but more importantly i think at least the work i do is less on autonomous decision making of any sort it's assisting humans and making better decisions so you could have a perfectly fair ai system resulting in horrible outcomes because the human that takes that those recommendations does horrible things or vice versa a

00:19:46 horribly biased ai system given to a human who understands how it works and can use it to create more equitable outcomes and so i think that there is kind of you know this fake trust in in in data where data is never objective um there is no such thing as you know good data it's whatever you know sensors we use to collect that that information and how we use it um and the other risk i think is to connect to that is often people may

00:20:17 use these systems as an excuse to do what they would have done anyway uh and now it gives them another you know evidence that see uh i i this is what the computer told me to do and there have been many cases again in in the criminal justice system of of you know judges sort of claiming well i i didn't make this decision the computer told me to do that well then why do you why do we have you uh why not just have the computer take over uh so i think those are kind of some initial thoughts and you know we can go into those um later more fantastic

00:20:49 um that brings us to our uh final panelist uh athena who i know has thought a lot about how humans interact with systems especially when they serve as assistants and also on how to educate people so they become more technically literate um tina you want to take it yes thank you very much and uh one update to my bio i was promoted to full professor in july even though it sounds like 100 years ago now but thank you thank you um so from my perspective i teach a class

00:21:20 called algorithms that affect lives for freshmen for incoming freshmen and i don't require any math or programming and my whole goal for them is should we do this so this goes back to where rahid was saying just because we have the technology and somebody will pay for it should we do it as citizens of this country should we deploy these algorithms for xyz just because we can do it and somebody's willing to pay for it maybe not right so we should really

00:21:51 think about that as one aspect of it the other aspect of it is what he touched in is that at least from the computer science perspective there's this notion that data comes from gods and when you talk to for example physicists or more natural scientists no there's a distribution right and so which part of this distribution did you actually see right um it would be good for example to actually say you know what this algorithm only works on white men

00:22:23 who are between 25 and 35 but computer scientists typically are not that honest we are all about it's a master algorithm right my algorithm is the best algorithm on the planet at least for five minutes until the next paper is published right and knocks it down so these are some of the things that both in terms of basically putting doubt into the human in terms of that this algorithm is fabulous and also making them think that there are certain scenarios in which the algorithm is really being treated as an

00:22:53 expert witness right it's as if it's an expert witness but it's not really being treated like an expert witness right where like you can actually like audit the algorithm and ask you different questions it seems like it's a one-way street from the algorithm saying you know i think tina is going to default on this loan right and not being able to ask well why do you think tina is going to default on his loan right what if tina was a white male right and had the royal flush of hands would tina still default on their loan right what if

00:23:25 tina's zip code was not so on and so forth right so there's some of these aspects of it but i think in general just thinking about just because we can do it and somebody is willing to pay for it should we do it is it good for our society right so one of the aspects is automation for example and there is an element in terms of ai and ethics where robots and factories are taking away jobs maybe you know you can say oh that's productivity growth and that's a good thing but then on the other hand what are you

00:23:57 going to do with all these people who are unemployed do we not care about them or for example should we tax the companies who bring in robots into the factories and use that money to re-educate our population right and so there's some of these kinds of things that um i've been teaching the freshmen and they're amazing they are amazing and you know so i'm hopeful for the future um because they ask the right questions um and on that i'll stop well that's great because that see i

00:24:27 wanted this to be an increasing amount of optimism so i like the trend and i'm just going to reverse the order so tina keep going you're on a roll here solutions maybe what are some ideas for how to solve or how to address some of the x stamp problems i think you're already addressing some of that but maybe you could just take the next segment and then we'll just go in reverse order all right go for it i think some of it is a citizenry that actually knows what's happening like for example a lot of my students

00:24:57 ask well is there any way for me to figure out how much google knows about me right then perhaps i can make an informed decision as to whether i want to use all the google products whether i want to allow google in my life or for example many of them don't know about the privacy law that was passed in california where you can go to these tech companies and ask them about your data now of course you can pass laws but they may not have teeth right so there's that as as well

00:25:28 but there's also aspect of um just having algorithms have labels on them the same way prescription drugs have labels on them right because these algorithms adversely affect different populations so i need to know the risks and benefits and who the algorithm was for and was the algorithm audited and can i audit it what are the privacy risks and and rights so my colleague patricia williams always says what the hell does consent mean here when i consent

00:25:58 to apple to use their music their itunes what am i consenting to exactly right so there's this kind of stuff that we need to unpack for our population so that they know what they're getting themselves into i think that's one of the things that we don't know and we can do that right we do have the tools to do that and lastly um i think this was something that was mentioned before which is you can't just leave the algorithm designed to white men because then they just think about like other white men

00:26:28 so you really need representatives from other slices of the population to be in the room to have this what's called human valued or human-centered design to think about well how would this operate on you know somebody like me right um so those are some of the solutions and we can't do them it's just that you know why should i do it if somebody's already paying me a lot of money to just tell them you know tina's bad and brandon's good that's easier for me okay

00:27:00 can i ask but what about this issue where there's some value in a particular program that you're invited to use and it's so sexy and exciting and or and or useful and it's it's cheap to start up with and then the implications and the and the you know the acceptance that you're asked to grant it's uh a lot of people don't take the time where they sort of buy into it before they really evaluate all the risks they're taking what can we do about that is there

00:27:31 something yeah so i think that actually gets a very good question and that gets into a bigger question of what is the business model of these companies you know google is an advertising company whether you want it or not it is an advertising company should we change the uh the business model of them where you are willing to pay five dollars a month right uh where you pay them five dollars a month and they won't sell your data and they will stop being an advertising company now when i taught the same class in the spring

00:28:02 my students were like i am not willing to pay five dollars a month like i'm willing to pay 20 for a burrito but i'm not willing to pay five dollars a month now this semester i have students who are like no i'm willing to pay five dollars a month to google so that like they don't sell my data and so on and so forth now there's a bigger problem in that it's not just google that uses your data right your credit card company is selling your data right i'm sure you've heard these stories about like your friend may have moved from the west coast to the east coast

00:28:32 and uh amazon knows where they have moved before you tell your parents it's because there are all these data brokerage right um that collect and sell data so it's not just that you can like stop gap at one point right and this is also why you need basically a national um uh set of rules and regulations with teeth again like for example fda still has teeth i think it's because our country is so litigious like people have to get sued this goes back to i think it was uh gabby that mentioned

00:29:02 in terms of having somebody um who you say it's tina's fault right i can put up a software out there and bring half the country down i won't get sued like i am not like right i am not liable right microsoft is not liable for all the bugs that are out there right in their microsoft software so there's some of that going on here as well but you ask a very good question and that touches on changing the business model right awesome uh i'm gonna reverse order here

00:29:34 right you want to jump in on possible solutions or ways of addressing some of the current pitfalls regulation i guess that's the that's the easy answer connecting with rotina saying yes we can change the business model yes we can educate consumers uh you know it takes it it's it's it takes time and it may not happen but i think as you said fda ftc cfpb uh fec now pick your favorite uh

00:30:05 regulatory agency and i think that's the solution except they're not ready yet they don't know how to do it they're not trained they don't have the the expertise um and gao for example right like a couple weeks ago shockingly you know this country's ceo had a had a two-day event on how do we audit and govern ai systems um and it was actually pretty reasonable uh so so i think i think that's that's one path that's i don't think it's the only path um i think the some of the things that i'm

00:30:36 working on in sort of anything are worth pushing is is is coming in from both angles from what tracy talked about procedural um but but also outcome focus right i mean i think neither of them are enough um procedural is is today it's very hard to audit procedural uh things right it's just so complicated outcomes are a little bit easier to audit so that's a good starting point but that's not a long term you know we need both um and the reason i say outcomes instead of procedural is at least in the ai world you know a

00:31:08 lot of the focus in this area has been on fair algorithms or whatever that means but that's this little tiny box in the middle and then there's all this stuff that happens before and all this stuff happens after it and if we just make those better the world doesn't change um for example you know if you sort of have again a let's say a system to recommend who should be provided you know prioritized for covet testing or vaccine someday in the future well you could have a fair system but then the outreach to do those

00:31:40 vaccinations so testing is happening in english then it doesn't matter how fair your for your algorithm is the system isn't fair and and in in measuring outcomes there is really helpful right to kind of audit those things the second thing is i think today again you can have perfect data but there's still you know these systems optimize what the people building those systems up tell them to optimize for it's not just this sort of magical truth that they build um and we don't we don't really teach those system developers to think

00:32:11 about what to optimize for you know it's like oh it's the best system is the one that best replicates the past that's a horrible way of building these systems and and that's what accuracy often translates to which is efficiency it's the more correct you are the more efficient you are but that's not the framing we talk about in the policy world there are trade-offs and and equity and effectiveness and efficiency so i think part of it is both training the people developing these systems into these notions of there is no objective accuracy it's not a trade-off

00:32:42 it's your performance measure has to include all of those things as opposed to just efficiency um i think the other thing is that the point is a lot of times are these systems are making are forcing us to make some of these um athena was saying these human societal policy values explicit you know before you know they were all implied and these systems these decisions were made inside people's heads or in political speeches or you know and now if you have to build a system i have to put these values as numbers into an algorithm and that's

00:33:14 uncomfortable and painful again we're not trained to have those discussions especially not with the people who are being affected by these types of systems right so it's not just i'd say human centered is useful but also culturally centered of the culture that you're trying to build it for um um and yeah so i think i think i think i'm thinking through all these things but i think in in terms of moving forward right i think we have to expand the existing regulatory environment in order to deal with with these types of things it's not and and rather than having being kind of ai

00:33:46 focused and need to be kind of focused on the area they're regulating right so it's ftc and again uh fda and cfpb and you know all of those that has to kind of that are then an ai regulatory body um i think we have to kind of create trainings for them in tools and processes and um other thing is i think we don't today have so that's one the second is we need the same type of things for policy makers like right now we don't have any tools and trainings and guidelines for them to take all the these systems being produced because for them the answer is either this is all magic

00:34:17 i'm just gonna use it because it solves a problem or this is scary i read this paper that says they're all biased so i shouldn't use any of this because again humans are so wonderful in what they do today and they also don't have any way of sort of procuring these types of systems that are that are you know so we have to kind of have procurements procurement policies training policy and then trainings for the people who are building these systems to to be able to have these conversations have these discussions and and and know build these things that

00:34:48 that actually get to what we care about thanks a lot right um gabby what what are you what's your take it's on what's been so i will yeah redirect us away from the practical and applied back to the theoretical as a philosopher um so i just completely second everything that reid and tina said i think um regulation and education are the way to go as far as applied safeguards but another thing that i

00:35:19 think we need to do is to redirect theoretical attention toward what sorts of aims we should be having and one of the things that i focus on in my own work to just drill down for a second is getting away from this idea of objectivity as an ideal so not only is objectivity something that we are incapable of achieving in a robust sense um but it's something that i think we shouldn't even be aiming for and so here

00:35:50 maybe i'll say some things that are controversial from a certain sort of perspective but i just want to be clear about what i'm saying when i'm using the term bias i don't mean it in any normatively latent sense so i want to be normatively agnostic about whether or not a bias is problematic either because it doesn't get us on to truth or it doesn't get us under justification or because we think it's doing something morally problematic so for me biases can be ethically good or bad epistemologically good or bad so one of the things that we recognize

00:36:22 from coming at both human decision making and algorithmic decision making from a naturalized perspective that is looking at how human beings actually make decisions is that we need bias and this is something that any computer scientist is going to say and then immediately be confronted with like skeptical eyes uh but it's something that we know just about how humans operate in the world uh so as philosophers of science we often focus on what's called the problem of under determination that is any data set that you're given will vastly under determine the possible hypotheses that you could come up with

00:36:52 in light of that data and i think if you're someone who favors objectivity or impartiality overall you'll notice that there's a problem here if we treated all of those many infinitely indefinitely many hypotheses equally we would just be crippled with indecision and so we wouldn't be able to make any decisions whatsoever and so this is just the proof of concept that objectivity in a strict sense is impossible and so for from one perspective if odd implies can uh we shouldn't even be aiming for it in that respect but i think a more

00:37:22 important one and again this comes out of like a naturalized viewpoint that is how humans and algorithmic decision making occurs is that bias actually helps us we know more not less when we have bias and so one of the things that i think is coming out of these more practical discussions about for example so-called color-blind approaches or uh conceptions of fairness that involve not having a marker for socially marginalized demographics and the actual features that you encode in your algorithm

00:37:53 what all those discussions are proving time and time again is that we're not able to erase the markers of systemic injustice by stripping away features of our data those patterns are so deeply ingrained in our environment that any program that we have that's intended to replicate or find consistencies in that data will necessarily imbibe those same biases just like the human visual perceptual system encodes various biases like light comes from above it's just how we as finite knowers come to decisions on

00:38:24 the basis of data um so i think that this uh redirection away from uh impartiality in this robust sense will help us a lot it's almost like if we uh thought of our algorithms as coming in at the beginning of the game then a level playing field would make sense but instead we're coming in midway through a race for which uh some members of marginalized demographics have been working uphill throughout most of the race and then we're saying okay from here on out we're gonna expect plateaus and so we

00:38:54 expect a level playing field it's like midway through the race is not the time to adopt impartiality and so taking into consideration the sorts of differences that have occurred up to this point will only help us better create what i think of as a move away from equality in general to equity that is what sorts of features or implementations do we have to adopt in order to give us a more playing field holistically thanks gabby that really that really resonated with some of the stuff raid

00:39:25 was saying because uh computer scientists will of course tell us that not only do we need bias if we want to learn we need variance too and that speaks to rajid's point about how if you eliminate too much variance you're not going to be able to learn or do anything either and make decisions so you need both bias and variation and those are definitely challenges given the given the current regime of a technology uh that brings us finally to back to tracy uh uh batting cleanup here on this topic tracy yeah batting cleanup um so

00:39:56 i actually wrote down a few notes of things i wanted to say after listening to tina and raid um and then gabby went in kind of a different direction and so when you're batting cleanup it's like okay how how am i going to do this so i think the way i'm going to do it is by making four points which are not necessarily like building on one another so you should be thinking of them as responding to different parts of the conversation

00:40:26 that we've heard so far and you know hopefully the connections will be self-evident and it will be productive so point number one i i just wanna note that when kena was talking about google and her students she said um something about paying google for your data think about that just that phrase or pay somebody for your data the point was is that you had to go to somebody

00:40:56 else for your stuff so um to a lawyer that might sound strange that you would go to someone else for your stuff and actually even have to pay them for your stuff because when you think about what property law is by definition it's yours and you are entitled to it um and why would you pay somebody for something that was already yours unless you had relinquished it in some other kind of transaction that just calls to mind um a relatively

00:41:29 recent supreme court opinion which some of you might be used to carpenter i might be aware of carpenter in which justice gorsuch tries to solve some of the problems around cell site location data if you've read this opinion you know that the justices were all over the place in it but one of the things that justice gorsuch tried to do was to you know try to rely on these older concepts of property

00:41:59 to solve these problems i mentioned this because i do want to say something about regulation in a second so you know just that point for a little bit of level settings so then that does bring up the question if if if we think there are all these problems that might be solved by regulation what should regulation look like typically when we're thinking about regulation um you know for most people ordinary folks intuitively you know regulation is about coming up with a set of rules

00:42:30 that you want other people to obey which means that you have to have a theory about compliance and the one that's typically available to most people is an idea that people will comply with rules or the law because they fear the consequences of failing to do so so then you know your regulatory regime would be very would be organized around punishment um and would be organized around identifying wrongdoers you know going back to something tina said trying to figure out blameworthiness

00:43:02 in in some sense um borrowing on concepts from uh criminal law but i i think or at least i hope that the conversation so far has revealed all the ways in which thinking about that kind of regulatory structure doesn't work very well um for this space that doesn't mean that it doesn't do something or even some things to address problems problems that might just be about identification of what's fair and unfair

00:43:34 but in the normal regulatory context we're trying to think about changing behavior wholesale in some way that is good right um and you know usually criminal law approaches and and punishment regimes don't really help us get there um so you know you might be thinking again about the alphabet soup of um administrative type interventions which often have to do with getting entities to document a bunch of stuff

00:44:05 like you can imagine for example i think andrew selpst who's a law professor at ucla has made arguments about requiring these companies to do a version of an environmental impact statement you know just sort of becoming much more self-aware about the ways in which their products algorithms and so on have um particularly bad uh consequences for certain groups right um but that brings up the question then

00:44:36 when we're talking about regulation um about what what authorities are we even talking about in this context right um so again when we talk about regulation at least in this country um we typically think about what the state is doing vis-a-vis some other actor whether it's an individual actor or an entity you can also imagine regulation being that concept being a little bit more um capacious like imagining if the the the

00:45:10 from a governmental perspective it would look like self-regulation which a lot of people don't like but it might be depending on the the regulatory regime we adopt something like requiring these entities to have certain kinds of rules for themselves that could then be audited in certain ways but outside of a punishment regime so a kind of uh connect a combination of rules towards moral serration because it

00:45:40 won't necessarily result in punishment hopefully that wasn't too confusing um and i can say more as as we talk about it but again i'm motivated by the work that we've done in the social media context so you know what have we done there um we have acknowledged the fact that in a lot of ways um that space is unregulated in the typical sense right and that they're not subject to the usual kinds of administrative rules

00:46:11 by the alphabet soup of administrative agencies for good or ill people have lots of views about whether the fcc should be doing more so on um but take that as a given you can actually think about having these entities impose their own regulatory regimes in ways that are transparent to users um consumers whatever you want to call the people who are interacting in that context and you can also look to see what features

00:46:43 you would want that regulation to have and so here's where i'm going to say a little bit more about you know my favorite regulatory approach that um depends on the social psychology of procedural justice but the goal of that work is to to encourage people in a space to voluntarily um comply with the rules that they set out right um now this this view this approach actually works

00:47:15 in the real world too in fact it was developed for the real world it's a recognition of the fact that despite the fact that most people seem to when they think about changing behavior move immediately to crime-based deterrence punishment regimes the reality is most people do not obey the law and rules because they fear the consequences of failing to do so most people obey the law and our rules because they agree with them period but the regulatory problem is

00:47:46 what you do in a context in which someone doesn't agree with the rule or they think it's silly or inefficient or something that's where the social psychology of procedural justice comes in and we know that people are more likely to conclude that authorities slash rules are fair in when four conditions obtained first when people have an opportunity and a particular interaction to tell their side of the story um or if we're talking about articulation of rules um

00:48:17 participate in the the creation of those rules and that's true even if that input doesn't have any particular impact on an outcome that's you know subject to those rules and there are limits to that but you know that's um it's about having voice that's what we call it second people care a lot about being treated with dignity and respect and being listened to so you know the conversations that we were talking about before about transparency

00:48:48 and the like have to do i think with these dignity concerns third people care a lot about being able to ascertain whether decisions are fair this gets to gabby's point i think about objectivity um you know i think what's interesting though about that is of course people never know what is in fact objective they're just looking for a dish of objectivity so they look for things they think are neutral they look for factuality they look for explanations that's where the

00:49:20 explanation piece come in comes in when you are explained when an outcome is explained to you you are more likely to think that it is neutral and without bias even if that's not true and then fourth people care a lot about being able to uh trust the motives of a decision maker so this is obviously a huge problem for machines um how are you gonna test the assess the motives of the machine or artificial intelligence you're going to go immediately to the programmer to the

00:49:52 owner of the machine to the um you know to the sponsor of the project that depends on these things to ascertain those things we know finally when all of those things happen um and people are able to to um to key in on those factors we know that that folks are more likely to conclude that what whatever experience they've had or the law or the rule um is they're more likely to follow that rule

00:50:23 they're more likely to engage with the promulgator of the rule and they're more likely to cooperate with the promulgator of that rule so all this to say if there's the possibility if we're talking about possible solutions infusing these ideas into how we go about a regulatory structure i think is really incredibly important so if i may uh follow up on something that tracy said so right now uh google is providing a

00:50:55 service and you are paying them with your data right hence they're in the advertising business they're in the attention economy right uh zainab to fecky who is a professor at unc chapel hill says it's like you're never hardcore enough for youtube they want you on their site right and so part of a solution is to think about can we change the business model of these companies one is okay well let's think of google as electricity right you pay for electricity you pay them

00:51:27 they don't sell your data right you because they need infrastructure and all that to be for you to search the internet right the web the world wide web now there are actually three um approaches that people are talking about one is the service model right like electricity like cable etc that you pay them monthly the other one is what paul romer has been talking about paul romer is a noted economist at nyu he won uh one of he was one of the winners of the nobel prize in 2018 where he says we will tax

00:51:59 them we will tax google right if they make over some certain number we will tax them of of profits and then we will you know use that then for good for our for our society and then there's uh um jaren uh lanier and glenn weil um jaren lanier is one of the fathers of virtual reality and glenn weil as an economist where that no no we'll keep the model as is but google will pay you for however much you know money they

00:52:30 make off of your data right now of course you can have that game right any of these systems can be game the same way like for example with the cable right you could game the cable uh payments but as part of a possible solution and i saw in one of the q and a's there was this thing about how people get hooked on the technology um is to think about well how what is a better business model as well right so not just regulation but what is a better business model and they go hand in hand so i just wanted to clarify that it's not so much

00:53:02 that you're paying google to get your data it's just that google will agree to not sell your data um and uh for you paying them as a service um and in fact there are certain other um companies like hulu for example right you pay them you see less ads right that doesn't mean that they're not selling your data by the way everybody's selling your data so rest assured at this point everybody's selling your data and you are just the data cloud as you walk by

00:53:33 right but tina i mean yes everybody's selling your data but certain industries have been better regulated right so so the all this industry selling data is all the consumer behavior credit card that industry is extremely regulated they can only sell your data to market to you and for nothing else it's the most well one of the worst things you can do with it but still there are these rules oh you want to do product development oh we can't sell you your the data for that so and i think the data ownership world

00:54:03 for the ai world is is still so well it's not new but it just hasn't been it's like fec not knowing how to regulate things on the internet when they know how to regulate things on the you know on print and tv the same so if google and uber self-driving cars are training on pedestrians and taxpayer paid you know stop signs and whose data are they training on so is the algorithm that they've built part owned by the people whose data they were using you know so i think that's where i would put the

00:54:34 the legal profession right now on you know like what about data ownership laws i think need to catch up so everybody's selling data but i think there are variations of that data and there are better and worse models of consent and every time we our data gets used do we get notified are there presets that we can choose uh as opposed to you know um and i think we just haven't caught up with that yeah so i would completely agree with that and for example in in health care right um there are a lot of uh rules and

00:55:04 regulations in terms of you know you getting access to let's say tina's medical data um so uh yeah it's it is it is the you know wild wild west in terms of uh the data that you are trailing uh is or is trailing behind you on all the apps that you use that you believe is good utility right um so i am somebody that has like about 500 apps on my iphone just to confuse them all if i could if i could just this is great

00:55:34 if i could just jump in just uh i'm seeing some connections here so i think uh tracy's remarks and also tina's there's there's something that they have in common if you change the incentive structure in such a way that the users feel like they have more say in how everything works and is regulated uh in a sense they have more power in the process then that plays into that social science social scientific what we know about people in their group deliberations does that does that make sense tracy

00:56:04 yes sorry i had my phone muted because somebody was calling and i figured you didn't want to hear my phone ringing oh this is great uh gabby you want to chime in if you'd like to yeah um the two things i was going to add is just insofar as we're attempting to theorize about various regulatory bodies that might transfer not transfer well over to the algorithmic or machine learning domain technology industry more generally um one of the issues that came up in the discussions is just that the algorithms are proprietary so

00:56:36 one issue is the data on which they're operating but again the actual algorithmic design is proprietary as well um and insofar as uh we have regulatory bodies for proprietary recipes as it were like the fda is already available and so that we could have some i think the main thing is just that we have something external to the industry itself and likewise um in scientific practice we have institutional review boards and it's a key function of those that they have individuals on the institutional review boards who are divorced from the actual

00:57:07 scientific practice itself so that you just have an external eye looking in on how things are operating and so all this just speaks back to the importance of um having a participatory voice and what's happening that the individuals who are the subjects or victims as it were of the technology industry um that they could have a say in how things are being regulated and so then this just shifts us back to the importance of not just regulation but also education it's important that people understand exactly how the algorithms are operating what their data

00:57:38 is being used for what it could be used for how they're playing a role in everything in order for them to have an informed voice about how things get regulated yeah oh sorry let me just ask a quick question didn't the eu pass certain regulations about the use of data and that every time you go on a site they ask you what data what you want what you don't want which is of course an annoyance because you just want to get the information and move on but is that a model that is going to be useful or is

00:58:12 a useless model well i guess from my perspective not without education because as you said you go and you say okay okay follow me through right and it's horrible i mean if you look at the cookies i mean there are software that you can see how many people are tracking you online and you're like okay perhaps i don't want to see how many people are tracking me online so and then actually that dovetail comment i was going to say is that even if you decide you know what i'm not going to be online uh try to live in america right now

00:58:42 without credit card or as soon as you go to the atm and get cash they know where you are right so you maybe they don't know as much about you but they know what where you are so so there's some of that going on on the flip side i guess on a positive note here is that for example if there are images of you up on the web but nobody has ever tagged you then they don't know that it's you right it's not that it's basically like we're just giving them this data right uh and so they take it and they

00:59:13 you know they use it to make a lot of money and in fact as part of that i think it was tracy that brought up explanation so supposedly on facebook actually i know this for a fact on facebook when you get something you can say why are you showing this to me why are you showing this ad to me or on google and many other platforms if you look at those explanations the explanations are too um general right oh i'm showing this because you're a woman between 20 and 50 whose primary residence is in the us

00:59:43 you know and i think that there's actually two aspects of it one is explanation is hard and two is they don't want to let you know how much they know about you because that's going to creep you out right so actually this is one of the experiments it's one of the assignments i give to my students i'm like okay go and see the kind of ads you're getting and look at the explanations they're giving you and tell me do you think that those explanations are good enough and it's always too general right it's not specific enough in terms of why and um just one last thing about google is even

01:00:14 if you go to the google like private incognito like you you clear everything you restart your machine you are in the uh chrome or safari or or mozilla's um firefox's uh incognito or private mode they're still tracking you they know your location just search and then you're gonna get ads for you know uh around the bistro around the corner so you know a lot of this has to do with us educating the public

01:00:45 and i'm by public i mean everybody right um and this is why for me it's a freshman course like these kids come in and like whoa so i'm just real quick yeah to um some of these connections so i really like tracy's comment about how a critical aspect of like um autonomy and self-governance is consent and as tina just pointed out um we should be thinking about certain algorithmic decision making our machine learning products more generally as things like utilities

01:01:17 and so then when we're confronted with these sorts of solutions where what we get is just bombarded with terms and conditions that everyone scrolls to the bottom of and clicks accept that's supposed to be a token of consent but of course if you think about it on the model of utilities that is we couldn't possibly opt out we couldn't possibly read through all the terms and conditions then from a conceptualized standpoint where consent you know you can't give consent under duress or coercion it seems like one of the elements that we're missing is not just that you need to be educated but the point of contact where

01:01:47 individuals consent to the use of these algorithms be more informed and robust as well amen to that i i had heard so it just is speaking back to the fda analogy a couple people mentioned that i had heard maybe tina you were telling me about this that uh there's some people are proposing that like drugs you should treat algorithms like drugs and and have a similar some similar kinds of regulatory regimes do you want to maybe say something about how would that work yeah so uh when you go get uh

01:02:19 prescription drug right uh the you you get this long pamphlet that nobody reads right and then you get this like very short uh label on the uh the bottle itself right and it would be good if we could do that for algorithms right uh and in terms of the long pamphlet there have been recent movements on that so um there was a group by margaret mitchell and another one by timnit uh hebrew from microsoft

01:02:50 and google and lots of other other universities where they came up with model cards uh for models where for a particular model like machine learning algorithm for the audience um you would say you know who created it what was it and his uses how was it trained how was it evaluated does it do well on the entire population what are some of the ethical issues so it's like a long-form birth certificate for the machine learning algorithm and then the other one was um data

01:03:20 sheets for data sets where you know again it's like a long-form birth certificate for the data set how was it collected who collected it how is it being maintained how was it cleaned right lots of other kinds of stuff now if you look at those papers and you look at these long-form birth certificates or these these pamphlets they're a little bit too inside baseball the same way for me i'm not going to read the big pamphlet for the for the prescription drug that i'm getting so we we also need to have some kind of a label that

01:03:51 the general public will understand that perhaps i don't want to use this algorithm because it will have some adverse effects for me because then i will be trailing data and somebody is going to use it and say well tina is not a good person to hire for this job right because of some data that they saw elsewhere that i did so these kinds of labels are extremely important and and i think the analogy to prescription drugs for algorithms is just spot on um here i mean i think i would i would want to go further right and i'm not disagreeing

01:04:22 with starting with because fda still has pretty macro uh outputs right they'll say this drug is safe and this drug is not safe whereas i think that the question we're asking with these algorithms is not that they're overall safe enough or not safe it's they're bad for lots of sub populations um and and today fda doesn't do a very good job of of of regulating that piece um you

01:04:53 know it has a laundry list of if you have these things you should be careful well yeah but i don't what about me uh you know what things i have is this going to be useful for me fda doesn't do that and i think same for me as then if i am you know if i'm allocating health resources or or or making criminal justice decisions i need sort of for my application um does this work right so one of the things we've developed over the last couple years is sort of this thing we call the fairness tree which sort of asks you what are you

01:05:25 using what are you trying to do what do you care about again it's not it's not at all for the sort of the the consumer as the public but consumers people policy makers decision makers are using these tools um it's sort of the input is what are you trying to do for example right if you're again if you're making punitive decisions uh or interventions then disparity in false positives is going to be much worse the disparity in false negatives if you're trying to help people and give them additional services the false negative disparities are much worse right and those are sort of

01:05:56 concepts you can explain but it's much easier so same system you use for two different things can have very different outcomes um so part of it is really kind of being much more deliberate about you know for this type of problems here are the issues for these types of problems and having audit tools and all those things so i agree that i think fda is a is the closest we have but i think we need to push much further in terms of um of of sort of auditing these types of

01:06:27 tools and and putting out these things the other thing is i think you know we we don't have the the this sort of the practitioner set you know sort of guidelines for people building these things and we're not a very mature field right uh we only as a field discovered there's a thing called ethics you know a few years ago right and and we should do something about it you know all this work on uh machine learning people trying you know discovering the field of ethics so i think we're just so new that we don't have

01:06:57 reproducibility we don't have sort of documentation guidelines we just need all of those things and if we had those this conversation would be much easier because we'd start from that and say well we need to make these tweaks as opposed to you know we call ourselves a science and we have no reproducibility guidelines yeah and actually we found ethics because we got bad publicity otherwise we wouldn't have found ethics and we still have i mean found mean you know we can spell it now so so that's a start but but yeah but

01:07:29 i think it's it's it's sort of embarrassing at some point right like no no no we're not all like that yeah i mean if i could jump in because you know at northeastern our ethics institute is one of the things we're really trying to do is be part of ethics education of technol technological people people who work in the field practitioners also policy makers and i think that's so well just to throw this in while we're talking about education i do agree that technological um literacy is more it's probably the most important thing of the general population

01:08:00 but ethical uh a little bit of thinking about ethics for the people doing the stuff and using that that's also probably a good idea too i'm just just to throw that in there um yeah throw in one little thing too when you're talking about the general population um you know one of the things that we've faced at yale law school which you know it's a pretty good school and we think that we get lots of really able students um we've been focused not just on in uh educating people about

01:08:32 technology but basic numeracy i mean you know the the level of numeracy education in this country is really poor um and you know i think that precedes even this question about how much you can understand technology which of course relates to the ethics i also point out that all of us is um each one of us works in a different kind of school and there are very little relationship between those schools in terms of conversation whether

01:09:04 we're talking about schools of information schools of journalism schools of communication you know the computer science stuff and then the regulatory people law not to mention business schools to get to tina's point about business models i mean you really need to create an entirely entire new field to to get this work done honestly yeah anybody go ahead i just want to ask there seems there is a difference between ethical and legal

01:09:35 so you were talking about fda so you fda approves a drug and then the company starts putting all sorts of ads one after the other and trying to have as many people on this drug as possible legally they protect themselves by reading you a whole list of their side effects and the including that you may die from it but uh so you can control it legally but

01:10:08 ethically it's a much harder thing to define and control isn't it yeah i would agree with that absolutely and so i would say that speaks to the importance of ethical thinking not just in the technical areas but also in government and in the regulatory bodies they should learn more ethics i don't have anyone in particular mind but you can imagine who i might be talking about but like for example some of the things that i again i just want to come back to education maybe because now i'm a professor like it's all about education uh you know i

01:10:39 haven't gone to the dark side but this notion that like when i go and talk to people i'm unlike for example we can tell who's your romantic partner on facebook because not everybody says who their romantic partner is and it's a very simple model right you're like the center of this flower there are petals around you this petal is high school the spiral is college this pedal is your book club so on and so forth people who are outside of these petals who who are friends with the people inside these petals they're either your sibling or your romantic partner

01:11:09 because you are introducing them to different facets of your life now if you stop doing those introductions it's a leading indicator that you will break up in two months and we can start pushing you single bar ads and other kinds of things right i think most people will find that very intrusive right but people don't know you know we know a lot about you guys you know we could find out really easily be up to a lot more like bad stuff which we are not so friends tread softly

01:11:40 well okay so it's about 3 42 i was thinking maybe between 15 and 20 more minutes before we get to q a possibly so maybe or we don't have to take that long but maybe we could sort of turn towards the future since i'm an incorrigible optimist um sorry uh maybe each of you could try to throw in something about how you think what are some ways we might be able to turn this turn the ship around so to speak and actually use it to leverage good moral outcomes

01:12:11 and not be having to play catch up with all the pathology so much um who wants to take that first gabby sure well hang it first um so i just want to maybe bring it back to something that rajit said that i really liked which is well now putting it in my own words um every algorithm is an artifact it's like a tool that we use and so we can decide whether we want to use that tool for good or for bad and so as rahid was saying

01:12:43 in a context where we're distributing resources we might decide that a certain decision procedure that increases false positives isn't as bad as in a case where we're predicting recidivism risk say one of the things that's common about these cases though and that i think really comes out of the influence and impact of computational procedures more generally is just a sort of computational prowess that we haven't seen before and that will allow for a lot of um i think positive impacts on the world so

01:13:15 going back to this issue of uh garbage and garbage out um so i always ask computer scientists you know you say that an algorithm is only as good as the data going in garbage and garbage out but at the same time it's supposed to be more objective than like human decision makers and so how do we reconcile these two claims that seem to be intention and they usually say something like well you know we'll be able to get rid of the hangry judges so like the judges who decide just before lunch and have harsher judgments at least computers don't get um hungry and so they won't make uh more

01:13:46 angry decisions um so there are some personal level biases uh that i've studied in my own work and that i think will be ameliorated by the use of objective more objective machine learning programs but what that objectivity means isn't necessarily robust strict objectivity rather i think it comes out of just being able to notice and pick up on certain features of our world that when we march through it in this individualized over-intellectualized fashion where we're focusing on human decision

01:14:17 making we think that we're better than we are um and so when we redirect focus so here's how i think about um the advantage that machine learning programs have on humans it's like if you think of just a simple well i won't get into the details but like a two-dimensional decision uh procedure uh where we have just two features that we're making a decision on and then we come to an inference on the basis of that and now you get into like 100 dimensional feature space where machine learning programs are picking up on like

01:14:49 countless features that they're collecting through data practice or collection practices that we're not even aware of so think of like three dimensions that's easy enough four dimensions that's harder now think of like a hundred thousand dimensions folded in on itself and you get something like a spiky ball just kind of existing there and computers are more objective than humans in the sense that they're able to deal with very spiky balls whereas we humans i think are inclined towards smooth surfaces we like for things to be easy and because of their computational

01:15:19 prowess i think some of those spikes in that ball that pick out things like uh injustice or patterns of oppression in the environment that they're picking up on them i think is good for us to redirect focus away from some of these questions about i i think the question of who's to blame is still an important one but it redirects us away from what individual person has made a decision that is biased against a particular person to more so focus on the environment in which these algorithms are being used

01:15:50 and insofar as what's common in all these applications of machine learning programs is that they're picking up on those patterns that are out there in the world where realists about those patterns we're not denying that those patterns exist then the question just becomes okay how do we leverage their ability to pick up on those patterns better than we can to ameliorate some of the problematic patterns that we see in the environment and so i think that's the main thing and that should be the focus of what we do with algorithms going forward right great yeah as opposed to using that process to sell you more

01:16:21 soap more effectively which is basically what's happening now ray do you want to jump in on this sure so i like i like abby's sort of description of these spiky balls right i think i think the the the part that's sort of extending that a little bit is is what a lot of these algorithms trying to do is they're given these these spikey balls and then they're given some outcome and and they're saying well figure out the patterns of spiky ball that lead to those outcomes now the problem is those outcomes are

01:16:53 not objective right those outcomes there's two types of problems we use ai for generally right for prediction machine learning prediction things for one is classification right where some human knows what a thing is it's just too slow for us humans to figure that out fast and we're too slow for it so is this an image of a person and all the all the horrible you know things we've heard about there there human biases and the outcome are the problem right the inputs the computer can adjust but if the outcome is wrong then the computer is by definition going to be wrong

01:17:24 because it's replicating that the and so one example that is some work in talking about earlier i've been doing with you know with police departments on identifying police officers who are going to do horrible things in the future like shoot people and then justify use of force and all those different things the key word that i just said was unjustified like who determines it was unjustified um use of force happened it got investigated and there some objective internal affairs team decided justified unjustified

01:17:54 um and you know if you're a department that's a pretty horrible department like a lot of large police departments are today it's going to be totally corrupt and you're going to say everything is justified and so the computer is just going to take the spike keyboard and as good as identifying patterns the outcome is not just you know perfectly justified so that's i think one big thing is that or even things like somebody is going to graduate high school on time that's not objective it's what support structures were in place what their backgrounds were how they grew up so you can't sort of say here's an

01:18:25 objective thing and let the computers figure out how to get to that objective outcome because there is no such thing as objective outcomes right we don't have counterfactuals and um so i'll give you another example of where we're trying to sort of again to to i think it was gaby's point about equality and equity right in the beginning um of this was work we're doing with los angeles city attorney's office on reducing misdemeanor recidivism through through social service interventions and diversion programs

01:18:56 and we sort of they wanted to help in building a system that would help them get ready for people who might be the police might be arresting and and booking so that when they're called to come in front of the judge they have a case file ready with all the the connections and social service programs in place and they didn't they would have a couple of hours and that wasn't enough time so we built the system and the first version of that system we found was about 80 efficient if all the 150 people they could they could have resources to prepare for

01:19:28 uh the list we would give them would be about 80 right and the challenge was that that system was more right for white people than hispanic people um and and so so playing that hour what that system does is helps both hispanic and white people but because hispanic recidivism rate is higher than white over time it results in both of the recidivism rates going down but the disparity is increasing um that's the most efficient system right so we said okay here's option number two which is focusing on equality so we

01:20:00 built a system tune it so that it's equally right for both um it's about two percent less efficient and what does that do well it reduces equally for both so it preserves the status quo disparity and so that's what you want but here's option number two here's option number three which is maybe another percent more expensive less efficient and it's better for hispanic people than white so not focused on on equality but what it results in is lowering the disparity and downstream you know a few years later it gets to

01:20:31 equity in in recidivism rates and now you have sort of these three policy options is the menu if you care about efficiency you use option number one and you increase disparities if you care about equality you use option number two it's two percent less more expensive and you get to equality but still preserving status quo and if you care about equity definition of equity you know you get to that and there's another one percent expense you know more expensive and they chose you know number three but i think that's kind of an example

01:21:01 where we can use these types of tools to help humans make decisions that lead to equitable outcomes but it requires policymakers to want that outcome and requires people like us to provide them this menu that they can understand and then all the math and everything else goes in the background right we can start to develop these algorithms and build these systems but the reasoning at that level um i think there's a lot of hope of many other examples like that and some recent work that we did for these types of resource allocation

01:21:32 problems we found that sort of the general assumption you often go to these talks on ai fairness most people will start with well there's a trade-off between accuracy and fairness and actually there is no empirical evidence that there is such a trade-off it's just a thing we say uh and so we actually found a paper and a review but we looked at five or six these problems that we've worked on with the last couple of years and we found that for certain causes of problems we could you know pretty um with some

01:22:02 explicitly focusing on equity and and kind of dealing with that issue we can actually reduce disparities you know equal without losing any efficiency or accuracy which i think again it gives us a path forward so that we can start talking about these things as kind of you know the equity is a first order goal in machine learning systems or any systems any human decision making systems um so that's the positive uh that that i'm going to leave everybody with

01:22:33 fantastic um we got a little bit of time left tina then i'll and then i'll let tracy uh take us on home yeah so i mean uh ai technology and machine learning in particular obviously have been used for lots of um good purposes for example disaster assistance or in medical informatics imaging right you have an mri and you know you can train a machine to say well you should look at this area right or for example right now um during

01:23:04 cobit 19 the network science institute that i'm part of we're doing a lot of work in terms of can we find better therapeutics for covet we have covet's fingerprint we have the fingerprints of the drugs we know the natural compounds we know can we find one that would be better for covet or for example in terms of network epidemiology you know can we for example predict um you know what is to come right and these are very complicated models but you know they're helping to

01:23:36 figure out what to do in terms of what policies should be enacted the other aspect of it is basically like policing the police with these algorithms right when you have a policy as i understand it that policy has to have some intent and then when you execute that policy you're you're getting data from that execution and you could try to reconstruct the intent of the policy and if they don't match then you can say well something has to change right stop and frisk in new york for example right if you were to collect that data try to reconstruct the policy it seems like the policy was to harass

01:24:09 uh young um black and brown males right and clearly that wasn't what they initially said right so there are a lot of these kinds of things that one can do to benefit society right but but these are more i would say contained right in terms of disaster relief or medical informatics and so on and so forth then it gets harder for example in terms of misinformation or democratic uh backsliding this is something i've worked at where like we know what to do to improve

01:24:39 our democracy it's just that we don't want to do them in terms of for example misinformation spreading through the internet etc etc it's just that we we don't have the willingness to do that thanks dude tracy maybe you could just take us on home here in the last class just just a couple of points um i i really like what rayed um said in terms of thinking about this you know that the old trope fairness versus accuracy it it um it also connects up with something

01:25:10 you said earlier which is that computer scientist data scientists who are working on these issues understand accuracy as predicting something that's happened in the past right which you know brings up the kind of path dependency point he illustrated with his three examples and so i guess the question i would have for the group and for people listening is you know what is it that's going to motivate the people who are actually doing these things i mean maybe some of them exist out there you know i've we've got

01:25:42 two great computer scientists on our panels who are who are doing this but to think forward in a forward-looking way themselves right so you know raid says we can do it um but you know who are you waiting to ask for someone to ask to do rather than generating your own models of like actually let us be the leaders let us show you through our technological prowess how to imagine a better world i mean so you know this is supposed to be the part of the the session

01:26:13 where we imagine the future and you know as a black woman who reads a lot of science fiction i want to say that you know the alpha afrofuturist vision is usually pretty pessimistic um you know so if i'm going to be optimistic you know what is going to be my model you know what's the world i'm imagining you know we have to think about that and i guess as we're imagining that world we want to live in we don't have to think necessarily about

01:26:46 what our technological limitations are you know any one of us can do this to imagine the world we want to live in and then i guess you know it's the job of the of the tech folks to do it i guess i just want to put a little bit more impetus on them to participate in the imagining of the future rather than being constrained by the world that has existed as you know the load star for perfection in in your work

01:27:17 great so great um what a terrific panel uh we're gonna shift over to question to q a now for we got about a half an hour uh and so our moderator alex has come on so i'll hand it to him to give us a tattoo please brandon before we go yes sorry uh tracy asked the great question i'm going to put right on this spot oh okay ray do you want to you want to take a step first to tracy's question and then we'll and then we'll go to alex with uh with questions from the front of the audience she asked a couple

01:27:49 of different questions so basically so i like the one about the the you know who are you waiting for yeah and i think i think that that's that's the question i ask a lot of the the computer scientists who are kind of on the deep end the theory of theorizing about fairness like what are you waiting for to actually do this and and i think that that is a problem with you know before we started the panel we were chatting about conferences and and there's this conference in that's kind of somewhat of an

01:28:20 intersection of different disciplines computer science social science law but it's still sort of computer sciencey more than it needs to be that's sort of focused on the theory of fairness and i think unfortunately a lot of this work is too theoretical without any actual context so i think as a field that there is there is a unfortunately a gap between practice and in in in the field and i think it has to be then so so at least

01:28:51 i'm trying to figure out how to you know all of my work is with governments and non-profits because i feel like that's where the implementation is happening that's where the actions are happening but that doesn't scale you know if you work with one city la it doesn't mean that every other city is going to do this if you work with one country one state so so i think the question is how how do we take i think what we need is is ways to expose the the computer science people to real

01:29:22 problems and real people and real you know i guess data because you know so it's a combination right not objective data just real data um but then i think we need to kind of have more of these types i mean this is a good example of different fields talking and we're using different vocabulary and and we're learning about what the words are and we do that you know again all of us do that and that's why we're here but that doesn't mean that that's the norm in any of our disciplines um so i don't

01:29:55 think we're waiting for anyone i think i think it has to be kind of right now it's it's it's both sides have to be proactive about going out and saying i just want to help i'm not in it for ten year or pay for tina's highly has ten years right so you don't care uh and i think that's what we that's part of it is our disciplines don't incentivize this type of work today uh at least in academia and and we need to change that and again that's you know and then we can sort of say well it's somebody else's problem but it is our problem uh absolutely that was my non-answer no

01:30:26 that's terrific and let me just say it's perhaps even more pronounced in philosophy because very often ethicists do not are really just working on very theoretical questions they're not they don't have the lived experience they're not even they're just not aware of actual ethical problems in the societies that in which they're living i hate to say that this is why we need more we need more uh rubber meets the road even in ethics in i think in every field yeah yeah and if i may follow up on that the incentive structure is not good across the board i mean if

01:30:59 you look at the papers that are coming out in computer science in these peer-reviewed conferences and journals it's really little tweaks to things right and then if you look at like the master's students i teach some of the money maker courses right they're just like teach me the algorithms that are going to make me a lot of money i don't care right and when i try to talk to them about ethics it's just they don't care but what is interesting is that they don't see that for example the algorithm that you're developing may

01:31:29 enable misinformation that may get somebody elected that will then change the law and you can no longer get h1b visa right they don't see that link right and so the incentive structure is just not theirs just i just want to make money so teach me the money making algorithms and it's either money or you know like i had this confidence i'm teaching a class right now it's a machine learning and public policy it's half the students our machine learning department have from the policy school it's painful and one of the students

01:32:00 machine learning phd student comes to me and says i'm a machine learning phd student i just want to do math why are you having us think about these these things and i think that's the problem we have you know another student came last year and said well i'm going to go to the public private sector so this ethics class that we did i don't think it's relevant to me like what that's exactly relevant to you just because it's so you know i mean and actually like when i go to my people and i give talks i'm like

01:32:31 how many of you are okay with your algorithm being used on you nobody raises their hand not even the white guys raise their own hand right so they know there's a problem but it's just like look whatever like they don't see that one leads to another to another to another and you know game over uh which is very very frustrating but the incentive structure within the the cs and the tech the stem fields have to change because right now we're just like oh look this is such an interesting problem

01:33:01 like i remember when we were living in the in near new york i would go out with my friends who work in finance i would come back and i'm like oh this is amazing problems and brandon would say get away from the cliff get away from because again exactly as right he said like this is a really cool problem i don't care about look at the math right and so you have to get away from that and like somebody could get hurt right and so you shouldn't this this panel is amazing it could go on forever i do want to give gabby a chance to maybe get a last word in here before we

01:33:33 go to questions if you've got something i'm in it for tenure don't talk to me about incentives point that's a great way to segue into q a alex you want to take this into q a uh yes i just want to know to i think s mason dan brought apologies to single you out in the in the zoom call but you have your hand up i can't relay any question you may have if your hands up you need to write it in the q a option within zoom so with that said um eradicate

01:34:07 rhythmic decision making that forced me to this sort of deflationary view of how biases can manifest uh just from data and innocuous processes in the human decision-making domain so it actually went the opposite direction for me but insofar as both are cases i think where we're getting away from this overly intellectual view as i was saying about how biases manifest that there is a person who is intentionally deciding to treat people differently on the basis of their belongings to

01:34:40 that these differences might be an emphasis more so than like categorical differences and so one of the things i'm trying to bring out is that some of these decisions to use like a simpler model like linear regression or like a non-parametric model like these decisions have ramifications that go up the line and so uh even though it's true that we're using relatively innocuous conceptualizations of bias i take it that some of the more systematic biases or the social biases that we're concerned about and share some commonalities with these

01:35:10 decision points earlier in the causal net and so there is a relationship there and uh yeah yeah i mean i think i think there's a deeper conversation which we can leave for a later time which would mean the design choices that a machine learning system developer makes in the data sources to use how you process them how you think about them and the downstream biases and i think that's a very we we don't talk about them very much you know there's no textbook that has uh things to think about at each step to

01:35:40 deal with bias and so what i'm teaching this was the things that we're teaching about but yeah i think that's it's a different conversation because that's a huge blind spot for for the developers right now you know before we were all online we had a little joke among ourselves about writing algorithms to check out our algorithms right and uh that did seem a little silly or aggressive but um you know there are some people who have this hope and we're talking about the far future uh i don't subscribe to this but i'm just gonna say it that you know there may be some way for

01:36:11 computers to learn to be more ethical and how would you go about doing that and is it possible even are they opposite you know are there just our algorithms and ethics just also to be ethical because that would be the only way for them to become super intelligent blah blah blah but it's interesting to think well uh could a supercomputer we we grant that maybe there'll be more intelligence that we will be but i don't think it would be possible isn't this a paradox that they would be more ethical than we could be

01:36:42 because how would we know that's the case that's weird i think it's a little bit of a paradox anyway anyone think that computers might ultimately evolve towards making ethical decisions themselves without supervision well let me jump in on this i think i mean i tend to think of things so in more of a virtue ethics kind of way like um who are the exemplars who do i look to uh who i really think that they they're acting in a really virtuous ways and uh to the extent that we're looking for exemplars and

01:37:13 we're and we're modeling exemplars then i think sure machine learning could also model examples i don't see why they couldn't yeah in fact so so that reminds me of something that uh rima basu said remo basu is a professor at claremont mckenna philosophy and she was like imagine a time where you could order an uber driver who's a consequentialist or an uber driver that is a virtue ethicist or something like that right with autonomous vehicles etc where you could put in

01:37:43 the requirement for the driver that you want with me it's an interesting idea they get different tips that's the next question awful i mean think about think about uh think about chess i mean the chess the the deep learning testing students are so much better than us but we still we know that they're better how if they are bad how do we know same it's the same thing i don't think it's fundamental well we know they're better because they beat us right no no but i mean specifically we

01:38:14 still have the sense that they're improving even when they're way better than us yeah yeah they could just search the space better you know it's a big bigger space they just you know well but maybe that's all ethics is too maybe maybe just as i said you have to have the right rules and the right exemplars you have to learn what that space is you know this is being taped i would love to find out what raheed and i don't know alex where we are in the in the in the q a so if this question isn't appropriate i just i'll throw it out you can manage the questions and then go ahead but i just want to tie

01:38:46 something that jerry said and and and brandon said to something right said earlier which is t that machines could model exemplars but of course if we think of exemplars of you know particularly virtuous people no one of course is virtuous uh humans are not you know unrelentingly virtuous like all the time except for jesus i'll use my own faith tradition

01:39:17 um so you know and and you know for a lot of people and that wasn't real right so like all of the exemplars that we have um are of real people are never virtuous all the time which brings to mind raheed's point about um he said well this is an incremental risk that the machine is going to whatever we program is going to do it all the time in a way in which you know humans never are that way and i just wondering if if

01:39:47 if you could reflect on just that idea a little bit um right about using humans as the um exemplar of the virtue that a machine could be modeled after in a world in which we know there is no such thing as any virtuous human all all the time does that even make sense yeah i mean it's an interesting and i think tina has talked about similar things before it's sort of right now these these systems

01:40:18 don't use people as exemplars they use let's say people's decisions or actions as examples right so we take historical judges we take all the judges and we take their decisions and we say okay let's build a computer to replicate and aggregate all these decisions and the computer is going to be wrong many of the time so then we tell the computer which mistakes are worth more than others um and right now we say every mistake is worth the same so then it gets most white decisions right and then that's what happens in in the world right and so now if we sort of

01:40:49 think about and i and i totally honestly haven't thought about that right but it's a really good point tracy you're making is what if the exemplars were humans and then some of this some of this variant sort of gets embedded where we're really using humans as examples and trying to figure out what would this human do versus that human do and and then it's maybe also as tina was saying it becomes kind of an expert witness or or a medical test which is just another input well here's what this human would say

01:41:20 and here's why and here's what this human would say and so you sort of have this committee that's advising you uh that you can talk to and then make a decision um some of it is auditable some of it is not so i think it makes it makes a lot of sense to kind of think about it that way and see um and again i mean the danger in all of these things is that eventually these these systems have some values embedded and and where are those values coming from does every every time a new owner or decision maker

01:41:51 takes over do they then change the values to suit so their values and all that kind of stuff but i think it's a it's i'm sure other people have better thoughts around this um but yeah so this is something i have thought about um and and usually i get a pushback in that for example there are better doctors or worse doctors right and in fact if you think about law or medicine it is very much this idea of apprenticeship learning right and so can you build a machine learning algorithm that's an apprentice to let's

01:42:22 say a good judge let's say ruth bader ginsburg who recently passed away right um the problem is that is a very difficult machine learning problem right to see why ruth bader ginsburg made certain decisions his reasoning processes that's very difficult it's not as simple as thumbs up or thumbs down right which is really what it is now in terms of a lot of the algorithms you're seeing in terms of pre-trial uh disposition hiring etc right they're like gladiator right t now thumbs down

01:42:54 brandon thumbs up um and so it's a hard problem and because it's a hard problem we typically don't tackle it because it takes longer to have a paper out but with an act of an act of mercy coming from a computer mean the same to someone as an act of mercy coming from a human even that we're perfectly well modeled um that's a good question i think for the computer it's just going to go with the objective function it has

01:43:25 right so in fact usually for for recommendation systems uh we try to model items and not so much humans because humans are more complicated um so you know we have we have this term mercy that means something in particular and it seems to be a very human quality it's anyway it's an interesting dilemma when it comes to applying it through a computer interface there are people who are working on this thing called affective computing which is where the computer will develop

01:43:56 empathy so my colleague stacy marcella is working on it and others are working on it too where you the computer will learn empathy and if like an element of being merciful is empathy then that's what they're working on hmm i'll see alex are there more questions yeah yeah we have we have three more we can get through all three i think is that is that kosher is that good okay um so we have uh sarah chen who who's asking or who wrote i really like

01:44:26 the idea of labeling algorithms based on user fit for example this algorithm is only useful such accurate with white males if this becomes a norm do you think this will encourage more diverse teams slash community involved developments or will it actually lead to more exclusionary products or algorithms um i guess for me i don't think of it that way i think of it more as like having forcing the algorithm designer to be more honest right because right now we tend to not

01:44:58 be honest we say my algorithm will work on everything under the sun right so i work on complex networks for a long time in computer science you would say my algorithm will work on any complex network you give it but that's clearly not true because biological networks are very different than social networks their structure is different so i think so i look at it in terms of um more holding the algorithm designer to be honest to say okay this algorithm works like this like the auditing that

01:45:29 raheed mentioned and it's only designed for this subpopulation which of course ezra he um nicely put right now drugs aren't so much like that right though i guess there's some of it right like if you're pregnant you should not take it if you're under 12 you should not take it right there's some of that in there um but i actually take it as just having the algorithm design to be honest it sounds so tina that you would think that people would be internally motivated though if they were honest with that kind of

01:46:00 specificity to try to do better precisely because of the ground setting claims about well we think it works all the time and so if they're constantly faced with it only works in this kind of context that you you know in the back of your head or left unsaid it's probably in the front of your head is that they're going to that you think they're going to change what they do but we don't know right yeah this relates to um the initial

01:46:32 point that i brought up about uh objectivity under the or bias under the guise of neutrality or under the guise of objectivity and i think what tina's mentioning is right that right now there's the skies that there's universal applicability when in fact it's actually only useful for particular demographics and i think this question brings out that making those assumptions or biases explicit might help in holding the programmers or designers feet to the fire but i think it also has another really

01:47:02 important element which is making good on some of the claims that people from marginalized demographics are already aware of that is giving voice to the disparate um effect of various um programs or drugs like that they're already trying to point out that this impartiality exists and that it's currently being ignored and so insulating the biases that are just a part of the the normal um run-of-the-mill programs i think is really important that we could give voice to those sorts of discrepancies

01:47:32 um so i think there is a risk for it being leading to more exclusionary practices but i think also just making good on the fact that people are uncomfortable with or with the programs already is important okay so next question is from ralph electual off of youtube before i get to his question um i just want to shout him out because he's been commenting the whole time on youtube very interesting comments and summarization uh occurring uh my favorite comment of his

01:48:04 good submission was how many of you are happy to have your algorithms used on you nobody that was i i like that comment a lot uh but to his to his question uh this is the i was trying to like i mean there may be some editorial work here because i'm trying to like formulate the best way because you kind of wrote a broken comment into a question so it's sort of like balancing between accuracy so using the passive protector in fairness what will motivate people to do the right thing

01:48:41 i think rahid you want to take that because this whole uh notion that there's a trade-off as you just um uh so rahid had mentioned something about the fact that there's this false notion of accuracy versus fairness right um so it's not like you have to have one versus the other um i don't know if that answers this question but right yeah i mean i wonder i think i think again i think of accuracy as sort of a made-up construct right it's a very subjective

01:49:12 like we come up with some definition and we stick with it in practice i think in most of again most of the work i do accuracy is kind of a another way of call saying efficiency which is i have resources to help this many people and what i find is the the the the list my model helps me select has as many people from that as possible so it's efficiency and so yes there might be a trade-off between efficiency and and equity and effectiveness and and i think you know we theoretically there would be

01:49:43 but i think what i was saying was that in practice for many problems we're finding there is no empirical evidence of of that but also you know i i don't like the term accuracy right because somehow it by definition seems like it's the right thing to do and an accuracy is just saying at least the definition we think of as accuracy in these types of systems is just saying replicate as much of the past as possible and treat every individual case equally

01:50:15 each error equally and that's very narrow definition of what what and so i think i often sort of you know at least when i'm working with students it's sort of our policy makers as well let's come up with the performance goals that we have what is the overall policy goal and then let's define the thing that that we then call performance and that might include a bunch of different things but and we do that all the time from for other types of systems what's why not do it for this um so yeah i mean i think i think the the

01:50:46 trade-off is kind of more how we think about these things in a more of a knee-jerk way but but it it doesn't always appear to to to exist in practice okay um do you have time for two more questions just one more what are you guys why don't you say both of them and then we can okay okay so so the first one's from janu i hope i'm pronouncing that correctly janu writes how widespread is the use of

01:51:16 facial recognition in the western world and what regulation is coming and then the the the second one is from an anonymous viewer uh how can the new supreme court justice make fair decisions i assume they're referring to the uh i guess uh what is the assumed nominee amy cohen barrett or coney bear excuse me if i i watched your name is religion a bias when it comes to social justice

01:51:48 maybe someone should take the first one well i'll say something about that since i am the law professor um not that anyone else would have anything else to say but the first thing to say is is it a bias i really like gabby's um discussion about biases before um earlier i don't know if the questioner was was on the webinar at that point but yes of course it is you're right um but i i

01:52:19 i think you know implicit in the question given the fact that that was the second part of the question in the first half was like how can this person make fair decisions is this idea that that bias you know whatever it is i already just mentioned uh you know my own faith tradition which of course is a bias in a sense is um pointing to some sense of this person should she be confirmed capability of making fair decisions i

01:52:49 think that goes back exactly to raid's question which is what does it mean to say um a fair decision the right decision it definitely ties into this question of how we understand what accuracy is and tying that to what's right and just to blow it up a little and and the work that i do and you know criminal legal systems people often talk about policy that works you know well this works well what the hell are you talking about when

01:53:20 you say that you know stop and frisk works works to do what is always you know the question so that probably wasn't a satisfying answer for you but like that is certainly an answer to the so an answer to the supreme court question i have no idea about facial recognition um yeah i guess in terms of facial recognition as you have probably heard

01:53:51 some companies like amazon microsoft and ibm have announced that they would like stop or pause their facial recognition offerings for law enforcement but they're not the big companies official recognition is being used i don't know how why you know i what is widely but is it seems to be prevalent so before the pandemic when i was flying to europe they're like we don't need your boarding pass right and they would just scan my face and i would go through or coming back

01:54:22 from germany there was a faster way of going through immigration where again they would do facial recognition and i would just go go through and i didn't have to talk to anybody right and so and of course you know the facial recognition problems with facial recognition have been very well documented uh by joy uh poliomini at al in the algorithmic justice league that she has um in terms of regulations i think somebody has to get sued and sued big

01:54:52 with a lot of money so that something would happen and right now that has not happened perhaps you know the unfortunate gentleman in detroit um that was arrested because the facial recognition couldn't tell one person or from another i think he is suing um this city and again like with the facial recognition in particular just so you know some of the ethical problems here google knows that it has a problem with facial recognition like they don't have enough

01:55:23 black people in their data set so what do they do they hire a contractor the contractor says okay which city has a lot of black people they like atlanta they go to atlanta then they start looking for um easy easy black people that they could take lots of pictures and have them sign a content consent form so they target homeless people and they target college students they have them sign a consent form they give them five dollars they take pictures of them and they're like okay now we got our black people well clearly that's

01:55:54 wrong right is because now the system believes that black people are either homeless predominantly male that's what they were going for or or college students i mean so they just go from one ethical quagmire to another you know and what i'm saying was was you know it's publicly available there were news about it and google's like it wasn't us it was a contractor right um so yeah and i think i mean so so there are some regulations already there right so over the last year san francisco seattle um and portland have been the larger

01:56:27 places oakland i think somerville massachusetts and have also banned so these cities have banned the use of face recognition tools by city agencies and the big one has been the police department but in general by city agencies the probably other 10 states that have pending legislation that's going on um not you know new york has a bunch actually not not if it is from what i remember has passed um so so a lot of this is happening on the other hand i was at this event a couple weeks ago

01:56:59 where police departments were talking about their experiences after these bans and that was a totally fascinating where they're coming up with all sorts of loopholes like well we can't use it directly but if a consumer or another a retail store that has cameras and face recognition if they give it to us we can use it so i think there's sort of this adversarial thing going on right now where how do we you know the intention of the hand wasn't who can own it and who can not own it where it comes from the intention was you cannot use it but the way it was

01:57:29 implemented very detailed and we're doing the same thing right now in pittsburgh and it's like it's all these exclusions exist um so i think regulation is is needs to happen but it it and it's it's happening but um but there also is going to be this adversarial thing going on gabby did you want to yeah yeah add real quick i think the example facial recognition software brings out really nicely something that ravi was saying about uh talking about accuracy really like narrowing us too much on a particular goal that might not

01:58:00 be the sort of thing that we want to focus on uh so i too have heard joy balulimi talk about um her audits of facial recognition software and when it initially started it was like and a lot of these discussions about on the illegality of them come back to the accuracy issue that is that they're only something like 50 accurate with uh non-pale male faces and so this focus on accuracy the response among corporations was of course okay we'll fix it exactly as tina said get more data that

01:58:31 makes it more accurate on uh women and people of color and elderly individuals and joyce talks about how the response anecdotally from the black activist community was like please don't do this like it wasn't we don't want uh accurate machines or facial recognition programs it's like we don't want facial recognition software at all and so this focus on accuracy i think is like a bit of a red herring because it makes it sound like if we could only get more accurate facial recognition software it's like we also need the safeguards against those accurate programs being used for nefarious ends

01:59:02 and so the discussion about what makes for an ethical or a fair uh machine learning program like it really goes well beyond the scope of just the pure mathematics behind the program to what context is it being used for and what sorts of ends is it trying to achieve that might be a good place to stop i think so um what a fantastic panel tracy tina gabby raid uh thank you and thanks uh so much to ed and jerry and and the center

01:59:32 um and thank you brandon for organizing it and doing such a great job moderating it thanks ed i hope to see you guys in new york city back in person soon you know so do we yep thank you all uh and uh everyone keep an eye out on our website and people on our mailing list future round tables will likely be um uh advertised shortly thank you thank you jay and thank you for

02:00:03 thanks everybody thank you thank you all be safe out there take care ciao thank you