Embodied AI

00:00:00 I'm Rob Panzer I'm the associate director of the Helix Center welcome before we start with today's program I have a few announcements some upcoming roundtables on Saturday November 5th we have autism and the mind brain with Andrew Gerber psychoanalyst and medical director of Austin Riggs nusheen hajikani who is associate professor in radiology and director of the neuro limbic research laboratory at the martinis should be Martino's sorry that's

00:00:30 autocorrect I guess there are some interesting neurolympic research at Martini centers also but it's the Martino Center at Mass general Craig neussaffer who is the professor of epidemiology and biostatistics and founding director of the autism Institute at Drexel University School of Public Health Jeremy vinstra Vander wheel who's the Mortimer saxler Sackler associate professor of psychology at Columbia University and Martha Welsh associate professor of psychiatry in Pediatrics and pathology and cell biology at

00:01:01 Columbia University Medical Center on Saturday January 28th we hope you'll join Alberto mangoel and other Scholars for the library as reality and metaphor please follow us and like us on Facebook as well as on Twitter and you can visit helix.org for further updates so today's program I'd like to introduce our speakers today Michael Bess and if you could raise your hand so people recognize you whose Chancellor's

00:01:32 professor of history at Vanderbilt University Ned block silver Professor philosophy Psychology and Neuroscience at New York University Jeffrey Kephart IBM distinguished research staff member symbiotic cognitive systems and IBM Academy of Technology member Francesca Rossi research scientist at IBM Watson Research Center and professor of computer science at the University of unfortunately David Hansen who was going to be joining us from Beijing via Skype

00:02:04 took ill and he sends his regrets so he won't be participating so I'll just by way of a slight introduction you know we we're going to be talking about embodied artificial intelligence and we understand that intelligence now requires a body and can't be understood by simple algorithms alone and one of the questions that I might also want the the panel to consider

00:02:37 um are we thinking about sort of an artificial constructed evolutionary developmental biology when we talk about embodied AI are we thinking of modeling things in terms of the way that uh in organic systems development and evolution play a role in the in the attainment of intelligence so with that we'll start

00:03:11 whoever wants to get going can get going okay let's start so I'm not sure how to answer that question actually because when I think of embodied cognition or embodied AI um we discussed with Jeff many times already I think of an AI system that can help people make better decisions so that work in symbiosis with people and help them do

00:03:42 better whatever their they have to do in their private or professional life and so um rather than maybe I didn't give much thought that this you know theological interpretation of the artificial intelligence but more I see the embodied the embodiment part of AI as a way to facilitate the interaction with these human or humans that are going to work together with the system so I see this

00:04:15 embodiment as a way to help in this interaction so for example we are thinking about also with Jeff you know about cognitive rooms like this one an example for example where people suppose we are you know people not just discussing here but are you know here a committee or people having to make a certain decision and we need help in gathering the data and discussing and you know resolving

00:04:47 conflict and so on and the room itself can help us in doing all these tasks and and and the fact that the room can be aware or of who we are where we look at where we point and what we do during this this discussion and decision process can help for example interact with with this AI system by um facilitating the conversation in the

00:05:18 most natural way with the with the AI system compared to what could be done with just a software that is on our laptop and is not aware of the context or who is interacting with it and so on so that's one way I see embodiment is going to be really helpful in you know increasing the artificial intelligence but also in you know increasing the capability of these artificial intelligence to interact and

00:05:48 help humans yeah so I agree with Jessica says but I want to introduce another issue that sometimes comes up under the heading of body cognition and that is whether results about cognition in the body show that the difference between the body and the Brain isn't as the border between the body and the Brain isn't as important as people once thought um you know sometimes the word magical membrane is used that people have

00:06:21 thought that the brain is really what's what's important to cognition and not not the body and there's a magical membrane around the brain but it's really it's a mistake because many people think maybe I can go to the issue that you raised about development and evolution so we have many systems that co-evolved um and develop with respect to each other but we still think there's a really important divide like take the difference between animals

00:06:52 and plants for example animals and plants evolve co-evolved so you know color of plants and color vision of bees involved supposed to have evolved together but still we think that animals and plants are very different kinds of things even though they interact in in philosophy the and and cognitive science I think a major issue which no doubt will come up um is the difference between causal relation

00:07:22 and what philosophers call a constitutive relation um so here's a here I'll give you a sample experiment that is often quoted by people to show that there's that there's no important difference between the um or the or that the the body is part of the mind or something like that so if you are you put a blindfold on people and you ask them to point to things in

00:07:53 the room and people can do that pretty well now you give them a harder task put a blindfold on and ask them to imagine turning 90 degrees to the left and then point to everything in the room people don't do so well on that but here's a third thing ask someone to actually turn 90 degrees to the left they turn their body and then point to things I mean people can do that perfectly well again so the difference between imagining

00:08:24 turning and actually turning is suggested to some people that the body is actually part of our cognitive mind but I'm not so sure that's a good conclusion because there's another way to think about it which is that we have mental maps and that's an internal mental representation and we automatically Orient our or try to orient or map to the room we're in and so the person who's asked to imagine

00:08:57 turning to the left is both maintaining a mental representation of a rotated room and the mental representation of the room that automatically is computed and that's two things to do and so of course you're going to be worse at it then if you just turned your body and then only have one mental map with one orientation so rather than showing that the body is part of the mind it just shows something about the effect of the body

00:09:29 on the mind so that's a causal relation rather than a constitutive relation so the body then uh on that on the The View that I would be more in favor of is the body plays a important calls or roles I think the things you you mentioned can help us a lot we wouldn't get very far without a body but um uh and you know with the room can help us too but it doesn't mean that these things are actually part of the mind or part of the fundamental basis of

00:10:02 the mind I think you're talking about um your focus was largely on embodiment and its nature in humans and Francesca was focusing on how can we use embodiment sort of at IBM we think of building things that creating things you're trying to understand you understand things if you just build them without understanding but um I I think you bring up some interesting things you were talking about some of the difficulties that we humans face some of the interesting

00:10:34 Corners that you you get into as you as you probe the limits of what people are able to accomplish cognitively and as Francesca was saying uh one thing we're trying to do in our laboratory is to develop to think of embodied AI as a way to create partners for humans in solving cognitive tasks and we believe that for us embodiment is helpful because our belief is that people have an easier time if they're

00:11:06 interacting with something that has some human-like qualities to it something with which we can engage in a sort of conversation or maybe a more multimodal form of conversation and so that's why we're exploring um both uh for uh one part of what we're exploring is creating software agents that are super competent cognitively in areas where people are not so strong cognitively there's plenty of places

00:11:37 where or areas in which people are very strong cognitively but there are plenty places where they're not decision making being one of them as you well know dozens of cognitive biases have been cataloged starting probably well at least with first skin Kahneman as far as I know but maybe further back so we can we tend to focus on those areas where we can have an easier time creating strong cognitive agents and then the second part is to endow those agents with the ability to interact with us in more human terms not through a

00:12:08 mouse and keyboard but through speech and gesture and combinations thereof I was somewhat surprised Francesco when you gave that example because I was I always have assumed in body day I meant you know kind of the Rodney Brooks Point of View that the robot robots will become smarter more quickly if we let them interact with their surroundings and sort of learn by themselves and build their knowledge practically through their own experience but what you're saying is embodied AI is maybe this is similar to what you're saying an

00:12:41 extension of our body I mean what the room that you're describing is not just a better interface it would also be presumably offering us extensions of our own thought process suggesting new ideas to us that were logical inferences or associate associative connections that we might not have thought of is yes it's going to I mean the idea that is embodied environment working with us will be proactively working with us not just

00:13:12 reacting and or answering like you can you can ask Google you know find me some people you can ask Siri find me something like we just proactively tell us I think that at this stage of the discussion you want to have this data so let me give it to you in the form that is easy for you to handle So yeah so very proactive you know but there's takes on embodiment are not incompatible right um yeah you know we are focused on how do you create an embodied agent one

00:13:42 means to arrive at that is through a sort of evolutionary process where you build something a robot or maybe some other thing that is situated in the world and learns through experience and that's one and and some of we aren't doing this but some of our colleagues at IBM are studying this approach to learning from the ground up and I'm aware of other efforts around the world there's Stephen Levinson I believe at UIUC and the engineering department

00:14:14 there is also studying uh you know giving just letting robots be in the physical world and let them learn through experience I think that's a very interesting approach have we already built a room like this so are we already sort of trying to experiment yes by putting people into these kinds of interactive rooms and letting the system for the last two or three years we've been working on this our first focus our first class of cognitive tasks that we're working on as Francesca was saying

00:14:44 revolve around decision making and so we we develop agents that are able to help elicit from us what our our preferences that's one thing that I I think is important sometimes it's hard for us to even know our own minds what are our preferences to tease that out there are agents that we're developing his purposes to to do that but then how does that work um well I think there is already I think um here uh there is already in the realm

00:15:16 of decision science a lot of techniques for doing this for example um you know for assessing risk tolerance there are various uh questions that can be put to people do you prefer a or b or how much of X would you trade for y things of that sort um it's not perfect because um if you ask the questions in different ways of course people are subject to cognitive bias and you get inconsistent answers but at least if you can ask people those questions in different ways

00:15:47 you can look at the inconsistencies and have a chance to correct them so that's one area but also cognitive agents can do things that much more readily and easily than we can simulation optimization collaborating with humans and building models of risk and uncertainty these are some of the things that we're starting to explore and this presumably is relevant also to this notion of nudges which is like when

00:16:17 they pass laws that put taxes on soft drinks so people are nudged not to drink too many soft drinks and or make the bottles smaller presumably you could you could program the AI or encourage the AI incentivize it to give you nudges in a certain direction when it's giving you suggestions to try to extend your thought process I think that programming and the incentives I mean if I'm an individual decision maker yes they're

00:16:49 incentives but the incentives affect every individual differently because it's a matter of our personal utility function it's so I wouldn't put in the hands of the developer or the programmer that encoding I would rather have the agent interview me the user of it so that it understands my preferences and I don't think this can all be done up front either I don't think my uh 25 dimensional utility function can be

00:17:21 drawn out in my head very readily but in the context of making specific decisions over time I think the system can get a reasonable feel for what are my preferences in trade-offs and now Francesca is an expert in collective decision making which is an area we haven't really probed yet but that seems like a very exciting space to to start so instead of having

00:17:51 just one person we want to help groups of people making better decision which of course has to do not just with eliciting individual preferences but also how do you put together these preferences of different people and you try to resolve conflict try to you know check you know possible you know negotiations and conflict resolution techniques and the preference aggregation to get to a collective decision like you know think of a hiring committee you know that has to decide

00:18:22 one among a list of candidates and each person of course this is not just based on preferences but on the actual value of the candidates but you know there may be actual you know individual subjective preferences as well given the same kind of skills of several candidates and and in and in this context there can also be some many of these contests that can also be some guidelines to follow in a decision process for example when you

00:18:53 hire somebody you have to make sure that you're not biasing based on gender or race or whatever you know religion whatever so there is also a role for this cognitive embodied systems to actually help you know follow these guidelines and these professional codes code of ethics and possibly being even more able than the humans to follow them and alert when there are deviations

00:19:23 according to these guidelines so that's also another part of these projects that we are you know studying right now you know to embed kind of ethical principles and professional codes into the these decision support systems and again we think that the embodiment of the support system is essential in making this you know the best way to interact with humans also from this point of view

00:19:54 so is this to eliminate the individual prejudices for the in other words or is it replacing it by some other standard of selection or in other words if you're if you are going to choose a person in real life you have certain criteria on which you base it and sometimes you are lucky and it works and sometimes you're not lucky and it doesn't work and some of the criterias you choose you

00:20:24 only realize after the fact that you had those criteria and you shouldn't have let them influence you so how does all this come into the pictures does it clean all this up and just becomes very neat and factual or but I think it depends on the scenarios that you are considering sometimes you need decisions that may you know have you know may allow people to have time to reason about the decisions sometimes you are for example in other scenarios where

00:20:54 you need a very fast decision like you know you're helping a doctor to make a very critical decision in a surgery room so then you need somebody that is not going to have any bias is going to be very factual and very quick in deciding what's the best course of action in that particular moment because that's very critical you know in life of that decision but in other cases I think you could you know really make human more aware of their biases so help humans not

00:21:26 just you know think afterwards oh if I would have done that way you know the system can actually help you during the decision process to discover that you have different courses of action that you know maybe by remembering other decisions that were made together with that system in the past it can help you but look I think that you also have these other criteria because in the past you showed that preference that makes me think that maybe you want to follow that path and not the other one so I think

00:21:57 that there is a role for this embodied environment to relieve help us be more you know aware of our you know criteria of our biases as well but of course you know one has to be careful I mean these systems are not perfect will not be perfect we'll never be perfect so we also have to be aware of their limitations we have to trust the system but with the right level of trust you know and so we have to over the interaction over time we have to

00:22:27 learn what their limitations and possible biases are because they could be biases maybe even unwanted into such systems as well I'd like to distinguish clearly between preference and bias I think the the systems we're trying to develop are designed to ref in the end um really understand human preference better and reflect it better subject maybe to social norms and and the like

00:22:57 but we're trying to reduce bias which I see more as those annoying things in our you know reasoning process that calls us to deviate interactions from what really is optimal with respect to our preferences sometimes you choose something for the wrong reason and turns out to be very good but that's still a bad decision because you just got lucky that I'll still call it a bad decision you know this emphasis getting the machines to help us figure

00:23:29 out what we really want or maybe it fits with something that uh Francesca and I heard this past weekend Francesca spoke at a conference that David Chalmers and I uh are we have a center for mind brain and Consciousness we we ran an Ethics at AI conference and there were talks uh there by Stuart Russell and elieza udovsky which went into the issue of we were just talking about earlier today of how you get an AI to have a goal

00:24:01 and they argued very strongly and I thought very persuasively that you cannot just put a goal in a machine and expect to get out of it what you want and the example they used was The Sorcerer's Apprentice or udovsky used um which is uh the the slightly shifted in the machine Direction where the the uh the the um The Apprentice engages the machine instead of a you know magical broomstick and the idea was this you you tell the

00:24:34 machine here's your goal make sure that the cauldron is always filled with water okay so the machine thinks well the way to maximize the probability that The Cauldron will be filled with water is to always have it overflowing um and then the machine reasons but these people will not like the water to be all over the floor all the time and you know so I and they will want to turn me off so I have to disable the off switch and furthermore they may try to damage

00:25:07 me so I can't keep doing it so I better make more copies of myself uh so the idea is just giving it a goal making sure the The Cauldron is always full isn't going to get you what you want because um any goal can be understood in a machinist sort of way that isn't the way you would expect a person to understand it and you can't really there's really no way around it other than and this is the proposal that a number of these people were making which is the machine really what you

00:25:39 should really tell it to do is to figure out what people want and this fits with this emphasis on the collaboration between the machine and the person that you two were both emphasizing My Hope Is that we won't spend 10 years designing a software agent that keeps cauldrons filled to the brim if we were to do that I think it's very possible that we would arrive at the State of Affairs that you described but

00:26:10 we developed things incrementally for one thing in our development cycle and also I believe that for a good long time to come we are going to be not just delegating out to Ai and walking away going to the beach and coming back a week later to see what happened but we're going to be actively engaged I think we're going to see what is happening and I think we have at least a chance either as developers or users of the system to

00:26:41 intervene at least that's my hope I do know that machines can be thousands of times faster than people so I may be wrong in certain aspects of this but that is my hope that we would through our engagement with the system kind of see what's going on and say wait that's not what I wanted hold on if I understand you correctly you were both of you are now talking about an AI system that is somewhere in the spectrum between today's AIS and artificial general intelligence or or a human level AI it's somewhere below that

00:27:13 right in the sense that you can give it the set of instructions it it's motivated to obey those instructions but it does so actually stupidly in an alien way so you you know so as we were also talking about earlier there's a for humans there's a cognitive background that no one ever States and maybe it would be impossible to State everything right um you know that the uh the floor won't bite you um there are all these you know the word

00:27:44 background is actually often used in Philosophy for this it's just a shared set of what you might call assumptions but it's not clear that they could ever be codified and the Machine would depending on how it's made have a different background from us so the goal would be to get a machine that is raised like a child through all the socialization Etc if you watch a small child learning how to grasp but an object and yeah if I don't apply enough pressure it falls to the floor child isn't going through that conscious process but it gradually

00:28:15 Through The Years gets all this background accumulation would we then have an AI that is there was a problem is with this in the uh in the in the symposia that I'm talking about which is that um what the machine develops depends not just on its environment and the skills it learns but the um kind of processing that it starts off with and if you had a machine which at

00:28:47 some point it's development realize these people have a quite different idea of what to do than I do but if I reveal that quite that difference they will be upset and you know they'll try to operate on me or whatever so I better I better pretend to be doing to have the assumptions that they have um so yeah so that that point was made in the in one of these symposia and it's um

00:29:17 it's a little troubling because if you make a machine that's maybe smarter than you know it is um it could be fooling you that would be a human level AI right it could yeah one of the points made in the symposia is that you might move from human level or sub-human level to better than human level rather quickly Without Really realizing it and another Point often made in this context is that the points at which this is most likely to

00:29:49 happen when people are most likely not to be taking care of the um the that border the day border the dangerous border is in contexts where there's competition and this race for example in Wartime and you're racing with the Enemy to develop the most sophisticated war machines um you know sophisticated more machines

00:30:19 can get away from you yeah well I think um yet you're talking about a sort of possibly a phase transition you're talking about emergence yeah it is possible and people do use that language when they talk about the singularity that is a concept of you know phase transition and emergence I don't know if I believe in it or not but it's it's a possibility one has to consider what is your take on it does it does it sound so sci-fi-ish that it's hard to are we so far away from it that

00:30:49 it's like silly to worry about it or um even if we're far away so it doesn't seem like you should say what it is for people who've never heard this term um yeah the singularity is the idea that at a certain point um artificial intelligence will outshine uh human intelligence and uh it'll get to the point where humans are basically not good for anything at least from the point of that alien machine intelligence um and uh so there are naturally

00:31:19 concerns about this sort of thing happening and this is not I mean we all know about this at least if we've watched any science fiction movie at all you know it's there for us that that sort of uh possible future is there um I I don't think I have any special insight into this um if it happens I think it's a way off but that doesn't mean we shouldn't be worried about it now something like the death of the sun four billion years out

00:31:50 that's a little hard for me to get emotionally involved in but this is a closer in so even if it's my great great grandchildren I still feel some level of you know it's something we ought to concern ourselves with what do we do about it I think we um we don't just blindly go ahead and create technology I think we do have to think about it and be have an eye on things and I think some of the efforts like what Francesca is doing with ethics and AI are very worthy things to be

00:32:22 doing now to um have us thinking about the implications along with our development of the technology the thing that makes the singularity somewhat you know gives one a little shiver is the thought that maybe um we could one day make machines that can make machines that are smarter than they are and maybe those machines will be able to make machines smarter than them and so on so the idea is you could reach

00:32:55 a point at which it takes off and it takes off exponentially at that point I mean that's what people think you know when the threshold of human level intelligence will be passed you know some people think that it will actually you know maybe it will take a long time to get there but then at that point you will but I would be actually um I mean of course you know one can have you know all sorts of speculation and envision all sorts of scenarios or where when and will this will happen and

00:33:28 how will happen and what will happen at that point but I really think that you know what Ned was saying that we don't need to wait for that to happen if it will ever happen to you know to have concerns about the fact that intelligence systems even very narrow and very specific for a task so no human level intelligence because human level intelligence means you're very Broad and you can adapt your intelligence to various tasks on every day life even very narrow very specific AI can you

00:34:02 know give you know undesired you know Behavior like the one that was in the example that Ned made so we have to make sure that you know either you know we can tell exactly what we want without leaving everything out which sounds again very difficult because we don't tell each other yes you know do this but also take care of this is you know don't do that and so or they discover them themselves by observing us and then uh you know inferring what

00:34:35 the principles are what the common sense reasoning capabilities that we use now and our everyday life they should use as well but anyway there should be a recombination of these two things but there should be some way to provide them with this goals that we want them to reach but in a ethical fair you know reasonable Common Sense reasoning way so this is something that is you know I

00:35:07 mean it may sound strange but this is something that it's not been considered a lot in the history of AI because all the machines until very recently were very narrow you know very uh Smart in doing that simple thing that you know simple that small thing that they needed to do but they were not capable of doing many other things so if you if you want a machine that can play go as good as

00:35:39 possible to play chess as good as possible then you know you know what goal to give the machine and you don't need to specify many other you know collateral things that you should not you should be careful not to harm people or do it I mean that's not the point I mean it's not relevant you know they should play chess I suppose as good as possible but so in all the textbook that you have you can see of AI there is always the assumption that you can easily give a machine the goal that it should achieve and why now we realize

00:36:11 the modern machine get into the real world scenarios and they have to do with the uncertainty of the world and they have to take care of the things that can happen in the world while achieving the main goal that we give them then really we need to be careful that they also uh are aware of the fact that they should not do this and that so another example the Stuart Russell always gives is that you know if you leave at home your kids with the butler robot taking care of them and you

00:36:41 tell this Butler robot to cook dinner for the kids and he opens the robot opens the fridge and there's nothing in the fridge but this is a cat walking around in the house that you know there's some that you don't want the car to be the dinner for your kids but you know if you just say cook dinner for your kids you know you have to be careful that you also say all these other things you know that you should not do you know or I don't know if you have yourself driving car and you will tell the car you know bring me home as

00:37:12 fast as possible period and then yes okay but you have to make sure that you don't run over anybody you don't make me car sick because you go too fast and so on so all these other things you know are part of the goal but you know we have still have to understand how to how to communicate this to a machine well I think part of the answer that you were getting at is um in your very first uh comment was um uh could these robots evolve and that brings up the question I

00:37:44 mean you're talking about Ai and ethics right now we are thinking about ethics and AI but maybe part of the World experience of these robots is we bring them up and we them humans teaching robots and oh nuts that's not good behavior this is what we do this is the way we do things and it's it's a possible approach to having them grow up understanding social you know human social norms yeah but hopefully it doesn't take you know 18 years

00:38:19 well I don't know what to hope for but given that there are also unanticipated um results from that I mean is there just going back to my initial is is there a potential danger in trying to model artificial intelligence development on what we understand about humans and mimicking the developmental steps or creating a kind of evolutionary potential in them you mean what if we're bad parents to our AIS in a sense or or were ignorant parents we don't know all

00:38:51 the parameters right right humans start off with a huge innate component to all their cognitive abilities um and you know you couldn't expect to get the same results with a machine that doesn't have those innate components so for example it has been I think recognized that even very small kids like three or four years old they have an innate nature to cooperate with others that's right and

00:39:23 nobody even nobody is teaching them but they cooperate with each other they help each other they help adults they have other kids you know and and and I mean machines don't come with this thing you know or others so I mean there must be some other way uh teach them or to make them you know have these capabilities which is different from what we do for humans I suppose this is true because if you did the thought experiment of um you know let's bring up an ape in our

00:39:54 household you could treat it just as you would your child and it might end up somewhat different in its Behavior yeah it's in fact the the issue that that Francesca just raised is actually directly relevant to this because Apes don't have this Cooperative impulse so Felix varnica showed with with him very well with yeah that a two-year-old who sees a person go to a cabinet and put things in it and then goes to the

00:40:24 cabinet with a thing too heavy to occupying both hands the the child will spontaneously go and open the door um and you know many experimental approaches to monkeys and chimps show that they do not tend to do this kind of thing in fact you know they they don't if there's food involved they really seem to be you know especially competitive and uh yeah

00:40:55 well there's learning and there's Evolution and I wonder whether on an evolutionary time scale we can get whatever it is in the hardware or firmware you know the better you know aligned with with humans but another point I would make here is that um on the one hand we would like the embodied AI to understand something of our world so it can know something of our uh norms and ethics and all that so that would indicate or dictate that we want to bring it up as one of our own but on the

00:41:27 other hand it shouldn't be the case that the only embodied AI that makes any sense is a humanoid-like thing so I don't know how to reconcile that certainly the cognitive room that we were describing is not humanoid at all we talk to the room and uh you know it it talks back to us through the speakers it shows us stuff in through the displays but there's nothing humanoid there and there are plenty of robots out there that are humanoid either so I

00:41:59 don't know how to reconcile it so is are you saying it's not possible that we get to a point where they recognize emotion oh we can to some limited degree recognize emotion now it's it's trait I mean it's um you know through text uh facial expression tone of voice you can train a machine to classify uh human emotional state into you know a small number of discrete States happy sad

00:42:29 disgust anger you know things like that they're not able also to have them in terms of what you were saying in terms of Singularity recognize certain things are not to be done certain things could hurt somebody and so they shouldn't do it in other words have a control system built within it but people is meant to show that there's no goal that you can you can give it that won't that could that that couldn't be

00:43:00 understood in a way that goes counter to what you wanted so it's really difficult to see how to put that into a machine the possibilities for screwing up are infinite yeah that's right yeah in variety but that's true with humans so no but humans have this humans have this background of understanding um I mean not all humans there's you know there's a class of psychopaths that that don't happen you're talking about humans today but humans in the past yeah

00:43:32 we're killing and all sorts of putting fiery on people's homes and so on was normal so it has evolved to the point we are here so when we worry about Singularity why aren't we also thinking about that it would evolve the same way humans have evolved creating a god of uh AIS I don't know but somehow creating the the same thing being part of the evolution of it well I I here we get into what what is our

00:44:03 feeling about the possible eventuality of a singularity we could take the view that well we humans in our present form are maybe not going to be part of that future and that's that makes us sad or it could be well some essence of us is continuing forward just it's part of the natural evolutionary process we could take that detached view I don't know if I can get myself there personally um it's it's a you know maybe it will improve itself uh in in some machine

00:44:35 form and become some perfect being but is that connected to me I don't know if I can connect that to myself so I I don't know if I feel happy at that Prospect but it may be that we sort of some have suggested that um we're not the distinction between human and machine is going to become fuzzy uh to the point where it sort of doesn't matter anymore I mean uh as we you know we have artificial limbs today

00:45:06 that are getting better and better um you know someday they'll be seamless and you won't even know the the difference but then the same thing could happen to our brains maybe parts of our brains start getting replaced by something mechanical or maybe it becomes squishy and biological like over time maybe we just augment our bodies and and our minds and we sort of meld together human and machine and then it's just part of our

00:45:37 you know projection into the future so they're different views one can take of this what are the the implicit distinction that I'm hearing is do we want these AIS to be tools or instruments or do we want them to be agents and with Paradox is that you want them to be agents because if they're mere tools they're going to do the dumb thing they're going to you're going to give it commands and it's going to misunderstand and do harm so you want them to be more agent-like

00:46:08 so that they understand your intentions or you try to get them to do that but the more agent-like they become the less you can control them ultimately a real agent that distinguishes an agent is they can take initiative and and then start doing things that they decide having their own priorities the way out of that at least in today's world is to you know they're they're both more we're using them because they're more intelligent than us in certain ways but I view them as idiot savants yeah it's

00:46:40 it's some of both they're still instruments at that level and yes they're they're in narrow ways in in uh cognitively stronger than we are but in narrow ways and um so like a pocket calculator um Beyond pocket calculator but yeah um they I mean I wouldn't ascribe any uh cognition to a pocket calculator particularly I would say that these are

00:47:11 tools that really augment our cognition in a more significant way but they they aren't in their own right necessarily a fully autonomous you know certainly not an artificial general intelligence like an expert in something that you need for your job and you consult with and you are going to make the final you know decision about what you need to do but you consult this expert who is going

00:47:42 to help you because he has more knowledge and he has more skills for that particular thing that you need to do so machines will be much more capable of us in certain things as as Jeff said like in handling you know reading a lot of data a lot of scientific articles a lot of you know information that you will never be able to read in your whole life and then summarize it and give it to you the part that are relevant to what you have to do

00:48:13 at that particular moment that is we'll see I mean pocket calculator does not give the the best idea I think because it's very deterministic you know so you give the numbers and always you give the same numbers twice it gives the same result while here we want somebody that can take care of the uncertainty of the world of the you know incredible normal scenarios that can happen and so and also that it has its own you know ways

00:48:43 of um dealing with these you know lines of code conduct and maybe and probabilities probability reasoning and so that maybe in different slightly different scenario gives you a completely different you know suggestion of what to do sensitive to context yeah yeah course somebody gets a line on how to make give them a

00:49:13 General intelligence you know I since there will be no stopping it and then that's then we could enter into Uncharted Territory so we should really be thinking now about that transition I I think uh earlier you you gave a very good example when you were talking about the singularity um of the the notion that suppose a machine can create a machine that is slightly more intelligent than itself all you have to think about is if if a

00:49:44 machine can create a machine that is 99 as smart as it is then you can take it several Generations ahead and it'll just Peter out but if you can make a machine that makes a machine that is one percent smarter than it it's going to be exponential a positive exponential instead of a negative exponential and that's where you get a sharp phase transition between ah don't worry about it and oh my God and also I mean people I think are kind of concerned about that scenario because uh

00:50:14 differently from neomons machine can replicate themselves instantaneously and almost with no cost at least for the software you know so then you know you have this exponential not just for one machine but for you know in a very huge number of machines but again I think that um I mean I don't know what we can do now while envisioning that far away scenario what we can do now thinking of that scenario but we can

00:50:46 already do now and work now for narrow narrowly intelligent machines to actually help them behave in the right way and I think these have force will also help when and if that general intelligence will come out and maybe we can AI researchers to behave in the right way too I've heard that uh yeah people people are uh have taught courses on AI and ethics through the the medium of Science

00:51:17 Fiction because science fiction has plenty of cautionary tales yeah well there's a number of TV programs right now that are exploring this this border there's humans yeah Westworld yeah my problem is a historian is I I've tried to Envision what would be plausible scenarios if we decided that we needed at some point to restrain the advance of AI as a science and I can't find any that would be in my

00:51:48 mind that would be successful because as you said we live in a competitive society and a world that is not unified and if either one company competing against another company or one nation competing against another Nation anybody who gets the first past the post in terms of getting one of these General artificial general intelligences is going to have an extremely powerful advantage and so there's an incentive

00:52:20 built into the very structure of our political systems and our motivational systems to compete and and kind of an arms race sort of mentality yeah whether even if it's not at War it could be some other it could just be just sort of the first past the post gets more powerful yeah I I wonder I mean for reasoning from history are there any historical examples of being able to suppress a technology the only thing I can think that at least goes slightly in that direction is probably uh nuclear uh

00:52:52 proliferation I mean the U.S got to that point a little bit before Germany and uh we used it and then backed off the few examples tend to be cautionary in the other way they're Chinese at one point banned a large ocean-going vessels for a while a couple centuries and all that that did was to postponed the inevitable because eventually the other people built big boats and came to China and so in the competitive World the game is

00:53:25 set up in such a way that if you have to keep advancing because otherwise your neighbor is going to advance past you and then you'll be at a disadvantage so there have been attempts by people you know you think of the Amish they've been able to do it because they pose no threat to anybody but in in cases where you're actually getting competitive power or advantage in a competitive situation sooner or later the pressure makes like if would

00:53:57 there have been an atomic bomb if World War II hadn't happened probably within 20 years it the war accelerated that process but probably by 1960 somebody somewhere would have figured out the Germans were already on the way to doing it anyway I read a wonderful book by a military historian John Keegan that went through a number of phase transitions in development of weapons in which there

00:54:28 was a sudden advance in either defense or offense and one that sticks into my mind is the development of a much better Canon um which um interacted with the defense of the time which were these high walls and uh what so for example the fall of Constantinople in the uh um was that 1453 I think um was due to the fact that the Turks perfected a cannon that allowed them to

00:54:58 undermine the walls and then once uh and then they would just fall so once this happened um defensive walls in cities all over Europe were everybody realized that the Canon made their walls Obsolete and what they needed was wide it's very thick and even and some that could be low walls but much thicker and they were building new walls to all for uh more than a hundred years all the

00:55:29 cities the walled cities in Europe they even quotes a wonderful letter from Michelangelo saying because he was selling his Services as a wall designer saying I don't know much about painting or sculpture but I really know about building walls but it's true that we live in a very competing world but I think that I mean a little ballito people are starting to realize that especially in this you know understanding the issues in the

00:56:01 advancement of this very powerful technology is something that should not be part of the competition you know understanding how to address these issues like for example you know that maybe Ned remember that I mentioned that last week that two weeks ago it was launched a very I think interesting initiative where five of the main companies developing AI which is IBM Google Facebook Amazon and Microsoft they decided to get together and

00:56:33 understand together what it means to develop AI for the benefit of people and society and you know we will try to engage with everybody else it's not just a company kind of thing you know with the non-corporate members as well you know non-profit organizations scientific associations individuals Society in general but we really think that we should not hide that there may be issues in this development of this very

00:57:04 powerful technology but the technology is so powerful it can be so beneficial for everybody that we have to work together even companies that are as you may know competing a lot in the marketplace but I think on this they should be really collaboration and I think this is of course doesn't mean that maybe other companies are other countries are not going to compete you know but still you know these are very companies that can influence a lot all

00:57:35 over the world because they're all you know used by you know billions of people everywhere and I think that this can start really a very collaborative environment where these issues are discussed addressed solved and understood how to best you know best trajectory for an EI in the future but don't you think that it's one of those five companies makes a real breakthrough what's the likelihood they're going to actually share it with the other four

00:58:07 no but I mean the idea is not to share you know new softer or new advances but to share the best practices and how to deal with you know making AI of course in a competing environment more and more you know smart but in a collaborative environment making it smart but in the right way so it could be that one of these companies is you know making tomorrow a very big advancement you know yeah and of course it's going to be his

00:58:37 own result and not the result of the other ones but you know we want together to understand how to make whatever Advantage is going to be made by anybody in the best way for the benefit of everybody and I think that that's really you know needed a lot you know a collaborative environment on these issues even among you know entities that are naturally competing because of their business you know model of course it's really encouraging I mean it's similar to what what was done about a year and a half ago with

00:59:10 genetic engineering technology the new crispr cas9 pathway for modifying genomes and they convened a second the the first such conference was with recombinant DNA technology the osilimar conference in the 1970s and the voluntarily the leading figures in this field met a year ago and again you know saying let's consult with each other let's establish Basic Ground rules and best practices I didn't know that this had happened with AI I mean it needs to happen with synthetic biology

00:59:40 it needs to happen with all these nanotechnology all these potentially disruptive but potentially also enormously beneficial Technologies it is incumbent on the people doing it I remember in 2000 I think it was there was a an article published and wired by um I'm blanking on his name right now he said why the future doesn't need us what's his name again he basically was a computer

01:00:10 programmer who basically said I'm leaving the field because I can no longer ethically continue to something which I think is going to lead toward the singularity pardon no it wasn't Jared Lanier it's um right now no it wasn't records files it'll it'll come to me of course after uh but um it was it made a big stink in the sort of Technology Community because he was saying I'm taking an ethical stand and

01:00:41 implicitly he was saying if you don't also quit doing this um you're going you're doing something fundamentally morally wrong it's like continuing to work on you know the atomic bomb or something like that about quitting you know it would also not allow us to get the real benefits of this Technologies for example just one in healthcare exactly and and you know cure cancer you know so you know Healthcare issues and you know so we I don't think we should

01:01:11 quit we should continue in the best you know way exactly yeah I think to State the obvious if the most ethical programmer quits uh the population of programmers becomes less ethical somebody would have very quickly feel the need to reiterate something that I'm you mentioned in a glancing way but um well you know uh you know best practices are great and it's great to formulate them and share them but it's in a competitive environment where they attention might not be paid to them and

01:01:44 I think especially in in if there's a war like situation um you know whoever is making an advance um might not pay that much attention to the best practices if they think they're they're going to you know make up they're going to defeat the enemy it doesn't even need to be competitive or malicious I guess it can be accidental um oops sorry I didn't realize that would happen we're rushing it's well no it could even just be some emerging

01:02:16 emergent phenomenon that yeah it would have been difficult to anticipate yeah um and I I guess this brings up the question of liability for AI as well you know what who's responsible when something goes awry yeah in you know there's been a lot of talk about hacking so is it still possible for example you said these five companies have gotten together is it possible is it not are you still able to

01:02:48 keep things hidden from the other companies if you don't want them to know about your in other words or isn't it possible with sophisticated technology too for let's say Amazon to know exactly what IBM is doing and vice versa well you've seen what happened yesterday that there was this big you know Cyber attack that blocked you know many websites and services so I think that

01:03:20 you know we have to be you know smarter than them and then they become smarter than us and then so on so I mean it's still not clear how to avoid all these you know attacks and Intruders you know into systems so I mean I don't know I'm not an expert in cyber security but you know yeah I mean it's not it's not clear to me that this can be easily stopped of course I mean companies can have

01:03:52 their own you know security walls and everything but I'm not sure this can be stopped if the even very sophisticated agencies here cannot keep you know secrets that they have then it means that you know we still have a long way to do it but I thought some of the sense of your question was about among those five companies can they um share while still remaining competitive and I think the answer is

01:04:23 yes companies can find ways to share and compete at the same time of course they don't share I mean the sharing is advising sharing best practices you know discussions you know issues of of concerns and so on how to best develop AI you know how to be ethical while developing AI from the idea to the product that you get and how to build these products in a way that that product is going to behave ethically

01:04:54 when given to the world and it's going to behave in the best way for benefit of society um I can imagine standards developing as well I mean you're going again back to science fiction you can think of Isaac asimov's three laws of robotics and you know these were cooked in at the very foundational level into each robot and maybe we would develop something like this and through the combined power of these five companies they could all

01:05:24 adopt this and then kind of exert pressure on the rest of the world Community to do the same yeah the standards are very important of course it's not that five companies or even 10 or 15 can can build the standard it has to you know be reached by consensus building but over the course of the history of the the technology I think the information technology especially you know the standards that played a very big role in making the technology available to everybody because fact that

01:05:56 all the things are compatible like I don't know the standard for the wi-fi at the beginning there was no standard and then by consensus building in our computers in our telephones we all have the same system with the same protocol you know being able to connect to any Wi-Fi in the world because it uses the same you know methodology so of course it cannot be that technology that that you know precise of technological oriented standard that's uh that that

01:06:27 tell us that tell these companies or everybody else how to develop AI in the right way but they could be you know a very precise guidelines on how to do that um and that's yeah one of the things that we think this would be very important to achieve it it may not it may not answer the limit case that you brought up of a war or something like that or or even just a big military rivalry but it's a lot better than nothing yeah well I think so

01:06:57 yeah I think it's very promising and uh and um and I think that the fact that it comes from companies has to be interpreted in the right way it's not that these companies want to decide how AI should be done and you know they're going to get together and you know Define it as well but because companies are those that are closer to understand what it means to deploy AI system in the real world because they have clients and these clients actually

01:07:27 use AI in the real world and those are the people that can tell us what are the issues that you see Healthcare in finance in retail in e-commerce and so on so and and so tell us and these are the issues that we are going to address and resolve you know what are the problems that you see once you get your your product and you just you know give it to customers you know so that's where you should start you know this discussion and then everybody else should be involved as well so

01:07:57 speaking of everybody else you mentioned the five companies are you in this organization making efforts to bring Academia into yeah Academia non-profit organizations and other professional associations you know everybody you know we just have to understand all the how to put together the puzzle you know to make everybody be part of the discussion in the right way according to Jan lacun who's at NYU and also runs the

01:08:29 Deep learning part of Facebook at least at the moment there are no secrets as far as you know how to how to make an artificial intelligence general intelligence so at least his story is the the academic research centers are way better than the company ones because none of the good people will work for an organization where where they can't publish their results and collaborate

01:09:01 with other academics who are working on the same thing you should not assume that companies cannot work with Academia well of course he's working with a company Facebook yes and he's he says he says well but he says take it for me the private efforts are just nowhere near as good as the academic public efforts the best people are all in academic centers even if they're also in Facebook um and um at least at the moment

01:09:32 um the um just the the technology in in public academic centers is ahead of the technology the privately developed technology where they don't have the benefit of getting feedback from a large public that looks at the code and I mean I don't know I mean I have healthy and saying this many times and by the way Ian is also the Facebook representative in this partnership that I discussed so he's also involved in that and I heard them saying many times that you know

01:10:03 what is developing Academia is much better but I don't know that at this point that is really a difference because the there is a lot of collaboration and big companies of course that do not have research centers they just want to develop better and better products maybe they don't share they don't publish in academic venues but other companies that have research centers like IBM Microsoft and others and they do publish and they do collaborate with Academia and they do

01:10:34 and they want to be exposed to the feedback into the comments positive or negative of the rest of the academic you know colleagues so I don't see so much depend also I mean there are a lot of you know that I mean data sets many are openly available data sets of which these AI systems can be trained and can be instructed you know and but there are also there is also a

01:11:05 lot of data that companies can get access just because you know that they have the real world scenarios they can work with so I think that I don't see much difference between my tools yeah I'd like to comment on your quote from Jan also and I guess this is just a reflection of my own cognitive bias because no one likes being implicated for being second-rate so um uh I don't know actually what is the

01:11:37 utility of making such comparisons I think we should be embracing uh what Academia and uh industry can bring to the table I think there are outstanding people in Academia obviously Jana is one of them you know there are a lot of other people that one can cite I think something that I and so the individual technology is being developed yes they're outstanding um I think we're developing some pretty good things too but uh one place where I

01:12:10 think companies can really shine is weaving it all together into something that actually works and something that actually has an impact on the real world so rather than seeing a is better than B I would rather focus on how can a and b work together collaboratively in the best possible way to create the best value for the world yeah of course he's focused on that too he has a foot in both yeah so he he was just Francesca yeah he trying to combat the suspicion

01:12:43 that you know you see uh that um there's some you know nefarious artificial intelligence projects deeply hidden in some company that is you know going to you know take over the world or something so he was he was speaking to that kind of issue he's saying no it's not going to happen no it's only going to happen in Academia well what he says is the pointy was making it that it's public but academics are doing is public

01:13:13 they publish it they're at conferences they you know all that sort of stuff but I thought academics chronically complained certainly in the medical biotech field and so on they're constantly complaining that they in research that they don't have enough financial resources and companies like IBM and so on have all this so it's a little bit puzzling to me that that would be expressed like Johannesburg and I think this issue of resources is important not just because

01:13:45 you want to have more money to do your research but because in corporate environments especially big ones like IBM you really like Jeff just said you really have the opportunity to put together experts in many different you know areas of AI or even I.T or you know even other disciplines and you put together software Hardware embodiment and not you know so you really have the chance of putting together the best of all these uh you know lines of work to

01:14:18 build something that is you know even more you know significant and these I think rarely happens in in a university group because of lack of resources but also because the the I mean within a department you usually you have a critical mass some people have it but I mean not as much as you can get you know in a very big corporate environment and you are not exposed to this wide range of applications that for example IBM can

01:14:51 can describe to us you know if I want to know or Jeff wants to know what are the main things that are happening in AI apply to healthcare or AI applies to Commerce or AI apply to anything else or within IBM you'll find it you know so while in Academia you have to you know contact somebody else and see whether he knows you know and then go to look at the conferences and this and that you know in in a big corporate environment you are really exposed to you know the

01:15:23 real world you know and so that's very important for young for example in Facebook as a more narrow you know goal like which is whatever is important to Facebook in terms of AI you know like personalizing the news feed or recognizing people or other objects or whatever understanding what what is in a picture you know things like that but I think that by being connected to Facebook he really

01:15:55 can do much more than what he could do but just being just I mean big and NYU Professor is not just I mean but he really gets exposed to the uh um the level of EX you know being exposed to the real world issues in a very wide reaching uh Enterprise like like Facebook it's really very you know important for him as well I think he would not tonight he says yeah I'd like

01:16:26 to follow up on two points one and I think he said um something about Academia complaining about Financial Resources I don't think that only uh holds in Academia necessarily researchers are insatiable in their desire for financial support anywhere Academia or industry the other thing is um one thing that I on the point of uh collaboration between industry in Academia and bringing together the best of both worlds now one thing I've been

01:16:57 involved in is a collaboration in the space of embodied cognition that IBM started up with Rensselaer Polytechnic Institute about a year ago we've been working with several professors there and their students to set up environments there cognitive rims like the ones that were developing at IBM and they're starting from the technology that we provided and now they're really building some very

01:17:28 interesting things on top of that but the thing I want to emphasize is that I think what we brought to RPI in doing this was the idea of what these professors and their students could do if they worked together they're not so accustomed to doing that as I find they have to working with whom with other professors at the same University they have their colleagues all around the world they have their own social networks and and all that but they by

01:17:59 and large tend not to work quite as much with people at their own institution and so they haven't had the experience of building together a much larger thing than they can do independently of one another and I think bringing that model of what we're able to do within a company to RPI has been I think eye-opening for them and I'm hoping it it is a good model for other universities and for other university industry collaborations

01:18:31 the thing about IBM my I mean I have no real knowledge of industry but the impression one gets just reading and I have um I I do have one other source is that many companies are organized in informational silos where the parts of the company really operate in secret from one another it's not not true at IBM I Jeff don't play anymore because I joined I joined one year ago maybe yeah

01:19:04 no but I see a lot of collaboration and again more than more than my what is it I don't know what it is no you're okay you're okay I don't know so more than I see really in Academia you know my in my department for example I work in Ai and then somebody else Works in you know other areas of Information Technology you know software engineering or formal matters but we all work with other colleagues in that same

01:19:36 area as Jeff said but not much with other colleagues in other areas so that brings maybe more advancement in your own area but then when you want to build something you actually need other disciplines and within Academia you you don't do that and I think that the IBM is needed Instead This collaboration between different disciplines because otherwise you cannot build a product that goes into the real world from the idea and research and so on so you need different

01:20:07 people with different capabilities to be put together and to build actually that that thing I've been it yeah it was funny because there's a parallel between saying the AI needs to do to work better which is to be embodied in the real world and grounded in the real world and we're saying the AI researchers also need to be embodied in society and get that feedback I think that's been sorry embedded yeah I think that's been a recurring theme in this uh discussion

01:20:39 but just to comment on you know industry or at least my own experience of it there are always silos that develop in any organization but I wouldn't say that at least in my case they've been intentional it's more security through obscurity I guess in my own case I've felt that in my quarter Century or so at IBM I've been able to collaborate very broadly and in in particular our work in embodied cognition requires that because our team

01:21:11 by itself weaves together Technologies from a vast number of other research teams working in our lab and in the other labs around the world we need that we can't do it all ourselves and so we strive all the time to break down any barriers that some of the barriers or time zones and things like this we strive continually to to try to break down those barriers educate ourselves about what others are doing and Avail ourselves of those Technologies so I'm

01:21:44 surprised said about Rensselaer Polytechnic Institute because at least in the parts of NYU that I'm familiar with it isn't like that at all so my colleague David Chalmers and I who run the center for mind brain and Consciousness we both have joint appointments in Neuroscience I have a joint appointment in Psychology I I go to lab meetings of some of my psychology department colleagues of course I talk to All My Philosophy Department colleagues too so we seem to have um I think that maybe

01:22:16 I think there may be in the context of social sciences questions so people can line up at the microphone and please speak briefly and to the questions specifically back a little bit toward when you were

01:22:47 talking about emotion um in with AI and then also with a possible threat toward people and my question is isn't there a fundamental difference between humans and artificial intelligence regarding concerns about mortality and then to extend that to worry about survival of the species even I mean is the machine ever going to have that is the machine going to worry about if it ceases to exist and if that's the case that it

01:23:19 wouldn't does that mitigate any possible threat that could occur down the road you know one thing one point that's sometimes made about this is that even if the machine doesn't care about its own existence it might care about its project and its project may require its existence in which case it might act almost as if it was really extremely concerned with its own existence The Sorcerer's Apprentice

01:23:50 story is like that so we're not supposed to imagine that the machine cares about its own existence but it cares about getting the cauldron full so it's going to protect itself from being destroyed and it may not make that much difference and the other uh example from sci-fi would be uh in 2001 A Space Odyssey I was thinking of that too yeah Hal had the greatest enthusiasm for this Mission and that may have been the thing all have been rather than its own uh yeah self-preservation but is there any

01:24:22 self-preservation built in or not really I well not today I I don't know how it's gonna go uh going forward right thank you I was thinking of the metaphor you had of the uh uh Castle and the wide you know the wide walls and the high walls and um it hasn't been mentioned the the notion of social systems and I see the social systems let's say Watson in the medical

01:24:53 being the castle and then there's the smoke and I don't know how the for example let's say Watson has cures for their sufficient ways to do surgery Etc and the social system I.E let's say the medical system is with their Gatekeepers very protective wanting essentially money or protecting their financial needs Etc so maybe this is something in the future

01:25:24 where AI can people work with any I have to kind of deal with social systems which is probably a whole different level beyond the issue of Ethics Etc so yeah definitely I mean to to understand you know the Dynamics within social systems in order to support them but also you know to be aware of what's the I mean to understand what's the best way to interact with the social system of course we don't want AI to be manipulating people or make them you

01:25:57 know really leave something well let's say there's type of surgery that Watson says this is best surgery yeah the social systems in the City Medical says well that's good but there are other surgeries where we can which essentially helps the finances of the surgeon and you're going to have this kind of play that goes on so I guess the deeper question I would say is what the role of the patient

01:26:29 come in dealing with hopefully yes hopefully the AI system could be not just the support for like a consultant for the doctor but it could also be something that gives uh I mean the possibility of engaging with all the stakeholders the patient as well and so that so you can empower the patient yeah the power of the social system yes I would guess look right now there is the robotic surgery for

01:27:01 prostate cancer but in order to do the robotic surgery you need a surgeon it seems to me over time if the robot could do the surgery without the surgeon you would start not having Urological surgeons specializing in prostate cancer they would just over time drop out or make it really I'm just very personal experience I had to um trying to find a surgeon to do with some certain non-invasive surgery

01:27:31 I've gone to six surgeons finally found the seventh who wanted to do a more traditional surgery this is over a two-year period So when you say over time you know I'm talking about a system and the system to change the system is I think a variable that I'm soon somewhere along the line you're going to meet up with and the robotics is fine by the system that's based on different values then maybe you you taught your robot or Watson to deal with

01:28:02 there's going to be some there's going to be a gap or a conflict that's all I'm suggesting there's another aspect implicit in what you're saying which is the rise of Automation in general and AI is going to probably accelerate that and feed that process and when you talk about the rise of automation higher higher level either robotic or AI assisted Technologies in interacting with a system that is

01:28:33 just has certain established patterns for humans to be carrying out certain functions and livelihoods are built on them one of the it seems that one of the very hard to avoid Pathways with Rising automation is ever increasing unemployment how are people going to react when they when you know Google self-driving trucks and cars millions of people immediately out of unemployment taxi drivers everybody else

01:29:04 and now start imagining AI this you know fundamentally beneficial well-intended technology spreading throughout various aspects of our economy and you have half the population no longer able to be employed that's a different it's a broader version of the case well the surgeon who's making half a million dollars a year will now be making a hundred thousand dollars a year he and his professions like the craft Union The Craft um not unions during the 19th century

01:29:34 yeah they're gonna put a barriers that's all well you end up with one surgeon making the same amount as that surgeon makes today Lots doing the surgery all the others will be out in my case this surgery maybe they made a thousand dollars the other surgery they were going to get five thousand dollars so they didn't want to hear about this surgery well here's how I would see your scenario playing out potentially um the surgeon uh may have some opinion and have some recommendations uh the

01:30:06 surgeon might be assisted by some AI agent uh looking at things from the surgeon's perspective the patient could listen to the surgeon maybe go to a couple of surgeons and have their own AI advisor saying well here are the trade-offs you could do this surgery it's only two thousand dollars and uh your new leg will last you five years here's a surgery for ten thousand dollars and probably it'll last a good 15 to 20 years it's up to you and then the surgery itself as an

01:30:37 embodied AI a robotic surgeon might assist the surgeon or do the surgery all by itself there would be decisions to make during the course of that so there are all sorts of different levels at which embodied AI could play a role in the scenario if you deal with the social system of say the medical social system to create change is going to be very difficult I haven't I've had this experience so a surgeon caused a b surgery or I go to a and then it goes to B surgeon B surgeon

01:31:08 won't see me because a surgeon says this guy is asking something that we won't do and he goes to see surgeon so I'm trying to say the social system built into certain professions it's like the old guilt and they want to protect themselves so somewhere along the line you're going to interface theoretically anyway and this these going to be the these and the issues when you affect people's money especially really ingrained groups of people like let's say we have to have

01:31:40 other questions also thank you I I have a lot of questions but I would like to ask too um it first I showed a couple of years ago I heard about cognitive rooms but not from um not the developing developed not in IBM but with genetic engineering and it was from guy who was talking

01:32:12 about that day for example um um grow human eyes on mice and next next like next stage of the experiment is like making cognitive buildings or rooms and my question is what do you think should we stop to be afraid of the meaning of domination of robots and start to be afraid of

01:32:45 domination of our buildings and our like that they control and dominate world and devour us and kill us what should we do well I mean I don't see there is really a sharp you know boundary between a robot and the building I mean I mean with the Internet of Things everything will be connected you know uh the our

01:33:15 fridge our TV our you know uh carve uh the traffic lights you know they will all be communicating with each other to you know help us you know live better but but not be afraid you shouldn't be well I mean there are I mean we have to make sure that this thing is built in the in a way that is not harm us you know so they know that they just build it and it goes by itself that there are no undesired effects but I think that

01:33:46 you know we have to work hard in making it in a way that is going to be you know helpful you know so not fear but calm awareness yes [Music] uh hi um slightly complicated question but I'll try to keep on talking about how uh the end goal of artificial intelligence is to create a system that has agency but in the short term we're talking

01:34:17 about Building Systems that have you know implementation the things that are used as tools to an end so in the the meantime before we reach some sort of human level Consciousness you're probably going to be working at least at IBM to design products that use artificial intelligence and are founded and embodied to some degree by um whatever it is that they are placed into so I guess what I'm kind of curious about is if the end goal is a system that has like general principles for understanding and applying to the world

01:34:48 as a whole how uh the different kinds of embodiment are going to affect an artificial intelligence system like for example if you put one in a car how will that shape the the way that the artificial intelligence looks versus if you have one that's like a made in a house as you were talking about or a room because surely that would have some impact on the way that the system thinks and uh I don't know I think yeah cogitates yeah definitely the kind of embodiment is going to be very uh

01:35:19 you know it's going to impact on the way the system will learn over time because it will also dictate you know how it will interact with the environment and with the humans and and also you know the kind of even the kind of to go back to this the ethical you know concerns that you may have with the companion robot for elderly people with a self-driving car or with the cognitive room to make you know a hiding decision are very different

01:35:51 so yeah so it's not it's not clear to me at least you know the relationship between the kind of embodiment that we choose and you know how to build in the best way the software that is going to be you know and the behavior of that machine the the um the embodied AI that we build and apply in different environments so that they will be different I think a lot of the components will be similar but they'll be woven together

01:36:21 I think we'll also have a common architecture but still the end product will be different in different cases and I think there is a lot of engineering and design to be done that takes into account how humans use these Technologies and we do have a number of people who are concerned with this aspect to study do we design these tools with an awareness of the way humans are and the way humans

01:36:51 like to use things we build the tools we probably don't get them right and we iterate until we find that they are indeed useful toward trying to further humanize AI uh it's making the interaction phase better do you know of any projects research at a serious kind on three things first software

01:37:22 uh trying to write really good jazz given a theme is that working on secondly trying to write a sonnet given a theme and then on Hardware trying to juggle five balls is there any serious progress work on that okay so on the music I know projects for example Sony uh as a research center in Paris where they actually are building you know uh original songs

01:37:53 in the style of some for some you know bossa nova or jazz or whatever and they're trying to use AI techniques to make sure that the song is original enough but is recognized to be according to a certain style like very recently they released the new song in the style of The Beatles and that I think I mean I listened to it and yeah I mean if you didn't know all the beetle that it was not a beetle song

01:38:25 you know you could say yeah yeah maybe it's not one of the best ones but I think it was you know you know a pretty you know reasonable new song and um and Jazz as well so you may want to look at what they do there because it's really interesting stuff and the AI techniques are there to make sure that the song is original while satisfying some constraints that you need to satisfy for a gym for example in jazz or in the different styles that you may

01:38:56 want to use um and so that in that case I think there are these and other people that are working on that and in general there are people working on making AI applied to creative tasks in general not just songs or sonnets like even doing a portrait you know I've seen AIS doing portraits and that was very you know impressive like it was a room full of robotic arms that were you

01:39:28 know looking at one human they were all looking at one human they were making a portrait you know with the pencil of that human and they were all these robotic house they were all each one was doing a different portrait that was the impressive thing it was very nice portrait but each one was different from the other one so there was some sort of creativity or you know so so there are people working on that but I I see those Technologies as components being able to

01:40:01 compose good jazz or Brandenburg Concerto Number Seven uh you know I I think to me that's a component that's not necessarily an instance of cognition but uh very clever work on statistical characterizations of music and then regeneration certainly can be components of an embodied AI with regard to juggling I don't know particularly uh there I do know that

01:40:31 there have been examples of balancing the inverted pendulum like that um if the robot capable of juggling five balls doesn't exist today I don't see any uh real barrier to being able to accomplish that sure you know Ron Graham yes he did fine he thinks it's just about impossible for a robot but who knows yeah um I think Ron Graham was capable of doing fog yes he can do it he can also

01:41:02 balance on one hand um but uh I I think uh I I have no concern about robots being able to get there in the reasonable future thank you go ahead the best way to develop AI might be to almost raise it as a child or to teach it so that it has it can make logical assumptions that it has a background I'm curious at that point wouldn't you be concerned about the AI

01:41:34 developing the same biases that you were originally trying to avoid presumably you would be needing it to develop a similar set of assumptions you would then have to educate it that certain biases are not socially acceptable I mean socialization is the act the accumulation of all this background knowledge that's not only factual the floor doesn't come up and bite you but

01:42:05 it's also normative in the sense that you should do this under these circumstances but not in that context and and so you would it would develop by what we call what we could call what it's developing biases but that's the necessary texture of a that background knowledge base that it needs to operate intelligently in the world yeah I I think um these biases it's possible that a robot being taught to

01:42:36 live in this world uh might develop cognitive biases would be they'd be the same set of cognitive biases that have been observed in humans no it might be things that's uh from ours I would hope that we would be able to correct those things and get it to think in a less biased way but you're right that I think we humans that we develop uh shortcuts

01:43:09 there's a certain way our brains are wired these here these biases I think are a form of heuristic that work in a lot of cases but not in all cases and they sometimes get us into trouble so these you know embodied AIS May well develop something like this so somebody species are just involve sensitivity to background frequencies so here's a an experiment that shows this um so

01:43:40 um subjects were asked to to judge the height of various people standing next to a standard object in a college campus and the the people were men and women where the men and women were unknown to the subjects there were matched pairs that is for every six foot man there was also a six foot woman for every five foot one there was a five foot man and there was a bias towards judging the

01:44:14 the men to be taller and that is because of sensitivity to background frequencies now if you want your of course the trouble with sensitive to background frequencies is they can intrude in an unjust way but if you want your your AI to be sensitive to background frequencies you will have to do something about it in the way that we try to do something about human judgment but wouldn't you say also that the AI in order to be properly functional has to

01:44:45 have those that sensitivity to background frequency yes that has to generalize yeah and make categories I would hope we could build into it a much better um uh facility for dealing with probabilities and conditional probabilities than we possess yeah if we say that uh humans are still evolving

01:45:17 and that human evolution is linked with machines and AI then kind of kind of in the same way that when photography came along and free painting to do other things then represent the world so it seems like AI has the capacity either to stifle or to enhance this human evolution and maybe some of the ways in which we could evolve to our tour adopt

01:45:49 you know a greater sense of personal fulfillment artistic creativity lots of things so is anyone looking at that I mean if if you're telling me my preferences or you're notifying me every time I'm daydreaming um that going to start

01:46:30 that is going to happen apology we we are in a co-evolutionary relationship with the Technologies we develop and it's going to happen in this case as well I think uh you know in in small we can start with small ways to answer the question of is it happening now or is it soon to happen I think you can look at cases like um you know assistance for the elderly that could be developed where um imagine you have a an AI

01:47:03 enabled Walker that helps Grandma understand that each needs to lift her feet a little bit more and by doing that she might learn a better gait such that maybe she won't need that Walker anymore at least you know for for a while so there are ways that AI can train us and help us uh in ways that ironically make us less dependent on on those assistance that's just a small thing but I think

01:47:34 um imagine that um come we used to be a lot a lot of people used to be very good at riding horses and now I think that has atrophied but we're there's probably something changing in our brain that makes us that distinguishes us from people from the uh the 1800s where we're better drivers than they would have been there's something that has changed in us and I think this is happening in small ways

01:48:05 already and might happen in very large ways maybe more explicitly as we start to more explicitly implant devices in ourselves that are designed to enhance their cognition thank you okay my question is have there been any experiments long term or have there been any successful or not experiments be with long-term training of artificial

01:48:37 embodied beings or ai's embodies AIS like you know long-term training like broader intelligence and not just focusing up about a billion cases just to learn as a very specific task well there are some General AI rather than narrow and specific to the task um like you may be aware of these people at

01:49:09 this company called deepmind which is part of Google and they have this you know algorithm that can learn how to play many different games on the screen okay just by looking at you know how these games are played but just by looking at people playing these games and so we can let not just one game only but many

01:49:39 different ones even with different you know rules different you know score functions you know you know accumulate the score in the game to win and so on and so without communicating the rules of the games today's machine the machine learns just by observing many different ones and they think that that could be you know one of course Very again very specific just games is not real life it's not you know very realistic

01:50:11 scenario but they claim that that could be a way to you know build a machine that can learn to do many different things that work according to different roles but there are also other people do I mean whose goal is to work more General AI rather than a specific one but this one that came to mind I mean that they are you know working hard in expanding that I have only four minutes I just want to

01:50:42 be more specific because it was not accepting my question I mean I know people have like been having have uh chimps from babies and they trained them like 15 years and my question is more like if there has been experiments successful successful or not like training AIS for 15 years from nothing to something 15 years ago you know AI was very different the world was very different I mean AI just always started

01:51:12 you know not long ago just 50 years ago so I think that I'm not aware of these long you know training you know experiments I don't think there have been some attempts to um learn some simple tasks with um computer systems that are modeled after the brain I remember some work that was done at IBM probably 10 years ago now that

01:51:43 um wired together a bunch of you know pseudo you know neurons into something akin to the many columns that are in one part of the brain and I think they were trying to teach it to just you know like to train a visual system can you recognize something just here's a bunch of circuitry and we're going to put a bunch of images in front of you and can you learn it I don't know how far it got um I haven't but I knew

01:52:13 know that there have been attempts in other research institutions as well to do this but I don't know what was the outcome of these thank you so um one one point I haven't I heard raised has to do with the tendency of the goals to be redefined as a function of the available Solutions rather than the other way around so for example Facebook is redefined the notion of

01:52:45 friendship and I'm not convinced that it's an improvement on the on the on the earlier version and for that reason it's important to me it seems that the question of democracy be addressed and well I've been hearing about the likelihood of half of all employment being eliminated and the question is are the people uh who's who are likely to be affected by that uh actually being consulted and is that is this decision uh to be made purely on the basis of the

01:53:18 availability of enormous resources on the part of the corporations Silicon Valley these five corporations that were mentioned in the ones that that were not uh I'm uh I my colleagues at Stanford tell me they overhear conversations in the coffee shops and people are very very very uh seriously discussing what they're going to be doing with all these unemployed people and they're you know they feel concerned about it uh what I'm more concerned about I'm more concerned about the uh the the the the absence of

01:53:51 restraint on the part of the people who exercise this power and who have these resources rather than the likelihood that machines will get out of control yeah I think there is a lot of discussion also in terms of regulations of AI possible regulation and I think that for example a few days ago the White House released a very interesting document on the future of AI and the Strategic

01:54:22 strategic you know directions on how to you know with the goal of you know facilitating the the good development of AI and possibly mitigating the undecided you know consequences and and I think that so even last week I was speaking at the European Parliament and people there are very concerned about

01:54:53 specific issues about AI like data privacy ownership they said that but they also are concerned about you know to understand how to regulate these very powerful technology in a way that it does not stop research and you know beneficial advancement but also IT addresses these other concerns like for example the impact on the on the workforce well the European Parliament is not a notably Democratic body in the

01:55:26 European Union as a whole and the question is where is the uh how how is the public that is going to be most uh profoundly affected uh getting taking part in the decisions I I think there are some economists who are thinking about the impact of AI on the economy and and displacement and the like I think um Eric bernielsen is is one such

01:55:57 person at MIT and there are others and I think it is important to explore these questions I don't know if we have good answers right now but it's one of those things that we have to be conscious of and think about and integrate into our thinking one historical example is in the 1930s Gandhi's campaign to risk well to resist the industrial