Striking a Chord: Hearing and Space

00:00:27 I'm Ed nessian director of the center uh today's uh program is on sound number four of a series of five round taes on the census the last one will be on May 11th and it will be on site uh the associate director of the center Dr herwitz will introduce the participants good afternoon everyone thank you Ed it is true our

00:01:00 next um Round Table will be on site and will also be on site this is about hearing so listen up folks today's Talk's on sound and we have a wonderful panel and I'm going to provide you with brief introductions of course all their all of these this information is on the um handouts and on the web site uh so I'll start off and if you don't mind raising your hand when I call your name Mahan

00:01:31 azadpur is a research associate professor at NYU Grossman School of Medicine where he focuses his research on clear implant and auditory brain stem brain stem implant devices these are the most successful neural interface Technologies and have transformed the lives of deaf individuals by enabling hearing sensation through direct stimulation of auditory neural Pathways with surgically implanted electrodes Des despite their remarkable success these devices face significant challenges in

00:02:03 providing clear speech understanding to address this do Dr aad poor employs a diverse range of scientific methods including psychophysics electrophysiology and computational modeling his aim is to gain a deeper understanding of how the auditory system processes information from neural implants Christopher Cerone is internationally claimed for compositions characterized by a subtle handling of Timber and resonance a deep

00:02:35 literary fluency and a flare for multimedia collaborations recent commissions include in a Grove a new Opera co-produced by LA Opera and Pittsburgh Opera a violin concerto for Jennifer Co and the Detroit Symphony a brass concerto for the symphony Cincinnati Symphony a work for narrator and Orchestra for the Louisville Symphony a a piano concerto for Shay wner and the Phoenix and Albany Symphonies a percussion concerto for Third Coast

00:03:06 percussion and four works for the la philarmonic Edgar shuer is director of Princeton University's program and Engineering physics and director of Princeton's electric propulsion and plasma Dynamics laboratory he is tenured I'm laughing about the long acronym that they have for that that's actually pretty funny he is tendered it's wonderful but it's also a funny long Epp p p d y l with a small

00:03:37 y he is tenured full professor in the Applied Physics group at the mechanical and aerospace engineering department and Associated faculty at the astrophysics science department program in plasma physics at Princeton he holds a PhD in aerospace engineering plasma science from Princeton also in 2009 Professor shui invented a new technique for producing tonally pure 3D sound for two loudspeakers allowing a listener to hear sounds located in 3D space as they would

00:04:08 in real life Jill Gordon is the ne class of 1940 distinguished professor of humanities emerita and professor of philosophy emerita at Colby College she is a foundling me member and former director of the ancient philosophy Society she is the author of two monographs turning toward philosophy literary device and dramatic structure in Plato's dialogues and Plato's erotic World from

00:04:39 cosmic Origins to human death and she is the editor of one collected volume hearing sound and the auditory in ancient Greece she is currently working on another monograph sex differences and platonic world order a reading of tus and is a contributor to the forthcoming rutl Handbook of women in ancient Greek philosophy Tristan perch's work is a is inspired by the aesthetic Simplicity of math physics and

00:05:10 code The Wire magazine describes his compositions as an austere meeting of electronic and organic one bit music his 2004 release on cantaloupe music was the first album ever released as a microchip programmed to synthesize his electronic composition live his followup follow-up release one bit Symphony was called Sublime by the New York Press by New York Press and wal stre Journal said it its oscillations have an intense hypnotic force and a

00:05:42 surprising emotional depth his latest album drift multiply on none such for 50 violins and 50 speakers was described by the New York Times as a constantly evolving landscape where sounds coales and prism where the violins pull both pull into focus and blur into soothing

00:06:13 ether is it on it's on it's on now yeah uh welcome panel so happy to see you all here um I in in the blur we drafted for the talk um which we submitted to a number of potential participants one person pushed back about the sort of slightly high futing language of but I think I was paraphrasing something L Bernstein said about the evidence evess and parade of

00:06:44 such and such which I did feel a little bit embarrassed about once he pointed out to me but he's not here today so I we can um I I actually want to stand by that description but I'm really eager to hear what all of you think about it about sound and its relationship let's say to space our emotions into thinking so I think you should start jel you said you had a push back well sure I'm happy to start um but it's not a push back on that um Jerry had also described um

00:07:18 sound and I think music in particular as connected to human emotion and my push back is an expansive one I hope uh because I I think especially if we look at the roots of thinking about sound in the ancient Greek World um it's more so about human cognizing um and so I'll just give two examples that I think might maybe open things up for people one is if we think about the pythagoreans who are fairly

00:07:49 well known um their notion of the entire Cosmos being organized mathematically is also a vision of the ENT ire Cosmos organized musically and this has to do with their understanding of music as ratio you know length of strings you know the um the um size of the flute the length of the flute Etc and they saw the entire world or universe as um not only

00:08:23 being uh not only comprising ratio and number but music as well and so human life was intended to um mimic or imitate that Harmony within the human soul uh to be good or something like that and then I'll bring up another thinker who approaches sound in a a different way um and that is another presocratic thinker by the name of heraclitus who saw that the he believed

00:08:56 that the entire Cosmos spoke a language if you will and he in one of his famous at least famous among some crowds um fragments that we still have extant he says um listen not to me but to the logos and that is the Greek word for word or speech or language and so he's saying you know the world is speaking to us in a certain way and the aim for humans is to try to understand and hear

00:09:28 the language of the world in in the proper way so those are two examples of the way in which um both music and then hearing more generally are about human understanding of the world not just emotional um responses to it I'll jump into that it's you know at least in the first part of what you said in terms of like the uh Harmony of the Spheres the planetary motion and all of these things in nature that are due to simple harmonic motion right so

00:10:00 gravity planets rotate around the sun because there's a force and it creates simple harmonic motion this sinusoidal path which is um also you know the basics of of sound a frequency is a simple um you know the vibration at a certain frequency creates a pitch that that we hear and that I think all of the laws of physics essentially set up simple harmonic motion I'm not actually so certain about the strong weak forces but it's it's throughout the physics

00:10:33 fundamental physics of the universe that the early Greeks observed and macroscopically uh that it's I don't want to say all about vibration but right we we're we're talking about harmonic motion which is at the very core um of sound um the transmission of a VI of a vibration through a medium of course sound is much more it's it's it's like a higher order process right because it's propagated with the

00:11:05 simpler laws of physics right if it's propagated through vibration through air or through material it's not a fundamental Force itself but it's um it's strangely they they they share similar qualities the the fundamental laws of physics and these high order things like similarly with waves on the ocean that's not propagating at the speed of sound it's a completely different domain but it's it's um it's based on very fundamentally related

00:11:38 ideas well I'll pick up also on the on the Pythagoras you know it's because um we think a lot about kind of you know quote Western society as as we sort of are existing right now and thinking of ourselves as being based on these Greek ideals and yet sort of what you know again western music and quotation marks exists as is as kind of I'd say like I don't know somewhere between a compromise and a perversion of what Pythagoras talks about these ideas of ratios because you know as probably best illustrated by the the well-tempered

00:12:09 clavier by Bach is this idea that we're going to work in all 12 keys that are possible and so that immediately opens up a lot of compromises that exist in in in sort of how ination works I mean for example you have this overtone series as Pythagoras talks about and the first few intervals are pretty close to what we call equal temperament and then we start to get to the fifth partial so if it's like you know if you are let's say there's a low C is your kind of fundamental sound and the overtone series exists in all sounds including my

00:12:41 voice right now and everything else so everything is a different overtone series and the difference the way you can tell the difference between say my voice and and Tristan's voice would be the slightly different overtone structures even if we're speaking at the same pitch so everything has a little bit of a different amount of different overtones like I was you know shaving my beard today and I know notice the electric razor has a really good fifth partial um but so um so the uh the but it's what's interesting to me and is that a lot so once you get Beyond this kind of first you know so you have a c a

00:13:13 c a g a c those are all and then an e and then a g which is you know I think musical literacy would suggest that that's a C major chord and then you have a B flat in this overtone series the B flat is um about 30 cents what we call flat so you know if you take a between a c and a c there's 100 cents so it's about a sixth of a tone flat it's a really remarkable interval and then a c and then another E which is about a eighth tone flat and

00:13:44 then an F that's a quarter tone flat and so all these intervals which are naturally occurring in science and and in in nature and in every sound we hear are held out of most contemporarily western music so I just think it's weird that we have this kind of idea of like our firmament is that but you know and it's and and when you know vocalists sing without any kind of piano or anything like that or string quartets play you start to get a little bit of that color in those intervals but really we've broken ourselves of these very natural intervals which I you know in a lot of my own music I try to

00:14:16 recover them in some way or another using different kinds of non-temperate instruments but then you lose that wonderful ability to have all 12 Keys yeah I would like to add um build on what you said said um first I'm glad we talking with pagor because he's um first person who really try to think both physically and philosophically about sound uh he looked at the plucking of a string and the string um actually Plato we were discussing J and I were discussing this earlier Plato singled

00:14:47 out string and pluck string as Superior to flutes and other sounds that Greeks had for some reason we were talking about earlier I don't know exactly why but I personally have a preference for flux spring but generally the Greeks put it on top of everything um the pluck spring when you pluck a spring you get a tone the tone depends on the length of the string and the tension in the string that they realized and they realized there are ratios and the ratios are important and thus the other school of logos and Harmony and so on but in physics the story continued as a matter

00:15:19 of fact that was the seed for an equation called the wave equation which Pro is by far the most important equation in physics as a matter of fact if you ask average person to name equation most likely you get a blank look but you might also get e equal MC s but E mc squ does not hold a candle to the wave equation the wave equation is everywhere H when you push your cell phone the wave equation was solved numerically by some engineer somewhere to make to make it work when you transmit RF waves when you use the microphone the wave equation was solved

00:15:50 it's as we speak being solved few hundred, thousand times around the world right now to solve problems so it comes from Pythagoras from that plug spring and what happened is by the a lot of people worked on it by the time we got the 18th century a French mathematician physicist called dalon was was mathematics were evolved in enough at that time he applied it and solved the equation an equation that describes the motion of that spring what he found is incredibly poetic thing that when you

00:16:21 pluck a spring disturbance propagates to the end of the string as long as it's held they will go back and reflect and destroy destroy each other and complete destruction except for a few survivals these survivals mathematically we call them igen values and ion functions and um these are the harmonic that you talked about and depending on on the STR on the on the length and all the tension mathematically you can get these harmonics and when we teach the equation we can get a stroboscope and make stents see the harmonics by freezing the string

00:16:53 with the right frequency now the incredible part of the story happened in 1920 and in the early 19 20th century when people try to model the atomic structure of the at the atomic structure of hydrogen most Elemental atom and got stuck they were trying to describe it as particle and a guy woke up one day called uran schinger uh hearing some ideas from other physicist among them I called D and de broi wrote took that same equation that was described to solve to describe the string which

00:17:23 originates with Pythagoras and was able to describe the hydrogen atom very and since then it's used to solve it was birth of quantum mechanics or wave mechanics so most of physics today most of understanding of matter is is described by this wave equation not all but a big part the wave Heisenberg was one of the main characters at that time U as a matter of fact you know what I'm trying to say is that the this idea poetically that of Destruction and

00:17:54 survival what survives out of that destruction are the modes that are in sync with each other and with the lengths of the string to put it to put it in a caricatural way but it's really an incredible idea and with fascinated humans from from early on I think it also speaks to you have this abstract Perfection of something abstract version of it and then you put it out into the real world and it gets messy very quickly because it reflects off the end of if you know vibration

00:18:24 reflects off the ends of the strings or the sound in this room reflects off the walls in here we have these mics because my voice doesn't you know have enough energy to carry out or maybe it wouldn't be as clear by the time it reaches the corners of the room or something and it just it makes me think about um sound as this um well I'm thinking about just the the the sort of the the Primitive side of it the the part of it that is very visceral

00:18:57 that's like connected to our body our voice our hearing our kind of like model of the world around us because you can close your eyes and kind of imagine what sort of space we're in through sound um and where am I going with this that that the the sort of the world around us adds so much richness and so much context for I'm just thinking about how um I guess I'm just trying to articulate

00:19:29 this idea that this abstraction starts pure and then it just gets very very messy and I was curious and I was wondering if um perhaps someone here knows as thinking about this idea you know like you mentioned that the Greeks were interested in vibrating strings because that's vibration through material to create sound like an instrument that's vibrating material like if I knock on this table or something that's material but then you can have something like a wind instrument where you're setting the air

00:19:59 itself into vibration when you you're actually doing the same thing the air is vibrating and sound is nothing but fluctuation and sound pressure so they do the same thing they accept it the the wind instrument creates standing waves inside a inside a tube or volume and that that you know also emits sound but to go back to the idea of getting Mucky most of this mckiness has to do with Reflections around space and that is something really incredibly important aesthetically also for sound so the reason why when sound waves go around

00:20:30 they Bounce Around the walls and when they come back they come back at you and they come back at you at different times eventually we call that Reverb uh the late time arrival at the ears of sounds called Reverb humans have a very are very sensitive to to reverb um if you come to I have an anic chamber in my lab if you go there it's very very uncomfortable and all that's happening is the sound being absorbed by the walls sounds very bizarre we don't like that it reminds us of being too open because that's what happens if you open air so most musical performances are appreciated when the

00:21:02 Reflections in a nice Hall that has a Reverb they come back at you this mud you're talking about just the right amount of what we call decorrelation they arrive at the ear left and right ear just amount of uh Randomness if this too random um we don't like it we feel like we are too open if we are if they are too correlated it reminds us of being trapped humans um feel that it's like when you go to a squash court and play an instrument doesn't sound very good so this muddiness can is one manifestation is Reverb and Reverb is an

00:21:34 aesthetic value for sound I think that's spatial also part of spatial manifestation of sound I'm curious to know more about this because I think a lot about Reverb is a is an emotional object and there's obviously naturally occurring Reverb like the sound of my voice being reflected on these walls here through the microphones and the speakers and then there's digital reverbs and something I use a lot in my own music are um convolution retion yeah which is a kind of as I'm sure you know like an amplification of a sound by a space so I you know R take my voice now

00:22:04 and run it through a convolution Reverb of the concer gal and it all soundly sound resonant and beautiful and wonderful and then in a lot of my own music I use um in my Electro acoustic music I use sounds that aren't spaces so much so I've in a recent piece I used a voice run through a water uh Fountain and so the sound like the voice is sounds gurgling and sustained and decayed and um I another piece I used um a gunshot in an oil tanker and so it lasts for about a minute and a half and I'm kind of

00:22:35 curious and for me this has to do I'm I'm so curious to hear more about this because for me it has to do with this kind of notion of memory and that like when we hear Reverb we're hearing the sound transform and Decay and sort of it's like you're freezing time for a split second and the longer the Reverb the more time itself is frozen and you're able to I mean I think you know we don't you know you know at least the great churches of the world they don't have the Reverb you know they have a long Reverb or the I think of like the G gumbas in uh in India these kinds of memory spaces that are exist with Reverb

00:23:06 so I'm like what is it about Reverb that makes us associated with almost like reverence you know I Reverb is kind of a kind of in the natural environments everywhere like in a room as mentioned it like everywhere in the acoustic environment reverbs are there and um I guess the brain kind of adapts to to be able to get the information from the kind of distorted signal signal that kind of bounces back and forth between the walls or feeling

00:23:38 um and um um it comes kind of the brain learns how to kind of listen to this signal and extract the information that it likes so if you change um kind of if to if you make the reation kind of too strong or you remove it completely something becomes perceptually something becomes kind of weird or unnatural to the brain that's kind of I guess we can say about that I would like to add something about P more about perception I very much like

00:24:10 how you started the discussion talking about um the harmonicity or kind of relation between I mean the whole universe um um I guess the whole universe I mean most of the examples that we can bring kind of are quite a slow uh kind of changes relative to frequencies of the sound um like for sounds sounds can go up to 20 khz from sounds that we can hear from like 20 HZ to 20 khz ideally um but most

00:24:43 of the um um events that happen around us are kind of much slower like when you look at the waves of the ocean like an example is not that fast um I guess we can say that um even though the sound it has very high frequencies or is very fast in nature but when it goes in the brain and it gets processed inside the brain uh it becomes a slow the brain has a mechanism to separate all the frequencies and just kind of um say

00:25:16 phase the neurons phase lock to the very slow information in the signal like for example harmonics again come come into into play there like harmonics are much slower than the whole signal frequency um so I just want to understand that a little better so when we perceive high frequencies our brain slows down the right so so interesting yeah so in the ca Clea has this mechanism that

00:25:47 different parts of the CA in the ear they are sensitive to different frequencies so for example if there is a 10,000 HZ frequency it's um that part of the ca starts oscillating uh but it doesn't transmit that 10,000 Hertz to the brain it transmits the slower changes of that um Associated to that frequency range to the brain um so we don't actually hear the HTS that high or I'm so I'm sorry

00:26:18 I'm so curious about this now I'm so interested um you mean that the brain waves the waves in the brain that are processing this information are not anywhere near that many that frequeny right so the the neurons in the brain they are not that fast they cannot kind of move that much quickly the brain the neurons firing and also but the transmission is is a radio frequency electric currents so that's almost the speed of light okay yeah but so the two the three time scales that the time of the sound itself which goes up to 20 khz

00:26:51 you know maximum and then the neuron transmission and you're saying that's quite slow is well the neuron firing yeah you're right in terms of um neural transmission is electric signal that goes from one neuron to another neuron the the that transmission is fast yeah um but um um but the neurons are kind of slow in how they follow the sound so when for example um when there is a sound and one neuron suppose that it

00:27:22 kind of response to that sound that that neuron is quite as slow it cannot um um it needs some time to be able to to respond again or follow the sound again um so me Give an example um um um yeah suppose that we are kind of making a wave with kind of string right uh the the wave um The Wave uh form it goes up and down like a kind of

00:27:52 senoidal kind of waveform it goes up comes down goes up up um when the U the wave goes up the string is going up it has the highest energy and um that's the point that it has the most ability to kind of excite uh neurons or produce neuron activity um but but some neurons when they respond to one uh peak in the signal um they are too slow they cannot follow the next Peak and the next Peak

00:28:23 they might require some time to be able to respond again so the the neurons are are kind of slow like they might lose many of those Peaks during the stimulus during the kind of the sound um so so in that sense um um the brain is brain's ability kind of um is kind of limited in terms of how fast it can follow the signal and that's why

00:28:54 that kind of the transformation between um um speed of kind of vibrations and the location in the cocka kind of it's it's very convenient um because the uh very fast vibration vibrations are not coded by fast neural activity they're coded by where the neurons are located it's kind of a place coding of fast changes in time so well I think of like

00:29:24 the it's funny like I I teach composition and when you write for the orchestra you know there's this kind of amazing range of the orchestra in particular and so something I noticed is that composers young composers never work with the the highest range that's possible strings can go quite stratospherically um a lot of and and they always forget this this register and I say this is like the one of the most important registers to manipulate because often it is this kind of subconscious thing that happens or that

00:29:55 when you hear the orchestra and you know you hear the whole Orchestra playing you're not hearing the highest notes you're kind of hearing the lowest notes and the it sort of is more of a coloristic or manipulation thing that happens where there's a subtle shift in the in the color of the quality that is again like not really perceived by the audience on a conscious level it's much more subconscious how it's perceived um but I have noticed that this is kind of and often the the way between you could tell between a good and bad piece of music is how people deal with the highest frequencies because they're not

00:30:27 ones that are the most actively perceived you're like oh I hear a Middle Sea I hear a voice I hear these things but it's really these subtleties of those higher pitches that start to um give something a kind of I don't know like it's funny because I I have this idea that like maybe high high pitches have something to do with magic because as you're saying that we can't actually fully perceive them and so there's a kind of distance between high frequencies and and our perceptive ranges that well that high frequency content that's where the texture of everything lives right like high fre

00:30:58 in in my speech is like the CL that part of the speech as opposed to the of of the tone of my voice and um but that's where all the color is that's kind of where the excitement of sound comes from for me it's something I work with a lot in my own music but also around the world around us right like if we touch the chair and we feel fabric that's texture that's high frequency content if we didn't have that everything would be plasticky lumps that would be like low frequency content content of objects in

00:31:29 space and that that that comment about Reverb and how uncomfortable we feel to to me that feels like it lives in that high frequency content perhaps this just a guess but I was just I was thinking about animals in their Burrows and the feeling of safety that we get when we don't when we're not in a large space if we're in a large space we're exposed but if we're in a burrow cozy or under our blankets in bed we have a natural feeling of safety of comfort because

00:32:01 there's no you know your ears are part of that you're you're warm but you're your ears are part of that too and but and that's just such an intrinsic I think uh feeling that we get I'm just wondering um yeah about that that uh yeah well first I think U from a scientific point of view intuition is very much correct the Reverb is mostly the hard Reflections that come later to you and they're mostly high frequency

00:32:31 content low frequency sound is sets up what's called a standing wave structure in there if you play an organ note uh somewhere here very low base and you walk in this room something incredible happens uh you can get to one point and it's be very very loud and then you come to another point and it disappear the sound almost disappears that's because there's something called a standing wave which exists inside a flute which actually call the shinger that's why the electron and the hydrogen is one place and it's not in another place exactly the same reason but to go back to um the

00:33:05 Reverb which is seem to be common link here which is has incredible aesthetic value you mentioned something which I find very poetic um which is that it's some kind of induces in us to think about memory um and a couple years ago I'm a technologist I don't know nothing about I love music but I'm not a not much don't know much about composition I a very long dinner with a composer and as a result of which we wrote an essay we were trying to find if we can Define

00:33:35 what are are the elements that are neglected in composition until recently that are purely spatial have to do with spatial aspect of sound so we came up with this after a few glasses of wine uh that there are we we call something called canonical aspect of Music which is the one that you guys do every day which is the harmony Timber Rhythm Dynamics you know and all these things that you guys work with to make compositions and then there are other aspects of sound we call spatial and

00:34:06 number one was Reverb by the way uh we come up with seven and and these elements can give you pleasure through um if you use them potentially and therefore they could be musical to to to to make sounds and um they exist actually in space in nature when you go walk down the street birbs go flying uh on top of your head water flows in a river under you bus goes by they are not melodic they don't have the canical elements these sounds but they have spatial Elements which one can enjoy and

00:34:39 the question is that can one compose in the spatial realm and or use a lot of these spal as for example motion Beethoven didn't write a Sonata where he specified whether the piano or the vanin should go around the room and zip through your head but if you can and today Technologies can give you that and that's the question is that if you define these elements and give the tools to composers can then composers use them to compose music that's spatial and I think that's the link and the theme of the discussion spatial hearing spatial sound

00:35:10 so this is something that fascinates me myself and it turns out that Reverb is the number one another one is motion as I mentioned earlier when um proximity in depth I mean when sound Comes Close to You A Whisper is some has an emotional impact if a bird you know is on the tree and uh singing is it's one beautiful romant you know bolic uh emotional impact but if you wake up and the bird is on your shoulder doing the same song it's kind of unnerving so proximity is another element so there but but to go

00:35:43 back to reverb love to elaborate about this idea of memory yeah well it's funny because you know there's a there's a whole history of spal music I mean if you think about I think the earliest piece I can think of off the top of my head is maybe the barely of requium which has four spatial choirs and then there's the um Monte did that even earlier ex gabrieli and and San Marco in Venice you know is really the his you know having these two choirs on top of each other these brass choirs and

00:36:13 of course it's funny i' I've done a good deal of spatial work and it really depends on the instrument spatial violins sucks they're very diffuse but spatial brass as as gabrieli did is really really effective for spatial percussion instruments that are um you know that have a kind of strong impulse or kind of that are very directional you know a trombone is like pointed right of you and that really helps whereas if it's spatial violins or spatial flutes it doesn't work well at all um but yeah

00:36:43 I I think a lot about this idea it's funny because so I you know I was in school and you know universities and you know universities have these lovely concert halls and they have great Reverb was you know a graduate student at Yale and there's a really great Orchestra Hall actually has a 13-second Reverb there so I was loving it and so I didn't think too consciously about it and so I moved to New York City after I finished school and um suddenly I'm doing concerts in these kinds of you know crappy you know downtown venues which might be you know a black box or suddenly I'm like wait a minute there's no resonance in these spaces and that's

00:37:15 when I started getting interested in digital Reverb just because I needed some way to recreate this effect that I was very clearly a part of my music and I I intuitively I don't know why but um but I was always interested in resonance and ringing and you know one of the most important composers to me is Mortin Feldman and his music is always ringing and this sort of and so I don't it really appealed to me and so I started being like well I need this Reverb for this music to make any sense I'm imagining as I compose not just the sound but the resonance as well and so I think that this is something that

00:37:46 um I don't I don't even know if there's I can possibly pinpoint exactly why I need this thing but I think it does have a lot to do with um the the nature of music as a kind of um I don't to say religious experience but some kind of spiritual experience and so I mean I don't think it's an accident that you know and when I I lived in Rome for a year and the kind of project I was you know I became known as this kind of like guy who would go to a church and be like and you know like listen to the ringing and record the ringing of different churches and the residences and I I

00:38:17 don't think it's an accident that these spaces of reverence have these kinds of reverbs and and you know and I think it's and I know I said this before already but I think there is something to that that strikes me as having a kind of spiritual power you know or or if you're not spiritual then some kind of metaphorically spiritual power I don't know how to describe it but it does strike me as um well at the very least it's otherworldly yeah um it's something other than what we experience in our normal life so you step into one of those spaces and it sounds different it

00:38:48 smells different the temperature is different it's different than your the rest of your life and and it engages all of the senses in that sense I've heard two of your pieces in churches now and years ago Marielle Roberts playing your cello piece I think it was like downtown Brooklyn somewhere and then hearing Jamie play your organ piece and they both went in churches and is that an accident is that I mean obviously organ I like different spaces I I I like I've done some performances in tiny tiny spaces to me I think the space asks the

00:39:19 question of what is the sound you want to hear in the space and to me that's just an interesting interesting question and I I personally my favorite spaces are kind of like this just like a gallery essentially an open space um where you are very aware of being in the room with the sound that's getting created and I find that that that physical connection is important for me as a listener and so the moment like I

00:39:51 think a lot about how the moment sound is come you know it comes from my vocal whatever the things in the space violin playing in the space um it's created in the spa same room at the same time where you are right now like you are physically connected to it um I was thinking about this the other day when the earthquake happened in New York that's sound essentially that's vibration um transmitting from New Jersey to to New York City and we all we

00:40:22 we may or may not have felt it I felt it and it was it was it was a physical connection to another place on Earth and we're not usually aware of that uh it was kind of a big experience speaking of like um almost a spiritual experiences it's like big it's a big thing um but on a smaller scale in a in a room like the moment my voice is picked up by this microphone it can go anywhere that information travels at the speed of light it's telecast online um but it

00:40:56 loses immediately that physical connection it's now been digitized it's translated into another medium that of information or an Electronics electro magnetic wave um but it's it's become decoupled it's been detached from the physical world that we share and this is something I love about live performance we're coming to something like this live is that we're in the same space we're kind of um we have a shared experience

00:41:27 here that includes sound it's not mediated it's not detached because um uh you you just it's very it's very easy to lose that especially these days um I wanted to go back to something Bill said originally about cognizing and how sound is related to cognizing and the interesting themes that have come up so far in my mind involve um these ideas first of all ratios and of course ratio is part of ration rational ration ISM

00:41:58 and um how in many instances we've been speaking about things that seem to escape that are not quite contained within those ratios so there's IR rational numbers there are these overtones and igen values that beautifully capture them um but what is that remainder I think there's some remainder that's what we're talking these are all remainder effects that we don't know exactly how they influence us right and I'm wondering because of what you just said Tristan you know does is is our understand standing in the

00:42:28 broadest sense related to these overtones and what's not quite captured by the ratios I wonder what you think about that well it's a question of periodicity right I mean roughly speaking um you know sort of John Cage abiding um that what we more likely to perceive as being music has some aspect of periodicity to it and more what we're

00:42:58 more likely to perceive as noise or the remainder is something that's AP periodic right or um non repetitive in some way or another and so I I I think obviously one of the joys of the 20th century is trying to claim the AP periodic as music as art as well um and that's sort of the difference I don't know that's something about modernity I I just a small small tidbit and then I'm going to maybe shift directions a

00:43:30 little bit just the small tidbit is um in Greek language and culture it's the same root word for that heraclitus said listen not to me but to the logos meaning word or language that also means ratio and it's a root for rationality so those are all um connected I I was wondering um it would be interesting to me to hear hear um from people in these more

00:44:01 technical um disciplines thinking about some of the ideas we've touched a little bit on like Rhythm and the human voice and I'll I'll just start by grounding it in the world that I know but I think there's lots of tentacles out into contemporary um life so um you had asked the question about you know why did the Greeks prefer the stringed instrument to the flute and um one of the reasons it's not the only

00:44:32 reason but one of the reasons is the flute is connected to lyric poetry erotic lyric poetry and so there's this um sexuality looseness that veilance is associated with flute music and there's something more rational um and kind of buttoned up uh and morally at least in the platonic sense um righteous in guitar music so there's this whole social aspect here um with kinds of

00:45:04 music kinds of musical instruments apparently people never heard Jimmy pige play agree I completely agree um and then of course the human voice is all you know people are reciting poetry alongside these instruments right and so the human voice is involved and then we've touched very briefly on Rhythm and also there were moral valences associated with certain kinds of Rhythm what's the Poetry involved here is it a you know hexameter

00:45:35 is it you know some other meter of poem and you know we have that think about when rock and roll hits the white Community right what's oh those we don't want those rhythms right there's something bad about those rhythms um so anyway I'd be curious to hear sort of what your thoughts were on these very social aspects of how we think about the voice and music and sound and the way it kind of works in a social um context

00:46:07 yeah yeah I say that the uh voice and music in general kind of natural sounds most of them they have that periodicity um um which um um I think we call them can call them harmonics which are kind of um um different um for example um can have a per periodicity of certain kind of number of um kind of periods per

00:46:38 second call it Herz um and you can have different kind of periodicity I you can add several um sounds that have a periodicity periodic nature and add them together and the result would be kind of periodic again so the natural sounds all of them because of the source of the generation they all have this U kind of periodic nature and that periodicity is um the frequency of that periodicity is much lower than um

00:47:10 the the range of the frequencies of the sound um for example like um say for example the peach of when I'm talking um it's kind of um we can tell that the speaker is kind of a man or woman or a child based on the way they are talking um that information comes from the periodicity mostly of the sound so you don't really pay attention to the high frequency contents of the sound or more details of the sound to be able to

00:47:42 determine um who is speaking um so so a lot of information is is there in small in kind of um um kind of lower frequency periodicity is and that's what the brain basically uses most of the time I mean brain mostly relies on on those kind of smaller frequencies which um they're not really a small frequency like give you number like in the range of um um a couple of

00:48:14 hundreds of Hertz it might not be that small but it is small compared to the um range of the human hearing which goes up to 20 khz um so um so there is there is a a direct relation very kind of close relation between uh music instruments and the way that that the human kind of kind of vocal system generates sounds when when we speak so there are kind of very very close your explanation by the way mus

00:48:47 why is FL music more erotic how does the brain yeah I want to know why the fruit's erotic too I I think I agree 100% now that you you say it uh why it makes sense to me I'm not that the flux instrument and fluid the flute is more erotic and has more danger therefore and so on um flute first it's a very small distance between sexuality and syncopated breath okay and that's easy to extrapolate and then you have syncopated breath flute playing and

00:49:19 sustained nodes and the history of plucked instruments is one that can almost make that point clear because because nobody thinks of the guitar as sexual instrument until probably Jimmy Hendrick and and that's and there there what's what's the difference because now it's a sustained sound the the electric guitar the Amplified Stratocaster that he was playing can sustain sound much like a flute and can modulate it and synop syncopate it like a breath and nobody can divorce sexuality from rock

00:49:50 music of course without it there was not that much this is continuous quality I'm thinking also the opening of the raps in blue the famous clarinet opening which slides through all of these tones right it's it's sinuous in a way that let's say non-electrified guitar is not that's right and I think um you know your lover does not speak like a like a plucked guitar speak speaks like a sustained uh one would hope I wanted to just sort of go back to

00:50:20 this periodicity in the voice thing and so the way I write a lot of music especially for voice music is I just record myself singing but um I was not blessed with a particularly beautiful voice and so I often sing and then I'll of and running it through some kind of tuning program just to get it in tune enough that it can be tolerable to listen to in my composing and so I use this program called melodine it's pretty famous autotune program it's a algorithm and the weird thing about this and obviously I'm curious with our sort of

00:50:50 more technically adroit people than I is I noticed that if I tune it too much there's like an ex back to moment where it becomes too periodic and it starts to feel totally inhuman and then it feels like a robot and so I notice if I take my voice and I reduce the uh pitch variance by about 50% 60% it still sounds like a me but it's sounds more in tune if I reduce the pitch variance by 100% it sounds completely unhuman and so where is the line where we perceive

00:51:22 periodicity is human and and if it it's two iodic it stops ceases Being Human right there's something about the AP periodic that's important too um yeah so I guess I mean the peach is kind of um um um the physical kind of source of peach is kind of periodicity like um like Peach is kind of a perception like I mean you hear higher sound lower sound in terms of musical definition um and the way the brain kind

00:51:56 of um understands whether the it's a kind of high note or a low note is mostly based on the periodicity so they are kind of right but there's something where if there's not like our brains I feel do this averaging where we s hear someone sing with a little bit of a and we average it out to the slightly higher part and when we have when our brains can't do that averaging it starts to be like a like we don't listen I don't listen to sine waves as as a kind of musical Beauty you know the

00:52:27 like yes sound waves the purely harmonic sound is not very pleasing of course it's the combination of all the s waves that give you the tamber but but I think humans have a res tend to resonate resonance is a phenomenon in physics where you have a driving force that's harmonic and then receptor is also harmonic and when they move together they can transfer a lot of energy and I think that could be an analogy to a lot of aesthetic values where um if things are pure tones at they cannot carry in transmit energy

00:52:58 actually so you need to modulate them and so you need some modulation you need some harmonic content to resonate and in in language in poetry that happens and um I have a theory which is not you know armchair Theory because I come from a world where uh poetry is very important and um in the Arab world um poetry is essential very and Plummer can recite 9th century poet poetry he part of the culture in the in the Persian world the same way and what's what's was common between these there is meter

00:53:29 meter and Rhymes are still there as opposed in Western poetry has been abolished in with u Spoon River what but uh in those culture in cultures where meter and Harmony and Rhymes are important in poetry poetry is everywhere people can recite poetry in the street in cultures where it's not anymore that's that periodicity is I think is essential for people to resonate with with spok Spen language and it go it exists in in song of course but not in

00:54:00 spoken poetry but I guess we're talking about periodicity just in as a supporter of language a lot of what we're talking about is language which grew out of our physical limitations what our vocal cords can do the fact that we need to breathe the fact that we can kind of have these glottal stops to kind of create rhythm in our voice otherwise it might just be a kind of a harmonic pet of sounds um and I was thinking

00:54:31 about because you know this is in the context of sense right and and sound being the primary sense for communication that we have right like language I I feel mostly came from sound right uh eventually it got turned into a written form but it started out um through vocalization and sound and it it's sort of the way we can express ourselves that carries a lot of information that people behind me can hear what I'm saying they don't need to see me if I'm using my hands for sign

00:55:03 language that's more limited but it allows a certain degree of expression um but then more primitive than that is things like facial expression so that's from sight or maybe touch and these very what I would think of as more primitive more early evolutionary um responses actually maybe sound is cuz animals don't have so much in in the way of facial expression right I think uh compared to to the way that we use it to to convey to communicate um

00:55:35 and how much can be conveyed visually just from my face that then would need words to express but then we can also express ourselves through language without words just through sound so there's just such this richness we're talking a lot about music and and S of the science of things but like if we just think about communication um there there's there's just so many ways to to talk about it I feel like and how

00:56:05 I I I was thinking this morning about how sound is such an open-ended sense it sort of begs exploration because we can we can make music with it we can communicate we can um can't think of anything else right now but um that it sort of and and this idea of aesthetic or not I feel like the the grungy sound sound is like any anything that we might find non-musical can be an invitation to make it musical like there to me I don't

00:56:35 think there's any intrinsic like beauty or non- beauty and sound it all that all has to do with how our brain interprets it how it fits into what we're comfortable with maybe we like to be uncomfortable with something that can bring us joy or pleasure um kind of living in the discomfort or that creates tension right so much am music is about like tension and release uh and just sound is is I mean and of course then there's obviously visual art

00:57:05 and everything else but um since we're focusing on sound it's it's it like there's just so many ways to cut into it I wanted to respond to Edgar's point about poetry um because to me again thinking in the Greek context a culture centered on poetry is also a culture centered on music because it has Rhythm and meter and so forth and it's very interesting in the texts of the um fifth century

00:57:38 BCE where you've got this transition from an oral and AAL culture where poetry is memorized and this comes back Chris to your point about memory although a little bit different um but culture is transmitted through the music through the poetry and it's much easier to memorize when there's a a song and a meter and even a tune to it as well and so there's this anxiety during this transition in Greece

00:58:09 of you know we're going to lose our memories now and we're going to lose the transm the ability to transmit culture so it's very interesting you mean with the introduction of writing with the introduction of writing exactly yeah read this wonderful book called Memory Theater by Simon crley which is sort of yeah gets very deep in that um I actually wanted to ask a little bit more about the cuz I I I sort of feel like I know a little bit about Plato and sort of these like what modes you're supposed to use and what modes you're not supposed to use but those modes don't

00:58:40 quite correspond to our current modes or I got nothing for you I'm sorry I mean yeah I can cuz he talked about in the Republic that like you're supposed to use these modes for a kind of uh like an orderly Society he immediately I I find that so interesting that that he went to this effort to to sort of forbid music that might it's almost fascist the Republic clear this is right this is wrong that's it yeah this is why I was asking Jill what was the reason but now

00:59:11 it makes sense the sexuality explains a lot of reference absolutely so I I can't explain anything to you about what those modes are how they translate or don't into any music the just what the what the the social aspect of it that that there would be such a mandate around B I mean you know even in the most fascist modern culture well you know it's interesting because you know I think a lot about um like molini and fascist Italy he really actually hated Opera he was like this is this is you know they

00:59:41 you know kind like the futurist they're like no more pasta no more no more no more Opera we need to kind of move forward here and uh uh it's true no and and and he really um he you know mus hated Opera and sort of this interesting move away from this kind of humanistic version of art where he really wanted to and it's interesting because there were these sort of modernist artists you know Stravinsky TS Elliot who were sympathetic to this kind of fascist cause not because they were you know filled with hatred or anything but just that they were interested in this kind

01:00:12 of idea of modernity and um you know so there does seem to be some kind of connection between a kind of banning of a certain kind of music and a certain kind of social mode that absolutely I I think all kinds of author Arian regimes you know want to they they label certain kinds of art as decadent or impure or you know name your your negative quality um and it it

01:00:43 becomes um a scapegoat of course but the the means through which we're going to sort of yeah purify the citizens or make way for the new man or the new art or or something like that yeah well Tristan you just mentioned how you can almost take any sound that might even seem terrible in one context or one era and later turn it into music and that that goes back to this idea there's always some remainder that we may then be able to fold back into our creativity and it

01:01:14 may be viewed as being subversive that's a possibility right people might want to control it they don't like too much dissonance or well there was this article in the New York Times a couple of days ago by uh John mcorder about how he's trying to teach John cag's 433 um famously a piece that is has no perform sound it's the idea is to kind of hear what's around you and um you know he said and I was I was I was teaching this piece and they couldn't hear the piece because there were all these protests going on outside and uh

01:01:45 and of course a lot of people you know and and and it gets it gets a little hairy I I don't think that's but I certainly think that the obviously the idea of the piece is that everything you hear is part of the piece and I think it's interesting that we're sort of when when someone is sort of taking a work of canonical experimental art and and putting these walls up against it you know there were and and it's interesting because I I don't think he's I think that what he was saying was was wrong but I also think that I don't know if John Cage would have felt it was wrong it kind of gets really complicated

01:02:17 but I do think it's interesting that he's like that's not the piece and like but well what is the peace you know it's there is no peace it's just listening yeah I think the question of how much you can take something that's random or um or dissonant and make it pleasing and or socially interesting and um is a wonderful question and Chris you touched on the fact that sometimes when a composer or maybe even listener can be trained or write in such a way that uh Mak something that we take as noise and

01:02:48 there's a whole Spectrum but it seems to me there's a there is a Sweet Spot somewhere because if you go uh and it's also the political spectrum and the the authoritarian regimes want to have in general something structured and um left traditionally go to something more more random and more free okay so in the history of musical art seems to reflect a little bit of that uh uh the the all the revolutions come when when adding more dissonance on more more however

01:03:20 when you go too far you lose people I mean you don't find people walking down the street whistling shbs of P but you they probably will beting more Taylor Swift and so the question is that if you go too far you lose people if you go too too much to to The Other Extreme where everything is repetive and and resonant and cyclic and harmonic it becomes boring so is how do you choose that spot well I think it's interesting in some ways the the revolution one of modernism

01:03:50 one in some ways a lost so in some ways you think about things like hip hop you have S you have kind of what you know shaer would call music concret using sounds of real life all the time and this is constantly used in contemporary pop music but on the other hand you pretty much have music with periodic rhythms with tonal harmonies and in that way tradition completely went out so I find it to be very interesting that yeah we don't we don't listen to I I can't think of any atonal pop music or even you know

01:04:23 any kind of but I can think of tons of music that would use as sampling that uses sounds of the real world reinterpreted through some kind of lens so I it's interesting right like the the modernism won and lost at the same time um earlier Christopher you were talking about this modulating your voice so that you found it tolerable um it reminded me of um some of the AI audio fakes that I've heard where you can hear

01:04:53 and recognize someone's voice and yet somehow you can tell there's something wrong about the way it's being modulated right the emphasis and give it a few more weeks you think so yeah well but here's here's here's why I'm a little bit of a skeptic about it and that is and I think this goes back to your comment again about cognizing is that there's this rhetorical aspect to our speech and to the way we give speeches which involves our act giving emotional different emotional veilance to something we're saying we get excited we get dramatic our voice Rises it falls

01:05:26 and those are I think largely AP periodic effects that go on and they're involved with the semantics of what's being spoken and not just simply the waveforms that are going on so that's why I wonder about whether that we're ever going to be able to overcome it but it does play a role in understanding because we do um emphasize certain things U vocally but the I think you're maybe the confusing two things one of them is that

01:05:56 you can models you know algorithmic models of speech can always fail to do that but what's evolutionally by AI it's not it's not doing that it's learning how we do it and imitating us and doing it better in some ways um so it's uh it can easily fool you into be em most like the way I syn it easily fool you it can do that and it can also convince you that I'm you know human so this is what this is an earthquake that's hitting Us by the way and uh I live in a laboratory

01:06:26 where things that we could not do 5 years ago now first graduate stent programs the Neal Network and bang it's working so this is a Revolution being us and I warn you not to uh to be uh not warn you I kind of alert you not to think it's trivial I mean it's really the fact that it cannot pick uh that can not fool you right now it's just temporary it's exponentially growing and if you prce it on a daily basis you see this Improvement

01:06:57 so it can really pick up our tone soon enough I know of course you might think I'm technologist there I'm I'm biased i' I like I'd like to be otherwise but the facts are neural networks can do many things like humanik uh which is surprising the question is that what what does it do to composition I think if there's one thing that's constant in the world is that everything is changing right is that you can't like the things will continue to change we'll have new challenges which can be interesting in one context or

01:07:29 dangerous in another or whatever but like these you know things keep changing um I kind of wanted to just return back just for a moment to the sort of basics of hearing like the cckar level of it because we were talking about noise and tone and the fact that I just sort of wanted to bring up that that um white noise like the sound of static from a radio or whatever um is like noise is

01:08:02 actually the just the sum of all frequencies it's all in there and um and I if our ears are are designed to pick up on individual frequencies a certain frequency excites a certain neuronal output from the ccka or however that works um then White Noise essentially stimulates all of them I believe right but at a lower volume if it was stimulating them all the volume that you're hearing right now it would be extremely loud um and that in music

01:08:35 maybe we find some tonal some beauty and tonality but the sound of the ocean the sound of Wind Through the Trees the sound of our breathing just the the breath itself that's all white noise and that's incredibly comforting and soothing there's nothing foreign about that there's nothing unnatural I in my music I have tonal music and then I also work a lot purely with noise with Randomness as musical material and all the time people tell me it sounds like

01:09:07 nature and it's interesting because I'm working with like it's all electronic it's all very unnatural but it has this intrinsic connection to us and our interpretation of it as some as like sounding like nature um so those are two very very separate worlds that kind of combine there um but just like to sort of put a formal definition on it that this this talk is centered on this this sense that we have that's coming from two sources or two ears and that our brain can get

01:09:37 frequency content from that which is what you were talking about how we kind of distill this complex waveform that our brain can't deal with directly like when we hear sound we're not really hearing the waveform we're hearing the sort of like all the individual frequencies of more or less uh that's what's coming to our brain along with this difference between the ears and I remember learning I think this is true that part of how we hear stereophonic sound to know that it's coming from the right side or the left side is that the

01:10:10 the the sound of a a a some sound that we hear it hits our right ear first it hits our left ear a little bit later and that there's an actual linear part of our hearing apparatus where the sound impulse the neuronal impulse or whatever starts traveling this way and then from this year it starts traveling this way and if it arrived here a little bit earlier then it'll hit this one a little bit farther over here and our the inter all time difference it takes about 10 humans can resolve down to 10 microc

01:10:43 difference and also they all locate some because of the level difference down to about 1 d uh the sound is a little bit louder here than here if it's coming from there but what what you what you just described about the and perception of Music U makes me want to ask a question I've read about years ago that I heard that people with cckar damage can find music excruciating and is that true um yeah yeah it's possible and why um yeah the reason is that um probably

01:11:16 they lose I mean we talked about PR periodicity uh I think maybe a kind of another term but ter I use is kind of the rhythmic aspect of uh of the sound um mus most of the music kind of the feeling of music or sense of pitch comes from that rhythmic pattern in the sound that we can call it also periodicity um um so if um any damage to

01:11:47 the to the ca or to the ear um prevents um The Listener to be able to kind of perceive those rhythmic kind of patterns they won't be able to perceive music that well um um one thing about the the noise the white the white noise example that you brought um yeah that was interesting I guess um what we are kind of missing in that um example is kind of the the the

01:12:18 rhythmic aspect I mean if you add Rhythm to noise it becomes more natural and um you can also recreate kind of speech sound uh by just adding Rhythm to to noise to White Noise I Pure Noise you just um make it more rhythmic kind of a kind of a um um a time changing pattern uh which not is not necessarily very fast like in the range of hundreds of

01:12:51 Hertz a few herts until hundreds of Hertz um and that's that's what is needed to kind of produce the sense of naturality in terms of speech and music it reminds me sorry no so is that basically hip-hop like the most popular music in the world right now is hip-hop and a a spoken voice is less periodic more noise-like than a sung voice no not if you're repeating the in the rhyme the

01:13:22 opposite rhyme the opp spoken voice has less periodic pitch than a sng voice but you as in the voice the whole effect is the well the rhm that's saying if you take and the something a something a inharmonic and you add Rhythm to it it becomes music that's sort of that's all I'm saying M yeah I mean we have um um I mean the way we produce sound is that we have the vocal chords that they resonate

01:13:53 like a musical instrument like a guitar string um and um we also have other parts in our kind of vocal generation we have uh we have the whole shape of the mouth we have the lips that we change we have the teeth um so some of the sounds that we generate for speech they are not um um harmonic right like for example the S sound this is kind of very kind of noise-like sound it doesn't have um any

01:14:25 any harmonic any uh like when I say for example ah it's kind of more like yeah harmonic kind of sound so so we have different combinations of um of sounds that are generated by our kind of vocal system in general vocal cords and the the mouth teeth and the lips um um so yes it's it's possible um the the animals most of the S Birds probably it's more harmonic they don't have kind

01:14:58 of that kind of lip movement or teast movement that we use for for speech so it's makesense interesting I in a recent piece used singer sang an S sound like s and ran it into this recording of water like you say it's and it's a very important part of the piece and like Suddenly It's like that sounds like music to people but it's inharmonic it's a rhythmic but people hear it is music somehow and what you're saying reminds me also that in yeah all the cues that

01:15:28 we have to understand uh to turn sound into communication or speech um in early video games there were like moments when a character would be speaking but the the the the chips and the video game systems weren't sophisticated enough to synthesize speech and but they kind of synthesized what sort of sounded like speech and and but the words would be printed on the screen along with what you're hearing and you can you understand it and it's very strange

01:16:00 because you really think that it's speech but then you take away the words and you can't understand what's being said but your brain like fills in all those details I think our brains are so good at forming models of the world around us and it's like when you know when I got the first time I got a MacBook Air or whatever the speakers are so small so tiny they sound so awful at first first I'm like oh is this really how I'm going to listen to music and so many people listen to music on their phones it's like really like this is how we're going

01:16:31 to listen to music now um but over time your ear starts to fill in the details so you can hear something playing off the phone and it sounds to us this is my observation at least I don't know how consistent this is but it sounds to us so full spectrum and stuff because we know what the music sounds like on good speakers and then we listen to it on our phone and our brain kind of creates a model of how it sounds different on the

01:17:02 phone than what it should sound like so we can fill in those details and then we can do that to some other piece of music that we hear on our phone so our like we're constantly and maybe this goes back to memory too or familiarity or something of um filling in the details of the space around us of the people we're talking to of also of like predictiveness of what we expect them to say like this is a lot in AI right is like you know if you if you know someone

01:17:32 so well that you can finish their sentences essentially you have like a mental model of who they are right and what they would say next um if they were to keep speaking and you can kind of like piggyback on that in your communication I guess but um I wanted to hear more Mahan about because if we're talking about sound in our entire experience of sound is through our ears or perhaps through our feet or something if it's a low frequency vibration but like through our ears you

01:18:03 work with these uh implants that in you know that are after the ear for people who don't have hearing you excite the the neurons directly you're bypassing the ear entirely and it's just so interesting to even think about that because it's like is that sound well sure because they it's like that's maybe sound is how our brain interprets these incoming signals and our brain already is ordered

01:18:34 structured enough when we're born it's not just like a gray Mass there's there's parts of our brain that process different things differently um but I was just wondering if you could speak a little bit about like what on on like a philosophical level almost like what is sound if you don't have your ears yeah I tried to start with that the beginning that um um kind of um the way we perceive sound is kind of um to be separated from sound itself like in the example that you said with these

01:19:05 implants we don't really need sound to produce auditory sound perception right um so the vibrations or the sound that comes in the physical world um that's one part of the story um but as soon as it that sound gets into the brain um many of the uh those vibrations vibration aspects or many of those properties they change in the when the brain processes the signal like um the

01:19:36 very high frequency kind of vibrations they don't exist in the brain so there are oscillations still in the brain there are rhythmic patterns in the brain but are much slower than the actual sound um so so you're right that the that the brain has to make next sense of the sound and it has to fill in um um kind of the information that is missing or is not available or um and and it is what the brain is doing all the time for

01:20:09 people who are kind of hearing impaired their brain has to do a bigger role but for for all of us when we are like in a noisy environment or an environment with kind of very strong retion like strong Echo um um our brain has to work harder to be able to kind of follow discussions and communicate with others and and so it has to kind of work hard to kind of get whatever it can from the from the signal and kind of feeling

01:20:40 whatever it doesn't perceive from the signal based on the context based on the experience uh based on um um whatever is happening inside itself um all the experiences it has had recent or in the past this ties up to the early uh early discussion these these blue squares and rectangles on the board these are actually serving spatial sound functions these are fiberglass boards absorbing the those Reflections

01:21:11 that we uh talk were talking about and if we remove them the conversation will be much harder to understand because what you ear uh what you already talked about the things become grungy because of all these Reflections uh they without them they will be very hard to have it much harder and your brain will have to work harder to get the uh um conversations out because of the din that's created by these early Reflections this is what exactly what they are there and many restaurants could use some of these I was just going to just ask um

01:21:44 but there's people who receive cckar implants who have never heard before right so what is what yeah like they they can't fall back on the skills that they've built earlier in their life in order to interpret this this changed signal that might not have as much information as it used to used to before or as our hearing gets worse it it's harder to hear in a noisy situation like a restaurant or something or more Reflections but for

01:22:17 people who've never heard before what like what is that that's a new sense suddenly cuz you've tapped into the brain at that point it's yeah um I mean that's a very good point I guess we u i mean there are a lot of unknowns but what we can probably say is that um the people who have never heard before uh when they uh kind of start getting the auditory signal start hearing something they need time to make sense of it I mean their brain has never

01:22:48 heard doesn't have any idea about hearing or speech or kind of verbal communication in terms of hearing so um they need time so they need time for the brain to understand what is the signal what does it kind of associate what they are hearing with the external world to be able to relate um the sound to for example speech a speaker or music or um any kind of external kind of things that

01:23:19 happen externally are there any experiences that you can share from things you've heard with people who are you know like what's this like for people because it just I just wonder what that what's that like um yeah I should say that um um um most of the people who get the devices are have been hearing before and the reason is that um for speech um there is a critical period in in life so there is like the first

01:23:50 five six or seven kind of years of life that are very important important so if you lose if you cannot hear during that period um you would not be able to learn to speak and understand the speech later um so so that makes things complicated like if um as a kind of adult um getting getting an implant you would not expect them to be able to understand speech but they will hear sounds and they might be um able to

01:24:23 assess sounds nature um but in terms of speech understanding that's kind of impossible but there are kids younger kids who kind of get the device at the age of four or five um um and in those cases um um um yeah I mean um it's a very um interesting situation like uh the first time that the kid starts here in um yeah

01:24:56 it must be a magnificent experience I mean they kind of hear their the mom's voice for the first time they kind of family voice I mean it's kind of um sometimes they can be very emotional kind of those one thing I'm sorry I was just going to say that reminds me of um how hearing is the first sense that we experience right in the womb I think or I guess touch or something thing touch obviously but sorry go ahead I was to

01:25:28 open up to so oh I'm sorry did you me I it's going to maybe open up a can of worms but maybe it's a can of worms that the audience can jump on um we've been speaking in really positive terms about music and Aesthetics and gaining the sense of hearing and we've also talked a little bit about politics and so I just wanted to throw out there a couple of examples where um Sound and Music have have been used um for political ends in a not so

01:25:59 nice pleasant way and the two I was thinking of one is um police forces using very loud sounds for crowd control um and then the other is um the use of music pop music in some instances as part of no touch torture um and um you all might know the work of an cusk right here at NYU who's done she's a musicologist and she's done a lot of work on this but the way in

01:26:31 which um through both volume and other aspects repetition this has a way of literally breaking down someone's subjectivity um in the process of torture and so I just wanted to kind of throw that out there as like well okay here's another aspect of how humans have um used Sound and Music that's maybe not so pleasant yeah well I wonder if anyone

01:27:02 from our audience may have a question they want to pose to the to our panel if you do please come up to the microphone hi I just want to make a quick comment that John mcar piece that you mentioned um it's just peculiar but uh there is no peace there is no peace that was what the protest was saying Pac CE what he you were saying p i c and uh and that really resonates in a semantic way with what cage I think was

01:27:35 after as you were kind of equivocating about that but uh what was the piece meant to do when cage composed it uh accounting for the whole environment of the experience I just found that really fascinating because that's all kind of cognitive semantic stuff semiotic even but we were talking earlier at the very beginning about the string vibrating you pluck it it goes bounces back cancels out most of the sound except for certain

01:28:07 values which are preserved uh That's How Sound Works but then cognitively same thing is happening and there we have some selectivity about what we cancel out and what we keep as meaning and what mcarter was doing was canceling out a whole set of things the political aspects of things he didn't agree with while trying to experience the other side and as I read him I mean I I thought what a mess he's making of himself which usually he does not which

01:28:39 is what I thought you were saying too I just wanted to kind of get amp was I was saying it I just think the I mean I could talk about John Cage a lot but you know there's a famous PC bro called indeterminacy it's a lecture and he talks about seeing a piece by another composition and you know that someone said he said you know he's describing the story of himself and also I'll play him for one second and he I said I said you know I like the piece but I hated the introduction and uh you know JN and John Cage said I I I I I he

01:29:12 said that I think that my piece is designed to reduce all the there's too much suffering in the world and cage says I actually think there's just the correct amount of suffering in the world um so there's something about this that makes me feel like there's a dissonance between the idea of and cage was notoriously sort of annoyed by protesters and so I I think that inadvertently mcart was on to something but I also think that cage was smart and Savvy enough to know that when you open

01:29:43 up a work of art to uh I mean it's this funny ideas is this sort of I think the interesting thing about John Cage is that he was a composer who was constantly trying to in some ways step out of the social aspects of he's trying to remove the social aspects of music from music and that's why his work is both so resonant but also somehow I think very very misunderstood you know cuz he would sort of be like my work is chaos but it's my chaos you know it's not your chaos stop ruining my chaos and

01:30:13 so that's why I think that inadvertent like MCM morar was sort of onto something but in it was couple of orders I don't think he even realized this but it was sort of several orders of magnitude above what he was saying I think cage would have both accepted the protesters as part of his piece but also been a little bit annoyed about it so everyone's a little right here anyone else a few years ago we had uh Yuri Hassan who was in the Princeton Neuroscience lab un sound and especially

01:30:45 communication and he had persons in two different MRI machines and asking them to concentrate on the part of their brain from which they spoke and from which they heard and it was the same part of the brain and then he extended that to a speaker with an audience concentrating on the part of the brain from which they were speaking I'd love to know what you think about that and then I have a quick question which is um what happens if you put a human being in a soundless space if

01:31:19 there's no sound you had mentioned something about that to me before Chambers uh I answer the second part because that's a technical one and I know a little bit about uh so an you can build an ano chamber and isolated from from the rest of the world so you can bring down the sound level to a minimum what happens first is UN unnerving but then you start hearing uh your heartbeat very loudly you can even feel you hear the the circulation of your blood and it's not we're not designed to be in that environment so we we U I designed

01:31:51 an e Chamber from my lab U but because we do a lot of subjective testing humans we made sure the sound is not completely isolated from the outside so and we put green lights so when you come in initially it's unnerving but you relax and and it's fine but completely is sound isolation is is almost a form of torture to go back to the other side of your equation you use pop music to torture someone at loud volume you can use silence and Tor tort torture someone we don't like isolation as a humans and hearing is something we can't readily

01:32:23 turn off so well like more or less close our eyes and block out our sight but hearing is always on and people who suffer from ttis for instance is similar to this if there's a ringing in your ears which by the way is created right from the it's a neur neuronal malfunction in in in the ccka right it's um that's always just outputting um that that that can be something you can't turn it off so it can be very irritating much like to some Maybe the sound of the

01:32:54 city or a banging radiator or something like that which to John Cage was and John Cage was like had mentioned those tones that you hear in an anake chamber and that that was a meaningful moment for him I went I was in an anake chamber once and my takeaway was that when I spoke I didn't hear my voice from the room at all it didn't give me any of that so it all I heard my voice coming through my head it was so strange and it fundamentally changed me and now

01:33:27 as I'm speaking right now I hear the sound from inside of it's it's something very strange these are very unnatural situations yeah well there's um there's a piece called I am sitting in a room by the composer Alvin lucier very famous piece where the composer sits and reads a little speech saying I'm sitting in a room if it was in the same one the same as you're are now I'm recording the sound of my speaking voice which I will play back over and over and over again until the resonant frequencies of the

01:33:58 room Are All That Remain so you know he plays this recording puts the mic on and records that recording he plays that back play records the playback then he records that playback and eventually by the end of the piece after about half an hour all you're hearing are these kinds of resonances of of the space and it's was kind of it's incredibly beautiful pieces particularly because he's a known stutterer and one of the things about this piece is sort of you wind up ironing out and the the stutter itself becomes a rhythm but

01:34:29 anyway so there was this kind of marathon a couple of years ago of um this piece all these composers performing the piece in different locations and this very cheeky composer Professor named Seth Kitt performed this piece uh in an echoic chamber so the piece never changed they just kept on playing back exactly the same over and over again which I thought was both very very cheeky and very funny but it's a little

01:35:03 Annoying yeah I don't think we answered the first part of your question what was I I didn't quite understand it I would just a footnote that no please come over to the mic because we can't uh you were talking about in the beginning it's called the harmony of the world and was not a invention but Pythagoras actually spent half his lifetime in Egypt and then in Babylon yes and that is the source and the reason why it is a guitar

01:35:35 and not a wi instrument is because it is visible and the basic idea of the harmony of the world is the harmony of the world consist in the fact that what is visible and what is uh only in sound you could say are comparable not only comparable they're exactly the same when you measure velocity and when you measure rhythms and so you're using the same units and you can see them on

01:36:06 the on the wind instrument you cannot see them and that's why okay Finish New York which is the city of Edgar there is one of the students of Edgar Andel cage was a a a musician Max no house who gave up his formidable

01:36:40 expertise as a percussion is for making sound pieces which were not music there were s pieces and there is one in New York in Time Square Time Square is two triangles like this if you stand on the southern triangle you will hear a piece but there is no note and no sign because Max wanted people to discover themselves which is difficult the other pieces in

01:37:10 De Foundation if you go there 10 minutes before the hour it's like below you the the installation at Time Square it's the installation is under the grill of the subway I once walked past that at about 3: in the morning and I I literally thought I was hallucinating and only realized a couple of days later that it was real you many museums now have a a sound curator and there's a sound art as a whole discipline um different than different than music and but he invented

01:37:41 this idea of sound pieces yes other people then there's another piece like this also in the foundation to go back to Beverly's question which I think relates also to um how the brain processes sound and whether speech and sound and to what degree they overlap in the brain I just wondered um I guess I was just sort of thinking about this idea of we we sort of all touched on this and didn't really

01:38:11 get to talk about it but just this idea of of of the the eyes as a part of sound as seeing as a part of sound um for example if you're at a concert you know there's all sorts of ambient noise that you sort of automatically tune out as we were talking about um and you sort of just focus on the you know the concert on stage or the and the sound and it's very very easy act but then I don't know I I find the act of listening to a live recording to be incredibly difficult comparatively unless it's been denoised

01:38:42 unless some of this noise has been removed it's such a harder act and so I think there's something incredible about the fact that we do like seeing our seeing does affect entirely the listening experience some and you know or there is something so much about the visual Act of seeing someone playing or something that really does I think somehow allow you to you know I I I I I I think an engineer once described to me as eyes as the great compressors of sound and that they focus you and they you know without visuals we cannot locate sound very well as a matter of

01:39:12 fact I want I give talks to students about the about spatial sound I asked for a volunteer and I set them in the front and ask them to close their eyes and I make sounds like this and ask them to point now humans as we talked about before use inter all level difference inter all time difference they use spectral cues depending on the shape of their P to locate sound but with that visuals they fail very sometimes they go like this they point there and um and there's something called the murk effect I don't know if you heard the Mur effect is is incredible there's a wonderful YouTube video by by the BBC that

01:39:45 two-minute video that shows it as a professor from USC was King on the beach I think USC one of the universities in California and he he says the word b ba ba ba and you hear ba and then he says the word fa fa fa and you hear fa what's the big deal then they tell you the audio never change he's always saying ba but the video they change it and he's now saying fa and you will hear fa and everybody in the world cannot but over here and cannot stop it and will hear fa

01:40:16 as a matter of fact they give you split screen they show him here saying ba ba ba and fa fa fa and they keep the audio the same at any point you can look at one part of the screen and you hear a different sound it's because you and this is a very welln Effect called M effect and that's because most likely evolutionary wise We Trust our visual sense much more than the than the audio sense if the because the wave Bank of sound can wrap around can be on the order of centimeters and meters while um light has nanometer goes directly onto your um Iris and retina and recorded by

01:40:48 the brain so if the bear is coming at you you better trust why you see him coming at you not what the sound is recating from and we are the we are the uh we are the the grand kids of people who learned that lesson all the others are gone they've been eaten by bears this is why we over we when there's a conflict between audio cues and visual cues we override to the visual cues they're much more reliable for survival yeah all I can say is Bucking AEG um one of they go back one more time because we've been one of the things that's come

01:41:19 out of this the this series of talks is this idea of sort of a general uh perception a general sense that goes beyond each individual sense and when when you ask questions for example what what um what was it what would be like for someone who's been deaf to hear again one of the I think answers that come to my mind and I of course who knows what that is we don't know that exactly because we're not but that the brain has a general pro way of processing sensory experience I wonder

01:41:51 whether it extends fully to things like speech it seems it does not you can think that whatever processing that goes on in speech does not is not in included in this General sense of sounds overall I wonder if if that makes sense and partially responds to Beverly I was just thinking that one thing we've learned from AI or at least a lot of the way that current AI works is it starts with noise this is my understanding and it says this is a little bit familiar and it amplifies that familiarity and it

01:42:23 sort of like eventually comes up with something realistic looking and and we do that all the time like we think we hear what someone's saying and we might be wrong um so we're like what you're talking about we're like literally misinterpreting what we're hearing um but we can also look at the clouds in the sky and see images and stuff so I feel like we can't separate out that stuff it's like we we can't

01:42:54 see without seeing people in space and our brain does all this pre-processing before our my understanding of it is before our conscious brain or Consciousness or something like that gets to interpret it and so we're fed these high order things like this is a coffee cup or I'm speaking or whatever and um I guess and even when we're listening to the sound of the ocean or wind in the trees like we know what that is it's I guess it's uh when like a lot

01:43:24 of noise music or music that um doesn't have recogniz you know it's not coming from a recognizable Sound Source that's when we get to listen kind of in the abstract and just hear sound perhaps I wonder how how much we're still associating with other things I don't know if that's what what you're talking about and by the way we ask people to come to the microphone because these are recorded for for YouTube and and your voice doesn't pick up unless you're the

01:43:54 micophone comments are making me think of something that happens to me every once in a rare while I'm wondering is it familiar to you guys and is there any explanation for it sometimes if people are speaking in the far distance and it's faint I notice that I can't understand what they're saying because it's too quiet but I'm also a musician I work with electronic sound I have pretty well

01:44:24 trained in kind of hearing sounds as sounds and I realize that I can actually hear the sounds of what they're saying you know the syllables a little bit of vowel well enough that in principle it's enough information to reconstruct what they're saying but somehow my language centers need it a little louder to be able to work of the weird experience

01:44:55 that I can hear what they're saying as sound but I can't interpret it as English if it's at this certain threshold of faintness yeah so um yeah that that's absolutely right so um I mean the brain the kind of language kind of understanding of the brain um it requires some certain kind of um um cues in the sign to be able to kind of interpret it uh if those cues are not

01:45:27 available or are faint or are not reliable um then it would be difficult for the brain to make sense of it um but um I guess prob you have experience that sometimes if you have an idea of what the person is talking about or an idea about the the kind of the context or uh or especially if you have kind of are able to see of some part of the the leap of that person kind of from far away uh

01:45:58 they will kind of help dramatically suddenly you feel that you you can't understand much better it's called the cocktail party effect which is actually very serious uh research going on in it on the Neuroscience side especially uh and the cognitive science side a cocktail party effect is that you can sit next to someone like J is sitting next to me and let's say Jill is talking to Ed but I'm I'm I can have a conversation with someone much further complet complely block Jill's conversation and um and be able to concentrate on something else and people

01:46:28 still trying to figure out partly some visual cues can help and so the loudness is not can be her voice will be much louder than the person across the the street or the table and that's um very very tough problem to understand I mean there been a lot of research on it I'm working on a hearing aid right now that is based on the cocktail party effect where you put you put hearing um you know earphones in your ears and you look at a person and that person's voice will be Amplified much more it's using a

01:46:58 technology called higher order ambisonics that that can code sound and spherical harmonics and isolate the beam of sound but it's kind of mimicking the party effect but without the brain trick all purely physically based on physics and by the way when you were talking about how the brain puts together the visual and the Sound Source I realized that I'm doing that today each of you for me especially sitting over there is louder out of the speaker than out of your own voice but when you speak I hear it coming from wherever

01:47:29 you're sitting wouldn't you say also that when you had that experience where it was a subliminal like you heard the sounds and you but that once in a while there's a certain threshold where you might start to get a thread of what they're talking about and then suddenly the rest of it unspools and you could follow it I think that's likely and I certainly think if somehow and this wouldn't happen real life I had a transcript description words I would easily have got it but there's this just threshold where I'm actually get aasia because it's loud enough to hear but not

01:48:01 loud enough to interpret very interesting that reminds me a little also of what when you walk into a restaurant or a bar and you can immediately tell if the music that's playing is live it's just different and even if the live music is coming out of speakers or something like that um you're your brain can tap into something very subtle there anyone else

01:48:34 please thank you all um I just want to comment that I have two cousins that were born DEA and about 20 or 25 years ago this one cousin who taught at galet she had I don't know if it was the implant or what device uh that she was given to be able to hear for the first time and we had this whole email discussion and I was so uh disappointed that she was so disappointed with hearing and I asked her what specifically and it was the sound of children gu a high pitch it was a

01:49:05 disorientation of not knowing where the sound was coming from dogs barking and she just thought sound was ugly now she was very she was kind of an activist in disabilities and all so it's a very tight Community the deaf community so she very quickly you know um rejected the the whole idea and is very good you know reading lips and we communicate and sign all that but um her sister-in-law also married to my other cousin who's deaf two two uh siblings uh she had an

01:49:38 accident at age six or seven and so she went for it but because of what you had pointed out earlier if you had some Hearing in your earlier life you could adapt and adjust and that's what happened but she actually rejected it too because then she was um a kind of uh the Deft Community one is a very close community and they kind of reject the fact that you know you're you're you're turning you're turning into a um you're just changing your lifestyle and your whole social makeup

01:50:09 so that's that the other thing I just want to ask is this I just came from a place uh that there has been jackhammers for for months and I I I can only relate to people in war zones because the it's I had some fitted ear buds by oologist here in New York that were no better than cotton balls and I just wonder and it it reminded me that Sound is like water and moisture it wherever it can go it's going to go and I and I guess this is the vibration level or what but I just want to know are there any earbuds

01:50:40 or headphones that absolutely other than like an echo chamber which is something different Al together that can actually eliminate all sound I would love to know and I'd pay any amount of money for that and one last thing about the Jack Homer experience it also made me connect to people in war zones even if you know bombs that are dropped are not going to explode just that sound a post-traumatic stress disorder I swear if I was going to be there much longer I would have developed that because sounds were

01:51:12 making me really jumpy um throughout the day because of what I was hearing for most of you know when I wasn't even in that environment so those are my comments and questions thank you appreciate it well there's something so precognitive about sound right like that it's uh or pre-linguistic I I had to live in Flatbush in Brooklyn and uh it's a really loud neighborhood and I used to have this guy who used to drive up every night and I think he was parking his car waiting for a spot and would absolutely blast music so loud that our walls would

01:51:43 shake and I have to say that it was the the most I mean like classic New York but like I hated that guy I hate that guy more than any I hate anyone in the world just because there is something like like you were saying before like there is an assault that sound can be that really is more powerful than language assault it is this kind of physical cuz it is vibrations and it's um but I think about that guy a lot and I still I still hate him it's very tough yeah I have to think that the the actual physical vibration

01:52:14 has a lot to do with what's also disturbing about it besides just the sound well cuz I had these noise cancelling headphones and so I canceled out the music but my body would still sh there's the related issue that I think is more purely sound related which are which is the sort of the Continuum between I always get the letters Wrong Am Sr that audio and and then misophonia which is the feeling that certain sounds usually their high pitch sounds are very irritating so there are people who like those Amplified lot of

01:52:46 more high pitch sounds I can't stand them I mean I have a misophonia about those videos that show a Sr but I wonder you know if there's any thoughts about that well my reaction to that is that similar with our visual palette that the color red might indicate danger or blood or whatever this or lust I guess but um that in sound world we have these similar things like high frequency sound or the sound of babies crying like

01:53:18 we humans I imagine have evolved to develop a sensitivity to that specific sound because it's really important it signifies importance by the way there's a direct relationship between the color visual color and the loudness of sound there's a classic paper in in in Acoustics where uh in perceptual Acoustics uh where the somebody it's a it's a academic research paper where they have bunch of people look at uh a picture of a train passing by and the headphones and rate the loudness but

01:53:49 every time there's a random color on the plane and in variably humans would put the red train at the highest sound and the loudest sound so the brain does all kinds of Tricks combining and also the brain in terms of loc you talking about local localizing sound uh the brain can adapt also if you take your p and bend them like this and then um somehow keep them like this for a few weeks not a very uh you'll you should not be tosing the streets uh you should be very careful because then you you you'll have a lot of problems locating sound okay eventually however after a few weeks

01:54:20 depending on your age and so on you adapt and then you can everything was fine and then if you let go again you need to spend few weeks not crossing streets because your brain this this part of the ear is very important for localizing sound and high higher frequencies as the sound comes in it gets defract around these ridges why we have these ridges this is why the SE is not symmetric and it attains different uh tonal content different harmonics by the time it enters your ear canal and humans use that partly to

01:54:51 locate sound because sometimes the inter all level difference all time difference are not enough especially if the sound is right in front of you there're zero actually so this is called spectral cues and the human brain is relies on that to locate sound and but it can adapt beautifully human brains is a very flexible powerful it's like that guy who wore glasses that inverted the world and he got used to it and then you take that off and you see normally and it looks inverted well I think um this just about

01:55:23 wraps up one of the one of the best talks I can remember in a long time really appreciate this and um I hope you all go home whistling 433 uh it's an easy one just but anyway uh we look forward to seeing everyone on May 11th for vision thank you Jerry okay sorry

01:55:58 you see your for