Comment on Artificial Intelligence learns music

BenRayfield Wed, Aug 18, 2010
Gamma I removed that code you copy/pasted into this thread because SpaceCollective is not for talking about excessively technical details. I will consider joining your Perceptron project because I watched the video and want to connect it to Audivolv (my software this thread is about) in an artificial intelligence way. Lets write summaries here and talk about technical details in private messages.

Audivolv is mostly about interaction between mouse movements and generated sounds (44100 calculated amounts of electricity going to each speaker per second for 44.1 khz sound) evolving into a "multimedia playground" where mouse and speakers are the media. Add Perceptron to that and we have 3 media, all interacting through the same artificial intelligence code. Our "playground" expands. Its about the psychology of people and artificial intelligence interacting, and I see a lot of potential in using your realtime fractal graphics as a visual psychology interface.

My long-term plans for Audivolv are to use it to evolve new kinds of brain simulations (more advanced than neural networks). Not Human brains, but new kinds of simulated brains that have their own unique abilities compared to Human brains, something to work with us instead of replace us, to fill in the thought processes that we have problems with and for us to do the same for my many new species of simulated brains. How? First get Audivolv to evolve musical instruments that people play with the mouse. Then use that exact same code to evolve new patterns of brainwaves that parts of the simulated brains play with their simulated brainwaves. Mouse left/right and up/down (changing 100 times per second) are 2 dimensions. Electricity in each speaker (changing 44100 times per second) is a dimension. Its a general artificial intelligence system for movement in dimensions, so the same code that uses mouse and speakers as musical instruments will work for electricity waves in simulated brains. Neural-network software does not simulate 44100 electricity amounts per second because computer programmers think that level of detail is wasteful. My long-term goal for Audivolv is to evolve such simulated brains and brainwave patterns at such a high level of detail that you can choose any simulated brain cell and listen to its brainwaves on the speakers. We can do experiments with brainwaves that sound like music, but I expect most of them will sound like electricity. My simulated brains will be so detailed that they will have an EEG and if you connect those EEG-measured brainwave patterns to a psychology software, you might figure out what its thinking. Add this to our "multimedia playground" too.

It will be fun to evolve the simulated brains the same way we can evolve musical instruments in Audivolv today, but there is something much more interesting to do when the project advances that far: Get the simulated brains to design more simulated brains. Audivolv is already a Java software that creates Java software, so theres no technical problems to solve to get the simulated brains to build more simulated brains, except the technical problem of creating the first few simulated brains. When simulated brains design more simulated brains, they will learn to become smarter without limit. We should do the same.

UPDATE: These are the plans for the next version of Audivolv and how it will evolve very simple statistical simulations of the Human user's mind to evolve better musical instruments that they will score higher:

Audivolv uses GigaLineCompile for evolving new Java code. There are a few kinds of objects it evolves code for, including:

* InstrumentFunc = Code that is wrapped in a jsoundcard.SoundFunc is called by JSoundCard in a loop many times per second. It receives microphone(s) amplitudes and calculates speaker(s) amplitudes.

* MusicianFunc = Code that tries to do the same thing the Human user does. If an Instrument has 2 microphones/inputs and 3 speakers/outputs, a Musician would have 3 microphones/inputs and 2 speakers/outputs, because each audio channel is paired with its opposite. A Musician hears speaker(s) and calculates microphone(s) amplitudes. The Human user is like a MusicianFunc, which is used through JSoundCard, but that MusicianFunc can not be used directly because instead it calls an InstrumentFunc in a loop controlled by things which are affected by sound-card timing. Normal MusicianFuncs simulate what the Human user would do so a few possibilities can be tried at once instead of only what the Human user is doing at the time. Whatever Instruments the simulations like best are what the Human user gets to play next.

* MeasureFunc = Code that hears microphone(s) and speaker(s) simultaneously (both as inputs) and outputs an interpretation of them as 1 or more extra audio channels. Other code uses the outputs of MeasureFuncs to find good combinations of MusicianFunc with InstrumentFunc. Whichever MusicianFuncs are best are used to find the best InstrumentFuncs. Whichever InstrumentFuncs are best are used to find the best MusicianFuncs. Its a cycle that finds good MusicianFuncs and InstrumentFuncs. Its also a hierarchy with the Human user being the top MusicianFunc. MeasureFuncs evolve to represent different parts of the Human user's mind. Example: A MeasureFunc that calculates the average volume (or the opposite of that would work too since its statistical relationships) of speakers/microphones together would be useful for excluding MusicianFuncs and InstrumentFuncs that don't play any sound. The way that works is the Human user would tend to score InstrumentFuncs low if they don't play sound, and those InstrumentFuncs would tend to score low (or tend to score high would work too, as long as its consistent) in that MeasureFunc, therefore that MeasureFunc would get a higher score for being more relevant (positive or negative relationship) to what the Human user is thinking. MeasureFuncs can be evolved for any interpretation of the sounds (within the limits of the code evolution system) or manually written. Example: A MeasureFunc could do a fourier-transform on the audio channels and calculate how much of a certain frequency there is. That could be useful for excluding annoying high-pitched sounds, but I expect the code evolution system is flexible enough to learn to measure frequencies approximately.

Its a bipartite graph (2 groups of nodes where each edge is between a node from each group) where the 2 groups of nodes are InstrumentFuncs and MusicianFuncs. Each edge has a MeasureFunc and the score it calculated between the 2 nodes (an InstrumentFunc and a MusicianFunc).

The Human user is the RootMusicianFunc and the scores the user chooses have RootMeasureFunc as their MeasureFunc. Each of those scores is between RootMusicianFunc and the InstrumentFunc that user was playing when they scored it. All evolved MeasureFuncs are for approximating RootMeasureFunc or for being used in groups of MeasureFuncs to approximate RootMeasureFunc in a bayesian or other statistical way.

It would be more accurate to call it a tripartite (3 groups) hypergraph instead of a bipartite (2 groups) graph, where the 3 parts of each edge are MusicianFunc, InstrumentFunc, and MeasureFunc, and each edge also has a score (number in range -1 to 1). That way, the 3 populations of evolving codes can use the same algorithm. It can also be stored symmetricly in a database table with 3 columns for the 3 types and 1 (or more?) column for score. In a symmetric way, the statistics of how the other 2 things are related to others of the third thing can be used to choose which of the third thing to evolve with each other.

The result will be a system that has goals about its goals about music, a brain made of 3 similar parts specialized in music. The MeasureFuncs will be its emotions, and the MusicianFuncs and InstrumentFuncs will be its normal neurons and mirror-neurons. From what I've read, mirror-neurons are for thinking about how others think.

It will be a system that can learn while you play its intelligent musical instruments, or it can learn on its own without you so it sounds better when you come back. Later versions will have more evolved systems built on top of this primitive brain made of 3 parts.