Render’s favorites
Recently commented on
Algorithms are the intellectual currency of the future
It is undeniable that the world has grown increasingly complex, and it has become increasingly complex to live in. I mean this not only in the sense of there being more and more factors and information to know of and respond to, but in the sense that new technology and global interconnectedness creates new and increasingly bizarre situations, which pose new moral dilemmas to each person that lives here. We therefore have no choice but to invent new and more poignant methods of thinking and making decisions about the world.
What we truly would like to study is thought itself, but it seems at the moment that no one can yet formalize the notion of a thought so that it can be attacked mathematically. If someone could sit down and explain to me precisely what is meant by a thought, precisely what an idea is; if someone could say for sure what things can and cannot be ideas, in a comprehensive way... I would be very happy and truly amazed. There are many who have thought about this problem, and I am very interested to read Eric Baum's new book, previews at http://www.whatisthought.com/. However, while there are plenty of ideas, there is nothing that quite answers the question with confidence at the moment.
A related idea is the idea of an algorithm. The precise notion of an algorithm is an informal one but a crucial one in computer science. It is effectively a method for solving a certain problem, making a calculation or decision based on some information, and describes precisely a sort of reasoning strategy that could be implemented by a programmer for any computer programming language. Every program has an underlying algorithm — but there are many, many ways of implementing any algorithm in any particular programming language. Proving a program is correct typically amounts to proving that it correctly implements the algorithm. An algorithm is only worth anything if there is a proof of correctness — the discovery of this proof is synonymous with the discovery of the algorithm. In this way, algorithms form a link between the working processes of real machines (or real people) and mathematical truth.
The concept of an algorithm *has* been studied in great depth and formalized with the idea of a Turing Machine, the foundational concept of computer science. This has become the widely accepted formalization of the idea of a computer, and it is now considered that an algorithm exists to solve a problem if and only if there exists a Turing Machine which solves the problem. The study of computational complexity is the study of resource-bounded computation, and attempts to determine whether a given amount of computational resources (time, space, randomness, communication, quantum entanglement, and it gets more exotic from there...) is sufficient to solve a problem.
Humans seem pretty naturally well-suited to the task of conceiving of algorithms for certain problems. Finding the most efficient algorithm is still very difficult. However, one of the major problems in this field is proving that for some problem, an algorithm cannot exist, or that a better algorithm cannot exist. No one really knows how to do this well, and ultimately, it is due to our fundamentally poor understanding of the limitations of algorithms. Compare this with our (or at least *my*) poor understanding of the limitations of our thoughts.
One hope is that in addition to each of the resources used by an algorithm, we can discover other metrics for the progress of an ongoing computation, the information content of the data being used by an algorithm, or some other sort of computational invariant, and prove strong bounds on its growth; then, if solving a problem requires much of this invariant, it cannot be done quickly by any algorithm.
The discovery of such a measure would provide tremendous insight into the nature of computation, and, I believe, of thought. Especially if we could characterize exactly which problems have algorithms in terms of it. Of course, this would represent a tremendous scientific breakthrough.
What is the value of a new algorithm to humanity? At least as important as the ability to use it is the insight we gain from understanding it. Algorithms work by exploiting latent mathematical truths, and moreover, proofs against the existence of algorithms are proofs against the tremendous power of creative mathematics.
There will come a day when Moore's law has bottomed out, when what are essentially the best computers we can build have been built, when the top of the line machine does not differ in computational power from the everyman's PC, when cryptography relies on problems with no efficient parallel solutions and the largest computer network at the government's disposal will be no better prepared to crack state of the art codes than an affordable machine owned by a smart civilian. When this happens, the cryptographic arms race will rest solely on the algorithms employed by the code makers and the code breakers. Individuals will enjoy the same level of privacy as the government, and cyberspace could never be owned by legislators the way it is today.
When that day comes, when people try to handle problems that cannot be solved by throwing hardware at them, algorithms will be only relevant factor. As more and more jobs become automated, but NP-complete tasks like Theorem Proving remain intractable for computers, the discovery of new algorithms will be a central task for human beings. We have already seen the day when wars have hinged on the discovery of effective algorithms, when hundreds of German U-boats are destroyed because of insufficiently secure codes. We are currently seeing the day where hedge funds rise and fall on their ability to perform statistical analysis as accurately and robustly as possible on fantastic amounts of data in an economy that only grows more and more complicated. It is inevitable that algorithms will only increase in importance.
This is not to emphasize the existing uses we see for algorithms as the most important. On the contrary, I expect that the most fantastic and important uses for algorithms will be things we have not seen yet. The rise of computer games and computer simulations which feed the imagination is already on its way. The potential for algorithmic art is vast and wide open — we have already seen the beginnings of algorithmic music breaking onto the scene. Artificial intelligence research continues to inspire believers in a future previously only dreamed of in science fiction. As we speak, hundreds of researchers all over the globe are fine-tuning solutions to the various problems of computer vision.
In short, I don't know entirely where it is all going, but I am convinced algorithms are going to play a major role, and will at some point represent a major portion of human creative energy. In doing so, we will learn much about ourselves and our limits, and maybe come closer to really understanding what it means to think after all. With any luck.
What we truly would like to study is thought itself, but it seems at the moment that no one can yet formalize the notion of a thought so that it can be attacked mathematically. If someone could sit down and explain to me precisely what is meant by a thought, precisely what an idea is; if someone could say for sure what things can and cannot be ideas, in a comprehensive way... I would be very happy and truly amazed. There are many who have thought about this problem, and I am very interested to read Eric Baum's new book, previews at http://www.whatisthought.com/. However, while there are plenty of ideas, there is nothing that quite answers the question with confidence at the moment.
A related idea is the idea of an algorithm. The precise notion of an algorithm is an informal one but a crucial one in computer science. It is effectively a method for solving a certain problem, making a calculation or decision based on some information, and describes precisely a sort of reasoning strategy that could be implemented by a programmer for any computer programming language. Every program has an underlying algorithm — but there are many, many ways of implementing any algorithm in any particular programming language. Proving a program is correct typically amounts to proving that it correctly implements the algorithm. An algorithm is only worth anything if there is a proof of correctness — the discovery of this proof is synonymous with the discovery of the algorithm. In this way, algorithms form a link between the working processes of real machines (or real people) and mathematical truth.
The concept of an algorithm *has* been studied in great depth and formalized with the idea of a Turing Machine, the foundational concept of computer science. This has become the widely accepted formalization of the idea of a computer, and it is now considered that an algorithm exists to solve a problem if and only if there exists a Turing Machine which solves the problem. The study of computational complexity is the study of resource-bounded computation, and attempts to determine whether a given amount of computational resources (time, space, randomness, communication, quantum entanglement, and it gets more exotic from there...) is sufficient to solve a problem.
Humans seem pretty naturally well-suited to the task of conceiving of algorithms for certain problems. Finding the most efficient algorithm is still very difficult. However, one of the major problems in this field is proving that for some problem, an algorithm cannot exist, or that a better algorithm cannot exist. No one really knows how to do this well, and ultimately, it is due to our fundamentally poor understanding of the limitations of algorithms. Compare this with our (or at least *my*) poor understanding of the limitations of our thoughts.
One hope is that in addition to each of the resources used by an algorithm, we can discover other metrics for the progress of an ongoing computation, the information content of the data being used by an algorithm, or some other sort of computational invariant, and prove strong bounds on its growth; then, if solving a problem requires much of this invariant, it cannot be done quickly by any algorithm.
The discovery of such a measure would provide tremendous insight into the nature of computation, and, I believe, of thought. Especially if we could characterize exactly which problems have algorithms in terms of it. Of course, this would represent a tremendous scientific breakthrough.
What is the value of a new algorithm to humanity? At least as important as the ability to use it is the insight we gain from understanding it. Algorithms work by exploiting latent mathematical truths, and moreover, proofs against the existence of algorithms are proofs against the tremendous power of creative mathematics.
There will come a day when Moore's law has bottomed out, when what are essentially the best computers we can build have been built, when the top of the line machine does not differ in computational power from the everyman's PC, when cryptography relies on problems with no efficient parallel solutions and the largest computer network at the government's disposal will be no better prepared to crack state of the art codes than an affordable machine owned by a smart civilian. When this happens, the cryptographic arms race will rest solely on the algorithms employed by the code makers and the code breakers. Individuals will enjoy the same level of privacy as the government, and cyberspace could never be owned by legislators the way it is today.
When that day comes, when people try to handle problems that cannot be solved by throwing hardware at them, algorithms will be only relevant factor. As more and more jobs become automated, but NP-complete tasks like Theorem Proving remain intractable for computers, the discovery of new algorithms will be a central task for human beings. We have already seen the day when wars have hinged on the discovery of effective algorithms, when hundreds of German U-boats are destroyed because of insufficiently secure codes. We are currently seeing the day where hedge funds rise and fall on their ability to perform statistical analysis as accurately and robustly as possible on fantastic amounts of data in an economy that only grows more and more complicated. It is inevitable that algorithms will only increase in importance.
This is not to emphasize the existing uses we see for algorithms as the most important. On the contrary, I expect that the most fantastic and important uses for algorithms will be things we have not seen yet. The rise of computer games and computer simulations which feed the imagination is already on its way. The potential for algorithmic art is vast and wide open — we have already seen the beginnings of algorithmic music breaking onto the scene. Artificial intelligence research continues to inspire believers in a future previously only dreamed of in science fiction. As we speak, hundreds of researchers all over the globe are fine-tuning solutions to the various problems of computer vision.
In short, I don't know entirely where it is all going, but I am convinced algorithms are going to play a major role, and will at some point represent a major portion of human creative energy. In doing so, we will learn much about ourselves and our limits, and maybe come closer to really understanding what it means to think after all. With any luck.
8 comments
| Promote (15) |
|
Add to favorites (4)
|
| Synapses (3) |
|
Connected to
