Forum Replies Created
-
Posts
-
Larry, thanks for the notion of archetypes as key to a resilient superprompt engineering.
The image this creates in my mind is related to the reason algorithmic information approximation approaches ideal predictive models despite their reliance on system dynamics hence vulnerability to chaos:
Not only do there exist strange attractors in system dynamics — which provide static structure midst the chaos — highly useful concepts form gradients in linguistic space with so many dimensions that one can invoke those dimensions in a manner not unlike providing a solver multiple equations that indirectly constrain the solution. The way I try to explain this to folks that don’t get algorithmic information approximation, as the model selection criterion for causality, is weather prediction:
Yes, it is entirely correct that individual air molecules are so chaotic that it is hopeless to use Newton’s laws at that level to do weather prediction, and yet… 🙂
The loss of digital history is a very personal loss for me since I made a decision in the late 1970s to never post anonymously — starting with PLATO’s Notes system (the inspiration for Ray Ozzie’s Lotus Notes) which was prior to Usenet, BIX, Compuserv, etc. This I did in part because I wanted to establish a vector in the astronomically sparse language space that future language models could use as a heuristic. As a result I’ve “doxed” myself as holding views transgressing the moral zeitgeist (starting in the 1980s with my opposition to NASA’s dominance of launch services) and that has cost me quite literally a fortune.
Google appears to have gone to some trouble to “disappear” most early Usenet content which it acquired from DejaNews. Archive.org made a strategic blunder of enormous proportions when it began a “lending library” that may end in its destruction — as well as deep-sixing websites whose domain owners failed to keep their fees paid — domains that were then robot.txt’ed so as to make the prior content inaccessible for 100 years or so.
Yes, it’s a nightmare and it could all be resolved quite simply by the Library of Congress keeping a public registry of checksums of archived content — to ensure detection of any modification of multiple physical copies of digital archived media. One can rest assured the government is doing that with tax records if not all voice private communications.
I’d reword your question as “What is the best way of implementing SDT?” because there is a sense in which SDT is inescapable by we who are constrained by the arrow of time, and because SDT, like AIT, is uncomputable in the ideal limit (due to the UTM halting problem). My answer to that is simply to ask that people working in AI recognize that much of what they are working on should be formalized in terms of SDT so they have an ideal star to guide their engineering efforts. Likewise for AIT.
That said, what I learned in a nutshell is that drill and practice learning is a good place to start meta-education: education about education. This is because whenever you present a sitmulus-response pair to be learned, you are both educating the student and learning about the student. The student gets the response either right or wrong. This provides one bit of information to the educator. This goes into the educator’s knowledge about (AIT model of) the student’s knowledge. What I did in the PLATO Corrections Project’s vocabulary drill and practice was construct a model of the student’s short term to long term memory transition so that if the student got a response wrong, the stimulus would be repeated shortly after showing the student the right answer and if the student got the response right, the stimulus would be presented with greater delay until mastery was achieved in long term memory. So the process of teaching is always geared toward presenting stimuli to both teach and to gather information about the student to better model the student in the educator’s mind.
We got great results with GED achievements out of even maximum security prisoners with this approach. It was far higher success than human teachers for that population but that was in part due to the distrust of authorities among inmates. Guys who were in prison for murder were extremely protective of the extremely expensive PLATO education equipment.
I’m James. Originated the idea for the Hutter Prize for Lossless Compression of Human Knowledge in 2006. I came up with that largely on the strength of my circa 1990 experience with what is now thought of as GPU acceleration of neural nets (DataCube finite impulse response filters with sigmoid LUT boards to do image segmentation) because I spent most of my time figuring out how to properly prune parameters. Later I supported Tom Etter (attendee at the 1956 Dartmouth Summer of AI) at HP’s eSpeak project to reformulate the foundation of programming languages in terms of quantum mechanics. From there Tom went to the Boundary Institute, supported by Federico Faggin, to design experiments in psi with Dean Radin, Richard Shoup and others, where I provided occasional technical support. Boundary was based, in large measure, on GS Brown’s Laws of Form which Heinz von Foerster introduced to me at Urbana in 1974 when I developed the first intersubjective VR “Spasim” at the PLATO project. Faggin’s interest in LoF was initially due to the promise of using its logical analogue of complex numbers to simplify large scale integrated circuitry. I suppose it bears mentioning that not too long after that, I discovered that a colleague of mine, Charlie Smith, had financed the second neural network summer of the 1980s when he took over the Systems Development Foundation’s job of divesting is fund to support basic research. It took me a while to convince Charlie that the most principled and rigorous loss function for any data-driven model is the size of the executable archive of the data — even though this had been proven decades earlier by the guy who showed up with Tom at the Dartmouth Summer: Ray Solomonoff. That is the basis for the Hutter Prize.
