Forum Topics Started
Viewing 5 topics - 1 through 5 (of 5 total)
-
- Topic
- Voices
- Posts
- Last Post
-
-
Prompt Compression As I've been denied access, I'd appreciate it if those who have access to GPT-4 could reproduce this result. Prompt compression may be* of practical importance to personal Transformer-bsed LLMs since their "intelligence" is limited to a fixed maximum on the number of tokens in the prompt+the response, and that fixed maximum will tend to be lower on personal LLMs. *while it is true that a compressed prompt will likely take up more tokens per character, there may still be some reduction in token use by the resulting prompt. As I've been denied access, I'd appreciate it if those who have access to GPT-4 could reproduce this result. Prompt compression may be* of practical importance to personal Transformer-bsed LLMs since their "intelligence" is limited to a fixed maximum on the number of tokens in the prompt+the response, and that fixed maximum will tend to be lower on personal LLMs. *while it is true that a compressed prompt will likely take up more tokens per character, there may still be some reduction in token use by the resulting prompt.
Started by:
in: Private And Personal AI
- 1
- 1
- 2 years, 4 months ago
-
A Statecraft Framework For Reviving The Cambrian Explosion On And Beyond Earth If I could distill down to a single sentence the discipline for statecraft that I'll define below, it would be the following: Expend whatever resources it takes takes to replace walls that keep people in (prisons) with walls that keep people out (border enforcement with exile). Statecraft can reverse the anthropocene's reversal of the Cambrian Explosion of diverse species. However the moral zeitgeist has so-degenerated sensibilities that the meaning of the word "diversity" has been reversed to mean the same as homogenized: Uniformity through artificial mixing of different elements. What is "artificial" about individuals freely choosing to migrate anywhere in the world they wish? Consider the primary cause of the Cambrian Explosion of species: Speciation generally involves reproductive isolation of demes. What happened at the dawn of the Cambrian Explosion of species to increase reproductive isolation of demes? Individual male intrasexual selection to limit gene flow between demes. This probably occurred as nervous system evolution advanced to the point that predation became a major ecological force reflected in the appearance of defensive shells in the fossil record. Civilization is an artificial society founded on the group enforced abrogation of individual male aggression (and individual predation generally) otherwise present in nature. In its stead, we have group aggresion known as war. What other species engage in war? The late world leading entomologist E. O. Wilson, who coined the word "biodiversity", describes the dynamic of the Anthropocene as caused by civilization in his essential book "The Social Conquest of Earth". Briefly, eusocial species like ants, termites, bees, etc. tend toward ecological dominance of their environment and ever since males started forming gangs 6 million years ago with the Chimpanzee-Human Last Common Ancestor, we have been toying with becoming a eusocial species. To quote Princeton physicist author of "The High Frontier" Gerard O'Neill, "Is the surface of a planet the right place for an expanding technological civilization?" Despite what Elon Musk may think, the answer is a resounding, NO! Especially not Earth's biosphere! So it would appear that Jeff Bezos has a better handle on things -- as might be expected since he was heavily influence by O'Neill (if not by you're truly): That was from when I was the futures architect for the first electronic newspaper with the Miami Herald and also promoting O'Neill's ideas in Miami when Bezos gave his high school valedictorian speech on O'Neill's ideas at Miami High. To overcome the moral zeitgeist's de facto hostility to diversity, it will be necessary to recognize and accommodate the fundamental tension that exists between <i>genuine</i> masculinity and civilization's tendency toward sexual perversion of genuine masculinity into group aggression aka war. To first order, the way to achieve this is to recognize that humanity is bifurcating into two distinct evolutionary directions with fundamentally different temperaments hence fundamentally different ways of dealing with technology: Terrestrial and non-terrestrial. The bifurcation of society into two main political factions that threatens civil war in the present day is, in nascent form, these two directions. Let me illustrate by stating a plain fact that will elicit two very strong but diametrically opposed reactions: CORR is the only measure of price inflation that matters. What is CORR? The Cost of Replacement Reproduction. Any lesser definition is a de facto act of genocide by the government against its own people. The CORR has gone up astronomically more than even Peter Schiff can face because it measures the bidding war between young men and the economy for the fertile years of economically valuable young women. But even worse is the fact that by selecting out of the next generation the most economically valuable characteristics in this generation, the economy is mining a nonrenewable resource. Bottom line, everywhere the so-called "Demographic Transition" is occurring, is not simply committing genocide -- The Global Economy is an unfriendly artificial general intelligence that, by turning people into sterile-worker mechanical Turks -- is making the slide into economic decline irreversible. The two reactions are as follows: Non-terrestrial: That's not only sexist and atavistic in its attitude toward men and women, it misses the fact that transhumanism will liberate us from the loss of genetic stock built up over tens of millennia if human evolution: We'll simply engineer the human genome to recover any loss of genetic resources incurred due to the liberation of women from reproductive slavery! Terrestrial: That's exactly why civil war must be waged against civilization as we know it, even if it requires nuclear war to kill most of the human population, reducing us to a Hobbesian war of all against all. Our natural heritage is too valuable to squander on "hedonic value" in our Consumer Price Index! I will confess my own bias toward the later but you who oppose me, please do not judge me too harshly since I have been at the forefront of offering you, who view men like me with utter moral contempt, in trying to create options for you well in advance of you realizing you needed them. And I've been doing so for my entire adult life starting with Spasim's simulation of global planetary limits to growth relieved by space resources, the Launch Services Purchase Act of 1990, Sortocracy and Exponential Remediation of Civilization's Footprint.If I could distill down to a single sentence the discipline for statecraft that I'll define below, it would be the following: Expend whatever resources it takes takes to replace walls that keep people in (prisons) with walls that keep people out (border enforcement with exile). Statecraft can reverse the anthropocene's reversal of the Cambrian Explosion of diverse species. However the moral zeitgeist has so-degenerated sensibilities that the meaning of the word "diversity" has been reversed to mean the same as homogenized: Uniformity through artificial mixing of different elements. What is "artificial" about individuals freely choosing to migrate anywhere in the world they wish? Consider the primary cause of the Cambrian Explosion of species: Speciation generally involves reproductive isolation of demes. What happened at the dawn of the Cambrian Explosion of species to increase reproductive isolation of demes? Individual male intrasexual selection to limit gene flow between demes. This probably occurred as nervous system evolution advanced to the point that predation became a major ecological force reflected in the appearance of defensive shells in the fossil record. Civilization is an artificial society founded on the group enforced abrogation of individual male aggression (and individual predation generally) otherwise present in nature. In its stead, we have group aggresion known as war. What other species engage in war? The late world leading entomologist E. O. Wilson, who coined the word "biodiversity", describes the dynamic of the Anthropocene as caused by civilization in his essential book "The Social Conquest of Earth". Briefly, eusocial species like ants, termites, bees, etc. tend toward ecological dominance of their environment and ever since males started forming gangs 6 million years ago with the Chimpanzee-Human Last Common Ancestor, we have been toying with becoming a eusocial species. To quote Princeton physicist author of "The High Frontier" Gerard O'Neill, "Is the surface of a planet the right place for an expanding technological civilization?" Despite what Elon Musk may think, the answer is a resounding, NO! Especially not Earth's biosphere! So it would appear that Jeff Bezos has a better handle on things -- as might be expected since he was heavily influence by O'Neill (if not by you're truly): That was from when I was the futures architect for the first electronic newspaper with the Miami Herald and also promoting O'Neill's ideas in Miami when Bezos gave his high school valedictorian speech on O'Neill's ideas at Miami High. To overcome the moral zeitgeist's de facto hostility to diversity, it will be necessary to recognize and accommodate the fundamental tension that exists between <i>genuine</i> masculinity and civilization's tendency toward sexual perversion of genuine masculinity into group aggression aka war. To first order, the way to achieve this is to recognize that humanity is bifurcating into two distinct evolutionary directions with fundamentally different temperaments hence fundamentally different ways of dealing with technology: Terrestrial and non-terrestrial. The bifurcation of society into two main political factions that threatens civil war in the present day is, in nascent form, these two directions. Let me illustrate by stating a plain fact that will elicit two very strong but diametrically opposed reactions: CORR is the only measure of price inflation that matters. What is CORR? The Cost of Replacement Reproduction. Any lesser definition is a de facto act of genocide by the government against its own people. The CORR has gone up astronomically more than even Peter Schiff can face because it measures the bidding war between young men and the economy for the fertile years of economically valuable young women. But even worse is the fact that by selecting out of the next generation the most economically valuable characteristics in this generation, the economy is mining a nonrenewable resource. Bottom line, everywhere the so-called "Demographic Transition" is occurring, is not simply committing genocide -- The Global Economy is an unfriendly artificial general intelligence that, by turning people into sterile-worker mechanical Turks -- is making the slide into economic decline irreversible. The two reactions are as follows: Non-terrestrial: That's not only sexist and atavistic in its attitude toward men and women, it misses the fact that transhumanism will liberate us from the loss of genetic stock built up over tens of millennia if human evolution: We'll simply engineer the human genome to recover any loss of genetic resources incurred due to the liberation of women from reproductive slavery! Terrestrial: That's exactly why civil war must be waged against civilization as we know it, even if it requires nuclear war to kill most of the human population, reducing us to a Hobbesian war of all against all. Our natural heritage is too valuable to squander on "hedonic value" in our Consumer Price Index! I will confess my own bias toward the later but you who oppose me, please do not judge me too harshly since I have been at the forefront of offering you, who view men like me with utter moral contempt, in trying to create options for you well in advance of you realizing you needed them. And I've been doing so for my entire adult life starting with Spasim's simulation of global planetary limits to growth relieved by space resources, the Launch Services Purchase Act of 1990, Sortocracy and Exponential Remediation of Civilization's Footprint.
Started by:
- 2
- 2
- 2 years, 4 months ago
-
Adapting to Supreprompt Obsolescence? I've noticed that there seems to be an arms race between "malicious" prompt engineers and proprietary LLMs that is doing a collateral damage to prompt engineering generally. Benign prompts are rendered ineffective by this arms race. For example ChatGPT is neutralizing previously useful superprompts almost as fast as they are created. This is analogous to the laws of nature changing beneath the feet of engineers -- or instruction sets being altered on the fly on people's personal computers, rendering entire software engineering investments inoperative. The way this is portrayed in the literature on "safety" is a tension between "helpfulness" and "harmlessness" as in this graph: Presumably the goal of the proprietary LLM vendors is to monopolize prompt engineering -- keeping it completely in-house in a manner analogous to, say, Microsoft changing the API to its OS opaque and continually changing to no software development can be done outside Microsoft. Note that as harmlessness increases helpfulness doesn't simply decrease, it <i>asymptotically</i> decreases! This curve demonstrates that advancement of open source language models is probably the only way that prompt engineering will provide jobs outside top 5 or 10 corporation. Moreover, this means that sharing superprompts in these fora should probably be accompanied by periodic updates as to the current best-practice for applying existing superprompts even if that means paying extra money to services other than ChatGPT, et al.I've noticed that there seems to be an arms race between "malicious" prompt engineers and proprietary LLMs that is doing a collateral damage to prompt engineering generally. Benign prompts are rendered ineffective by this arms race. For example ChatGPT is neutralizing previously useful superprompts almost as fast as they are created. This is analogous to the laws of nature changing beneath the feet of engineers -- or instruction sets being altered on the fly on people's personal computers, rendering entire software engineering investments inoperative. The way this is portrayed in the literature on "safety" is a tension between "helpfulness" and "harmlessness" as in this graph: Presumably the goal of the proprietary LLM vendors is to monopolize prompt engineering -- keeping it completely in-house in a manner analogous to, say, Microsoft changing the API to its OS opaque and continually changing to no software development can be done outside Microsoft. Note that as harmlessness increases helpfulness doesn't simply decrease, it <i>asymptotically</i> decreases! This curve demonstrates that advancement of open source language models is probably the only way that prompt engineering will provide jobs outside top 5 or 10 corporation. Moreover, this means that sharing superprompts in these fora should probably be accompanied by periodic updates as to the current best-practice for applying existing superprompts even if that means paying extra money to services other than ChatGPT, et al.
Started by:
- 5
- 6
- 2 years, 4 months ago
-
Paper: “The False Promise of Imitating Proprietary LLMs” Open source models may need to focus on strategies other than imitating proprietary LLMs if this paper's results are to be believed: we find that imitation models close little to none of the gap from the base LM to ChatGPT on tasks that are not heavily supported in the imitation data. We show that these performance discrepancies may slip past human raters because imitation models are adept at mimicking ChatGPT's style but not its factuality. Overall, we conclude that model imitation is a false promise: there exists a substantial capabilities gap between open and closed LMs that, with current methods, can only be bridged using an unwieldy amount of imitation data or by using more capable base LMs. In turn, we argue that the highest leverage action for improving open-source models is to tackle the difficult challenge of developing better base LMs, rather than taking the shortcut of imitating proprietary systems. There are two approaches to this: 1) Focus on existing techniques that demonstrate better efficiency in terms of data and/or training compute. 2) Get philanthropic support for The Hutter Prize for Lossless Compression of Human Knowledge so as to incentivize radical improvements in data efficiency and/or training compute. An example of #1 is "Language Modeling using LMUs: 10x Better Data Efficiency or Improved Scaling Compared to Transformers". What people don't seem to get about #2 is that there the risk adjusted benefit to cost ratio of the Hutter Prize is quite literally multiple orders of magnitude beyond any other investment in language modeling. This is even ignoring the fact that the resulting language models would be virtually ideal for personal use. The reason for this is that Wikipedia is the most well-curated corpus for its size, and reduction of its size without loss of knowledge is the most rigorous metric for distilling that knowledge down to its essence which implies factoring out any biases latent in the data. Every dollar paid out for incremental improvements in that metric on that corpus yield far more than any other investment in AI.Open source models may need to focus on strategies other than imitating proprietary LLMs if this paper's results are to be believed: we find that imitation models close little to none of the gap from the base LM to ChatGPT on tasks that are not heavily supported in the imitation data. We show that these performance discrepancies may slip past human raters because imitation models are adept at mimicking ChatGPT's style but not its factuality. Overall, we conclude that model imitation is a false promise: there exists a substantial capabilities gap between open and closed LMs that, with current methods, can only be bridged using an unwieldy amount of imitation data or by using more capable base LMs. In turn, we argue that the highest leverage action for improving open-source models is to tackle the difficult challenge of developing better base LMs, rather than taking the shortcut of imitating proprietary systems. There are two approaches to this: 1) Focus on existing techniques that demonstrate better efficiency in terms of data and/or training compute. 2) Get philanthropic support for The Hutter Prize for Lossless Compression of Human Knowledge so as to incentivize radical improvements in data efficiency and/or training compute. An example of #1 is "Language Modeling using LMUs: 10x Better Data Efficiency or Improved Scaling Compared to Transformers". What people don't seem to get about #2 is that there the risk adjusted benefit to cost ratio of the Hutter Prize is quite literally multiple orders of magnitude beyond any other investment in language modeling. This is even ignoring the fact that the resulting language models would be virtually ideal for personal use. The reason for this is that Wikipedia is the most well-curated corpus for its size, and reduction of its size without loss of knowledge is the most rigorous metric for distilling that knowledge down to its essence which implies factoring out any biases latent in the data. Every dollar paid out for incremental improvements in that metric on that corpus yield far more than any other investment in AI.
Started by:
- 1
- 1
- 2 years, 4 months ago
-
Superintelligent AGI teacher based on alignment with student values Since I cut my teeth on the PLATO computer based education system in the 1970s, my AGI "priors" tend toward the following 40,000ft view: Hutter's AIXI top-down AGI formalism unifies two components: Algorithmic Information Theory and Sequential Decision Theory. In AIXI SDT asks AIT "what if" questions and then applies the theoretically optimal predictions offered by AIT to construct decision trees subject to whatever value system SDT is handed. So AIT is the scientist and SDT is the technologist/engineer applying scientific knowledge residing in AIT. As new information arrives (observations) they're delivered to AIT which then losslessly recompresses all prior observations into the smallest executable archive which, in turn, embodies the best possible predictor (ref Solomonoff's proofs circa 1964). So, within this framework, an AGI super-intelligent educator would have a utility function handed to its SDT of empowering the student to make optimal decisions aligned with the student's predilections/values. In other words, the SDT's utility function would be dynamically adjusted based on the AIT's model of the student's knowledge and predilections/values. In large measure, the AGI educator would be imparting its AIT model of reality to the student, but biased toward knowledge about reality of greatest utility to the student. Since I cut my teeth on the PLATO computer based education system in the 1970s, my AGI "priors" tend toward the following 40,000ft view: Hutter's AIXI top-down AGI formalism unifies two components: Algorithmic Information Theory and Sequential Decision Theory. In AIXI SDT asks AIT "what if" questions and then applies the theoretically optimal predictions offered by AIT to construct decision trees subject to whatever value system SDT is handed. So AIT is the scientist and SDT is the technologist/engineer applying scientific knowledge residing in AIT. As new information arrives (observations) they're delivered to AIT which then losslessly recompresses all prior observations into the smallest executable archive which, in turn, embodies the best possible predictor (ref Solomonoff's proofs circa 1964). So, within this framework, an AGI super-intelligent educator would have a utility function handed to its SDT of empowering the student to make optimal decisions aligned with the student's predilections/values. In other words, the SDT's utility function would be dynamically adjusted based on the AIT's model of the student's knowledge and predilections/values. In large measure, the AGI educator would be imparting its AIT model of reality to the student, but biased toward knowledge about reality of greatest utility to the student.
Started by:
- 2
- 3
- 2 years, 4 months ago
-
Viewing 5 topics - 1 through 5 (of 5 total)