Résultats de recherche pour « Emergent abilities »

Let us assume that gravity is an emergent low-energy phenomenon arising from a topologically stable defect in momentum space -- the Fermi point. What are the consequences? We discuss the natural values of fermion masses and cosmological constant; flatness of the Universe; bounds on Lorentz violation; etc.

The success of foundation models in language and vision rests on a simple empirical promise: predictable improvements, and occasionally qualitatively new capabilities, emerge as models, data, and compute are scaled together. Single-cell foundation models (scFMs)-transformers pretrained by self-supervision on tens of millions of single-cell transcriptomes-were built on the implicit assumption that the same promise transfers to biology. This review asks whether that assumption holds: can scaling actually make scFMs smarter, and if so, smarter in what sense? We distinguish four notions of "smarter" that the literature conflates-downstream task accuracy, zero-shot generalization, biological knowledge captured in representations, and the richness of internal computation-and review the evidence for each. On the optimistic side, controlled scaling studies on masked-reconstruction transformers show clean power-law improvements in reconstruction loss, and large generative cell-language models exhibit emergent-looking gains on complex tasks. On the skeptical side, multiple independent evaluations find that zero-shot scFM embeddings are frequently matched or beaten by classical pipelines and simple linear baselines, that benefits saturate well below language-model scales, and that data redundancy rather than data volume governs returns. We argue that the single-cell setting differs from natural language in ways-bounded biological information, heavy technical noise, and a low effective-dimensionality ceiling-that make naive extrapolation of language scaling laws unsafe. Mechanistic-interpretability evidence suggests that scale may reorganize how scFMs compute (toward more biologically coherent, convergent circuits and more interpretable features) even when leaderboard metrics move little, implying that "smarter" should be measured by the structure of learned representations and not only by benchmark accuracy. We close with concrete recommendations for compute-optimal pretraining, evaluation beyond saturated benchmarks, and the role of interpretability as a scaling diagnostic.

Compositionality is an important explanatory target in emergent communication and language evolution. The vast majority of computational models of communication account for the emergence of only a very basic form of compositionality: trivial compositionality. A compositional protocol is trivially compositional if the meaning of a complex signal (e.g. blue circle) boils down to the intersection of meanings of its constituents (e.g. the intersection of the set of blue objects and the set of circles). A protocol is non-trivially compositional (NTC) if the meaning of a complex signal (e.g. biggest apple) is a more complex function of the meanings of their constituents. In this paper, we review several metrics of compositionality used in emergent communication and experimentally show that most of them fail to detect NTC - i.e. they treat non-trivial compositionality as a failure of compositionality. The one exception is tree reconstruction error, a metric motivated by formal accounts of compositionality. These results emphasise important limitations of emergent communication research that could hamper progress on modelling the emergence of NTC.

This article explores psychological factors influencing schoolchildren's career choices and ability development. The study aims to identify how motivation, personal traits, and social context shape students’ professional orientation. Using a qualitative analytical method, the research draws from psychological and pedagogical literature, incorporating comparative and theoretical analysis. Results show that self-awareness, family influence, and social stereotypes significantly affect students’ career decisions, highlighting the need for integrated career guidance approaches involving schools, families, and society.

Recent work claims that large language models display emergent abilities, abilities not present in smaller-scale models that are present in larger-scale models. What makes emergent abilities intriguing is two-fold: their sharpness, transitioning seemingly instantaneously from not present to present, and their unpredictability, appearing at seemingly unforeseeable model scales. Here, we present an alternative explanation for emergent abilities: that for a particular task and model family, when analyzing fixed model outputs, emergent abilities appear due to the researcher's choice of metric rather than due to fundamental changes in model behavior with scale. Specifically, nonlinear or discontinuous metrics produce apparent emergent abilities, whereas linear or continuous metrics produce smooth, continuous predictable changes in model performance. We present our alternative explanation in a simple mathematical model, then test it in three complementary ways: we (1) make, test and confirm three predictions on the effect of metric choice using the InstructGPT/GPT-3 family on tasks with claimed emergent abilities; (2) make, test and confirm two predictions about metric choices in a meta-analysis of emergent abilities on BIG-Bench; and (3) show to choose metrics to produce never-before-seen seemingly emergent abilities in multiple vision tasks across diverse deep networks. Via all three analyses, we provide evidence that alleged emergent abilities evaporate with different metrics or with better statistics, and may not be a fundamental property of scaling AI models.

The scale-up of AI models for analyzing largescale power systems necessitates a thorough understanding of their scaling properties. Existing studies on these properties provide only partial insights, showing that loss function decreases predictably with increased model scales; yet no scaling law for power system AI models has been established, and model performance remains unpredictable due to “emergent abilities”. This study pioneers the discussion on the emergent abilities of graph neural network (GNN) for analyzing large-scale power systems, revealing that model performance improves dramatically once model scale exceeds a threshold. Furthermore, we introduce an empirical power-law formula to quantify the relationship between this threshold and the power system size. Our theory accurately predicts the threshold for the appearance of emergent ability in large-scale power systems, including a synthetic 10,000- bus and a real-world 19,402-bus systems.

Scaling up language models has been shown to predictably improve performance and sample efficiency on a wide range of downstream tasks. This paper instead discusses an unpredictable phenomenon that we refer to as emergent abilities of large language models. We consider an ability to be emergent if it is not present in smaller models but is present in larger models. Thus, emergent abilities cannot be predicted simply by extrapolating the performance of smaller models. The existence of such emergence implies that additional scaling could further expand the range of capabilities of language models.

Emergent behavior arising in a joint human-robot system cannot be fully predicted based on an understanding of the individual agents. Typically, robot behavior is governed by algorithms that optimize a reward function that should quantitatively capture the joint system's goal. Although reward functions can be updated to better match human needs, this is no guarantee that no misalignment with the complex and variable human needs will occur. Algorithms may learn undesirable behavior when interacting with the human and the intrinsically unpredictable human-inhabited world, thereby producing further misalignment with human users or bystanders. As a result, humans might behave differently than anticipated, causing robots to learn differently and undesirable behavior to emerge. With this short paper, we state that to design for Human-Robot Interaction that mitigates such undesirable emergent behavior, we need to complement advancements in human-robot interaction algorithms with human factors knowledge and expertise. More specifically, we advocate a three-pronged approach that we illustrate using a particularly challenging example of safety-critical human-robot interaction: a driver interacting with a semi-automated vehicle. Undesirable emergent behavior should be mitigated by a combination of 1) including driver behavioral mechanisms in the vehicle's algorithms and reward functions, 2) model-based approaches that account for interaction-induced driver behavioral adaptations and 3) driver-centered interaction design that promotes driver engagement with the semi-automated vehicle, and the transparent communication of each agent's actions that allows mutual support and adaptation. We provide examples from recent empirical work in our group, in the hope this proves to be fruitful for discussing emergent human-robot interaction.