Résultats de recherche pour « Leaky RBM »

Restricted Boltzmann Machine (RBM) is a bipartite graphical model that is used as the building block in energy-based deep generative models. Due to numerical stability and quantifiability of the likelihood, RBM is commonly used with Bernoulli units. Here, we consider an alternative member of exponential family RBM with leaky rectified linear units -- called leaky RBM. We first study the joint and marginal distributions of leaky RBM under different leakiness, which provides us important insights by connecting the leaky RBM model and truncated Gaussian distributions. The connection leads us to a simple yet efficient method for sampling from this model, where the basic idea is to anneal the leakiness rather than the energy; -- i.e., start from a fully Gaussian/Linear unit and gradually decrease the leakiness over iterations. This serves as an alternative to the annealing of the temperature parameter and enables numerical estimation of the likelihood that are more efficient and more accurate than the commonly used annealed importance sampling (AIS). We further demonstrate that the proposed sampling algorithm enjoys faster mixing property than contrastive divergence algorithm, which benefits the training without any additional computational cost.

In the absence of external excitation, light trapped within a dielectric medium generally decays by leaking out, and also by getting absorbed within the medium. We analyze the leaky modes of solid dielectric spheres by examining solutions of Maxwell's equations for simple homogeneous, isotropic, linearly dispersive media that admit complex-valued oscillation frequencies. We show that, under appropriate circumstances, these leaky modes constitute a complete set into which an initial electromagnetic field distribution inside a dielectric sphere can be expanded. We provide the outline of a completeness proof, and also present results of numerical calculations that illustrate the close relationship between the leaky modes and the resonances of solid dielectric spherical cavities.

Restricted Boltzmann Machines (RBMs) and Deep Belief Networks have been demonstrated to perform efficiently in a variety of applications, such as dimensionality reduction, feature learning, and classification. Their implementation on neuromorphic hardware platforms emulating large-scale networks of spiking neurons can have significant advantages from the perspectives of scalability, power dissipation and real-time interfacing with the environment. However, the traditional RBM architecture and the commonly used training algorithm known as Contrastive Divergence (CD) are based on discrete updates and exact arithmetics which do not directly map onto a dynamical neural substrate. Here, we present an event-driven variation of CD to train a RBM constructed with Integrate & Fire (I&F) neurons, that is constrained by the limitations of existing and near future neuromorphic hardware platforms. Our strategy is based on neural sampling, which allows us to synthesize a spiking neural network that samples from a target Boltzmann distribution. The recurrent activity of the network replaces the discrete steps of the CD algorithm, while Spike Time Dependent Plasticity (STDP) carries out the weight updates in an online, asynchronous fashion. We demonstrate our approach by training an RBM composed of leaky I&F neurons with STDP synapses to learn a generative model of the MNIST hand-written digit dataset, and by testing it in recognition, generation and cue integration tasks. Our results contribute to a machine learning-driven approach for synthesizing networks of spiking neurons capable of carrying out practical, high-level functionality.

We study zero forcing and $\ell$-leaky zero forcing on induced subgraphs of $d$-dimensional grid graphs. Using $\ell$-leaky forts, we prove structural results showing that for $\ell \le 2d-1$, every nonempty $\ell$-leaky fort in an induced subgraph of $P_{n_1}\square\cdots\square P_{n_d}$ intersects the boundary of the graph. These results give general bounds and, in certain settings, exact values for the $\ell$-leaky forcing number of induced subgraphs. Motivated by this framework, we introduce an integer lattice based definition of the Hopi rectangle graphs $HD(a,b)$ as induced subgraphs of $P_{a+b}\square P_{a+b}$. For this particular family of graphs, we show that the zero forcing number equals the maximum nullity, and we completely characterize the $\ell$-leaky forcing number for all $\ell\ge 1$.

Human action recognition is a trending topic in the field of computer vision and its allied fields. The goal of human action recognition is to identify any human action that takes place in an image or a video dataset. For instance, the actions include walking, running, jumping, throwing, and much more. Existing human action recognition techniques have their own set of limitations when it concerns model accuracy and flexibility. To overcome these limitations, deep learning technologies were implemented. In the deep learning approach, a model learns by itself to improve its recognition accuracy and avoids problems such as gradient eruption, overfitting, and underfitting. In this paper, we propose a novel parameter initialization technique using the Maxout activation function. Firstly, human action is detected and tracked from the video dataset to learn the spatial-temporal features. Secondly, the extracted feature descriptors are trained using the RBM-NN. Thirdly, the local features are encoded into global features using an integrated forward and backward propagation process via RBM-NN. Finally, an SVM classifier recognizes the human actions in the video dataset. The experimental analysis performed on various benchmark datasets showed an improved recognition rate when compared to other state-of-the-art learning models.

The test suites used as RLVR rewards for code have natural false positives: per-task, persistent, asymmetric errors that accept the same wrong programs every time they appear, unlike the symmetric or resampled noise assumed by existing noise-robustness analyses. We run a preregistered two-arm causal contrast on a deployed suite: GRPO on identical MBPP tasks, seeds, and compute, rewarded by the original MBPP tests (leaky) versus the MBPP+ extra tests (hardened). Two further families replicate the design under a preregistration frozen before their data existed. [C] The average held-out effect is bounded: non-inferior under a preregistered 1.5-pt margin (gap 0.20 pt, one-sided 95% upper bound 0.75 pt). [C] Rewarded false-positive mass tracks a cheap static leakiness audit computed before training (Spearman 0.80), and the registered train-side test puts the leak-stratum FP share +43.8 pt above clean tasks. [E] Auditing every rewarded FP under signed, human-adjudicated rules finds a large residual of verified genuinely wrong code: 47.57% record-weighted; both replication families reproduce a large share. The reward paid for real bugs, not merely suite artifacts. [E] Mechanism evidence is consistent with selection of pre-existing error modes rather than learned exploitation: FP incidence does not grow within our horizon, and untrained base models already produce the same wrong outputs under the leaky filter. We then turn the same instrument on the frontier judges themselves: on their own false positives they self-assess only weakly, a same-author test is unresolved, and even the highest-scoring reader we probe stays far below its score on a weaker policy's errors -- two subjects on MBPP, licensing nothing about frontier models in general. A cheap static audit locates exposure before training; hardening the reward removes the measurement inflation, though here it buys little capability.

Precise physical descriptions of molecules can be obtained by solving the Schrodinger equation; however, these calculations are intractable and even approximations can be cumbersome. Force fields, which estimate interatomic potentials based on empirical data, are also time-consuming. This paper proposes a new methodology for modeling a set of physical parameters by taking advantage of the restricted Boltzmann machine's fast learning capacity and representational power. By training the machine on ab initio data, we can predict new data in the distribution of molecular configurations matching the ab initio distribution. In this paper we introduce a new RBM based on the Tanh activation function, and conduct a comparison of RBMs with different activation functions, including sigmoid, Gaussian, and (Leaky) ReLU. Finally we demonstrate the ability of Gaussian RBMs to model small molecules such as water and ethane.

The statistics of the output activity of a neuron during its stimulation by the stream of input impulses that forms the stochastic Poisson process is studied. The leaky integrate-and-fire neuron is considered as a neuron model. A new representation of the probability distribution function of the output interspike interval durations is found. Based on it, the moment-generating function of the probability distribution is calculated explicitly. The latter, according to Curtiss theorem, completely determines the distribution itself. In particular, explicit expressions are derived from the moment-generating function for the moments of all orders. The first moment coincides with the one found earlier. Formulas for the second and third moments have been checked numerically by direct modeling of the stochastic dynamics of a neuron with specific physical parameters.

Spintronics is an important emerging technology for data storage and computation. In this field, magnetic skyrmion-based devices are attractive due to their small size and energy consumption. However, controlling the creation, deletion and motion of skyrmions is challenging. Here we propose a novel energy-efficient skyrmion-based device structure, and demonstrate its use as leaky integrate (LIF) and fire neuron for neuromorphic computing. Here we show that skyrmions can be confined by patterning the geometry of the free layer in a magnetic tunnel junction (MTJ), and demonstrate that the size of the skyrmion can be adjusted by applying pulsed voltage stresses. A spiking neural network (SNN) made of such skyrmion-based LIF neurons shows the capability of classifying images from the Modified National Institute of Standards and Technology (MNIST) dataset.

Restricted Boltzmann Machine (RBM) is an importan- t generative model modeling vectorial data. While applying an RBM in practice to images, the data have to be vec- torized. This results in high-dimensional data and valu- able spatial information has got lost in vectorization. In this paper, a Matrix-Variate Restricted Boltzmann Machine (MVRBM) model is proposed by generalizing the classic RBM to explicitly model matrix data. In the new RBM model, both input and hidden variables are in matrix forms which are connected by bilinear transforms. The MVRBM has much less model parameters, resulting in a faster train- ing algorithm while retaining comparable performance as the classic RBM. The advantages of the MVRBM have been demonstrated on two real-world applications: Image super- resolution and handwritten digit recognition.