TY - JOUR
T1 - On the Number of Memories that can be Perfectly Stored in a Neural Net with Hebb Weights
AU - Sussmann, H. J.
N1 - Funding Information:
Manuscript received November 5. 1987; revised April 19, 198X. This work was supported in part by the National Science Foundation under Grant DMS83-01678-01 and in part by the CAIP Center, Rutgers University. The author is with the Department of Mathematics, Rutgers University, New Brunswick, NJ 08903. IEEE Log Number 8825700.
PY - 1989/1
Y1 - 1989/1
N2 - Let {wij} be the weights of the connections of a neural network with n nodes, calculated from m data vectors v1,…, vm in {1,-1}, according to the Hebb rule. We prove that, if m is not too large relative to n and the vk are random, then the wij constitute, with high probability, a perfect representation of the vk in the sense that the vk are completely determined by the wij up to their sign. The conditions under which this is established turn out to be less restrictive than those under which it has been shown that the vk can actually be recovered by letting the network evolve until equilibrium is attained. In the specific case where the entries of the vk are independent and equal to 1 or - 1 with probability 1/2, the condition on m is that m should not exceed n/0.7log n.
AB - Let {wij} be the weights of the connections of a neural network with n nodes, calculated from m data vectors v1,…, vm in {1,-1}, according to the Hebb rule. We prove that, if m is not too large relative to n and the vk are random, then the wij constitute, with high probability, a perfect representation of the vk in the sense that the vk are completely determined by the wij up to their sign. The conditions under which this is established turn out to be less restrictive than those under which it has been shown that the vk can actually be recovered by letting the network evolve until equilibrium is attained. In the specific case where the entries of the vk are independent and equal to 1 or - 1 with probability 1/2, the condition on m is that m should not exceed n/0.7log n.
UR - https://www.scopus.com/pages/publications/84941456568
UR - https://www.scopus.com/pages/publications/84941456568#tab=citedBy
U2 - 10.1109/18.42187
DO - 10.1109/18.42187
M3 - Letter
AN - SCOPUS:84941456568
SN - 0018-9448
VL - 35
SP - 174
EP - 178
JO - IEEE Transactions on Information Theory
JF - IEEE Transactions on Information Theory
IS - 1
ER -