Resource-efficient perceptron has sparse synaptic weight distribution

Cengiz Pehlevan, Anirvan Sengupta

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Resource-efficiency is important for biological function of neurons. Using the perceptron as a model of a neuron, we show that resource-efficient learning implies sparse neural connectivity. The perceptron associates inputs to outputs by adjusting its synaptic weights. The learned synaptic weights are proposed to be the most resource-efficient by minimizing a biological resource cost given by the total absolute synaptic weight (l1-norm). Analytical methods from statistical physics and numerical simulations demonstrate that a resource-efficient perceptron has sparse connectivity. Sparseness decreases and resource usage increases with the number of associations to be learned. Our results have implications for synaptic connectivity in the cerebellum, where supervised learning is believed to happen.

Original languageEnglish (US)
Title of host publication2017 25th Signal Processing and Communications Applications Conference, SIU 2017
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781509064946
DOIs
StatePublished - Jun 27 2017
Event25th Signal Processing and Communications Applications Conference, SIU 2017 - Antalya, Turkey
Duration: May 15 2017May 18 2017

Publication series

Name2017 25th Signal Processing and Communications Applications Conference, SIU 2017

Other

Other25th Signal Processing and Communications Applications Conference, SIU 2017
CountryTurkey
CityAntalya
Period5/15/175/18/17

Fingerprint

Neural networks
Neurons
Supervised learning
Physics
Computer simulation
Costs

All Science Journal Classification (ASJC) codes

  • Computer Networks and Communications
  • Signal Processing

Keywords

  • Purkinje cells
  • neuron
  • perceptron

Cite this

Pehlevan, C., & Sengupta, A. (2017). Resource-efficient perceptron has sparse synaptic weight distribution. In 2017 25th Signal Processing and Communications Applications Conference, SIU 2017 [7960683] (2017 25th Signal Processing and Communications Applications Conference, SIU 2017). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/SIU.2017.7960683
Pehlevan, Cengiz ; Sengupta, Anirvan. / Resource-efficient perceptron has sparse synaptic weight distribution. 2017 25th Signal Processing and Communications Applications Conference, SIU 2017. Institute of Electrical and Electronics Engineers Inc., 2017. (2017 25th Signal Processing and Communications Applications Conference, SIU 2017).
@inproceedings{c948b3efccec464cad6e262d36f6602e,
title = "Resource-efficient perceptron has sparse synaptic weight distribution",
abstract = "Resource-efficiency is important for biological function of neurons. Using the perceptron as a model of a neuron, we show that resource-efficient learning implies sparse neural connectivity. The perceptron associates inputs to outputs by adjusting its synaptic weights. The learned synaptic weights are proposed to be the most resource-efficient by minimizing a biological resource cost given by the total absolute synaptic weight (l1-norm). Analytical methods from statistical physics and numerical simulations demonstrate that a resource-efficient perceptron has sparse connectivity. Sparseness decreases and resource usage increases with the number of associations to be learned. Our results have implications for synaptic connectivity in the cerebellum, where supervised learning is believed to happen.",
keywords = "Purkinje cells, neuron, perceptron",
author = "Cengiz Pehlevan and Anirvan Sengupta",
year = "2017",
month = "6",
day = "27",
doi = "10.1109/SIU.2017.7960683",
language = "English (US)",
series = "2017 25th Signal Processing and Communications Applications Conference, SIU 2017",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
booktitle = "2017 25th Signal Processing and Communications Applications Conference, SIU 2017",
address = "United States",

}

Pehlevan, C & Sengupta, A 2017, Resource-efficient perceptron has sparse synaptic weight distribution. in 2017 25th Signal Processing and Communications Applications Conference, SIU 2017., 7960683, 2017 25th Signal Processing and Communications Applications Conference, SIU 2017, Institute of Electrical and Electronics Engineers Inc., 25th Signal Processing and Communications Applications Conference, SIU 2017, Antalya, Turkey, 5/15/17. https://doi.org/10.1109/SIU.2017.7960683

Resource-efficient perceptron has sparse synaptic weight distribution. / Pehlevan, Cengiz; Sengupta, Anirvan.

2017 25th Signal Processing and Communications Applications Conference, SIU 2017. Institute of Electrical and Electronics Engineers Inc., 2017. 7960683 (2017 25th Signal Processing and Communications Applications Conference, SIU 2017).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Resource-efficient perceptron has sparse synaptic weight distribution

AU - Pehlevan, Cengiz

AU - Sengupta, Anirvan

PY - 2017/6/27

Y1 - 2017/6/27

N2 - Resource-efficiency is important for biological function of neurons. Using the perceptron as a model of a neuron, we show that resource-efficient learning implies sparse neural connectivity. The perceptron associates inputs to outputs by adjusting its synaptic weights. The learned synaptic weights are proposed to be the most resource-efficient by minimizing a biological resource cost given by the total absolute synaptic weight (l1-norm). Analytical methods from statistical physics and numerical simulations demonstrate that a resource-efficient perceptron has sparse connectivity. Sparseness decreases and resource usage increases with the number of associations to be learned. Our results have implications for synaptic connectivity in the cerebellum, where supervised learning is believed to happen.

AB - Resource-efficiency is important for biological function of neurons. Using the perceptron as a model of a neuron, we show that resource-efficient learning implies sparse neural connectivity. The perceptron associates inputs to outputs by adjusting its synaptic weights. The learned synaptic weights are proposed to be the most resource-efficient by minimizing a biological resource cost given by the total absolute synaptic weight (l1-norm). Analytical methods from statistical physics and numerical simulations demonstrate that a resource-efficient perceptron has sparse connectivity. Sparseness decreases and resource usage increases with the number of associations to be learned. Our results have implications for synaptic connectivity in the cerebellum, where supervised learning is believed to happen.

KW - Purkinje cells

KW - neuron

KW - perceptron

UR - http://www.scopus.com/inward/record.url?scp=85026319128&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85026319128&partnerID=8YFLogxK

U2 - 10.1109/SIU.2017.7960683

DO - 10.1109/SIU.2017.7960683

M3 - Conference contribution

AN - SCOPUS:85026319128

T3 - 2017 25th Signal Processing and Communications Applications Conference, SIU 2017

BT - 2017 25th Signal Processing and Communications Applications Conference, SIU 2017

PB - Institute of Electrical and Electronics Engineers Inc.

ER -

Pehlevan C, Sengupta A. Resource-efficient perceptron has sparse synaptic weight distribution. In 2017 25th Signal Processing and Communications Applications Conference, SIU 2017. Institute of Electrical and Electronics Engineers Inc. 2017. 7960683. (2017 25th Signal Processing and Communications Applications Conference, SIU 2017). https://doi.org/10.1109/SIU.2017.7960683