A Neural Network for Semi-supervised Learning on Manifolds

Alexander Genkin, Anirvan M. Sengupta, Dmitri Chklovskii

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Semi-supervised learning algorithms typically construct a weighted graph of data points to represent a manifold. However, an explicit graph representation is problematic for neural networks operating in the online setting. Here, we propose a feed-forward neural network capable of semi-supervised learning on manifolds without using an explicit graph representation. Our algorithm uses channels that represent localities on the manifold such that correlations between channels represent manifold structure. The proposed neural network has two layers. The first layer learns to build a representation of low-dimensional manifolds in the input data as proposed recently in [8]. The second learns to classify data using both occasional supervision and similarity of the manifold representation of the data. The channel carrying label information for the second layer is assumed to be “silent” most of the time. Learning in both layers is Hebbian, making our network design biologically plausible. We experimentally demonstrate the effect of semi-supervised learning on non-trivial manifolds.

Original languageEnglish (US)
Title of host publicationArtificial Neural Networks and Machine Learning – ICANN 2019
Subtitle of host publicationTheoretical Neural Computation - 28th International Conference on Artificial Neural Networks, 2019, Proceedings
EditorsIgor V. Tetko, Pavel Karpov, Fabian Theis, Vera Kurková
PublisherSpringer Verlag
Pages375-386
Number of pages12
ISBN (Print)9783030304867
DOIs
StatePublished - Jan 1 2019
Event28th International Conference on Artificial Neural Networks, ICANN 2019 - Munich, Germany
Duration: Sep 17 2019Sep 19 2019

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11727 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference28th International Conference on Artificial Neural Networks, ICANN 2019
CountryGermany
CityMunich
Period9/17/199/19/19

Fingerprint

Semi-supervised Learning
Supervised learning
Neural Networks
Neural networks
Feedforward neural networks
Graph Representation
Learning algorithms
Labels
Feedforward Neural Networks
Weighted Graph
Network Design
Locality
Learning Algorithm
Classify
Demonstrate

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Computer Science(all)

Keywords

  • Manifold learning
  • Online learning
  • Semi-supervised learning

Cite this

Genkin, A., Sengupta, A. M., & Chklovskii, D. (2019). A Neural Network for Semi-supervised Learning on Manifolds. In I. V. Tetko, P. Karpov, F. Theis, & V. Kurková (Eds.), Artificial Neural Networks and Machine Learning – ICANN 2019: Theoretical Neural Computation - 28th International Conference on Artificial Neural Networks, 2019, Proceedings (pp. 375-386). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11727 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-030-30487-4_30
Genkin, Alexander ; Sengupta, Anirvan M. ; Chklovskii, Dmitri. / A Neural Network for Semi-supervised Learning on Manifolds. Artificial Neural Networks and Machine Learning – ICANN 2019: Theoretical Neural Computation - 28th International Conference on Artificial Neural Networks, 2019, Proceedings. editor / Igor V. Tetko ; Pavel Karpov ; Fabian Theis ; Vera Kurková. Springer Verlag, 2019. pp. 375-386 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{20aee49e23c94802961af9218af275ae,
title = "A Neural Network for Semi-supervised Learning on Manifolds",
abstract = "Semi-supervised learning algorithms typically construct a weighted graph of data points to represent a manifold. However, an explicit graph representation is problematic for neural networks operating in the online setting. Here, we propose a feed-forward neural network capable of semi-supervised learning on manifolds without using an explicit graph representation. Our algorithm uses channels that represent localities on the manifold such that correlations between channels represent manifold structure. The proposed neural network has two layers. The first layer learns to build a representation of low-dimensional manifolds in the input data as proposed recently in [8]. The second learns to classify data using both occasional supervision and similarity of the manifold representation of the data. The channel carrying label information for the second layer is assumed to be “silent” most of the time. Learning in both layers is Hebbian, making our network design biologically plausible. We experimentally demonstrate the effect of semi-supervised learning on non-trivial manifolds.",
keywords = "Manifold learning, Online learning, Semi-supervised learning",
author = "Alexander Genkin and Sengupta, {Anirvan M.} and Dmitri Chklovskii",
year = "2019",
month = "1",
day = "1",
doi = "10.1007/978-3-030-30487-4_30",
language = "English (US)",
isbn = "9783030304867",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer Verlag",
pages = "375--386",
editor = "Tetko, {Igor V.} and Pavel Karpov and Fabian Theis and Vera Kurkov{\'a}",
booktitle = "Artificial Neural Networks and Machine Learning – ICANN 2019",
address = "Germany",

}

Genkin, A, Sengupta, AM & Chklovskii, D 2019, A Neural Network for Semi-supervised Learning on Manifolds. in IV Tetko, P Karpov, F Theis & V Kurková (eds), Artificial Neural Networks and Machine Learning – ICANN 2019: Theoretical Neural Computation - 28th International Conference on Artificial Neural Networks, 2019, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11727 LNCS, Springer Verlag, pp. 375-386, 28th International Conference on Artificial Neural Networks, ICANN 2019, Munich, Germany, 9/17/19. https://doi.org/10.1007/978-3-030-30487-4_30

A Neural Network for Semi-supervised Learning on Manifolds. / Genkin, Alexander; Sengupta, Anirvan M.; Chklovskii, Dmitri.

Artificial Neural Networks and Machine Learning – ICANN 2019: Theoretical Neural Computation - 28th International Conference on Artificial Neural Networks, 2019, Proceedings. ed. / Igor V. Tetko; Pavel Karpov; Fabian Theis; Vera Kurková. Springer Verlag, 2019. p. 375-386 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11727 LNCS).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - A Neural Network for Semi-supervised Learning on Manifolds

AU - Genkin, Alexander

AU - Sengupta, Anirvan M.

AU - Chklovskii, Dmitri

PY - 2019/1/1

Y1 - 2019/1/1

N2 - Semi-supervised learning algorithms typically construct a weighted graph of data points to represent a manifold. However, an explicit graph representation is problematic for neural networks operating in the online setting. Here, we propose a feed-forward neural network capable of semi-supervised learning on manifolds without using an explicit graph representation. Our algorithm uses channels that represent localities on the manifold such that correlations between channels represent manifold structure. The proposed neural network has two layers. The first layer learns to build a representation of low-dimensional manifolds in the input data as proposed recently in [8]. The second learns to classify data using both occasional supervision and similarity of the manifold representation of the data. The channel carrying label information for the second layer is assumed to be “silent” most of the time. Learning in both layers is Hebbian, making our network design biologically plausible. We experimentally demonstrate the effect of semi-supervised learning on non-trivial manifolds.

AB - Semi-supervised learning algorithms typically construct a weighted graph of data points to represent a manifold. However, an explicit graph representation is problematic for neural networks operating in the online setting. Here, we propose a feed-forward neural network capable of semi-supervised learning on manifolds without using an explicit graph representation. Our algorithm uses channels that represent localities on the manifold such that correlations between channels represent manifold structure. The proposed neural network has two layers. The first layer learns to build a representation of low-dimensional manifolds in the input data as proposed recently in [8]. The second learns to classify data using both occasional supervision and similarity of the manifold representation of the data. The channel carrying label information for the second layer is assumed to be “silent” most of the time. Learning in both layers is Hebbian, making our network design biologically plausible. We experimentally demonstrate the effect of semi-supervised learning on non-trivial manifolds.

KW - Manifold learning

KW - Online learning

KW - Semi-supervised learning

UR - http://www.scopus.com/inward/record.url?scp=85072872064&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85072872064&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-30487-4_30

DO - 10.1007/978-3-030-30487-4_30

M3 - Conference contribution

AN - SCOPUS:85072872064

SN - 9783030304867

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 375

EP - 386

BT - Artificial Neural Networks and Machine Learning – ICANN 2019

A2 - Tetko, Igor V.

A2 - Karpov, Pavel

A2 - Theis, Fabian

A2 - Kurková, Vera

PB - Springer Verlag

ER -

Genkin A, Sengupta AM, Chklovskii D. A Neural Network for Semi-supervised Learning on Manifolds. In Tetko IV, Karpov P, Theis F, Kurková V, editors, Artificial Neural Networks and Machine Learning – ICANN 2019: Theoretical Neural Computation - 28th International Conference on Artificial Neural Networks, 2019, Proceedings. Springer Verlag. 2019. p. 375-386. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-030-30487-4_30