Beyond Backpropagation

Overview

Is backpropagation the ultimate tool on the path to achieving artificial intelligence as its success and widespread adoption would suggest?

Many have questioned the biological plausibility of backpropagation as a learning mechanism since its discovery. The weight transport and timing problems are the most disputable. The same properties of backpropagation training also have practical consequences. For instance, backpropagation training is a global and coupled procedure that limits the amount of possible parallelism and yields high latency.

These limitations have motivated us to discuss possible alternative directions. In this workshop, we want to promote such discussions by bringing together researchers from various but related disciplines, and to discuss possible solutions from engineering, machine learning and neuroscientific perspectives.

Accepted Papers

Papers accepted for oral presentations

Papers accepted for poster presentations

1.	A biologically plausible neural network for local supervision in cortical microcircuits - Siavash Golkar, David Lipshutz, Yanis Bahroun, Anirvan Sengupta, Dmitri Chklovskii
2.	A More Biologically Plausible Local Learning Rule for ANNs - Shashi Kant Gupta
3.	A Theoretical Framework for Target Propagation - Alexander Meulemans, Francesco S Carzaniga, Johan Suykens, João Sacramento, Benjamin F. Grewe
4.	Align, then Select: Analysing the Learning Dynamics of Feedback Alignment - Maria Refinetti, Stéphane d'Ascoli, Ruben Ohana, Sebastian Goldt
5.	Architecture Agnostic Neural Networks - Sabera Talukder, Guruprasad Raghavan, Yisong Yue
6.	Backpropagation Free Transformers - Dinko D Franceschi
7.	Biophysical Neural Networks Provide Robustness and Versatility over Artificial Neural Networks - James Hazelden, Michael I Ivanitskiy, Daniel Forger
8.	BP2T2: Moving towards Biologically-Plausible BackPropagation Through Time - Arna Ghosh, Jonathan Cornford, Blake Richards
9.	Convolutional Neural Networks from Image Markers - Barbara C Benato, Italos Estilon de Souza, Felipe L Galvao, Alexandre X Falcão	[Video]
10.	Deep Networks from the Principle of Rate Reduction - Kwan Ho Ryan Chan, Yaodong Yu, Chong You, Haozhi Qi, John Wright, Yi Ma
11.	Deep Neural Network Training without Multiplications - Tsuguo Mogami
12.	Deep Neural Networks Are Congestion Games - Nina Vesseron, Ievgen Redko, Charlotte Laclau
13.	Deep Reservoir Networks with Learned Hidden Reservoir Weights using Direct Feedback Alignment - Matthew S Evanusa, Aloimonos Yiannis, Cornelia Fermuller	[Video]
14.	Direct Feedback Alignment Scales to Modern Deep Learning Tasks and Architectures - Julien Launay, François Boniface, Iacopo Poli, Florent Krzakala
15.	Feature Whitening via Gradient Transformation for Improved Convergence - Shmulik Markovich-Golan, ‪Barak Battash‬‏, Amit Bleiweiss
16.	Front Contribution instead of Back Propagation - Swaroop Ranjan Mishra, Anjana Arunkumar	[Video]
17.	Gated Linear Networks and Extensions - Eren Sezener, David Budden, Marcus Hutter, Christopher Mattern, Jianan Wang, Joel Veness
18.	Generalized Stochastic Backpropagation - Amine Echraibi, Joachim Flocon-Cholet, Stéphane W Gosselin, Sandrine Vaton
20.	HebbNet: A Simplified Hebbian Learning Framework to do Biologically Plausible Learning - Manas Gupta, ArulMurugan Ambikapathi, Ramasamy Savitha	[Video]
21.	Hindsight Network Credit Assignment - Kenny Young	[Video]
22.	How and When does Feedback Alignment Work? - Stéphane d'Ascoli, Maria Refinetti, Ruben Ohana, Sebastian Goldt
23.	Ignorance is Bliss: Adversarial Robustness by Design through Analog Computing and Synaptic Asymmetry - Alessandro Cappelli, Ruben Ohana, Julien Launay, Iacopo Poli, Florent Krzakala
24.	Improving Multimodal Accuracy Through Modality Pre-training and Attention - Aya Abdelsalam A Ismail, Faisal Ishtiaq, Mahmudul Hasan
25.	Investigating Coagent Networks for Supervised Learning - Dhawal Gupta, Matthew Schlegel, James Kostas, Gabor Mihucz, Martha White
26.	Investigating the Scalability and Biological Plausibility of the Activation Relaxation Algorithm - Beren Millidge, Alexander D Tschanz, Anil Seth, Christopher Buckley
27.	Layer-wise Learning of Kernel Dependence Networks - Chieh Tzu Wu, Aria Masoomi, Arthur Gretton, Jennifer Dy	[Video]
28.	Layer-wise Learning via Kernel Embedding - Aria Masoomi, Chieh Tzu Wu, Arthur Gretton, Jennifer Dy	[Video]
29.	Learning Flows By Parts - Manush Bhatt, David I Inouye	[Video]
30.	MEAL V2: Boosting Vanilla ResNet-50 to 80%+ Top-1 Accuracy on ImageNet without Tricks - Zhiqiang Shen, Marios Savvides
31.	Menger: Large-Scale Distributed Reinforcement Learning - Amir Yazdanbakhsh, Junchao Chen, Yu Zheng
32.	Meta-Learning Backpropagation And Improving It - Louis Kirsch, Jürgen Schmidhuber	[Video]
33.	MPLP: Learning a Message Passing Learning Protocol - Ettore Randazzo, Eyvind Niklasson, Alexander Mordvintsev	[Video]
34.	Neighbourhood Distillation: On the benefits of non end-to-end distillation - Laëtitia M Shao, Max Moroz, Elad Eban, Yair Movshovitz-Attias
35.	Optimizing Neural Networks via Koopman Operator Theory - William T Redman, Akshunna S. Dogra
37.	Predicting Pretrained Weights of Large-scale CNNs - Boris Knyazev, Michal Drozdzal, Graham Taylor, Adriana Romero	[Video]
38.	PyTorch-Hebbian: facilitating local learning in a deep learning framework - Jules Talloen
40.	Scaling Equilibrium Propagation to Deep ConvNets by Drastically Reducing its Gradient Estimator Bias - Axel Laborieux, Maxence M ERNOULT, Benjamin Scellier, Yoshua Bengio, Julie Grollier, Damien Querlioz	[Video]
41.	Scaling up learning with GAIT-prop - Sander Dalm
42.	Self Normalizing Flows - Thomas A Keller, Jorn W.T. Peters, Priyank Jaini, Emiel Hoogeboom, Patrick Forré, Max Welling	[Video]
43.	Short-Term Memory Optimization in Recurrent Neural Networks by Autoencoder-based Initialization - Antonio Carta, Alessandro Sperduti, Davide Bacciu	[Video]
44.	Slot Machines: Discovering Winning Combinations of Random Weights in Neural Networks - Maxwell M Aladago, Lorenzo Torresani
45.	Supervised Learning with Brain Assemblies - Akshay Rangamani, Anshula Gandhi	[Video]
46.	Symbiotic Learning of Dual Discrimination andReconstruction Networks - Tahereh Toosi, Elias B Issa
47.	The Interplay of Search and Gradient Descent in Semi-stationary Learning Problems - Shibhansh Dohare, Rupam Mahmood, Richard S Sutton	[Video]
48.	Towards self-certified learning: Probabilistic neural networks trained by PAC-Bayes with Backprop - Maria Perez-Ortiz, Omar Rivasplata, John Shawe-Taylor, Csaba Szepesvari
49.	Towards truly local gradients with CLAPP: Contrastive, Local And Predictive Plasticity - Bernd Illing, Guillaume Bellec, Wulfram Gerstner	[Video]
50.	Unintended Effects on Adaptive Learning Rate for Training Neural Network with Output Scale Change - Ryuichi Kanoh, Mahito Sugiyama	[Video]

Accepted papers will be presented during the poster sessions in [Gather.Town]. This is the map:

[This document] provides guidelines on the Gather.Town virtual platform used for the Beyond Backpropagation Workshop.

Reviewers

We would like to thank all of our reviewers for their great work:

Andrea Zugarini, Apoorva Nandini Saridena, Devansh Bisla, Enrico Meloni, Francesco Giannini, Gabriele Ciravegna, Giovanna Dimitri, Giuseppe D'Inverno, Giuseppe Marra, Haoran Zhu, Jing Wang, Joao Carreira, Lapo Faggi, Laura Sanabria-Rosas, Lisa Graziani, Maryam Majzoubi, Shihong Fang, Takieddine Mekhalfa, Yunfei Teng