Structural and dynamical strategies to prevent runaway excitation in reservoir computing

Engineering notes

Key topics: autonomous driving, control. See the paper for implementation details and experimental results.

Chinese explanation / 中文解读

中文解读待补充：本站会优先为端到端自动驾驶、BEV感知、3D目标检测、轨迹预测、路径规划、LiDAR感知等高价值论文补充中文说明。

Original abstract

Reservoirs, typically implemented as recurrent neural networks (RNNs) with fixed random connection weights, can be combined with a simple trained readout layer to perform a wide range of computational tasks. However, increasing the magnitude of reservoir connection weights to exploit non-linear dynamics can cause the network to develop strong spontaneous activity that drives neurons into saturation, dramatically degrading performance. In this work, we investigate two distinct countermeasures against such runaway excitation. The first approach introduces a subtle non-homogeneous structure into the matrix of connection weights. w ij , without altering the overall probability distribution p ( w ). We identify several favorable structuring principles, such as creating a small subset of neurons with weaker-than-average input connections. Even if the rest of the reservoir falls into runaway saturating behavior, this weakly coupled subset remains in a mildly non-linear regime whose dynamics can still be exploited by the readout layer. The second approach implements a form of automatic gain control (AGC), in which a dedicated control unit dynamically regulates the reservoir's average global activation toward an optimal setpoint. Although the control unit modulates the excitability of the reservoir only via a global gain factor, this mechanism substantially enlarges the dynamical regime favorable for computation and renders performance largely independent of the underlying connection statistics.

Links and sources

• PDF from original source

Comments