a community library of Nengo models, components & networks

← all submissions

ssp-slam

A spiking neural SLAM subnetwork: continuously integrate velocity into a Spatial Semantic Pointer (SSP) estimate of self-position, correct the estimate on landmark sightings, and learn a semantic associative memory mapping landmark identities to SSP locations.

network advanced slam ssp path-integration grid-cells associative-memory semantic-pointers subnetwork

ssp-slam

A spiking neural SLAM subnetwork: continuously integrate velocity into a Spatial Semantic Pointer (SSP) estimate of self-position, correct the estimate on landmark sightings, and learn a semantic associative memory mapping landmark identities to SSP locations.

Description

This is a curated NengoZoo wrapper around the SLAM network from Dumont et al. (2023). It composes three pieces:

  1. Path integrator — a velocity-controlled-oscillator network maintaining a continuous SSP estimate of self-position. (Same algorithm as the ssp-path-integrator submission, used internally here.)
  2. Position correction — on each landmark sighting, the network shifts its self-position SSP toward landmark_position − vector_to_landmark, snapping the estimate back to ground truth and arresting integrator drift.
  3. Associative memory — a Voja+PES learning loop pairs landmark identities (semantic pointers) with the corresponding SSP locations. After training, semantic queries like "blue triangle" or "all blues" return SSP similarity maps over the domain.

The wrapper exposes the upstream SLAMNetwork's ports as clean attributes (velocity_input, landmark_vec_ssp, landmark_id_input, no_landmark_in_view, plus pathintegrator, position_estimate, and assomemory for downstream probing). It's intended as a drop-in component for navigation models that want both a corrected self-position estimate and a queryable semantic map.

Installation

python -m venv .venv
source .venv/bin/activate          # Windows: .venv\Scripts\activate
pip install -r requirements.txt

sspslam is pulled from Nicole Dumont's GitHub repository (not on PyPI). The nengo-loihi line is a workaround — sspslam hard-imports it through its utils.network_diagram even when you don't use Loihi.

Usage

import nengo
import numpy as np
import sspslam
from ssp_slam import SSPSlam

ssp_space = sspslam.HexagonalSSPSpace(
    domain_dim=2, n_scales=2, n_rotates=3,
    domain_bounds=np.array([[-1, 1], [-1, 1]]),
    length_scale=0.3, seed=0,
)
d = ssp_space.ssp_dim

# Build your landmark SP space (n_landmarks, d).
landmark_sps = np.random.randn(4, d); landmark_sps /= np.linalg.norm(landmark_sps, axis=1, keepdims=True)
lm_space = sspslam.SPSpace(4, d, seed=0, vectors=landmark_sps)

with nengo.Network() as model:
    slam = SSPSlam(
        ssp_space=ssp_space, lm_space=lm_space, n_landmarks=4,
        view_rad=0.3,
        pi_n_neurons=500,
        mem_n_neurons=10 * d,
        circconv_n_neurons=100,
        vel_scaling_factor=vel_scaling_factor,  # see example
    )

    nengo.Connection(velocity_node, slam.velocity_input, synapse=None)
    nengo.Connection(init_ssp_node, slam.pathintegrator.input, synapse=None)
    nengo.Connection(landmark_vec_node, slam.landmark_vec_ssp, synapse=None)
    nengo.Connection(landmark_id_node, slam.landmark_id_input, synapse=None)
    nengo.Connection(no_view_gate_node, slam.no_landmark_in_view, synapse=None)

    self_ssp = nengo.Probe(slam.pathintegrator.output, synapse=0.05)

See examples/example_usage.py for the full toy-environment demo (path generation, environment with objects + a wall, semantic memory queries).

How it works

The path-integrator half is the same as in ssp-path-integrator: velocity is projected onto each of the SSP's Fourier components, and a recurrent oscillator (with attractor dynamics) tracks each component's phase over time.

On top of that, two additional pathways:

  • Landmark binding. When the agent is within view_rad of a landmark, an upstream input function provides (i) the landmark's semantic-pointer ID and (ii) the SSP-encoded vector from agent to landmark. Circularly convolving the inverse of the current self-SSP with the vector-to-landmark SSP gives the SSP-encoded landmark location; the corrected self-SSP is then this location bound with the inverse vector-to-landmark.
  • Semantic memory. A Voja+PES learning loop trains an associative memory from landmark IDs (semantic pointers) onto SSP locations. After enough sightings, querying the memory with a semantic term like bind(triangle, sum(color_sps)) returns an SSP whose similarity map over the domain peaks at every triangular landmark — a single neural circuit performs semantic-conditioned spatial recall.

For the math, see Dumont et al. (2023).

Citation

@article{dumont2023exploiting,
  author  = {Dumont, P. Michaela Nicole and Furlong, P. Michael and Orchard, Jeff and Eliasmith, Chris},
  title   = {Exploiting semantic information in a spiking neural {SLAM} system},
  journal = {Frontiers in Neuroscience},
  volume  = {17},
  year    = {2023},
  doi     = {10.3389/fnins.2023.1190515}
}

License

GPLv2 for this wrapper (see LICENSE). The underlying sspslam package is distributed by Nicole Dumont under its own license (Apache-2.0 per the upstream README).

Figures

environment

The toy 2-D environment used by examples/example_usage.py.

A seeded nengo.processes.WhiteSignal generates the agent's trajectory (gray) starting at the black star, of which the first T = 8 s are simulated. Three semantic objects — a blue triangle, an orange triangle, and an orange square — are placed at fixed points along the path; their positions are determined by the seed and the path period (held fixed at 10 s), so the layout stays the same regardless of T. A single black wall sits just off the trajectory in view of the agent without occupying the path.

Each object's semantic identity is the binding of a shape semantic pointer (triangle or square) with a color semantic pointer (blue or orange); this is what enables the memory-query figure to resolve queries like "all triangles" or "all blues" after training.

error_curves

Self-position estimation accuracy over the 8-second run, for SSP-SLAM (blue) versus a stand-alone SSP path integrator (orange).

The metric is cosine error in the SSP space: 1 − cos(estimate, true SSP). This is the same robust measure used in the source paper — it sidesteps the grid-decoder noise that would otherwise dominate Euclidean distance error at this CI-sized SSP dimension (ssp_dim = 37).

Both networks are initialised to the true SSP for the first 50 ms. SSP-SLAM stays well below the PI-only curve for essentially the entire run: each landmark sighting after t ≈ 0.5 s lets SLAM snap its position estimate back toward the corresponding memory recall, holding cosine error in the 0.1 – 0.4 range. The pure path integrator, with no correction signal available, drifts up past 0.8 by the midpoint of the run.

The headline at higher fidelity (the Dumont et al. (2023) paper's ssp_dim ≈ 240, ~60-second runs) is that the SSP-SLAM curve stays well below the PI-only curve for the entire trajectory; the CI-sized demo captures the same correction dynamic in compressed form.

memory_queries

Three semantic queries against the associative memory SSPSlam.assomemory at the end of the run.

After training, the memory has learned a mapping landmark_SP → SSP_location for the four landmarks it saw (3 objects + 1 wall). To answer a semantic query, we compose a query semantic pointer by binding shape × color terms, feed it as the input to the memory, and read out the output — an SSP. We then compute that SSP's similarity to a 60×60 grid of position SSPs over the domain and contour-plot the result.

Left — "Blue triangle" (bind(triangle_SP, blue_SP)): a tight peak at the blue-triangle landmark in the middle-left of the domain.

Middle — "All triangles" (bind(triangle_SP, blue_SP + orange_SP)): the peak anchors at the blue triangle, then extends through the centre out to the orange triangle in the lower-right — the memory has identified both triangular objects via SP algebra applied at query time.

Right — "All blues" (bind(triangle_SP + square_SP, blue_SP)): a single peak at the blue triangle (the only blue object in the scene).

At this CI-sized ssp_dim = 37, some smearing of the query response onto the agent's transit corridors between landmarks is expected — the memory associates each landmark SP with the SSPs it observed while learning it, which span a short interval of the path. The higher-fidelity setup in Dumont et al. (2023) produces sharper localisation; the compositional structure of the queries is already visible here.

This is the headline semantic capability that distinguishes SSP-SLAM from a plain path integrator — the same neural circuit that holds a self-position estimate also fields compositional spatial queries about what kinds of things are where.