Hands-on Geometric Deep Learning

Hands-on Lie Geometry for Data Scientists

Patrick R. Nicolas — Wed, 26 Aug 2026 04:29:31 GMT

Lie Geometry is a critical element of Geometric Deep Learning and consequentially world models. However the topic can be intimidating to reader with even satisfactory knowledge of differential geometry.

This article introduces Lie groups and algebras with intuitive examples before diving into the mathematical formalism and implementation in Python.

Why this Matters
Key Takeaways
Basics
The Challenges
🏛️ Riemannian Manifolds
🏛️ Lie Geometry
Applications of Lie Geometry
- Common Lie groups
- Practical Applications
Deep Dive
⚙️ Python Environment
📚 Geomstats Library
- Overview
- Installation
🏛️ Theory
⚙️ Special Orthogonal Group
⚙️ Special Euclidean Group
References
⚙️Appendix
Q & A
Paper Review

👉 Patrick Nicolas is a 30-year software engineering veteran and consultant specializing in Geometric Deep Learning and World Models, author of Scala for Machine Learning, and writer of Geometric Learning in Python.

Why this Matters

Purpose: Standard machine learning treated data in flat Euclidean space and therefore struggles to represent many real-world domain such as 3D poses, protein backbones, directional data, climate flows, or dynamic physical system. Lie groups & algebras provide data scientists with the mathematical language to operate on these non-linear spaces efficiently.

Audience: Engineers involved in Robotics, computer vision or data scientists exploring world models or geometric deep learning.

Value: Learn the key properties of Lie groups and algebras using intuitive example before diving into the mathematics and implementation of Special Orthogonal Groups and Special Euclidean Groups.

Key Takeaways

Intuitive foundations: You can grasp the core intuition of Lie geometry well before tackling the formal machinery of differential geometry and Riemannian manifolds.
Dual nature of Lie groups: A Lie group bridges algebra and geometry, behaving simultaneously as a smooth manifold and an algebraic group with the properties of both.
Key applied groups: The Special Orthogonal group SO_n, Special Euclidean group SE_n and Symmetric Positive Definite (SPD) manifolds equipped with Riemannian metrics are central to robotics, climate modeling, computer vision, and drug discovery.
Lie algebra as local linearization: A Lie algebra represents the tangent space at the group’s identity element, effectively linearizing the group’s local curved geometry.
Generators and exponential maps: Elements of SO₃ and SE₃ are constructed in their respective Lie algebras via linear combinations of base generators and mapped back to the group using the exponential map.
Computational tooling: The Python library Geomstats equips data scientists with practical tools to analyze, test, and implement geometric and Lie-group-based algorithms.

Basics

This section describes how Lie groups and algebras addresses the most common limitations of current deep learning and world models.

The Challenges

Most of deep learning models, including world models fail to respect or enforce continuous symmetry naturally occurring with real-world objects such as 3D poses, point clouds or molecular structures (images or video scenes) [ref 1]

Existing deep learning models, including Large Language Models fail to represent the natural symmetry found in the physical world.

Let’s consider the following sequence of objects that are created from continuous deformation (rotation, translation, scaling, ) of an original object [ref 2]

Fig. 1 Illustration of the continuous deformation/transformation of a 3D object

Can we create the sequence from a single original object using a geometric and algebraic transformation of infinitesimal steps?

The answer is yes. The geometric deformation is known as a Lie Group and its algebraic operation as the Lie Algebra.

🏛️ Riemannian Manifolds

Smooth and Riemannian manifolds have been the subject of previous articles in this newsletter.

For a brief overview A Friendly Primer on Geometric Deep Learning - Manifolds

For detailed description

The study of Smooth and Riemannian manifolds is part of Differential Geometry.

Here is a quick review of key concepts.

Why Smooth Manifolds

Geometric Deep Learning addresses the difficulties of limited data, high-dimensional spaces, and the need for independent representations in the development of sophisticated machine learning models, including graph-based and physics-informed neural networks. Data scientists face challenges when building deep learning models that can be addressed by geometric, non-Euclidean representation of data [ref 3, 4].

Here are some of the limitations of relying Euclidean space to build models.

Euclidean space is flat while real-world data distributions lie on highly curved manifolds. Forcing a curved data manifold into a flat Euclidean latent space introduces significant distortion
To embed a hierarchical tree structure into Euclidean space with low distortion would require high dimensional space. This leads to the curse of dimensionality,
The real world is filled with non-trivial global structures: periodic cycles, loops, and closed surfaces. Euclidean spaces can be continuously shrunk to a single point and cannot natively represent topological features such as holes, voids without tearing the data apart.
Euclidean space treats all directions uniformly. It naturally pulls interpolations toward the mean, reducing the information from the complex world dynamics.

Core Concepts

⚠️ This section briefly introduces key concepts in differential geometry; see the References section for a comprehensive treatment.

A manifold is a topological space that, around any given point, closely resembles Euclidean space. Specifically, an n-dimensional manifold is a topological space where each point is part of a neighborhood that is homeomorphic to an open subset of n-dimensional Euclidean space. Examples of manifolds include one-dimensional circles, two-dimensional planes and spheres, and the four-dimensional space-time used in general relativity [ref 5].

Smooth or Differential manifolds are types of manifolds with a local differential structure, allowing for definitions of vector fields or tensors that create a global differential tangent space.

A Riemannian manifold is a differential manifold that comes with a metric tensor, providing a way to measure distances and angles.

The tangent space at a point on a manifold is the set of tangent vectors at that point, like a line tangent to a circle or a plane tangent to a surface. Tangent vectors can act as directional derivatives, where you can apply specific formulas to characterize these derivatives.

A geodesic is the shortest path (arc) between two points in a Riemannian manifold.

An exponential map is a map from a subset of a tangent space of a Riemannian manifold while the logarithm map assigns a tangent vector to a point on the manifold as illustrated below.

Fig. 2 Illustration of key elements of Riemannian geometry on a 3D sphere

A vector is an arrow pointing through space. A covector is an object that measures vectors moving through space. A covector is a measurement dual to that arrow. While a vector is visualized as an arrow, a covector is best visualized as a set of parallel, evenly spaced hyperplanes(lines in 2D, planes in 3D).

Tensors are multidimensional arrays of numbers that obey specific transformation laws when changing coordinate systems. They track complex, multi-directional relationships.

A vector field assigns a single vector to every point in a space. You can visualize it as an infinite collection of arrows, where each arrow’s base sits at a specific point, its direction points where something is moving or pulling, and its length represents intensity.

A tensor field is a generalization of a vector field. Instead of assigning a simple arrow to every point, it assigns a tensor to every point.

The Riemannian metric g is an operator that assigns an inner product to the tangent space at every single point on the manifold. The Riemannian Metric is a smoothly varying, positive-definite inner product on each tangent space. It allows the calculation of length, angle and volume associated with vector fields.

The Levi-Civita connection solves the issue of comparing vectors on two different spaces (or two different points on the manifold. It provides a consistent rule for directional differentiation and parallel transport. Levi-Civita Connection is a torsion free (symmetric) Riemannian connection.

A Killing field is a vector field on a Riemannian or pseudo-Riemannian manifold that preserves the metric.

Riemannian Geometry is a key component of Geometric Deep Learning, the topic of this newsletter [ref 6]

🏛️ Lie Geometry

Some readers are intimidated by the mathematical formalism of Lie group and more generally Riemannian manifolds. I relegate the formal, mathematical definition of Lie geometry to the Deep Dive section of this article. However, I assume the reader is familiar with the foundation of differential geometry such as Riemannian manifold, tangent space and exponential maps at this point [ref 7].

Fundamentally a Lie group is a smooth manifold AND an algebraic group as illustrated below:

Fig. 3 Illustration of Lie groups leveraging properties of smooth manifold and algebraic group

🧠 Geometric Deep Learning leverages Lie geometry to enforce symmetry, invariance and equivariance with the architecture of neural networks.

Convolutional neural networks leverage translation invariance (Kernel + pooling)
Graph neural networks use invariance of permutation of node indices.
Equivariant Neural Networks rely group equivariance (e.g., rotation)

Lie groups

Intuition for 2-dimensional rotation

A geometric space allows us to visualize all possible 2D rotations at once, without an enumeration of numbers.

Fig. 4 Intuitive visualization of 2-dimensional rotations as a Lie group using a clock

Let’s consider the following scenario

Step 0: This point at 12 ‘o clock represents the “Identity” operation—doing nothing, or a θ=0° rotation.
Step 1: As you move along the circle, you are moving through smoothly changing continuously, stopping at θ=48°
Step 2: You move by an infinitesimal value ∂θ=0.00001°

The circle is a Lie group, SO2, because

The circle contains an infinite number of values therefore it is a manifold
The range of values [0, 360] is limited and countable therefore it is a group

Intuition for 3-dimensional rotation & translation

Fig 5. Intuitive representation of a rotation and a translation on a sphere as Lie group illustrated with a flying drone

Step 0: The drone sits in the launch pad that represents the identity in the operation.
Step 1: The drone moves upward along Z-axis as a translation with infinitesimal increment
Step 2: The drone gyrates, rotates and move along all 3 axes

The drone rotates over the 3 axes and move along the 3 axes, requiring 6-degree of freedom. The drone doesn't snap abruptly but glides smoothly along a continuous trajectory through 6-dimensional poses.

The flying space is a Lie group/manifold, SE3

More formerly

A Lie group G models continuous symmetries (e.g., smooth 3D rotations - Special Orthogonal Group SO3, rigid body transformations - Special Euclidean Group SE3). Because manifolds are curved, directly interpolating or taking gradients on G is non-trivial [ref 7].

Lie Algebra

Being a manifold a Lie group is related to the Lie algebra (tangent space at identity) through exponential and logarithm maps. The Lie algebra g solves this by serving as the flat vector space tangent to the group at the identity. This creates a bridge between curved geometry and linear calculus:

Logarithmic Map: Maps elements from the non-linear group G to the linear tangent space g, allowing standard vector addition, averaging, and optimization.
Exponential Map: Maps linear operations back onto the curved group manifold G, guaranteeing the transformation remains exact without drifting off the manifold.

📌 The mathematical symbols related to Lie algebra use the Fraktur font.

This article uses Upper case for Lie group elements and lower case for Lie algebra.

Here is an illustration of the components of SO₃ Lie group and algebra - exponential and logarithmic maps.

Fig. 6 Illustration of a Lie algebra as tangent vector at identity on a sphere illustrated by a sphere.

Let’s consider the two previous intuitive examples.

In the case of rotation within the plan (clock) the algebra can be expressed as a rotation of angle θ. The continuous move from any point to the circle to another is defined by the 2D rotation matrix.

The angular velocity component at identity t = 0

The unitary matrix is called the standard generator of so2

The Lie algebra for the flying drone in 3D Euclidean space is defined as 4 x 4 matrix combining a 3D rotation or angular velocity [θ_x, θ_y, θ_z] with a linear velocity vector [v_x, v_y, v_z].

📌 The Lie algebra is defined by the tangent plane to the manifold at identity. Therefore its components are first order derivative of motion - velocity [ref 9].

Animation

The animation relies on Manim library [ref 16]. Manim is the gold standard for high-quality, explanatory math videos. It is particularly suitable for Calculus, linear algebra, and complex equations where precise “morphing” between formulas and shapes is needed. Manim is the most commonly used animation on YouTube.

📌 In the two previous examples, the operation associated with Lie algebra elements corresponding to the multiplication of matrix in the Lie group is known as the Lie Bracket.

Applications of Lie Geometry

Common Lie Groups

Although it is mathematically impossible to list all Lie groups (continuous, infinite families), we can categorize them by simple groups using the Cartan-Killing classification.

📌 The Lie manifolds most commonly used in machine learning such as Special Orthogonal, Special Euclidean and Symmetric Positive Definite groups have been studied in more technical oriented article in this newsletter. They will be evaluated with Python implementation in the Deep Dive section.

Every connected simple Lie group corresponds uniquely to a root system, which is classified completely by connected Dynkin diagrams, falling into two categories:

Infinite Families

A_n: Special Linear groups over complex numbers C with a Lie algebra of dimension n(n+2) for n > 0
B_n: Odd-dimensional Orthogonal groups, corresponding to the Lie algebras so_2n+1 of dimension n(2n+1) for n > 1
C_n: Symplectic groups, corresponding to the Lie algebras sp_2n ( complex numbers) of dimension n(2n+1) for n > 2
D_n: Even-dimensional Orthogonal groups, corresponding to the Lie algebras so_2n (complex numbers) of dimension of n(2n-1) for n > 3

Exceptional Groups

G₂ of dimension 14 with rank 2
F₄ of dimension 52 with rank 4
E₆ of dimension 78 with rank 6
E₇ of dimension 133 with rank 7
E₈ of dimension 248 with rank 8

Matrix Groups

GL(n, F) - General Linear - Invertible, square n x n matrices over F = {R, C}
SL(n, F) - Special Linear - Matrix with determinant = 1 over F = {R, C}
O(n) - Orthogonal - Matrix over R preserving Euclidean inner product
SO(n) - Special Orthogonal - O(n) with determinant = 1
SE(n) - Special Euclidean - SO(n) with affine transformation of rotation and translation
U(n) - Unitary - Complex matrix preserving the Hermitian inner product
SU(n) - Special Unitary - U(n) group with determinant = 1
SPD(n) - Symmetric Positive Definite under Log-Euclidean or
Sp(2n, F) - Simplectic - Matrices over F = {R, C} preserving a non-degenerate skew-symmetric bilinear form.

⚠️ The Symmetric Positive Definite (SPD) Group is not a Lie group under standard matrix multiplication. It is a Lie group under two specific algebraic condition: Log-Euclidean and Affine Invariant Riemann metrics (Symmetric Positive Definite Matrices with specialized metric)

The following deep-dive section explores the essential operations on the SO₃ and SE₃ Lie groups, which find widespread application in fields such as robotics and molecular modeling.

Practical Applications

Lie groups and algebras have numerous applications

Table 1 Sample of applications of Lie Geometry in machine learning.

🔓 The rest of this deep dive is exclusive to paid subscribers. By upgrading, you unlock this full article along with a comprehensive archive of engineering articles, paper review, concept breakdowns, code walkthrough, GitHub repository and Q&A.

The Geometric Future of World Models

Patrick R. Nicolas — Wed, 12 Aug 2026 16:51:15 GMT

Have you ever wondered how World Models can represent our observed reality—without any built-in knowledge of the physical laws, geometric invariants, and constraints that govern it

Geometric Deep Learning (GDL) aggregates and modularizes the mathematical concepts that enable reasoning in the latent space of World Models.

Geometric Deep Learning, World Models use complex math to overcome limitations of traditional deep learning. We make that math accessible through hands-on tutorials.

Why this Matters
Key Takeaways
Basic Concepts
World Models
WM Challenges
Geometric Deep Learning (GDL)
GDL for WMs?
Deep Dive
Manifolds & Graph Structures
- Benefits
Topological Domains & Constraints
- Benefits
Invariance & Equivariance
- Benefits
- Symmetry in Real World
- Mathematics of Symmetry
Inductive Biases & Conservation Laws
Summary
Q & A
References
Paper Review

Why this Matters

Purpose: Current World Models fail to enforce symmetries, encode topological invariant & geometric constraints or preserve physics & conservation laws. Geometric Deep Learning (GDL) concepts borrows concept from topology, graph theory and differential geometry to support non-Euclidean geometry, particularly suited to enhance World Models.

Audience: AI Practitioners involved in Robotics, Physical AI and development of World Models.

Value: Explore how key features of GDL overcome the limitations of World Models when processing latent space data.

⚠️ This article analyzes and categorizes the properties in GDL addressing some of the limitations of World Models. It does not contain any implementation or evaluation code.

Key Takeaways

Latent Representation in World Models: Latent spaces form the operational foundation of World Models, driving simulation, optimization, planning, and inference tasks.
Key Structural Deficiencies: Current architectures lack intrinsic mechanisms to preserve physical conservation laws, enforce spatio-temporal symmetries (translation, rotation, reflection, scale), or disentangle action and reward dynamics.
Geometric Deep Learning Integration: GDL equips models with formal tools to embed symmetry, low-dimensional manifold constraints, and topological invariants into Generative models, Spatial Intelligence, and JEPAs.
Data and Compute Efficiency: Explicitly baking geometric priors into the network geometry minimizes sample complexity and dramatically accelerates training convergence.
Noise and Perturbation Immunity: Topological feature extraction ensures stability against metric noise, local variance, outliers, and missing data features.

Basics Concepts

This article asks the question: Is Geometric Deep Learning (GDL) a good fit to implement and enhance world models?

Fig. 1 Illustration of Geometric Deep Learning Features enhancing world models?

This article explores whether Geometric Deep Learning (GDL) is well-suited to implement and enhance world models. Because symmetry is central to GDL, I address this question through a structured process:

Review three key world model architectures.
Summarize key performance requirements for world models.
Introduce essential GDL concepts and features.
Map GDL capabilities directly to world model requirements to evaluate the fit.
Understand symmetry and its foundational importance.

World Models

Why World Models?

World models aim to predict, represent, and reason about the world from raw sensory inputs [ref 1, 2 ] as exemplified by

Generative World Model is often cited as the catalyst for the modern interest in this field. This model decomposed the problem into three distinct components managed/orchestrated by an agent - The Vision Model, The Memory Model and The Controller [ref 3]
Joint Embedding Predictive Architecture (JEPA) focuses on predicting embeddings, not pixels, structured through energy-based principles [ref 4, 5]
Spatial Intelligence models emphasize 3D reasoning, spatial affordances, object permanence, physical laws, and agent–environment interaction [ref 6].

📌 Conditional vs. Unconditional World Models

In the unconditional model, you start with the state of the world at time t. The world is a 𝗱𝘆𝗻𝗮𝗺𝗶𝗰𝗮𝗹 𝘀𝘆𝘀𝘁𝗲𝗺. Can I predict the state of the world at time t + 1? The longer the time horizon you want to run your prediction, the more difficult it is to make accurate, detailed predictions. What you can do is make predictions at a higher level of abstraction, where many details are not present.

Then there are conditional models where, given the state of the world at time t and a 𝗵𝘆𝗽𝗼𝘁𝗵𝗲𝘁𝗶𝗰𝗮𝗹 𝗶𝗻𝘁𝗲𝗿𝘃𝗲𝗻𝘁𝗶𝗼𝗻 can I predict the state of the world at time t + 1?

Standard or Generative WM

The standard (generative) World Model was introduced in 2017 paper by Ha & Schmidhuber [ref 3] as a neural network designed to understand and simulate the dynamics of the observed world, including its physical and spatial properties:

Reconstruction Goal: These models often use a Generative approach, where they attempt to reconstruct original data, such as future video frames or images, from a compressed “latent” representation.
Multimodal Input: They typically process video, sensor streams (like LiDAR), and sometimes language to create a unified internal simulation.
Simulated Training: By predicting future outcomes, an agent can “dream” or plan in a virtual sandbox, which reduces the need for dangerous or expensive observed-world trials.
Common Architectures: Standard world models frequently utilize Variational Autoencoders (VAE) for state compression and Recurrent Neural Networks (RNN), LSTM or Transformers to predict how those states change over time.

This brings us to the Vision-Memory-Controller Architecture as described below:

Fig.2 Illustration of the architecture of the Vision-Memory-Controller Generative World Model - Ha & Schmidhuber

The Vision model encodes the high dimensional data into a latent vector z_t as time or step t. In the original paper [ref 3], the encoder is implemented as a Variational Autoencoder.
The Memory model preserves the current latent vector z_t, representing the physical world then predicts a future temporal state(s) h_t+1using a variant of recurrent neural networks and the predicted latent vector z_t+1 give the reward of action triggered by the controller.
The Controller takes the current temporal state h_t the new state h_t+1 and latent vector z_t+1 as input to select the optimum action. The action triggers a reward that is used to guide the memory model.

📌 The original paper relies on an RNN or LSTM to implement the memory model that can be replaced by deep reinforcement learning strategies.

Spatial Intelligence

Spatial Intelligence models developed by Dr Fei-Fei Li, focuses on understanding of 3D space and time [ref 6]. Unlike LLMs that process tokens, Spatial Intelligence models build persistent 3D worlds to interact with the physical world. These simulated worlds represent the geometric structure of the observable world in represented in the latent space. Therefore, these spatial models are inherently multimodal.

These models incorporate geometry and physical laws to predict an action and the resulting next state. The latent space is often structured as a 3D/4D occupancy grid or a neural radiance field (NeRF). It isn’t just a vector of numbers.

Conceptually, Spatial Intelligence is an abstract world model because it aims to provide the “scaffolding” for cognition. Like the JEPA family, it seeks to move beyond pixel-level correlations toward a deep understanding of physical laws, causality, and geometry.

The spatial intelligence architecture defines the same sequence of components as the standard world model, namely Encoder-Predictor-Renderer with action/reward to guide the trajectory of latent states.

Fig. 3 Illustration of the Encoder-Latent Space-Predictor-Renderer components of Spatial Intelligence World Model

The Encoder (or Perception Module) generates a latent vector z_t as time t from observation x_t. The encoder is implemented as a vision transformer
The Predictor (or World Model Module) predicts the latent state z_t+1 given an agent action a_t and reward and the current context.
The Renderer (or Generation Module) decodes the predicted latent state z_t+1 into the original high dimension media (e.g., video frame, image) x_t+1
The Agent computes the loss between the original observation x_t and the reconstructed data x_t+1 then selects the most appropriate action and collect the reward.

JEPA

Unlike generative models that rely on observation-level reconstruction, Joint Embedding Predictive Architectures (JEPAs), introduced by Yann LeCun, optimize for representation consistency across multiple data views in the latent manifold [ref 7]. This approach avoids the computational burden of density estimation, enabling the encoder to capture abstract, task-invariant features. By operating independently of raw input constraints, the architecture offers superior flexibility in feature encoding. JEPA embeds features that are useful for predicting the next state while discarding unpredictable features.

📌 The JEPA framework has evolved swiftly since late 2023. The following discussion outlines the foundational concepts of the Joint-Embedding Predictive Architecture but does not provide a comprehensive technical audit of its various version-specific functionalities.

In a nutshell, JEPA differs from Generative WM and Spatial Intelligence WM as it computes the loss strictly in the latent space, between target latent values sy_t+1 and predicted latent value sx_t+1 - Loss(sx_t+1,sy_t+1) - instead the generative loss Loss(x_t+1,y_t+1) on observed data.

Fig. 4 Illustration the dual encoder-predictor-energy function minimizer of the Joint Embedding Predictive Architecture

A Encoder generates a latent vector sx_t as time t from observation (or context input) x_t, using a vision transformer
Another Encoder generates a latent vector sy_t from the target y_t
The Predictor computes the next representation sx_t+1from the current latent state z and the encoded context input sx_t
The latent target vector for the next step sy_t+1, is computed through an exponential moving average, using the target value y_t+1 aligned with the context sx_t
The predicted latent value sx_t+1and latent target value sy_t+1, are compared by minimizing the related energy function.

While Reinforcement Learning (RL) and Model Predictive Control (MPC) both aim to solve the same fundamental problem—finding a sequence of actions that maximizes a reward or minimizes a cost over time—they come from completely different philosophical backgrounds.

📌 JEPAs are particularly relevant in processing geometric priors and manifolds because they treat the latent space as the primary arena for learning.

WM Challenges

World models must encode structure, symmetries, and spatial relationships. Here is a list of requirements:

Embedding geometric, physical constraints as inductive biases bridges the gap between purely statistical pattern matching and true physical grounding. It avoids the need to learn basic physical principles from scratch through observations.
Incorporating translation, rotation, reflection, scaling invariance reduces the need for large data sets. For example, there is no need to train a model on rotated and translated objects as they referred to the same objects.
Constraining a WM eliminates the compounding effect of small errors along the trajectory in latent space. These geometric constraints include spatial manifolds and Topological continuity.
Conserving laws to structure the latent space through Hamiltonian and Lagrangian.
Enforcing 3D spatial relationships enable WM to reason about distance, angles … independently of texture, shape.
Factorizing some actions on objects, allows WM to avoid retraining the transition functions.
Sharing model parameters by enforcing symmetry

📌 These requirements will be reviewed in detail in the Deep Dive section.

Geometric Deep Learning (GDL)

Geometric Deep Learning (GDL) is a branch of machine learning dedicated to extending deep learning techniques to non-Euclidean data structures. Currently, there is no universally agreed-upon, well defined scope for Geometric Deep Learning, as its interpretation varies among authors.

GDL unlocks the ability to model complex, structured, and relational data, enabling advanced problem-solving in areas like social networks, molecular science, design optimization, and transportation systems.

Geometric Deep Learning (GDL) has been introduced by M. Bronstein, J. Bruna, Y. LeCun, A. Szlam and P. Vandergheynst in 2017 [ref 8]. Readers can explore the topic further through a tutorial by M. Bronstein [ref 9, 10].

I arbitrary break down the field into several categories

Graph Neural Networks
Mesh Modeling
Manifold & Group-Based Learning
Topological Deep Learning

A Graph Neural Network (GNN) is an optimizable transformation on all attributes of the graph (nodes, edges, global context) that preserves graph symmetries (permutation invariances).

A mesh in discrete differential geometry is a finite, piecewise-linear approximation of a smooth geometric object. The most common type are simplicial and combinatorial complexes (vertices, edges, faces) together with a geometric embedding of its vertices in Rn (most often 3D).

Machine learning models based on manifolds leverage differential geometry [ref 11]. A manifold is essentially a space that, around every point, looks like Euclidean space, created from a collection of maps (or charts) called an atlas, which belongs to Euclidean space.

Topological Deep Learning (TDL) is an emerging area of machine learning that extends traditional deep learning and graph neural networks (GNNs) to operate on higher-order topological structures beyond simple graphs [ref 12, 13]. While GNNs model pairwise relationships (nodes and edges)

GDL for WMs?

Encoding WMs

For sake of simplification, I break down GDL features into two generic properties:

Property 1: Provides the mathematical tools to learn over geometric structures or domains: graphs, meshes, manifolds, groups, topological spaces.

Property 2: Focuses on symmetries, invariances, and inductive (geometric) biases that correspond to physical laws or geometric constraints.

Digging a bit deeper, property 1 can be defined as supporting

Symmetry, invariance & equivariance
Inductive geometric bias & conservation laws

while property 2 enables

Representation on non-Euclidean (Riemannian) geometry
Modeling with topological complexes and constraint such as persistent homology

For the rest of this article, I will follow the color coding

Geometric Priors

The key question is how can I define and enforce symmetries, conservation, physical laws, non-Euclidean structures and topological constraints into a World Model?

GDL enforces the prior knowledge about symmetries, physics and conservation allows local geometric structures and global topology of observed space directly into the architecture of the architecture of encoder and predictor components as illustrated in the diagram below.

Fig. 5 High level of Geometric Deep Learning properties applicable to the improvement of World Models

GDL enforces geometric, physical, and topological constraints—including symmetry, conservation laws, and non-Euclidean domains—directly through its encoder-decoder architecture. By embedding structural constraints into the encoder and decoder, GDL intrinsically respects physical conservation laws, symmetries, non-Euclidean geometries, and global topology.

📌 Encoder & predictor modules are implemented as deep neural networks (e.g. transformers)

GDL enforces prior knowledge on the observed world, such as symmetries, Physics law and topological constraints directly through the architecture of neural networks such as

Equivariant, Steerable neural layers
Graph, Cell and Simplicial networks
Hamiltonian & Lagrangian neural networks

The mechanism to enforce symmetries, conservation laws, manifolds … is described in detail in the Deep Dive section

Quest for Symmetry

Symmetry play an important role in World Models where reasoning task such as prediction, planning or simulation are performed entirely in the latent space.

The concept of symmetry is central to Geometric Deep Learning and world models. It is the glue they link these two concepts. Let’s start with a formal definition.

“The property of remaining invariant under certain changes (as of orientation in space, of the sign of the electric charge, of parity, or of the direction of time flow) —used of physical phenomena and of equations describing them” - Webster dictionary

The benefits of symmetry are:

Simplification: Symmetry makes geometry easier to handle by turning intricate geometric problems into simpler algebraic ones.
Insight: It reveals how local and global properties of manifolds are connected, providing a clearer view of their structure.
Applications: In physics, symmetric spaces are central to theories such as general relativity and string theory, which use them to model the universe.

Fractal Dimension for Configuring Convolutional Networks

Patrick R. Nicolas — Thu, 30 Jul 2026 00:51:33 GMT

Challenged by the configuration of Convolutional Neural Networks for complex images?

Incorporating fractal dimension analysis offers a promising strategy. By quantifying structural complexity, this method aids in rationally configuring essential parameters such as kernel size, padding, and pooling.

Geometric Deep Learning, World Models use complex math to overcome limitations of traditional deep learning. We make that math accessible through hands-on tutorials.

Why this Matters
Key Takeaways
Fractal Complexity Basics
Why Fractal Dimension
What is a Dimension After All
🏛️ Fractal Dimension
🏛️ Box Counting Method
🏛️ Log-log Regression
Fractal Complexity Deep Dive
🏛️ Reference Shapes
Python Environment
Fractal Dimension Pipeline
⚙️ Setup
⚙️ Image Processing
⚙️ Box Counting Implementation
⚙️ Evaluation
References
Q & A
Appendix
⚙️ Fast Box Counting
⚙️ Koch Curve Generation
Paper Review

Why this matters

Purpose: Configuring the parameters of a 2D convolutional neural network (CNN), such as kernel size and padding, can be challenging because it largely depends on the complexity of an image or its specific sections. Fractals help quantify the complexity of important features and boundaries within an image and ultimately guide the data scientist in optimizing his/her model.

Audience: Data scientists looking for guidelines to estimate the architectural parameters of a 2D CNN.

Value: Learn basic concept and implementation of the computation of the fractal dimension using the box counting method.

Key Takeaways

Fractal dimension calculations guide tuning of CNN parameters like kernel size, stride, and pooling.
The box-counting algorithm estimates the complexity of objects within an image.
An image pre-processing pipelines removes background artifacts to yield precise fractal dimension measurements.
The Koch curve is commonly used to validate fractal dimension implementation

Fractals Complexity Basics

I apply fractal dimension computation to the parameterization of convolutional neural networks, though the approach is equally applicable to other deep learning models."

Why Fractal Dimension?

Fractal dimension quantifies the ‘geometric complexity’ of an object or image. Therefore it is useful in configuring parameters and architecture of convolutional networks (CNNs). It bridges the gap between spatial data complexity and model parameterization.

Configuring receptive fields. A receptive field grows in accordance with the spatial information scaling of the data. For instance, large kernel and stride would accommodate sparse geometrical features to avoid redundancy. Likewise, complex object such as medical imaging or geographic coastlines, need smaller, denser kernels with overlapping (small) strides.
Fractal Pooling; Max pooling ensures translation invariance by stripping out spatial layout information that may cause the CNN to underperform. A fractal pooling module captures the actual spatial distribution of features - known as Lipschitz invariance.

Fig. 1 Logic for the application of the fractal dimension to the configuration of a Convolutional Neural Network

Lipschitz invariance is the property of a function or transformation to preserve relative distance between data points tup to a bounded scaling factor.

Given 2 metric space (X, d_X) and (Y, d_Y) and a map f: X → Y

The topology of the space is preserved under deformation if

⚠️ How to update a CNN’s configuration using the fractal dimension of embedded objects lies outside the scope of this article and will be covered in a future piece.

What is a dimension after all?

Here are 3 definitions of dimension

Dimension in Euclidean space (line:1, plane: 2, Sphere: 3, ..)
Number of variables in a dynamic system or features in a model
Hausdorff dimension: Given an object in a D-dimensional Euclidean space, the measure increases by N=r^D each time its spatial dimension decreases by 1/r

Fig. 2 Illustration of the Hausdorff dimension.

📌 The Hausdorff dimension is not restricted to integers, It can be a floating point values as with fractals.

Fractal Dimension

A fractal dimension is a measure used to describe the complexity of fractal patterns or sets by quantifying the ratio of change in detail relative to the change in scale [ref 1].

Initially, fractal dimensions were used to characterize intricate geometric forms where detailed patterns were more significant than the overall shape. For ordinary geometric shapes, the fractal dimension theoretically matches the familiar Euclidean or topological dimension.

However, the fractal dimension can take non-integer values. If a set’s fractal dimension exceeds its topological dimension, it is considered to exhibit fractal geometry [ref 2].

There are many approaches to compute the fractal dimension [ref 1] of an image or a 3D object among them:

Variation method
Structure function method
Root mean square method
R/S analysis method
Box counting method

Box Counting Method

The box counting method is similar to the perimeter measuring technique we applied to coastlines. However, instead of measuring length, we overlay the image with a grid and count how many squares in the grid cover any part of the image. We then repeat this process with progressively finer grids, each with smaller squares. By continually reducing the grid size, we capture the pattern's structure with greater precision.

📌 The box counting method is also known as the Minkowski–Bouligand dimension.

Given the number of boxes of side length ε, N(ε) the box count dimension is computed as

The limit can be approximated with a Log Power-Law formula

The limit superior and inferior limits are used If the above limit does not exist. Computing the limit can be intractable as we cannot take the limit to absolute zero in a discrete digital grid.

Log-log Regression

A reliable approximation method is to fit a straight line to the log-transformed data points using Ordinary Least Squares (OLS) regression. The steps are

Overlay a grid of square or cubic boxes of side length ε over the object.
Count the number of boxes intersecting with the object’s boundary
Repeat 1-2 for decreasing value of ε
Apply regression log N( ε) vs. log( ε)

Given a sequence {ε_i } of n box sizes and the corresponding n count {N_i}, the OLS regression minimizes the dimension D

Q & A Answers

Patrick R. Nicolas — Sun, 26 Jul 2026 18:20:23 GMT

You have questions … We have answers

Deep Dive Q&A is an exclusive feature for paid subscribers.

Persistent Homology for the Rest of Us

Patrick R. Nicolas — Thu, 16 Jul 2026 17:38:06 GMT

Persistent homology has numerous applications in material science (namomaterial), chemistry, biology (protein folding, drug discovery), computer vision (texture & shape recognition), financial time-series analysis, neural science (neural activity) and above all topological deep learning. It offers an elegant and robust framework for separating structural features from noise, revealing the presence of loops, cycles, and singularities without relying on a metric or coordinate systems.

Geometric Deep Learning, World Models use complex math to overcome limitations of traditional deep learning. We make that math practical through hands-on tutorials.

Why this Matters
Key Takeaways
Persistent Homology Basics
🏛️ Topological Data Analysis
🏛️ Persistent Homology
🏛️ Filtration
📚 TDA Libraries
- Scikit-learn TDA
- Ripser
Persistent Homology Deep Dive
⚙️ Python Environment
⚙️ Evaluation
- Data Generation
- Evaluation Filtration
⚙️ Persistence Diagrams
⚙️ Persistence Barcodes
References
Q&A
Appendix
Paper Review

👉 Patrick Nicolas is a 30-year software engineering veteran and consultant specializing in Geometric Deep Learning and World Models, author of Scala for Machine Learning, and writer of Geometric Learning in Python.

Why this Matters

Purpose: Analyzing the overall geometric structure of data on a low-dimensional smooth manifold is notoriously difficult. Persistent homology simplifies this by providing a reliable way to distinguish real features from background noise—allowing us to detect loops, cycles, and singularities based purely on the data’s underlying distance relationships.

Audience: Data scientists and engineers developing world models, geometric or topological deep network or involved in topological data analysis.

Value: Learn to apply Vietoris-Rips filtration to noisy shaped-data, and create persistence diagrams and persistent barcodes using Scikit-learn TDA

Key Takeaways

Persistent Homology: A tool that maps the overall shape of complex data by identifying and tracking structural features (like holes or cavities) at different levels of detail.
Filtration: The process of expanding spheres around data points to build interconnected shapes, noting exactly when new structural features form and when they fill in.
Birth-Death Diagrams: A chart mapping the lifespan of these features, acting as a distinct geometric signature for the dataset.
Persistence Barcodes: A barcode-like chart that illustrates how long each shape feature survives as the scale changes during filtration.
Python Libraries: Scikit-learn TDA along with Ripser library provides a unified interface to specialized Python libraries for persistent homology, making it one of the most convenient tools for generating and manipulating persistence diagrams and persistent barcodes.

Persistent Homology Basics

🏛️ Topological Data Analysis

Many challenges in data analysis echo two fundamental integration problems:

How to recover high-dimensional structure from low-dimensional representations
How to assemble discrete samples into a coherent global form

Topological Data Analysis (TDA) is a methodology that applies concepts from algebraic topology and computational geometry to analyze and extract meaningful patterns from complex datasets. It provides a geometric and topological perspective to study the shape and structure of data [ref 1]. TDA seeks to develop rigorous mathematical, statistical, and algorithmic techniques to infer, analyze, and leverage the intricate topological and geometric structures underlying data, often represented as point clouds in Euclidean or more general metric spaces.

The most common topological domains [ref 2, 3] are

Simplicial Complexes
Cellular Complexes
Hypergraphs
Combinatorial Complexes

Some of the most common concepts used in TDA are:

Persistent homology captures the topological structure of data across different spatial resolutions. Features that remain stable over many scales are interpreted as genuine properties of the space, while short-lived ones are viewed as noise or artifacts.

A chain complex is the study of spaces and shapes in algebraic topology, consisting of a sequence of mathematical objects connected by homomorphisms. Chain complexes are structured so that the composition of any two consecutive maps is zero.

While chain complexes move downward in dimension in the sequence of objects, the cochain complexes, associated with cohomology moves upward in dimension.

A Cohomology from a cochain complex gives algebraic invariants of a space. Compared to homology, cohomology often carries additional structure.

Filtrations of a space across scales give rise to persistent homology, summarizing the birth and death of features; distances between summaries are stable under small perturbations.

Homotopy theory is a branch of algebraic topology that studies the properties of spaces that are preserved under continuous deformations, called homotopies. It helps in distinguishing spaces that are topologically the same from those that are not and offers tools for working with complex spaces in mathematics and applied sciences

🏛️ Persistent Homology

Persistent homology is the process of capturing the global geometric and topological structure of complex data and identifying and tracking features [ref 4, 5, 6] such as loops, cycles, and voids across multiple scales, as illustrated below:

Fig. 1 Homology class dimension 0, 1 & 2 for a sphere and Torus

📌 The rank of a homology class Hn is directly associated with the Betti number.

Persistent homology can be summarized as a three-step procedure as follows:

Filtration: Generate a nested sequence of simplicial complexes derived from the data. Through filtration, Persistent Homology tracks the emergence and disappearance of topological features across multiple scales, revealing structural properties of the data that traditional methods often miss.
Persistence diagrams: Record the evolution of topological features across this sequence.
Machine learning integration: Transform the persistence diagrams into vectors for efficient use in ML models.

🏛️ Filtration

The filtration method the most commonly used in Topological Data Analysis is the Vietoris-Rips filtration that can arbitrary be broken down into 3 steps

Growing Spheres: We place a ball of radius ε around each point in space (point cloud).
Building Complexes: We increase ε 0 → infinity, to expand these balls. When two balls intersect, a line segment (1-simplex) connects them. When three intersect, a triangle (2-simplex) forms, creating a simplicial complex.
Birth and Death: The topological features slowly appear and eventually get filled in.
- Birth: The radius ε at which a specific topological hole first appears.
- Death: The radius ε at which that same hole is filled in or merges into a larger component.
  Fig. 2 Visualization of a Vietoris-Rips filtration for 10-point cloud

The Čech Filtration is a variant of the Vietoris-Rips filtration. The key difference is adding higher-dimensional simplex only when all the respective balls mutually intersect. This method is more accurate but requires greater computation resources.’

A chain complex C is a sequence of R modules (e.g. vector spaces) and linear maps ∂ called boundary operators, such as

A cochain complex C is a sequence of R modules (e.g. vector spaces) and linear maps d called coboundary operators, such as

A filtration is defined as a sequence of topological X_ispaces of increasing size

The dimension of the homology group is defined as

k=0: Connected components
k=1: Loops
k=2: Cavities
…

The k-th homology groups Hk for each of the topological space Xi and its induced linear map i associated to the inclusion.

📌 A chain complex moves downward in dimension while a cochain moves upward in dimension

I select a simple 6-point cloud evenly distributed in a circle to illustrate the steps of filtration.

Fig. 3 6-stage Vietoris-Rips filtration for circle shape point cloud

Filtration stages:

A ball of diameter ε is assigned to each of the original 6 components (or points).
Although the size of the ball increases the components are still independent
As ε increases, the simplices overlap around a circle to create a loop form as a new complex
At this stage, ε = 0.8, the only edges created are between consecutive components of the loop simplex
The filtration discovers new edges between components across the loop for ε = 1.0
The filtration process identifies all possible edges (15), as ε increases toward infinity

📌 The maximum number of edges of an undirected graph with n nodes are

📚 TDA Libraries

Scikit-learn TDA

The scikit-learn TDA package generally refers to the ecosystem of Topological Data Analysis tools that follow the scikit-learn API, most notably the packages from the scikit-TDA organization [ref 7].

Practically, scikit-TDA is a collection of Python libraries that bring Topological Data Analysis techniques into the scikit-learn workflow such as:

Persistent homology & persistence diagrams
Vietoris–Rips complexes
Topological feature vectors
Machine-learning models leveraging topological domains

Scikit-learn TDA leverages two 3rd party packages

Ripser for fast persistent homology computation
Persim for creating and visualizing persistence diagrams

Installation:

pip install scikit-tda

Ripser

Ripser.py is a lean persistent homology package for Python built upon the fast C++ Ripser engine [ref 8]. It supports:

Computing persistence homology & cohomology
Handling both sparse and dense data sets
Approximating sparse filtration
Visualizing persistence diagrams
Computing lowerstar filtrations on images
Computing representative cochains.

Installation:

pip install Cython
pip install Ripser

Adding Premium Section

Sun, 12 Jul 2026 19:25:34 GMT

Over the past year, this publication has become a deeply technical library for geometric deep learning and world models. I have recently upgraded the newsletter’s structure to optimize how you navigate our articles and resources. To sustain the heavy research required to deliver these guides twice a month, I am adding a tiered section to the existing articles beginning July 16, 2026.

Basics Subscription: Get full access to foundational theory, core concepts, and key takeaways. Includes complete introductory articles, mathematical formalism, curated Master References, and Index pages - Free

Deep Dive Subscription: Unlock everything in the Free tier, plus advanced topics, hands-on Python code walkthroughs, practical evaluation and applicability guides, interactive Q&As, and comprehensive reviews of research papers and books - $9/month or $85/year

Feel free to visit the extensive FAQ page for details information about the newsletter and the posts.

Thank you for reading, building, and supporting this newsletter.

Geometric Deep Learning, World Models use complex math to overcome limitations of traditional deep learning. We make that math practical through hands-on tutorials.

Hands-on Geometric Deep Learning - staff

Research Paper & Book Reviews

Patrick R. Nicolas — Wed, 08 Jul 2026 19:12:41 GMT

Geometric Deep Learning, World Models use complex math to overcome limitations of traditional deep learning. We make that math practical through hands-on tutorials.

🔐 Access to book and research paper reviews is exclusive to paid subscribers.

The Irreverent Geometry of Consciousness

Patrick R. Nicolas — Mon, 08 Jun 2026 04:44:35 GMT

Can differential geometry and algebraic topology help us decoding neural dynamics?

Geometric Deep Learning and World Models rely heavily on the mathematical frameworks of Differential Geometry and Topology. Can we extend these same abstractions to map the landscape of the mind—representing neural dynamics, perception, and even states of consciousness as geometric structures?

Let’s dive into this radical hypothesis.

Geometric Deep Learning, World Models use complex math to overcome limitations of traditional deep learning. We make that math practical through hands-on tutorials.

Why this Matters
Key Takeaways
What is Consciousness
Geometric Transcendence Pipeline
Neural Activity
Neural Manifolds
Perceptual Manifolds
Space of Qualia
Conscious Coherence
- Theory
- Interpretation
References
Q & A
Paper Review
Appendix
Sheaf Theory
Cohomology
Sheaf Cohomology

Why this Matters

Purpose: This newsletter has emphasized the increasing critical role of advanced mathematics and geometry in World Models and Geometric Deep Learning. Would it be a stretch to evaluate how geometry would benefit our understanding of brain activity?

Audience: Scientists and engineers exploring the frontier where geometry, topology, and neuroscience collide.

Value: Analyze the multi-layered neural abstractions that correspond to advanced geometric and topological structures, specifically focusing on manifolds, sheaves, and cohomology.

Key Takeaways

From a geometric standpoint, consciousness is the process of translating raw physical reality (sensory data) into increasingly abstract layers of understanding.
This abstraction moves upward, starting at raw neural activity, progressing to neural manifolds, then to perceptual manifolds, entering the Space of Qualia, and ultimately culminating in (geometric) consciousness.
A neural manifold is a continuous geometric space that mathematically model how the brain maps these sensory inputs using dimensionality-reduction techniques such as Isomap.
A perceptual manifold is built by refining the neural manifold, untangling sensory data, removing background to achieve perceptual invariance.
The sheaf theory enables to stitch these individual perceptual manifolds together into a unified Space of Qualia,.
Algebraic cohomology, helps identify global coherence (absence of obstruction) within a specific Quale, as an approximation of consciousness.

What is Consciousness?

Let’s start with some famous quotes about consciousness …

“The past perpetuates itself through lack of presence. The quality of your consciousness at this moment is what shapes the future.” Eckhart Tolle

“It was the realization by physicists that photons have consciousness, and not just limited consciousness, but awareness of the entire cosmos” Kenneth Schmitt

“consciousness — in its phenomenal dimension, in the sense of what it is like to have an experience at all, the qualia of seeing red, of feeling pain, of being anything — belongs in that same category” Bradley Love - Consciousness, AI, and the Limits of Scientific Explanation.

… conscious machines

“We see minds in machines the way we see faces in clouds. The projection is ours...” Anil Seth, Neuroscientist

“Is ChatGPT conscious? No. It has no sense of the world, no sense of self, no experience of anything. It’s just a text predictor. It’s a very good one, but it’s not a mind.” Yann LeCun, Former chief AI Scientist at Meta

… and a definition:

“Consciousness: The quality or state of being aware especially of something within oneself.” Webster Dictionary

👉 From a philosophical perspective, consciousness is the capacity for subjective, first-person experience (Qualia) or the distinct feeling of what it is like to be a conscious being. It is related to the mind-body relation within ourselves.

👉 From a neurobiological perspective, consciousness is the emergent property of complex networks within the brain. It relies on the continuous integration of sensory input and internal representations across the thalamocortical system (connection thalamus ←→ cerebral cortex) to generate a unified, subjective experience of the world and the self.

👉 From this article, geometric perspective, we make the following assumption: As we process sensory data from the physical world, it transforms through successive stages of abstraction. This cognitive ascent can be modeled mathematically as a pipeline of geometric concepts, mapping physical reality to higher-level understanding - Geometric Transcendence Pipeline

⏭️ The debate related to AI consciousness and AGI have been raging on in the social media in recent months. It is beyond the scope of this article. For those interested, I would suggest the following articles [ref 2, 3, 4]

Geometric Transcendence Pipeline

Here are the key mathematical concepts introduced in this article

Individual neurons fire in high-dimensional space, but their collective population activity restricts itself to a lower-dimensional smooth manifold.
Sheaf Theory is applied to link small clusters of neurons activity to what the brain is actually “thinking” while ensuring consistency.
Cohomology is used to detect structural properties, obstructions, or “loops and holes” within that neural data shape or decision-making states.

📌 The concept of Geometric Transcendence Pipeline is unique to this article and visualize the layering architecture of perception. It is inspired by the paper On Brain as a Mathematical Manifold: Neural Manifolds, Sheaf Semantics, and Leibnizian Harmony which is reviewed at the end of the article.

The data doesn't just pass through layers; it transcends its previous representation via geometric operations—shifting from a discrete point-process space into a continuous Hilbert space, untangling into invariant sub-manifolds, and finally closing topologically into a unified conscious experience as illustrated below:

Fig. 1 Illustration of the Hierarchical Neural Architecture of Brain Activity - Geometric Transcendence Pipeline

📌 The sections Neural Activity and Neural Manifolds have been introduced and illustrated in detail in the previous article Decoding Neural Manifolds

Neural Activity

Following our detailed description of neural activity [ref 1], this section provides a quick recap of the core concepts.

👉 Neural activity refers to the complex electrochemical processes by which neurons generate, and process information. For this article, I restrict the definition as the action potential or spike generated by a biological neuron. Neural activity is obviously constrained by the properties of the physical network of neurons [ref 5, 6]

Researchers collect brain activities (neuron firing) through electrodes. The neural activity can be visualized as spikes as illustrated below:

Fig. 2 Illustration of neural activity for 4 biological neurons with a sliding time window

Neural manifolds are fundamentally smooth manifolds and therefore we need to convert the train of spikes associated with each neuron into a smooth function.

Smoothing with Convolution

Given a Kernel K and a spike train S, the Spike Density Function is computed as a convolution f

Using an exponential kernel

The exponential kernel is used in computational neuroscience because it can be computed recursively without needing to store the entire history of spikes as follows:

Using a Gaussian kernel

The following Manim dynamic visualization illustrates the key points related to Neural Activity.

Fig. 3 Illustration of Spike Density Function applied to the neural activity of 5 neurons

The value of the smoothing Spike Density Function is proportional to the number of spike (neuron firing) within the time window.

Neural Manifolds

Following our introduction to neural manifolds in Decoding Neural Manifolds, this section provides a quick recap of the core concepts.

The next step is to represent the neural activity as a neural trajectory in the Euclidean space of dimension = number of neurons. Neural manifolds are abstraction of neural trajectories and neural dynamics as low representation of high dimensional neural activity [ref 1].

👉 A neural manifold is a continuous topological space that represents how the brain (or an artificial neural network) organizes objective sensory inputs. For example, the brain continuously maps an object moving into space into a continuous surface - smooth manifold - embedded in a n-dimensional space of neural activity.

Fig. 4 Visualization of a 2-dimensional neural manifold representation of the neural activity of 3 biological neurons

There are several techniques to extract the low-dimensional manifold from the neural activity. Isometric Features Mapping technique (Isomap) is the most commonly used. At its core, it provides data scientists with a method to estimate the intrinsic geometry of a manifold based on each data point’s neighbors[ref 7].

Isomap relies on two principles:

Isometry: The geodesic distance on the manifold is invariant under the mapping to the tangent, Euclidean space.
Convexity: The manifold is a convex subset of the ambient Euclidean space

Isometric Feature Mapping (ISOMAP)

The core idea is to represent the data set (Neural Activity) as a graph, G with n data points, defined as vertices x and Euclidean distance DG as edges. For the points belonging to the neighborhood E, the weight of edges W is computed as [1]

For points that are non-neighbors the distance is computed along the geodesic of the manifold.

The geodesic distance is the shortest path distance between any sequence of vertex in G. I use the Dijkstra’s algorithm. For instance, a path through 3 nodes, i, j & k is simply computed through minimization [3]. The geodesic distance on a smooth manifold is theoretically computed by increasing the number of vertices to infinity (or weights of edges to zero) [4].

Finally, we apply the Multidimensional Scaling to construct the d-dimensional embedding

The Eigendecomposition of the Gram, symmetric matrix B generate n eigenvalues. The Eigenvectors associated with the d « n largest eigenvalues are the Isomap embeddings.

In a previous article, Decoding Neural Manifolds we generated a synthetic neural activity by sampling a S1 (Circle) manifold embedded in the neural data space with added Gaussian noise. We then applied a ISOMAP to extract the geometry of the manifold in 2 dimensions.

Fig. 5 Pipeline to generate a Neural Manifold from a synthetic Neural Activity sampled from S1 using Isomap manifold learning

I used Matplotlib to display the scatter plot between the 2 principal dimensions as illustrated below

Fig. 6 2-component Isomap for Neural Manifold from synthetic Neural Activity sampled from S1 manifold

📌 Contact and symplectic geometry on 3 and 4-dimensional manifolds have been used as a representation of the visual cortex [ref 8].

Perceptual Manifolds

A general neural manifold just shows how a circuit fires. A Perceptual Manifold is a specialized sub-manifold organized around meaningful, invariant features. It is where the brain utilizes geometric transformations to untangle raw sensory variations (like a face shifting under different light and angles) into a smooth, stable, and linearly separable representation of an object's identity.

Recognition

A Perceptual Manifold (or Object Manifold) is a specific type of neural manifold organized around meaningful features. Perceptual manifolds bridge raw data with recognition as follows [ref 9]:

Untangling: Most often raw sensory input (e.g., photon hitting the retina) is tangled. A human face looks completely different if the subject turns his/her head.
Invariance/Equivariance: The brain untangles the sensory data by mapping all variations of an object (lights, angles, distances, obfuscations, motion … ) onto a single, smooth manifold. For example, an object is always at a specific coordinate of the manifold, regardless of the angle.
Categorization: Perception happens when the brain can linearly separate these manifolds. Do the Car Manifold and Bicycle Manifold overlap?

👉 To summarize, the recognition (or categorization) of an object in the brain relies on geometric and topological properties such as invariance, equivariance or stable deformation (homotopy).

Global Structure

The next step is to determine the perceptual manifold’s global topological structure and properties [ref 9].

Persistent Homology: Use tools like Vietoris-Rips complexes and persistence diagrams to identify/extract persistent topological features. For example, if a stimulus (e.g., spike train) is periodic, the neural manifold should topologically resemble a 1-dimensional circle (S1 manifold).
Decoding Geometry: Persistent homology or Betti numbers can be applied to identify holes or voids in the neural manifold. For example, if the manifold is a torus, it suggests the brain is encoding two independent periodic variables.

Perceptual Metric

The identification of entanglement in neural data required the computation of a perceptual distance between two points over a geodesic on the neural manifold.

The two most common approaches that rely on Riemannian geometry are:

Fisher Information Metric for discriminating between concepts and noise. The distance between two stimulus points on the manifold is defined by how easily an ideal observer could discriminate between them based on the neural noise [ref 10].
Curvature Analysis for untangling concepts: The computation of the Ricci curvature tensor is applicable to identify perceptual bottlenecks and perceptual diffusion. A high positive curvature in a specific region of the neural manifold indicates over sensitivity and bottleneck. A negative curvature indicates that the region is not very sensitive to stimulus changes.

Equivariance

As described in the previous section Recognition, equivariance is key to understand that various actions (translation, rotation, scaling, stretching … ) on an object does not affect its identification.

Invariance and equivariance have been described in detail in previous articles [ref 11]. Here is a summary of the mathematics behind equivariance and symmetry (Lie) groups.

Symmetry & Equivariance

Let’s consider a neural encoder f that links stimulus or sensory input x to a neural state space y = f(x); f is equivariant, given a group G if for any transformation g, a group action on input ρ|x and a group action on outputs ρ|y

The relation is simplified in the case the group actions are identical

Finally, the simple scenario of no group actions.

Example:

Fig. 7 Illustration of equivariance from Sensory data to Neural Statesfor the rotation of an object

In the sensory data space, x is an image of a bunny and group element g is a rotation. The rotated image is defined by g.x. In the neural state space, the representation y=f(x) is transformed by ρ(g)(f(x))

X: Sensory data space (e.g., pixels of image of bunny)
Y: Neural representation space (e.g., edge, contour of the bunny)
G: Symmetry group (e.g., rotations, translations, scaling of the bunny).
Q = Y/G: Base or quotient space that contains the invariant identities like (e.g., Bunny)
O: Orbits or Fibers (e.g., continuous variations like position, hue).

In the context of neural manifolds, the symmetry group G is known the Qualia Modality, the base space Q is known as the Qualia Signatures and the Orbit O as the Qualia Attributes

👉 The concept of Orbit can be thought of as a pathway containing all possible changes in the attributes of an experience corresponding to a given object.

Example An orbit defines all the neural states associated with the bunny (rotation, scaling, translation, …). The sensory parameters of the experience may change along the orbit while the identity of the bunny does not as illustrated below.

Fig. 8 Illustration of Orbit in the identification of an object subject to equivariant transformation (rotation + scaling).

💡 As a side note, there is a compelling conceptual bridge between the Feynman Technique and the way World Models (particularly those using Latent Space) process information. Both are rooted in the principle of compression as a proxy for understanding.

The core of the Feynman technique is to take a complex topic and “explain it to a child, stripping away non-essential details. Similarly, the encoder of a World Model remove noise from sensory data and compress it into low-dimensional latent space. Both identify the underlying invariants (or gist).

The Feynman’s 2^nd step identifies the gap in your understanding of a topic while a World Model agent evaluates if the prediction of the next state from its latent representation is correct.

Finally, both attempt to build a mental model, through imagination or simulation.

Decoding Neural Manifolds

Sat, 23 May 2026 05:02:23 GMT

With differential geometry and topology already playing a central role in physics, biochemistry, machine learning …, it’s no surprise that smooth manifolds have become a key tool for understanding the brain and its activity.

Geometric Deep Learning, World Models use complex math to overcome limitations of traditional deep learning. We make that math practical through hands-on tutorials.

Why this Matters
Key Takeaways
Neural Manifolds - Basics
🏛️ Biology
🏛️ Dynamics of Population of Neurons
🏛️ Spike Density Function
🏛️ Neural Trajectory
Neural Manifolds - Deep Dive
⚙️ Python Environment
⚙️ Synthetic Neural Activity
🏛️ Neural Manifold Pipelines
🏛️ Latent Representation
⚙️ Isomap for Neural Manifold
🎬 Animation
References
Q & A
Paper Review

Why this Matters

Purpose: Structuring subjective experiences geometrically is a pre-requisite to link them to physical data and the patterns of firing neurons in the brain.

Audience: Data scientists and engineers curious about application of manifolds to represent brain activity.

Value: Develop a practical intuition for neural dynamics and latent manifolds by simulating spike trains with Poisson distributions and extracting topological structures using kernels and Isomap.

Key Takeaways

Signal Generation: Neural activity is an electrochemical process where neurons communicate via discrete events known as spikes.
Data Preprocessing: Raw spikes are recorded per neuron and normalized across specific time bins to ensure statistical consistency.
Manifold Extraction: Neural manifolds can be derived as discrete structures via eigendecomposition of the activity matrix, or as continuous representations using convolutional methods.
Signal Smoothing: Discrete spike trains are transformed into continuous temporal functions, f(t), by convolving them with Gaussian or exponential kernels.
Dimensionality Reduction: The high-dimensional Hilbert space containing these functions is projected onto a low-dimensional subset, defining the neural manifold.

Neural Manifolds - Basics

This section covers the essential Basics, while the Deep Dive section delivers advanced concepts, real-world applicability, dedicated Q&A and review.

⚠️ My knowledge in geometric learning does not make me an expert in neuroscience or biology, far from it. However, I thought it would be helpful to describe biological neuron prior to introducing neural manifolds - I hope expert readers will forgive me. Instructive references are listed at the end of this article.

🏛️ Biology

Biological neurons are the fundamental building blocks of the entire nervous system including the brain. They responsible for receiving, processing, and transmitting information throughout the body [ref 1].

This picture worths a thousand words …

Fig. 1 Illustration of a biological neuron

The basic functional components are:

Dendrites: Branch-like structures extending from the cell body that act as antennas, receiving signals from other neurons.
Soma: The metabolic center of the cell containing the nucleus and DNA, which sums up all incoming information from the dendrites.
Axon: A long tail-like structure that carries electrical impulses (action potentials) away from the soma to other neurons, muscles, or glands.
Axon Terminals: The end of the axon where chemical signals called neurotransmitters are released to pass the message to the next cell.

📌 Biological neurons are also called as nerve cells or neural cells

You may wonder how biological neurons compare with artificial neural units. Artificial neural networks are over-simplified versions of biological cell networks as illustrated in the following table.

Table 1 Basic comparison between biological and artificial neurons

Biological neurons can be categorized as Unipolar (single process extending from the body), Bipolar (one pair axon-dendrite), Multipolar (one axon - many dendrites) & Pseudo-unipolar (multiple process branches).

🏛️ Dynamics of Population of Neurons

Neural activity refers to the complex electrochemical processes by which neurons generate, and process information. For this article, I restrict the definition as the action potential or spike generated by a biological neuron.

Neural activity is obviously constrained by the properties of the physical network of neurons. These constraints reduce the neural activity patterns to low-dimensional subspace — manifold — within a high-dimensional ambient Euclidean neural space. While a population of N neurons could theoretically produce an infinite variety of firing patterns within an N-dimensional space, their actual activity is constrained to a much simpler, lower-dimensional surface [ref 2, 3, 4].

Researchers collect brain activities (neuron firing) through electrodes. The neural activity can be visualized as spikes as illustrated below:

Fig. 2 Illustration of neural activity for 4 biological neurons with a sliding time window

Since information is distributed across a population of neurons, it raises a fundamental question: what are the correlative or causal relationships between the spikes within that group?

One approach to answering this question is to analyze the collective firing patterns across multiple neurons within a specific (sliding) time window.

📌 Some articles may use the term neural discharge instead of neural spikes.

🏛️ Spike Density Function

Neural manifolds are fundamentally smooth manifolds and therefore we need to convert the train of spikes associated with each neuron into a smooth function. Mathematically speaking, those spikes are Dirac delta functions and can be smoothed as Spike Density Functions (SDF).

Convolution

Given a Kernel K and a spike train S, the SDF is computed as a convolution f

Using an exponential kernel

Using a Gaussian kernel

📌 There are many alternatives to the exponential kernel. The exponential kernel is used in computational neuroscience because it can be computed recursively without needing to store the entire history of spikes as follows:

The value of the smoothing Spike Density Function is proportional to the number of spike (neuron firing) within the time window as illustrated below:

Fig. 3 Illustration of Spike Density Function applied to the neural activity of 5 neurons

🏛️ Neural Trajectory

The next step is to represent the neural activity as a neural trajectory in the Euclidean space which dimension is equal to the number of neurons. Neural manifolds are abstraction of neural trajectories and neural dynamics as low representation of high dimensional neural activity [ref 5].

The following diagram illustrated a 2-dimensional manifold extracted from the neural activity of 3 biological neurons.

Fig. 4 Visualization of a 2-dimensional neural manifold representation of the neural activity of 3 biological neurons.

Think of a neural manifold as a mathematical map of what a group of neurons can actually do. While thousands of neurons could theoretically fire in endless combinations, their physical wiring and the task at hand force their activity onto a much simpler, low-dimensional surface. By studying these shapes of neural activity, we gain a clearer window into how the brain coordinates complex behavior.

Hands-on Stochastic Gradient Langevin Dynamics

Patrick R. Nicolas — Tue, 12 May 2026 16:24:01 GMT

Although a powerful and ubiquitous optimization method, the Stochastic Gradient Descent has fundamental structural limitations that make it unsuitable for some types of complex landscapes and Bayesian inference.

The Stochastic Gradient Langevin Dynamics (SGLD) fills the gap between optimization and random (Monte Carlo) sampling.

Geometric Deep Learning, World Models use complex math to overcome limitations of traditional deep learning. We make that math practical through hands-on tutorials.

Why this Matters
Key Takeaways
SGLD Basics
🏛️ Gradient Descent Family
🏛️ Stochastic Gradient Langevin Dynamics
- Langevin Dynamics
- Quadratic Loss Function
SGLD Deep Dive
🏛️ Multimodal Benchmark Functions
- Ackley’s Benchmark
- Rosenbrock’s Benchmark
⚙️ Python Environment
⚙️ Setup
⚙️ Validation
⚙️ Evaluation
References
Q & A
Appendix
Paper Review

Why this Matters

Purpose: Scientists have been trying to combine function optimization and random sampling to improve reliability of Bayesian inference and the minimization of the loss function in training of large models. Stochastic Gradient Langevin Dynamics (SGLD) mitigates the risk of converging to local minima by injecting Gaussian noise proportional to the square root of the learning rate, thereby facilitating global exploration of the loss landscape.

Audience: Data scientists and researchers looking to expand their understanding and evaluate alternative for Bayesian inference or training deep learning models.

Value: Learn the behavior of the Stochastic Gradient Langevin Dynamics as compared to Stochastic Gradient Descent and Adam algorithm using landscape loss function (e.g. Ackley).

Key Takeaways

By injecting “controlled jitter” (Gaussian noise) into the update step, SGLD successfully hops out of the shallow local minima that often trap standard gradient-based methods.
Using mathematical “playgrounds” like the Ackley or Rosenbrock functions allows us to isolate and visualize how an optimizer handles complex curvatures before testing on actual data.
PyTorch’s modular design makes SGLD easy to build; you just inherit the Optimizer class and plug your custom logic into the step() method.
Interestingly, SGLD acts as a bridge between worlds: it mirrors the broad exploration of SGD when the learning rate is high and shifts toward the precision of ADAM as the rate cools down.

SGLD Basics

This section covers the essential Basics, while the Deep Dive section delivers advanced concepts, real-world applicability, dedicated Q&A and review.

The best strategy to evaluate SGLD is to compare it with optimization methods we are all familiar with - Stochastic Gradient Descent and Adam.

🏛️ Gradient Descent Family

Gradient Descent (GD)

The Gradient Descent is the simplest form of optimization and a direct application of calculus. This is an iterative optimization algorithm used to minimize loss functions in machine learning.

Give a learning rate α, a loss function L, feature values vector x and label y

Stochastic Gradient Descent (SGD)

The vanilla Gradient Descent is notorious for slow convergence toward global minimum as it requires to process the entire datasets.

The stochastic version enables faster, highly efficient, and scalable learning for large datasets with two improvements has been made to the original Gradient Descent [ref 1]:

Selecting the random training pair features-label {xi, yi}
Organizing these random training samples into mini-batch of size m

SGD is noisy but converges faster than the original Gradient Descent.

Nesterov Accelerated Gradient (NAG)

Given the momentum v, the momentum coefficient β, the learning rate α and the mini-batch of size m

Nesterov acceleration changes the convergence rate of gradient descent on convex functions.

Gradient Descent: Converges at a rate of O(1/t)
Nesterov Momentum: Converges at a rate of O(1/t**2)

Adaptive Moment Estimation (ADAM)

Adam computationally efficient optimization algorithm for training deep learning models, addressing the short comings of SGD by combining the benefits of AdaGrad and RMSProp [ref 2]

It updates the learning rates for each parameter as described by the formulas below, using two momenta

First moment - similar to the mean - m (1)
Second moment or adjusted variance v (2)

The two momentum coefficients are normalized by their respective momentum scale factors (3).

Adam with Weight Decay (AdamW)

The update formula (4) can be improved by applying a linear decay function to the model parameters w with a factor λ

🏛️ Stochastic Gradient Langevin Dynamics (SGLD)

SGLD is derived from the Langevin mathematical formulation of the dynamics of molecular systems. From a Bayesian perspective, the Langevin method samples from the posterior distribution p(w |x).

Langevin dynamics

Langevin dynamics explores the landscape, escapes shallow valleys, and converges to a Gibbs distribution that places more weight on low-energy regions. In other words, it bridges optimization and inference: it can act like a noisy optimizer [ref 3, 4]

Fig. 1 Illustration of the trajectory of a Stochastic Gradient Langevin dynamics on a Ackley loss function/landscape

Stochastic gradient Langevin dynamics (SGLD) combines the two ideas, mixing minibatch gradients with structured (Gaussian) noise. SGLD uses mini-batching to create random gradient estimate similarly to SGD. Therefore, SGDL has two components:

Deterministic drift (the gradient of the loss)
Stochastic diffusion (the injected Gaussian noise)

This makes it scalable to large datasets while retaining the ability to balance exploitation with exploration, a concept borrowed from reinforcement learning.

The update formula is

📌 Beside been used as a stochastic optimization method, SGLD is a sampling method related to Markov Chain Monte Carlo and Bayesian learning [ref 5]

Quadratic Loss Function

The first test is to compare various optimizers with a convex function (single global minimum). The simplest form of convex function is the ubiquitous quadratic symmetric function. For a two parameters w1, w2 (vector w of n parameters) and a scale coefficient λ.

Fig. 2 Illustration of the symmetric quadratic loss function

Presentation: Geometric World Models Beyond Tokens

Patrick R. Nicolas — Tue, 28 Apr 2026 16:31:46 GMT

Presentation slides - We aren’t just forecasting pixels. World models map physical laws into latent space to enable genuine reasoning, while Geometric Deep Learning bakes the actual structure of the physical world directly into the neural network.

The presentation reviews the limitations of current models including the inability for Large Language Models to reason on interconnected data with complex geometric structure.

This talk explores the foundational roles of latent space encoding and self-supervised learning in the development of World Models. It further provides a visual and theoretical breakdown of continuous symmetries—specifically invariance and equivariance—and multiscale representations, demonstrating how these geometric priors are natively integrated into geometric deep neural architectures like Graph, Topological, and Equivariant Neural Networks.

The talk concludes with practical tips to develop foundational knowledge and leverage AI to apply it to your specific task.

🎯 Overview

Notes:

Outcome Prediction (The Old Way): This is like a student memorizing the answers to a physics test. The model learns a direct mapping from X to Y. It can tell you the ball will hit the ground in 2 seconds, but it still doesn’t know why.
Latent Representation (The World Model): This is like the student learning the laws of gravity. The “latent space” becomes a condensed, mathematical version of reality where concepts like mass, friction, and velocity exist as coordinates
Reasoning tasks are simulation, prediction, planning, optimization defined and executed in the latent space. They rely on the representation of real world object, state, action, with temporal component in low dimensional space.

Notes: Geometric priors includes facts extracted from physical world such as shapes, physics laws, chemistry, …

Notes:

The objective is to build a representation of the world as we collect more data or experience
The representation of the physical world evolves over-time so new observations should reflect the changing world (data distribution shift).
Convolutional Neural Networks are shift-equivariant (translation in Lie Algebra)
Example: PINNs loss regularization using differential equation with learnable parameter.

⚠️ The Problem

Notes:

These are just an overview of limitations of understanding our world through the lens of a sequence of tokens
Geometric Deep Learning (and more recently World Models) have gained traction in 2024 when the limitations of LLMs in term of reasoning become apparent

Notes:

Geometric or physical world constraint can be categorized as Hard (architecture) vs. Soft (loss regularization).

Graph representation entails node features + link features + graph features)
Manifold representation assumes data lie on a manifold but can be manipulated in the tangent space
Topological representation includes Simplicial complexes, Cell complexes, hypergraphs …
Spatial modeling: It combines shapes & Physics laws (Gravity, Protein folding,…)

Notes:

Large dataset requirements: LLMs require pre-training on large corpus as they are not identify symmetry such as rotations ..
Dependency on coordinates: Same object with different coordinates may not be identified along the geodeic
Do not respect physical laws: Physical laws have a cause-effect component that is difficult to model as a sequence of tokens.
Lacks inductive reasoning: LLMs do not have the ability to derive laws or general rules from specific observations in the physical worlds.

Notes: Patterns on observations can be difficult to identify on the Euclidean Space but easier to analyzed once projected over a manifold. Algebraic operations can be then applied on the tangent space (e.g., Lie algebra).

🌐 World Models

Notes: World models were introduced in 2017 and Geometric Deep Learning in 2021 but have become popular over the last 2 years are the limitation of LLMs in reasoning becomes apparent. The latent space encapsulates the level of abstraction that enables reasoning tasks.

Notes: A significant number of ‘World Models’ have been introduced for the last 3 years with an estimated 170+ related research papers or articles. This categorization - Generative, Latent and probabilistic is somewhat arbitrary.

Notes:

Generative reconstruction (with masking), used in LLMs is also applicable to prediction in latent space
Training an encoder (e.g., visual transformer) on objects, images subject to invariance or equivariance are expected to produce very similar embeddings

Notes: Symmetry in physical world can be encoded directly in the architecture (e.g., convolution & pooling, message passing & aggregation) or defined as a regularization term of the training loss.

📐Geometric Deep Learning

Notes:

Concepts in Algebraic Topology (global invariance) and Differential Geometry (local invariance) are key to represent symmetry
Symmetry can be encoded in the neural architecture (e.g. Graph Neural Networks, Equivariant Networks): Geometric Deep Learning
Symmetry (e.g. Lie) groups can be used in the latent space of World Models

Notes:

How can I practically incorporate the knowledge of the physical world - aka geometric priors - into the neural architecture?
Symmetry: Invariance, Equivariance, Affine transformation …
CNNs are the simplest example of invariant neural architecture (shift invariance)
Graph Neural Networks are perfect examples of these 2 principles - Symmetry of permutation and message aggregation and pooling

Notes:

There are several benefits in grouping images deterministically using continuous transforms.

Require less raw, labeled data as the original dataset can be augmented through continuous transformation
Geometric priors reduce the need of large fully-connected networks for classification or regression
Algebraic operations related to symmetry are deterministic (e.g. convolution)

Notes:

Actions on manifold such translation or rotation - is executed through infinitesimal change (Lie group)
These actions are defined algebraically (Lie algebra)
CNN (SE2) and Graph Neural Network Invariance of node indices are good examples.

Notes:

High frequencies => Overfitting
Low frequencies => Underfitting

Dispersion of density of node degree ó Gaussian or Ricci curvature (same Ricci curvature used in Field equations.) Micro and Meso scale are local message aggregation and Macro-scale is the permutation-invariant global pooling layer or Readout High-frequency components are the projections of the signal onto the eigenvectors of the Graph Laplacian associated with the largest eigenvalues – Outliers, Anomalies

Notes: Curvature of smooth manifold is used to define the scope/resolution/scale of data aggregation for continuous signals in robotics, computer vision ….

Notes: Differential operators such as Curl, Gradient, Divergence and Laplacian are used to generate a geometric (smooth or discrete) representation of a physics law.

🧮 Neural Architectures

Notes: The ubiquitous convolution (translation invariance in pixels) reduce the need for large MLPs (fully –connected).

Notes:

Many type of graph neural networks that can be categorized as Inductive vs. Transductive

Graph Convolutional Networks
Graph Attention Networks
Generalized Message-Passing Graph Networks

Notes: Applications like protein interaction, cybersecurity, traffic forecasting, and molecular engineering often demand global insights that go beyond traditional graph-based representations. To capture such complexity, these domains benefit from models built on richer topological structures such as simplicial complexes, cell complexes, and hypergraphs.

Notes: Curvature can be defined on meshes (e.g., triangular) models through topological complexes (Ollivier-Ricci curvature).

Notes: An equivariant neural network is a type of architecture designed to preserve the structure of symmetries, such as rotation, translation, or reflection, within the data. It ensures that if the input is transformed (e.g., rotated), the resulting output is transformed in the exact same way, allowing the model to understand the relationship between objects regardless of their orientation or position.

📚 Where to Start

Notes: More mathematics but less code and computation power required. The latent space is smaller compact but mathematically challenging.

Notes: Building a geometric learning/world model may require dealing with concepts spread among multiple disciplines.

Notes: What matters is to understand the concepts, foundation not the nitty-gritty => AI agent to collect the right information you can read

Here is technique borrowed from Richard Feynman, I have applied for thorny subjects:

Write the name of the topic or subject
Explain (or teach) the concept in simple language (as you will describe to something): Course, Podcast, Newsletter …
Identify the areas you are struggling
Pinpoint any complicated terms and simplify them

Notes: This list is a subset of tools available to model reasoning in the latent space or building geometric deep learning models.

🧠 Takeaways

📘 References

World Models

1 World Models: The Next Frontier in Our Path to AGI is Here - I. de Gregorio - Medium, 2023

2 AI and World Models. - R. Worden - Active Inference Institute, 2026

3 LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristic - R. Balestriero, Y. LeCun, 2025

Geometric Deep Learning

4 Limitations of Deep Neural Networks - S. Tsimenidis – 2020

5 Geometric and spectral limitations in generative adversarial networks - K. Mahyar - Rutgers University Libraries, 2021

6 Geometric DeepLearning Grids, Groups, Graphs, Geodesics, and Gauges – M Bronstein, J. Bruna, T. Cohen, P. Veličković, 2021

7 Geometric deep learning: going beyond Euclidean data - M. Bronstein, J. Bruna, Y. LeCun, A. Szlam, P. Vandergheynst, 2017

8 Introduction to Geometric Deep Learning Hands-on Geometric Learning, 2025

9 A Comprehensive Introduction to Graph Neural Networks - A. Awan - Datacamp, 2021

10 Position: Topological Deep Learning is the New Frontier for Relational Learning - T. Papamarkou et all, 2024

11 Geometric Deep Learning and Equivariant Neural Networks J. Gerken, J. Arronson, H. Linander, F. Ohlsson, C. Peterson, D. Persson, 2021

12 Pytorch Geometric - PyG Documentation, 2024

13 Github geomstats Documentation, 2023

14 Github topoxDocumentation, 2023

Patrick Nicolas is a software and data engineering veteran with 30 years of experience in architecture, machine learning, and a focus on geometric learning. He writes and consults on Geometric Deep Learning, drawing on prior roles in both hands-on development and technical leadership. He is the author of Scala for Machine Learning(Packt, ISBN 978-1-78712-238-3) and the newsletter Geometric Learning in Python on LinkedIn.

A Guided Tour of the Joint Embedding Predictive Architecture

Patrick R. Nicolas — Sat, 11 Apr 2026 18:17:50 GMT

Large Language Models struggle to understand the physical world because they only see data as a flat string of words. In contrast, the Joint-Embedding Prediction Architecture (JEPA) acts as a "World Model" by processing information in a deeper, abstract space, allowing it to plan and reason through complex, interconnected data more like a human would.

Geometric Deep Learning, World Models use complex math to overcome limitations of traditional deep learning. We make that math practical through hands-on tutorials.

Why this Matters

Purpose: The sequence of tokens representation in LLMs is inadequate to represent the physical world and its highly interconnected data. The goal of world models and JEPA in particular is to develop an abstract, latent-space encoding of physical reality that enables internal reasoning rather than merely optimizing for predictive accuracy on specific outcomes.

Audience: Engineers exploring world models as an alternative to large language models and deep learning networks operating in Euclidean space.

Value: This article explores how JEPA integrates into the broader framework of World Models. It details essential components such as self-supervised learning, latent planning, and the Model Predictive Control, while highlighting the fundamental role of symmetry.

Key Takeaways

Dual-Component Architecture: JEPA consists of a Contextual/Target Encoder pair that maps raw observations into a latent manifold, and a Predictor that operates entirely within that abstract space.
Rapid Architectural Evolution: The framework has undergone extensive iterative refinements since its inception, moving toward more efficient world models like LeWorldModel.
Self-Supervised Regularization: Advanced regularization techniques (such as VICReg or SIGReg) are fundamental to the training process, ensuring stable and informative latent representations.
Hierarchical Latent Planning: The model executes planning by decomposing high-level goals into a sequence of action-driven intermediate states within the latent space.
Efficiency and Robustness: Primary advantages include significantly lower computational and data overhead, increased resilience to sensory noise, and superior performance in spatial reasoning.
Ongoing Research Challenges: Critical frontiers include perfecting mitigation strategies for representation collapse and enhancing the reliability of both Spatio-temporal masking and long-horizon latent planning.

JEPA Basics

‘This section covers the essential Basics, while the Deep Dive section delivers advanced concepts, real-world applicability, dedicated Q&A and review.

World Models

Latent space

Although very powerful, Large Language Models (LLMs) face fundamental limitations that prevent them from being fully autonomous or universally reliable.

The sequence of tokens representation with a linear context window is inadequate to reason on complex, multi-dimensional or interconnected data such as 3D spatial molecular biology, drug discovery (Alpha fold), physics simulation, mesh dynamics and relational data. They have limited knowledge of the underlying reality (hallucinations).

Large Language Models (LLMs) face several fundamental limitations that prevent them from being fully autonomous or universally reliable:

Table 1 Summary of limitations of Large Language Models

The ultimate goal of world models is to develop an abstract, latent-space encoding of physical reality that enables internal reasoning, simulation … rather than merely optimizing for predictive accuracy on specific outcomes [ref 1, 2].

World model learning can capture meaningful representations by embedding complex, high-dimensional data into lower-dimensional abstract spaces that reside in the latent space.

The encoder generates embeddings from the observed physical world as illustrated below.

Fig. 1 Illustration of the relation between observed world and the qbstract World Model in latent space through encoding and decoding.

Note: (*) Not every world model contains a decoder that is used for generative reconstruction.

Generative World Models

Introduced by Ha & Schmidhuber [ref 3] and often cited as the catalyst for the modern interest in this field, this model decomposed the problem into three distinct components managed/orchestrated by an agent - The Vision Model, The Memory Model and The Controller.

Group-structured Latent Spaces

Geometric priors ensure that an agent’s internal representation of states and actions respects the underlying symmetry of its environment. Because a Markov Decision Process typically consists of a mix of symmetric and non-symmetric elements, embedding these as structural priors within the latent space allows the agent to generalize more efficiently from limited experience [ref 4].

Spatial Intelligence

Spatial Intelligence models developed by Dr Fei-Fei Li, focuses on understanding of 3D space and time [ref 5]. Unlike LLMs that process tokens, Spatial Intelligence models build persistent 3D worlds to interact with the physical world. These simulated worlds represent the geometric structure of the observable world in represented in the latent space].

Joint Embedding Predictive Architecture (JEPA)

Unlike generative models that rely on observation-level reconstruction, Joint Embedding Predictive Architectures (JEPAs), introduced by Yann LeCun, optimize for representation consistency across multiple data views in the latent manifold [ref 6]. This approach avoids the computational burden of density estimation, enabling the encoder to capture abstract, task-invariant features. By operating independently of raw input constraints, the architecture offers superior flexibility in feature encoding. JEPA embeds features that are useful for predicting the next state while discarding unpredictable features.

📌 JEPA’s core philosophy is to avoid data reconstruction, eschewing decoders and generative outputs entirely. In contrast, the Generative Latent Prediction (GLP) framework argues that reconstructing raw data is essential for validating latent-space accuracy. Currently, the AI community remains divided on which approach provides a more robust world model [ref 7].

JEPA

JEPA Models

The need for abstraction in representing the physical world: “Abstract representations simplify complex information into a form that is more manageable and meaningful for specific tasks or analyses. By focusing on the essential aspects and ignoring the less important details, these representations help systems (whether human or machine) to process information more efficiently and effectively. “ Y. LeCun

📌 The JEPA framework has evolved swiftly since late 2023 [ref 24]. The following discussion outlines the foundational concepts of the Joint-Embedding Predictive Architecture but does not provide a comprehensive technical audit of its various version-specific functionalities.

JEPA models have branched out significantly, with the main differences lying in the tasks they target and the tricks they use to stop the model from 'collapsing'—or taking the easy way out by ignoring the complexity of the data.

JEPAs are particularly relevant in processing geometric priors and manifolds because they treat the latent space as the primary arena for learning. They have evolved considerably since their introduction in 2022 [ref 6], differing mainly in their target tasks and in the strategies used to learn non-collapsing representation.

Several versions of JEPA have been released since the original conceptual JEPA proposing prediction in latent space (partial list):

V-JEPA (Video JEPA): Self supervised video latent models: Motion in representation space without generative reconstruction [ref 8]
I-JEPA (Image-based JEPA): Self supervised learning semantic image features: Vision Transformers and Invariance-based single encoder [ref 9]
LeJEPA (Latent Euclidean JEPA): Introduction of SIGreg regularization with Isotropic embedding improving both accuracy and scalability [ref 10]
Point-JEPA: Extension of JEPA for point clouds and more precise encoding of geometric priors [ref 11]
ThinkJEPA: VLM-guided JEPA modeling of latent world and large vision-language reasoning model that introduces long-term reasoning and planning [ref 12]
LeWorldModel: Introduction of loss regularization using Gaussian-distributed latent embeddings and improved execution performance [ref 13].
Temporal Straightening: Enhancement to training by incorporating curvature as a regularization of loss function [ref 14]
VLA-JEPA: JEPA for enhanced vision language action [ref 15]
Var-JEPA: Variational formulation of JEPA with generative self-supervised learning [ref 16]
Social-JEPA: Emergent geometric isomorphic [ref 17]
Le MuMo JEPA: Multi-Modal self-supervised learning with fusion tokens [ref 18]
Laya: LeJEPA variant for EEG [ref 19]
BiJEPA: Bi-directional JEPA with symmetry prediction [ref 20]
GeoWorld: Hyperbolic JEPA that preserves hierarchical relations [ref 21]
US-JEPA: JEPA for ultrasound [ref 22]
...

Fig. 2 Genealogy of JEPA models and their purpose as of March 2026 (graphics courtesy of TuringPost.com)

Enhancements to the original architecture are regularly introduced

Rendering vs. Reasoning

The physical world is composed of high-entropy, noisy, fine-grained signals (such as images and video), creating a significant challenge for models attempting to represent or capture underlying dynamics.

Masked-token training objectives typically underperform on tasks requiring high-level spatial understanding. Although reconstructing observed data is sufficient for rendering fine-grained textures and geometric lines, it does not facilitate a true semantic grasp of the world's dynamics—a limitation that necessitates moving toward predictive modeling in latent space. Therefore. these models underperform on tasks requiring spatial reasoning.

Benchmarking Topological Deep Learning

Patrick R. Nicolas — Sat, 28 Mar 2026 17:06:26 GMT

While Topological Data Analysis and Deep Learning rely on a wide range of mathematical structures and lifting/transforming algorithms, TopoBench simplifies the entire research cycle. It automates the design and evaluation process by providing a ready-to-use pipeline for configuring and training various graph and topological models.

Geometric Deep Learning, World Models use complex math to overcome limitations of traditional deep learning. We make that math practical through hands-on tutorials.

Why this Matters
Key Takeaways
TopoBench Basics
🏛️ Overview
🏛️ Topological Domains
🏛️ Topological Lifting
🏛️ TopoBench
TopoBench Deep Dive
⚙️ Python Environment
⚙️ Wrapper
⚙️ Training Implementation
⚙️ Evaluation
- Datasets
- Training & Testing
References
Q & A
Appendix
Paper Review

Why this Matters

Purpose: Topological Data Analysis (TDA) and Deep Learning encompass a diverse array of mathematical frameworks, higher-order structures, and sophisticated lifting and transformation algorithms. TopoBench streamlines this complexity by automating the design and assessment of topological architectures through an integrated pipeline for the configuration, training, and evaluation of Graph and Topological Neural Networks.

Audience: Data scientists and engineers using topological data analysis or developing topological neural models.

Value: Learn the basics of TopoBench architecture, configuration, training and evaluation with a simple dataset.

Key Takeaways

TopoBench is a versatile Python-based library engineered to accelerate the development and benchmarking of Topological Deep Learning (TDL) architectures.
The framework offers a modular suite of lifting and transformation tools designed to handle diverse graph structures and higher-order topological complexes.
Built-in PyTorch Lightning integration ensures a seamless workflow across the entire modeling, training, and evaluation pipeline.
By utilizing PyTorch Geometric and TopoX as its core engines, the platform minimizes boilerplate code for handling graphs, simplicial complexes, and hypergraphs.

TopoBench Basics

Overview

Topobench is a modular Python framework built to standardize benchmarks and streamline research within Topological Deep Learning (TDL). It enables the seamless training and comparative analysis of various Topological Neural Networks (TNNs) across multiple domains, including graphs, simplicial complexes, cellular complexes, and hypergraphs [ref 1].

Fig. 1 TopoBench Library Logo

Topological Domains

The most common topological domains - a supported in TopoX and TopoBench are

Simplicial Complexes
Cellular Complexes
Hypergraphs
Combinatorial Complexes

A simplicial complex is a graph with faces. It generalizes graphs that model higher-order relationships among data elements—not just pairwise (edges), but also triplets, quadruplets, and beyond (0-simplex: node, 1-simplex: edge, 2-simplex: Triangle, 3-simplex: Tetrahedron, ) [ref 2].

Cell complexes (or CW complexes) represent objects of flexible shape which are built out of basic ball-shaped building blocks (cells) of arbitrary dimension. Cells of different dimensions are rigidly related. For example, an area is enclosed by lines, which in turn are enclosed by points. This rigid structure describes the underlying topology. These complexes can be used as an alternative to Graph Neural Networks when data modeling requires high-order relationships [ref 3].

A hypergraph is a generalization of a graph where a hyperedge can connect any number of nodes or vertices. Similarly to Cell and Simplicial Complexes, a node and a hyperedge are said to be incident if the vertex is a member of the hyperedge. Hypergraphs are obviously ideal to encode relationships that are not strictly pairwise such as a chemical reaction of multiple molecules or a conference call involving many participants [ref 4].

Combinatorial complexes are structures used to represent topological spaces—like surfaces or multi-dimensional shapes—by breaking them down into discrete pieces like points, line segments, triangles, and their higher-dimensional counterparts. They are more flexible than Simplicial and Cell complexes.

Table 1 Comparative summary of Topological Domains

📌 This section emphasizes complexes, assuming they are less familiar to the general reader. Note, however, that TopoBench also natively supports traditional topological domains, including graphs and point clouds. Graphs (nodes and edges) and point clouds (nodes only) are simplest forms of complexes after all.

Topological Lifting

Lifting is the mathematical and architectural process of transforming a graph (composed of nodes and edges) into a higher-order topological structure supporting cells, faces, volumes or hyperedges.

The most common scenario is the topological lifting of a graph to a simplicial complex as described and illustrated in a previous article [ref 5]. The resulting components are

0-Simplices (Nodes): The original vertices.
1-Simplices (Edges): The original connections.
2-Simplices (Faces): Created by identifying cliques (like a triangle of three connected nodes) and filling them in as a solid surface.
3-Simplices (Tetrahedra): Created by identifying fully connected groups of four nodes.
…

There are many lifting techniques, the most common among them being

Clique Complex Lifting: This is the most common method. Every k-clique (a set of k nodes where every node is connected to every other node) in the graph is mapped to a (k-1) simplex. For example, every 3-node cycle becomes a triangular face.
Ring/Cycle Lifting: Often used in chemistry (for molecules like Benzene), this method identifies cycles of a certain length and lifts them to 2-cells. This is particularly useful for Cellular Neural Networks.
Curvature-Based Lifting: Using measures like Ollivier-Ricci curvature, you can lift regions of high connectivity or specific geometric properties into higher-order structures to better represent the manifold’s “flow.”

While graph-to-topological-complex transformations are the standard entry point for TDL, the lifting process is not limited to graphs; one topological complex can be further lifted into another, more sophisticated higher-order structure.

Fig. 2 Illustration of the hierarchy of lifting techniques from lower to higher order topological structures

📌 This article focuses on graph and topological complexes. However, a simple topological structure, point set or point cloud can be also lifting into a graph or any of the complexes.

TopoBench

TopoBench provides a unified benchmarking infrastructure for Topological Deep Learning (TDL) by integrating and expanding upon current software tools. It combines NetworkX for graph processing with the TopoX suite—specifically TopoNetX [ref 6] for building complex structures and TopoModelX for model implementation. Additionally, it supports out-of-the box PyTorch Geometric (PyG) [ref 7] models and original research code.

At its core, TopoBench is a unified and flexible workflow that supports a variety of datasets, data transformation, preprocessing methods along with deep learning models (e.g., Graph Neural Networks) and customizable metrics.

The key components are

Customizable loader: Extends the capability of PyG’s InMemoryDataset.
Data loader: Provides interface to data batch for graphs, hypergraphs, simplicial, cell and combinatorial complexes.
Pre-processor: Defines a pipeline of sequential transforms that processes dataset only once.
Transforms: Inherited PyG’s BaseTransform, categorized as Data manipulation, Topology lifting and full or feature lifting.
Models: Torch Lightning modules - Neural networks defined as backbone Topological Neural Network - can be imported from either PyG or TopoX.

A standout capability of TopoBench is its support for 'lifting,' which allows users to transform basic graph data into higher-order topological structures. This process enables the elevation of both raw features and the underlying connectivity, with the primary transformation routes detailed in the diagram below.

Fig. 3 Illustration of various topological models and lifting methods supported in TopoBench

Graphs Deserve Some Attention

Patrick R. Nicolas — Wed, 11 Mar 2026 16:45:18 GMT

Graph Attention Models offer a hybrid solution to the limitations of spectral and spatial graph processing addressing shortcoming of graph convolutional networks (fixed weight and transductive learning strategy) and graphSAGE (variable size of node neighborhood).

Geometric Deep Learning, World Models use complex math to overcome limitations of traditional deep learning. We make that math practical through hands-on tutorials.

Why this Matters
Key Takeaways
Graph Attention Network Basics
🏛️ Overview
🏛️ Graph Attention Model
PyTorch Geometric Library
🏛️ Torch Attention Modules
🏛️ Internal Attention Function
Graph Attention Model Deep Dive
⚙️ Class Design
⚙️ Python Environment
⚙️ Implementation Graph Attention Block
⚙️ Implementation Graph Attention Model
Evaluation
References
Q & A
Appendix
Message Passing Block
Graph Attention Model Builder
Training Configuration
Paper Review

Why this Matters

Purpose: Graph Attention Networks (GATs) overcome the traditional constraints found in both spectral-based models and spatial inductive methods for sequences. Unlike GCNs, which often rely on static or degree-based weights, GATs employ a dynamic attention mechanism to optimize node embeddings. Additionally, by accommodating variable neighborhood sizes, GATs offer superior data retention over GraphSAGE’s neighborhood sampling approach.

Audience: Data scientists and engineers developing complex, non-linear models using graph representation.

Value: Learn how to use PyTorch Geometric to implement a GAT while leverage reusable neural blocks.

Key Takeaways

Because GAT is inductive and can learn weights for varying neighborhood sizes, it generally trains more effectively and achieves better results than GCN or GraphSAGE.
The PyTorch Geometric (PyG) library is quite versatile, offering seven different models that handle everything from static/dynamic attention to semi-supervised tasks.
It is worth noting that many of these PyG modules come with LeakyReLU activations already baked into their internal logic.
To keep our implementation flexible, we adopted a builder pattern using reusable blocks to assemble our GAT architectures dynamically.

Graph Attention Network Basics

This section covers the essential Basics, while the Deep Dive section delivers advanced concepts, real-world applicability, dedicated Q&A and review.

🏛️ Overview

GATs were introduced in 2018 [ref 1] to address the shortcoming of both Convolutional Networks (CNNs) and Graph Convolutional Networks (GCNs)

My evaluation of GATs follows my articles on Graph Sampling [ref 2, 3] Graph Convolutional Networks (GCN) [ref 4] and Graph SAGE [ref 5].

Here is a comparative table for GCN, GraphSAGE and GAT

Table 1 Comparison of GAT, GCN and GraphSAGE models

GAT vs. GCN

GAT uses learnable attention weights assigned to different neighboring nodes based on their features while GCN assigns a fixed weight to neighbors. Moreover, GAT have an inductive learning strategy which does not require loading the entire graph as with GCN.

GAT vs. GraphSAGE

GraphSAGE samples a fixed number of neighbors that may cause loss of valuable information, while GAT process neighbors of any size each with a unique weight.

Graph Attention Models were introduced to address the limitation of both Spectral representation of graph (Laplacian) and Spatial induction (convolution on graph) for sequence-based tasks. The basic idea is to compute the hidden representation of each node from its neighbors using self-attention strategy. It is also directly applicable to inductive learning problems - generalization over unseen graph nodes (e.g., GraphSAGE) [ref 6].

Fig. 1 Illustration of the transformation of graph nodes features in kth layer of a graph attention network

🏛️ Graph Attention Model

Generic Message Passing and Aggregation

Graph Attention

Notes:

The symbol || represents the concatenation
The parameters a and W are learned

📌 Transformers can be constructed as GATs. Transformer can be interpreted as a fully connected graph where token embeddings act as nodes. Each node can attend to every other node through attention connections, enabling unrestricted information flow

Fig. 2 Illustration of Transformers as Graph Attention Models

The outer product - query (Q) key (K) vectors - generates a message between nodes, and the multiplication with the value (V) vector corresponds to message aggregation and node state update.

📌 GAT is a discretization of the graph diffusion. The diffusion is defined by the heat equation.

Given a latent variable z and the ith node with positional value u and feature x.

Applied to graph given two nodes i and j

Applying the Euler discretization formula, at step k with an interval tau

PyTorch Geometric Library

Library Overview

PyTorch Geometric has emerged as a leading library for exploring and implementing Graph Neural Networks (GNNs) using PyTorch.

The key Features of PyTorch Geometric are:

Efficient Graph Processing: Optimizes memory and computation using sparse graph representations.
Flexible GNN Layers: Covers GCN, GAT, GraphSAGE, GIN, and other advanced architectures.
Batching for Large Graphs: Supports for mini-batching for handling graphs with millions of edges.
Seamless PyTorch Integration: Provides full compatibility with PyTorch tensors, autograd, and neural network modules.
Diverse Graph Support: PyTorch Geometric handles directed, undirected, weighted, and heterogeneous graphs.

The most important PyG Modules are:

torch_geometric.data to manages graph structures, including nodes, edges, and features.
torch_geometric.nn to provide data scientists prebuilt GNN layers like convolutional and gated layers.
torch_geometric.transforms to pre-process input data (e.g., feature normalization, graph sampling).
torch_geometric.loader to handle large-scale graph datasets with specialized loaders.

Installation

Verify environment

python -c “import torch; print(torch.__version__)”

PIP Installation

pip install torch_geometric

Conda Installation

conda install pyg -c pyg

Extension libraries and dependencies

pip install pyg_lib torch_scatter torch_sparse torch_cluster torch_spline_conv -f https://data.pyg.org/whl/torch-${TORCH}-{VERSION}+${CUDA}-{VERSION}.html

Core

pip install torch_geometric

🏛️ Torch Attention Modules

GATConv: Basic linear layer applied after concatenating features. It uses static attention (where the ranking of importance between neighbors is the same for all query nodes
GATv2Conv: Contrary to GATConv , GATv2Conv implements dynamic attention, applicable if your model seems to be smoothing features too much
TransformerConv: It implements a multi-head dot-product attention mechanism instead of the additive attention used in GATConv. It only computes the transformed and aggregated node features, scheme particularly beneficial if you want to leverage the gated residual connections.
RGATConv: It learns separate attention weights for different relation types of heterogeneous graphs. This module is particularly suitable for complex knowledge graphs and datasets with multi-type edges.
AGNNConv: Suitable for semi-supervised learning as it focuses on attentional propagation without a learnable linear transformation in every layer.
FusedGATConv: An extension to the basic GATConv layer that fuses the message-passing operations into a single kernel to reduce memory footprint.
GPSConv: A hybrid, wrapper-style module local message-passing layer-global attention layer.

📌 As a general rule, graph attention layers do not need a pooling mechanism for node classification and link prediction as the attention mechanism already pools information from its neighbors. However, graph classification may benefit from pooling.

🏛️ Internal Attention Function

The following PyTorch attention modules, GATConv, GATv2Conv, RGATConv and FusedGATConv have a default, internal activation module, LeakyReLU with a negative slope 0.2. Therefore, there is no need for developer to specify an activation function.

Being a wrapper layer the GPSConv has a default ReLU activation function.

Finally TransformerConv and AGNNConv may need a user-specified activation function depending on the application.

Mathematics of Abstract World Models

Patrick R. Nicolas — Fri, 27 Feb 2026 19:58:28 GMT

While LLMs have made massive strides, they remain hindered by high training costs and a lack of grounded, experiential reasoning. This has sparked a surge in World Model research—specifically Abstract World Models—which leverage advanced geometric and mathematical frameworks to bridge the gap between statistical prediction and true autonomous reasoning.

Geometric Deep Learning, World Models use complex math to overcome limitations of traditional deep learning. We make that math practical through hands-on tutorials.

Why this Matters
Key Takeaways
World Model Math Basics
World Models
Latent Manifold Models
World Model Math Deep Dive
Mathematics of Abstract World Models
Graph Theory
- Overview
- Applicability
Algebraic Topology
- Overview
- Applicability
Differential Geometry
- Overview
- Applicability
References
Appendix
Q & A
Paper Review

Why this Matters

Purpose: Although Large Language Models (LLMs) have advanced significantly in their reasoning and multimodal capabilities, they still face several fundamental limitations that prevent them from being fully autonomous or universally reliable.

Interest in World models have been growing in recent years to address LLMs shortcomings with noisy, uneven and in some case scarce data, lack of interpretability, lack of experiential knowledge of the observed world and costly training. This article deals into Abstract World Models and their reliance of advanced geometric and mathematical concepts to deliver true reasoning capabilities.

Audience: Scientists and engineers investigating World Models as an option to use representation of the natural, observed world as their models.

Value: Learn an overview of World Model principles, highlighting the shift toward Abstract architectures. We examine established models to illustrate how Differential Geometry and Topology provide the necessary geometric structure for sophisticated latent-space reasoning

Key Takeaways

Research Trajectory: World models represent a rapidly evolving frontier, with novel methodologies and applications emerging weekly.
Architectural Taxonomy: These frameworks are broadly classified into three paradigms: generative, latent-predictive, and Bayesian.
Compression & Fidelity: By encoding raw data alongside physical laws, geometric constraints, and prior knowledge, these models create a compressed mirror of reality.
Geometric Foundations: Abstract world models are formally structured as manifolds within a high-dimensional latent space.
Mathematical Underpinnings: The execution of predictive and generative tasks on these manifolds necessitates advanced frameworks, including differential geometry, topology, and graph theory.

World Models Math Basics

World Models

World model learning can capture meaningful representations by embedding complex, high-dimensional data into lower-dimensional abstract spaces that reside in the latent space [ref 1, 2, 3].

📌 Contrary to the popular narrative that World Models are a brand-new breakthrough, the field has actually been evolving since 2016—and arguably even earlier.

There are many ways to categorize world models as illustrated in the diagram below.

Fig. 1 Example of categorization of world models

This review focuses on three primary architectural frameworks for world models:

Standard (or Generative) World Models: Systems designed to reconstruct or predict high-dimensional sensory input.
Latent Manifold World Models: Architectures like JEPA or Spatial Intelligence that prioritize representation learning within a latent space.
Probabilistic World Models: Frameworks that utilize uncertainty estimation and probabilistic inference to navigate the environment.

⚠️ This classification of world models, though arbitrary, is intended to highlight the underlying relationship between these architectures and the principles of Differential Geometry, Graph Theory and Topology. The following list of world models is far from being exhaustive!

Standard World Models

Introduction

The standard (generative) World Model is a neural network designed to understand and simulate the dynamics of the observed world, including its physical and spatial properties:

Reconstruction Goal: These models often use a Generative approach, where they attempt to reconstruct original data, such as future video frames or images, from a compressed “latent” representation.
Multimodal Input: They typically process video, sensor streams (like LiDAR), and sometimes language to create a unified internal simulation.
Simulated Training: By predicting future outcomes, an agent can “dream” or plan in a virtual sandbox, which reduces the need for dangerous or expensive observed-world trials.
Common Architectures: Standard world models frequently utilize Variational Autoencoders (VAE) for state compression and Recurrent Neural Networks (RNN), LSTM or Transformers to predict how those states change over time.

We described two examples of architectures of generative world models:

Traditional (or Original) World Models
OpenAI’s Sora

Generative World Models

Introduced by Ha & Schmidhuber [ref 4] and often cited as the catalyst for the modern interest in this field, this model decomposed the problem into three distinct components managed/orchestrated by an agent:

The Vision Model (V): A Variational Autoencoder (VAE) that compresses raw pixel input into a small latent vector z
The Memory Model (M): A Recurrent Neural Network (RNN) that predicts the next z based on previous states and actions.
The Controller (C): A simple linear model that decides which action to take based solely on the internal representations provided by V and M.

OpenAI’s Sora

Although primarily discussed as a video generation tool, OpenAI describes Sora as a “world simulator” [ref 5].

Spatiotemporal Patches: It treats video as a collection of 3D patches, allowing it to model the persistence of objects and physical interactions over time.
Physics Heuristics: While it doesn’t have an explicit physics engine, its scale allows it to “learn” certain physical properties, like gravity and fluid dynamics, through observation.

Probabilistic World Models

This model was proposed by Yoshua Bengio [ref 6]. Unlike the JEPA approach, which focuses on latent consistency, or Fei-Fei Li’s Spatial Intelligence, which focuses on 3D geometric realism, Bengio’s model focuses on epistemic uncertainty—the model’s ability to know what it does not know,

This is Bayesian probabilistic model that forms hypotheses about the world, and estimates the posterior probabilities of those hypotheses using prior probabilities and Bayesian inference from evidence. The world model could have strong prior probability of the known laws of physics, geometry and so on.

📌 It is reasonable to categorize Bengio’s Bayesian World Model as an abstract. I treat is a separate category because if unique focus on safety and causality.

Latent Manifold World Models

Introduction

Latent Manifold World Models - sometimes referred as Abstract World Models - are structured representations that purposely omit unnecessary details to focus on key patterns, symmetries, and causal structures.

No Reconstruction: Unlike generative models, abstract models like the Joint-Embedding Predictive Architecture (JEPA) do not reconstruct the complete image or video frame. Instead, they predict missing parts or future states directly within an abstract representation space.
Efficiency: By ignoring irrelevant details (like the exact shape of a cloud or the texture of a wall that doesn’t affect a robot’s path), these models operate at a higher level of abstraction and are much more computationally efficient.
Latent Reasoning: The “center of gravity” shifts from generating data to reasoning in latent space, allowing the model to focus on the semantic meaning of a scene rather than its pixel-perfect appearance.
Group Symmetries: These models often incorporate formal mathematical foundations, such as Equivariant Transition Structures, to ensure that learned representations respect physical symmetries like rotation or translation.

Let’s introduce 3 well-known Abstract World Models

Joint Embedding Prediction Architecture
Markov Decision Process Abstraction
Spatial Intelligence

Joint Embedding Prediction Architecture (JEPA)

JEPAs are particularly relevant in processing geometric priors and manifolds because they treat the latent space as the primary arena for learning.

📌. The earliest and most common JEPAs models are purely self-supervised and do not use Reinforcement learning (RL) during pre-training. However, some of the latest variants explicitly integrate RL.

Group-structured Latent Spaces

The two main characteristics are

Geometric Priors: These models encode SE(3) (3D Euclidean motions) or SO(3) (rotations) directly into the latent space.
Equivariance: If an object moves in the real world, its representation in the latent space moves along the manifold according to the laws of symmetry. This allows the model to generalize “object permanence” and “physics” across different viewpoints without needing billions of examples.

Einsteinian World Model

This model explicitly learns globally consistent solutions using the Space-Time manifold.

Metric Learning: Instead of learning abstract features, the network predicts the local metric tensor—the mathematical object that defines distances and curvature—across overlapping coordinate patches.
Curvature as Loss: The model adds the Einstein field equations to the training loss to ensure the world model is physically valid. The loss (energy) increases if it predicts a move that violates the curvature rules of its space-time manifold.
Global Consistency: To prevent the model from becoming a “bag of heuristics,” it enforces consistency between patches, ensuring the latent space is a smooth, continuous manifold.

Spatial Intelligence

Spatial Intelligence models developed by Dr Fei-Fei Li, focuses on understanding of 3D space and time [ref 8]. Unlike LLMs that process tokens, Spatial Intelligence models build persistent 3D worlds to interact with the physical world. These simulated worlds represent the geometric structure of the observable world in represented in the latent space. Therefore, these spatial models are inherently multimodal.

Conceptually, Spatial Intelligence is an abstract world model because it aims to provide the "scaffolding" for cognition. Like the JEPA family, it seeks to move beyond pixel-level correlations toward a deep understanding of physical laws, causality, and geometry.

👉 For the reminder of this article we focus on the Abstract World Models and more specifically JEPA and Group-structured Latent Space Model to illustrate the role of advanced mathematics.

Latent Manifold Models

Overview

There is currently no universal, strictly standardized definition of an “abstract world model” in the AI research community. While the term is widely used, its boundaries shift depending on whether the researcher focuses on generative reconstruction or latent prediction.

However, the community generally agrees on several core functional pillars that characterize these architectures:

Core Functional Pillars

Latent State Representation: They must compress high-dimensional sensory data into a lower-dimensional “latent” manifold.
Temporal Dynamics (Memory): They must include a component that predicts how the latent state evolves over time, often based on internal physics or learned transitions.
Predictive Ability: A true world model must be able to “hallucinate” or simulate future outcomes without receiving new external data.

To understand the shift toward Abstract World Models, we must contrast them with traditional deep learning and standard Transformer architectures. As the following table illustrates, the fundamental divergence lies in their geometric assumptions—moving from flat Euclidean spaces to smooth or discrete manifolds—and the underlying mathematical frameworks that govern their representations.

Table 1. Deep Learning & Transformer models vs. Abstract World Models

Latent Representation

Let’s consider a bouncing ball. Each video frame consists of millions of pixel values changing every millisecond. However, the underlying attributes are its position (Geometry), and its velocity, gravity (Physics Law). The latent space is the mathematical space that maps the pixels into these underlying attributes.

One established fact is that entities and transformations/operations on the latent space is critical to abstract world models. The abstract world model replicate & compress the observed world (data, physics laws, geometric constraint, prior knowledge ….) into the latent space along with the target.

Therefore, the state of the ‘world’ is predicted as a latent variable and loss/objective are computed in a low dimensional manifold.

📌 A significant number of world models enforce geometric priors in the latent space through regularization term in the loss function in training

Invariance constraint

Equivariance constraint

The decoder reconstructs or renders the updated latent state back to the observed world.

How do we define the latent space in this context? It is best understood through three distinct lenses:

Geometry: Representations are constructed as smooth or discrete manifolds, meshes, and graphs, or within topological domains such as hypergraphs and simplicial complexes.
Variables: The space serves as a container for latent variables, including states, rewards, objectives, and operational constraints.
Activities: This low-dimensional manifold provides the playground for computation, where prediction, planning, simulation, and optimization are executed with high efficiency.

⚠️. When learning state representations, the most significant obstacle is the collapse of the latent space. This occurs when the encoder maps distinct data points x to an identical or nearly identical embedding z, causing the model to lose the ability to differentiate between unique inputs.

Furthermore, even if the embeddings are distinct, the model may fail to properly preserve the geometric distances between states during the prediction process, leading to inaccurate simulations of reality.

Group-structured Latent Space Models

World Models are advanced AI systems learning the rules of reality (physics, cause-and-effect) from data like videos to simulate and predict dynamic 3D environments, enabling more robust reasoning, planning, and creation beyond static content [ref 9].

World Models go beyond Large Language Models (LLMs) by understanding spatial relationships and physical interactions, allowing AI to generate immersive, interactive worlds for robotics, design, medicine, and complex problem-solving, representing a significant leap towards human-level intelligence.

Let’s consider the Markov Decision Process (MDP) fully defined by a state space S, an action space A, a reward function R and a state transition T

A self-supervised world model is defined in a latent space Z with the following learnable components

Learnable encoding that projects states into a low-dimensional manifold

Learnable transition function

Learnable reward function

JEPA

JEPA’s approach avoids the computational burden of density estimation, enabling the encoder to capture abstract, task-invariant features [ref 10, 11].

📌 The original JEPA framework has evolved into a diverse ecosystem of specialized architectures. Notable variants include V-JEPA for video, LeJEPA, and Hierarchical JEPA, alongside targeted iterations such as Multi-Resolution, Cell-JEPA, and Lp-JEPA.

The following mathematical formalism reflects the original definition of JEPA

In JEPA and its video-based extension (V-JEPA), the core mathematical framework is Energy-Based. The energy is minimized in the latent space.

The energy measures the L2 norm of the predicted latent representation and the target latent representation

x: Input and contextual data tensor
y: Target (labeled) data
z: Latent vector
s_xLatent representation of the context/data from the input encoder
s_yLatent representation of the target from the target encoder
Pred: Prediction neural model for the latent state using the latent feature tensor and the latent variable z

Let’s look at the predictor model. It simply generates an estimate of the target latent state from the contextual features s_xand the latent variable z

JEPA uses variance-based regularization as architectural constraint to prevent information to collapse [ref 11, 12]

📌 Traditionally, self-supervised deep learning models would use contrastive learning.

The loss known as the Variance-Invariance-Covariance Regularization component is expressed as:

lambda, mu, nu: Hyperparameters of the model
Inv: Invariance as the mean-squared error between two batch of embeddings
v: Variance as the hinge loss pushing the standard deviation of each embedding dimension to be above a certain threshold
C Covariance that penalizes the off-diagonal elements of the covariance matrix of the embeddings
Z, Z’: Batch of embeddings
n: Size of batch
d: Dimension of the latent variable

The total loss is then computed as

The weights of the target encoder are computed using a simple exponential moving average on the context weights theta

The following diagram illustrates the 3 top components of an abstract world model: Encoder, Latent space and Decoder with associated energy minimization, and actuation policy.

Fig. 2 Illustration of the relation between Observed World and the Abstract World Model in Latent Space through encoding and decoding.

📌 Beyond JEPA and Markovian frameworks, the landscape of abstract world models includes several specialized paradigms, such as Equivariant World Models [ref 12] , Symplectic World Models, [ref 13] and Spacetime World Manifolds [ref 14] . A technical overview of these alternatives is provided in the Appendix.

Visualization Tools for Geometric Deep Learning

Patrick R. Nicolas — Fri, 13 Feb 2026 00:15:12 GMT

Have you ever found it challenging to translate complex AI architectures or machine learning outcomes into a clear, compelling narrative?

Fortunately, Python offers a powerful suite of libraries specifically designed to bring these models and concepts to life through visualization and animation.

Geometric Deep Learning, World Models use complex math to overcome limitations of traditional deep learning. We make that math practical through hands-on tutorials.

Why this Matters
Key Takeaways
Animation Frameworks - Basics
🏛️ Matplotlib/Animation
- Functionality
- Installation
🏛️ Manim
- Functionality
- Installation
Animation Frameworks - Deep Dive
Which Animation Library?
⚙️ Python Environment
⚙️ Lie SE3 Euclidean Group (Matplotlib)
⚙️ Graph Convolutional Neural Network (Manim)
⚙️ 2D Normal Distribution (Matplotlib)
⚙️ Lie groups (Matplotlib)
References
Q & A
Paper Review

Why this Matters

Purpose: Abstract mathematical frameworks, including algebraic topology and differential geometry, are often best elucidated through dynamic visualizations and animations. These tools are invaluable for both conceptual demonstrations and the interpretation of experimental results. To meet this need, the Python ecosystem offers data scientists a robust selection of specialized animation libraries.

Audience: Scientists and engineers who needs to describe their model through animation or create demos to illustrate a new concept

Value: Explore essential Python libraries as powerful communication vehicles for visualizing and animating complex models and experimental results.

Key Takeaways

Matplotlib.animation: An accessible extension of the standard Matplotlib library, offering a gentle learning curve for those looking to add dynamic updates to existing visualizations.
Manim: A high-fidelity engine supporting complex scenes, camera controls, and LaTeX integration; however, its extensive feature set requires a significantly higher time investment to master.
Use Cases: Matplotlib excels in real-time monitoring and data analysis, whereas Manim is optimized for polished educational content, professional demonstrations, and high-quality storytelling.
Performance: Due to its rendering overhead, Manim is less efficient for large-scale sequences. It is best used with a “low-quality first” iterative workflow, scaling up to high-resolution output only for the final release.

Animation Frameworks - Basic

This section covers the essential Basics, while the Deep Dive section delivers advanced concepts, real-world applicability, dedicated Q&A and review.

📌 Although various specialized 3D rendering frameworks are available [ref 1], they are often either underutilized in research or incompatible with the requirements of geometric learning.

Matplotlib/Animation

While Matplotlib remains the industry standard for static and interactive data visualization, its animation features are frequently underutilized and less widely understood.

📌 This article focuses specifically on animation; therefore, I will not delve into Matplotlib’s standard plotting functionality.

Functionality

While often seen as “basic,” Matplotlib remains the most versatile for scientific plotting and is often the best “first step” for simple animations. It is best suited for Real-time signal processing, simple 2D dynamic systems, physics and math simulation [ref 2].

Matplotlib enables animation generation via its animation module, treating an animation as a series of individual frames plotted on a single, condensed figure.

The animation process relies on two Python classes [ref 3].

FuncAnimation that generates animation by iteratively updating the data of a single plot. It is generally the more efficient choice for memory and speed.
ArtistAnimation that creates animation from a pre-defined list of “Artist” objects (drawings) for each frame. This approach is better suited for complex or highly creative scenes.

Installation

pip install matplotlib

Basic design is illustrated by the following pseudo-code

draw:
   initialize animation configuration parameters
   set up static elements
   -> update(frame)   # Nested function to update the dynamic component

   invoke FuncAnimation(fig, update, frames=100, interval=300, ...)
   save animation to MP4 file

Manim

Manim is the gold standard for high-quality, explanatory math videos. It is particularly suitable for Calculus, linear algebra, and complex equations where precise “morphing” between formulas and shapes is needed. Manim is the most commonly used animation on YouTube [ref 4].

The key features are Native LaTeX support, built-in coordinate systems, and beautiful object to object transformations.

Functionality

A Manim animation implements a dynamic scene, defined in the class Scene (or its subclasses). Objects to be displayed and/or animated in a scene have their type inheriting from the class Mobject [ref 5, 6]

Here a list of the most common objects, scenes and animation functions.

Objects

Mobject: base class for objects that can be displayed on screen.
VMobject: A vectorized Mobject
VGroup A group of vectorized objects of type VMobject
NxGraph: Generic mathematical graph
Graph: Undirected graph
DiGraph: Directed graph
ValueTracker: An object that track real-value parameters
Arc: A circular arc defined by its start and end angle (radiants)
Circle: A circular arc (0, 2.pi)
Line: A curved or straight line between 2 mobjects
Arrow: Graph configuration arrow
Text: Ascii or Latex content
Angle: A circular arc mobject defined by its angle
…

Scenes

Scene: Set of tools to manage mobject and animations
ThreeDScene: 3D scene with camera angle
SpecialThreeDScene: 3D scene with support for spheres and camera shades

The key functions for scenes are

construct: Add content mobjects or vmobjects to the scene
add: Adds a mobject
clear: Removes all objects
remove: Removes a given mobject
bring_to_front/bring_to_back: Brings mobjects in front or back in the scene
pause : Pauses the scene
play: Plays an animation in the scene
wait: Plays animation without change

Animation

Create: Shows a mobject
Write: Simulate hand-drawing or handwriting
Unwrite: Simulate the erasure of handwriting or hand-drawing mobject
FadeIn: Alternative to create by fading in an mobject into the scene
FadeOut: Fades out an mobject into the scene
Rotate: Animation that rotates an mobject in a scene
Transform: Transform one mobject to another one
…

📌 ManimGL is a separate version of Manim (by 3Blue1Brown) that uses OpenGL for hardware-accelerated, real-time rendering. If you find the standard ManimCE too slow for your complex manifolds, ManimGL might be the solution.

Installation

Installation Linux

sudo apt update   
sudo apt install build-essential python3-dev libcairo2-dev 
                 libpango1.0-dev ffmpeg   # Manim dependency
pip install manim

Installation MacOS

brew install manim
brew install py3cairo ffmpeg pango pkg-config scipy     # Dependencies

Installation verification

manim checkhealth

Initializing a Manim project in folder ‘my_folder’

manim init project my-folder --default

Execution of a scene - class MyScene

manim -pql main.py MyScene
ql: low quality video
qm: medium quality video
qh: high quality video

📌 Although the Seaborn library is a compelling extension to Matplotlib plotting capabilities, it does not support animation.

2025 Retrospective Research Papers on Geometric Deep Learning

Patrick R. Nicolas — Wed, 28 Jan 2026 21:17:44 GMT

Curvature-informed Graph Learning

Patrick R. Nicolas — Thu, 15 Jan 2026 17:42:15 GMT

Is your Graph Neural Network struggling with over-squashing or over-smoothing during training? These message-passing bottlenecks are often geometric in nature—and the Ollivier-Ricci curvature algorithm offers the solution!

Geometric Deep Learning, World Models use complex math to overcome limitations of traditional deep learning. We make that math practical through hands-on tutorials.

Why this Matters
Key Takeaways
Ollivier-Ricci Curvature Basics
🏛️ Graph Neural Network Shortcoming
🏛️ Why Discrete Curvature
🏛️ Discrete Differential Geometry
🏛️ Ollivier-Ricci Curvature (ORC)
Ollivier-Ricci Curvature Deep Dive
🏛️ Impact of Curvature on Graph Neural Network
⚙️ Python Environment
⚙️ Ollivier-Ricci Algorithm
⚙️ Sinkhorn-Knopp Approximator
⚙️ Evaluation
- Stochastic Layout
- Manifold Layout
⚙️ Appendix
- Floyd-Warshall Algorithm
- Spherical Geodesic Distance
References
Q & A
Paper Review

Why this Matters

Purpose: Over-squashing and over-smoothing represent the primary obstacles in training Graph Neural Networks (GNNs). While over-squashing generates critical information bottlenecks, over-smoothing leads to the excessive generalization of node features. By utilizing Ollivier-Ricci curvature, researchers can identify and remediate graph regions that obstruct effective information flow.

Audience: Scientists and engineers developing and deploying graph neural networks.

Value: Learn the foundational understanding of discrete curvature and the practical implementation of the Ollivier-Ricci metric via iterative Wasserstein distance approximations.

Key Takeaways

Discrete differential geometry provides a robust mathematical framework for resolving fundamental GNNbottlenecks, including over-squashing, over-smoothing, and the inability to capture higher-order topological structures.
Edge and node curvature are essential metrics for governing the structural layout and connectivity constraints of graphs (such as cliques) and triangle meshes.
The Ollivier-Ricci metric adapts Ricci curvature from smooth manifolds to discrete settings by utilizing the Wasserstein distance paired with a shortest-path cost matrix.
Efficient computation of this curvature is achieved by approximating optimal transport through the Sinkhorn-Knopp algorithm.

Ollivier-Ricci Curvature Basics

'This section covers the essential Basics, while the Deep Dive section delivers advanced concepts, real-world applicability, dedicated Q&A and review.

🏛️ Graph Neural Network Shortcoming

Discrete Differential Geometry has recently been used to address fundamental limitations in Graph Neural Network (GNN) on how information flows through graphs:

Information Bottlenecks: Message passing in GNNs can lead to over-squashing, where high negative curvature identifies specific edges that restrict efficient data flow [ref 1].
Generalization Errors: GNNs often overfit to noise or fail to generalize when undetected curvatures in the decision boundaries create unnecessary complexity.
Dynamic Modeling: Representing the evolution of a graph over time requires sparse modeling to map high-dimensional temporal changes onto a lower-dimensional manifold.

Geometric Constraints: Models utilizing orthogonal weights to prevent gradient vanishing often require mathematically intensive projections to remain valid within standard Euclidean spaces.
Convergence Hurdles: Extreme curvature variance across a mesh or graph can destabilize the learning process and prevent the model from reaching a global minimum.
Adaptive Optimization: Severe training instability may necessitate Natural Gradient Descent via the Fisher-Rao metric to better navigate the parameter manifold than standard SGD.

🏛️ Why Discrete Curvature

Discrete Differential Geometry (DDG) is a field that translates the principles of smooth differential geometry—the study of curved shapes using calculus—into a language that computers can process, primarily using meshes and polygons.

Discrete differential geometry offers a rigorous mathematical formalism to solve several well-documented limitations in Graph Neural Networks (GNNs): Over-squashing, Over-smoothing and lack of higher order topological structures.

For specifically elements of discrete manifold such as curvature and flow improve training and architecture design of graph models.

Reducing Over-squashing: Information from vast neighborhoods is compressed into fixed-size vectors, creating bottlenecks. Using Ollivier-Ricci or Forman-Ricci Curvatures helps identify these bottlenecks—typically found in areas of highly negative curvature—allowing for strategic graph rewiring along geodesics.[ref 1].
Mitigating Over-smoothing: Node features become indistinguishable after multiple layers of message passing. Implementing anisotropic diffusion or the discrete Laplace-Beltrami operator allows the feature space manifold to evolve dynamically across epochs, preserving essential signal variance [ref 2].
Capturing Higher-Order Structures: Standard GNNs are often limited to simple node-to-edge interactions. Utilizing simplicial or cell complexes provides a richer representation of data; these complex interactions are efficiently computed using discrete exterior calculus.
Optimizing Geometric Convergence: Determining the ideal network depth can be difficult and computationally expensive. By monitoring the Discrete Ricci Flow of the feature manifold, researchers can more accurately identify convergence points and reach optimal network depth.

🏛️ Discrete Differential Geometry

Discrete Differential Geometry (DDG) is a field that translates the principles of smooth differential geometry—the study of curved shapes using calculus—into a language that computers can process, primarily using meshes and polygons.The core elements of this field are categorized by how they discretize the geometry of shapes and the operators that act upon them [ref 3, 4].

Rather than dealing with infinite, smooth surfaces, DDG uses combinatorial structures to represent geometry such as Simplicial Complexes, Cell Complex, Hypergraphs or meshes [ref 5, 6].

One of the most important elements is defining how a jagged mesh “curves by measuring the curvature.

Fig. 1 Illustration of the value of Gaussian or Mean curvature for spherical (clique), flat (grid) and hyperbolic (tree) graphs.

Since a flat triangle has no curvature, DDG assigns curvature to the vertices and edges:

Gaussian Curvature (Angle Defect): This curvature is intrinsic to the Riemannian manifold and does not require any ambient Euclidean space. For a vertex, this is calculated as 2\pi minus the sum of the interior angles of the triangles meeting at that vertex.
Mean Curvature: This extrinsic computation of the curvature is often defined through the Cotan formula, which relates the movement of a vertex to the change in the total surface area.

📌 The Gaussian (intrinsic) and Mean (extrinsic ) Curvature are related to the Discrete Laplace-Beltrami operator and will be the topic of a future article.

🏛️ Ollivier-Ricci Curvature (ORC)

Drawing from Riemannian geometry, the concept of curvature measures the convergence (>0) or divergence (< 0) of geodesic paths. Applied to graphs, Ollivier-Ricci Curvature (ORC) serves as a discrete approximation that uncovers local topology and geometric properties through the lens of optimal transport. By utilizing the Wasserstein distance, ORC identifies network flow patterns: negatively curved edges represent high-traffic ‘bottlenecks’ critical for efficient data transfer, whereas positively curved edges indicate highly redundant, well-connected regions where ‘commodities’ flow through multiple alternative paths [ref 7].

Let G be a graph with a shortest-path metric (a.k.a. cost matrix) d, mu the probability measure on graph for a given node v. The Ollivier-Ricci curvature of a pair of node (i, j) is computed from the Wasserstein distance W

The challenge is to compute the two values: Wasserstein distance between probability distributions over two nodes and the shortest path metric

Let’s start with the computation of the shortest paths for all pairs of nodes in a graph, introducing the Floyd-Warshall algorithm

Shortest Paths

The Floyd-Warshall (FW) algorithm is a commonly applied method to compute all-pairs shortest paths in a directed graph. The algorithm assumes that the weights are all positive.

Let’s consider a subset S={1, 2, … k} subset of k vertices (or nodes) from a graph V = {1, 2, …, k, .., n}. For any pair of nodes i, j in V, FW considers all the paths whose intermediate vertices are all down from S. Let p be a minimum weight path from among them. FW exploits the relationship between the path p and shortest path i → j with all intermediate nodes in the set {1, 2, …, k-1}

The algorithm is implemented recursively (step k):

Wasserstein Distance

The Wasserstein Distance measures the cumulative “cost” to transform one distribution into another, focusing on distances between probability masses. it is also known as the Earth Mover’s Distance.

Fig. 2 Illustration of the Earth Mover’s 2D Wasserstein distance

It is independent of overlap and can handle significant distribution shifts in shape or location. By measuring the overall “cost” of moving the distribution, is less affected by minor tail variations or outliers [ref 8].

📌 Unlike the KL Divergence, the Wasserstein distance is a true metric (it is symmetric and satisfies the triangle inequality).

For two distributions r and c, the p-Wasserstein Wp distance is defined as

📌 While Wasserstein distance is traditionally expressed using the measures mu and nu, I have adopted the r and c notation to maintain consistency with the standard conventions used in graph-based optimal transport.

The distance between graph node relies on the 1-Wasserstein distance, W1 which has a simple, closed form - area between two cumulative functions F for r and c distributions.

📌 The Wasserstein distance is not the only metric to compare probability distributions.

The 𝗠𝗮𝗵𝗮𝗹𝗮𝗻𝗼𝗯𝗶𝘀 distance computes the number of standard deviations between two distributions along the principal covariance axes. It requires that both distributions shared the same covariance matrix. It is commonly used in outlier, anomaly detection, clustering and Kalman filtering.
The 𝗕𝗵𝗮𝘁𝘁𝗮𝗰𝗵𝗮𝗿𝘆𝘆𝗮 measures the affinity between probability densities breaking it down into a Mahalanobis term and a covariance determinant term. It is restricted to distributions that have no disjoint supports.

The computation of 1-Wasserstein is daunting, so a numerical approximation is in order.

Wasserstein Distance Approximation

The computation of 1-Wasserstein is daunting, so I approximate using the iterative Sinkhorn-Knoppe algorithm [ref 9].

Sinkhorn distance

For a given transport plan Pij, representing how much ”mass” is moved from one point i in distribution r to point j in distribution c, and cost matrix (or shortest paths) between i & j, Mij - the Sinkhorn distance is defined as

The optimal transport plan, P* is computed through a traditional minimization of the transport plans Pij under constraints and the cost Mij

The smoothing factor epsilon is the scale factor for the entropy logP and converge to the 1-Wasserstein distance as epsilon → 0.

Iterative Sinkhorn-Knoppe algorithm

The Sinkhorn-Knopp algorithm provides an efficient way to compute the Sinkhorn distance by solving an entropic-regularized optimal transport problem.

Given an optimal transport plan P, the cost matrix M, the entropy regulation factor epsilon, a source distribution r and a destination distribution c

Note: u, v are normalization variables.

The set of formulas [6] can be easily converted into the following pseudo-code

Compute K
Init v <- [1]
DO
   u <- r/K.v
   v <- c/KT.u
WHILE u < threshold and v < threshold
P = diag(u).K.diag(v)

Estimating an appropriate entropy regularization parameter, epsilon, to approximate the Earth Mover’s Distance remains a significant challenge. For the purposes of this study, the following condition was selected empirically.

Q & A: Answers -2 🛠️

Patrick R. Nicolas — Thu, 15 Jan 2026 03:18:10 GMT

Turbocharging Neural Networks with Taichi Language

Patrick R. Nicolas — Fri, 02 Jan 2026 19:18:42 GMT

You don't have to rewrite your entire codebase to get better performance from existing numerical computation routines. Taichi is a domain-specific language (DSL) that accelerates NumPy and Torch applications while keeping their original structure intact, critical for development of machine learning models.

Geometric Deep Learning, World Models use complex math to overcome limitations of traditional deep learning. We make that math practical through hands-on tutorials.

Why this Matters
Key Takeaways
Taichi Language Basics
🏛️ Overview
🏛️ Taichi Data Types
🏛️ Taichi Processing Annotators
🏛️ Limitations
Taichi Language Deep Dive
Use Case: Forest Cover Types
⚙️ Python Environment
⚙️ Taichi Kernels
⚙️ Taichi Neural Layer
Evaluation
References
Q&A
Paper Review
Appendix
Matrix Transposition in Taichi
Taichi Autograd Implementation
Evaluation with Mac Metal

Why this matters

Purpose: Python has many tools for speeding up numerical work, but they often come with steep learning curves. Taichi bridges this gap by functioning as a DSL that integrates directly with existing NumPy and Torch pipelines. It uses simple Python annotations to enable high-performance parallel computing for tasks like deep learning.

Audience: Scientists and engineer in need of improving the performance of existing numerical computation or machine learning models.

Value: Learn the basics of Taichi DSL and compare the training performance of a Taichi-enhanced deep learning model against its baseline PyTorch implementation.

Key Takeaways

Taichi is an open-source, Python-integrated parallel programming language optimized for high-performance numerical tasks, including simulation, visualization, and deep learning
By offloading computationally intensive code directly to GPU kernels, it bypasses standard Python limitations.
While its performance is sensitive to specific model architectures and hardware targets, preliminary evaluations indicate that Taichi offers superior scalability compared to PyTorch modules as neural network density increases.
The appropriate application of annotators such as ti.field, ti.function or ti.kernel can be challenging at time.
The Taichi language imposes restrictions on using some of the native Python data structures.

Taichi Language Basics

🏛️ Overview

Taichi is an open-source, imperative, parallel programming language designed for high-performance numerical computation [ref 1].

It is widely used in areas such as real-time physical simulation, scientific computing, augmented reality, artificial intelligence, computer vision and robotics, visual effects for films and games, and general-purpose GPU computing [ref 2]

⚠️ While alternatives to the Taichi programming language exist—such as Numba and CuPy—it is important to note that these frameworks are often designed with different computational objectives in mind. A future article will evaluate Numba.

Taichi is embedded in Python and relies on just-in-time (JIT) compilation to transparently offload compute-intensive Python code to native GPU kernels.

Taichi supports multiple GPU backends, including CUDA, Metal (Apple MPS/Metal), Vulkan, and OpenGL. At runtime, it automatically probes the available backends in a predefined order and falls back to CPU execution if no compatible GPU is found: CUDA → Vulkan → OpenGL → Metal → CPU.

This article is merely an introduction to Taichi domain-specific language (DSL). I recommend the reader to browse through the various tutorials on the subject [ref 3, 4].

📌 Focusing on the intersection of geometric learning and high-performance computing, this issue explores the application of Taichi’s domain-specific language to a standard multi-layer perceptron. Renowned for its efficiency in physical simulation, ray tracing, and intensive numerical tasks, Taichi offers a compelling framework for optimizing neural architectures.

🏛️ Taichi Data Types

ti.fp16, ti.fp32, ti.fp64: Pointers to the corresponding Numpy/PyTorch floating point types
ti.types.ndarray: Maps external arrays in Numpy or PyTorch to an external pointer so Taichi can access it without copying. It uses ti.f32 for the type of elements by default
ti.types.vector(n, dtype): Defines a 1D array of n elements or type type (int16, int32, f32,). It supports operations like .dot(), .norm() … once defined in a kernel
ti.types.matrix(n, m, dtype): Represents a 2D array that Taichi optimizes for small, fixed-size matrix math.

🏛️ Taichi Processing Annotators

ti.field is a generic, global memory container.
ti.kernel defines an entry point to execution within Taichi system for parallelization from anywhere in a program
ti.function implements a function that is executed with the taichi system but invoked from another function or a kernel.
ti.static is used within kernels to force compile-time evaluation
ti.ndrange uses as substitute to Python range as part of vectorization process
ti.loop_config(block_dim) is a performance hint used to manually tune how a Taichi kernel is mapped onto the underlying GPU threads. It groups exactly ‘block_dim’ threads together into a single block for parallelizing a loop. The setting is different on CUDA and Metal.
ti.init(arch, fast_math) is the mandatory entry point that sets up the Taichi runtime environment. It serves as the bridge between your Python script and the high-performance backend (CPU or GPU) that will execute your code.
Architecture: arch
- ti.cpu: Runs on your multi-core CPU (default).
- ti.gpu: Attempts to find the fastest available GPU backend in this order: ti.cuda → ti.vulkan → ti.metal -> ti.opengl -> ti.cpu
Floating point precision: fast_math
- True ti.fp32 for performance
- False ti.fp64 for precision
ti.data_oriented This decorator is used to bridge the gap between Object-Oriented Programming and Taichi’s kernels. When you define a Python class that contains Taichi kernels, the Taichi compiler needs to track the data associated with that specific class instance.

📌 ti.function can only be invoked from another ti.function or a ti.kernel while ti.kernel can be invoked from anywhere in the code except another ti.kernel or ti.function

Taking full advantage of the performance of the Taichi Language would involve digging into the internals components to estimate which part of a training and evaluation workflow is a good candidate [ref 6, 7].

Taichi has established a reputation for high-performance scientific computing and real-time visualization, as demonstrated by the benchmarks and examples provided in [ref 5]. This study provides a comparative analysis of the computational performance of the Taichi programming language relative to PyTorch within the context of deep learning architectures.

🏛️ Limitations

Developers may find some annotators counter-intuitive
You cannot use arbitrary Python libraries (like Matplotlib or PIL) in kernels. Python list comprehensions, generation expressions, list or dictionaries are not allowed inside kernels.
You must declare field types upfront to enforce static typing, and variables cannot change types once defined.
The initial call to a @ti.kernel implementation incurs overhead.
Moving large data set between PyTorch and Taichi creates a performance bottleneck.
Out of bounds access of data (incorrect indexing) does not generate a warning or error message.