Unsupervised Learning Meets Emergent pattern

Author:Murphy | View: 24052 | Time: 2025-03-23 18:47:14

Photo by *William L. Stefanov on NASA-JSC. (Cities growth can be understood as a second-order phase transition and an emergent phenomenon).*

This article was inspired by the paper Discovering Phase Transitions with Unsupervised Learning by Lei Wang [1].

Introduction

One of the most intriguing phenomena within the natural sciences is phase transition or critical phenomena, which can usually be seen in the transition between states of matter such as solid, liquid and gas. However, it is a phenomenon that transcends the usual and connects the brain, a sheet of ferromagnetic material and population dynamics, among others [2,3]. However, those emergent patterns of phase transitions are not always easily detectable. Therefore, unsupervised learning techniques can be employed to help with this task. In this article, I'll use as a toy model, for the phase transition recognition with Principal Component Analysis (PCA), one of the simplest models for critical phenomena analysis, a.k.a the 2D Ising model. It was introduced in 1920 by Wilhelm Lenz to describe ferromagnetic materials in a simplified way.

Phase transition and emergent phenomena

When a system undergoes a phase transition, it shows a variety of complex macro-structural patterns generated by the interaction of micro-structural components, giving birth to emergent patterns. The simplest description of these phenomena was made by Landau in the 1930s, where he introduced the concept of an order parameter φ, which can be a scalar, a vector, or even a tensor. The order parameter provides information about the phases of the system and the transition between them. It is possible to define the order of phase transition as:

First order transition

In a first-order phase transition, the order parameter φ will jump discontinuously at the phase change.

Second order transition

In a second-order phase transition, there will be no discontinuous jump in the order parameter φ at the phase transition; however, its derivative will have a discontinuous jump.

Difference between the order parameter in the first and second phase transition order. Image by the author.

One of the Simplest phase transition model

The 2D Ising model is one of the simplest models for ferromagnetic materials and consists of a grid of spins that are described by a discrete variable that takes just two values σᵢ = -1, 1, i.e., spin "up" and spin "down". The interaction between these spins is given by the function called Hamiltonian which is controlled by the temperature T of the system, but I won't go into more details. This system is famous for going through a second-order phase transition, in a critical temperature Tc, with a very simple description.

Illustration of the 2D Ising model, where each grid node is a spin, blue being the turn "down" and red "up" button. Image by the Author.

So, for T < Tc, the temperature is sub-critical, where the spins are practically all aligned. For T > Tc, at the supercritical temperature, where the spins are all approximately randomly distributed. The model at the critical point, i.e. the phase transition, presents very interesting space-time patterns called emergent patterns, with several implications and are studied in various areas.

Simulation of the 2D Ising model showing the spatial patterns for different temperatures. The emergent phenomena appear in the critical temperature. Image by the author.

The order parameter of the 2D Ising model is known and was solved analytically by Onsager in 1944. The solution is simple φ = ∑ᵢ σᵢ ( it's just the sum of every spin value in the lattice) and will be captured by the unsupervised learning without any input besides the system state, i.e. the matrix containing all the spin values [1]. For the 2D Ising model, PCA can be performed and even lead to the identification of the order parameter.

Unknown phase transition and unsupervised learning

Phase transitions or order parameters are not always easily found, so certain unsupervised learning and Dimensionality Reduction techniques can help find phase transitions and critical points. One advantage of these techniques is that none assume the presence or locality of the critical point. A simple and common technique for dimensionality reduction is Principal Component Analysis (PCA), which is widely used in various physical systems to help with this problem.

PCA is a dimensionality reduction technique through principal components, which are mutually orthogonal directions along which the data variances grow monotonically as one approaches the first components. PCA finds the principal components through a simple linear transformation of the original coordinates Y = XW. The orthogonal transformation is made on column vectors W = (w₁,w₂,…,wₙ) and denotes wₗ as weights of principal components in configuration space. They are determined by an eigenvector and eigenvalue problem [1].

Where XᵗX is equivalent to the covariance matrix. I won't go into detail, but there's something really interesting happening here.

By keeping only the first principal components, PCA is an efficient dimensionality reduction approach that captures most of the linear variations in the original data as shown below. It's possible to see that the first component represents the axis where the data is more spread out.

An illustrative example of PCA components with two-dimensional data. Image by the author.

When applied to state configurations, i.e., the multidimensional space in which each system state is located, PCA finds the most significant linear variations in data. Thus, the method will make a linear combination of the base vectors that form each system state so that the variance grows monotonically. Hence, it's possible in many cases to observe phase transitions if they exist.

Searching for the phase transition

For the 2D Ising model, PCA can be performed and can even lead to the identification of the order parameter.

Configurations of uncorrelated spins can be generated at different temperatures ranging from T/J = 1.6 to 2.9, assuming the existence of the critical temperature within this range, and joining each simulation into an _M_xN matrix, where M=900 is the total number of configurations and N is the number of spins on the lattice. Thus, all elements of the first row of the matrix, (S₁ⱼ), will be a spin σᵢ = -1,1 of the lattice of simulation 1 at temperature T = 1.6, and in the second row of the matrix, (S₂ⱼ), will be a spin σᵢ = -1,1 of the lattice of simulation 2 at temperature T = 1.62.

First two PCA components, where each point is a simulation of the model of 2D Ising at a certain temperature. It's possible to see a defined cluster for the super-critic and sub-critic states, on the other hand, the phase transition is spread across the components. Image by the author.

With the use of only two components, the appearance of clusters for different temperatures is evident, with well-characterized clusters for the supercritical T > Tc and subcritical T < Tc cases, while configurations near the phase transition are dispersed. At this point, it's possible for other supervised algorithms to separate those clusters. It's important to remark that using other components will not make a significant contribution, as the vast majority of the variance in the simulation data is found in the first two components.

Cumulative explained variance over the PCA components. The first component contains almost all of the variance. Y axis is in log scale. Image by the author.

Now it's possible to see that the PCA first component indeed captured the order parameter for the 2D Ising Model! Whence, more advanced Unsupervised Learning methods can be used to find phase transitions and emergent phenomena for more complex data and models.

Comparison between the first component and order parameter at a second-order phase transition. Image by the author.

Conclusion

The use of unsupervised learning and dimensionality reduction methods can detect the critical behavior of the model without even assuming the location and existence of the phase transition, or even input information besides the system states data. This proves to be quite useful in more complex models that do not have an analytical solution, with complicated phase transitions and emergent phenomena.

The notebook for this article is available here.