=Paper=
{{Paper
|id=Vol-3777/paper16
|storemode=property
|title=Methodology for an Intelligent System Constructing for Modeling the Conduction of an Electrical Impulse in Nerve Axons
|pdfUrl=https://ceur-ws.org/Vol-3777/paper16.pdf
|volume=Vol-3777
|authors=Michael Z. Zgurovsky,Pavlo O. Kasyanov,Liudmyla B. Levenchuk,Vladyslav R. Novykov
|dblpUrl=https://dblp.org/rec/conf/profitai/ZgurovskyKLN24
}}
==Methodology for an Intelligent System Constructing for Modeling the Conduction of an Electrical Impulse in Nerve Axons==
https://ceur-ws.org/Vol-3777/paper16.pdf
Methodology for an Intelligent System Constructing for
Modeling the Conduction of an Electrical Impulse in
Nerve Axons
Michael Z. Zgurovsky1, Pavlo O. Kasyanov2, Liudmyla B. Levenchuk1 and Vladyslav R.
Novykov1
1
National Technical University of Ukraine ‘Igor Sikorsky Kyiv Polytechnic Institute’, Beresteiskyi Ave. 37 build.35, Kyiv,
03056, Ukraine
2
Institute for Applied System Analysis, National Technical University of Ukraine ‘Igor Sikorsky Kyiv Polytechnic Institute’,
Beresteiskyi Ave., 37 build. 35, Kyiv, 03056, Ukraine
Abstract
This article presents a methodology for building an intelligent system for approximate solutions of the
Hodgkin-Huxley equation, which models the conduction of an electrical impulse in nerve axons. Given the
complexity of this nonlinear differential equation and the non-uniqueness of its solutions, we use advanced
computational methods, including Physical Information Neural Networks (PINN) and Deep Learning
Galerkin Method (DLGM). These methods allow us to transform infinite-dimensional stochastic
optimization problems into finite-dimensional ones, providing efficient and accurate numerical simulations.
Our approach combines machine learning with classical biological modeling to overcome the limitations of
traditional numerical methods. We develop an algorithm that approximates solutions for electrical impulse
conduction models by capturing both quantitative and qualitative characteristics of nerve impulse
dynamics. Numerical results confirm the effectiveness of our methodology, demonstrating accurate
approximations and stability of traveling wave solutions for various parameter settings. This research
provides a deeper understanding of neuronal behavior and offers potential applications in the development
of new therapeutic strategies.
Keywords
reaction-diffusion equations; multivalued interaction functions; machine learning; physics-informed
neural networks; approximate solutions
MSC2020: 35-04, 35R70 1
1. Introduction
The Hodgkin-Huxley equation describes the conduction of the nerve impulse in the optic nerve. The
equation has the form
where is a complicated nonlinear function of and its past values. A detailed description of the
function is not necessary here. The physiological fact modeled by this equation is as follows: if the
nerve is stimulated below the threshold, the disturbance simply dampens out, but if stimulated above
the threshold, the disturbance quickly forms a particular shape and moves along the line like a
traveling wave. Hodgkin [4] presents the physiological background and compares the results of
numerical integration with experimental observations. The mathematical problem is to classify the
possible waveforms and prove that any disturbance of appropriate initial size and shape,
traveling at the appropriate speed , stabilizes to a translation of one of these forms:
ProfIT AI 2024: 4th International Workshop of IT-professionals on Artificial Intelligence (ProfIT AI 2024), September 25–27,
2024, Cambridge, MA, USA
mzz@kpi.ua (M. Zgurovsky); kasyanov@i.ua (P. Kasyanov); levenchuk.liudmyla@lll.kpi.ua (L. Levenchuk);
vlad.novykov@gmail.com (V. Novykov)
0000-0001-5896-7466 (M. Zgurovsky); 0000-0002-6662-0160 (P. Kasyanov); 0000-0002-8600-0890 (L. Levenchuk); 0009-
0003-4249-6454 (V. Novykov)
© 2024 Copyright for this paper by its authors.
Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR
Wor
Pr
ks
hop
oceedi
ngs
ht
I
tp:
//
ceur
-
SSN1613-
ws
.or
0073
g
CEUR Workshop Proceedings (CEUR-WS.org)
CEUR
ceur-ws.org
Workshop ISSN 1613-0073
Proceedings
see [7] and references therein. The complexity of this equation has led to the introduction of
simplified models of the Hodgkin-Huxley equation to understand the stabilization mechanism in
similar but simpler circumstances. This issue is reviewed in the works of Cohen [2] and Rinzel [9].
The first simplification is the Fitzhugh-Nagumo equation [3]. Considering that the process is
described by a reaction-diffusion differential equation with discontinuous nonlinearity, the
uniqueness of solutions to the Cauchy problem is not always guaranteed. Therefore, the quantitative
analysis of solutions to such equations requires a methodology for finding all solutions to the Cauchy
problem or, at the very least, a sufficient variety of approximate methods that can potentially provide
an approximation of another solution to the corresponding problem without uniqueness. In this
work, we continue the developments laid out in the work on the quantitative study of solutions of a
class of differential inclusions using so-called physics-informed neural networks (PINNs) [6]. We
note that the qualitative analysis and stability theory for differential inclusions with partial
derivatives were developed in [11]. In the future, the developed methodology will be applied to
systems of discontinuous nonlinearities with applications to both the original Cohen [2] and Rinzel
[9] models, FitzHugh-Nagumo, as well as to other mathematical models such as climatology models,
heat-mass transfer, unilateral problems, problems on manifolds with and without boundaries,
differential-operator inclusions with pseudomonotone type operators, and evolutionary
hemivariational inequalities with possibly nonmonotone potentials. In addition to computational
studies, considerable attention will be given in the future to empirically validating the obtained
results through a series of experiments
2. Problem definition
In this paper, we study an intelligent system methodology designed to approximate solutions for the
model of electrical impulse conduction in nerve axons. We explore the integration of advanced
computational techniques with biological modeling, aiming to overcome the limitations of classical
numerical methods. Recent advancements in intelligent systems, including machine learning and
neural networks, offer promising new avenues for developing approximate solutions to these
complex models. By leveraging the computational power and adaptability of intelligent systems, we
can enhance the accuracy and efficiency of simulations, providing deeper insights into neuronal
behavior and potentially informing the development of novel therapeutic strategies.
Consider the problem:
(1)
with initial conditions:
(2)
where and is the Heaviside step function.
For a fixed let According to [13] (see the book and references
therein), there exists a weak solution , with
, of Problem (1) – (2) in the following sense:
(3)
for all where be a measurable function such that
(4)
The main goal of this paper is to develop an algorithm for approximation of solutions for classes
of electrical impulse conduction in nerve axons con with multivalued interaction functions allowing
for non-unique solutions of the Cauchy problem (1) – (2) via the PINNs; [1, 5, 6, 8] and references
therein.
3. Methodology of Approximate Solutions for Electrical Impulse
Conduction Equations with Multivalued Interaction Functions
Fix an arbitrary and a sufficiently smooth function with compact support
We approximate the function by the following
Lipschitz functions:
(5)
For a fixed consider the problem:
(6)
with initial conditions:
(7)
According to [11] and references therein, for each Problem (6)–(7) has an unique
solution Moreover, [12] implies that each convergent sub-sequence
of corresponding solutions to Problem (6)–(7) weakly converges to a
solution of Problem (1)–(2) in the space
(8)
endowed with the standard graph norm, where
Thus, the first step of the algorithm is to replace the function in Problem (1)–(2) with
considering Problem (6)–(7) for sufficiently large cf [6].
For this purpose let us now consider Problem (6)–(7) for sufficiently large Theorem 16.1.1 from
[5] allows us to reformulate Problem (6)–(7) as an infinite dimensional stochastic optimization
problem over a certain function space. More exactly, let , let
be a probability space, let and be independent random variables.
Assume for all , that
Note that be Lipschitz continuous, and let satisfy for all
that
Theorem 16.1.1 from [5] implies that the following two statements are equivalent:
1. It holds that .
2. It holds is the solution of Problem (6)–(7).
Thus, the second step of the algorithm is to reduce the regularized Problem (6)–(7) to the infinite
dimensional stochastic optimization problem in
(9)
However, due to its infinite dimensionality, the optimization problem (9) is not yet suitable for
numerical computations. Therefore, we apply the third step, the so-called Deep Galerkin Method
(DGM) [10], that is, we transform this infinite dimensional stochastic optimization problem into a
finite dimensional one by incorporating artificial neural networks (ANNs); see [5, 10] and references
therein. Let be differentiable, let satisfy
, and let satisfy for all that
(10)
where is the -dimensional version of a function that is,
is the function which satisfies for all , with
that
for each satisfying and for a function
we denote by the realization function of the
fully-connected feedforward artificial neural network associated to with layers with
dimensions and activation functions defined as:
for all and for each satisfying
the affine function from to associated to is defined as
for all
The final step in the derivation involves approximating the minimizer of using stochastic
gradient descent optimization methods [5]. Let , , for each
let and be random variables. Let for each
(11)
Let is defined as
(12)
for each and let satisfy for all
that
(13)
Ultimately, for sufficiently large the realization is chosen as an
approximation:
of the unknown solution of (1)–(2) in the space defined in (8). So, the following theorem is
justified.
Theorem 1 Let and Then the sequence of
defined in (12)–(13) has an accumulation point in the weak topology of defined in (8). Moreover,
each partial limit of the sequence in hands is weakly converges in to the solution of Problem (1)–
(2) in the sense of (3)–(4).
The empirical risk minimization problems for PINNs and DGMs are typically solved using SGD
or variants thereof, such as Adam [5]. The gradients of the empirical risk with respect to the
parameters can be computed efficiently using automatic differentiation, which is commonly
available in deep learning frameworks such as TensorFlow and PyTorch. We provide implementation
details and numerical simulations for PINNs and DGMs in the next section; cf. [5, 6, 8] and references
theerein.
4. Numerical Implication
Let us present a straightforward implementation of the method as detailed in the previous Section
for approximating a solution of Problem (1)–(2) with and
the initial condition where
(14)
Let This implementation follows the original proposal by [8], where 2.000 realizations
of the random variable are first chosen. Here, is uniformly distributed over and is
uniformly distributed over A fully connected feed-forward ANN with 4 hidden layers, each
containing 50 neurons, and employing the Swish activation function is then trained. The training
process uses batches of size 256 sampled from the 2.000 preselected realizations of
Optimization is carried out using the Adam SGD method [5]. A plot of the resulting approximation
of the solution after training steps is shown in Figure 1.
Figure 1: The birth of a soliton: plots for the functions where
and is an approximation of the solution
of Problem (1)–(2) with where is defined in (14), computed by means of the PINN
method as implemented in Source code 1.
import os
import torch
import matplotlib.pyplot as plt
from torch.autograd import grad
from matplotlib.gridspec import GridSpec
from matplotlib.cm import ScalarMappable
dev = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
T = 1.0 # the time horizon
M = 2000 # the number of training samples
k = 5 # the parameter
torch.manual_seed(0)
x_data = torch.rand(M, 1).to(dev) * torch.pi # Sampling x from (0,
\pi)
t_data = torch.rand(M, 1).to(dev) * T
# The initial value
def phi(x):
return 2 / torch.cosh(10 * (x - torch.pi/2)).unsqueeze(1)
# The interaction function H_k(s)
def H_k(s, a, k):
return torch.where(
s < a, torch.tensor(0.0, device=s.device),
torch.where(s < a + 1/k, k * (s - a), torch.tensor(1.0,
device=s.device))
)
# Define the network
def create_network():
return torch.nn.Sequential(
torch.nn.Linear(2, 50), torch.nn.SiLU(),
torch.nn.Linear(50, 50), torch.nn.SiLU(),
torch.nn.Linear(50, 50), torch.nn.SiLU(),
torch.nn.Linear(50, 50), torch.nn.SiLU(),
torch.nn.Linear(50, 1),
).to(dev)
optimizer = torch.optim.Adam
J = 256 # the batch size
a_values = [0.1, 0.15, 0.2, 0.25, 0.3, 0.35]
# Function to train the model for a specific value of `a`
def train_for_a(a, epochs=M):
model = create_network()
optim = optimizer(model.parameters(), lr=1e-3)
for i in range(epochs):
if i % 100 == 0:
print(f"Iteration {i} for a = {a}")
# Choose a random batch of training samples
indices = torch.randint(0, M, (J,))
x = x_data[indices, :]
t = t_data[indices, :]
x.requires_grad_()
t.requires_grad_()
optim.zero_grad()
# Compute u(0, x) for each x in the batch
u0_pred = model(torch.hstack((x, torch.zeros_like(t))))
# Compute the loss for the initial condition
initial_loss = (u0_pred - phi(x)).square().mean()
# Compute the partial derivatives using automatic
differentiation
u = model(torch.hstack((x, t)))
ones = torch.ones_like(u)
u_t = grad(u, t, ones, create_graph=True)[0]
u_x = grad(u, x, ones, create_graph=True)[0]
u_xx = grad(u_x, x, ones, create_graph=True)[0]
# Compute the loss for the PDE
H_pred = H_k(u, a, k)
pde_loss = (u_t - u_xx - H_pred + u).square().mean()
# Compute the total loss and perform a gradient step
loss = initial_loss + pde_loss
loss.backward()
optim.step()
if i % 100 == 0:
print(f"Loss at iteration {i} for a = {a}: {loss.item()}")
return model
# Function to plot the solution at different times
def plot_solution(a_index, a, model):
mesh = 128
x = torch.linspace(0, torch.pi, mesh).to(dev).unsqueeze(1)
t = torch.linspace(0, T, mesh).to(dev).unsqueeze(1)
x_grid, t_grid = torch.meshgrid(x.squeeze(), t.squeeze(),
indexing="xy")
x_flat = x_grid.reshape(-1, 1)
t_flat = t_grid.reshape(-1, 1)
z = model(torch.cat((x_flat, t_flat), 1))
z = z.detach().cpu().numpy().reshape(mesh, mesh)
return a_index, a, z
def save_plot(results):
gs = GridSpec(2, 4, width_ratios=[1, 1, 1, 0.05])
fig = plt.figure(figsize=(16, 10), dpi=300)
# Find the min and max values for color normalization
z_min = min(result[2].min() for result in results)
z_max = max(result[2].max() for result in results)
for a_index, a, z in results:
ax = fig.add_subplot(gs[a_index // 3, a_index % 3])
ax.set_title(f"a = {a}")
im = ax.imshow(
z, cmap="plasma", extent=[0, torch.pi, 0, T],
aspect='auto', origin='lower', vmin=z_min, vmax=z_max
)
ax.set_xlabel('x')
ax.set_ylabel('t')
# Add the colorbar to the figure
sm = ScalarMappable(cmap="plasma", norm=plt.Normalize(vmin=z_min,
vmax=z_max))
cax = fig.add_subplot(gs[:, 3])
fig.colorbar(sm, cax=cax, orientation='vertical')
# Create the directory if it does not exist
output_dir = "../plots"
os.makedirs(output_dir, exist_ok=True)
fig.savefig(os.path.join(output_dir, "ceur.pdf"),
bbox_inches="tight")
if __name__ == '__main__':
results = []
for i, a in enumerate(a_values):
model = train_for_a(a)
result = plot_solution(i, a, model)
results.append(result)
save_plot(results)
Source code 1: Modified version of source code from Section 16.3 of [5].
5. Conclusions
In this paper, we have developed and validated an intelligent system methodology for approximating
solutions to models of electrical impulse conduction in nerve axons. The core of our approach
integrates advanced computational techniques, notably Physics-Informed Neural Networks (PINNs)
and the Deep Learning Galerkin Method (DLGM), with biological modeling to address the
complexities inherent in these systems.
Our methodology transforms the infinite-dimensional stochastic optimization problems
associated with the Hodgkin-Huxley and FitzHugh-Nagumo equations into tractable finite-
dimensional problems based on the use of artificial neural networks (ANNs). This approach not only
enhances the accuracy and efficiency of the numerical simulations but also provides a robust
framework for accounting for the qualitative characteristics of the solutions, such as stability and
traveling wave phenomena.
The numerical implementation and empirical results confirm the effectiveness of our method. By
successfully approximating the solutions for various parameter values, our method demonstrates its
capability to overcome the limitations of classical numerical methods. The results visually and
quantitatively affirm that the proposed computational method captures both the quantitative
approximation and qualitative behaviors of the model solutions.
Future work will aim to refine the methodology further, exploring more complex models and
extending the approach to other types of partial differential equations encountered in biological and
medical applications. The adaptability and computational power of intelligent modeling systems
promise to further expand the horizons of understanding complex biological processes.
To summarize, the methodology for constructing an intelligent system for approximating
solutions to the conduction equations of electrical impulses in nerve axons presented in this study
represents the next step forward in the numerical analysis of neuronal dynamics, offering a powerful
tool for both theoretical research and practical applications in computational neuroscience.
Acknowledgements
All authors contributed equally to this work. The third author was partially supported by NRFU
project No. 2023.03/0074 "Infinite-dimensional evolutionary equations with multivalued and
stochastic dynamics." The authors express their gratitude to Prof. Oleksiy Kapustyan for a thorough
and productive discussion on the potential use of machine learning methods for approximate
solutions of partial derivative equations that admit non-uniqueness of solutions to Cauchy problems.
References
[1] Christian Beck, Martin Hutzenthaler, Arnulf Jentzen, and Benno Kuckuck. An overview on deep
learning-based approximation methods for partial differential equations. Discrete and
Continuous Dynamical Systems-B, 28(6):3697–3746, 2023.
[2] Hirsh Cohen. Mathematical developments in hodgkin-huxley theory and its approximations.
Lect. Math. Life Sci, 8:89–123, 1976.
[3] Richard FitzHugh. Impulses and physiological states in theoretical models of nerve membrane.
Biophysical journal, 1(6):445–466, 1961.
[4] Alan L Hodgkin. The conduction of the nervous impulse. Liverpool University Press, 1964.
[5] Arnulf Jentzen, Benno Kuckuck, and Philippe von Wurstemberger. Mathematical introduction
to deep learning: methods, implementations, and theory. arXiv preprint arXiv:2310.20360, 2023.
[6] Pavlo O. Kasyanov, Oleksiy V. Kapustyan, Liudmyla B. Levenchuk, and Vladyslav R. Novykov.
Machine learning method for approximate solutions for reaction-diffusion equations with
multivalued interaction functions. Preprints, July 2024.
[7] Henry P McKean. Stabilization of solutions of a caricature of the fitzhugh-nagumo equation.
Communications on Pure and Applied Mathematics, 36(3):291–324, 1983.
[8] Maziar Raissi, Paris Perdikaris, and George E Karniadakis. Physics-informed neural networks:
A deep learning framework for solving forward and inverse problems involving nonlinear
partial differential equations. Journal of Computational physics, 378:686–707, 2019.
[9] John Rinzel. Simple model equations for active nerve conduction and passive neuronal
integration. Lectures on mathematics in the life sciences, 8:125–164, 1976.
[10] Justin Sirignano and Konstantinos Spiliopoulos. Dgm: A deep learning algorithm for solving
partial differential equations. Journal of computational physics, 375:1339–1364, 2018.
[11] Michael Z Zgurovsky and Pavlo O Kasyanov. Qualitative and quantitative analysis of nonlinear
systems. Springer, 2018.
[12] Mikhail Z Zgurovsky, Pavlo O Kasyanov, Oleksiy V Kapustyan, José Valero, and Nina V
Zadoianchuk. Evolution Inclusions and Variation Inequalities for Earth Data Processing III:
Long-Time Behavior of Evolution Inclusions Solutions in Earth Data Analysis, volume 27.
Springer Science & Business Media, 2012.
[13] Mikhail Z Zgurovsky, Valery S Mel’nik, and Pavlo O Kasyanov. Evolution Inclusions and
Variation Inequalities for Earth Data Processing I: Operator Inclusions and Variation
Inequalities for Earth Data Processing, volume 24. Springer Science & Business Media, 2010.