The methods of optimization for coefficient determination in the mathematical model of electroretinal signal were analyzed for the task of detecting neurotoxicity risks (identification of neurotoxicants, evaluation of their type, quantitative characteristics, duration of exposure, etc.). A comparison of the computation time was performed for the coefficient determination algorithm using brute force search, Hooke-Jeeves method, and gradient descent based on a simulated electroretinal signal. The coefficient search algorithm was implemented in the Matlab programming environment.
The history of electroretinography dates back to the late 19th century with the work of physiologist and Nobel laureate Alfred F. Fuchs. He used electrical signals to study the physiology of animal eyes. However, the first person to apply electroretinography to study the human retina was French ophthalmologist Jules Gonin in the 1920s. He used a simple electrode construction based on Foster's discovery.
Electroretinography (ERG) is a method for investigating the electrical activity of the retinal tissue of the eye, which allows for the evaluation of the functional state of various components of the retina, including photoreceptors (cones and rods) and bipolar cells.
Fig. 1. is shown: the electrooculogram (EOG) represents the electrical response from the outer retina (photoreceptor–RPE complex); electroretinogram (ERG) measures the electrical response from the photoreceptor and inner retina; the visual evoked potential (VEP) represents the response from ganglion cells to the occipital cortex; NFL nerve fibre layer, GCL ganglion cell layer, IPL inner plexiform layer, INL inner nuclear layer, OPL outer plexiform layer, ONL outer nuclear layer, RPE retinal pigment epithelium, PR photoreceptor.
This technique holds significant importance in ophthalmology for the diagnosis and monitoring of diseases such as retinal degeneration, glaucoma, and diabetic retinopathy. It is also used in studying the functional state of the human body in the early stages of neurotoxicity.
In the second half of the 20th century, electroretinography (ERG) underwent significant development, leading to improvements in research methods and understanding of retinal functions. The emergence of computer technology enabled the recording and analysis of ERG using digital methods, making data processing easier and facilitating more accurate interpretation of research results. The use of photo stimulators based on xenon lamps and LEDs allowed for control over the intensity and wavelength of light during retinal stimulation, improving the quality and standardization of ERG studies. The use of electrodes with reduced contact area increased the accuracy and sensitivity of the electroretinography method, along with subsequent filtering of the useful electroretinosignal, measurement of thresholds, amplitudes, and durations of signal components, providing more detailed information about the functional state of the retina.
The improvement of standard ERG protocols, which have become widely accessible, has allowed for the standardization of procedures and the comparison of results between different methods and approaches to obtaining electroretinosignal (ERS).
Since the year 2000, electroretinography (ERG) has undergone significant development and enhancement: 1. More efficient electrodes with reduced contact area and improved design have emerged, providing more stable and accurate results in experimental ERG recordings. 2. The use of high-speed analog-to-digital converters (ADCs) has enabled higher signal sampling rates, improving the quality of information obtained from ERG. 3. ERG is employed to study various aspects of retinal functioning, including phototransduction, signal transmission, and response to low-intensity light, in the investigation and diagnosis of neurotoxicity and eye disorders. 4. The development of standardized ERG protocols has facilitated result comparability among different laboratories and researchers. International organizations such as the International Society for Clinical Electrophysiology of Vision (ISCEV) contribute to the development and updating of standards and recommendations for conducting ERG studies.
In modern electroretinography (ERG), new methods and additional investigations are emerging, allowing for more comprehensive information about the functional state of the retina. These include multifocal ERG (mfERG), the combination of ERG with optical coherence tomography (OCT) for complementing retinal structure analysis with functional assessment, and the integration of ERG with genetic studies and mutation analysis to understand the relationship between genetic anomalies, retinal structure, and functional status [2-4].
However, electoretinographic research (especially at low light intensities) is accompanied by a number of challenges. Informative parameters of the ERS can be significantly corrupted, complicating its analysis. The presence of artifacts (eye movements or blinking) can distort the signal and lead to inaccurate results. Processing a large volume of data collected from the ERS can be complex and require the use of advanced algorithms and machine learning methods for effective analysis. The ERS can vary considerably between individuals, making comparison and interpretation of results challenging. Interpreting the ERS can be difficult, especially in the case of complex pathological conditions or changes occurring at different levels of the retina and early stages of neurotoxicity. To build an expert system for the analysis of the ERS (Figure 2), similar to data processing in IoT for smart city systems [5], big data processing in medicine [6] or in complex technical systems of various purposes [7], taking into account the aforementioned problems, it is necessary to employ an appropriate mathematical model and further methods for processing the experimentally obtained ERG. Previous works have justified a mathematical model of ERG as a damped oscillatory process using a function that is a solution to a linear second-order differential equation with constant coefficients (with the function representing the model of retinal light stimulation) [8].
To determine the parameters of the mathematical model (coefficients of the difference equation), a method of direct exhaustive search (brute force) was used, which guarantees a predefined accuracy and convergence but is computationally intensive. The significant processing time of the ERG by the expert system hinders its application for remote, automated, real-time monitoring of human body conditions (particularly in different toxicities).
An improved method of parametric identification of the mathematical model based on the HookeJeeves method exists, which combines exploratory search with cyclic variable change and pattern search [10]. The Hooke-Jeeves method is a simple and efficient optimization method, especially in cases where the function lacks an analytical derivative or is non-differentiable. It is typically employed for local optimization, finding the optimal solution within a constrained parameter space. Although the Hooke-Jeeves method is effective for local optimization, it has some drawbacks: a) Dependency on the initial point - if the initial point is chosen incorrectly, the algorithm may converge to a local optimum and fail to achieve global optimality. b) High number of iterations - the algorithm may require a considerable number of iterations to reach the desired solution, particularly in complex parameter spaces. c) Sensitivity to noise - even small amounts of noise can lead to significant search deviations and affect the discovery of the optimal solution.
Given the aforementioned drawbacks, several optimization methods are considered more effective than the Hooke-Jeeves method. These include the Nelder-Mead method, the Broyden-FletcherGoldfarb-Shanno (BFGS) method, the Conjugate Gradient method, and others [11].
The Nelder-Mead method is one of the most widely used derivative-free optimization methods. It employs an iterative process in which a set of points, called a simplex, is modified at each iteration to find the minimum or maximum of a function. The main idea of the Nelder-Mead method is to gradually expand or contract the simplex in the direction of optimization based on function comparisons at different points.
Simplex is a polyhedron composed of vertices that correspond to points in the parameter space." "The Nelder-Mead method has several disadvantages to consider when applying it: slow convergence speed (it can be quite slow to converge to the global minimum or maximum, especially for complex functions or in high-dimensional parameter space. This is due to its local simplex transformation strategies), tendency to get stuck in local minima or maxima (especially when the function has many local extrema or when the simplex enters an area where the function has poor shape), sensitivity to the initial approximation (incorrect placement of the simplex can lead to convergence to an incorrect solution), and others." "The Broyden-Fletcher-Goldfarb-Shanno (BFGS) method is an iterative algorithm for unconstrained nonlinear function minimization. It combines ideas from quasi-Newton methods and conjugate gradient methods, making it efficient and widely used in optimization problems." "The main idea of the BFGS method is to approximate the quasi-Newton Hessian matrix, which estimates the second derivatives of the function. The initial Hessian matrix can be chosen arbitrarily, but typically an identity matrix or a diagonal matrix is used. During the iterative process, the Hessian matrix is updated at each step based on changes in the gradients.
The main advantages of the BFGS method are: it has fast convergence, especially compared to the gradient descent method (it can quickly find the function minimum, especially in the case of smooth and nonlinear functions); it can be applied to a wide range of unconstrained optimization problems (it does not require formulating constraints on variables); compared to methods that require computing the full Hessian matrix, the BFGS method has lower computational costs (the Hessian matrix is updated iteratively, saving time and resources). However, the BFGS method also has its disadvantages: it requires computing and storing the function's Hessian matrix at each iteration (which can be very costly for large problems with a large number of parameters); storing the Hessian matrix at each iteration may require significant memory, especially for large problems; it does not guarantee convergence in unbounded problems (if the function is unbounded in the optimization direction, the method can diverge or get stuck in local minima); the choice and determination of the initial approximation can greatly impact the speed and quality of convergence of the BFGS method. Therefore, applying this method for optimizing the parameter determination algorithm of the ERS model makes it impractical for use in an expert system for real-time monitoring of a human body's condition.
The conjugate gradient method is an efficient algorithm for minimizing quadratic functions in nonlinear spaces. It is commonly used for solving optimization problems, especially in the case of large systems of linear equations or machine learning tasks. The main idea of the conjugate gradient method is that each subsequent descent direction is chosen to be conjugate to all previous directions, allowing for faster convergence to the function minimum.
The ordinary gradient descent method is used for minimizing a function of a single variable. The formula for updating the parameter in each iteration of the gradient descent method is as follows: θi = θi-1 - α * ∇J(θi-1) (1) where: θi-1 – the previous value of the parameter of the objective function, θi – the updated value of the objective function, α – learning rate (step size) that determines how large the update step will be, ∇J(θi) – the gradient of the function at the point θi-1, which is the vector of partial derivatives of the function with respect to each parameter at the point θi-1.
This process is repeated until a specified accuracy or a certain number of iterations is reached. If we have a function with multiple parameters, the formula for updating the parameters is as follows: θi = θi-1 - α * ∇J(θi-1) ……………………….. ………………………… θj = θj-1 - α * ∇J(θj-1) (2) For each parameter from θi to θj.
In the gradient descent method, it is important to set the learning rate α correctly, wh determines the convergence speed of the algorithm. If α is large, instability or divergence may oc while if α is too small, the algorithm may work slowly. Additionally, various variations of t gradient descent method can be used, such as stochastic gradient descent (SGD), mini-batch gradient descent (mini-batch GD), and others. Each of these variations has its own characteristics and ways of updating parameters. The main drawbacks of the conjugate gradient method are as follows: the conjugate gradient method is particularly efficient for minimizing quadratic functions, but its efficiency may decrease for general nonlinear functions (complex functions may exhibit slow convergence or stagnation); the effectiveness of the conjugate gradient method can depend on the initial approximation (if the initial point is chosen far from the minimum, the algorithm may converge slowly or not converge at all); if the problem has constraints, additional methods need to be used to incorporate these constraints.
Considering the advantages of the conjugate gradient method, we will apply it for optimizing the parameter identification of the mathematical model of the ERC. To do this, we will perform modeling of a test ERC in the MATLAB environment and find the optimal values of the model coefficients, b1_opt and b2_opt. The determination of these optimal coefficients will be evaluated based on the closeness of the modeled ERS sˆk to the known ERS sk using a criterion value:
N11 nN1 sk sˆk 2 (3)
To compare and evaluate the prototype method and the improved method, we determine the time of selection of coefficients by the brutal force method and Hook-Jeeves method [10] and conjugate gradient method at different numbers of search points (Table 1).
As seen from the provided data, the brute-force algorithm is a straightforward and exhaustive method of searching for an optimal solution by systematically evaluating all possible combinations of variables within a given range. It involves discretizing the search space into a grid of points and evaluating the objective function at each point. The algorithm requires a large number of objective function evaluations, resulting in longer processing times. As the number of points increases, the processing time significantly grows due to the exponential increase in the number of combinations to evaluate.
The Hooke-Jeeves method, also known as the pattern search method, is an iterative optimization algorithm that explores the search space by moving from one point to another based on a pattern or direction. The method dynamically adjusts the step sizes and directions based on the improvement in the objective function value. The processing time of the Hooke-Jeeves method is usually faster compared to the brute-force algorithm since it focuses on exploring promising regions of the search space rather than evaluating all possible combinations. However, the method still requires a considerable number of iterations for convergence.
The conjugate gradient method is an iterative optimization algorithm commonly used for solving unconstrained optimization problems. It combines the advantages of the gradient-based approach and the search in conjugate directions. The method updates the search direction iteratively based on the gradient and conjugate direction, allowing efficient navigation in the search space. To determine the optimal step size along the search direction, it is necessary to compute the gradient of the objective function and perform a line search. Compared to the brute-force algorithm and the Hooke-Jeeves method, the conjugate gradient method typically converges faster and requires fewer iterations to reach an optimal solution. As a result, this method can provide faster processing times for optimization problems with a large number of points.
In summary, the brute-force algorithm exhaustively evaluates all combinations, leading to longer processing times. The Hooke-Jeeves method focuses on exploring promising regions and has faster processing times. The conjugate gradient method combines the gradient-based approach and search in conjugate directions for efficient convergence, resulting in faster processing times compared to other methods.
The brutal-force method is the simplest but most time-consuming approach. It requires enumerating all possible combinations of parameters to find the optimal values. Therefore, depending on the number of parameters and the desired accuracy, the brute-force method can be very slow and impractical for large optimization problems. The Hooke-Jeeves method can be effective in cases where the function has a smooth surface and continuous gradients. However, for complex functions with multiple local minima, the Hooke-Jeeves method may get stuck in suboptimal solutions or require many iterations to find the global minimum. In the case of searching for optimal values of the coefficients in the ERS model, the Hooke-Jeeves method requires significantly less execution time. The advantage of using the gradient descent method is that it quickly converges to a local minimum, so the algorithm's processing time is even shorter than that of the Hooke-Jeeves optimization algorithm. Overall, the choice of method depends on the characteristics of the problem and the requirements for speed and accuracy in synthesizing an expert system for diagnosing the human body's condition based on the obtained ERS. The conjugate gradient method is found to be more efficient than the brute force search and Hooke-Jeeves method in terms of computation time. The study contributes to the development of an expert system for real-time monitoring of a human body's condition based on ERS analysis.
[1] Cornish, E. E., Vaze, A., Jamieson, R. V., & Grigg, J. R. (2021). The electroretinogram in the genomics era: outer retinal disorders. Eye, 35(12), 2406–2418. https://doi.org/10.1038/s41433021-01659-y [2] McCulloch, D. L., Marmor, M. F., Brigell, M. G., Hamilton, R., Holder, G. E., Tzekov, R., & International Society for Clinical Electrophysiology of Vision. (2015). ISCEV Standard for fullfield clinical electroretinography (2015 update). Doc Ophthalmol, 130(1), 1-12.