Time series forecasting & parameter estimation

Introduction

Ordinary Differential Equations (ODEs) are often used to model dynamic systems and have been successfully applied to many fields including population prediction, decay of radioactive material, electric circuits and biological systems. Trajectories (with respect of time) of different types of variables such as current and voltage / concentration of molecules can be produced according to ODE systems.

In this project, we model an ODE-based human wound healing process according to [Segal et al.]. This ODE-based mathematical model contains 6 variables ($FI{B}_p,FI{B}_m,FI{B}_a,COL,N,P$) representing concentrations of molecules, and 42 coefficient parameters (see the Fig. below).

Based on this ODE system, our task is threefold:

1. Data synthesis: Synthesize time series data according to the ODE given the parameter values.

2. Time series forecasting: Predict near future variable trajectories based on historical data.

3. Parameter estimation: Estimate the value and posterior distributions of the 42 coefficient parameters.

Task 1: data synthesis

We built a simulator to synthesize the data acording the ODE system. Given different sets of parameter values, variable trajectories for the 6 variables are generated with noise added on top of them . The figure below visualizes four settings with different parameter values.

Task 2: time series forecasting

We used LSTM cells to build Seq2seq for the forecasting task. Specifically, we implemented the two types of Many to Many Seq2Seq models listed in the fourth and fifth figure below.

For the little to non-overlap Seq2seq, it is often used in machine translation task. Here, we used it to predict the 6 values from hidden states for the future several time steps. For the most to all-overlap Seq2seq, it is often used in time series forecasting. Here, we used it to predict the 6 values for the next step from the current hidden state. Some results for the next-step prediction are shown below.

Task 3: parameter estimation

For a toy experiment, we first estimate the parameter given a simple ODE system:

with two observable variables ($C_a,C_b$) and two parameters ($k_1,k_2$) with ground truth $k_1=0.18$ and $k_2=0.91$. We used Markov chain Monte Carlo (MCMC) method with 2 sets of 7 data points to estimate the two parameters and the results are shown below.

For the 42-parameter ODE system for the wound healing process, the result of the posterior estimation is shown below.

Conclusion

For the time serires forecasting task, by comparing the MSEs and the visualizations of the forecasting, the time series models can learn the underlying patterns of ODEs well enough in one step forecasting settings, while performs worse in multiple future time step forecasting Long history memory will benefit both one and multiple time step forecasting.

For the parameter estimation task, this work shows the possibility of given the data for a large-scale ODE system for wound healing process, we are able to estimate the parameters by Parallel Tempering MCMC method and obtain the posterior probability distribution of the parameter that potentially can represent different individuals across population.