在邯郸学步后，想要深入用好Tensorflow中的STS model，还是要静下心来，好好阅读点材料。
f ( t ) = f 1 ( t ) + f 2 ( t ) + . . . + f n ( t ) + ε ; ε ∼ N ( 0 , σ 2 ) f(t) = f_1(t) + f_2(t) + ... + f_n(t) + \varepsilon; \varepsilon \sim N(0, \sigma^2) f(t)=f1(t)+f2(t)+...+fn(t)+ε;ε∼N(0,σ2)
1、 时间序列模型的用处，除了实现预则立之外，还可以有以下3方面作用：
2、STS的核心是Variation Inference and Hamiltonian Monte Carlo，可以实现单点和不确定性的预测，可以利用GPUs实现平行计算及与深度神经网络的整合。
TensorFlow Probability (TFP) now features built-in support for fitting and forecasting using structural time series models. This support includes Bayesian inference of model parameters using variational inference (VI) and Hamiltonian Monte Carlo (HMC), computing both point forecasts and predictive uncertainties. Because they’re built in TensorFlow, these methods naturally take advantage of vectorized hardware (GPUs and TPUs), can efficiently process many time series in parallel, and can be integrated with deep neural networks.
3、95% 预测置信区间。
We can see that the forecast uncertainty (shading ± 2 standard deviations) increases over time, as the linear trend model becomes less confident in its extrapolation of the slope. The mean forecast combines the seasonal variational with a linear extrapolation of the existing trend, which appears to slightly underestimate the accelerating growth in atmospheric CO2, but the true values are still within the 95% predictive interval.
4、分解各部分的作用
在不太理解算法内部原理的情况下，猜测从下图各部分中预测置信区间的大小看，hour-of-day 、temperature在电量预测模型中起到的比较重要的作用；day-of-week、autogressive次之，其中autogressive贡献了大部分的预测不确定性。
We see that the model has quite reasonably identified a large hour-of-day effect and a much smaller day-of-week effect (the lowest demand appears to occur on Saturdays and Sundays), as well as a sizable effect from temperature, and that it produces relatively confident forecasts of these effects. Most of the predictive uncertainty comes from the autoregressive process, based on its estimate of the unmodeled (residual) variation in the observed series.
5、LinearRegression除了可以处理相关时间序列数据外，也可以处理节假日和特定日期。
For time series depending on additional, time-varying covariates. Regression components can also be used to encode holiday or other date-specific effects.
As a more advanced application, we might use the design matrix to encode holiday effects. For example, suppose we are modeling data from the month of December. We can combine day-of-week seasonality with special effects for Christmas Eve (Dec 24), Christmas (Dec 25), and New Year’s Eve (Dec 31), by constructing a design matrix with indicators for those dates. Source
holiday_indicators = np.zeros([31, 3])
holiday_indicators[23, 0] = 1 # Christmas Eve
holiday_indicators[24, 1] = 1 # Christmas Day
holiday_indicators[30, 2] = 1 # New Year's Eve
holidays = tfp.sts.LinearRegression(design_matrix=holiday_indicators,
name='holidays')
day_of_week = tfp.sts.Seasonal(num_seasons=7,
observed_time_series=observed_time_series,
name='day_of_week')
model = tfp.sts.Sum(components=[holidays, seasonal],
observed_time_series=observed_time_series)
6、Seasonal要素. For time series depending on seasonal factors, such as the hour of the day, the day of the week, or the month of the year.
day_of_week = tfp.sts.Seasonal(num_seasons=7,
num_steps_per_season=24,
observed_time_series=y,
name='day_of_week')
hour_of_day = tfp.sts.Seasonal(num_seasons=24,
num_steps_per_season=1,
observed_time_series=y,
name='hour_of_day')
model = tfp.sts.Sum(components=[day_of_week, hour_of_day],
observed_time_series=y)
month_of_year = tfp.sts.Seasonal(
num_seasons=12,
num_steps_per_season=[31, 28, 31, 30, 30, 31, 31, 31, 30, 31, 30, 31],
drift_scale_prior=tfd.LogNormal(loc=-1., scale=0.1),
initial_effect_prior=tfd.Normal(loc=0., scale=5.),
name='month_of_year')
对于4年（包含闰月）的month_of_year处理方式类似如下，这一方式可用于处理农历.
num_days_per_month = np.array(
[[31, 28, 31, 30, 30, 31, 31, 31, 30, 31, 30, 31],
[31, 29, 31, 30, 30, 31, 31, 31, 30, 31, 30, 31], # year with leap day
[31, 28, 31, 30, 30, 31, 31, 31, 30, 31, 30, 31],
[31, 28, 31, 30, 30, 31, 31, 31, 30, 31, 30, 31]])
month_of_year = tfp.sts.Seasonal(
num_seasons=12,
num_steps_per_season=num_days_per_month,
drift_scale_prior=tfd.LogNormal(loc=-1., scale=0.1),
initial_effect_prior=tfd.Normal(loc=0., scale=5.),
name='month_of_year')
10、LocalLevel
为什么说在观察到温度与AR的spikes基本一致，就表明addtional featues或data transformations有助于更好地捕捉到温度feature的作用？
A modeler might use this decomposition to understand how to improve the model. For example, they might notice that some spikes in temperature still seem to coincide with spikes in the AR residual, indicating that additional features or data transformations might help better capture the temperature effect.
关于这个问题，有人提出了类似的问题:
It was mentioned that since peaks in the residual correspond to peaks in temperature other features could be engineered to capture this - are you referring to say polynomial expansions of temperature? any other hints? 针对这个问题，Dave Moore的回复如下：
I think we were imagining polynomial expansions, though you could also consider other transformations like log, or one-hot vectors from quantizing temperature into bins, which at a fine enough discretization can represent arbitrary nonlinear functions. Or to be really fancy you could consider interactions between these (e.g., instead of a piecewise-constant response from quantized features, you’d use the one-hot bins as a masking pattern on the original series to get a piecewise-linear response, or on polynomial features to get a piecewise-polynomial response, etc).
模型中添加平方、立方和乘积等非线性features，往往是比较有用的。
Often it’s useful to add complexity to the model by considering nonlinear features of the input data. A simple and common method to use is polynomial features, which can get features’ high-order and interaction terms. Source
The way of creating polynominal features:
| X 1 X_1 X1 | X 2 X_2 X2 |
|---|---|
| 0 | 1 |
| 2 | 3 |
| 4 | 5 |
>>> poly = PolynomialFeatures(degree = 2)
>>> poly.fit_transform(X)
| 1 | X 1 X_1 X1 | X 2 X_2 X2 | X 1 2 X_1^2 X12 | X 1 X 2 X_1X_2 X1X2 | X 2 2 X_2^2 X22 |
|---|---|---|---|---|---|
| 1 | 0 | 1 | 0 2 0^2 02 | 0 × 1 0 \times 1 0×1 | 1 2 1^2 12 |
| 1 | 2 | 3 | 2 2 2^2 22 | 2 × 3 2 \times 3 2×3 | 3 2 3^2 32 |
| 1 | 4 | 5 | 4 2 4^2 42 | 4 × 5 4 \times 5 4×5 | 5 2 5^2 52 |
>>> X = np.arange(6).reshape(3, 2)
>>> X
array([[0, 1],
[2, 3],
[4, 5]])
>>> poly = PolynomialFeatures(2)
>>> poly.fit_transform(X)
array([[ 1., 0., 1., 0., 0., 1.],
[ 1., 2., 3., 4., 6., 9.],
[ 1., 4., 5., 16., 20., 25.]])
Polynomial expansions for time series:
Refer to the Polynomial Regression and Generating polynomial features, the polynomial expansions for time series maybe like the following code:
>>> from sklearn.preprocessing import PolynomialFeatures
>>> import numpy as np
>>> x = np.arange(5)
>>> x = x[:, np.newaxis]
>>> x
array([[0],
[1],
[2],
[3],
[4]])
>>> polynomial_features= PolynomialFeatures(degree=2)
>>> x_poly = polynomial_features.fit_transform(x)
>>> x_poly
array([[ 1., 0., 0.],
[ 1., 1., 1.],
[ 1., 2., 4.],
[ 1., 3., 9.],
[ 1., 4., 16.]])
>>> polynomial_features= PolynomialFeatures(degree=3)
>>> x_poly = polynomial_features.fit_transform(x)
>>> x_poly
array([[ 1., 0., 0., 0.],
[ 1., 1., 1., 1.],
[ 1., 2., 4., 8.],
[ 1., 3., 9., 27.],
[ 1., 4., 16., 64.]])
Add
l
o
g
(
t
e
m
p
e
r
a
t
u
r
e
)
log(temperature)
log(temperature) transformation to build_model.
temperature_log_effect = sts.LinearRegression(
design_matrix=tf.reshape(np.log(temperature),
(-1, 1)), name='temperature_log_effect')
def build_model(observed_time_series):
hour_of_day_effect = sts.Seasonal(
num_seasons=24,
observed_time_series=observed_time_series,
name='hour_of_day_effect')
day_of_week_effect = sts.Seasonal(
num_seasons=7, num_steps_per_season=24,
observed_time_series=observed_time_series,
name='day_of_week_effect')
temperature_effect = sts.LinearRegression(
design_matrix=tf.reshape(temperature - np.mean(temperature),
(-1, 1)), name='temperature_effect')
# added feature
temperature_log_effect = sts.LinearRegression(
design_matrix=tf.reshape(np.log(temperature),
(-1, 1)), name='temperature_log_effect')
autoregressive = sts.Autoregressive(
order=1,
observed_time_series=observed_time_series,
name='autoregressive')
model = sts.Sum([hour_of_day_effect,
day_of_week_effect,
temperature_effect,
# added feature
temperature_log_effect,
autoregressive],
observed_time_series=observed_time_series)
return model
模型预测精度有了较大改善，如下图所示：
官网model的预测结果：
添加了
l
o
g
(
t
e
m
p
e
r
a
t
u
r
e
)
log(temperature)
log(temperature)后model的预测结果：
