| System Identification Toolbox | |
Step 4: Fine Tuning Orders and Disturbance Structures
For real data there is no such thing as a "correct model structure." However, different structures can give quite different model quality. The only way to find this out is to try out a number of different structures and compare the properties of the obtained models. There are a few things to look for in these comparisons.
Fit Between Simulated and Measured Output
Keep the Model Output View open and look at the fit between the model's simulated output and the measured one for the Validation Data. Formally, you could pick that model, for which this number is the highest. In practice, it is better to be more pragmatic, and also take into account the model complexity, and whether the important features of the output response are captured.
Residual Analysis Test
You should require of a good model that the cross correlation function between residuals and input does not go significantly outside the confidence region. Otherwise there is something in the residuals that originate from the input, and has not been properly taken care of by the model. A clear peak at lag k shows that the effect from input u(t-k) on y(t) is not correctly described. A rule of thumb is that a slowly varying cross correlation function outside the confidence region is an indication of too few poles, while sharper peaks indicate too few zeros or wrong delays.
Pole Zero Cancellations
If the pole-zero plot (including confidence intervals) indicates pole-zero cancellations in the dynamics, this suggests that lower order models can be used. In particular, if it turns out that the orders of ARX models have to be increased to get a good fit, but that pole-zero cancellations are indicated, then the extra poles are just introduced to describe the noise. Then try ARMAX, OE, or BJ model structures with an A or F polynomial of an order equal to that of the number of noncanceled poles.
What Model Structures Should be Tested?
Well, you can spend any amount of time to check out a very large number of structures. It often takes just a few seconds to compute and evaluate a model in a certain structure, so that you should have a generous attitude to the testing. However, experience shows that when the basic properties of the system's behavior have been picked up, it is not much use to fine tune orders in absurdum just to press the fit by fractions of percents.
Many ARX models: There is a very cheap way of testing many ARX structures simultaneously. Enter in the Orders text field many combinations of orders, using the colon (":") notation. You can also press the Order Selection button. When you select Estimate, models for all combinations (easily several hundreds) are computed and their (prediction error) fit to Validation Data is shown in a special plot. By clicking in this plot the best models with any chosen number of parameters will be inserted into the Model Board, and evaluated as desired.
Many State-space models: A similar feature is also available for black-box state-space models, estimated using n4sid. When a good order has been found, try the PEM estimation method, which often improves on the accuracy.
ARMAX, OE, and BJ models: Once you have a feel for suitable delays and dynamics orders, if is often useful to try out ARMAX, OE, and/or BJ with these orders and test some different orders for the disturbance transfer functions (C and D). Especially for poorly damped systems, the OE structure is suitable.
There is a quite extensive literature on order and structure selection, and anyone who would like to know more should consult the references.
| | Step 3: Examining the Difficulties | Multivariable Systems | |