In most cases point estimates are not satisfactory when an estimation problem is faced, reliability certificates are also required. In case of non-linear estimation problems these certificates are usually derived using the asymptotic distribution of the estimate as the sample count tends to infinity.
The combination of non-linear problems with finite sample counts results in faulty uncertainty estimates. We present a family of randomized hypothesis testing methods, called data perturbation (DP) methods, that allow hypothesis testing with exact confidence level for practically any model structure.
When carefully constructed, DP hypothesis tests result in well structured confidence regions. We are going to see an 'appropriate' DP method that works well in general. It results in connected and bounded confidence regions for linear regression problems if the joint distribution of the noise is invariant under a subgroup of the unitary group (i.i.d. noise, componentwise symmetric noise, conditionally uniform noise)
The algorithm is going to be illustrated using a simple non-linear estimation example.
The structure of the confidence regions depend on an intimate relationship between the assumed noise characteristics, model structure and external input. To illustrate this we are going to investigate the power function of the proposed test. As we will see, if the input of the problem does not satisfy certain excitation conditions then the test will reduce to a simple coin toss.