View source for Efficient local validation of partially ordered models via Baysian directed sampling

{{Paper
|Title=Efficient local validation of partially ordered models via Baysian directed sampling
|Authors=Kellan Moorse and Richard Murray
|Source=Submitted, 2024 American Control Conference (ACC)
|Abstract=We consider the problem of estimating the subset of test conditions under which a simplified model—or set of simplified models—accurately approximates the behavior of a true system. We approach the problem by proposing a compact set of possible test conditions, and an unknown but samplable continous validity function over that set that quantifies the accuracy of the model under each possible condition. We propose a novel Bayes estimator that optimally directs function sampling to greedily minimize the expected posterior misclassification rate of the valid set, which we call minimum posterior misclassification sampling (GP-MPM), and we show that the the method can be extended to approximate the valid sets of a partially ordered set of models, with sample complexity growing sublinearly with the number of models. In testing against a safety-focused model, we show that the algorithm’s estimated valid set approaches the true valid set much more quickly than undirected sampling, even with small sample sizes.
|URL=https://www.cds.caltech.edu/~murray/preprints/mm24-acc_s.pdf
|Type=Conference submission
|ID=2023i
|Tag=mm24-acc
|Funding=AFOSR T&E2
|Flags=NCS
}}