Efficient local validation of partially ordered models via Baysian directed sampling

From Murray Wiki
Jump to navigationJump to search
Title Efficient local validation of partially ordered models via Baysian directed sampling
Authors Kellan Moorse and Richard Murray
Source Submitted, 2024 American Control Conference (ACC)
Abstract We consider the problem of estimating the subset of test conditions under which a simplified model—or set of simplified models—accurately approximates the behavior of a true system. We approach the problem by proposing a compact set of possible test conditions, and an unknown but samplable continous validity function over that set that quantifies the accuracy of the model under each possible condition. We propose a novel Bayes estimator that optimally directs function sampling to greedily minimize the expected posterior misclassification rate of the valid set, which we call minimum posterior misclassification sampling (GP-MPM), and we show that the the method can be extended to approximate the valid sets of a partially ordered set of models, with sample complexity growing sublinearly with the number of models. In testing against a safety-focused model, we show that the algorithm’s estimated valid set approaches the true valid set much more quickly than undirected sampling, even with small sample sizes.
Type Conference submission
URL https://www.cds.caltech.edu/~murray/preprints/mm24-acc s.pdf
DOI
Tag mm24-acc
ID 2023i
Funding AFOSR T&E2
Flags