Powered by webMathematica Monash University Engineering IEEE IEEE Instrumentation & Measurement Society Wolfram webMathematica

Benchmark Test Distributions


To date, the evaluation of expanded uncertainty is handled exclusively by Monte Carlo (MC) method. However, there are cases where MC cannot be applied and thus alternative methods are necessary. One typical scenario is when type-A evaluation is required. Currently, the only other mainstream solution is to identify an appropriate distribution based on the information obtained from the high-order moments of the data.


On the other hand, there is currently no comprehensive test distributions that exist to standardize the performance evaluation of moment-based distribution fitting techniques. Without such benchmark test distributions, it is not possible to compare the performance of one technique to the other. The red shaded region in figure (a) below shows where most test distributions used for performance assessment of the fitting techniques lie on skewness-kurtosis plot. The square and diamond points in the same figure show test distributions used for expanded uncertainty estimation using distribution fitting in literature. Therefore, while the fitting techniques can be reliably used for distributions that fall within the shaded region, there is insufficient information on their performance for distributions outside the shaded region.


Powered by webMathematica

For that reason, the objective of the set of distributions in given here is to establish a benchmark distribution set and a framework for performance comparison between various parametric distribution fitting methods especially for expanded uncertainty estimation. A fitting technique can be assessed easily using the framework shown in figure (b) above. The Distribution List and Distribution Solution in figure (b) can be obtained by clicking on the buttons below.


DISTRIBUTION LIST DISTRIBUTION SOLUTION

The button below provides a lookup table for solving g and h parameter for distribution fitting using Tukey's gh distribution


LOOKUP TABLE FOR PARAMETER g AND h IN TUKEY'S GH DISTRIBUTION

This webpage is deployed using webMathematica. Minor part of the benchmark test distributions has been published in 2015 IEEE International Instrumentation and Measurement Technology Conference (I2MTC) in Pisa, Italy. Hence the best citation for using the benchmark test distributions is:

  • A. Rajan, Y. C. Kuang, M. P.-L. Ooi, and S. Demidenko, "Benchmark Test Distributions for Expanded Uncertainty Evaluation Algorithms," IEEE Transactions on Instrumentation and Measurement, vol. 65, 2016

  • This webpage is supported by:

  • Monash University
  • Wolfram Mathematica and webMathematica
  • IEEE Instrumentation and Measurement Society, Technical Committee 32 - Fault Tolerance Measurement Systems

  • Please send any your questions by emailing us.



    Team


    Arvind Rajan

    Monash University Scholar
    Electrical and Computer Systems Engineering
    arvind.rajan@monash.edu

     

    Dr Kuang Ye Chow

    Associate Head of School (Research Training)
    Electrical and Computer Systems Engineering
    Monash University
    kuang.ye.chow@monash.edu

    Dr Melanie Po-Leen Ooi

    Associate Professor
    Electrical, Electronic & Computer Engineering
    Heriot-Watt University
    melanie.ooi@monash.edu
     

    Prof Serge Demidenko

    Associate Head
    School of Engineering & Advanced Technology
    Massey University
    s.demidenko@massey.ac.nz