Hi Fabian, All,

I will extend the validation service accordingly. One thing I am
wondering: is CDE is a special case of regression, or is it a separate
prediction type?

My question is targeted on the representation within OpenTox, rather
within the Validation. At the moment a validation object has general
figures (e.g. model, training_dataset, num_predicted_instances, ...).
In case of classification it further has classification_statistics
(e.g. confusion_matrix, area_under_roc, ...).
In case of regression it has regression_statistics ( e.g. r_square, rmse, ...).

So if CDE is a subclass of regression, coverage and size would be
added to the regression_statistics. (This would propably only make
sense, if there are models that predict both, the regression target
value as well as an interval.)
If CDE is a seperate prediction type, I would establish a new type of
statistics, cde_statistics.

What do you think?

Best regards,
Martin


On Wed, Nov 17, 2010 at 11:38 AM, Fabian buchwald <FabianBuchwald at gmx.de> wrote:
> Hi Martin,
>
> for our integrated conditional density estimators (CDEs) we need two new validation measures, coverage and size.
> With a CDE you predict an interval. For the validation you want to check whether the true target value is in the interval (coverage) and want to know how large the predicted interval is (size) compared to the range of target values in the training set. The formulas of these two quality measures can be found at http://wwwkramer.in.tum.de/people/research/research/pubs/AAAI10_CDE(page 4, left column).
> Can you provide these new quality measures?
>
> Best regards,
>
> Fabian
> --
> GMX DSL Doppel-Flat ab 19,99 €/mtl.! Jetzt auch mit
> gratis Notebook-Flat! http://portal.gmx.net/de/go/dsl
> _______________________________________________
> Development mailing list
> Development at opentox.org
> http://www.opentox.org/mailman/listinfo/development
>


-- 
Dipl-Inf. Martin Gütlein
Phone:
+49 (0)761 203 8442 (office)
+49 (0)177 623 9499 (mobile)
Email:
guetlein at informatik.uni-freiburg.de