This should probably better be posted to the development list...

---------- Forwarded message ----------
From: Martin Guetlein <martin.guetlein at googlemail.com>
Date: Tue, May 24, 2011 at 8:26 PM
Subject: Predicted variables and confidence --- was: [OTP] Lazar models
To: opentox partners mailing list <partners at opentox.org>, Nina Jeliazkova <
jeliazkova.nina at gmail.com>
Cc: Christoph Helma <helma at in-silico.ch>


Hi all,

I just managed to produce the first validation report that utilizes
non-lazar 'confidence' values, with a j48 model from ambit:
http://local-ot/validation/report/validation/47
(Once again this is just proof of concept, this is a training data
validation and the confidence value is the class-probability value coming
from WEKA, I asked Nina to add this information to the model predictions
some time ago.)

Both model services (ambit and lazar) now add the confidence as a separate
feature to the prediction dataset which is nice, I think we should keep it
that way.

One deviation is that Ambit adds both features (prediction and confidence)
to Model#predictedVariables while IST puts them into
PredictionDataset#features. IST is doing this because we do not have a
feature service, features do only exist in datasets (which makes A&A
easier). I am fine with both solutions, but we maybe should agree on a
common way to do it?

The second deviation is how the actual prediction and confidence features
look like. To unify this, my proposition would be:
* The predicted feature is of type OT:ModelPredictionFeature (subclass of
OT:Feature)
* The confidence feature is of type OT:ModelConfidenceFeature (subclass of
OT:Feature)
* The confidence feature has a property OT:confidenceOf which points to the
predicted feature (in case a model has more than one prediction feature)

Best regards,
Martin


-- 
Dipl-Inf. Martin Gütlein
Phone:
+49 (0)761 203 8442 (office)
+49 (0)177 623 9499 (mobile)
Email:
guetlein at informatik.uni-freiburg.de