Hi Tobias, All,

On Fri, Dec 4, 2009 at 8:41 AM, Tobias Girschick
<tobias.girschick at in.tum.de> wrote:
> Hello Martin,
>
> thanks for the visulization of the Validation and Reporting Workflows.
> It would be interesting to see the "API-Version" (e.g. sequence of curl
> calls) of the graphical overviews, too. This could also be helpful to
> check if the API in its current state is capable of handling the full
> validation and reporting.

Thats a nice idea, I will add the curl calls.

On Fri, Dec 4, 2009 at 8:55 AM, Tobias Girschick
<tobias.girschick at in.tum.de> wrote:
> Hello Martin,
>
> another thing, that is not clear to me is that you write "The following
> chart illustrates the possible working process of validating an
> algorithm" (http://www.opentox.org/data/documents/development/validation/validation-and-reporting-overview-and-data-flow)
> and further below you say the reports described are "reports for model
> validation".
> In my opinion, the OpenTox user usually will validate a model, not an
> algorithm. On the other hand, if you build "the same" (everything except
> algorithm identical) model with two or three different algorithms (or
> algorithm parameters), you can validate the algorithms (regarding this
> dataset/model).

I'm not quit sure if I got your point right.
I use the term 'validate an algorithm' for the procedure 'use
algorithm to build model on training set, make predictions on test
set, compare predictions to actual values'.
And the term 'validate a model' to 'make predictions on test set,
compare predictions to actual values'.
Both are of course possible with the validation webservice (I just
sketched the first case on the web page, because it is more
complicated, and it includes the second case).

If a developer wants to compare his new algorithm to others, he could
uses the 'validate an algorithm' command (with the new algorithm, as
well as other algorithms, maybe on a range of data sets). Other
techniques like cross-validation are possible as well, of course.

If a developer has a model for a certain endpoint, he will use the
'validate model' command.
Does that answer your question?

Regards,
Martin




>
> best Regards,
> Tobias
>
> On Thu, 2009-12-03 at 18:35 +0100, Martin Guetlein wrote:
>> Hello All,
>>
>> as discussed in the virtual meeting yesterday, I prepared a web page
>> to give some insight into the validation and reporting services:
>>
>> http://www.opentox.org/data/documents/development/validation/validation-and-reporting-overview-and-data-flow
>>
>> (You will find a link to this page on the validation api site as well.)
>>
>> Comments and suggestions for improvement are highly appreciated.
>>
>> Regards,
>> Martin
>>
>
>
> --
> Dipl.-Bioinf. Tobias Girschick
>
> Technische Universität München
> Institut für Informatik
> Lehrstuhl I12 - Bioinformatik
> Bolzmannstr. 3
> 85748 Garching b. München, Germany
>
> Room: MI 01.09.042
> Phone: +49 (89) 289-18002
> Email: tobias.girschick at in.tum.de
> Web: http://wwwkramer.in.tum.de/girschick
>
> _______________________________________________
> Development mailing list
> Development at opentox.org
> http://www.opentox.org/mailman/listinfo/development
>



-- 
Dipl-Inf. Martin Gütlein
Phone:
+49 (0)761 203 8442 (office)
+49 (0)177 623 9499 (mobile)
Email:
guetlein at informatik.uni-freiburg.de