[OTDev] How to upload a dataset ...

Nina Jeliazkova jeliazkova.nina at gmail.com
Mon Sep 6 08:57:16 CEST 2010


Hi,

On Fri, Sep 3, 2010 at 8:58 PM, surajit ray <mr.surajit.ray at gmail.com>wrote:

> Hi,
>
> Lemme take a look into that ....
>
> heres my code creating the dataset ... can you tell me whats wrong with it
> ?
>



>
> *****************************************
>             this.resultDataset = createModel();
>             Individual dataset =
>
> resultDataset.createIndividual(resultDataset.getOntClass(OT.OTClass.Dataset.getNS()));
>             OT.OTClass.Dataset.createOntClass(resultDataset);
>

Have to check if it indeed makes different to Jena, but may be worth
changing the order of the two statements above (first creating the dataset
class and then using it)


>             OT.OTClass.DataEntry.createOntClass(resultDataset);
>             OT.OTClass.Feature.createOntClass(resultDataset);
>             OT.OTClass.FeatureValue.createOntClass(resultDataset);
>             OT.OTClass.Compound.createOntClass(resultDataset);
>
>
>             for (int i=0 ; i<this.compoundURIs.length ; i++) {
>                 //create data entry
>                 Individual dataEntry =
>
> resultDataset.createIndividual(OT.OTClass.DataEntry.getOntClass(resultDataset));
>                 dataset.addProperty(OT.dataEntry, dataEntry);
>                 //add compound to data entry
>                 Individual newcompound =
> resultDataset.createIndividual(this.compoundURIs[i],
> OT.OTClass.Compound.getOntClass(resultDataset));
>                 dataEntry.addProperty(OT.compound, newcompound);
>                 //First value
>                 Individual feature1 =
>
> resultDataset.createIndividual(MaxtoxApplicationSettings.getServerRootPath()
> + "/feature/" + modelNumber + "_prediction",
> OT.OTClass.Feature.getOntClass(resultDataset));
>                 Individual featureValue1 =
>
> resultDataset.createIndividual(OT.OTClass.FeatureValue.getOntClass(resultDataset));
>                 featureValue1.addProperty(OT.feature, feature1);
>                 featureValue1.addLiteral(OT.value,
> resultDataset.createTypedLiteral(this.predictions[i],
> XSDDatatype.XSDstring));
>
>                 //Second value
>                 Individual feature2 =
>
> resultDataset.createIndividual(MaxtoxApplicationSettings.getServerRootPath()
> + "/feature/" + modelNumber + "_fingerprint",
> OT.OTClass.Feature.getOntClass(resultDataset));
>                 Individual featureValue2 =
>
> resultDataset.createIndividual(OT.OTClass.FeatureValue.getOntClass(resultDataset));
>                 featureValue2.addProperty(OT.feature, feature2);
>                 featureValue2.addLiteral(OT.value,
> resultDataset.createTypedLiteral(this.fingerprints[i],
> XSDDatatype.XSDdouble));
>
>                 //and finally add values to the data entry
>                 dataEntry.addProperty(OT.values, featureValue1);
>                 dataEntry.addProperty(OT.values, featureValue2);
>             }
> *******************************************************
>
>
>
Can't really spot what's wrong, will try to run it.

>
>
> On 3 September 2010 22:34, Nina Jeliazkova <jeliazkova.nina at gmail.com>
> wrote:
> > Hi Surajit,
> >
> > I assume resultDataset variable is a Jena Model?
> >
> > Here is one of the RDF files you've tried to upload
> >
> > http://www.ideaconsult.net/downloads/maxtox/ambit2_22761.rdf
> >
> > First observation there is no any ot:Dataset instance. There are multile
> > ot:DataEntry, but no dataset to where they belong.  This is definitely a
> > reason the dataset could not be imported - the parser simply doesn't find
> a
> > dataset to work with and does nothing.
> >
> > Second observation - you have defined value of type double, but the value
> > itself is a string of 0 and 1s (the fingerprint).  This is at least not
> > correct and will most probably result in errors when parsing values.
> >
> >        <ot:FeatureValue>
> >             <ot:value rdf:datatype="
> http://www.w3.org/2001/XMLSchema#double"
> >
>
> >>0,0,0,1,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0</ot:value>
> >             <ot:feature
> > rdf:resource="http://localhost:8080/MaxtoxTest/feature/1_fingerprint"/>
> >           </ot:FeatureValue>
> >
> Corrected that ... I had interchanged the types between prediction and
> fingerprint
>
> > Ah, and (relatively minor)  issue - the feature URI
> > http://localhost:8080/MaxtoxTest/feature/1_fingerprint will not be
> > dereferencable, even if you are running feature service at your machine -
> > just because "localhost" means your machine for you, but my machine to
> me.
> >
>
> Yeah ... that was during testing from my machine ...I have loaded the
> same on the freiburg server which has the correct (absolute) server
> path.
>
> > If you are not running feature service elsewehere it would be better to
> > either have anonymous node for it, or create it beforehand  (POST to
> > http://{host}/ambit2/feature) and provide the URI in the dataset RDF.
> TUM
> > and NTUA developers can share their experience/code as well.
> >
>
> Will try that in a future iteration ... however I have a thought
> regarding this. Remoting is good as long as the information load to be
> maintained is high and the connection is good. I believe localizing
> the smaller stuff is better - as it leads to fewer steps that can fail
> (in the course of a bad connection)
>

Both approaches above a valid, the point is to have the information about
the feature stored somewhere and linked to the rest.
Low quality connection is what stateless protocols are good for.


>
> > Third,  this  code will not work as you might expect:
> >
> > this.uploaded_URI = output.getText(); ,
> >
> > because task RDF will be returned, not a single URI (you have specified
> > request.getClientInfo().getAcceptedMediaTypes().add(new
> > Preference<MediaType>(MediaType.APPLICATION_RDF_XML));
> >
>
> Do I have to drill through the RDF representation to reach the URI ?
>

If requesting RDF, yes.  But you might also set accepted media type to
text/uri-list , this will return URI only.


> Also if its taking a long time what is your polling strategy ?
>
>
The polling strategy is decided by the client side.


> Are there any ready pieces of code for polling ?
>
>
> Look at RemoteTask class at the opentox-client SVN link

Nina


>
> > Hope this helps,
> > Nina
> >
>
> Lastly some links I had left out (for reference)
>
> models at :
> http://opentox2.informatik.uni-freiburg.de:8080/MaxtoxTest/model
> particular (for ex model no 1) :
> http://opentox2.informatik.uni-freiburg.de:8080/MaxtoxTest/model/1
>
> models take a dataset as the input and generate the fingerprints and
> the predictions.
>
> Regards
> Surajit
>
> >
> > On Fri, Sep 3, 2010 at 6:33 PM, surajit ray <mr.surajit.ray at gmail.com>
> > wrote:
> >>
> >> Hi,
> >>
> >> Tried uploading using this piece of code
> >>
> >> ************************************
> >> System.out.println(result_dataset_URI);
> >>            Representation representation = new
> >> OutputRepresentation(MediaType.APPLICATION_RDF_XML) {
> >>
> >>                @Override
> >>                public void write(OutputStream out) throws IOException {
> >>                    resultDataset.write(out, "RDF/XML-ABBREV");
> >>                }
> >>            };
> >>
> >>            Request request = new Request();
> >>            Client client = new Client(Protocol.HTTP);
> >>            request.setResourceRef(result_dataset_URI);
> >>
> >>  request.setReferrerRef(MaxtoxApplicationSettings.getServerRootPath());
> >>            request.setMethod(Method.POST);
> >>            request.getClientInfo().getAcceptedMediaTypes().add(new
> >> Preference<MediaType>(MediaType.APPLICATION_RDF_XML));
> >>            request.setEntity(representation);
> >>
> >>
> >>            Response response = client.handle(request);
> >>            // Get the output from the response
> >>            Representation output = response.getEntity();
> >>            this.uploaded_URI = output.getText();
> >> ************************************
> >>
> >>
> >>
> >>
> >>
> >> I got this as the result
> >>
> >>
> >>
> >>
> >> *************************************
> >> 0.0 Error 1283527568408 File import application/rdf+xml [-1] The
> >> server has fulfilled the request but does not need to return an
> >> entity-body, and might want to return updated metainformation The
> >> server has fulfilled the request but does not need to return an
> >> entity-body, and might want to return updated metainformation 500
> >> *************************************
> >>
> >> Could you clarify ?
> >>
> >> I am using the http://apps.ideaconsult.net:8080/ambit2/dataset for
> >> testing as the other server was down - sorry for that.
> >
> >
> >>
> >> I just uploaded
> >> twice so you can delete 2 vague looking databases ...
> >
> >
> >
> >>
> >> Regards
> >> Surajit
> >>
> >>
> >>
> >> On 3 September 2010 11:46, Nina Jeliazkova <jeliazkova.nina at gmail.com>
> >> wrote:
> >> > Hi Surajit,
> >> >
> >> > On Fri, Sep 3, 2010 at 1:50 AM, surajit ray <mr.surajit.ray at gmail.com
> >
> >> > wrote:
> >> >>
> >> >> From your previous example I gathered this
> >> >>
> >> >> RDFRepresentation r = new RDFRepresentation();
> >> >> Representation rep = r.createRepresentation(rdfDataset,
> >> >> MediaType.APPLICATION_RDF_XML);
> >> >>
> >> >> The problem is this representation does not exist in the stable
> >> >> version of Restlets !
> >> >
> >> > If you refer to opentox-client examples, then RDFRepresentation is not
> a
> >> > Restlet class at all, but custom one, using Jena writers for
> serializing
> >> > RDF.
> >> >
> >> >
> >> >
> https://ambit.svn.sourceforge.net/svnroot/ambit/branches/opentox/opentox-client/src/main/java/org/opentox/rdf/representation/RDFRepresentation.java
> >> >
> >> > Restlet RDF support is rather minimalistic (graph representation, no
> >> > querying, let alone reasoning), I am not sure if any of the partners
> >> > actually uses it. If you are working with 1.x Restlet , there is
> indeed
> >> > no
> >> > RDF support there. But Restlet RDF support can be safely ignored if
> you
> >> > are
> >> > already using Jena (though I would be curious to see performance
> >> > comparison
> >> > between the two).
> >> >
> >> > Using StringRepresentation/ FileRepresentation with media type
> >> > "application/rdf+xml" does the same job.  RDFRepresentation class
> >> > discussed
> >> > above actually creates OutputStreamRepresentation.
> >> >
> >> > Upload examples can be found here
> >> >
> >> >
> >> >
> https://ambit.svn.sourceforge.net/svnroot/ambit/branches/opentox/opentox-client/src/test/java/org/opentox/dataset/test/DatasetTest.java
> >> >
> >> >
> https://ambit.svn.sourceforge.net/svnroot/ambit/branches/opentox/opentox-client/src/test/java/org/opentox/dataset/test/FeatureTest.java
> >> >
> >> > The examples above are cleaned up versions of ambit code. JUnit tests
> of
> >> > ambit dataset services can be found here
> >> >
> >> >
> >> >
> https://ambit.svn.sourceforge.net/svnroot/ambit/trunk/ambit2-all/ambit2-www/src/test/java/ambit2/rest/test/dataset/DatasetsResourceTest.java
> >> >
> >> >
> >> >
> >> >>
> >> >> Also the example uses a RemoteTask to achieve the actual upload.
> >> >> Therefore I could not exactly fathom how to do the upload.
> >> >
> >> > Here is a summary
> >> >
> >> > curl -X POST -d "@filecontent" -H "Content-type:application/rdf+xml"
> >> > http://datasetservice/dataset
> >> >
> >> > This returns Task object (either URI, if -H "Accept:text/uri-list" is
> >> > specified or Task RDF representation if -H
> "Accept:application/rdf+xml"
> >> > is
> >> > specified).  Task RDF representation is defined in
> >> > http://opentox.org/api/1_1/opentox.owl
> >> >
> >> > If the task is still running you'll get result codes 201 or 202 , this
> >> > means
> >> > you need to query again the returned task URI.  If code 200 is
> returned,
> >> > the
> >> > URI returned is where the result should be found.   In the examples
> >> > there is
> >> > a RemoteTask class that handles all this.
> >> >
> >> >
> >> >
> https://ambit.svn.sourceforge.net/svnroot/ambit/branches/opentox/opentox-client/src/main/java/org/opentox/task/RemoteTask.java
> >> >>
> >> >> On 3 September 2010 03:47, surajit ray <mr.surajit.ray at gmail.com>
> >> >> wrote:
> >> >> > Hi,
> >> >> >
> >> >> > I have managed to get a dataset (RDF) made using the results of a
> >> >> > model prediction run. Could you please tell me a dataset service
> >> >> > where
> >> >> > I could upload the same ?
> >> >
> >> > For testing
> >> >
> >> > http://ambit-uni-plovdiv.bg:8080/ambit2/dataset
> >> >
> >> > For your final models/results , especially if you would like them
> >> > appearing
> >> > in ToxPredict.
> >> >
> >> > http://apps.ideaconsult.net:8080/ambit2/dataset
> >> >
> >> >
> >> >>
> >> >> Also it would be very helpful if you could
> >> >> > give an example of an upload request using restlets ...
> >> >
> >> > See above.
> >> >
> >> > Hope this helps,
> >> > Nina
> >> >
> >> >
> >> >> >
> >> >> > Thanx
> >> >> > Surajit
> >> >> >
> >> >
> >> >
> >> >
> >
>
> >
>



More information about the Development mailing list