[OTDev] How to upload a dataset ...

surajit ray mr.surajit.ray at gmail.com
Thu Sep 9 20:54:31 CEST 2010


Hi,

Tried changing the order of the statements, putting this

dataset.addProperty(OT.dataEntry, dataEntry);

right after all the feature and feature value creation steps. I am
still getting the error on upload. Could you suggest the reason - so I
can troubleshoot it at our end

Thanks
Surajit

On 6 September 2010 12:27, Nina Jeliazkova <jeliazkova.nina at gmail.com> wrote:
> Hi,
>
> On Fri, Sep 3, 2010 at 8:58 PM, surajit ray <mr.surajit.ray at gmail.com>
> wrote:
>>
>> Hi,
>>
>> Lemme take a look into that ....
>>
>> heres my code creating the dataset ... can you tell me whats wrong with it
>> ?
>
>
>>
>> *****************************************
>>             this.resultDataset = createModel();
>>             Individual dataset =
>>
>> resultDataset.createIndividual(resultDataset.getOntClass(OT.OTClass.Dataset.getNS()));
>>             OT.OTClass.Dataset.createOntClass(resultDataset);
>
> Have to check if it indeed makes different to Jena, but may be worth
> changing the order of the two statements above (first creating the dataset
> class and then using it)
>
>>
>>             OT.OTClass.DataEntry.createOntClass(resultDataset);
>>             OT.OTClass.Feature.createOntClass(resultDataset);
>>             OT.OTClass.FeatureValue.createOntClass(resultDataset);
>>             OT.OTClass.Compound.createOntClass(resultDataset);
>>
>>
>>             for (int i=0 ; i<this.compoundURIs.length ; i++) {
>>                 //create data entry
>>                 Individual dataEntry =
>>
>> resultDataset.createIndividual(OT.OTClass.DataEntry.getOntClass(resultDataset));
>>                 dataset.addProperty(OT.dataEntry, dataEntry);
>>                 //add compound to data entry
>>                 Individual newcompound =
>> resultDataset.createIndividual(this.compoundURIs[i],
>> OT.OTClass.Compound.getOntClass(resultDataset));
>>                 dataEntry.addProperty(OT.compound, newcompound);
>>                 //First value
>>                 Individual feature1 =
>>
>> resultDataset.createIndividual(MaxtoxApplicationSettings.getServerRootPath()
>> + "/feature/" + modelNumber + "_prediction",
>> OT.OTClass.Feature.getOntClass(resultDataset));
>>                 Individual featureValue1 =
>>
>> resultDataset.createIndividual(OT.OTClass.FeatureValue.getOntClass(resultDataset));
>>                 featureValue1.addProperty(OT.feature, feature1);
>>                 featureValue1.addLiteral(OT.value,
>> resultDataset.createTypedLiteral(this.predictions[i],
>> XSDDatatype.XSDstring));
>>
>>                 //Second value
>>                 Individual feature2 =
>>
>> resultDataset.createIndividual(MaxtoxApplicationSettings.getServerRootPath()
>> + "/feature/" + modelNumber + "_fingerprint",
>> OT.OTClass.Feature.getOntClass(resultDataset));
>>                 Individual featureValue2 =
>>
>> resultDataset.createIndividual(OT.OTClass.FeatureValue.getOntClass(resultDataset));
>>                 featureValue2.addProperty(OT.feature, feature2);
>>                 featureValue2.addLiteral(OT.value,
>> resultDataset.createTypedLiteral(this.fingerprints[i],
>> XSDDatatype.XSDdouble));
>>
>>                 //and finally add values to the data entry
>>                 dataEntry.addProperty(OT.values, featureValue1);
>>                 dataEntry.addProperty(OT.values, featureValue2);
>>             }
>> *******************************************************
>>
>>
>
> Can't really spot what's wrong, will try to run it.
>>
>> On 3 September 2010 22:34, Nina Jeliazkova <jeliazkova.nina at gmail.com>
>> wrote:
>> > Hi Surajit,
>> >
>> > I assume resultDataset variable is a Jena Model?
>> >
>> > Here is one of the RDF files you've tried to upload
>> >
>> > http://www.ideaconsult.net/downloads/maxtox/ambit2_22761.rdf
>> >
>> > First observation there is no any ot:Dataset instance. There are multile
>> > ot:DataEntry, but no dataset to where they belong.  This is definitely a
>> > reason the dataset could not be imported - the parser simply doesn't
>> > find a
>> > dataset to work with and does nothing.
>> >
>> > Second observation - you have defined value of type double, but the
>> > value
>> > itself is a string of 0 and 1s (the fingerprint).  This is at least not
>> > correct and will most probably result in errors when parsing values.
>> >
>> >        <ot:FeatureValue>
>> >             <ot:value
>> > rdf:datatype="http://www.w3.org/2001/XMLSchema#double"
>> >
>>
>> >> >>0,0,0,1,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0</ot:value>
>> >             <ot:feature
>> > rdf:resource="http://localhost:8080/MaxtoxTest/feature/1_fingerprint"/>
>> >           </ot:FeatureValue>
>> >
>> Corrected that ... I had interchanged the types between prediction and
>> fingerprint
>>
>> > Ah, and (relatively minor)  issue - the feature URI
>> > http://localhost:8080/MaxtoxTest/feature/1_fingerprint will not be
>> > dereferencable, even if you are running feature service at your machine
>> > -
>> > just because "localhost" means your machine for you, but my machine to
>> > me.
>> >
>>
>> Yeah ... that was during testing from my machine ...I have loaded the
>> same on the freiburg server which has the correct (absolute) server
>> path.
>>
>> > If you are not running feature service elsewehere it would be better to
>> > either have anonymous node for it, or create it beforehand  (POST to
>> > http://{host}/ambit2/feature) and provide the URI in the dataset RDF.
>> > TUM
>> > and NTUA developers can share their experience/code as well.
>> >
>>
>> Will try that in a future iteration ... however I have a thought
>> regarding this. Remoting is good as long as the information load to be
>> maintained is high and the connection is good. I believe localizing
>> the smaller stuff is better - as it leads to fewer steps that can fail
>> (in the course of a bad connection)
>
> Both approaches above a valid, the point is to have the information about
> the feature stored somewhere and linked to the rest.
> Low quality connection is what stateless protocols are good for.
>
>>
>> > Third,  this  code will not work as you might expect:
>> >
>> > this.uploaded_URI = output.getText(); ,
>> >
>> > because task RDF will be returned, not a single URI (you have specified
>> > request.getClientInfo().getAcceptedMediaTypes().add(new
>> > Preference<MediaType>(MediaType.APPLICATION_RDF_XML));
>> >
>>
>> Do I have to drill through the RDF representation to reach the URI ?
>
> If requesting RDF, yes.  But you might also set accepted media type to
> text/uri-list , this will return URI only.
>
>>
>> Also if its taking a long time what is your polling strategy ?
>>
>
> The polling strategy is decided by the client side.
>
>>
>> Are there any ready pieces of code for polling ?
>>
>>
> Look at RemoteTask class at the opentox-client SVN link
> Nina
>
>>
>> > Hope this helps,
>> > Nina
>> >
>>
>> Lastly some links I had left out (for reference)
>>
>> models at :
>> http://opentox2.informatik.uni-freiburg.de:8080/MaxtoxTest/model
>> particular (for ex model no 1) :
>> http://opentox2.informatik.uni-freiburg.de:8080/MaxtoxTest/model/1
>>
>> models take a dataset as the input and generate the fingerprints and
>> the predictions.
>>
>> Regards
>> Surajit
>>
>> >
>> > On Fri, Sep 3, 2010 at 6:33 PM, surajit ray <mr.surajit.ray at gmail.com>
>> > wrote:
>> >>
>> >> Hi,
>> >>
>> >> Tried uploading using this piece of code
>> >>
>> >> ************************************
>> >> System.out.println(result_dataset_URI);
>> >>            Representation representation = new
>> >> OutputRepresentation(MediaType.APPLICATION_RDF_XML) {
>> >>
>> >>                @Override
>> >>                public void write(OutputStream out) throws IOException {
>> >>                    resultDataset.write(out, "RDF/XML-ABBREV");
>> >>                }
>> >>            };
>> >>
>> >>            Request request = new Request();
>> >>            Client client = new Client(Protocol.HTTP);
>> >>            request.setResourceRef(result_dataset_URI);
>> >>
>> >>  request.setReferrerRef(MaxtoxApplicationSettings.getServerRootPath());
>> >>            request.setMethod(Method.POST);
>> >>            request.getClientInfo().getAcceptedMediaTypes().add(new
>> >> Preference<MediaType>(MediaType.APPLICATION_RDF_XML));
>> >>            request.setEntity(representation);
>> >>
>> >>
>> >>            Response response = client.handle(request);
>> >>            // Get the output from the response
>> >>            Representation output = response.getEntity();
>> >>            this.uploaded_URI = output.getText();
>> >> ************************************
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> I got this as the result
>> >>
>> >>
>> >>
>> >>
>> >> *************************************
>> >> 0.0 Error 1283527568408 File import application/rdf+xml [-1] The
>> >> server has fulfilled the request but does not need to return an
>> >> entity-body, and might want to return updated metainformation The
>> >> server has fulfilled the request but does not need to return an
>> >> entity-body, and might want to return updated metainformation 500
>> >> *************************************
>> >>
>> >> Could you clarify ?
>> >>
>> >> I am using the http://apps.ideaconsult.net:8080/ambit2/dataset for
>> >> testing as the other server was down - sorry for that.
>> >
>> >
>> >>
>> >> I just uploaded
>> >> twice so you can delete 2 vague looking databases ...
>> >
>> >
>> >
>> >>
>> >> Regards
>> >> Surajit
>> >>
>> >>
>> >>
>> >> On 3 September 2010 11:46, Nina Jeliazkova <jeliazkova.nina at gmail.com>
>> >> wrote:
>> >> > Hi Surajit,
>> >> >
>> >> > On Fri, Sep 3, 2010 at 1:50 AM, surajit ray
>> >> > <mr.surajit.ray at gmail.com>
>> >> > wrote:
>> >> >>
>> >> >> From your previous example I gathered this
>> >> >>
>> >> >> RDFRepresentation r = new RDFRepresentation();
>> >> >> Representation rep = r.createRepresentation(rdfDataset,
>> >> >> MediaType.APPLICATION_RDF_XML);
>> >> >>
>> >> >> The problem is this representation does not exist in the stable
>> >> >> version of Restlets !
>> >> >
>> >> > If you refer to opentox-client examples, then RDFRepresentation is
>> >> > not a
>> >> > Restlet class at all, but custom one, using Jena writers for
>> >> > serializing
>> >> > RDF.
>> >> >
>> >> >
>> >> >
>> >> > https://ambit.svn.sourceforge.net/svnroot/ambit/branches/opentox/opentox-client/src/main/java/org/opentox/rdf/representation/RDFRepresentation.java
>> >> >
>> >> > Restlet RDF support is rather minimalistic (graph representation, no
>> >> > querying, let alone reasoning), I am not sure if any of the partners
>> >> > actually uses it. If you are working with 1.x Restlet , there is
>> >> > indeed
>> >> > no
>> >> > RDF support there. But Restlet RDF support can be safely ignored if
>> >> > you
>> >> > are
>> >> > already using Jena (though I would be curious to see performance
>> >> > comparison
>> >> > between the two).
>> >> >
>> >> > Using StringRepresentation/ FileRepresentation with media type
>> >> > "application/rdf+xml" does the same job.  RDFRepresentation class
>> >> > discussed
>> >> > above actually creates OutputStreamRepresentation.
>> >> >
>> >> > Upload examples can be found here
>> >> >
>> >> >
>> >> >
>> >> > https://ambit.svn.sourceforge.net/svnroot/ambit/branches/opentox/opentox-client/src/test/java/org/opentox/dataset/test/DatasetTest.java
>> >> >
>> >> >
>> >> > https://ambit.svn.sourceforge.net/svnroot/ambit/branches/opentox/opentox-client/src/test/java/org/opentox/dataset/test/FeatureTest.java
>> >> >
>> >> > The examples above are cleaned up versions of ambit code. JUnit tests
>> >> > of
>> >> > ambit dataset services can be found here
>> >> >
>> >> >
>> >> >
>> >> > https://ambit.svn.sourceforge.net/svnroot/ambit/trunk/ambit2-all/ambit2-www/src/test/java/ambit2/rest/test/dataset/DatasetsResourceTest.java
>> >> >
>> >> >
>> >> >
>> >> >>
>> >> >> Also the example uses a RemoteTask to achieve the actual upload.
>> >> >> Therefore I could not exactly fathom how to do the upload.
>> >> >
>> >> > Here is a summary
>> >> >
>> >> > curl -X POST -d "@filecontent" -H "Content-type:application/rdf+xml"
>> >> > http://datasetservice/dataset
>> >> >
>> >> > This returns Task object (either URI, if -H "Accept:text/uri-list" is
>> >> > specified or Task RDF representation if -H
>> >> > "Accept:application/rdf+xml"
>> >> > is
>> >> > specified).  Task RDF representation is defined in
>> >> > http://opentox.org/api/1_1/opentox.owl
>> >> >
>> >> > If the task is still running you'll get result codes 201 or 202 ,
>> >> > this
>> >> > means
>> >> > you need to query again the returned task URI.  If code 200 is
>> >> > returned,
>> >> > the
>> >> > URI returned is where the result should be found.   In the examples
>> >> > there is
>> >> > a RemoteTask class that handles all this.
>> >> >
>> >> >
>> >> >
>> >> > https://ambit.svn.sourceforge.net/svnroot/ambit/branches/opentox/opentox-client/src/main/java/org/opentox/task/RemoteTask.java
>> >> >>
>> >> >> On 3 September 2010 03:47, surajit ray <mr.surajit.ray at gmail.com>
>> >> >> wrote:
>> >> >> > Hi,
>> >> >> >
>> >> >> > I have managed to get a dataset (RDF) made using the results of a
>> >> >> > model prediction run. Could you please tell me a dataset service
>> >> >> > where
>> >> >> > I could upload the same ?
>> >> >
>> >> > For testing
>> >> >
>> >> > http://ambit-uni-plovdiv.bg:8080/ambit2/dataset
>> >> >
>> >> > For your final models/results , especially if you would like them
>> >> > appearing
>> >> > in ToxPredict.
>> >> >
>> >> > http://apps.ideaconsult.net:8080/ambit2/dataset
>> >> >
>> >> >
>> >> >>
>> >> >> Also it would be very helpful if you could
>> >> >> > give an example of an upload request using restlets ...
>> >> >
>> >> > See above.
>> >> >
>> >> > Hope this helps,
>> >> > Nina
>> >> >
>> >> >
>> >> >> >
>> >> >> > Thanx
>> >> >> > Surajit
>> >> >> >
>> >> >
>> >> >
>> >> >
>> >
>>
>> >
>
>
>



More information about the Development mailing list