[OTDev] How to upload a dataset ...
Nina Jeliazkova jeliazkova.nina at gmail.comMon Sep 13 11:30:25 CEST 2010
- Previous message: [OTDev] How to upload a dataset ...
- Next message: [OTDev] How to upload a dataset ...
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Hi, This one should be sufficiently small (3 compounds, 2 features - 1 name, one TUM fragment descriptor ). http://apps.ideaconsult.net:8080/ambit2/dataset/2675?max=3&feature_uris[]=http://apps.ideaconsult.net:8080/ambit2/feature/27962&feature_uris[]=http://apps.ideaconsult.net:8080/ambit2/feature/31185&media=application%2Frdf%2Bxml&rdfwriter=jena <rdf:RDF xmlns:ac="http://apps.ideaconsult.net:8080/ambit2/compound/" xmlns:ot="http://www.opentox.org/api/1.1#" xmlns:bx="http://purl.org/net/nknouf/ns/bibtex#" xmlns:otee="http://www.opentox.org/echaEndpoints.owl#" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:ar="http://apps.ideaconsult.net:8080/ambit2/reference/" xmlns="http://apps.ideaconsult.net:8080/ambit2/" xmlns:am="http://apps.ideaconsult.net:8080/ambit2/model/" xmlns:af="http://apps.ideaconsult.net:8080/ambit2/feature/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:ad="http://apps.ideaconsult.net:8080/ambit2/dataset/" xmlns:ag="http://apps.ideaconsult.net:8080/ambit2/algorithm/" xmlns:owl="http://www.w3.org/2002/07/owl#" xmlns:xsd="http://www.w3.org/2001/XMLSchema#" xmlns:ota="http://www.opentox.org/algorithmTypes.owl#" xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#" xml:base="http://apps.ideaconsult.net:8080/ambit2/"> <owl:Class rdf:about="http://www.opentox.org/api/1.1#Dataset"/> <owl:Class rdf:about="http://www.opentox.org/api/1.1#Compound"/> <owl:Class rdf:about="http://www.opentox.org/api/1.1#Feature"/> <owl:Class rdf:about="http://www.opentox.org/api/1.1#FeatureValue"/> <owl:Class rdf:about="http://www.opentox.org/api/1.1#NumericFeature"> <rdfs:subClassOf rdf:resource="http://www.opentox.org/api/1.1#Feature"/> </owl:Class> <owl:Class rdf:about="http://www.opentox.org/api/1.1#DataEntry"/> <owl:ObjectProperty rdf:about="http://www.opentox.org/api/1.1#dataEntry"/> <owl:ObjectProperty rdf:about="http://www.opentox.org/api/1.1#hasSource"/> <owl:ObjectProperty rdf:about="http://www.opentox.org/api/1.1#values"/> <owl:ObjectProperty rdf:about="http://www.opentox.org/api/1.1#compound"/> <owl:ObjectProperty rdf:about="http://www.opentox.org/api/1.1#feature"/> <owl:DatatypeProperty rdf:about="http://www.opentox.org/api/1.1#units"/> <owl:DatatypeProperty rdf:about="http://www.opentox.org/api/1.1#value"/> <owl:AnnotationProperty rdf:about="http://purl.org/dc/elements/1.1/type"/> <owl:AnnotationProperty rdf:about="http://purl.org/dc/elements/1.1/title "/> <owl:AnnotationProperty rdf:about=" http://purl.org/dc/elements/1.1/description"/> <ot:Dataset rdf:about="dataset/2675"> <ot:dataEntry> <ot:DataEntry> <ot:values> <ot:FeatureValue> <ot:value rdf:datatype="http://www.w3.org/2001/XMLSchema#double" >1.0</ot:value> <ot:feature> <ot:NumericFeature rdf:about="feature/31185"> <dc:creator>Default</dc:creator> <ot:hasSource rdf:resource=" http://localhost:8080/OpenTox-dev/algorithm/FTM5/c%28%3Ac%28%3Ac%28-C%29%29%29%28%3Ac%28%3Ac%28%3Ac%29%29%29 "/> <owl:sameAs rdf:resource=" http://www.opentox.org/api/1.1#TUM_FTM_c%28%3Ac%28%3Ac%28-C%29%29%29%28%3Ac%28%3Ac%28%3Ac%29%29%29 "/> <ot:units>count</ot:units> <dc:title>TUM_FTM_c(:c(:c(-C)))(:c(:c(:c)))</dc:title> <rdf:type rdf:resource=" http://www.opentox.org/api/1.1#Feature"/> </ot:NumericFeature> </ot:feature> </ot:FeatureValue> </ot:values> <ot:values> <ot:FeatureValue> <ot:value rdf:datatype="http://www.w3.org/2001/XMLSchema#string" >zolpidem tartrate</ot:value> <ot:feature> <ot:Feature rdf:about="feature/27962"> <dc:creator>195.178.207.240</dc:creator> <ot:hasSource>290.sdf</ot:hasSource> <owl:sameAs rdf:resource=" http://www.opentox.org/api/1.1#ChemicalName"/> <ot:units></ot:units> <dc:title>Name</dc:title> </ot:Feature> </ot:feature> </ot:FeatureValue> </ot:values> <ot:compound> <ot:Compound rdf:about="compound/153272/conformer/505875"/> </ot:compound> </ot:DataEntry> </ot:dataEntry> <ot:dataEntry> <ot:DataEntry> <ot:values> <ot:FeatureValue> <ot:value rdf:datatype="http://www.w3.org/2001/XMLSchema#double" >1.0</ot:value> <ot:feature rdf:resource="feature/31185"/> </ot:FeatureValue> </ot:values> <ot:values> <ot:FeatureValue> <ot:value rdf:datatype="http://www.w3.org/2001/XMLSchema#string" >amoxicillin</ot:value> <ot:feature rdf:resource="feature/27962"/> </ot:FeatureValue> </ot:values> <ot:compound> <ot:Compound rdf:about="compound/150568/conformer/505874"/> </ot:compound> </ot:DataEntry> </ot:dataEntry> <ot:dataEntry> <ot:DataEntry> <ot:values> <ot:FeatureValue> <ot:value rdf:datatype="http://www.w3.org/2001/XMLSchema#double" >1.0</ot:value> <ot:feature rdf:resource="feature/31185"/> </ot:FeatureValue> </ot:values> <ot:values> <ot:FeatureValue> <ot:value rdf:datatype="http://www.w3.org/2001/XMLSchema#string" >pioglitazone hydrochloride</ot:value> <ot:feature rdf:resource="feature/27962"/> </ot:FeatureValue> </ot:values> <ot:compound> <ot:Compound rdf:about="compound/150563/conformer/505873"/> </ot:compound> </ot:DataEntry> </ot:dataEntry> </ot:Dataset> </rdf:RDF> Hope this helps, Nina On Mon, Sep 13, 2010 at 12:19 PM, surajit ray <mr.surajit.ray at gmail.com>wrote: > Hi, > > It would be helpful to have a working sample of a short dataset in > RDF. That way I can compare this to my dataset representation to check > why the upload task is failing on the ambit server. > > Regards > Surajit > > On 10 September 2010 00:24, surajit ray <mr.surajit.ray at gmail.com> wrote: > > Hi, > > > > Tried changing the order of the statements, putting this > > > > dataset.addProperty(OT.dataEntry, dataEntry); > > > > right after all the feature and feature value creation steps. I am > > still getting the error on upload. Could you suggest the reason - so I > > can troubleshoot it at our end > > > > Thanks > > Surajit > > > > On 6 September 2010 12:27, Nina Jeliazkova <jeliazkova.nina at gmail.com> > wrote: > >> Hi, > >> > >> On Fri, Sep 3, 2010 at 8:58 PM, surajit ray <mr.surajit.ray at gmail.com> > >> wrote: > >>> > >>> Hi, > >>> > >>> Lemme take a look into that .... > >>> > >>> heres my code creating the dataset ... can you tell me whats wrong with > it > >>> ? > >> > >> > >>> > >>> ***************************************** > >>> this.resultDataset = createModel(); > >>> Individual dataset = > >>> > >>> > resultDataset.createIndividual(resultDataset.getOntClass(OT.OTClass.Dataset.getNS())); > >>> OT.OTClass.Dataset.createOntClass(resultDataset); > >> > >> Have to check if it indeed makes different to Jena, but may be worth > >> changing the order of the two statements above (first creating the > dataset > >> class and then using it) > >> > >>> > >>> OT.OTClass.DataEntry.createOntClass(resultDataset); > >>> OT.OTClass.Feature.createOntClass(resultDataset); > >>> OT.OTClass.FeatureValue.createOntClass(resultDataset); > >>> OT.OTClass.Compound.createOntClass(resultDataset); > >>> > >>> > >>> for (int i=0 ; i<this.compoundURIs.length ; i++) { > >>> //create data entry > >>> Individual dataEntry = > >>> > >>> > resultDataset.createIndividual(OT.OTClass.DataEntry.getOntClass(resultDataset)); > >>> dataset.addProperty(OT.dataEntry, dataEntry); > >>> //add compound to data entry > >>> Individual newcompound = > >>> resultDataset.createIndividual(this.compoundURIs[i], > >>> OT.OTClass.Compound.getOntClass(resultDataset)); > >>> dataEntry.addProperty(OT.compound, newcompound); > >>> //First value > >>> Individual feature1 = > >>> > >>> > resultDataset.createIndividual(MaxtoxApplicationSettings.getServerRootPath() > >>> + "/feature/" + modelNumber + "_prediction", > >>> OT.OTClass.Feature.getOntClass(resultDataset)); > >>> Individual featureValue1 = > >>> > >>> > resultDataset.createIndividual(OT.OTClass.FeatureValue.getOntClass(resultDataset)); > >>> featureValue1.addProperty(OT.feature, feature1); > >>> featureValue1.addLiteral(OT.value, > >>> resultDataset.createTypedLiteral(this.predictions[i], > >>> XSDDatatype.XSDstring)); > >>> > >>> //Second value > >>> Individual feature2 = > >>> > >>> > resultDataset.createIndividual(MaxtoxApplicationSettings.getServerRootPath() > >>> + "/feature/" + modelNumber + "_fingerprint", > >>> OT.OTClass.Feature.getOntClass(resultDataset)); > >>> Individual featureValue2 = > >>> > >>> > resultDataset.createIndividual(OT.OTClass.FeatureValue.getOntClass(resultDataset)); > >>> featureValue2.addProperty(OT.feature, feature2); > >>> featureValue2.addLiteral(OT.value, > >>> resultDataset.createTypedLiteral(this.fingerprints[i], > >>> XSDDatatype.XSDdouble)); > >>> > >>> //and finally add values to the data entry > >>> dataEntry.addProperty(OT.values, featureValue1); > >>> dataEntry.addProperty(OT.values, featureValue2); > >>> } > >>> ******************************************************* > >>> > >>> > >> > >> Can't really spot what's wrong, will try to run it. > >>> > >>> On 3 September 2010 22:34, Nina Jeliazkova <jeliazkova.nina at gmail.com> > >>> wrote: > >>> > Hi Surajit, > >>> > > >>> > I assume resultDataset variable is a Jena Model? > >>> > > >>> > Here is one of the RDF files you've tried to upload > >>> > > >>> > http://www.ideaconsult.net/downloads/maxtox/ambit2_22761.rdf > >>> > > >>> > First observation there is no any ot:Dataset instance. There are > multile > >>> > ot:DataEntry, but no dataset to where they belong. This is > definitely a > >>> > reason the dataset could not be imported - the parser simply doesn't > >>> > find a > >>> > dataset to work with and does nothing. > >>> > > >>> > Second observation - you have defined value of type double, but the > >>> > value > >>> > itself is a string of 0 and 1s (the fingerprint). This is at least > not > >>> > correct and will most probably result in errors when parsing values. > >>> > > >>> > <ot:FeatureValue> > >>> > <ot:value > >>> > rdf:datatype="http://www.w3.org/2001/XMLSchema#double" > >>> > > >>> > >>> >> > >>0,0,0,1,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0</ot:value> > >>> > <ot:feature > >>> > rdf:resource="http://localhost:8080/MaxtoxTest/feature/1_fingerprint > "/> > >>> > </ot:FeatureValue> > >>> > > >>> Corrected that ... I had interchanged the types between prediction and > >>> fingerprint > >>> > >>> > Ah, and (relatively minor) issue - the feature URI > >>> > http://localhost:8080/MaxtoxTest/feature/1_fingerprint will not be > >>> > dereferencable, even if you are running feature service at your > machine > >>> > - > >>> > just because "localhost" means your machine for you, but my machine > to > >>> > me. > >>> > > >>> > >>> Yeah ... that was during testing from my machine ...I have loaded the > >>> same on the freiburg server which has the correct (absolute) server > >>> path. > >>> > >>> > If you are not running feature service elsewehere it would be better > to > >>> > either have anonymous node for it, or create it beforehand (POST to > >>> > http://{host}/ambit2/feature) and provide the URI in the dataset > RDF. > >>> > TUM > >>> > and NTUA developers can share their experience/code as well. > >>> > > >>> > >>> Will try that in a future iteration ... however I have a thought > >>> regarding this. Remoting is good as long as the information load to be > >>> maintained is high and the connection is good. I believe localizing > >>> the smaller stuff is better - as it leads to fewer steps that can fail > >>> (in the course of a bad connection) > >> > >> Both approaches above a valid, the point is to have the information > about > >> the feature stored somewhere and linked to the rest. > >> Low quality connection is what stateless protocols are good for. > >> > >>> > >>> > Third, this code will not work as you might expect: > >>> > > >>> > this.uploaded_URI = output.getText(); , > >>> > > >>> > because task RDF will be returned, not a single URI (you have > specified > >>> > request.getClientInfo().getAcceptedMediaTypes().add(new > >>> > Preference<MediaType>(MediaType.APPLICATION_RDF_XML)); > >>> > > >>> > >>> Do I have to drill through the RDF representation to reach the URI ? > >> > >> If requesting RDF, yes. But you might also set accepted media type to > >> text/uri-list , this will return URI only. > >> > >>> > >>> Also if its taking a long time what is your polling strategy ? > >>> > >> > >> The polling strategy is decided by the client side. > >> > >>> > >>> Are there any ready pieces of code for polling ? > >>> > >>> > >> Look at RemoteTask class at the opentox-client SVN link > >> Nina > >> > >>> > >>> > Hope this helps, > >>> > Nina > >>> > > >>> > >>> Lastly some links I had left out (for reference) > >>> > >>> models at : > >>> http://opentox2.informatik.uni-freiburg.de:8080/MaxtoxTest/model > >>> particular (for ex model no 1) : > >>> http://opentox2.informatik.uni-freiburg.de:8080/MaxtoxTest/model/1 > >>> > >>> models take a dataset as the input and generate the fingerprints and > >>> the predictions. > >>> > >>> Regards > >>> Surajit > >>> > >>> > > >>> > On Fri, Sep 3, 2010 at 6:33 PM, surajit ray < > mr.surajit.ray at gmail.com> > >>> > wrote: > >>> >> > >>> >> Hi, > >>> >> > >>> >> Tried uploading using this piece of code > >>> >> > >>> >> ************************************ > >>> >> System.out.println(result_dataset_URI); > >>> >> Representation representation = new > >>> >> OutputRepresentation(MediaType.APPLICATION_RDF_XML) { > >>> >> > >>> >> @Override > >>> >> public void write(OutputStream out) throws > IOException { > >>> >> resultDataset.write(out, "RDF/XML-ABBREV"); > >>> >> } > >>> >> }; > >>> >> > >>> >> Request request = new Request(); > >>> >> Client client = new Client(Protocol.HTTP); > >>> >> request.setResourceRef(result_dataset_URI); > >>> >> > >>> >> > request.setReferrerRef(MaxtoxApplicationSettings.getServerRootPath()); > >>> >> request.setMethod(Method.POST); > >>> >> request.getClientInfo().getAcceptedMediaTypes().add(new > >>> >> Preference<MediaType>(MediaType.APPLICATION_RDF_XML)); > >>> >> request.setEntity(representation); > >>> >> > >>> >> > >>> >> Response response = client.handle(request); > >>> >> // Get the output from the response > >>> >> Representation output = response.getEntity(); > >>> >> this.uploaded_URI = output.getText(); > >>> >> ************************************ > >>> >> > >>> >> > >>> >> > >>> >> > >>> >> > >>> >> I got this as the result > >>> >> > >>> >> > >>> >> > >>> >> > >>> >> ************************************* > >>> >> 0.0 Error 1283527568408 File import application/rdf+xml [-1] The > >>> >> server has fulfilled the request but does not need to return an > >>> >> entity-body, and might want to return updated metainformation The > >>> >> server has fulfilled the request but does not need to return an > >>> >> entity-body, and might want to return updated metainformation 500 > >>> >> ************************************* > >>> >> > >>> >> Could you clarify ? > >>> >> > >>> >> I am using the http://apps.ideaconsult.net:8080/ambit2/dataset for > >>> >> testing as the other server was down - sorry for that. > >>> > > >>> > > >>> >> > >>> >> I just uploaded > >>> >> twice so you can delete 2 vague looking databases ... > >>> > > >>> > > >>> > > >>> >> > >>> >> Regards > >>> >> Surajit > >>> >> > >>> >> > >>> >> > >>> >> On 3 September 2010 11:46, Nina Jeliazkova < > jeliazkova.nina at gmail.com> > >>> >> wrote: > >>> >> > Hi Surajit, > >>> >> > > >>> >> > On Fri, Sep 3, 2010 at 1:50 AM, surajit ray > >>> >> > <mr.surajit.ray at gmail.com> > >>> >> > wrote: > >>> >> >> > >>> >> >> From your previous example I gathered this > >>> >> >> > >>> >> >> RDFRepresentation r = new RDFRepresentation(); > >>> >> >> Representation rep = r.createRepresentation(rdfDataset, > >>> >> >> MediaType.APPLICATION_RDF_XML); > >>> >> >> > >>> >> >> The problem is this representation does not exist in the stable > >>> >> >> version of Restlets ! > >>> >> > > >>> >> > If you refer to opentox-client examples, then RDFRepresentation is > >>> >> > not a > >>> >> > Restlet class at all, but custom one, using Jena writers for > >>> >> > serializing > >>> >> > RDF. > >>> >> > > >>> >> > > >>> >> > > >>> >> > > https://ambit.svn.sourceforge.net/svnroot/ambit/branches/opentox/opentox-client/src/main/java/org/opentox/rdf/representation/RDFRepresentation.java > >>> >> > > >>> >> > Restlet RDF support is rather minimalistic (graph representation, > no > >>> >> > querying, let alone reasoning), I am not sure if any of the > partners > >>> >> > actually uses it. If you are working with 1.x Restlet , there is > >>> >> > indeed > >>> >> > no > >>> >> > RDF support there. But Restlet RDF support can be safely ignored > if > >>> >> > you > >>> >> > are > >>> >> > already using Jena (though I would be curious to see performance > >>> >> > comparison > >>> >> > between the two). > >>> >> > > >>> >> > Using StringRepresentation/ FileRepresentation with media type > >>> >> > "application/rdf+xml" does the same job. RDFRepresentation class > >>> >> > discussed > >>> >> > above actually creates OutputStreamRepresentation. > >>> >> > > >>> >> > Upload examples can be found here > >>> >> > > >>> >> > > >>> >> > > >>> >> > > https://ambit.svn.sourceforge.net/svnroot/ambit/branches/opentox/opentox-client/src/test/java/org/opentox/dataset/test/DatasetTest.java > >>> >> > > >>> >> > > >>> >> > > https://ambit.svn.sourceforge.net/svnroot/ambit/branches/opentox/opentox-client/src/test/java/org/opentox/dataset/test/FeatureTest.java > >>> >> > > >>> >> > The examples above are cleaned up versions of ambit code. JUnit > tests > >>> >> > of > >>> >> > ambit dataset services can be found here > >>> >> > > >>> >> > > >>> >> > > >>> >> > > https://ambit.svn.sourceforge.net/svnroot/ambit/trunk/ambit2-all/ambit2-www/src/test/java/ambit2/rest/test/dataset/DatasetsResourceTest.java > >>> >> > > >>> >> > > >>> >> > > >>> >> >> > >>> >> >> Also the example uses a RemoteTask to achieve the actual upload. > >>> >> >> Therefore I could not exactly fathom how to do the upload. > >>> >> > > >>> >> > Here is a summary > >>> >> > > >>> >> > curl -X POST -d "@filecontent" -H > "Content-type:application/rdf+xml" > >>> >> > http://datasetservice/dataset > >>> >> > > >>> >> > This returns Task object (either URI, if -H "Accept:text/uri-list" > is > >>> >> > specified or Task RDF representation if -H > >>> >> > "Accept:application/rdf+xml" > >>> >> > is > >>> >> > specified). Task RDF representation is defined in > >>> >> > http://opentox.org/api/1_1/opentox.owl > >>> >> > > >>> >> > If the task is still running you'll get result codes 201 or 202 , > >>> >> > this > >>> >> > means > >>> >> > you need to query again the returned task URI. If code 200 is > >>> >> > returned, > >>> >> > the > >>> >> > URI returned is where the result should be found. In the > examples > >>> >> > there is > >>> >> > a RemoteTask class that handles all this. > >>> >> > > >>> >> > > >>> >> > > >>> >> > > https://ambit.svn.sourceforge.net/svnroot/ambit/branches/opentox/opentox-client/src/main/java/org/opentox/task/RemoteTask.java > >>> >> >> > >>> >> >> On 3 September 2010 03:47, surajit ray <mr.surajit.ray at gmail.com > > > >>> >> >> wrote: > >>> >> >> > Hi, > >>> >> >> > > >>> >> >> > I have managed to get a dataset (RDF) made using the results of > a > >>> >> >> > model prediction run. Could you please tell me a dataset > service > >>> >> >> > where > >>> >> >> > I could upload the same ? > >>> >> > > >>> >> > For testing > >>> >> > > >>> >> > http://ambit-uni-plovdiv.bg:8080/ambit2/dataset > >>> >> > > >>> >> > For your final models/results , especially if you would like them > >>> >> > appearing > >>> >> > in ToxPredict. > >>> >> > > >>> >> > http://apps.ideaconsult.net:8080/ambit2/dataset > >>> >> > > >>> >> > > >>> >> >> > >>> >> >> Also it would be very helpful if you could > >>> >> >> > give an example of an upload request using restlets ... > >>> >> > > >>> >> > See above. > >>> >> > > >>> >> > Hope this helps, > >>> >> > Nina > >>> >> > > >>> >> > > >>> >> >> > > >>> >> >> > Thanx > >>> >> >> > Surajit > >>> >> >> > > >>> >> > > >>> >> > > >>> >> > > >>> > > >>> > >>> > > >> > >> > >> > > > > > > -- > Surajit Ray > Partner > www.rareindianart.com >
- Previous message: [OTDev] How to upload a dataset ...
- Next message: [OTDev] How to upload a dataset ...
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
More information about the Development mailing list