Dear all,

I am posting this suggestion as a follow-up to Vedrins request for a testing and release plan on the partners list.

I think we should make a clear distinction between distributed webservices and local installations. In the initial phase we should focus on providing reliable webservices. At a later point of time, we can bundle OT components for some example platforms and release them as local applications.

1. Webservices

For webservices I propose a test-driven release policy. High-level test should be fairly easy to implement, because all we have to know is the REST request and the expected response (and we can even reuse this as examples in the API documentation).

It assumes that we have a release-management component, that provides

	- dependency tracking for all OT components
	- automated testing facilities
	- maybe API documentation and bug tracker (they could also come from another component/application)

Each version of an OpenTox component will run through the following stages:

	- Developers code and test in their local environment.
	- As soon as the application passes all local tests, they can deploy it as a public webservice and submit to the release-management component
			- the REST API 
			- API tests (REST requests/expected output)
			- dependencies on other components (maybe also as tests)

	The release-management component
		- publishes the new version and its API
 		- runs the tests for the new version (which will hopefully succeed)
		- informs the developer about failed tests
		- informs developers of components utilizing the webservice about the update

	As more and more components/users access the webservice, additional bugs will be detected. Ideally they should be entered as tests (REST request/expected response/actual response) into the bug tracker. (developer may need to help if the bug tracker has been used by an inexperienced user).

	The release-management component
		- pulls new tests from the bug-tracker
		- informs developers about new bugs (can be delegated to the bug tracker)
		- tests incoming bug fixes

	- Developers fix bugs, merge bug fixes into other branches and redeploy their webservice.

This strategy should increase the number of test cases (and thus the reliability of components) over time, without putting too much burden on individual developers to create tests for all possible (mis)uses of their component.

Versioning and dependencies:

I think that it is impractical to keep all versions of OT webservices that have been published alive (server resources, maintenance efforts, ...). To cut down the number of supported webservices I suggest to

	- to redeploy bug fixes and refactored versions under the same URI (no API and response changes - except for bugs)
	- to assign new URIs to: API changes, algorithm changes, feature additions (tests of current version will fail for previous version)
	- use dependency trees to identify webservice versions, that are not needed anymore

2. Local Installations

I think, that providing well tested OT components for all possible architecture/operating system/version/distribution combinations is near to impossible, even if we restrict ourselves to a few examples. Instead I would suggest to start installing example OT applications (i.e. webservices + interface) on single machines. This would involve:

	- identification of required webservices (through dependency trees)
	- identification of required libraries (through dependency trees)
	- creation of (platform independent) installation procedures for all required components
	- installing webservices locally
	- resolving versioning problems (e.g. if components depend on different system library versions)
	- local integration testing

Providing reliable cross-platform installation scripts can be in my experience very annoying, time consuming and error prone (especially with library versioning problems). For this reason I would leave the decision of providing local installations to the individual application (GUI) programmer - it could be also an excellent business opportunity to sell this as a service.