18 November, 2014

IBM Bluemix ... Abusing Autoscaling Function

For almost nine months Bionia has been trying to implement a workflow to create a service for the analysis of mass spectrometric data. While the workflow is theoretically established and has been hardcoded before, we need more processing power so that parallelization is prerogative. We have tried different workflow managment systems like Taverna, Galaxy, and KNIME, but each of them were missing an important functionality so that we were not able to implement our desired workflow.

Recently, we investigated cloud solutions and our current work on IBM Bluemix is promising in so far, that our workflow seems to be possible. The only problem that seems to exist is scalability and we solved it in the following way. One instance accepts user data and splits it into tasks that can be calculated in a small amount of time. Other instances are ready to do calculations with the data and are set to autoscale. The former instance sends the data repeatedly to the processing instances and by that forces it to autoscale so that we achieve the desired parallelization. 

Clearly, IBM Bluemix was not designed for parallel calculations, but in this way, we can achieve (force) parallel execution of multiple tasks.