Programming language Python
Part 1: Design and implement a workflow tool for managing and coordinating cloud resources to finish the workflow [login to view URL] is required to be able to launch the instances, upload the data/application/scripts and download the data after the execution. It is regarded as a client to be able to communicate with Cloud Service Provider. It should include the following functions:
• Launch the instance with proper configuration, which includes, user authentication, instance configuration, storage configuration.
• Check/track the status of the instances.
• Access the launched instances though ssh.
• Checkout the output of the execution.
Platform you can choose: You can choose to work on Amazon AWS or Google Cloud. You can refer to BEE ([login to view URL]), SLURM-GCP ([login to view URL] ), starcluster ([login to view URL]) and cfncluster ([login to view URL]) to design your workflow tool engine based on the platform you choose.
Benchmark Application: You are required to use BLAST as the benchmark application ([login to view URL]) to test your workflow tool. From the link you can find the multiple steps that you have to manually launch the containerized application and you are required to batch them in a workflow and send the “workflow” to cloud to process through the workflow tool you build.
Part 2. Integrate continuous integration software (Travis for example [login to view URL]) to the workflow tool you build (consider it as a workflow application). When the developers check in the changes to the source codes. The continuous integration will invoke a unit test, which will automatically launch instance/containers on cloud then compile and execute the examples (you need to setup examples and scripts). You choose “your application” for demonstration as well as the examples and scripts. You are expected to demonstrate the whole continuous integration process.
NOTE: It is not acceptable that you use the web interface of AWS or Google Cloud to configure, launch and run your workflow.
The project will have the following deliverables: Demo, Project Report and Program Files
• Demo: you are required to show the demo as a team .
• Project Report: generally, introduce your design and the flow chart based on what you build.
• Program Files. All files, including source code, sample output & README, should be able to access on github; If you don’t have a github account, create one and document your files well. Meanwhile, if you need to include your cloud account credentials in the source code,