Faculty of Engineering

Building intelligent computing clouds

The process of managing the underlying hardware/software infrastructure behind a computing cloud is known as cloud orchestration.   This research involves the development of a set of modules for cloud orchestration that utilise sophisticated mathematical algorithms to improve the efficiency of the cloud infrastructure. These modules will be able to plug into any cloud operating system or distributed system and can be customised, for example, to increase performance, to reduce power consumption or to reduce infrastructure costs.


Clouds are no longer the future of ICT - they are the present, with cloud computing already being used throughout the world. The benefits of cloud computing are well documented and include improved agility, better reliability and lower power consumption. Forecasts of cloud computing investment predict that $85 billion will be spent on cloud computing over the next 5 years. In addition to public cloud offerings from Google, Amazon and others, private clouds are also being deployed within organisations looking to streamline their IT systems.


Key focus areas/issues

New modules currently being developed are:

  1. Sensory – measures and communicates information about the system, key factors here include passive measurement and low overhead communication.
  2. Predictive – uses sensory information to predict future levels of, for example, workload, power consumption, storage requirements.
  3. Adaptive – uses Sensory and Predictive information to adapt the orchestration of the current infrastructure, eg, reconfigures the network routing to improve performance, moves VMs to conserve power, a key factor here is solution speed.
  4. Evolving – uses Sensory and Predictive information to evolve the infrastructure, eg, adds new technology, remove old technology, redeploys the infrastructure components.

Existing modules we will integrate with are:

  1. Storage – there are many distributed storage systems appropriate for use in cloud storage:
    • Swift (part of OpenStack – an open source cloud operating system) – distributed, object based system offering block, file and object storage.
    • Ceph – distributed, object based system offering block, file and object storage.
    • Ibrix (Hewlett Packard) – distributed, tiered, file-based system.
    • GPFS (IBM) – distributed, tiered system offering file storage.
    • Gluster.
    • IRODS.
  2. Compute – this component schedules jobs on the cloud. There are several platforms currently available including:
    • Nova (part of OpenStack).
    • Eucalyptus.
  3. Image – this component manages VM images, including base images for standard VMs and specialised image for particular images.

Current major developments

  • We are testing a Ceph system within The University of Auckland’s data centre to investigate its possible inclusion as part of the University’s data storage infrastructure.
  • We are developing APIs for our new modules within OpenStack and Ceph.
  • We are investigating how to utilise OpenFlow alongside our cloud orchestration modules.


Key achievements


We have implemented a prototype cloud infrastructure using OpenStack and compared OpenStack’s storage platform (Swift) with Ceph (which can also be embedded within OpenStack), including some testing with VMs.

We recently presented our first optimisation model for designing cloud infrastructure (specifically storage tiers) at the Symposium for Applied Computing (2011).

Click the image to the left to view an enlargement of the underlying graph theory model for designing storage nodes (within a storage tier).  The generic graph structure is shown underneath the automatically generated optimal design with the resources provided/required by individual components within the storage node.


Key people



Cameron Walker
Email: c.walker@auckland.ac.nz
Phone:  +64 9 373 7599 ext 87009

Michael O’Sullivan
Email: michael.osullivan@auckland.ac.nz
Phone: +64 9 373 7599 ext 87907

Related publications

O'Sullivan MJ, Walker CG, Lee D., 2012. Designing Data Storage Tier using Integer Programing, 27th Symposium On Applied Computing, Trento, Italy, 26 Mar 2012 - 30 Mar 2012. Proceedings of 27th Symposium On Applied Computing. ACM. 8 pages.

Lee, D., O'Sullivan, M. J., Walker, C. G., & MacKenzie, M. L. , 2011. Robust Benchmarking for Archival Storage Tiers. In Proceedings of the Sixth workshop on Parallel Data Storage (pp. 6 pages). Seattle, USA.

Lee, D., O'Sullivan, M., & Walker, C., 2011. Measurement for improving the design of commodity archival storage tiers. Proceedings - 2011 4th IEEE International Conference on Utility and Cloud Computing, 275-280.

Lee, D., O'Sullivan, M., & Walker, C., 2011. Benchmarking and modeling disk-based storage tiers for practical storage design. In PMBS'11 - Proceedings of the 2nd International Workshop on Performance Modeling, Benchmarking and Simulation of High Performance Computing Systems, Co-located with SC'11 (pp. 21-22). Seattle, WA, USA: ACM. doi:10.1145/2088457.2088472

Lee D., O’Sullivan M. J., Walker C. G., 2010. Practical Measurement of Typical Disk Performance and Power Consumption using Open Source SPC-1, Annual International Conference on Green Information Technology – GreenIT, Singapore, Oct 25-26.

Walker, C.G., O'Sullivan, M.J., 2010. Core-Edge Design of Storage Area Networks - a Single-edge formulation with problem-specific cuts, Computers and Operations Research, 37(5), pp 916-926.

Walker, C.G., O'Sullivan, M.J., Thompson, T.D., 2009. 'A Mixed-Integer Approach to Core-Edge Design of Storage Area Networks', Computers and Operations Research, 34, (10), p2976-3000

O'Sullivan, M. J., & Walker, C. G., 2009. Bi-Criterion Design of Core-Edge Storage Area Networks. In 2009 International Conference on Telecommunications Systems - Modeling and Analysis (pp. 47-71). Monterey.
O'Sullivan, M. J., Shahoumian, T., & Ward, J. M. (2009). US 7502839, Module-building method for designing interconnect fabrics. United States. Retrieved from http://www.patentlens.net/patentlens/patent/US_7502839/en/

Brownlee, J. N., Halytskyy, Y., Jones, N., Kharuk, A. F., O'Sullivan, M. J., Walker, C. G., . . . Ziedins, I., 2009. Storage Network Planning for KAREN/BeSTGRID.

O’Sullivan M.J., Walker C.G., O’Sullivan M.L., Thompson T.D. and Philpott A.B., 2006. Protecting local access telecommunications networks: Toward a minimum-cost solution. Telecommunication Systems Volume 33, pp 353-376.

O’Sullivan M.J., Walker C.G., 2005. A Mixed-integer Approach to Storage Area Network Design using Generic Network Components, School of Engineering Technical Report No. 626. Department of Engineering Science, School of Engineering, University of Auckland, pp1-27.

Walker C.G., O’Sullivan M.J., Elangasinghe M, 2005. Evaluation of Core-Edge Storage Area Network Designs using Simulation, School of Engineering Technical Report No. 627. Department of Engineering Science, School of Engineering, University of Auckland, pp1-24.