Faculty of Engineering


Robot picking platform and software tools

Improvement of collaborative work between robots and humans in materials handling operations is the focus for this project.

Increased use of online ordering has created an opportunity for increased automation in factories and warehouses, such as the use of robot picker assistants. To achieve this, the robot must understand commands, recognise and detect objects, have mobility to be able to fetch the object and be capable of delivery to the appropriate location.

 

Robot picking platform and software tools

Key focus areas/issues


This research focuses on two main areas:

  • Human robot interaction
  • Robot motion.

This will be done by:

  • Improvement to current functional models to incorporate:
    • speech synthesis – creating a New Zealand English voice including commonly used Maori words, demonstrating capability to customise for local speech variations.
    • speech recognition – for noisy background environments.
    • emotion – improved expression and interpretation of emotions.
  • New functional modules will be developed including:
    • vision – object and people detection and recognition, tracking, recognition of facial expressions.
    • motion – to be able to receive an instruction, locate and collect the desired object and deliver it to the appropriate place.
    • intelligent decision making – development of the cognitive capability to learn to be an effective assistant to the human operator in a specific environment.
  • System integration to ensure a robust system for commercial use incorporating existing and new functional modules.
  • Evaluation to determine effective operation and implementation.
Top

Current major developments


Significant progress has already been made in adding functions to the speech synthesis module, integrating the animated face module with commercial face-generating software to display different face models, solving synchronization between voice and sound and face display, and developing the speech recognition module based on CMU’s PocketSphinx software.

 

Key people


  • Bruce MacDonald
    Electrical and Chemical Engineering
  • Burkhard Wunsche
    Computer Science
  • Xun Xu
    Mechanical Engineering
  • Catherine Watson
    Electrical and Chemical Engineering
  • Xingyan Li
    Electrical and Computer Engineering

Contact


Bruce MacDonald
Email: b.macdonald@auckland.ac.nz 
Phone: +64 9 373 7599 extn 88157

Related publications


The following publications are from previous relevant research:

Chen, IY, Macdonald, BA, Wuensche, BC, 2008 'Markerless Augmented Reality for Robotic Helicoptor Applications', In:Gerald Sommer, Reinhard Klette (ed.), Robot Vision (Proceedings of the 2nd Workshop Robot Vision), New York, Springer Academic Publishers, p.125-138.

Chen, IY, Macdonald, B, Wuensche, BC, 2009. ‘Mixed reality simulation for mobile robots’, Proceedings of the IEEE International Conference on Robotics and Automatio (ICRA).

Wimmer, M, MacDonald, B, Jayamuni, D, Yadav, A, 2008. ‘Facial expression recognition for human-robot interaction – a prototype’. In: Gerald Sommer, Reinhard Klette (ed), Robot Vision, (Proceedings of the 2nd Workshop Robot Vision), New York, Springer Academic Publishers, pp 139-152.

Macdonald, BA., Biggs, GM, Collett, THJ, 2007.'Software environments for robot programming', In:Davide Brugali (ed.), Software Engineering for Experimental Robotics, vol. 30 of Springer Tracts in Advanced Robotics., Berlin/Heidelberg, Springer, p.107-124.

Canas, JM, Matellan, V, Macdonald, BA, Biggs, GM, 2007. 'Sidebar - programming commercial robots', In:Davide Brugali (ed.), Software Engineering for Experimental Robotics, vol. 30 of Springer Tracts in Advanced Robotics., Berlin/Heidelberg, Springer, p.125-132.

Brugali, D, Agah, A, Macdonald, BA, Nesnas, I, Smart, WD, 2007. 'Trends in robot software domain engineering', In:Davide Brugali (ed.), Software Engineering for Experimental Robotics, vol. 30 of Springer Tracts in Advanced Robotics., Berlin/Heidelberg, Springer, p.3-8.

Yuen, DCK, Macdonald, BA, 2001. ‘Robot localisation using an omnidirectional colour image', In:Reinhard Klette, Shmuel Peleg, and Gerald Sommer (ed.), Lecture Notes in Computer Science., Springer-Verlag, p.167-175.
 

Top