It's not what you say, but how you say it
In the future it might only take an internet connection, a microphone and a computer for a company to determine whether or not you are suitable for a job as an air traffic controller.
Using a combination of speech processing software and an understanding of how speech patterns change when a person is under stress, job applicants could be screened on how well they can work in high pressure situations.
This is the idea that underpins NICTA's BrainGauge product, which applies apeech analysis and cognitive assessment.
"In speech you can detect the mental effort people put in to do a task," NICTA project leader Dr Fang Chen told ZDNet Australia.
"Often if you're being asked a tough question you start to have long delays in your responses and have lots of 'ahs' and 'ums' to delay your speech. These are all symptoms of you running out of brain space because we have to use a very limited working memory to concentrate on a task, which can be quite difficult and challenging."
By using cognitive assessment through speech analysis, Chen and her team can measure how much difficulty or how much cognitive load is being used and how much capacity you still have in doing a task. BrainGauge uses a person's own voice, speaking in low stress situations, as a reference point to assess cognitive load.
"We've done a few studies with airport flight controllers. While they're talking they can sound calm and friendly, but we wanted to know if there were indicators in their speech that came out when they were overloaded with flights and questions," she said.
According to Chen, BrainGauge could have a range of applications, from defence all the way to video games. The product has made its way around the world and is currently being used in Canada by Ottawa's Emergency Services. In Australia, the technology is being used by the NSW Roads and Traffic Authority and in defence situations, although she wouldn't provide further details.
The technology has already found a way to enter the mass market as a form of quality assurance for call centre staff.
"This technology can help us tap into a telephone or a computer, and as long as we catch the agent's speech, we can provide real-time 24-hour monitoring. So, a supervisor can look at the dashboard and say 'oh this guy has got a lot of red bars' and have a review over the calls," she said.
Call centres, according to Chen, are conventionally high-stress jobs with a high rate of staff turnover. BrainGauge can help centres with recruitment.
In a field test, agents are given activities, such as number memorisation.
"We found that results the agents score on these tests highly correlated with their [on the job] performance and their attrition rate," said Chen, saying that the information allowed centres to find the applicants who have the skills and disposition to perform in the high-stress job.
Chen believes that the technology could also be placed onto iPhones for people to monitor their own stress levels. A prototype app was created in 2008, which uses basic speech modelling, and the company plans to roll it out to the mass market in the near future.
Speech is only the first in a wide range of areas to be explored using this technology. Human handwriting and eye movement details are also currently being researched to explore more robust and accurate measurements on mental effort.