Cutting commutes with data mining
Sydney drivers on the notoriously congested M4 tollway could soon shorten their commutes courtesy of a recently completed data mining competition coordinated by local start-up Kaggle.
The competition was to produce an algorithm to forecast travel times on the M4 for intervals ranging from 15 minutes to 24 hours with the greatest accuracy. It was contested by 364 teams from the Kaggle community — consisting of professional data analysts, statisticians and students from around the world — who used over two years worth of historical data from the NSW roads and traffic authority.
The algorithms were ranked based on the "root mean squared error" (RMSE), a measure of the gap between predicted travel times and actual travel times. The lower the RMSE, the smaller the average error is.
The $10,000 prize was taken out by Jose Gonzalez-Brenes (Carnegie Mellon University) and Guido Matias Cortes (University of British Columbia), two Costa Rican-born students based in North America whose algorithm will power the RTA live-traffic website and inform M4 management and operations, according to Kaggle founder Anthony Goldbloom.
"If you're trying to work out when to leave work, you can jump on the site and get predictions for how long it'll take to get home if you leave now, in 15 minutes, 30 minutes, 45 minutes, etc," Goldbloom said. "The hope is that people will be able to make more informed decisions on when to drive home, which might smooth traffic across peak times."
It's the first competition Kaggle has hosted for government, but chairman Nick Gruen, who led the Federal Government's Government 2.0 taskforce, said it won't be the last, since discussions are underway with other agencies to use data-mining competitions to improve public services.
"NSW's chief scientist Mary O'Kane is a big fan of Kaggle's and has championed us through the bureaucracy," he said in an email interview. "I've addressed their heads of departments on Kaggle and ... there is genuine interest from several."
"In one discussion we had with a NSW department, they brought their data analyst to a meeting — but it turned out she was working for us as well! She was a competitor on Kaggle!"