How Google keeps its database ticking

case study Google is used to sifting through huge amounts of information to generate its search results, but a 12 gigabyte database proved something more of a challenge for its own financial management and planning systems.

Since 2001, Google has been using a variety of products from Hyperion to manage its internal financial systems. As the company grew rapidly following its listing in 2004, its approach of using a single database for both forward financial planning and actual financial results become troublesome.

The database grew in size to more than 12 gigabytes, and the period restructuring required to ensure accuracy could see the system, which is now used by more than 150 staff, taken offline for two hours at a stretch.

"The database that we had was already at the limit," Chris Schulze, technical program manager for Google, said during a presentation at Hyperion's Solutions 2007 conference in Orlando.

"There's an inherent conflict having both applications on the same cube," said Chad Coughlin, manager for consultancy Answerthink, which worked with Google to solve the problem.

Rather than ditching the system, Google decided to use partitioning to split the single existing databases into three components: one for the current financial planning projections, one for the actual current data from existing HR and general ledger systems, and one storing historic information.

"We wanted the user experience to be the same, even if it was more complex in the background," Coughlin said. Google also wanted to make it possible to drill down through existing information to more detailed information from its Oracle Financials system.

Using a transparent partition enabled Google to split the three systems while maintaining a single user interface, and built a series of interfaces to enable drill-down. The partitioning process commenced in November 2006, and was completed by February this year.

The forward planning and actual results databases are now each less than one gigabyte in size. The archival database remains relatively large, and can only be fully restructured on weekends as it still takes a number of hours.

Future plans for the system include shifting to Hyperion's System 9 platform and ditching the current Windows server used to run the service.

"Right now, we're on a not very powerful Windows box," Couglin said. "We definitely are wanting to go to Unix when we go to System 9."

Angus Kidman travelled to Orlando as a guest of Hyperion.

Advertisement

Talkback 12 comments

    shill posts lame press release Brad Eleven -- 28/04/07

    Angus is clearly a Hyperion sock puppet. Somehow I find it hard to believe that Google depends on a Windows machine to handle an important database, and 12GB isn't that much. If you've got trouble with 12 billion bytes in a database, you need a different product, probably a different DBA.

    12GB?? Is that a typo??? x man -- 28/04/07

    12GB?? Is that a typo??? Why would anyone have a problem with a 12GB database???

    12GB Anonymous -- 12/02/09 (in reply to #320078516)

    because this is "dense" data, this is the result of the OLAP requests which leads to the cube

    if you would have a look to the original data of the activity you'd probably get TB of data

    use kdb+ Anonymous -- 28/04/07

    www.kx.com, this is the fastest DB in the market, Google is wasting money on Hyperion

    12GB? --->12 TB? Anonymous -- 28/04/07

    12 gigs is nothing, at all!, that must be a typo i have over a terabyte consisting of several hundred thousand files and GDS(google desktop search) does just fine, im sure GOOGLE can handle 12 GB at .0001% capacity, damn reporters need to understand what their writing about.

    incorrect word Anonymous -- 30/04/07 (in reply to #320078524)

    they're (not their)

    12Gb Anonymous -- 01/05/07 (in reply to #320078524)

    Clearly, you guys discussing how "12Gb of data isn't very much" don't really know what you're talking about.

    Have you ever tried building a complex SQL query on a table even 1/20th that size? I just did a quick addition in a customer table for one of Australia's largest companies (where I'm working on a project )... the row size was 561 bytes (roughly half a K, of which there are 1024 in one Mb). So for, say a 1Gb table (1024 Mb), there would be 1024/561 x 1024 x 1024 = 1,913,978 rows.

    A complex query might involve up to 5 (say) of those tables. Even a single join between them is MULTIPLICATIVE - which means you're then considering 2,000,000 x 2,000,000 rows (4,000,000,000,000).

    Can anyone see how 12Gb of data might be a bit overwhelming for the average server now?

    Multi Users Anonymous -- 01/05/07 (in reply to #320078673)

    Not to mention the fact that there are 150 users on the system... potentially 150 of those big queries at any one time.

    Again, a somewhat different scenario to GDS ;)

    12Gig really is Nothing Lars Olson -- 24/09/07 (in reply to #320078673)

    I run a wikipedia clone on a single web server that is in a shared hosting environment and the wikipedia data is over 6gb compressed. Surely if I can run such a website on a shared server, google can manage 12GB of data?

    Database System san -- 31/05/07

    What kind of database is being used by google, any idea?

    www.indiainternet.in

    wrong word Anonymous -- 21/12/07

    1st paragraph: Became not become.

    do they me TB www.ingreferral.com -- 31/12/07

    12 gb is very small

Add your opinion

Latest Videos

Sponsored content

Power Centre - Content from our premier sponsors

Blogs

  • David Braue Forget the NBN, 100Mbps is already here
    Telstra and TransACT will shortly begin offering 100Mbps broadband to many customers. By moving early, the companies have not only raised the bar for Australia's broadband services, but thrown down a challenge to a government that now faces increased pressure to deliver the NBN as promised.
  • Array IT: Govt's cost-cutting bitch
    The government needs to stop looking at IT as a necessary evil or the place to remove costs when the Treasurer comes calling.
  • Array Can complaints on mobile content be cut?
    On 1 July this year the new Mobile Premium Services Code was introduced. It sounds like it's had a good impact, but is it enough?
  • More blogs »

Tags

Back to top

Featured