How Google keeps its database ticking

case study Google is used to sifting through huge amounts of information to generate its search results, but a 12 gigabyte database proved something more of a challenge for its own financial management and planning systems.

Since 2001, Google has been using a variety of products from Hyperion to manage its internal financial systems. As the company grew rapidly following its listing in 2004, its approach of using a single database for both forward financial planning and actual financial results become troublesome.

The database grew in size to more than 12 gigabytes, and the period restructuring required to ensure accuracy could see the system, which is now used by more than 150 staff, taken offline for two hours at a stretch.

"The database that we had was already at the limit," Chris Schulze, technical program manager for Google, said during a presentation at Hyperion's Solutions 2007 conference in Orlando.

"There's an inherent conflict having both applications on the same cube," said Chad Coughlin, manager for consultancy Answerthink, which worked with Google to solve the problem.

Rather than ditching the system, Google decided to use partitioning to split the single existing databases into three components: one for the current financial planning projections, one for the actual current data from existing HR and general ledger systems, and one storing historic information.

"We wanted the user experience to be the same, even if it was more complex in the background," Coughlin said. Google also wanted to make it possible to drill down through existing information to more detailed information from its Oracle Financials system.

Using a transparent partition enabled Google to split the three systems while maintaining a single user interface, and built a series of interfaces to enable drill-down. The partitioning process commenced in November 2006, and was completed by February this year.

The forward planning and actual results databases are now each less than one gigabyte in size. The archival database remains relatively large, and can only be fully restructured on weekends as it still takes a number of hours.

Future plans for the system include shifting to Hyperion's System 9 platform and ditching the current Windows server used to run the service.

"Right now, we're on a not very powerful Windows box," Couglin said. "We definitely are wanting to go to Unix when we go to System 9."

Angus Kidman travelled to Orlando as a guest of Hyperion.

Like this article? Click below to send it to your mobile for free!

Talkback 11 comments

  1. shill posts lame press release Brad Eleven -- 28/04/07

    Angus is clearly a Hyperion sock puppet. Somehow I find it hard to believe that Google depends on a Windows machine to handle an important database, and 12GB isn't that much. If you've got trouble with 12 billion bytes in a database, you need a different product, probably a different DBA.

  2. 12GB?? Is that a typo??? x man -- 28/04/07

    12GB?? Is that a typo??? Why would anyone have a problem with a 12GB database???

  3. use kdb+ Anonymous -- 28/04/07

    www.kx.com, this is the fastest DB in the market, Google is wasting money on Hyperion

  4. 12GB? --->12 TB? Anonymous -- 28/04/07

    12 gigs is nothing, at all!, that must be a typo i have over a terabyte consisting of several hundred thousand files and GDS(google desktop search) does just fine, im sure GOOGLE can handle 12 GB at .0001% capacity, damn reporters need to understand what their writing about.

    1. incorrect word Anonymous -- 30/04/07

      they're (not their)

    2. 12Gb Anonymous -- 01/05/07

      Clearly, you guys discussing how "12Gb of data isn't very much" don't really know what you're talking about.

      Have you ever tried building a complex SQL query on a table even 1/20th that size? I just did a quick addition in a customer table for one of Australia's largest companies (where I'm working on a project )... the row size was 561 bytes (roughly half a K, of which there are 1024 in one Mb). So for, say a 1Gb table (1024 Mb), there would be 1024/561 x 1024 x 1024 = 1,913,978 rows.

      A complex query might involve up to 5 (say) of those tables. Even a single join between them is MULTIPLICATIVE - which means you're then considering 2,000,000 x 2,000,000 rows (4,000,000,000,000).

      Can anyone see how 12Gb of data might be a bit overwhelming for the average server now?

    3. Multi Users Anonymous -- 01/05/07

      Not to mention the fact that there are 150 users on the system... potentially 150 of those big queries at any one time.

      Again, a somewhat different scenario to GDS ;)

    4. 12Gig really is Nothing Lars Olson -- 24/09/07

      I run a wikipedia clone on a single web server that is in a shared hosting environment and the wikipedia data is over 6gb compressed. Surely if I can run such a website on a shared server, google can manage 12GB of data?

  5. Database System san -- 31/05/07

    What kind of database is being used by google, any idea?

    www.indiainternet.in

  6. wrong word Anonymous -- 21/12/07

    1st paragraph: Became not become.

  7. do they me TB www.ingreferral.com -- 31/12/07

    12 gb is very small

Add your opinion


Sponsored content

Power Centre - Content from our premier sponsors

Blogs

  • Renai LeMay StartupCamp Melbourne: The review
    StartupCamp Melbourne looks to have produced just as interesting ideas as the Sydney event which immediately preceded it, but the Victorian start-ups appear to have stumbled during execution. Sydney 1, Melbourne 0.
  • Array Google should come clean on datacentres
    It's nice that Google says it has put an effort into making its datacentres more energy efficient, but the search giant's pledges won't mean much until it discloses just how many of the beasties it's actually running.
  • Array US shows what OPEL could have been
    Sprint's WiMAX roll-out in Baltimore will prove the Australian government's decision to worm its way out of the Opel WiMAX contract was a short-sighted, and ultimately damaging, political stunt that has benefited nobody.
  • More blogs »

Tags

Back to top

Featured