Wednesday, March 21, 2012

Query performance during incremental processing

We're doing some testing, and finding that query performance goes to hell while performing incremental processing on the partition (ProcessAdd). Specifically during the partition merge part of the processing, queries take from 3 to 30 times longer.

I guess it's not totally surprising that this would be the case, but can anyone confirm that this is the expected behavior? Is there anything that we can do to improve the situation?We have the same issue, same problem with Process Index as well. MS never gave us any solutions, so all processing is done off hours.|||

This is probably your best bet.
Processing operation is quite heavy. It often requires tons of memory and I/O. It also tries to aqure a lock at it's final stage, meaning your queries got to finish by then to allow processing to commit.

Many Analysis Services customers implement updates to their cubes during special period of time, usually night hours. If your applicaiton requires to keep your cubes up-to-date with recent updates you can utilize several techniques that require some carefull planning from your side.

You partition your data say by month and setup a small partition that you will be fully processing ( with last day of data). Benind the sense ProcessAdd for partition is creating a new parttion and then merges it with the original one. It is often faster to re-process small dailiy partition fully.

You can also get yourself a hiher end hardware that is going to sustain processing loads while users are querying your data.

Another solution is to have dedicated processing machine that from time-to-time is synchronized with production server.

Take a look at the project REAL for some clues on how to partition your data and implement batch updates.

Edward.
--
This posting is provided "AS IS" with no warranties, and confers no rights.

No comments:

Post a Comment