Friday, March 30, 2012
Query Processor Error after SP4
We have a database with approx. 460 tables linked to a foreign key (UserID)
in a table called Users. After installing SQL SP4, when trying to delete an
unreferenced entry in the Users table (even using Enterprise Manager), we get
the following error: "[Microsoft][ODBC SQL Server Driver][SQL Server]Internal
Query Processor Error: The query processor encountered an unexpected error
during execution."
Using the same database on a SP3a SQL server, it works fine.
Thanks in advance
Johan Fourie
Hi,
Johan Fourie,
Did u restarted ur sql services after installation of sp4.
hope this help
from
killer
|||Hi Johan,
Thanks for your post.
From your descriptions, I understood you SELECT statements will report the
error message after applied SP4. If I have misunderstood your concern,
please feel free to point it out.
Based on my knowledge, SQL Server SP4 should fix a related error message.
Please refer the KB for more detailed information.
FIX: Internal Query Processor Error 8623 When Microsoft SQL Server Tries to
Compile a Plan for a Complex Query
http://support.microsoft.com/kb/818729
Is it possible for you to reproduce it on Northwind database? Would you
please generate a sample table, with which I could reproduct it on my side?
Any more error logs on your side? I understand the information may be
sensitive to you, my direct email address is v-mingqc@.online.microsoft.com
(please remember to remove "online" before click SEND as it's only for
SPAM), you may send the sample script file to me directly and I will keep
secure.
Thank you for your patience and cooperation. If you have any questions or
concerns, don't hesitate to let me know. We are always here to be of
assistance!
Sincerely yours,
Michael Cheng
Microsoft Online Partner Support
When responding to posts, please "Reply to Group" via your newsreader so
that others may learn and benefit from your issue.
================================================== ===
This posting is provided "AS IS" with no warranties, and confers no rights.
|||Yes, we did many reboots since the upgrade to SP4
Thanks
Johan Fourie
"doller" wrote:
> Hi,
> Johan Fourie,
> Did u restarted ur sql services after installation of sp4.
> hope this help
> from
> killer
>
|||Hi Michael
Ok, from SQL Query Analyzer the following command "DELETE FROM Users WHERE
UserID = 19" returns the following error: "Server: Msg 8630, Level 17, State
34, Line 1
Internal Query Processor Error: The query processor encountered an
unexpected error during execution."
I do not think it would be possible to simulate this problem on the
Northwind database as you need many FK links to the affected table. As you
can see, it is actually not a complex query, but SQL must determine whether
the entry being deleted is in use by one of the FK relationships (>460)
Thanks and much appreciated
Johan Fourie
"Michael Cheng [MSFT]" wrote:
> Hi Johan,
> Thanks for your post.
> From your descriptions, I understood you SELECT statements will report the
> error message after applied SP4. If I have misunderstood your concern,
> please feel free to point it out.
> Based on my knowledge, SQL Server SP4 should fix a related error message.
> Please refer the KB for more detailed information.
> FIX: Internal Query Processor Error 8623 When Microsoft SQL Server Tries to
> Compile a Plan for a Complex Query
> http://support.microsoft.com/kb/818729
> Is it possible for you to reproduce it on Northwind database? Would you
> please generate a sample table, with which I could reproduct it on my side?
> Any more error logs on your side? I understand the information may be
> sensitive to you, my direct email address is v-mingqc@.online.microsoft.com
> (please remember to remove "online" before click SEND as it's only for
> SPAM), you may send the sample script file to me directly and I will keep
> secure.
> Thank you for your patience and cooperation. If you have any questions or
> concerns, don't hesitate to let me know. We are always here to be of
> assistance!
>
> Sincerely yours,
> Michael Cheng
> Microsoft Online Partner Support
> When responding to posts, please "Reply to Group" via your newsreader so
> that others may learn and benefit from your issue.
> ================================================== ===
> This posting is provided "AS IS" with no warranties, and confers no rights.
>
|||Hi Johan,
Thanks for the quick response.
It is very strange the SP4 enroll this new error message. I would like to
check it further. Is it possible for you to send me sample tables or sample
databases? (mdb file or the scripts with your tables)
I understand the information must be sensitive to you. Alternatively, to
efficiently troubleshoot a this issue, we recommend that you contact
Microsoft Customer Service and Support and open a support incident and work
with a dedicated Support Professional.
Please be advised that contacting phone support will be a charged call.
However, if it turn out to be a bug in SP4 then charges are usually
refunded or waived.
To obtain the phone numbers for specific technology request please take a
look at the web site listed below.
http://support.microsoft.com/default...S;PHONENUMBERS
If you are outside the US please see http://support.microsoft.com for
regional support phone numbers.
Sincerely yours,
Michael Cheng
Microsoft Online Partner Support
When responding to posts, please "Reply to Group" via your newsreader so
that others may learn and benefit from your issue.
================================================== ===
This posting is provided "AS IS" with no warranties, and confers no rights.
|||Hi Michael
Thanks for the quick response. We took the database again to a SP3a SQL
server and it works fine there, must be a SP4 problem. I'll be sending you
the database via e-mail shortly. Please treat the data and structure of the
database as very sensitive. Hope to hear from you soon.
Regards and many thanks.
Johan Fourie
"Michael Cheng [MSFT]" wrote:
> Hi Johan,
> Thanks for the quick response.
> It is very strange the SP4 enroll this new error message. I would like to
> check it further. Is it possible for you to send me sample tables or sample
> databases? (mdb file or the scripts with your tables)
> I understand the information must be sensitive to you. Alternatively, to
> efficiently troubleshoot a this issue, we recommend that you contact
> Microsoft Customer Service and Support and open a support incident and work
> with a dedicated Support Professional.
> Please be advised that contacting phone support will be a charged call.
> However, if it turn out to be a bug in SP4 then charges are usually
> refunded or waived.
> To obtain the phone numbers for specific technology request please take a
> look at the web site listed below.
> http://support.microsoft.com/default...S;PHONENUMBERS
> If you are outside the US please see http://support.microsoft.com for
> regional support phone numbers.
>
> Sincerely yours,
> Michael Cheng
> Microsoft Online Partner Support
> When responding to posts, please "Reply to Group" via your newsreader so
> that others may learn and benefit from your issue.
> ================================================== ===
> This posting is provided "AS IS" with no warranties, and confers no rights.
>
|||Hi Johan,
Thanks for your kindly sending me the database files.
After looking into the execution plan, I believe this is a known issue of
SQL Server service pack 4.
Since we add stack overflow checks in SP4, it raise this new problem. For
now, we do not have a pretty workaround for this issue.
However, if you want to apply SP4 and this issue has big business impact to
you. I would like to recommand you opening a grace case with Microsoft
Customer Service and Support (CSS). Please understand that we could not
share you the QFE request in the newsgroup.
Please be advised that contacting phone support will be a charged call.
However, if the support engineer confirmed this is a bug and no other
support then charges are usually refunded or waived.
To obtain the phone numbers for specific technology request please take a
look at the web site listed below.
http://support.microsoft.com/default...S;PHONENUMBERS
If you are outside the US please see http://support.microsoft.com for
regional support phone numbers.
You could share the information below with the CSS support engineer
Bug# SQL Server 8.0 20000179
I do apologized for this new known issue that caused to you and thanks so
much for your patience and cooperation. If you have any questions or
concerns, don't hesitate to let me know. We are always here to be of
assistance!
Sincerely yours,
Michael Cheng
Microsoft Online Partner Support
When responding to posts, please "Reply to Group" via your newsreader so
that others may learn and benefit from your issue.
================================================== ===
This posting is provided "AS IS" with no warranties, and confers no rights.
|||Hi Michael
Thanks. Is there a quick fix available for this problem? We've advised all
our customers to go to SP4 (when it was released) and they wont be happy if
we tell them to go back to SP3a (as they all have this problem). Can you also
supply us with the KB number for this problem. Would it be possible for you
to send the QFE via e-mail to me?
Thanks in advance.
Johan Fourie
"Michael Cheng [MSFT]" wrote:
> Hi Johan,
> Thanks for your kindly sending me the database files.
> After looking into the execution plan, I believe this is a known issue of
> SQL Server service pack 4.
> Since we add stack overflow checks in SP4, it raise this new problem. For
> now, we do not have a pretty workaround for this issue.
> However, if you want to apply SP4 and this issue has big business impact to
> you. I would like to recommand you opening a grace case with Microsoft
> Customer Service and Support (CSS). Please understand that we could not
> share you the QFE request in the newsgroup.
> Please be advised that contacting phone support will be a charged call.
> However, if the support engineer confirmed this is a bug and no other
> support then charges are usually refunded or waived.
> To obtain the phone numbers for specific technology request please take a
> look at the web site listed below.
> http://support.microsoft.com/default...S;PHONENUMBERS
> If you are outside the US please see http://support.microsoft.com for
> regional support phone numbers.
> You could share the information below with the CSS support engineer
> Bug# SQL Server 8.0 20000179
> I do apologized for this new known issue that caused to you and thanks so
> much for your patience and cooperation. If you have any questions or
> concerns, don't hesitate to let me know. We are always here to be of
> assistance!
>
> Sincerely yours,
> Michael Cheng
> Microsoft Online Partner Support
> When responding to posts, please "Reply to Group" via your newsreader so
> that others may learn and benefit from your issue.
> ================================================== ===
> This posting is provided "AS IS" with no warranties, and confers no rights.
>
|||Hi Johan,
Thanks for your update.
Unfortunately, I am so sorry to say that there is temporarily no public
Knowledge Base articles and document describing this issue. This issue only
occurs in SQL Server 2000 SP4, SQL Server 2000 SP3 and SQL Server 2005 do
not have this issue.
Unfortunately, I am unable to process the delivery of QFEs via email
directly. However, you should be able to quickly get this QFE by opening
the grace case with CSS.
Thank you for your patience and cooperation. If you have any questions or
concerns, don't hesitate to let me know. We are always here to be of
assistance!
Sincerely yours,
Michael Cheng
Microsoft Online Partner Support
When responding to posts, please "Reply to Group" via your newsreader so
that others may learn and benefit from your issue.
================================================== ===
This posting is provided "AS IS" with no warranties, and confers no rights.
Query Processor Error after SP4
We have a database with approx. 460 tables linked to a foreign key (UserID)
in a table called Users. After installing SQL SP4, when trying to delete an
unreferenced entry in the Users table (even using Enterprise Manager), we ge
t
the following error: "[Microsoft][ODBC SQL Server Driver][SQL Se
rver]Internal
Query Processor Error: The query processor encountered an unexpected error
during execution."
Using the same database on a SP3a SQL server, it works fine.
Thanks in advance
--
Johan FourieHi,
Johan Fourie,
Did u restarted ur sql services after installation of sp4.
hope this help
from
killer|||Hi Johan,
Thanks for your post.
From your descriptions, I understood you SELECT statements will report the
error message after applied SP4. If I have misunderstood your concern,
please feel free to point it out.
Based on my knowledge, SQL Server SP4 should fix a related error message.
Please refer the KB for more detailed information.
FIX: Internal Query Processor Error 8623 When Microsoft SQL Server Tries to
Compile a Plan for a Complex Query
http://support.microsoft.com/kb/818729
Is it possible for you to reproduce it on Northwind database? Would you
please generate a sample table, with which I could reproduct it on my side?
Any more error logs on your side? I understand the information may be
sensitive to you, my direct email address is v-mingqc@.online.microsoft.com
(please remember to remove "online" before click SEND as it's only for
SPAM), you may send the sample script file to me directly and I will keep
secure.
Thank you for your patience and cooperation. If you have any questions or
concerns, don't hesitate to let me know. We are always here to be of
assistance!
Sincerely yours,
Michael Cheng
Microsoft Online Partner Support
When responding to posts, please "Reply to Group" via your newsreader so
that others may learn and benefit from your issue.
========================================
=============
This posting is provided "AS IS" with no warranties, and confers no rights.|||Yes, we did many reboots since the upgrade to SP4
Thanks
--
Johan Fourie
"doller" wrote:
> Hi,
> Johan Fourie,
> Did u restarted ur sql services after installation of sp4.
> hope this help
> from
> killer
>|||Hi Michael
Ok, from SQL Query Analyzer the following command "DELETE FROM Users WHERE
UserID = 19" returns the following error: "Server: Msg 8630, Level 17, State
34, Line 1
Internal Query Processor Error: The query processor encountered an
unexpected error during execution."
I do not think it would be possible to simulate this problem on the
Northwind database as you need many FK links to the affected table. As you
can see, it is actually not a complex query, but SQL must determine whether
the entry being deleted is in use by one of the FK relationships (>460)
Thanks and much appreciated
--
Johan Fourie
"Michael Cheng [MSFT]" wrote:
> Hi Johan,
> Thanks for your post.
> From your descriptions, I understood you SELECT statements will report the
> error message after applied SP4. If I have misunderstood your concern,
> please feel free to point it out.
> Based on my knowledge, SQL Server SP4 should fix a related error message.
> Please refer the KB for more detailed information.
> FIX: Internal Query Processor Error 8623 When Microsoft SQL Server Tries t
o
> Compile a Plan for a Complex Query
> http://support.microsoft.com/kb/818729
> Is it possible for you to reproduce it on Northwind database? Would you
> please generate a sample table, with which I could reproduct it on my side
?
> Any more error logs on your side? I understand the information may be
> sensitive to you, my direct email address is v-mingqc@.online.microsoft.com
> (please remember to remove "online" before click SEND as it's only for
> SPAM), you may send the sample script file to me directly and I will keep
> secure.
> Thank you for your patience and cooperation. If you have any questions or
> concerns, don't hesitate to let me know. We are always here to be of
> assistance!
>
> Sincerely yours,
> Michael Cheng
> Microsoft Online Partner Support
> When responding to posts, please "Reply to Group" via your newsreader so
> that others may learn and benefit from your issue.
> ========================================
=============
> This posting is provided "AS IS" with no warranties, and confers no rights
.
>|||Hi Johan,
Thanks for the quick response.
It is very strange the SP4 enroll this new error message. I would like to
check it further. Is it possible for you to send me sample tables or sample
databases? (mdb file or the scripts with your tables)
I understand the information must be sensitive to you. Alternatively, to
efficiently troubleshoot a this issue, we recommend that you contact
Microsoft Customer Service and Support and open a support incident and work
with a dedicated Support Professional.
Please be advised that contacting phone support will be a charged call.
However, if it turn out to be a bug in SP4 then charges are usually
refunded or waived.
To obtain the phone numbers for specific technology request please take a
look at the web site listed below.
http://support.microsoft.com/defaul...US;PHONENUMBERS
If you are outside the US please see http://support.microsoft.com for
regional support phone numbers.
Sincerely yours,
Michael Cheng
Microsoft Online Partner Support
When responding to posts, please "Reply to Group" via your newsreader so
that others may learn and benefit from your issue.
========================================
=============
This posting is provided "AS IS" with no warranties, and confers no rights.|||Hi Michael
Thanks for the quick response. We took the database again to a SP3a SQL
server and it works fine there, must be a SP4 problem. I'll be sending you
the database via e-mail shortly. Please treat the data and structure of the
database as very sensitive. Hope to hear from you soon.
Regards and many thanks.
--
Johan Fourie
"Michael Cheng [MSFT]" wrote:
> Hi Johan,
> Thanks for the quick response.
> It is very strange the SP4 enroll this new error message. I would like to
> check it further. Is it possible for you to send me sample tables or sampl
e
> databases? (mdb file or the scripts with your tables)
> I understand the information must be sensitive to you. Alternatively, to
> efficiently troubleshoot a this issue, we recommend that you contact
> Microsoft Customer Service and Support and open a support incident and wor
k
> with a dedicated Support Professional.
> Please be advised that contacting phone support will be a charged call.
> However, if it turn out to be a bug in SP4 then charges are usually
> refunded or waived.
> To obtain the phone numbers for specific technology request please take a
> look at the web site listed below.
> http://support.microsoft.com/defaul...US;PHONENUMBERS
> If you are outside the US please see http://support.microsoft.com for
> regional support phone numbers.
>
> Sincerely yours,
> Michael Cheng
> Microsoft Online Partner Support
> When responding to posts, please "Reply to Group" via your newsreader so
> that others may learn and benefit from your issue.
> ========================================
=============
> This posting is provided "AS IS" with no warranties, and confers no rights
.
>|||Hi Johan,
Thanks for your kindly sending me the database files.
After looking into the execution plan, I believe this is a known issue of
SQL Server service pack 4.
Since we add stack overflow checks in SP4, it raise this new problem. For
now, we do not have a pretty workaround for this issue.
However, if you want to apply SP4 and this issue has big business impact to
you. I would like to recommand you opening a grace case with Microsoft
Customer Service and Support (CSS). Please understand that we could not
share you the QFE request in the newsgroup.
Please be advised that contacting phone support will be a charged call.
However, if the support engineer confirmed this is a bug and no other
support then charges are usually refunded or waived.
To obtain the phone numbers for specific technology request please take a
look at the web site listed below.
http://support.microsoft.com/defaul...US;PHONENUMBERS
If you are outside the US please see http://support.microsoft.com for
regional support phone numbers.
You could share the information below with the CSS support engineer
Bug# SQL Server 8.0 20000179
I do apologized for this new known issue that caused to you and thanks so
much for your patience and cooperation. If you have any questions or
concerns, don't hesitate to let me know. We are always here to be of
assistance!
Sincerely yours,
Michael Cheng
Microsoft Online Partner Support
When responding to posts, please "Reply to Group" via your newsreader so
that others may learn and benefit from your issue.
========================================
=============
This posting is provided "AS IS" with no warranties, and confers no rights.|||Hi Michael
Thanks. Is there a quick fix available for this problem? We've advised all
our customers to go to SP4 (when it was released) and they wont be happy if
we tell them to go back to SP3a (as they all have this problem). Can you als
o
supply us with the KB number for this problem. Would it be possible for you
to send the QFE via e-mail to me?
Thanks in advance.
--
Johan Fourie
"Michael Cheng [MSFT]" wrote:
> Hi Johan,
> Thanks for your kindly sending me the database files.
> After looking into the execution plan, I believe this is a known issue of
> SQL Server service pack 4.
> Since we add stack overflow checks in SP4, it raise this new problem. For
> now, we do not have a pretty workaround for this issue.
> However, if you want to apply SP4 and this issue has big business impact t
o
> you. I would like to recommand you opening a grace case with Microsoft
> Customer Service and Support (CSS). Please understand that we could not
> share you the QFE request in the newsgroup.
> Please be advised that contacting phone support will be a charged call.
> However, if the support engineer confirmed this is a bug and no other
> support then charges are usually refunded or waived.
> To obtain the phone numbers for specific technology request please take a
> look at the web site listed below.
> http://support.microsoft.com/defaul...US;PHONENUMBERS
> If you are outside the US please see http://support.microsoft.com for
> regional support phone numbers.
> You could share the information below with the CSS support engineer
> Bug# SQL Server 8.0 20000179
> I do apologized for this new known issue that caused to you and thanks so
> much for your patience and cooperation. If you have any questions or
> concerns, don't hesitate to let me know. We are always here to be of
> assistance!
>
> Sincerely yours,
> Michael Cheng
> Microsoft Online Partner Support
> When responding to posts, please "Reply to Group" via your newsreader so
> that others may learn and benefit from your issue.
> ========================================
=============
> This posting is provided "AS IS" with no warranties, and confers no rights
.
>|||Hi Johan,
Thanks for your update.
Unfortunately, I am so sorry to say that there is temporarily no public
Knowledge Base articles and document describing this issue. This issue only
occurs in SQL Server 2000 SP4, SQL Server 2000 SP3 and SQL Server 2005 do
not have this issue.
Unfortunately, I am unable to process the delivery of QFEs via email
directly. However, you should be able to quickly get this QFE by opening
the grace case with CSS.
Thank you for your patience and cooperation. If you have any questions or
concerns, don't hesitate to let me know. We are always here to be of
assistance!
Sincerely yours,
Michael Cheng
Microsoft Online Partner Support
When responding to posts, please "Reply to Group" via your newsreader so
that others may learn and benefit from your issue.
========================================
=============
This posting is provided "AS IS" with no warranties, and confers no rights.
Query Processor Error after SP4
We have a database with approx. 460 tables linked to a foreign key (UserID)
in a table called Users. After installing SQL SP4, when trying to delete an
unreferenced entry in the Users table (even using Enterprise Manager), we get
the following error: "[Microsoft][ODBC SQL Server Driver][SQL Server]Internal
Query Processor Error: The query processor encountered an unexpected error
during execution."
Using the same database on a SP3a SQL server, it works fine.
Thanks in advance
--
Johan FourieHi,
Johan Fourie,
Did u restarted ur sql services after installation of sp4.
hope this help
from
killer|||Hi Johan,
Thanks for your post.
From your descriptions, I understood you SELECT statements will report the
error message after applied SP4. If I have misunderstood your concern,
please feel free to point it out.
Based on my knowledge, SQL Server SP4 should fix a related error message.
Please refer the KB for more detailed information.
FIX: Internal Query Processor Error 8623 When Microsoft SQL Server Tries to
Compile a Plan for a Complex Query
http://support.microsoft.com/kb/818729
Is it possible for you to reproduce it on Northwind database? Would you
please generate a sample table, with which I could reproduct it on my side?
Any more error logs on your side? I understand the information may be
sensitive to you, my direct email address is v-mingqc@.online.microsoft.com
(please remember to remove "online" before click SEND as it's only for
SPAM), you may send the sample script file to me directly and I will keep
secure.
Thank you for your patience and cooperation. If you have any questions or
concerns, don't hesitate to let me know. We are always here to be of
assistance!
Sincerely yours,
Michael Cheng
Microsoft Online Partner Support
When responding to posts, please "Reply to Group" via your newsreader so
that others may learn and benefit from your issue.
=====================================================
This posting is provided "AS IS" with no warranties, and confers no rights.|||Yes, we did many reboots since the upgrade to SP4
Thanks
--
Johan Fourie
"doller" wrote:
> Hi,
> Johan Fourie,
> Did u restarted ur sql services after installation of sp4.
> hope this help
> from
> killer
>|||Hi Michael
Ok, from SQL Query Analyzer the following command "DELETE FROM Users WHERE
UserID = 19" returns the following error: "Server: Msg 8630, Level 17, State
34, Line 1
Internal Query Processor Error: The query processor encountered an
unexpected error during execution."
I do not think it would be possible to simulate this problem on the
Northwind database as you need many FK links to the affected table. As you
can see, it is actually not a complex query, but SQL must determine whether
the entry being deleted is in use by one of the FK relationships (>460)
Thanks and much appreciated
--
Johan Fourie
"Michael Cheng [MSFT]" wrote:
> Hi Johan,
> Thanks for your post.
> From your descriptions, I understood you SELECT statements will report the
> error message after applied SP4. If I have misunderstood your concern,
> please feel free to point it out.
> Based on my knowledge, SQL Server SP4 should fix a related error message.
> Please refer the KB for more detailed information.
> FIX: Internal Query Processor Error 8623 When Microsoft SQL Server Tries to
> Compile a Plan for a Complex Query
> http://support.microsoft.com/kb/818729
> Is it possible for you to reproduce it on Northwind database? Would you
> please generate a sample table, with which I could reproduct it on my side?
> Any more error logs on your side? I understand the information may be
> sensitive to you, my direct email address is v-mingqc@.online.microsoft.com
> (please remember to remove "online" before click SEND as it's only for
> SPAM), you may send the sample script file to me directly and I will keep
> secure.
> Thank you for your patience and cooperation. If you have any questions or
> concerns, don't hesitate to let me know. We are always here to be of
> assistance!
>
> Sincerely yours,
> Michael Cheng
> Microsoft Online Partner Support
> When responding to posts, please "Reply to Group" via your newsreader so
> that others may learn and benefit from your issue.
> =====================================================> This posting is provided "AS IS" with no warranties, and confers no rights.
>|||Hi Johan,
Thanks for the quick response.
It is very strange the SP4 enroll this new error message. I would like to
check it further. Is it possible for you to send me sample tables or sample
databases? (mdb file or the scripts with your tables)
I understand the information must be sensitive to you. Alternatively, to
efficiently troubleshoot a this issue, we recommend that you contact
Microsoft Customer Service and Support and open a support incident and work
with a dedicated Support Professional.
Please be advised that contacting phone support will be a charged call.
However, if it turn out to be a bug in SP4 then charges are usually
refunded or waived.
To obtain the phone numbers for specific technology request please take a
look at the web site listed below.
http://support.microsoft.com/default.aspx?scid=fh;EN-US;PHONENUMBERS
If you are outside the US please see http://support.microsoft.com for
regional support phone numbers.
Sincerely yours,
Michael Cheng
Microsoft Online Partner Support
When responding to posts, please "Reply to Group" via your newsreader so
that others may learn and benefit from your issue.
=====================================================This posting is provided "AS IS" with no warranties, and confers no rights.|||Hi Michael
Thanks for the quick response. We took the database again to a SP3a SQL
server and it works fine there, must be a SP4 problem. I'll be sending you
the database via e-mail shortly. Please treat the data and structure of the
database as very sensitive. Hope to hear from you soon.
Regards and many thanks.
--
Johan Fourie
"Michael Cheng [MSFT]" wrote:
> Hi Johan,
> Thanks for the quick response.
> It is very strange the SP4 enroll this new error message. I would like to
> check it further. Is it possible for you to send me sample tables or sample
> databases? (mdb file or the scripts with your tables)
> I understand the information must be sensitive to you. Alternatively, to
> efficiently troubleshoot a this issue, we recommend that you contact
> Microsoft Customer Service and Support and open a support incident and work
> with a dedicated Support Professional.
> Please be advised that contacting phone support will be a charged call.
> However, if it turn out to be a bug in SP4 then charges are usually
> refunded or waived.
> To obtain the phone numbers for specific technology request please take a
> look at the web site listed below.
> http://support.microsoft.com/default.aspx?scid=fh;EN-US;PHONENUMBERS
> If you are outside the US please see http://support.microsoft.com for
> regional support phone numbers.
>
> Sincerely yours,
> Michael Cheng
> Microsoft Online Partner Support
> When responding to posts, please "Reply to Group" via your newsreader so
> that others may learn and benefit from your issue.
> =====================================================> This posting is provided "AS IS" with no warranties, and confers no rights.
>|||Hi Johan,
Thanks for your kindly sending me the database files.
After looking into the execution plan, I believe this is a known issue of
SQL Server service pack 4.
Since we add stack overflow checks in SP4, it raise this new problem. For
now, we do not have a pretty workaround for this issue.
However, if you want to apply SP4 and this issue has big business impact to
you. I would like to recommand you opening a grace case with Microsoft
Customer Service and Support (CSS). Please understand that we could not
share you the QFE request in the newsgroup.
Please be advised that contacting phone support will be a charged call.
However, if the support engineer confirmed this is a bug and no other
support then charges are usually refunded or waived.
To obtain the phone numbers for specific technology request please take a
look at the web site listed below.
http://support.microsoft.com/default.aspx?scid=fh;EN-US;PHONENUMBERS
If you are outside the US please see http://support.microsoft.com for
regional support phone numbers.
You could share the information below with the CSS support engineer
Bug# SQL Server 8.0 20000179
I do apologized for this new known issue that caused to you and thanks so
much for your patience and cooperation. If you have any questions or
concerns, don't hesitate to let me know. We are always here to be of
assistance!
Sincerely yours,
Michael Cheng
Microsoft Online Partner Support
When responding to posts, please "Reply to Group" via your newsreader so
that others may learn and benefit from your issue.
=====================================================This posting is provided "AS IS" with no warranties, and confers no rights.|||Hi Michael
Thanks. Is there a quick fix available for this problem? We've advised all
our customers to go to SP4 (when it was released) and they wont be happy if
we tell them to go back to SP3a (as they all have this problem). Can you also
supply us with the KB number for this problem. Would it be possible for you
to send the QFE via e-mail to me?
Thanks in advance.
--
Johan Fourie
"Michael Cheng [MSFT]" wrote:
> Hi Johan,
> Thanks for your kindly sending me the database files.
> After looking into the execution plan, I believe this is a known issue of
> SQL Server service pack 4.
> Since we add stack overflow checks in SP4, it raise this new problem. For
> now, we do not have a pretty workaround for this issue.
> However, if you want to apply SP4 and this issue has big business impact to
> you. I would like to recommand you opening a grace case with Microsoft
> Customer Service and Support (CSS). Please understand that we could not
> share you the QFE request in the newsgroup.
> Please be advised that contacting phone support will be a charged call.
> However, if the support engineer confirmed this is a bug and no other
> support then charges are usually refunded or waived.
> To obtain the phone numbers for specific technology request please take a
> look at the web site listed below.
> http://support.microsoft.com/default.aspx?scid=fh;EN-US;PHONENUMBERS
> If you are outside the US please see http://support.microsoft.com for
> regional support phone numbers.
> You could share the information below with the CSS support engineer
> Bug# SQL Server 8.0 20000179
> I do apologized for this new known issue that caused to you and thanks so
> much for your patience and cooperation. If you have any questions or
> concerns, don't hesitate to let me know. We are always here to be of
> assistance!
>
> Sincerely yours,
> Michael Cheng
> Microsoft Online Partner Support
> When responding to posts, please "Reply to Group" via your newsreader so
> that others may learn and benefit from your issue.
> =====================================================> This posting is provided "AS IS" with no warranties, and confers no rights.
>|||Hi Johan,
Thanks for your update.
Unfortunately, I am so sorry to say that there is temporarily no public
Knowledge Base articles and document describing this issue. This issue only
occurs in SQL Server 2000 SP4, SQL Server 2000 SP3 and SQL Server 2005 do
not have this issue.
Unfortunately, I am unable to process the delivery of QFEs via email
directly. However, you should be able to quickly get this QFE by opening
the grace case with CSS.
Thank you for your patience and cooperation. If you have any questions or
concerns, don't hesitate to let me know. We are always here to be of
assistance!
Sincerely yours,
Michael Cheng
Microsoft Online Partner Support
When responding to posts, please "Reply to Group" via your newsreader so
that others may learn and benefit from your issue.
=====================================================This posting is provided "AS IS" with no warranties, and confers no rights.|||Thanks. I'll open a case with CSS.
Regards.
--
Johan Fourie
"Michael Cheng [MSFT]" wrote:
> Hi Johan,
> Thanks for your update.
> Unfortunately, I am so sorry to say that there is temporarily no public
> Knowledge Base articles and document describing this issue. This issue only
> occurs in SQL Server 2000 SP4, SQL Server 2000 SP3 and SQL Server 2005 do
> not have this issue.
> Unfortunately, I am unable to process the delivery of QFEs via email
> directly. However, you should be able to quickly get this QFE by opening
> the grace case with CSS.
> Thank you for your patience and cooperation. If you have any questions or
> concerns, don't hesitate to let me know. We are always here to be of
> assistance!
>
> Sincerely yours,
> Michael Cheng
> Microsoft Online Partner Support
> When responding to posts, please "Reply to Group" via your newsreader so
> that others may learn and benefit from your issue.
> =====================================================> This posting is provided "AS IS" with no warranties, and confers no rights.
>|||Hi Johan,
Thanks for your cooperation.
I just wanted to follow up to let you know that the Support Engineer that
you are working with in CSS has contacted me directly to let me know that
the hotfix I recommended is not currently available to the public. I
apologize that I was not previously aware of this situation.
Please continue to work with the CSS Support Engineer for a workaround or
resolution to your issue.
Again, I am sorry that I was unaware that this hotfix is not available to
the public.
Thank you for your patience and cooperation.
Sincerely yours,
Michael Cheng
Microsoft Online Partner Support
When responding to posts, please "Reply to Group" via your newsreader so
that others may learn and benefit from your issue.
=====================================================Business-Critical Phone Support (BCPS) provides you with technical phone
support at no charge during critical LAN outages or "business down"
situations. This benefit is available 24 hours a day, 7 days a week to all
Microsoft technology partners in the United States and Canada.
This and other support options are available here:
BCPS:
https://partner.microsoft.com/US/technicalsupport/supportoverview/40010469
Others: https://partner.microsoft.com/US/technicalsupport/supportoverview/
If you are outside the United States, please visit our International
Support page: http://support.microsoft.com/common/international.aspx
=====================================================This posting is provided "AS IS" with no warranties, and confers no rights.
Wednesday, March 21, 2012
Query performance problems with join on UDF-based computed column
select * from TheTable A, FKeyTable B
where A.ComputedColumn1 = B.KeyColumn
but this one sends the CPU usage of SQL Server to 99% for a very long time:
select * from TheTable A, FKeyTable B
where A.ComputedColumn2 = B.KeyColumn
The main difference we can see that the computed column that causes problems is based on a UDF, and the other one isn't (but again, both are computed). When I look at the execution plan, the slow query shows a Nested Loop (Inner Join) with a "No Join Predicate" warning, with the estimated # of rows being 70 million (which correponds to the product of 1016 rows in TheTable and 69K rows in FKeyTable). The fast query doesn't have that warning, and shows 1016 rows (the # of rows in TheTable).
Does anyone know why the usage of a UDF would induce this horribly inefficient join behavior? Anything we can do to fix it?
This is SQL Server 2005 SP2, btw.
By default, the engine does not know anything about the udf (i.e. has no statistics). So, bad plan can be generated.
To avoid this, I suggest that you create your udf with schemabinding and persist your computed column.
|||oj,Thanks for the suggestions. I tried changing the UDF to use schema binding, but it didn't seem to effect the query plan at all. Persisting the computed column isn't a possibility here, as its value changes over time.
|||
Scalar udf is executed once per row which tends to be the cause for horrible performance. Now that we know that the values are non-deterministic, performance hit is almost unavoidable.
Can you post some ddl+sample data (insert)+sample code. We might be able to devise a solution.
|||I'll see if I can come up with a simple repro.
What I don't understand is why a regular inline computed column (which is also non-deterministic) performs just fine, but the UDF version performs so horribly. I tried taking the computed column, which is essentially computing a time duration by calling datediff between "now" and a datetime stored in another column of the table, and converting the exact logic into a UDF. Once I do that, it has the same (bad) performance characteristics as my original problem - a Nested Loop with a "No Join Predicate" warning, 70M rows.
BTW, actually querying the column value is quite fast. It's the join that craters the performance.
|||Have you tried explicitly creating a statistic for the column?
e.g.
create statistics _stats on table1(compute_col) with fullscan, norecompute
|||No luck. It complained that it "cannot be used in an index or statistics or as a partition key because it is non-deterministic."|||Kevin, please post the requested info. Don't forget the udf code, too.
Note: you can use either free tool, objectscriptr or qalite from rac4sql.net to generate ddl+sample data.
|||Here's the scripts (as far as I can tell, there's no attachment support on the forums....bummer). This is a seriously simplified version of our tables that reproduce the issue we're having.
The Create script:
Code Snippet
CREATE FUNCTION [dbo].[ComputeFKey](@.startTS datetime)
RETURNS float
with schemabinding
AS
BEGIN
RETURN datediff(minute,@.startTS,getutcdate())/(5)*(5)
END
go
CREATE TABLE [dbo].[JoinTable](
[KeyCol] [int] NOT NULL,
[MoreData] [int] NOT NULL,
CONSTRAINT [PK_JoinTable] PRIMARY KEY CLUSTERED
(
[KeyCol] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
go
CREATE TABLE [dbo].[MainTable](
[KeyCol] [int] IDENTITY(1,1) NOT NULL,
[CompCol] AS ((datediff(minute,[TSCol],getutcdate())/(5))*(5)),
[UdfCompCol] AS ([dbo].[ComputeFKey]([TSCol])),
[TSCol] [datetime] NOT NULL,
CONSTRAINT [PK_MainTable] PRIMARY KEY CLUSTERED
(
[KeyCol] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
go
Script to populate the join table:
Code Snippet
declare @.Start int
declare @.KeyVal int
declare @.RowsToAdd int
declare @.MoreData int
select @.RowsToAdd = 1000
if exists (select * from JoinTable)
select @.Start = max(KeyCol) + 5 from JoinTable
else
select @.Start = 0
select @.KeyVal = @.Start
while (@.KeyVal < (@.Start + @.RowsToAdd * 5))
begin
select @.MoreData = @.KeyVal % 60
insert into JoinTable (KeyCol, MoreData) values (@.KeyVal, @.MoreData)
select @.KeyVal = @.KeyVal + 5
end
Script to populate the main table:
Code Snippet
declare @.Row int
declare @.RowsToAdd int
select @.Row = 0
select @.RowsToAdd = 1000
while (@.Row < @.RowsToAdd)
begin
insert into MainTable (TSCol) values (DateAdd("hh", -1, getutcdate()))
select @.Row = @.Row + 1
end
Queries that demonstrate the issue:
Code Snippet
-- Fast query
select A.KeyCol from MainTable A, JoinTable B where A.CompCol = B.KeyCol
-- Slow query (Nested Loop with table spool)
select A.KeyCol from MainTable A, JoinTable B where A.UdfCompCol = B.KeyCol
The second query joins against the computed column that uses the UDF, and ends up with a Nested Loop with a table spool (a million rows in the Nested loop if you use the default values of the populate scripts - 1000 rows per table).
|||Hi Kevin!
This is probably a bug, because MS SQL 2k behaves fine in this case (the execution plans are identical).
And btw, your function and column have different data types (float and integer).
|||Oops - that was an error in creating my simplified repro case. The actual application schema has a CAST in the computed column. Thanks for catching that (though it doesn't affect the result). Replace the UdfCompCol definition with this:
[UdfCompCol] AS (CONVERT([int],[dbo].[ComputeFKey]([TSCol]),0))
|||
Thank you for posting the requested info. It's helpful to see what you're up against.
Now, to the _bad_ news. This is actually by design. Because you use getutcdate() within your udf, you make it non-deterministic. In sql2k, it is not possible to call getdate or getutcdate in a udf. Sql2k5 has loosen up this restriction but because the udf is marked as non-deterministric, the index seek (on jointable) is not possible. Therefore, the only option left for the engine is do a full scan of the jointable. Thus is the performance difference you see.
Btw, you can see that the perf is exactly identical for inline computed column versus udf computed column in sql2k. You will have to use the getdate_view trick though.
e.g.
Code Snippet
create view _utcdate
as
select getutcdate() [dt]
go
create FUNCTION [dbo].[ComputeFKey](@.startTS datetime)
RETURNS int
--with schemabinding
AS
BEGIN
RETURN (select datediff(minute,@.startTS,dt)/(5)*(5) from _utcdate)
END
go
-- Fast query
select A.KeyCol from MainTable A, JoinTable B where A.CompCol = B.KeyCol
-- Slow query (Nested Loop with table spool)
(1 row(s) affected)
StmtText
--
|--Hash Match(Inner Join, HASH.[KeyCol])=(
.[CompCol]), RESIDUAL
.[KeyCol]=
.[CompCol]))
|--Clustered Index Scan(OBJECT[tempdb].[dbo].[JoinTable].[PK_JoinTable] AS
))
|--Compute Scalar(DEFINE.[CompCol]=datediff(minute, MainTable.[TSCol], getutcdate)/5*5))
|--Clustered Index Scan(OBJECT[tempdb].[dbo].[MainTable].[PK_MainTable] AS
))
(4 row(s) affected)
StmtText
-
select A.KeyCol from MainTable A, JoinTable B where A.UdfCompCol = B.KeyCol
(1 row(s) affected)
StmtText
--
|--Hash Match(Inner Join, HASH.[KeyCol])=(
.[UdfCompCol]), RESIDUAL
.[KeyCol]=
.[UdfCompCol]))
|--Clustered Index Scan(OBJECT[tempdb].[dbo].[JoinTable].[PK_JoinTable] AS
))
|--Compute Scalar(DEFINE.[UdfCompCol]=[dbo].[ComputeFKey](Convert(MainTable.[TSCol]))))
|--Clustered Index Scan(OBJECT[tempdb].[dbo].[MainTable].[PK_MainTable] AS
))
I kind of figured that would be the answer. However, I still don't understand why the same computation, when done directly in a computed column (but equally non-deterministic), has good performance characteristics. It probably doesn't matter much, since I'm guessing there is no real fix here. I'm just curious.
|||As shown in the query plan for both queries on sql2k, the index/table scan is used. This is the same as in sql2k5 with the udf.
The inline computed column is calculated differently than the udf. Although, it in itself is non-deterministic, the sql2k5 is enhanced (made smarter) to use the available statistics based on the index key to perform an index seek.
In sql2k8, you can force an index seek but it's still not going to be possible for non-deterministic udf.
|||
Hi Oj!
It is not the same as in sql2k5.
I would understand if 2k5 spools the results of the function, but I don't understand why does it spool JoinTable? It does not make any sense...
Query performance problems with join on UDF-based computed column
select * from TheTable A, FKeyTable B
where A.ComputedColumn1 = B.KeyColumn
but this one sends the CPU usage of SQL Server to 99% for a very long time:
select * from TheTable A, FKeyTable B
where A.ComputedColumn2 = B.KeyColumn
The main difference we can see that the computed column that causes problems is based on a UDF, and the other one isn't (but again, both are computed). When I look at the execution plan, the slow query shows a Nested Loop (Inner Join) with a "No Join Predicate" warning, with the estimated # of rows being 70 million (which correponds to the product of 1016 rows in TheTable and 69K rows in FKeyTable). The fast query doesn't have that warning, and shows 1016 rows (the # of rows in TheTable).
Does anyone know why the usage of a UDF would induce this horribly inefficient join behavior? Anything we can do to fix it?
This is SQL Server 2005 SP2, btw.
By default, the engine does not know anything about the udf (i.e. has no statistics). So, bad plan can be generated.
To avoid this, I suggest that you create your udf with schemabinding and persist your computed column.
|||oj,Thanks for the suggestions. I tried changing the UDF to use schema binding, but it didn't seem to effect the query plan at all. Persisting the computed column isn't a possibility here, as its value changes over time.
|||
Scalar udf is executed once per row which tends to be the cause for horrible performance. Now that we know that the values are non-deterministic, performance hit is almost unavoidable.
Can you post some ddl+sample data (insert)+sample code. We might be able to devise a solution.
|||I'll see if I can come up with a simple repro.
What I don't understand is why a regular inline computed column (which is also non-deterministic) performs just fine, but the UDF version performs so horribly. I tried taking the computed column, which is essentially computing a time duration by calling datediff between "now" and a datetime stored in another column of the table, and converting the exact logic into a UDF. Once I do that, it has the same (bad) performance characteristics as my original problem - a Nested Loop with a "No Join Predicate" warning, 70M rows.
BTW, actually querying the column value is quite fast. It's the join that craters the performance.
|||Have you tried explicitly creating a statistic for the column?
e.g.
create statistics _stats on table1(compute_col) with fullscan, norecompute
|||No luck. It complained that it "cannot be used in an index or statistics or as a partition key because it is non-deterministic."|||Kevin, please post the requested info. Don't forget the udf code, too.
Note: you can use either free tool, objectscriptr or qalite from rac4sql.net to generate ddl+sample data.
|||Here's the scripts (as far as I can tell, there's no attachment support on the forums....bummer). This is a seriously simplified version of our tables that reproduce the issue we're having.
The Create script:
Code Snippet
CREATE FUNCTION [dbo].[ComputeFKey](@.startTS datetime)
RETURNS float
with schemabinding
AS
BEGIN
RETURN datediff(minute,@.startTS,getutcdate())/(5)*(5)
END
go
CREATE TABLE [dbo].[JoinTable](
[KeyCol] [int] NOT NULL,
[MoreData] [int] NOT NULL,
CONSTRAINT [PK_JoinTable] PRIMARY KEY CLUSTERED
(
[KeyCol] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
go
CREATE TABLE [dbo].[MainTable](
[KeyCol] [int] IDENTITY(1,1) NOT NULL,
[CompCol] AS ((datediff(minute,[TSCol],getutcdate())/(5))*(5)),
[UdfCompCol] AS ([dbo].[ComputeFKey]([TSCol])),
[TSCol] [datetime] NOT NULL,
CONSTRAINT [PK_MainTable] PRIMARY KEY CLUSTERED
(
[KeyCol] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
go
Script to populate the join table:
Code Snippet
declare @.Start int
declare @.KeyVal int
declare @.RowsToAdd int
declare @.MoreData int
select @.RowsToAdd = 1000
if exists (select * from JoinTable)
select @.Start = max(KeyCol) + 5 from JoinTable
else
select @.Start = 0
select @.KeyVal = @.Start
while (@.KeyVal < (@.Start + @.RowsToAdd * 5))
begin
select @.MoreData = @.KeyVal % 60
insert into JoinTable (KeyCol, MoreData) values (@.KeyVal, @.MoreData)
select @.KeyVal = @.KeyVal + 5
end
Script to populate the main table:
Code Snippet
declare @.Row int
declare @.RowsToAdd int
select @.Row = 0
select @.RowsToAdd = 1000
while (@.Row < @.RowsToAdd)
begin
insert into MainTable (TSCol) values (DateAdd("hh", -1, getutcdate()))
select @.Row = @.Row + 1
end
Queries that demonstrate the issue:
Code Snippet
-- Fast query
select A.KeyCol from MainTable A, JoinTable B where A.CompCol = B.KeyCol
-- Slow query (Nested Loop with table spool)
select A.KeyCol from MainTable A, JoinTable B where A.UdfCompCol = B.KeyCol
The second query joins against the computed column that uses the UDF, and ends up with a Nested Loop with a table spool (a million rows in the Nested loop if you use the default values of the populate scripts - 1000 rows per table).
|||Hi Kevin!
This is probably a bug, because MS SQL 2k behaves fine in this case (the execution plans are identical).
And btw, your function and column have different data types (float and integer).
|||Oops - that was an error in creating my simplified repro case. The actual application schema has a CAST in the computed column. Thanks for catching that (though it doesn't affect the result). Replace the UdfCompCol definition with this:
[UdfCompCol] AS (CONVERT([int],[dbo].[ComputeFKey]([TSCol]),0))
|||
Thank you for posting the requested info. It's helpful to see what you're up against.
Now, to the _bad_ news. This is actually by design. Because you use getutcdate() within your udf, you make it non-deterministic. In sql2k, it is not possible to call getdate or getutcdate in a udf. Sql2k5 has loosen up this restriction but because the udf is marked as non-deterministric, the index seek (on jointable) is not possible. Therefore, the only option left for the engine is do a full scan of the jointable. Thus is the performance difference you see.
Btw, you can see that the perf is exactly identical for inline computed column versus udf computed column in sql2k. You will have to use the getdate_view trick though.
e.g.
Code Snippet
create view _utcdate
as
select getutcdate() [dt]
go
create FUNCTION [dbo].[ComputeFKey](@.startTS datetime)
RETURNS int
--with schemabinding
AS
BEGIN
RETURN (select datediff(minute,@.startTS,dt)/(5)*(5) from _utcdate)
END
go
-- Fast query
select A.KeyCol from MainTable A, JoinTable B where A.CompCol = B.KeyCol
-- Slow query (Nested Loop with table spool)
(1 row(s) affected)
StmtText
--
|--Hash Match(Inner Join, HASH.[KeyCol])=(
.[CompCol]), RESIDUAL
.[KeyCol]=
.[CompCol]))
|--Clustered Index Scan(OBJECT[tempdb].[dbo].[JoinTable].[PK_JoinTable] AS
))
|--Compute Scalar(DEFINE.[CompCol]=datediff(minute, MainTable.[TSCol], getutcdate)/5*5))
|--Clustered Index Scan(OBJECT[tempdb].[dbo].[MainTable].[PK_MainTable] AS
))
(4 row(s) affected)
StmtText
-
select A.KeyCol from MainTable A, JoinTable B where A.UdfCompCol = B.KeyCol
(1 row(s) affected)
StmtText
--
|--Hash Match(Inner Join, HASH.[KeyCol])=(
.[UdfCompCol]), RESIDUAL
.[KeyCol]=
.[UdfCompCol]))
|--Clustered Index Scan(OBJECT[tempdb].[dbo].[JoinTable].[PK_JoinTable] AS
))
|--Compute Scalar(DEFINE.[UdfCompCol]=[dbo].[ComputeFKey](Convert(MainTable.[TSCol]))))
|--Clustered Index Scan(OBJECT[tempdb].[dbo].[MainTable].[PK_MainTable] AS
))
I kind of figured that would be the answer. However, I still don't understand why the same computation, when done directly in a computed column (but equally non-deterministic), has good performance characteristics. It probably doesn't matter much, since I'm guessing there is no real fix here. I'm just curious.
|||As shown in the query plan for both queries on sql2k, the index/table scan is used. This is the same as in sql2k5 with the udf.
The inline computed column is calculated differently than the udf. Although, it in itself is non-deterministic, the sql2k5 is enhanced (made smarter) to use the available statistics based on the index key to perform an index seek.
In sql2k8, you can force an index seek but it's still not going to be possible for non-deterministic udf.
|||
Hi Oj!
It is not the same as in sql2k5.
I would understand if 2k5 spools the results of the function, but I don't understand why does it spool JoinTable? It does not make any sense...
Query performance problems with join on UDF-based computed column
select * from TheTable A, FKeyTable B
where A.ComputedColumn1 = B.KeyColumn
but this one sends the CPU usage of SQL Server to 99% for a very long time:
select * from TheTable A, FKeyTable B
where A.ComputedColumn2 = B.KeyColumn
The main difference we can see that the computed column that causes problems is based on a UDF, and the other one isn't (but again, both are computed). When I look at the execution plan, the slow query shows a Nested Loop (Inner Join) with a "No Join Predicate" warning, with the estimated # of rows being 70 million (which correponds to the product of 1016 rows in TheTable and 69K rows in FKeyTable). The fast query doesn't have that warning, and shows 1016 rows (the # of rows in TheTable).
Does anyone know why the usage of a UDF would induce this horribly inefficient join behavior? Anything we can do to fix it?
This is SQL Server 2005 SP2, btw.
By default, the engine does not know anything about the udf (i.e. has no statistics). So, bad plan can be generated.
To avoid this, I suggest that you create your udf with schemabinding and persist your computed column.
|||oj,Thanks for the suggestions. I tried changing the UDF to use schema binding, but it didn't seem to effect the query plan at all. Persisting the computed column isn't a possibility here, as its value changes over time.
|||
Scalar udf is executed once per row which tends to be the cause for horrible performance. Now that we know that the values are non-deterministic, performance hit is almost unavoidable.
Can you post some ddl+sample data (insert)+sample code. We might be able to devise a solution.
|||I'll see if I can come up with a simple repro.
What I don't understand is why a regular inline computed column (which is also non-deterministic) performs just fine, but the UDF version performs so horribly. I tried taking the computed column, which is essentially computing a time duration by calling datediff between "now" and a datetime stored in another column of the table, and converting the exact logic into a UDF. Once I do that, it has the same (bad) performance characteristics as my original problem - a Nested Loop with a "No Join Predicate" warning, 70M rows.
BTW, actually querying the column value is quite fast. It's the join that craters the performance.
|||Have you tried explicitly creating a statistic for the column?
e.g.
create statistics _stats on table1(compute_col) with fullscan, norecompute
|||No luck. It complained that it "cannot be used in an index or statistics or as a partition key because it is non-deterministic."|||Kevin, please post the requested info. Don't forget the udf code, too.
Note: you can use either free tool, objectscriptr or qalite from rac4sql.net to generate ddl+sample data.
|||Here's the scripts (as far as I can tell, there's no attachment support on the forums....bummer). This is a seriously simplified version of our tables that reproduce the issue we're having.
The Create script:
Code Snippet
CREATE FUNCTION [dbo].[ComputeFKey](@.startTS datetime)
RETURNS float
with schemabinding
AS
BEGIN
RETURN datediff(minute,@.startTS,getutcdate())/(5)*(5)
END
go
CREATE TABLE [dbo].[JoinTable](
[KeyCol] [int] NOT NULL,
[MoreData] [int] NOT NULL,
CONSTRAINT [PK_JoinTable] PRIMARY KEY CLUSTERED
(
[KeyCol] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
go
CREATE TABLE [dbo].[MainTable](
[KeyCol] [int] IDENTITY(1,1) NOT NULL,
[CompCol] AS ((datediff(minute,[TSCol],getutcdate())/(5))*(5)),
[UdfCompCol] AS ([dbo].[ComputeFKey]([TSCol])),
[TSCol] [datetime] NOT NULL,
CONSTRAINT [PK_MainTable] PRIMARY KEY CLUSTERED
(
[KeyCol] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
go
Script to populate the join table:
Code Snippet
declare @.Start int
declare @.KeyVal int
declare @.RowsToAdd int
declare @.MoreData int
select @.RowsToAdd = 1000
if exists (select * from JoinTable)
select @.Start = max(KeyCol) + 5 from JoinTable
else
select @.Start = 0
select @.KeyVal = @.Start
while (@.KeyVal < (@.Start + @.RowsToAdd * 5))
begin
select @.MoreData = @.KeyVal % 60
insert into JoinTable (KeyCol, MoreData) values (@.KeyVal, @.MoreData)
select @.KeyVal = @.KeyVal + 5
end
Script to populate the main table:
Code Snippet
declare @.Row int
declare @.RowsToAdd int
select @.Row = 0
select @.RowsToAdd = 1000
while (@.Row < @.RowsToAdd)
begin
insert into MainTable (TSCol) values (DateAdd("hh", -1, getutcdate()))
select @.Row = @.Row + 1
end
Queries that demonstrate the issue:
Code Snippet
-- Fast query
select A.KeyCol from MainTable A, JoinTable B where A.CompCol = B.KeyCol
-- Slow query (Nested Loop with table spool)
select A.KeyCol from MainTable A, JoinTable B where A.UdfCompCol = B.KeyCol
The second query joins against the computed column that uses the UDF, and ends up with a Nested Loop with a table spool (a million rows in the Nested loop if you use the default values of the populate scripts - 1000 rows per table).
|||Hi Kevin!
This is probably a bug, because MS SQL 2k behaves fine in this case (the execution plans are identical).
And btw, your function and column have different data types (float and integer).
|||Oops - that was an error in creating my simplified repro case. The actual application schema has a CAST in the computed column. Thanks for catching that (though it doesn't affect the result). Replace the UdfCompCol definition with this:
[UdfCompCol] AS (CONVERT([int],[dbo].[ComputeFKey]([TSCol]),0))
|||
Thank you for posting the requested info. It's helpful to see what you're up against.
Now, to the _bad_ news. This is actually by design. Because you use getutcdate() within your udf, you make it non-deterministic. In sql2k, it is not possible to call getdate or getutcdate in a udf. Sql2k5 has loosen up this restriction but because the udf is marked as non-deterministric, the index seek (on jointable) is not possible. Therefore, the only option left for the engine is do a full scan of the jointable. Thus is the performance difference you see.
Btw, you can see that the perf is exactly identical for inline computed column versus udf computed column in sql2k. You will have to use the getdate_view trick though.
e.g.
Code Snippet
create view _utcdate
as
select getutcdate() [dt]
go
create FUNCTION [dbo].[ComputeFKey](@.startTS datetime)
RETURNS int
--with schemabinding
AS
BEGIN
RETURN (select datediff(minute,@.startTS,dt)/(5)*(5) from _utcdate)
END
go
-- Fast query
select A.KeyCol from MainTable A, JoinTable B where A.CompCol = B.KeyCol
-- Slow query (Nested Loop with table spool)
(1 row(s) affected)
StmtText
--
|--Hash Match(Inner Join, HASH.[KeyCol])=(
.[CompCol]), RESIDUAL
.[KeyCol]=
.[CompCol]))
|--Clustered Index Scan(OBJECT[tempdb].[dbo].[JoinTable].[PK_JoinTable] AS
))
|--Compute Scalar(DEFINE.[CompCol]=datediff(minute, MainTable.[TSCol], getutcdate)/5*5))
|--Clustered Index Scan(OBJECT[tempdb].[dbo].[MainTable].[PK_MainTable] AS
))
(4 row(s) affected)
StmtText
-
select A.KeyCol from MainTable A, JoinTable B where A.UdfCompCol = B.KeyCol
(1 row(s) affected)
StmtText
--
|--Hash Match(Inner Join, HASH.[KeyCol])=(
.[UdfCompCol]), RESIDUAL
.[KeyCol]=
.[UdfCompCol]))
|--Clustered Index Scan(OBJECT[tempdb].[dbo].[JoinTable].[PK_JoinTable] AS
))
|--Compute Scalar(DEFINE.[UdfCompCol]=[dbo].[ComputeFKey](Convert(MainTable.[TSCol]))))
|--Clustered Index Scan(OBJECT[tempdb].[dbo].[MainTable].[PK_MainTable] AS
))
I kind of figured that would be the answer. However, I still don't understand why the same computation, when done directly in a computed column (but equally non-deterministic), has good performance characteristics. It probably doesn't matter much, since I'm guessing there is no real fix here. I'm just curious.
|||As shown in the query plan for both queries on sql2k, the index/table scan is used. This is the same as in sql2k5 with the udf.
The inline computed column is calculated differently than the udf. Although, it in itself is non-deterministic, the sql2k5 is enhanced (made smarter) to use the available statistics based on the index key to perform an index seek.
In sql2k8, you can force an index seek but it's still not going to be possible for non-deterministic udf.
|||
Hi Oj!
It is not the same as in sql2k5.
I would understand if 2k5 spools the results of the function, but I don't understand why does it spool JoinTable? It does not make any sense...