You install Microsoft Dynamics CRM 4.0 and follow the instructions to the best of your ability only to find that when you try to access the CRM web site, it prompts you for a password three times then you get the message:
You are not authorized to view this page.
HTTP Error 401.1 – Unauthorized: Access is denied due to invalid credentials.
This has got to be the #1 most common error people run into when installing Microsoft Dynamics CRM 4.0. In all of the production, testing, and development environments I have set up, the 401.1 error is the one I have spent the most time troubleshooting. It is ALWAYS caused by a problem with Kerberos and double-hop authentication. The trouble is that there are SO MANY ways you can break Kerberos. Over the weekend, I went through another marathon troubleshooting session to resolve yet another Kerberos issue. This time it was in our Sandbox (testing) hosting environment. It came down to an incorrect SPN this time.
Service Principal Names (SPNs) and Delegation
For Kerberos double-hop authentication to work, you must have the proper SPNs configured and you must configure delegation for the computer account or service account that the service runs under. For CRM, this is the identity that your CRM web site’s application pool uses.
The SPNs you need to configure for Kerberos double-hop authentication to work properly are set up as follows:
setspn –A HTTP/servername:5555 domain\serviceusername_or_computername
setspn –A HTTP/servername.company.com domain\serviceusername_or_computername
If you configure the application pool to use an AD user account (service account) as its identity, you should use the name of that account in the above configuration. If you use Network Service, you should use the computer name (and you won’t need to prefix it with domain\). These commands add an SPN in Active Directory. You can also use ADSIEdit to configure this. Note that you need to use the port number for the short (NetBIOS) name version of the SPN but NOT for the FQDN version. This one got me recently.
Next, you need to allow delegation for the computer or service account using ADUC.
Incorrect or Missing SPN
You need to make sure that you have the correct SPN configured. Use setspn -L “computer_or_account_name” to list all SPNs for the service account and computer accounts.
Ex:
setspn -L CRMSERVERNAME
setspn -L Domain\service_account_name
In a recent troubleshooting experience, I had the port number added to the FQDN SPN and I got the following misleading error in the event log:
Event Type: Error
Event Source: Kerberos
Event Category: None
Event ID: 4
Date: 1/20/2009
Time: 1:55:03 PM
User: N/A
Computer: ONCCRM01
Description:
The kerberos client received a KRB_AP_ERR_MODIFIED error from the server host/crmdiscoveryserver.domain.net. The target name used was HTTP/crmdiscoveryserver.domain.net. This indicates that the password used to encrypt the kerberos service ticket is different than that on the target server. Commonly, this is due to identically named machine accounts in the target realm (MSCRMHOST.NET), and the client realm. Please contact your system administrator.
For more information, see Help and Support Center at
All of the info I found while searching indicated that the problem was due to the next issue, Duplicate SPNs, when in reality, it was because Kerberos was looking for an SPN without the TCP port and I had configured it with the TCP port.
Duplicate SPNs
Duplicate SPNs can be difficult to resolve if you don’t know how.
Knowledgebase Article Windows 2000 Server Prompts Domain User for Credentials describes how to use LDP and Adsiedit.msc to troubleshoot and fix a duplicate SPN problem. When you run LDP, make your search string http/* to return all SPNs configured for web sites. You can narrow that down further to http/computername* if the former string yields too many results. Do the same for host/* and MSSQLServer/* just in case.
Event ID 11 — Service Principal Name Configuration
Note the important comment that the “setspn -X” option is only available in Windows Server 2008, not 2003. If you have Server 2008, this is much simpler than using LDP which can be daunting to those unfamiliar with AD and LDAP.
“Negotiate,NTLM” not configured on the web site and/or virtual directories
Your web site must have Negotiate/NTLM configured or the client and server will not negotiate Kerberos. This is the default setting but any number of things can change this configuration. The following article discusses how to configure this parameter using adsutil. I prefer to use the Metabase Explorer in the IIS Resource kit because it lets me see all of the configuration parameters and edit them directly. Either way will work.
Keepalives not enabled on the CRM web site
Keepalives are turned on by default in IIS. If they are turned off, Kerberos will break. This is the most difficult issue I have ever had to troubleshoot related to the 401.1 error in CRM. I don’t know how it got turned off but the only thing that led me to the fix was to analyze the traffic with Netmon then use IIS Diagnostics to troubleshoot Kerberos. Use Metabase Explorer or adsutil to examine and change the keepalive setting.
Filed under: Tips & Tricks, Troubleshooting | Tagged: CRM, iis, kerberos, spn |
Dan Blake's CRM Architecture and Business Blog 
[…] entry is fully written in it’s author’s blog. Read all about this here. This entry was posted on Friday, January 23rd, 2009 and is filed under Uncategorized. You can […]
I’ve been hunting what seems a similar issue as your blog title, but I’m confused on some other issues during the installation.
I’m trying to get a simple test build up and make it use a wildcard SSL certificate.
If I use the default installation procedure it will use the http://ServerName:Port for all configuration entries in the MSCRM_Config.dbo.DeploymentProperites, and the registry which I assume is why I get the exception you solved here.
However, everything else seems to work if I hit it by IP. The workflows work, customization, etc. all work.
When I add my wildcard certificate to the website binding, update the deployment properties table, and restart everything it seems as everything is working over SSL.
However, now the AsyncService throws a bunch of errors that prevent workflows from successfully completing.
The exception relates to the ASP.NET calls. System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a send.
I apologize if this is the incorrect place to post these questions, and used the wrong protocol but I’m painfully laughing because I’ve configured and written many systems so I’m looking for some small configuration issue.
Thank you in advance.
Chris,
Feel free to post a comment on this blog any time and I’ll help you out if I can. You should also post your question on the CRM newsgroups. A couple things to check… Use the IFDConfig tool and make sure that you have your InternalNetworkAddress configured properly (this changes the corresponding registry setting). Also, make sure the IIS web site has both “Enable Anonymous Access” and “Integrated Windows Authentication” configured. You can refer to my earlier post, A Primer on Multitenancy for Microsoft Dynamics CRM, for more details on configuring IFD with a wildcard SSL certificate.
-Dan
Thanks Dan. Great community support.
The links are very helpful. I’m curious though.
I don’t want an internet facing deployment, and only need the Professional version.
I have the default install working and used the guidance you offered with SetSPN.
I’m confused on where the documentation lives for the basic steps to make this use a secure connection. It seems like it should be 5 or fewer steps and done.
I’m currently using Windows 2008 (IIS 7), and SQL 2008. My original thought was:
1. Add wildcard certificate to the web server
2. Add binding to CRM site for https using certifcate
I thought this would be it but seems the site won’t respond, and the Outlook Client won’t accept URL. It’s like the web server won’t fire the http modules correctly unless you update the deployment properties in the mscrm config.
I did this and then it responded via the Outlook Client, web client, and via web services. I’ve got integration code ready to go that wires the CRM with our internal systems.
However, the Async Service throws exceptions over and over, and all system jobs and workflows are dead.
Thanks again, and I don’t want to liter up your comment tree here but wanted to be verbose enough to articulate my basic intention.
I’ll take your guidance on other areas for help.
Thanks again for fantastic community support.
Chris
Chris,
You are quite welcome. I started this blog to be a resource to the community where I and others can help others in the community. So, you are not littering up the comment tree at all. Quite the contrary, you are adding value. If you have a question/issue, someone else surely does too.
I didn’t understand what you were trying to do after reading your first post. Since you were implementing a wildcard SSL cert, I assumed (incorrectly) that you were implementing IFD. Now I understand that you are trying to do a basic, single-tenant implementation but you want to secure the traffic with SSL. Did you follow the article titled Make Microsoft Dynamics CRM 4.0 client-to-server network communications more secure?
By the way, are you going to allow users to connect over the Internet and through a firewall or only while on the LAN/WAN? I’m just curious as to why you aren’t using IFD.
-Dan
Hi Dan,
Another great link, and I’m surprised that I never got this in my results from the numerous google searches.
However, I did find most of those steps from various documents.
It seems I tried all of those but I think I may have misconfigured it because I’m using a wildcard SSL.
For the attribute stored in the SQL table ADSdkRootDomain I entered the 3rd level domain name like crm.mycompany.com, but the documentation seems to indicate I need to use the *.mycompany.com:443. Any expert advice here?
The next step also is a little vague in editting the registry for the LocalSdkPort. The default install allows you to build a new site hosted on port 5555. The documentation in the article says if you are hosting on 80 or 443 do nothing, so do I change the LocalSdkPort to 80 or 443? My guess is 443.
I think I’ve made this a little more challenging than it really is but it seems that once you misconfigure and move forward the CRM install becomes unstable and the only option is rebuild.
I’m also using all latest technology like SQL 2008 which uses it’s own web server to host reporting services, IIS 7, and Window 2008 all on it’s own server for testing purposes. I know how other products are built around an older version of the product, and some of the alterations for making the product work with newer versions can have strange side effects. It seems everything is perfect for the OS, and web server, but I may need to use SQL 2005 to avoid any http and port collisions.
To answer your other question we will most likely force all access over our company VPN. It seems that the install is much easier, and it follows our approach to access for all other company related properties.
Thanks again for the insight, and I’ll try the configuration changes in proper order as the document describes. If you can answer the above on how CRM wants the certificate and port entered I’ll be more than happy to repost all the basic steps so others don’t have to fight this. :)
Chris
Chris,
Your questions:
1.) “For the attribute stored in the SQL table ADSdkRootDomain I entered the 3rd level domain name like crm.mycompany.com, but the documentation seems to indicate I need to use the *.mycompany.com:443. Any expert advice here?”
You definitely do not want to use the “*” here. If your org name is “crm” then you would want to use “mycompany.com:443” in the ADSdkRootDomain field. If your orgname something else and your root domain is “crm.mycompany.com” then you should use “crm.mycompany.com:443” here. Just make sure the FQDN your users utilize has a hostname that matches the unique org name.
2.) “The next step also is a little vague in editing the registry for the LocalSdkPort. The default install allows you to build a new site hosted on port 5555. The documentation in the article says if you are hosting on 80 or 443 do nothing, so do I change the LocalSdkPort to 80 or 443? My guess is 443.”
I would try “do nothing” here as the documentation states then if it doesn’t work, try 443. I haven’t tried this without doing a full IFD implementation.
3.) “but it seems that once you misconfigure and move forward the CRM install becomes unstable and the only option is rebuild.”
I know what you mean. Sometimes it seems that there is some hidden problem making it unstable and I also get to a point where I just want to start over with a clean install. However, logic tells me that it seems impossible to fix only because I either: a. can’t figure out the one small error causing the problem or b. there are multiple issues, all of which need to be fixed, but I’m not solving the problem by attempting one thing at a time. Usually, if you keep to a strict troubleshooting methodology of changing one thing at a time, the symptoms will change when you fix one of the issues. That said, many times the shorter path is to start over clean.
4.) “but I may need to use SQL 2005 to avoid any http and port collisions.”
You have probably read this kb article but, just in case:
Support for running Microsoft Dynamics CRM 4.0 together with Microsoft SQL Server 2008
There are 3 hotfixes you need to apply. However, the easiest route is to check “allow setup to check for updates” during the install. There is no mention of port conflicts in the above article. I’m embarrassed to say that I haven’t personally (my staff tried it but I haven’t had time) implemented CRM with SQL 2008 yet so I’m not aware of how the SRS site gets configured by default. With SQL 2005, I like to use the default web site for both CRM and SRS. SRS sets up virtual directories under the default web site. Then you install CRM and select the existing web site and it populates all of the CRM files in the same site. I would attempt to install SQL 2008 the same way.
Good Luck!
-Dan
Thanks Dan.
However, I’m at a loss again.
Followed steps to the letter, and the only thing I can see that may be off is the following.
My wildcard is for *.mycompany.com.
Entering crm.mycompany.com for the ‘host name’ on the binding in IIS site, and added this to the database properties as recommended with crm.mycompany.com:443.
My organization is named ‘mycompany’.
Updated the registry attribute with port 443, because it was set to 5555.
Added host name crm.mycompany.com to the local host file so it routes correctly during test.
Ran IISRESET
Net Stop <>
Net Start <>
Kicker is that web site responds, Outlook Client configuration works, and everything seems normal.
When reviewing System Jobs they are all in a ‘waiting status’ with error related to the System.Net.WebException that seems to indicate the Async Service can’t communicate using the SSL.
My only assumption is that the orgname and root zone name of ‘mycompany’ are identical and it’s causing some form of confusion.
I’ve run the Trace Tool and Diag Tools found on the web but nothing but the same exceptions in the logs for System Jobs.
Thanks for the feedback.
Looks like the NET Stop and Net Start comment items removed the Async Service reference. Basically just saying I restarted the Async Service so it would pickup any changes.
Thanks… I’ve got it to work finally after discarding the port number for the FQDN registration. I’ve burnt six hours of endless googling and reading… cheers!
Thank you for the help! And I think this talk of deleting SPAM or other inappropriate content from your blog breaking universal blogging rules is silly. You have to have some way of keeping things under control. Especially once you get high page rankings and the spam bots leave comments a couple times a week.
Matt,
You are quite welcome.
Thanks for the comment,
Dan
Dan – this is a great blog. Very helpful. Thanks and good luck.
Best regards.
Nari
Nari, You are welcome and good luck to you too!
Dan,
thank you so much. After about 20 hours of searching (with detours through the IE configuration and after reading an incredible amount of KB articles and now knowing everything about Kerberos no one should know) we could solve our problem – by switching on keepalives.
Thanks again, keep up the good work.
Regards,
Oliver from Zurich
[…] Here is what led me to the above solution. […]