Sunday, August 23, 2015

Cisco Quality Manager 10.5 upgrade issues

I'm sharing several important lessons learned during a Cisco Quality Manager upgrade from 9.0.1.57 to 10.5 SR6, given they are not well documented.  I hope it saves you some time.

Quality Manager administrator access

In environments where you are using MS Active Directory to associate users with recorded devices you may find your self unable to login to the QM administrator's application after the upgrade.  The normal Quality Manager local admin user and password doesn't seem to work, and neither do the AD accounts you've used for logging in as administrators previously. If you are really motivated and change the model to use QM Authentication, the local QM administrator works but now your administrative AD access is gone. 

What to do?

Note there is a new field buried in the Enterprise | Site Configuration | Enterprise Settings form.  You need to actually select and choose to edit one of your Active Directory entries to see it.  Of course you can't do any of this unless you run postinstall.exe because of course, your locked out.

When you choose to edit an Active Directory entry, you will find a field called Admin Group. Mine contained the value QMAdministrator.  I don't know where that value came from because the Admin Group is now actually enforced and represents a MS Active Directory group that contains users that may or may not be Quality Manager system administrators.

First, find or create a AD group that contains your potential QM admins.  Enter the display name of the group in the QM Admin Group field.

Then, when choosing OK you will be presented with a grid of users from that AD group.  Choose one or more to become system admins.  See below.



You're in.

CUCM Recording Server SIP Trunk

If you are using Cisco Built In Bridge recording / Calabrio network recording you will already have a SIP trunk configured in CUCM, pointing to your recording server.  This trunk carries information from CUCM to Quality Manager regarding the DNs that need to be recorded, and from Quality Manager regarding what recording server to use.  There's the rub.  Previously the CUCM pointed the trunk to the Quality Manager recording server and no one thought about it. Of course, you would point it to the recording server because of course that's where the recording are going to go. Right?

Calabrio has changed the basic architecture in 10.5.  You now point the SIP trunk in CUCM to the Quality Manager Base server, not the recording server.  Before you correct this, you might dig around in the QM logs and find references to your QM not receiving SIP invites on the base server and be confused. Why would you need them there? 

Reset your CUCM trunk, start WireSharking stuff, reread all the install and upgrade guides you can find from Cisco's site. Freak out a little.

Then just point the trunk to the Quality Manager base server, rather than the record server where it's been pointing forever. 

See below.


Quality Manager VoIP Devices and Recording Cluster and Signaling Group

New programmatic entities in Quality Manager probably make in more scalable and manageable. New information under Enterprise | Site Configuration | Telephony Groups allow you to parse up and group different recording resources and signalling types.  This is great, but after the upgrade you will need to verify those values have been associated with your VoIP devices correctly.  My experience was that none of my devices had been associated with a Recording Cluster or Signaling Group at all.  That won't work.

Luckily, given all my devices need to be associated with the same Recording Cluster and Signaling Group, it's an easy edit of the entire list.  If your environment has multiple recorders, this might be a more painful process.

Go to Enterprise | Record Server Configuration | VoIP Devices and clean up your mess.

See below.


Enjoy your recordings again.

Wednesday, June 24, 2015

Cisco PLM Prime License Manager login hangs

I've run into an issue twice now on CUCM 10.5.2 systems.  When attempting to log into Cisco Prime License Manager running co-resident with a CUCM server, after entering the correct credentials the login page never changes.  The small spinning status wheel below the credentials stops but the browser may look like it is still attempting access.  You can't access PLM.

You may likely be hitting  bug CSCur95552.

For some reason, Cisco TAC has not documented any work around or fixes at this time.

If running co-resident with CUCM, you need regenerate the IPSec certificate on the node where Prime License Manager is running. Then if this is your CUCM Publisher, restart the DRF Master and DRF Local services. If it's not the Publisher, just restart the DRF Local services.

You should then be able to login.

Regenerate Certificate:
Navigate to Cisco Unified OS Administration | Securtiy | Certificate Management and choose Find.
Scroll down to and choose the ipsec certificate associated with the server where PLM is running co-resident.

Choose Regenerate and wait a few moments until it indicates success.


Restart Services:
Navigate to Cisco Unified Serviceability | Tools | Control Center - Network Services.
Pick the server where PLM is running co-resident and choose Go.
Scroll to and choose Cisco DRF Master, click Restart, and wait until it has been restarted.
Scroll to and choose Cisco DRF Local, click Restart, and wait until it has been restarted.

You should now be able to log into PLM.

Restart Tomcat:
If you experience the same issue, restart Tomcat on the same server and repeat the above steps.
SSH to or open VMware console on server where PLM is co-resident.
Login as the platform administrator.
run "utils service restart Cisco Tomcat"


Monday, June 15, 2015

Cisco CUCM SIP URI Case Sensitive

Client indicated after starting to deploy B2B video, Jabber Guest, immersive telepresence endpoints, etc. that they were occasionally running into issues when dialing via SIP URI.  It was discovered that the entries in MS Active Directory may not have been consistently entered as lower case.  Some may have been entered using camel case, and I assume some may all be upper case as well.  When CUCM synchronized it's users with Active Directory, those cases were respected.

From the version 9 SRND, you will find:

Per RFC 3261 (section 19.1.4, URI Comparison) comparison of the userinfo of SIP URIs has to be case-sensitive. According to this standardized behaviors, Alice@cisco.com and alice@cisco.com are not to be considered equivalent. When routing directory URIs, Unified CM respects this standard and looks for a case-sensitive full match of the user portion and a case-insensitive match of the host portion. To avoid confusion, Cisco highly recommends provisioning only directory URIs with all lowercase userinfo so that all directory URIs can reliably be dialed by entering all lowercase information. Unified CM 9.1 and later releases can be configured to always use case-insensitive comparison of the user info portion of directory URIs. This can be achieved by configuring the enterprise parameter URI Lookup Policy accordingly. This setting applies to matching locally configured directory URIs and also to matching directory URIs for which an ILS lookup is done. The default setting of this enterprise parameter defines standard compliant case-sensitive matching of the user info portion of directory URIs.

Cisco is apparently doing the correct thing by considering Me@abc.com and me@abc.com two different things.  Unfortunately I can't imagine anyone in the wild assuming the same thing.  Regardless of whether an enterprise was lucky enough to develop and follow a procedure where all mail IDs or SIP URIs were entered with a consistent case, lower, upper or camel, they can not assume their users or clients will do the same.

With that, starting in version 9 there is an enterprise parameter that will toggle how lookups are done.

Navigate to System | Enterprise Parameters and in the Enterprise Parameters Configuration section, find URI Lookup Policy.

URI Lookup Policy
Change the value to Case Insensitive and the issue is resolved.  This is not business impacting and no resets are required.

From the help file:

This parameter specifies the way the match is done for the configured user part of the URIs in Cisco Unified Call Manager. If the parameter is set to Case Sensitive, we will match the exact case sensitive URI configured in Cisco Unified CM. If the parameter is set to Case Insensitive, then case insensitive lookup will be done.


 


Wednesday, May 13, 2015

Unity Connection Changing The From Name in Single Inbox / Unified Messaging

While performing a upgrade of CUCM 7 and Unity 7 to CUCM 10.5 and Unity Connection 10.5, the client pointed out an interesting difference in how voice mails were presented in the Outlook client.  When receiving a message from the old Unity system, Outlook showed the client's company name in the From: column when the caller was unidentified.  Unity provided variables to modify this field. When voice mails were received from the new Unity Connection system the From: column shows Cisco Unity Connection Messaging System.  No where in Unity Connection can this be obviously changed.

The name Cisco Unity Connection Messaging System is actually the display name for the user with the alias UnityConnection.  Unfortunately, it does not appear to be editable via the GUI.
 

If you are brave enough, it can be modified via CLI.  I do not know if it is officially supported, but it is technically.

From the CLI you can execute the following to see the user in question:
run cuc dbquery unitydirdb select displayname,objectid from tbl_user where displayname = 'Cisco Unity Connection Messaging System'

You should be returned something like:
displayname                              objectid
---------------------------------------  ------------------------------------ C
Cisco Unity Connection Messaging System  aeb80eab-1ec0-4add-8d4f-f29396639324

Then, cross your fingers and execute the following.  Here for example I am changing the name to The ABC Company:
run cuc dbquery unitydirdb update tbl_user set displayname = 'The ABC Company' where displayname = 'Cisco Unity Connection Messaging System'

In the GUI you will find the UnityConnection user's display name is now changed to the company name, here for example, The ABC Company.  Messages will still show From:Cisco Unity Connection Messaging System though.

I don't know what service(s) restart commits the change, but I had the luxury of rebooting the entire server.  After that, the From: address in UM does show the new name you defined.

Friday, April 03, 2015

Dial IP addresses from Cisco TelePresence endpoints registered to Cisco CUCM



HowTo: Support dialing IP addresses from Cisco TelePresence endpoints registered to CUCM to reach external H.323 video endpoints 

Information here is a compilation of that found in”Cisco Unified Communications Manager with Cisco VCS (SIP Trunk)Deployment Guide”, various http://supportforums.cisco.com discussions and a most excellent http://www.ucguerrilla.com post.  Note William Bell is the brain behind the ip@ prefix and what follows is my personal interpretation and experience in the wild.Screen shots are actual client configurations in production as of 4/2/2015.


Call flow was tested via a Cisco MX200 TelePresence endpoint registered to CUCM 9.1.2. Calls to external H.323 video endpoints dialed via IP address were sent from CUCM to Expressway 8.5.1 C and E, already providing B2B video calling via SIP URIs. External auto-answering H.323 and SIP video endpoints were called for testing purposes without permission and those IP addresses are not included here.

Background:


Video calling between endpoints registered to CUCM internally and B2B via Expressway was already in production.  Users were accustomed to dialing via SIP URI, like joe.smith@mycompany.net or john.smith@anothercompany.com.

Previously, B2B H.323 video calls were made from Polycom endpoints by dialing public IP addresses. As these older existing Polycom video endpoints were retired and replaced by Cisco video endpoints registered to CUCM, the ability to continue to dial external H.323 devices by IP address was required.

Issues:

 

Although the process of physically dialing IP addresses from Cisco TelePresence Touch (or other) interfaces is no more difficult than dialing SIP URIs, at this time CUCM does not support simply dialing IP addresses.


Cisco’s VCS deployment guide suggests the solution is to define a dialing access code specific to video calls, for example the digit 8. Then you are to instruct the users to dial 8 plus a string of twelve digits to represent the IP address they want to call, where leading zeros are to be added to each octet where necessary and the dots ignored.  For example, users accustomed to dialing 10.100.1.10 on a Polycom would now be required to convert the IP address and dial 8010100001010 from the new Cisco video endpoints.  This is not an acceptable solution.

Solution: 


Configure CUCM and Expressway C and E to support dialing IP addresses using the format ip@.  For clarity, where a user would previously dial 10.100.1.10 from a Polycom device, they would now dial ip@10.100.1.10 from the Cisco device.  The ip@ are literal characters to be dialed prefacing the IP address, but otherwise the IP address portion itself continues to look like and be dialed like an IP address.

Simply:
  1. CUCM will accept a call dialed with a SIP URI format like ip@10.100.1.10 and route it to Expressway C.
  2. Expressway C will accept the call in the ip@10.100.1.10:5060 format (or ip@10.100.1.10:5061), strip the ip@ and :5060 portions of the number and pass the remaining portion of the address, now looking like an IP address, to Expressway E.
  3. Expressway E will not attempt a DNS lookup on the address and will support SIP to H.323 conversions.



Configuration steps:

 


CUCM:

 


Navigate to Call Routing | SIP Route Pattern and add a route pattern that matches your destination video device IP address.  Verify the pattern is for IPAddress Routing and note the address can either be an exact match, or reflect a network address in the X.X.X.X/Y format.  The Sip Trunk / Route List should reflect a SIP trunk between CUCM and Expressway C.  


This is how calls placed from a CUCM registered endpoint to a SIP URI with an IP address rather than domain name (i.e. ip@10.100.1.10) are routed to Expressway C.


An example of a specific IP address being routed to a SIP trunk called ExpresswayCore-SIP-Trunk is below.  This was used for initial testing purposes. Do NOT use this IP address for your own tests.



Again, rather than creating a SIP Route Pattern for every public endpoint you need to contact, you can add addresses in the X.X.X.X/Y IP address and subnet mask format.  Unfortunately CUCM does not accept 0.0.0.0 as an IP address.  This means a single entry similar to an IP default route found in other Cisco IOS based devices doesn’t seem possible.  One work around is dividing the entire IPv4 space into two subnets, 1.0.0.0/1 and 128.0.0.1/1.  It is working successfully at this time.






Expressway C:


Navigate to Configuration | Dial Plan | Transforms and add a new transformation.  The type needs to be RegEx, the pattern string ip@(.*)(:506[01]) and the replace string \1.


Via the RegEx replacement, this transform is responsible for replacing the SIP URI presented by CUCM (i.e. ip@10.100.1.10:5060) with a string resembling simply an IP address (i.e. 10.100.1.10).
Below is the actual configured transform.  It is important to set the priority number so that it does not break any other calling.  Here the transform worked safely with the highest priority value (last to be matched).



Navigate to Configuration| Dial Plan | Search Rule, and add a rule with a Mode that matches Any IP Address.  The target should be a Traversal Zone that passes the call to Expressway E.  This how the string that now looks like an IP address (i.e. 10.100.1.10) gets routed to Expressway E.

Below is the actual search rule added, targeting a Traversal Zone called Traversal B2B that was already supporting B2B video calling vial SIP URIs.




Navigate to Configuration | Dial Plan | Configuration and verify Calls to unknown IP addresses are set to Indirect.  I don’t have a picture here, but it is the only option on the page and will likely not need to be changed. 


Expressway E:  


Navigate to Configuration | Dial Plan | Configuration and verify Calls to unknown IP addresses are set to Direct.



This routes calls to IP addresses directly to the destination on the Internet without attempting DNS lookups. 







Navigate to Configuration | Protocols and verify both SIP and H.323 protocols are enabled.  These are two basic settings required to support SIP calls from CUCM / Expressway C and H.323 calls to external endpoints.



Navigate to Configuration | Protocols | Interworking and enable SIP to H.323 internetworking. I don’t have a picture here but it is again the only option on the page and is likely default.  This provides support for your call being generated by a Cisco SIP based TelePresence endpoint to communicate with the target H.323 endpoint.



  
Notes 

  • On Cisco Telepresence endpoints, users will need to be instructed to dial ip@ in front of the IP address the intend to call.
  • The event logs in Expressway C and E are relatively easy to use and decipher. If B2B video calling is in production, once you add your CUCM SIP Route Pattern, look to the Expressway C and then E for troubleshooting.



Log Example 1: Here is Expressway C rejecting a call from a device at 7902@mycompany.net, my test MX200 registered to the CUCM at 10.181.1.8.  The destination IP address is 76.74.100.40.  The 404 response code was a result of the Expressway C missing the search rule designed to route IP addresses to Expressway E via the Traversal Zone.



Log Example 2: Here is Expressway C reporting a 480 response to an attempt to reach 66.135.98.168.  The lesson here is when making test calls to endpoints by IP address, first verify the IP address in question actually points to a working video endpoint. This one didn’t.



Log Example 3: Here is Expressway E reporting the destination 76.74.100.40 can’t be found.  This indicates the call has made it all the way from the CUCM endpoint, to CUCM, to Expressway C and to Expressway E and looks like an IP address.  We know this is a working endpoint though.  The issue was the Calls to unknown IP addresses value was not set to Direct.