Wednesday, June 29, 2016

Stream Multicast MOH from gateway notes

Cisco CUCM deployments can provide MoH / music on hold to callers that is streamed from the CUCM server(s) themselves.  If the devices you are putting on hold are far from the CUCM servers though and you don't want the music streams to cross lower bandwidth WAN links, one option you have is to stream the MoH from a voice gateway that is local to the devices on hold.

There are quite a few dependencies to make this work though. Below is a summary of the things I addressed today making it work at a new site.

In your CUCM, you need to enable multicast capabilities on the MoH source, the MoH server, and a Media Resource Group (MRG).

Audio Source with multi-cast enabled

MOH Server with multi-cast enabled

MRG with Multi-cast enabled

Note: You should configure multicast audio sources to increment on the IP address and not the port number (see server configuration above) because:

  • IP phones placed on hold join multicast IP addresses, not port numbers
    • Cisco IP phones have no concept of multicast port numbers. Therefore, if all the configured codecs for a particular audio stream transmit to the same multicast IP address (even on different port numbers), all streams will be sent to the IP phone even though only one stream is needed. This has the potential of saturating the network with unnecessary traffic because the IP phone is capable of receiving only a single MoH stream.
  • IP network routers route multicast based on IP addresses, not port numbers 
    • Routers have no concept of multicast port numbers. Thus, when it encounters multiple streams sent to the same multicast group address (even on different port numbers), the router forwards all streams of the multicast group. Because only one stream is needed, network bandwidth is over-utilized and network congestion can eventually result.

You then need to create and / or assign a MRGL to the trunk, device pool or other that includes the MRG configured above.

MRGL used by remote devices that includes multi-cast MRG from above

Note: You do NOT need to support multicast traffic generically across the network or across the WAN.  Because the phones and voice gateway used for streaming the MoH file were on the same subnet at the remote site, I only enabled support for multicast routing on the voice gateway itself.  The switch where the phones and voice gateway were connected at the remote site did not even have multicast routing enabled.

Now verify what multi-cast IP address your music will utilize.  It is not necessarily the IP address set in the MOH server you see above or even related to the number of the audio source in use. Rather it depends on what codec is being used and in what order the audio sources were added to your server.

Here's a great discussion on the topic:

On the MOH server where "Enable Multicast Audio Sources" is enabled, here's how I found the address I needed.

admin:run sql select m.mohaudiosourceid,m.multicastaddress,m.multicastport, from mohservermulticastinfo as m,typemohcodec as c where m.tkmohcodec = c.enum
mohaudiosourceid multicastaddress multicastport name
================ ================ ============= ========
1             16384         711 ulaw
51            16384         711 ulaw
2             16384         711 ulaw
1             16384         729
51            16384         729
2             16384         729
1             16384         wideband
51            16384         wideband
2             16384         wideband
1             16384         711 alaw
51            16384         711 alaw
2             16384         711 alaw

Note, if here I want to use audio source 2 and the G.711 ulaw codec, I should then use multicast address in my gateway config.

My gateway configuration then includes the following critical components:
ip multicast-routing distributed
ip source-address port 2000
moh enable-g711 "flash:/SampleAudioSource.ulaw.wav"
multicast moh port 16384

Here my gateway's LAN IP address is and I have chosen to stream Cisco's sample music from flash on the gateway.  If you are interested, here's my tips on how to download music from Cisco's music on hold servers to use elsewhere:  It's the technique I used get the sample music file off the server and on the gateway.

If all goes as planned, when a caller is put on hold you can see evidence of it being streamed from the gateway via the "show ccm-manager music-on-hold" command.
! with no calls on hold

my-router#show ccm-manager music-on-hold
Current active multicast sessions : 0

! with one call on hold
my-router#show ccm-manager music-on-hold
Current active multicast sessions : 1
 Multicast       RTP port   Packets       Call   Codec    Incoming
 Address         number     in/out        id              Interface
===================================================================        16384   0/0              25939 g711ulaw            

Friday, June 17, 2016

UCCX 11 Notes

The official Cisco install guide indicates the server hostname is case sensitive and must be lower case.  I suspect this is to alleviate issues with installing signed certificates later with all lower case SANs.

When modifying "Finesse Layout XML" it appears restarting TomCat or Finesse services is no longer required.  Users seem to have to log out and back in to see changes though.

If you are deploying UCCX 11.0 you should plan to hit bug CSCux33949.  Your default Finesse gadgets will all be broken.  At this time the fix requires TAC gaining root access, providing some new JAR files and restarting the UCCX Engine.  Budget time to have this fixed early in your implementation plan.

RTMT cannot be used to show telephony data on versions 11 or 11.5 per bug CSCuy72411.  We won't be able to use it to check triggers, call control groups and CTI ports until who knows when.

UCCX Editor is not compatible with Windows 8 or 8.1.

The Team Performance gadget served by /desktop/gadgets/TeamPerformance.jsp is not resizable in version 11.0.  The rumor is it will be in 11.5.

Friday, April 22, 2016

CUCM Authentication URL format

When testing authentication against CUCM v10 using the Authentication URL, here is the format:

Thursday, April 21, 2016

Singlwire Informacast Resiliency feature

Somewhere around version 9 Singlewire introduced a "resiliency" feature where you no longer had to depend on a single server to provide paging, alerting, bells, etc. to your masses.  You can now deploy multiple backup servers (much like CUCM, called subscribers) that would perform all the paging functions in the event the publisher or higher order subscribers failed.  Now that they are on version 11 and I haven't deployed one since about version 8, here are few tips to get you over the resiliency feature deployment that I bumped into today.

1. Send Commands to Phones by JTAPI

If you are familiar with the single server deployment model or audio paging solutions from other vendors, you may be familiar with the suggestion they all make to modify the CUCM cluster's Authentication URL.  The Singlewire documentation still addresses this in multiple places, which is good as it is a common point of failure in these types of deployments.  Singlewire and others will strongly recommend you change the default CUCM Authentication URL to a custom URL pointing to your fancy new paging server instead.  The idea here is to have all the phones authenticate to the paging server when they are sent a page request rather than potentially overloading the CUCM Publisher with those same authentication requests.  You don't want thousands of phones smashing on CUCM's Tomcat door just because your receptionist wants to announce she found a pair of sunglasses at the front desk. 

The challenge is now that if you have redundant paging servers, what do you use for the new Authentication URL?  Normally you would use a URL that points to the Singlewire server.  But now you have two (or three or whatever). If your first paging server fails and the second takes over all the paging functions, your phones are still going to look for the first paging server to authenticate against and nothing will work even if subscribers are up and healthy because there is no paging server to authenticate the page requests.

What do you do?

You use JTAPI instead of HTML requests.  HTML sounds way simpler to me and it's been how things were done for some time now.  Why not stick with it?  Quietly you will find the Singlwire subscriber servers use JTAPI regardless of how you've chosen to authenticate requests from the Singlewire publisher.  This solves the problem of what URL to change the CUCM Authentication URL to as you can now set the one and only option to your one and only paging server using HTML requests, the Singlewire publisher.  But since the subscribers use JTAPI rather than HTML, why not use JTAPI on the publisher as well?  Just do it.  When you buy another administration bolt on tool and it wants you to change the Authentication URL to itself as well you'll thank me.  How many daisy chained authentication server requests do you need?

On the publisher you will find the single checkbox option under Admin | Broadcast Parameters.

2. Unified Communications Application User

Having installed you subscriber and it looking healthy you decide to shutdown the publisher and actually test the resiliency function.  You dial a CUCM route pattern that was working perfectly when the publisher was up but now hear something like "We're sorry. No devices could be activated. Your broadcast will not be completed."  You know your SNMP and AXL settings are correct since your pages worked through the publisher and you know it's not a HTML authentication issue since you are using JTAPI now so what is the problem?

Although most of the configuration information in a resilient Singlewire solution is replicated from the publisher to the subscribers, the CUCM Application User settings are not. Why? Don't know.  They just aren't.

Log into your subscriber server HTML interface where you typically are prevented from really doing anything and set your CUCM Application User credentials there.  Go to Admin | Telephony | Cisco Unified Communications Manager Cluster | Edit Telephony Configuration and enter the user name and password there.

3. Stop / Restart the singlewireInformaCast service

How do you replicate a server failure to test resiliency?  You could shut down the whole server, maybe use a fancy network ACL, maybe just pull an Ethernet cable, whatever, because you just can't find a tool in the administrative interface to stop or restart services. 

From the main administration page, choose the Webadmin option.  This wil take you to another log in and adminstrative interface.  From there choose System | Bootup and Shutdown.  That's right.  You can simply stop and start services manually from the option labelled Bootup and Shutdown.  This is not terribly intuitive as far as I'm concerned but it works.  Scroll down the service list and click singlewireInformaCast.  You can Stop Now or Start Now on the Publisher server to simulate a server outage.

Here's what it looks like if you're in the right place:

Tuesday, March 29, 2016

Using Cisco UCCX CUIC as a wallboard and CSCun28885 Tomcat inactivity timeouts

A co-worker of mine was involved in a new Cisco UCCX roll-out where the client was looking to use a CUIC report on a large LED flat screen as a wallboard solution.  After creating the report, running it via a permalink and adjusting screen resolution and Internet Explorer zooming to make the report readable, the client found the result to be satisfactory.  They unfortunately realized that because Cisco Tomcat has a maximum inactivity timeout of 14400 seconds, their new wallboard would break overnight if there was no activity in the call center and CUIC would need to be shutdown and restarted every AM.

You can find some background here:

From some email exchanges it looked like every 4 hours or AM someone would need to:
  1. Kill Internet Explorer
  2. Start Internet Explorer back up.
  3. Navigate to the CUIC dashboard permalink.
  4. Login to the CUIC dashboard.
Being pressed for time he asked for help and here's the solution, as dirty as it may be, I came up with.

Powershell script to stop IE, start IE, navigate to a web page and log in:

# Script closes instances of IE,
# opens a new visible instance,
# navigates to a URL
# and logs into CUIC
# NOTE: user name and password stored here in clear text
# WebMaxtor
# 03/28/2016

# Edit this $Url to be the URL or IP address of the site to launch
# This could be the CUIC logon page or a dashboard permalink
# This assumes there will be no challenge to accept certificates
# Typical CUIC logon page URL below as example
$Url = ''
# Edit this to be the username
# Edit this to the corresponding password

# Close IE to eliminate timed out reports
# option 1:
# Get-Process iexplore | Foreach-Object { $_.CloseMainWindow() }
# option 2:
#(New-Object -COM "Shell.Application").Windows() |
#  ? { $_.Name -like "*Internet Explorer*" } |
#  % { $_.Quit() }
# option 3:
Get-Process iexplore | Stop-Process

# Wait a few seconds to be safe for processes stop if need be
while ($IE.Busy -eq $true)
Start-Sleep -Milliseconds 2000;

# Start a visible Internet Explorer instance
$IE = New-Object -com internetexplorer.application;
$IE.visible = $true;

# Wait a few seconds and then log on
while ($IE.Busy -eq $true)
Start-Sleep -Milliseconds 2000;

# fill in the form and click submit button
$IE.Document.getElementById('j_username').value = $Username

Windows Batch file to make running powershell script easier:

PowerShell.exe -NoProfile -Command "& {Start-Process PowerShell.exe -ArgumentList '-NoProfile -ExecutionPolicy Bypass -File ""%~dpn0.ps1""' -Verb RunAs}"

The beauty of the batch file is if the powershell script and the batch file are named the same except for the file extension (i.e. MyFile.bat and MyFile.ps1) and reside in the same folder, you can just click the BAT file and you are off and running.

There are multiple methods to close IE via powershell and I can't be sure what will work best in your environment so be sure to test the three options or come up with your own.

The final step was then calling the BAT file from Window's Scheduled Tasks on a regular basis to ensure the report was not effected by the Tomcat inactivity timer.