IPCelerate IPSession, Status Solutions, Dukane nurse call and stattap.exe crashing

Running IPCelerate IPSession version 5.8.4 with a Dukane nurse call integration.  Status Solutions provides the interface between the Dukane TAP page output and IPCelerate SAM.

After the server hung during a Microsoft update application it was hard booted. At first the problem was that alerts designed to be presented to a single 7925 WiFi phone were being presented to all phones in the IPSession page group.  Then I found the stattap.exe application (the "Status Solutions TAP Interface" service) was crashing shortly after startup.  Several pending alerts could be sent and then the stattap.exe would crash.  A symptom was that the IPCelerate SAM web GUI would display a "Got error 134 from table handler", rather than a grid of recent alerts.

Ultimately, the issue was a corrupted alarm_device table in the MySQL statsol database.

You can gain access to the MySQL database by running:
c:\ipcelerate\statsol\mysql\bin\mysql -u root -pstatsoldb statsol


You can see the tables in the statsol database by using:
SHOW TABLES;

The output should look similar to the following:
+-----------------------------+
| Tables_in_statsol           |
+-----------------------------+
| action                      |
| alarm                       |
| alarm_device                |
| alarm_device_image          |
| alarm_device_type           |
| alarm_log                   |
| alarm_log_2000_q1           |

The list continues but it's not shown here...

With that, wanting to check tables without being potentially destructive and having the most control during the process, I started checking tables by running:
CHECK TABLE ;

Given the number of tables in the list, I was happy to find the alarm_device table was an offender.  The check, repair and recheck output is shown below:

The bad check...

mysql> check table alarm_device;
+----------------------+-------+----------+-------------------------------------
---------------------+
| Table                | Op    | Msg_type | Msg_text
                     |
+----------------------+-------+----------+-------------------------------------
---------------------+
| statsol.alarm_device | check | warning  | 3 clients are using or haven't close
d the table properly |
| statsol.alarm_device | check | error    | Unexpected byte: 0 at link: 179536
                     |
| statsol.alarm_device | check | error    | Corrupt
                     |
+----------------------+-------+----------+-------------------------------------
---------------------+
3 rows in set (0.05 sec)

The repair...

mysql> repair table alarm_device;
+----------------------+--------+----------+------------------------------------
------+
| Table                | Op     | Msg_type | Msg_text
      |
+----------------------+--------+----------+------------------------------------
------+
| statsol.alarm_device | repair | warning  | Number of rows changed from 1645 to
 1644 |
| statsol.alarm_device | repair | status   | OK
      |
+----------------------+--------+----------+------------------------------------
------+
2 rows in set (0.09 sec)

A good check...

mysql> check table alarm_device;
+----------------------+-------+----------+----------+
| Table                | Op    | Msg_type | Msg_text |
+----------------------+-------+----------+----------+
| statsol.alarm_device | check | status   | OK       |
+----------------------+-------+----------+----------+
1 row in set (0.01 sec)

After some discussions with Status Solutions support staff, they confirmed that the alarm_device table is a common issue after unclean reboots.  They also indicated running mysqlcheck should not cause any issues while MySQL is running.

A much more efficient command is then:
C:\IPcelerate\statsol\mysql\bin\mysqlcheck --repair --check-only-changed -uroot -pstatsoldb statsol

The output should be similar to this:
statsol.action                                     Table is already up to date
statsol.alarm                                      OK
statsol.alarm_device                               OK
statsol.alarm_device_image                         Table is already up to date
statsol.alarm_device_type                          OK
statsol.alarm_log                                  OK
statsol.alarm_log_2000_q1                          Table is already up to date

The list continues below but is not shown...

mysqlcheck appears to run the manual check and appropriate repair in one process, as shown here:
http://dev.mysql.com/doc/refman/5.0/en/mysqlcheck.html

Cisco UCCX Application Manager in partial service

See here or he like for typical troubleshooting techniques: https://supportforums.cisco.com/thread/2131666

Problem: You find your Cisco UCCX server(s) in partial service, and after drilling down find the Application Manager is in partial service.  One option is to change the trace levels to debug on the APP_MGR, restart the engine(s) or server(s) and then examine the MIVR logs.  Searching the logs for PARTIAL_SERVICE or ERROR at the time of reboots can be telling as it may reference an offending script.  At at high profile installations, restarting / rebooting is not a practical option, and pulling logs from the Linux appliances is not as easy as it was on the older Windows based systems.

Possible easy solution:  Check the Real Time Reporting in UCCX.  From the administrative GUI, choose Tools | Real Time Reporting | Report | Applications.  You should find a grid of the applications on your deployment and a column on the right titled Valid.

If any of them have a value of false, you can be sure the partial service is at least partially attributable to that application.  Validating the script via the script editor, checking for valid sub flows, etc. is in order.

If you don't "own" the administration of UCCX and have various administrators not checking server health after their changes, this is a nice easy option, pointed out to me by a customer in just such an environment.  It appears to be available in at least versions 7 through 9.

Cisco CUCM check database replication via CLI basics

Cisco has made reviewing CUCM server replication on newer versions pretty easy via the GUI. Choose Cisco Unified Reporting in the upper right, log in and choose System Reports | Unified CM Database Status and then choose Generate a new report.

You can also look for 2s in RTMT if that is available.

Sometimes the CLI is the only way to go, and I never remember the exact commands.

Check the DB replication status on all Cisco Unified Communications Manager nodes in the cluster to ensure that all servers are replicating database changes successfully. You can check by using either RTMT or a CLI command.

via CLI:
show perf query class "Number of Replicates Created and State of Replication"

– 0—Replication Not Started. Either no subscribers exist, or the Database Layer Monitor service has not been running since the subscriber was installed.
– 1—Replicates have been created, but their count is incorrect.
– 2—Replication is good.
– 3—Replication is bad in the cluster.
– 4—Replication setup did not succeed.

Just as an aside, there's another CLI command I like, where validate_network is not available via the question mark.  To check network connectivity and DNS server configuration, enter the CLI command below:
utils diagnose module validate_network

Upgrades are prohibited during License Grace Period and licexpiry.txt location

You might sometime find yourself in the position where you need to manipulate files on Cisco CUCM, such as the licexpiry.txt file. Some background on that particular situation can be found here: https://supportforums.cisco.com/message/3923028#3923028

This is something you need to have TAC assist with, as they have methods to generate temporary root access to the Linux OS and these files can't be accessed via the normal platform administration CLI commands.

You may also realize there are plenty of documented methods to gain root access on a Cisco CUCM server yourself, but my favorite, in that it includes good screen shots and Vi text editing keystrokes help can be found here: http://htluo.blogspot.com/2011/08/root-access-on-cucm-and-other-uc.html

Of course, you shouldn't do this.  It's unsupported by Cisco, Cisco TAC, likely voids warranties and service contracts, could be illegal, and who knows what else.  Don't do it.

When Cisco TAC is assisting you, you might find some of the documentation associated with removing the licexpiry.txt file is incorrect.

So...

To find the actual location of the file, you Cisco TAC could run:

find / -iname licexpiry.txt

This should return the real location of the file, something like:

/usr/local/platform/conf/licexpiry.txt

With that information, to remove the file you can run:

cd /usr/local/platform/conf
rm licexpiry.txt

Good luck, maniacs.

Edit 9/30/2013: see http://webmaxtor.blogspot.com/2013/09/upgrades-are-prohibited-during-license.html for some additional information on gaining access to the CUCM file system.

Convert WAV to 8 bit, 8 KHz u-Law format, version 2

So you have a wonderful new recording you would like to upload to Cisco UCCX (or IPCC, Unity Connection, Unity Express, or whatever) and know that it needs to be in a 8bit, 8KHz, u-Law format.

How do you convert it?

In Windows XP, the Sound Recorder application would allow you to simply change the format by opening it, choosing to save it and then selecting Change in the dialog box. The correct format there is Format: CCITT u-Law and Attributes: 8.000 khz, 8 Bit, Mono.

Now that Microsoft has apparently sucked the life out of Sound Recorder in Windows 7, my next preference is using Audacity.  Unfortunately Audacity, while extremely feature rich, can be a bit overkill for this purpose.

If you are so inclined, you can find my original post regarding how to convert files in Audacity here: http://webmaxtor.blogspot.com/2012/05/convert-wav-to-8-bit-8-khz-u-law-format.html

My new preference is using the Windows XP version of Sound Recorder on Windows 7.

What?! How, you say?

The first step is to find an XP machine that you can steal files from. You can find the Sound Recorder executable on a typical XP machine at C:\WINDOWS\system32\sndrec32.exe.  Copy that file and move it to your Win 7 machine's C:\WINDOWS\system32 folder.

Edit 09/14/2015:  You can find a copy of the 32 bit Sound Recorder application from Windows XP by searching for sndrec32.zip here: https://sites.google.com/site/raymaslanka/

Now, running that executable may pop an error indicating "There was an error updating the registry".

Here I suggest you:

• Close the message box.
• Close the Sound Recorder application.
• Right click the sndrec32.exe and choose "Run as Administrator".
• The XP version of Sound Recorder should run without issue.
• Close the application and simply run or open it again, not as an administrator.

If it runs successfully, you should be all set.

To make life easy, now:

• Right click sndrec32.exe and choose Send to | Desktop (create Shortcut)
• Rename the short cut as Sound Recorder XP, or something similar.
• Drag that short cut to the Windows start bubble, over All Programs, over Accessories and wait for the accessories list to expand.  
• Then drop it in the list of accessories.

You should then be able to search for, find and run the Sound Recorder XP application with the CCITT u-Law options, as well as the newer version if you'd like.

TaDa.