- Setup a SFTP server somewhere
- Console or SSH to CUCM
- Run "file get install /*"
- When prompted, enter SFTP server address, username, and password
- Use "." as the directory path (or get fancier if you want)
Saturday, January 07, 2012
Cisco CUCM install logs via CLI
To get the install logs from Cisco CUCM without RTMT:
Wednesday, January 04, 2012
Cisco Switch Smartport configuration via CLI
After barking up the wrong troubleshooting tree for some time, I ran into a situation where a customer needed a UC540 switchport configured for trunking to a Linksys switch hiding in a ceiling. Due to VPN network configurations out of my control, I was unable to use Cisco CCA to modify it. Knowing using the CLI is somewhat of a no-no on UC540s, I wanted to apply the Smartport configurations as if they had been done so via CCA, verses just reconfiguring as if it was any other manageable switch.
You can display the macros used (I think) by CCA as follows:
! To show a short list of available macros, run:
!
UC540#show parser macro brief
!
! This returns:
!
default interface: cisco-desktop
default interface: cisco-phone
default interface: cisco-switch
default interface: cisco-router
default interface: cisco-wireless
! To show the macro details and variables that need to be set, run:
!
UC540#show parser macro
!
! This returns
!
Total number of macros = 5
--------------------------------------------------------------
Macro name : cisco-desktop
Macro type : default interface
# macro keywords $access_vlan
# Basic interface - Enable data VLAN only
# Recommended value for access vlan should not be 1
switchport access vlan $access_vlan
switchport mode access
# Configure port as an edge network port
spanning-tree portfast
--------------------------------------------------------------
Macro name : cisco-phone
Macro type : default interface
# Cisco IP phone + desktop template
# macro keywords $access_vlan $voice_vlan
# VoIP enabled interface - Enable data VLAN
# and voice VLAN
# Recommended value for access vlan should not be 1
switchport access vlan $access_vlan
switchport mode access
# Update the Voice VLAN value which should be
# different from data VLAN
# Recommended value for voice vlan should not be 1
switchport voice vlan $voice_vlan
# Configure port as an edge network port
spanning-tree portfast
--------------------------------------------------------------
Macro name : cisco-switch
Macro type : default interface
# macro keywords $native_vlan
# Access Uplink to Distribution
# Do not apply to EtherChannel/Port Group
switchport trunk encapsulation dot1q
# Define unique Native VLAN on trunk ports
# Recommended value for native vlan should not be 1
switchport trunk native vlan $native_vlan
# Update the allowed VLAN range such that it
# includes data, voice and native VLANs
switchport trunk allowed vlan ALL
# Hardcode trunk
switchport mode trunk
--------------------------------------------------------------
Macro name : cisco-router
Macro type : default interface
# macro keywords $native_vlan
# Access Uplink to Distribution
switchport trunk encapsulation dot1q
# Define unique Native VLAN on trunk ports
# Recommended value for native vlan should not be 1
switchport trunk native vlan $native_vlan
# Update the allowed VLAN range such that it
# includes data, voice and native VLANs
switchport trunk allowed vlan ALL
# Hardcode trunk
switchport mode trunk
# Ensure fast access to the network when enabling the interface.
# Ensure that switch devices cannot become active on the interface.
spanning-tree portfast
--------------------------------------------------------------
Macro name : cisco-wireless
Macro type : default interface
# macro keywords $native_vlan
# Access Uplink to Distribution
switchport trunk encapsulation dot1q
# Define unique Native VLAN on trunk ports
# Recommended native vlan should NOT be 1
switchport trunk native vlan $native_vlan
# Update the allowed VLAN range such that it
# includes data, voice and native VLANs
switchport trunk allowed vlan ALL
# Hardcode trunk to speed up convergence
switchport mode trunk
# Ensure that switch devices cannot become active on the interface.
spanning-tree portfast
--------------------------------------------------------------
! To apply a macro, make note of the macro name and keyword
!
! For example, the cisco-switch macro has a $native_vlan keyword
!
! Then, after removing commands already specific to the port, run (assuming configuring for a switch with native VLAN 1):
!
UC540(config)#int f0/1/2
UC540(config-if)#macro apply cisco-switch $native_vlan 1
UC540(config-if)#macro description cisco-switch
The macros don't do anything I wouldn't normally do, but at least keeps your CCA and CLI configurations consistent.
Pretty cool.
You can display the macros used (I think) by CCA as follows:
! To show a short list of available macros, run:
!
UC540#show parser macro brief
!
! This returns:
!
default interface: cisco-desktop
default interface: cisco-phone
default interface: cisco-switch
default interface: cisco-router
default interface: cisco-wireless
! To show the macro details and variables that need to be set, run:
!
UC540#show parser macro
!
! This returns
!
Total number of macros = 5
--------------------------------------------------------------
Macro name : cisco-desktop
Macro type : default interface
# macro keywords $access_vlan
# Basic interface - Enable data VLAN only
# Recommended value for access vlan should not be 1
switchport access vlan $access_vlan
switchport mode access
# Configure port as an edge network port
spanning-tree portfast
--------------------------------------------------------------
Macro name : cisco-phone
Macro type : default interface
# Cisco IP phone + desktop template
# macro keywords $access_vlan $voice_vlan
# VoIP enabled interface - Enable data VLAN
# and voice VLAN
# Recommended value for access vlan should not be 1
switchport access vlan $access_vlan
switchport mode access
# Update the Voice VLAN value which should be
# different from data VLAN
# Recommended value for voice vlan should not be 1
switchport voice vlan $voice_vlan
# Configure port as an edge network port
spanning-tree portfast
--------------------------------------------------------------
Macro name : cisco-switch
Macro type : default interface
# macro keywords $native_vlan
# Access Uplink to Distribution
# Do not apply to EtherChannel/Port Group
switchport trunk encapsulation dot1q
# Define unique Native VLAN on trunk ports
# Recommended value for native vlan should not be 1
switchport trunk native vlan $native_vlan
# Update the allowed VLAN range such that it
# includes data, voice and native VLANs
switchport trunk allowed vlan ALL
# Hardcode trunk
switchport mode trunk
--------------------------------------------------------------
Macro name : cisco-router
Macro type : default interface
# macro keywords $native_vlan
# Access Uplink to Distribution
switchport trunk encapsulation dot1q
# Define unique Native VLAN on trunk ports
# Recommended value for native vlan should not be 1
switchport trunk native vlan $native_vlan
# Update the allowed VLAN range such that it
# includes data, voice and native VLANs
switchport trunk allowed vlan ALL
# Hardcode trunk
switchport mode trunk
# Ensure fast access to the network when enabling the interface.
# Ensure that switch devices cannot become active on the interface.
spanning-tree portfast
--------------------------------------------------------------
Macro name : cisco-wireless
Macro type : default interface
# macro keywords $native_vlan
# Access Uplink to Distribution
switchport trunk encapsulation dot1q
# Define unique Native VLAN on trunk ports
# Recommended native vlan should NOT be 1
switchport trunk native vlan $native_vlan
# Update the allowed VLAN range such that it
# includes data, voice and native VLANs
switchport trunk allowed vlan ALL
# Hardcode trunk to speed up convergence
switchport mode trunk
# Ensure that switch devices cannot become active on the interface.
spanning-tree portfast
--------------------------------------------------------------
! To apply a macro, make note of the macro name and keyword
!
! For example, the cisco-switch macro has a $native_vlan keyword
!
! Then, after removing commands already specific to the port, run (assuming configuring for a switch with native VLAN 1):
!
UC540(config)#int f0/1/2
UC540(config-if)#macro apply cisco-switch $native_vlan 1
UC540(config-if)#macro description cisco-switch
The macros don't do anything I wouldn't normally do, but at least keeps your CCA and CLI configurations consistent.
Pretty cool.
Monday, December 05, 2011
UC520 and UC540 load 8.2.0 and CUE voicemail message delivery log samples with GMail TLS / SSL failures
The trace commands used on CUE are:
no trace all
clear trace
trace voicemail msgnotif all
trace configapi smtp debug
trace entitymanager NotifDevice all
trace smtp all
show trace buffer tail
Below is a successful voicemail notification to ray.maslanka@gmail.com, using smtp-server.roadrunner.com on port 25.
The WAN port is directly connected to a RoadRunner cable modem with a public IP address.
The UC520 firewall is set to "Low" via the CCA GUI and no command line modifications have been made.
4414 12/05 08:39:51.938 VMSS mnot 0 MessageNotification: Enter insertIntoMsgNotifQueue(): msg recipient: UOnenotif target: UOne
4414 12/05 08:39:51.976 capi smtp 0 SmtpServer: getSysdb(): Attribute: address
2322 12/05 08:39:51.977 capi smtp 0 SmtpSysdbNode: get(): address
4414 12/05 08:39:51.978 VMSS mnot 0 MessageNotification: query: INSERT INTO outcall_email_job VALUES ('UOne', 108, 1323092391940, 15, 0, 'UOne');
4414 12/05 08:39:51.996 VMSS mnot 0 MessageNotification: Notifying sender threads
3021 12/05 08:39:52.005 VMSS mnot 0 EmailSender: Processing job: 1
4414 12/05 08:39:52.006 VMSS mnot 0 MessageNotification: Exit insertIntoMsgNotifQueue(): notifyEmailThread=true, notifyPhoneThread=false
3021 12/05 08:39:52.113 VMSS mnot 0 EmailSender: Got a job: [owner_id=UOne, contact=ray.maslanka@gmail.com, extra_text=null, nextSendTime=1323092391940, device_type=32, device_id=108, retryCount=0, targetId=UOne, attach_vm=true, isUrgent=false, isPrivate=false, ndrreason=0, uid=15, messagetype=1, preferenceA, sender=203, callerName=User Three
3021 12/05 08:39:52.118 VMSS mnot 0 EmailSender: sendEmailNotification: checkSendPreConditions passed
3021 12/05 08:39:52.118 capi smtp 0 SmtpServer: getSysdb(): Attribute: address
2297 12/05 08:39:52.119 capi smtp 0 SmtpSysdbNode: get(): address
3021 12/05 08:39:52.119 capi smtp 0 SmtpServer: getSysdb(): Attribute: port
2307 12/05 08:39:52.120 capi smtp 0 SmtpSysdbNode: get(): port
3021 12/05 08:39:52.121 capi smtp 0 SmtpServer: getSysdb(): Attribute: userid
2322 12/05 08:39:52.122 capi smtp 0 SmtpSysdbNode: get(): userid
3021 12/05 08:39:52.122 capi smtp 0 SmtpServer: getSysdb(): Attribute: password
2318 12/05 08:39:52.123 capi smtp 0 SmtpSysdbNode: get(): password
3021 12/05 08:39:52.123 capi smtp 0 SmtpServer: getSysdb(): Attribute: authRequired
2297 12/05 08:39:52.124 capi smtp 0 SmtpSysdbNode: get(): authRequired
3021 12/05 08:39:52.130 VMSS mnot 0 EmailSender: Begin processing email job, UID=15
3021 12/05 08:39:52.158 VMSS mnot 0 EmailSender: Attachment body part added, UID=15
3021 12/05 08:39:53.476 VMSS mnot 0 EmailSender: Sent successfully, UID=15
3021 12/05 08:39:53.476 VMSS mnot 0 EmailSender: Send email rc=1
3021 12/05 08:39:53.476 VMSS mnot 0 EmailSender: Deleting email job: Device ID=108, UID=15
3021 12/05 08:39:53.495 VMSS mnot 0 EmailSender: calculateWaitTime(): current time: 1323092393492, job time: 0
3021 12/05 08:39:53.495 VMSS mnot 0 EmailSender: Wait time=0
Below is a failed voicemail notification to ray.maslanka@gmail.com, using smtp.gmail.com on port 587.
The CUE is configured to require SMTP authentication, and again using ray.maslanka@gmail.com.
Note the errors related specifically to TLS, a logical result of using port 587.
4414 12/05 09:11:59.431 VMSS mnot 0 MessageNotification: Enter insertIntoMsgNotifQueue(): msg recipient: UOnenotif target: UOne
4414 12/05 09:11:59.469 capi smtp 0 SmtpServer: getSysdb(): Attribute: address
2297 12/05 09:11:59.470 capi smtp 0 SmtpSysdbNode: get(): address
4414 12/05 09:11:59.470 VMSS mnot 0 MessageNotification: query: INSERT INTO outcall_email_job VALUES ('UOne', 108, 1323094319432, 17, 0, 'UOne');
4414 12/05 09:11:59.493 VMSS mnot 0 MessageNotification: Notifying sender threads
3021 12/05 09:11:59.502 VMSS mnot 0 EmailSender: Processing job: 1
4414 12/05 09:11:59.512 VMSS mnot 0 MessageNotification: Exit insertIntoMsgNotifQueue(): notifyEmailThread=true, notifyPhoneThread=false
3021 12/05 09:11:59.602 VMSS mnot 0 EmailSender: Got a job: [owner_id=UOne, contact=ray.maslanka@gmail.com, extra_text=null, nextSendTime=1323094319432, device_type=32, device_id=108, retryCount=0, targetId=UOne, attach_vm=true, isUrgent=false, isPrivate=false, ndrreason=0, uid=17, messagetype=1, preferenceA, sender=203, callerName=User Three
3021 12/05 09:11:59.606 VMSS mnot 0 EmailSender: sendEmailNotification: checkSendPreConditions passed
3021 12/05 09:11:59.606 capi smtp 0 SmtpServer: getSysdb(): Attribute: address
2297 12/05 09:11:59.607 capi smtp 0 SmtpSysdbNode: get(): address
3021 12/05 09:11:59.607 capi smtp 0 SmtpServer: getSysdb(): Attribute: port
2307 12/05 09:11:59.608 capi smtp 0 SmtpSysdbNode: get(): port
3021 12/05 09:11:59.609 capi smtp 0 SmtpServer: getSysdb(): Attribute: userid
2322 12/05 09:11:59.610 capi smtp 0 SmtpSysdbNode: get(): userid
3021 12/05 09:11:59.610 capi smtp 0 SmtpServer: getSysdb(): Attribute: password
2318 12/05 09:11:59.611 capi smtp 0 SmtpSysdbNode: get(): password
3021 12/05 09:11:59.612 capi smtp 0 SmtpServer: getSysdb(): Attribute: authRequired
2297 12/05 09:11:59.613 capi smtp 0 SmtpSysdbNode: get(): authRequired
3021 12/05 09:11:59.622 VMSS mnot 0 EmailSender: Begin processing email job, UID=17
3021 12/05 09:11:59.654 VMSS mnot 0 EmailSender: Attachment body part added, UID=17
3021 12/05 09:12:00.339 VMSS mnot 0 EmailSender: Error sending email, UID= 17 :javax.mail.SendFailedException: Sending failed;
nested exception is:
class javax.mail.MessagingException: 530 5.7.0 Must issue a STARTTLS command first. v18sm77175183ibh.4
3021 12/05 09:12:00.339 VMSS mnot 0 EmailSender: Send email rc=2
3021 12/05 09:12:00.339 VMSS mnot 0 EmailSender: Deleting email job: Device ID=108, UID=17
3021 12/05 09:12:00.362 VMSS mnot 0 EmailSender: calculateWaitTime(): current time: 1323094320360, job time: 0
3021 12/05 09:12:00.362 VMSS mnot 0 EmailSender: Wait time=0
Below is a failed voicemail notification to ray.maslanka@gmail.com, using smtp.gmail.com on port 465.
The CUE is configured to require SMTP authentication, and again using ray.maslanka@gmail.com.
Note the errors related "Exception reading response", not a very descriptive error but related to the 465 / SSL authentication.
4414 12/05 09:18:08.994 VMSS mnot 0 MessageNotification: Enter insertIntoMsgNotifQueue(): msg recipient: UOnenotif target: UOne
4414 12/05 09:18:09.083 capi smtp 0 SmtpServer: getSysdb(): Attribute: address
2297 12/05 09:18:09.088 capi smtp 0 SmtpSysdbNode: get(): address
4414 12/05 09:18:09.088 VMSS mnot 0 MessageNotification: query: INSERT INTO outcall_email_job VALUES ('UOne', 108, 1323094689041, 18, 0, 'UOne');
4414 12/05 09:18:09.120 VMSS mnot 0 MessageNotification: Notifying sender threads
4414 12/05 09:18:09.120 VMSS mnot 0 MessageNotification: Exit insertIntoMsgNotifQueue(): notifyEmailThread=true, notifyPhoneThread=false
3021 12/05 09:18:09.194 VMSS mnot 0 EmailSender: Processing job: 1
3021 12/05 09:18:09.212 VMSS mnot 0 EmailSender: Got a job: [owner_id=UOne, contact=ray.maslanka@gmail.com, extra_text=null, nextSendTime=1323094689041, device_type=32, device_id=108, retryCount=0, targetId=UOne, attach_vm=true, isUrgent=false, isPrivate=false, ndrreason=0, uid=18, messagetype=1, preferenceA, sender=203, callerName=User Three
3021 12/05 09:18:09.217 VMSS mnot 0 EmailSender: sendEmailNotification: checkSendPreConditions passed
3021 12/05 09:18:09.217 capi smtp 0 SmtpServer: getSysdb(): Attribute: address
2322 12/05 09:18:09.218 capi smtp 0 SmtpSysdbNode: get(): address
3021 12/05 09:18:09.218 capi smtp 0 SmtpServer: getSysdb(): Attribute: port
2297 12/05 09:18:09.220 capi smtp 0 SmtpSysdbNode: get(): port
3021 12/05 09:18:09.220 capi smtp 0 SmtpServer: getSysdb(): Attribute: userid
2318 12/05 09:18:09.221 capi smtp 0 SmtpSysdbNode: get(): userid
3021 12/05 09:18:09.222 capi smtp 0 SmtpServer: getSysdb(): Attribute: password
2307 12/05 09:18:09.223 capi smtp 0 SmtpSysdbNode: get(): password
3021 12/05 09:18:09.223 capi smtp 0 SmtpServer: getSysdb(): Attribute: authRequired
2322 12/05 09:18:09.224 capi smtp 0 SmtpSysdbNode: get(): authRequired
3021 12/05 09:18:09.230 VMSS mnot 0 EmailSender: Begin processing email job, UID=18
3021 12/05 09:18:09.261 VMSS mnot 0 EmailSender: Attachment body part added, UID=18
3021 12/05 09:20:09.532 VMSS mnot 0 EmailSender: Error sending email, UID= 18 :javax.mail.SendFailedException: Sending failed;
nested exception is:
class javax.mail.MessagingException: Exception reading response;
nested exception is:
java.net.SocketTimeoutException: Read timed out
3021 12/05 09:20:09.532 VMSS mnot 0 EmailSender: Send email rc=2
3021 12/05 09:20:09.532 VMSS mnot 0 EmailSender: Deleting email job: Device ID=108, UID=18
3021 12/05 09:20:09.560 VMSS mnot 0 EmailSender: calculateWaitTime(): current time: 1323094809558, job time: 0
3021 12/05 09:20:09.560 VMSS mnot 0 EmailSender: Wait time=0
Below is the software version information related to the UC520 CUE.
Unfortunately the CUE release notes at http://www.cisco.com/en/US/docs/voice_ip_comm/unity_exp/rel8_6/rel_notes/rel_notes86.html#wp364157 indicate SMTP over TLS and SSL to GMail, etc is supported first in version 8.6.
show software versions
Cisco Unity Express version (8.0.6)
Technical Support: http://www.cisco.com/techsupport Copyright (c) 1986-2010 by Cisco Systems, Inc.
Components:
- TimeCardView version 8.0.6
- PhoneConnect version 7.0.0
- CUE Voicemail Language Support version 8.0.6
- Monitor Agent version 7.0.0
- Madrox Multi-app Support version 8.0.6
There are multiple conversations on the Internet re: using GMail's SMTP server(s) as voicemail message delivery servers, but none with detailed resolutions.
A request for feedback in the Cisco supportforums re: the software version and related errors can be found here, with no replies at the time of this writing: https://supportforums.cisco.com/message/3503487#3503487
My strategy at this time will be to assume voicemail message delivery from UC5XX with software load 8.2.0 is not possible via GMail's SMTP servers.
no trace all
clear trace
trace voicemail msgnotif all
trace configapi smtp debug
trace entitymanager NotifDevice all
trace smtp all
show trace buffer tail
Below is a successful voicemail notification to ray.maslanka@gmail.com, using smtp-server.roadrunner.com on port 25.
The WAN port is directly connected to a RoadRunner cable modem with a public IP address.
The UC520 firewall is set to "Low" via the CCA GUI and no command line modifications have been made.
4414 12/05 08:39:51.938 VMSS mnot 0 MessageNotification: Enter insertIntoMsgNotifQueue(): msg recipient: UOnenotif target: UOne
4414 12/05 08:39:51.976 capi smtp 0 SmtpServer: getSysdb(): Attribute: address
2322 12/05 08:39:51.977 capi smtp 0 SmtpSysdbNode: get(): address
4414 12/05 08:39:51.978 VMSS mnot 0 MessageNotification: query: INSERT INTO outcall_email_job VALUES ('UOne', 108, 1323092391940, 15, 0, 'UOne');
4414 12/05 08:39:51.996 VMSS mnot 0 MessageNotification: Notifying sender threads
3021 12/05 08:39:52.005 VMSS mnot 0 EmailSender: Processing job: 1
4414 12/05 08:39:52.006 VMSS mnot 0 MessageNotification: Exit insertIntoMsgNotifQueue(): notifyEmailThread=true, notifyPhoneThread=false
3021 12/05 08:39:52.113 VMSS mnot 0 EmailSender: Got a job: [owner_id=UOne, contact=ray.maslanka@gmail.com, extra_text=null, nextSendTime=1323092391940, device_type=32, device_id=108, retryCount=0, targetId=UOne, attach_vm=true, isUrgent=false, isPrivate=false, ndrreason=0, uid=15, messagetype=1, preferenceA, sender=203, callerName=User Three
3021 12/05 08:39:52.118 VMSS mnot 0 EmailSender: sendEmailNotification: checkSendPreConditions passed
3021 12/05 08:39:52.118 capi smtp 0 SmtpServer: getSysdb(): Attribute: address
2297 12/05 08:39:52.119 capi smtp 0 SmtpSysdbNode: get(): address
3021 12/05 08:39:52.119 capi smtp 0 SmtpServer: getSysdb(): Attribute: port
2307 12/05 08:39:52.120 capi smtp 0 SmtpSysdbNode: get(): port
3021 12/05 08:39:52.121 capi smtp 0 SmtpServer: getSysdb(): Attribute: userid
2322 12/05 08:39:52.122 capi smtp 0 SmtpSysdbNode: get(): userid
3021 12/05 08:39:52.122 capi smtp 0 SmtpServer: getSysdb(): Attribute: password
2318 12/05 08:39:52.123 capi smtp 0 SmtpSysdbNode: get(): password
3021 12/05 08:39:52.123 capi smtp 0 SmtpServer: getSysdb(): Attribute: authRequired
2297 12/05 08:39:52.124 capi smtp 0 SmtpSysdbNode: get(): authRequired
3021 12/05 08:39:52.130 VMSS mnot 0 EmailSender: Begin processing email job, UID=15
3021 12/05 08:39:52.158 VMSS mnot 0 EmailSender: Attachment body part added, UID=15
3021 12/05 08:39:53.476 VMSS mnot 0 EmailSender: Sent successfully, UID=15
3021 12/05 08:39:53.476 VMSS mnot 0 EmailSender: Send email rc=1
3021 12/05 08:39:53.476 VMSS mnot 0 EmailSender: Deleting email job: Device ID=108, UID=15
3021 12/05 08:39:53.495 VMSS mnot 0 EmailSender: calculateWaitTime(): current time: 1323092393492, job time: 0
3021 12/05 08:39:53.495 VMSS mnot 0 EmailSender: Wait time=0
Below is a failed voicemail notification to ray.maslanka@gmail.com, using smtp.gmail.com on port 587.
The CUE is configured to require SMTP authentication, and again using ray.maslanka@gmail.com.
Note the errors related specifically to TLS, a logical result of using port 587.
4414 12/05 09:11:59.431 VMSS mnot 0 MessageNotification: Enter insertIntoMsgNotifQueue(): msg recipient: UOnenotif target: UOne
4414 12/05 09:11:59.469 capi smtp 0 SmtpServer: getSysdb(): Attribute: address
2297 12/05 09:11:59.470 capi smtp 0 SmtpSysdbNode: get(): address
4414 12/05 09:11:59.470 VMSS mnot 0 MessageNotification: query: INSERT INTO outcall_email_job VALUES ('UOne', 108, 1323094319432, 17, 0, 'UOne');
4414 12/05 09:11:59.493 VMSS mnot 0 MessageNotification: Notifying sender threads
3021 12/05 09:11:59.502 VMSS mnot 0 EmailSender: Processing job: 1
4414 12/05 09:11:59.512 VMSS mnot 0 MessageNotification: Exit insertIntoMsgNotifQueue(): notifyEmailThread=true, notifyPhoneThread=false
3021 12/05 09:11:59.602 VMSS mnot 0 EmailSender: Got a job: [owner_id=UOne, contact=ray.maslanka@gmail.com, extra_text=null, nextSendTime=1323094319432, device_type=32, device_id=108, retryCount=0, targetId=UOne, attach_vm=true, isUrgent=false, isPrivate=false, ndrreason=0, uid=17, messagetype=1, preferenceA, sender=203, callerName=User Three
3021 12/05 09:11:59.606 VMSS mnot 0 EmailSender: sendEmailNotification: checkSendPreConditions passed
3021 12/05 09:11:59.606 capi smtp 0 SmtpServer: getSysdb(): Attribute: address
2297 12/05 09:11:59.607 capi smtp 0 SmtpSysdbNode: get(): address
3021 12/05 09:11:59.607 capi smtp 0 SmtpServer: getSysdb(): Attribute: port
2307 12/05 09:11:59.608 capi smtp 0 SmtpSysdbNode: get(): port
3021 12/05 09:11:59.609 capi smtp 0 SmtpServer: getSysdb(): Attribute: userid
2322 12/05 09:11:59.610 capi smtp 0 SmtpSysdbNode: get(): userid
3021 12/05 09:11:59.610 capi smtp 0 SmtpServer: getSysdb(): Attribute: password
2318 12/05 09:11:59.611 capi smtp 0 SmtpSysdbNode: get(): password
3021 12/05 09:11:59.612 capi smtp 0 SmtpServer: getSysdb(): Attribute: authRequired
2297 12/05 09:11:59.613 capi smtp 0 SmtpSysdbNode: get(): authRequired
3021 12/05 09:11:59.622 VMSS mnot 0 EmailSender: Begin processing email job, UID=17
3021 12/05 09:11:59.654 VMSS mnot 0 EmailSender: Attachment body part added, UID=17
3021 12/05 09:12:00.339 VMSS mnot 0 EmailSender: Error sending email, UID= 17 :javax.mail.SendFailedException: Sending failed;
nested exception is:
class javax.mail.MessagingException: 530 5.7.0 Must issue a STARTTLS command first. v18sm77175183ibh.4
3021 12/05 09:12:00.339 VMSS mnot 0 EmailSender: Send email rc=2
3021 12/05 09:12:00.339 VMSS mnot 0 EmailSender: Deleting email job: Device ID=108, UID=17
3021 12/05 09:12:00.362 VMSS mnot 0 EmailSender: calculateWaitTime(): current time: 1323094320360, job time: 0
3021 12/05 09:12:00.362 VMSS mnot 0 EmailSender: Wait time=0
Below is a failed voicemail notification to ray.maslanka@gmail.com, using smtp.gmail.com on port 465.
The CUE is configured to require SMTP authentication, and again using ray.maslanka@gmail.com.
Note the errors related "Exception reading response", not a very descriptive error but related to the 465 / SSL authentication.
4414 12/05 09:18:08.994 VMSS mnot 0 MessageNotification: Enter insertIntoMsgNotifQueue(): msg recipient: UOnenotif target: UOne
4414 12/05 09:18:09.083 capi smtp 0 SmtpServer: getSysdb(): Attribute: address
2297 12/05 09:18:09.088 capi smtp 0 SmtpSysdbNode: get(): address
4414 12/05 09:18:09.088 VMSS mnot 0 MessageNotification: query: INSERT INTO outcall_email_job VALUES ('UOne', 108, 1323094689041, 18, 0, 'UOne');
4414 12/05 09:18:09.120 VMSS mnot 0 MessageNotification: Notifying sender threads
4414 12/05 09:18:09.120 VMSS mnot 0 MessageNotification: Exit insertIntoMsgNotifQueue(): notifyEmailThread=true, notifyPhoneThread=false
3021 12/05 09:18:09.194 VMSS mnot 0 EmailSender: Processing job: 1
3021 12/05 09:18:09.212 VMSS mnot 0 EmailSender: Got a job: [owner_id=UOne, contact=ray.maslanka@gmail.com, extra_text=null, nextSendTime=1323094689041, device_type=32, device_id=108, retryCount=0, targetId=UOne, attach_vm=true, isUrgent=false, isPrivate=false, ndrreason=0, uid=18, messagetype=1, preferenceA, sender=203, callerName=User Three
3021 12/05 09:18:09.217 VMSS mnot 0 EmailSender: sendEmailNotification: checkSendPreConditions passed
3021 12/05 09:18:09.217 capi smtp 0 SmtpServer: getSysdb(): Attribute: address
2322 12/05 09:18:09.218 capi smtp 0 SmtpSysdbNode: get(): address
3021 12/05 09:18:09.218 capi smtp 0 SmtpServer: getSysdb(): Attribute: port
2297 12/05 09:18:09.220 capi smtp 0 SmtpSysdbNode: get(): port
3021 12/05 09:18:09.220 capi smtp 0 SmtpServer: getSysdb(): Attribute: userid
2318 12/05 09:18:09.221 capi smtp 0 SmtpSysdbNode: get(): userid
3021 12/05 09:18:09.222 capi smtp 0 SmtpServer: getSysdb(): Attribute: password
2307 12/05 09:18:09.223 capi smtp 0 SmtpSysdbNode: get(): password
3021 12/05 09:18:09.223 capi smtp 0 SmtpServer: getSysdb(): Attribute: authRequired
2322 12/05 09:18:09.224 capi smtp 0 SmtpSysdbNode: get(): authRequired
3021 12/05 09:18:09.230 VMSS mnot 0 EmailSender: Begin processing email job, UID=18
3021 12/05 09:18:09.261 VMSS mnot 0 EmailSender: Attachment body part added, UID=18
3021 12/05 09:20:09.532 VMSS mnot 0 EmailSender: Error sending email, UID= 18 :javax.mail.SendFailedException: Sending failed;
nested exception is:
class javax.mail.MessagingException: Exception reading response;
nested exception is:
java.net.SocketTimeoutException: Read timed out
3021 12/05 09:20:09.532 VMSS mnot 0 EmailSender: Send email rc=2
3021 12/05 09:20:09.532 VMSS mnot 0 EmailSender: Deleting email job: Device ID=108, UID=18
3021 12/05 09:20:09.560 VMSS mnot 0 EmailSender: calculateWaitTime(): current time: 1323094809558, job time: 0
3021 12/05 09:20:09.560 VMSS mnot 0 EmailSender: Wait time=0
Below is the software version information related to the UC520 CUE.
Unfortunately the CUE release notes at http://www.cisco.com/en/US/docs/voice_ip_comm/unity_exp/rel8_6/rel_notes/rel_notes86.html#wp364157 indicate SMTP over TLS and SSL to GMail, etc is supported first in version 8.6.
show software versions
Cisco Unity Express version (8.0.6)
Technical Support: http://www.cisco.com/techsupport Copyright (c) 1986-2010 by Cisco Systems, Inc.
Components:
- TimeCardView version 8.0.6
- PhoneConnect version 7.0.0
- CUE Voicemail Language Support version 8.0.6
- Monitor Agent version 7.0.0
- Madrox Multi-app Support version 8.0.6
There are multiple conversations on the Internet re: using GMail's SMTP server(s) as voicemail message delivery servers, but none with detailed resolutions.
A request for feedback in the Cisco supportforums re: the software version and related errors can be found here, with no replies at the time of this writing: https://supportforums.cisco.com/message/3503487#3503487
My strategy at this time will be to assume voicemail message delivery from UC5XX with software load 8.2.0 is not possible via GMail's SMTP servers.
Tuesday, November 29, 2011
FXO calls disconnect: fxols_line_reversal_clid_wait. On_hook line reversal detected possibily due to crossed cable
Customer running CME on c2800nm-advipservicesk9-mz.124-4.XC4.bin with 6 FXO ports on 2 VWIC2-4FXO cards. The complaint was that callers in conversation with outside parties disconnect at random times. The duration of the call and destination didn't seem to matter.
Running "debug voip ccapi inout" and "debug vpm all" to investigate, occasionally found the following:
Nov 22 13:06:39: htsp_process_event: [0/1/0, FXOLS_ONHOOK, E_DSP_SIG_0110]
Nov 22 13:06:39: fxols_line_reversal_clid_wait. On_hook line reversal detected possibily
due to crossed cable
Nov 22 13:06:39: htsp_process_event: [0/1/1, FXOLS_ONHOOK, E_DSP_SIG_0110]
Nov 22 13:06:39: fxols_line_reversal_clid_wait. On_hook line reversal detected possibily
due to crossed cable
Nov 22 13:06:39: htsp_process_event: [0/1/2, FXOLS_ONHOOK, E_DSP_SIG_0110]
Nov 22 13:06:39: fxols_line_reversal_clid_wait. On_hook line reversal detected possibily
due to crossed cable
Nov 22 13:06:39: htsp_process_event: [0/1/3, FXOLS_ONHOOK, E_DSP_SIG_0110]
To confirm this was not a crossed cable or a flip in tip and ring leads, cables were inspected. No physical layer changes were made in years.
To confirm the issue was not related to the FXO card (errors were only related to one card), I swapped the two card positions in the router. The errors above continued to happen on the same ports (0/1/X), ruling out a card failure.
Running "show voice dsp group all" indicated the DSPs were up and looked normal.
DSP groups on slot 0:
dsp 1:
State: UP, firmware: 8.4.1
Max signal/voice channel: 16/16
Max credits: 240
Group: FLEX_GROUP_VOICE, complexity: FLEX
Shared credits: 240, reserved credits: 0
Signaling channels allocated: 12
Voice channels allocated: 0
Credits used: 0
dsp 5:
State: UP, firmware: 8.4.1
Max signal/voice channel: 16/16
Max credits: 240
Group: FLEX_GROUP_VOICE, complexity: FLEX
Shared credits: 240, reserved credits: 0
Signaling channels allocated: 0
Voice channels allocated: 0
Credits used: 0
Tracing physical cables further, I found the 4 ports in question were served by an Adtran Total Access 616, while the other 2 appeared to be standard POTS lines. In discussions with the TELCO carrier, errors on the T1 serving the Adtran were found and remote access to the device was not possible.
A site visit by a carrier representative indicated "this thing is toast" and the Adtran was replaced.
Problem solved.
Running "debug voip ccapi inout" and "debug vpm all" to investigate, occasionally found the following:
Nov 22 13:06:39: htsp_process_event: [0/1/0, FXOLS_ONHOOK, E_DSP_SIG_0110]
Nov 22 13:06:39: fxols_line_reversal_clid_wait. On_hook line reversal detected possibily
due to crossed cable
Nov 22 13:06:39: htsp_process_event: [0/1/1, FXOLS_ONHOOK, E_DSP_SIG_0110]
Nov 22 13:06:39: fxols_line_reversal_clid_wait. On_hook line reversal detected possibily
due to crossed cable
Nov 22 13:06:39: htsp_process_event: [0/1/2, FXOLS_ONHOOK, E_DSP_SIG_0110]
Nov 22 13:06:39: fxols_line_reversal_clid_wait. On_hook line reversal detected possibily
due to crossed cable
Nov 22 13:06:39: htsp_process_event: [0/1/3, FXOLS_ONHOOK, E_DSP_SIG_0110]
To confirm this was not a crossed cable or a flip in tip and ring leads, cables were inspected. No physical layer changes were made in years.
To confirm the issue was not related to the FXO card (errors were only related to one card), I swapped the two card positions in the router. The errors above continued to happen on the same ports (0/1/X), ruling out a card failure.
Running "show voice dsp group all" indicated the DSPs were up and looked normal.
DSP groups on slot 0:
dsp 1:
State: UP, firmware: 8.4.1
Max signal/voice channel: 16/16
Max credits: 240
Group: FLEX_GROUP_VOICE, complexity: FLEX
Shared credits: 240, reserved credits: 0
Signaling channels allocated: 12
Voice channels allocated: 0
Credits used: 0
dsp 5:
State: UP, firmware: 8.4.1
Max signal/voice channel: 16/16
Max credits: 240
Group: FLEX_GROUP_VOICE, complexity: FLEX
Shared credits: 240, reserved credits: 0
Signaling channels allocated: 0
Voice channels allocated: 0
Credits used: 0
Tracing physical cables further, I found the 4 ports in question were served by an Adtran Total Access 616, while the other 2 appeared to be standard POTS lines. In discussions with the TELCO carrier, errors on the T1 serving the Adtran were found and remote access to the device was not possible.
A site visit by a carrier representative indicated "this thing is toast" and the Adtran was replaced.
Problem solved.
Monday, November 07, 2011
Unity Connection 8.6 Delayed Messaging Complaints Troubleshooting
The issue to be addressed was a user claiming that they "cleared" their mailbox one afternoon, but upon arriving the next morning, heard new messages with time stamps earlier than the time they claimed they cleared it. This is interesting in so much as they were not depending on MWI to trigger their message retrieval, but rather checking via the TUI occasionally and paying attention to message counts and time stamps.
While I might normally attribute this to user error, the user complained of multiple instances, only occasionally, leading me to believe there might be some validity to their concern.
A combination of the "User Phone Login and MWI Report" and the "User Message Activity Report" in Unity Connection should provide a basic log of messages left, user retrieval and MWI actions, but they seemed slightly broken on this system, TAC can't confirm what I am presented with is to be expected and is working on it in their own lab. If these reports were reliable, they might be all I needed to confirm what the user was experiencing, or if they were in error.
Rather, I needed to use the UC logs and RTMT to decipher the user and system actions.
Step one was to enable micro traces on Unity Connection. You can access these trace setting via Unity Connection Serviceability. Below they are set for Conversation Manager, MiuSkinny and Notifier, although I was most concerned with notifier (related to MWI functions with information re: message counts as well), given I suspected other traces and output might be suspect per TAC's research.
After downloading the trace files, finding a tool to search giant ugly text files for you is invaluable. I personally use Windows Grep by Huw Millington, but that might not be the best and certainly isn't the only way to proceed.
You can start by searching for the user's alias or display name to find relevant data. Nice variables in the notifier logs also include NOTIFYQ_ACTION_MSG_NEW (associated with users receiving new messages), NOTIFYQ_ACTION_MSG_READ (users reading messages),
NOTIFYQ_ACTION_SEEN_MSG_DELETED (users deleting messages), etc. Your needs or results may vary.
You will also find things like "resyncCheck found expected msgcounts 0,4,0,0,0,0,0,0,0,4,0,4,0,0,0,0,0" that are easy to ignore due to the comma delimited overload, but are extremely helpful.
Depending on your text editor, you may not see the data schema unless you loosen your search. In this case its: URG VOI, VOI, RCPT, URG FAX, FAX, URG TXT, TXT, URG DISP, DISP, NONPROC,
SAVED URG VOI, SAVED VOI, SAVED RCPT, SAVED URG FAX, SAVED FAX, SAVED URG TXT, SAVED TXT.
So, the 0,4,0,0,0,0,0,0,0,4,0,4,0,0,0,0,0 indicates 4 voice messages, 4 not processed (?), and 4 saved messages associated with whatever user you are dwelling on.
Following the actions of users in the logs, you can determine exactly what times messages were received, lights lit, users acted on messages and how, lights extinguished, etc.
Ultimately, in this case I discovered, although the user claimed to check a mailbox at a specific time, they did not and all messages were retrieved and deleted the next time they reported to check messages. There was no evidence of delayed messages or lights. The user retrieval was delayed.
It is important to note after all this headache, the problem could have been averted by a better phone and message delivery design. The user responsible for checking messages in the box did not have a MWI available on their phone, but rather a secretary in another room (that was recently separated from the company) was the intended MWI recipient.
After some discussion re: my findings, and some gentle finger pointing, I simply added a line appearance associated with the mailbox to the users phone and explained the difference between a lit LED under the handset and the envelope icon next to the line.
So far so good, and the user is grateful to not have to guess when to check a mailbox.
While I might normally attribute this to user error, the user complained of multiple instances, only occasionally, leading me to believe there might be some validity to their concern.
A combination of the "User Phone Login and MWI Report" and the "User Message Activity Report" in Unity Connection should provide a basic log of messages left, user retrieval and MWI actions, but they seemed slightly broken on this system, TAC can't confirm what I am presented with is to be expected and is working on it in their own lab. If these reports were reliable, they might be all I needed to confirm what the user was experiencing, or if they were in error.
Rather, I needed to use the UC logs and RTMT to decipher the user and system actions.
Step one was to enable micro traces on Unity Connection. You can access these trace setting via Unity Connection Serviceability. Below they are set for Conversation Manager, MiuSkinny and Notifier, although I was most concerned with notifier (related to MWI functions with information re: message counts as well), given I suspected other traces and output might be suspect per TAC's research.
Once your traces are set and the problem is reproduced, you can use RTMT to retrieve the associated log files. You'll need as much detail from the end user as possible regarding their experience, but at very least, times associated with the message timestamps or user retrieval will be helpful. The logs can be quite voluminous and confusing. The correct version of RTMT can be downloaded from the Unity Connection server, not Cisco.com. If you are like me, you may already have multiple versions installed. It will be helpful to note what version you are installing so it can be run effectively when complete.
Below are screenshots of appropriate options to retrieve Conversation Manager and Notifier traces.
After downloading the trace files, finding a tool to search giant ugly text files for you is invaluable. I personally use Windows Grep by Huw Millington, but that might not be the best and certainly isn't the only way to proceed.
You can start by searching for the user's alias or display name to find relevant data. Nice variables in the notifier logs also include NOTIFYQ_ACTION_MSG_NEW (associated with users receiving new messages), NOTIFYQ_ACTION_MSG_READ (users reading messages),
NOTIFYQ_ACTION_SEEN_MSG_DELETED (users deleting messages), etc. Your needs or results may vary.
You will also find things like "resyncCheck found expected msgcounts 0,4,0,0,0,0,0,0,0,4,0,4,0,0,0,0,0" that are easy to ignore due to the comma delimited overload, but are extremely helpful.
Depending on your text editor, you may not see the data schema unless you loosen your search. In this case its: URG VOI, VOI, RCPT, URG FAX, FAX, URG TXT, TXT, URG DISP, DISP, NONPROC,
SAVED URG VOI, SAVED VOI, SAVED RCPT, SAVED URG FAX, SAVED FAX, SAVED URG TXT, SAVED TXT.
So, the 0,4,0,0,0,0,0,0,0,4,0,4,0,0,0,0,0 indicates 4 voice messages, 4 not processed (?), and 4 saved messages associated with whatever user you are dwelling on.
Following the actions of users in the logs, you can determine exactly what times messages were received, lights lit, users acted on messages and how, lights extinguished, etc.
Ultimately, in this case I discovered, although the user claimed to check a mailbox at a specific time, they did not and all messages were retrieved and deleted the next time they reported to check messages. There was no evidence of delayed messages or lights. The user retrieval was delayed.
It is important to note after all this headache, the problem could have been averted by a better phone and message delivery design. The user responsible for checking messages in the box did not have a MWI available on their phone, but rather a secretary in another room (that was recently separated from the company) was the intended MWI recipient.
After some discussion re: my findings, and some gentle finger pointing, I simply added a line appearance associated with the mailbox to the users phone and explained the difference between a lit LED under the handset and the envelope icon next to the line.
So far so good, and the user is grateful to not have to guess when to check a mailbox.
Subscribe to:
Posts (Atom)