Troubleshooting the

Cisco Meeting Server

Vernon Depee – Technical Leader

BRKCOL-3110
Cisco Spark
Questions?
Use Cisco Spark to communicate
with the speaker after the session

How
1. Find this session in the Cisco Live Mobile App
2. Click “Join the Discussion”
3. Install Spark or go directly to the space
4. Enter messages/questions in the space

Cisco Spark spaces will be cs.co/ciscolivebot#BRKCOL-3110

available until July 3, 2017.

© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Introduction
• Acano’s vision
• Bringing disparate solutions together
• Flexible customization via the API

• My troubleshooting methodology – Easy once you know how it should work

• Heavy security mindset

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 4
Agenda

• Introduction
• Logging Tools
• Troubleshooting CallBridge
• Troubleshooting Clients
• Troubleshooting Microsoft
Interop
• Common Issues
• Conclusion
Tools
Troubleshooting Tools
• WebAdmin Logs
• SFTP Files
• SFTP: logbundle.tar.gz (Version 2.2+)
• API
• CMSLC (CMS Log Collector)
• pcaps

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 7
WebAdmin Logs
• Basic logs are available in the
WebAdmin interface
• These logs do not contain all of the
information available and should not
be used for troubleshooting
• Use the SFTP log (syslog) instead

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 8
Logging Levels
• In WebAdmin we can increase the
logging level of certain services
• If logging is not enabled, detailed
information will not be written to the
syslog
• Logging automatically disables after
the timer expires

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 9
SFTP Files
• Certificates and Keys
• Live.json
• Boot.json
• Backups
• Audit log
• Log (syslog)
• Crash dumps
• pcaps

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 10
 Log Bundle
• Version 2.2 of CMS adds a new log
bundle that can be collected via SFTP
• Syslog
• Database
• Crash Dumps
• Live.json

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 11
API - GET https://<webadminIP>:8443/api/v1/calllegs
• Can be done with any browser
• Check Configuration/Status
• Returns XML
• Reference API guide
• Multiple API GETs need to be cross-
referenced

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 12
API – Call Diagnostic Logs
• Require a POST – Extension required
• POST to /api/v1/coSpaces/<cospace-
ID>/diagnostics
• Location Header in 200 OK gives
Diagnostic ID
• GET to <location>/contents
• Diagnostics can also be seen in
WebAdmin Status page

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 13
CMSLC
• Written by Cisco TAC Engineers
• Collects ALL info from the server
• Can take a while to run
• Not Cisco supported
• Live Call Troubleshooting
• https://www.cmslogcollector.com/

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 14
pcap – Packet Capture
• Show all network traffic in/out an interface
• Extremely useful in troubleshooting network impairment
• Captures roughly 100MB before ending
• Capture command issued via MMP
• File downloaded via SFTP

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 15
What log should I collect?
Usefulness of log types The Log Bundle does not pull call info (yet)
10
9 The API can pull everything except for service config
8
7
6
5 These tell you what happened, but not always why
4
3
2
1
0
WebAdmin Log Syslog Live.json API Log Bundle CMSLC
Problem with Configuration Problem with Quality Problem with Crashes

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 16
Troubleshooting
CallBridge
CallBridge Intro
• Heart of the CMS solution
Callbridge WebAdmin
• Configured via WebAdmin/API

Web
Space API
Interface

Call

CallLeg

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 18
Call Processing Flow
If domain in If domain in
Call enters CMS ‘incoming call ‘Forwarding
match domains’: Rules’:

Incoming calls can match:

Search for User
URI portion Forward call
• Spaces based on call based on rule
• Users match rule
• IVRs
• Lync
• Lync Simplejoin If match: Use Outbound
Connect Call Dial Rules

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 19
CallBridge XMPP connection issues

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 20
AdHoc Config
• HTTPS must be used
• HTTP Port must match WebAdmin
port

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 21
Adding CMS to CUCM as Conference Bridge
• CMS uses same API as Conductor for CUCM communications
• cucm_escalation Docker VM receives from CUCM and relays to CMS
• cucm-esc in logs
• cucm-esc.CodianToCms – CUCM to CMS
• cucm-esc.CmsApi – CMS loopback GET
/api/v1/system/status
CU
device.query CM
XZ

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 22
AdHoc Flow
• API creates new space
POST
• Call enters on SIP /api/v1/coSpaces
CU
conference.create CM
XZ POST
INVITE /api/v1/calls
• Call drops down to 2 participants
• Callers connect P2P and API tears down space DELETE
INVITE (HOLD) /api/v1/coSpaces/<id>

BYE
CU
conference.destroy CM
XZ
BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 23
 AdHoc Call Join Failure
cmsrtp1 host:server: INFO : SIP trace: INVITE sip:001036050001@join.vdepee.com:5061 SIP/2.0
cmsrtp1 host:server: INFO : call 4: incoming encrypted SIP call from "sip:3001@vdepee.com" to local URI
"sip:001036050001@join.vdepee.com:5061" / "sip:001036050001@join.vdepee.com"
cmsrtp1 host:server: INFO : call 4: ending; local teardown, destination URI not matched - not connected after 0:00

Call comes in to 001036050001@join.vdepee.com

Join@vdepee.com is not set to target spaces!

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 24
XMPP Clustering Failover – Good Config

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 25
XMPP Clustering Failover – Good Config

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 26
XMPP Clustering Failover – Bad Config
• Because a server is manually set
failover will never occur
• Should be left blank if using an
XMPP cluster
• DNS records will select the
appropriate server (_xmpp-
component._tcp.domain)

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 27
XMPP Clustering Failover – Bad Config
• Different Secrets
• Authentication failure will
occur if failover from
cmsrtp1 to cmsrtp2
occurs
• Make sure to use xmpp
callbridge add-secret to
manually specify secrets

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 28
CallBridge Clustering – “Connection Attempted”

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 29
CallBridge Clustering – “Connection Attempted”

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 30
CallBridge Clustering – “Connection Attempted”

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 31
Troubleshooting
Clients
 CMA Flow
DNS
SERVER CMS
XMPP 4 Database
1 DNS SRV
SERVER
3
2 CallBridge
6
6
CMA 5 LDAP
Client TURN

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 33
CMA Flow
• DNS SRV lookup for _xmpp-client._tcp.domain
• TCP connection to IP:Port returned (5222)
• Certificates Exchange
• From here on out all encrypted/hidden
• Credentials exchanged
• Contact List pulled
• Logged in
• TURN server credentials supplied

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 34
 CMA through CMS Edge Flow
DNS CMS
SERVER CMS Core
Edge
3 Database
1 DNS SRV
Trunk XMPP
2 Load Balancer XMPP SERVER

7 5

CMA XMPP 4
Client TURN
LDAP
7 CallBridge LDAP
STUN/RTP 6

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 35
 WebBridge Flow
DNS 3
SERVER 4 CMS
XMPP XMPP 5 Database
DNS A WebBridge 6
1 SERVER

2 CallBridge
7
8 8

WebRTC LDAP
TURN
Client

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 36
Guest Login Issues – Quick Check for Trust
• WebBridge trusts the CallBridge: • WebBridge does not trust the CallBridge
certificate. No Join Call Button

*Only true if guest access is configured to be enabled.

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 37
 WebBridge through Expressway Flow
DNS
SERVER CMS
5
Database
1 DNS A CallBridge
7
7 EXP-E
STUN/RTP XMPP 4
TURN 6
LDAP
XMPP
HTTPS Proxy SERVER
WebRTC 2
Client XMPP 3
2
EXP-C
HTTPS
HTTPS Proxy WebBridge
2

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 38
WebBridge thru Expressway Flow – Adding CMS
• _cms-web._tls lookup – ideal for using not port 443
• A record lookup
• Attempts communication – single TCP reset causes full failure
• If successful, adds to CMS list
• C tells the E the guest account client URI
• E checks its certificates for the guest account client URI
• Alarm raised on E if SAN does not include FQDN for WebBridge

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 39
Chrome WebRTC debugging
• Navigate to chrome://webrtc-
internals
• Shows ice candidates, TURN
servers, and SDP
• Shows graphs for media SSRCs
• Unofficial documentation for
Chrome’s webrtc-internals:
https://testrtc.com/webrtc-internals-
documentation/

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 40
STUN/TURN/ICE
• Clients need to advertise an address where they can receive media
• Typical SDP:
• c=IN IP4 192.168.1.20
• m=audio 40156

• 192.168.1.20 is a private address

• We may need to route the traffic through something else first.

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 41
STUN/TURN/ICE
• STUN is used to determine what addresses we can advertise
• These addresses become ICE candidates
• A TURN Relay address is provided as a last resort

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 42
ICE SDP Candidates
• HOST Entries (Direct Communication)
• a=candidate:5 1 udp 10.0.75.1 63930 typ host
• a=candidate:9 1 udp 192.168.1.173 63933 typ host

• Server Reflexive (The source IP as seen by the TURN server)

• a=candidate:10 1 udp 172.18.105.1 63933 typ srflx raddr 192.168.1.173 rport 63933
• TURN Relay (The IP of the TURN server and how it plans to route back to the
client)
• a=candidate:11 1 udp 172.18.105.2 24000 typ relay raddr 172.18.105.1 rport 63933

*SDP Truncated
BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 43
Client’s Ice flow
Client TURN CMS

Use STUN to find IPs

SDP

Bind Channel for remote IP:Ports

Binding Requests to all candidates

Binding Requests to all candidates

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 44
 Using STUN to find IPs
Client TURN

Allocate Request

401

Allocate Request with credentials

Allocate Successful Response with

XOR-MAPPED-ADDRESS and XOR-RELAY-ADDRESS

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 45
Troubleshooting
MS Interop
MS On-Prem Gateway Flow
CUCM EP CUCM CMS MS Front End
A/V Call
A/V Call A/V Call
Presentation Call

SIP

MSSIP

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 47
MS Presentation offer

• Skype initiates content with a new INVITE (new call-id) *SDP truncated to fit

• Multi-part SDP
a=x-mediabw:applicationsharing-video send=12000;recv=12000
m=applicationsharing 26733 TCP/RTP/AVP 127
a=setup:active
a=connection:new
a=rtcp:26733
a=mid:1
a=rtpmap:127 x-data/90000
a=rtcp-mux
a=extmap:1 http:\\www.webrtc.org\experiments\rtp-hdrext\abs-send-time
a=x-applicationsharing-session-id:1
a=x-applicationsharing-role:sharer
a=x-applicationsharing-media-type:rdp
a=x-applicationsharing-contentflow:sendonly
m=video 16308 RTP/AVP 122 123 Only in second SDP offer. CMS will reject this.
BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 48
MS On-Prem Problem

CMS CallBridge Certificate:

CN = cmsrtp1.vdepee.com
OU = Systems
O = Cisco
L = RTP
ST = NC
C = US

CMS CallBridge SANs:

DNS Name: cmsrtp1.vdepee.com
DNS Name: join.vdepee.com

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 49
MS On-Prem Solution – S4B validates CN for RX Traffic
• New-CsTrustedApplicationPool -Identity cmsrtp1.vdepee.com -ComputerFqdn cmsrtp1.vdepee.com -Registrar
S4B fe.vdepee.skype -site 1 -RequiresReplication $false -ThrottleAsServer $true -TreatAsAuthenticated $true

• New-CsTrustedApplication -ApplicationId meetingserver-application -TrustedApplicationPoolFqdn

S4B cmsrtp1.vdepee.com -Port 5061

• Enable-CsTopology
S4B

• Erase Lync Edge Settings in WebAdmin

CMS

• Re-configure Lync Edge Settings in WebAdmin

CMS

• Wait! This may take a few minutes

YOU

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 50
 MS Front
End
MS On-Prem Dual-Homing Flow
CUCM EP CUCM CMS

A/V Call A/V Call AVMCU

SIP IMMCU
MSSIP

Focus

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 51
SERVICE for Meeting ID
• In Lync/S4B calls are connected to by putting the “ID” in the URI

• 0L7HIO0 is the meeting ID

• We discover this meeting ID by sending a SIP SERVICE message to the front
end using the credentials configured in Lync Edge Settings on the CMS
• The 200 OK response includes the meeting ID
• <msci:conference-id>0L7H1IO0</msci:conference-id>
• In Dual-Homed O365 Meetings instead of a SIP SERVICE message we use
HTTPS
BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 52
MS O365/Federation Gateway Flow
CUCM Expressway-C Expressway-E

A/V Call A/V Call A/V Call Office 365/

Content Content Federated Lync
Call Call

Content
A/V Call

A/V Call

Call SIP

MSSIP

CMS

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 53
MS Federation - CUCM Configuration

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 54
MS Federation – Key
Expressway/VCS-C Configuration IM&P Federation
A/V Federation

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 55
 MS Gateway Federation - CMS Configuration

Incoming Call Forwarding

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 56
MS Federation – Key
Expressway/VCS-E Configuration IM&P Federation
A/V Federation

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 57
MS O365 calling Problem – SIP Variant
• If call outbound from CMS enters the Expressway-E as a Standards-Based SIP call the
Expressway-E will do Standards-Based SRV lookups (_sips._tcp,_sip._tcp,_sip._udp)

• Module="network.dns" Level="DEBUG": Detail="Sending DNS query"

Name="_sips._tcp.conftme.onmicrosoft.com" Type="SRV (IPv4 and IPv6)“
• Module="network.dns" Level="DEBUG": Detail="Sending DNS query"
Name="_sip._tcp.conftme.onmicrosoft.com" Type="SRV (IPv4 and IPv6)“
• Module="network.dns" Level="DEBUG": Detail="Sending DNS query"
Name="_sip._udp.conftme.onmicrosoft.com" Type="SRV (IPv4 and IPv6)"

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 58
MS O365 calling Problem – SIP Variant
• If call outbound from CMS enters the Expressway-E as a MS Variant SIP call
the Expressway-E will do Microsoft-Based SRV lookups (_sipfederationtls._tcp)

• Module="network.dns" Level="DEBUG": Detail="Sending DNS query"

Name="_sipfederationtls._tcp.conftme.onmicrosoft.com" Type="SRV (IPv4 and
IPv6)"

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 59
MS Federation Certificate Requirements
• EXP-E SAN must include the A-record tied to the _sipfederationtls._tcp record
• EXP-E SAN must include an entry in the SIP domain
• Certificate issues for outbound calls from the EXP-E will usually show a “Server
time-out” in the search history
• In Lync logs check for TL_WARN and TL_ERROR messages for calls to Lync
• The Text field will explain the error
• The Data field will provide more details

• In Lync logs check for 504 Server time-out message for calls from Lync
• Ms-diagnostics field will give details on the error

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 60
Common Issues & Misconceptions
Certificates & CMS
• Most certificates must be trusted • <key-file> - Private Key
directly – not via a CA
• <crt-file> - Server Certificate
• Certificate validation can be thought
• <crt-bundle> - Your CA chain
of as authentication between
services • <trust> - Server certificates allowed
to access this service
Root

Int(s)

We only care about the

Server server certificate

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 62
Tips to avoid unnecessary complications
• Don’t unnecessarily use NIC’s
• One Interface per subnet
• You can control what interface listens on what services, but the routing table determines
egress interface.
• Certificates can be re-used for many services
• For most of my lab environment I have only two certs – server (runs all services and db
server) and client (runs db client only).
• Be careful that your private keys don’t become compromised

• Utilize Expressway as edge when possible

• CMA over Expressway coming soon!

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 63
Common Problem – Chat Disabled
• Chat is disabled by default
• We need a default Call Profile that
allows chat

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 64
Common Problem – Chat Disabled
• Reference API guide for /callprofiles/
• The parameter
messageBoardEnabled allows text
chat
• We create a profile
“messageBoardEnabled=True” and
get the ID of the call profile in the
location header of the 200 OK

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 65
Common Problem – Chat Disabled
• Reference API guide for
/system/profiles
• The parameter callProfile
sets the system default
call profile
• Then we add this call
profile as the system
default profile with a PUT
• Now chat is enabled!

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 66
Common Problem – Call Quality
• Ensure QoS marking is configured and is trusted
• DSCP settings can be configured in the MMP on a per traffic type basis (reboot
required)
• To troubleshoot media streams, packet captures can be useful, but SIP media
encryption needs to be disabled under Configuration > Call Settings
• TAC can rebuild unencrypted pcaps into Audio/Video streams
• Check for missing RTP Sequence #s and validate DSCP markings

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 67
Common Problem – Downgrade without Restore
• After a downgrade a backup from the previous version MUST be restored
• Database differences between versions can cause major issues
• Errors in logs may talk about null values or keys not being present

• postgres[29415]: [3-1] 2017-05-17 18:17:51 UTC 192.168.1.20(40582) ERROR:

null value in column "uri_id" violates not-null constraint

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 68
Conclusion

• Continue talking in the Spark

Space!
• Anything not covered that you
want to see covered?
 Complete Your Online
Session Evaluation
• Give us your feedback to be
entered into a Daily Survey
Drawing. A daily winner will
receive a $750 gift card.
• Complete your session surveys
through the Cisco Live mobile
app or on www.CiscoLive.com/us.

Don’t forget: Cisco Live sessions will be

available for viewing on demand after the
event at www.CiscoLive.com/Online.

© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Thank you
Extras
 Adding CMS to CUCM as Conference Bridge - logs
Apr 18 16:37:14 user.info cmsrtp1 2017-04-18 16:37:14: [INFO] cucm-esc.CodianToCms - Handling device.query
Apr 18 16:37:14 user.info cmsrtp1 2017-04-18 16:37:14: [INFO] cucm-esc.CmsApi - GET request to "http://127.0.0.1:8080/api/v1/system
Apr 18 16:37:14 user.info cmsrtp1 host:server: INFO : API trace 6612: GET for "/api/v1/system/status" (from 127.0.0.1:55872)
Apr 18 16:37:14 user.info cmsrtp1 host:server: INFO : API trace 6612: sending 200 response, size 564
Apr 18 16:37:14 user.info cmsrtp1 host:server: INFO : API trace 6612: <status>
Apr 18 16:37:14 user.info cmsrtp1 host:server: INFO : API trace 6612: <softwareVersion>2.2(Beta3)</softwareVersion>
Apr 18 16:37:14 user.info cmsrtp1 host:server: INFO : API trace 6612: <uptimeSeconds>418593</uptimeSeconds>
Apr 18 16:37:14 user.info cmsrtp1 host:server: INFO : API trace 6612: <cdrTime>2017-04-18T16:37:13Z</cdrTime>
Apr 18 16:37:14 user.info cmsrtp1 host:server: INFO : API trace 6612: <activated>true</activated>
Apr 18 16:37:14 user.info cmsrtp1 host:server: INFO : API trace 6612: <clusterEnabled>true</clusterEnabled>
Apr 18 16:37:14 user.info cmsrtp1 host:server: INFO : API trace 6612: <cdrCorrelatorIndex>11</cdrCorrelatorIndex>
Apr 18 16:37:14 user.info cmsrtp1 host:server: INFO : API trace 6612: <callLegsActive>0</callLegsActive>
Apr 18 16:37:14 user.info cmsrtp1 host:server: INFO : API trace 6612: [ ... ]
Apr 18 16:37:14 user.info cmsrtp1 host:server: INFO : API trace 6612: </status>
Apr 18 16:37:14 user.info cmsrtp1 2017-04-18 16:37:14: [INFO] cucm-esc.CmsApi - Successful Response for GET request to http://127
Apr 18 16:37:14 user.info cmsrtp1 2017-04-18 16:37:14: [INFO] cucm-esc.CodianToCms - Response to device.query: {'currentTime':20

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 74
AdHoc Call Join Failure – full logs
This failure was caused by no incoming call rule for join.vdepee.com. Make sure you have inbound rules that match the domain portion of the URI.

Apr 18 17:03:20 user.info cmsrtp1 2017-04-18 17:03:20: [INFO] cucm-esc.CodianToCms - Handling conference.create
Apr 18 17:03:20 user.info cmsrtp1 2017-04-18 17:03:20: [INFO] cucm-esc.CodianToCms - ownerID present
Apr 18 17:03:20 user.info cmsrtp1 2017-04-18 17:03:20: [INFO] cucm-esc.CodianToCms - Creating space with {'name': '001036050001', 'uri': '001036050001', 'callId': '001036050001'}
Apr 18 17:03:20 user.info cmsrtp1 2017-04-18 17:03:20: [INFO] cucm-esc.CmsApi - POST request to "http://127.0.0.1:8080/api/v1/coSpaces", "{'name': '001036050001', 'uri': '001036050001', 'callId': '0
Apr 18 17:03:20 user.info cmsrtp1 host:server: INFO : API trace 6634: POST for "/api/v1/coSpaces" (from 127.0.0.1:57730)
Apr 18 17:03:20 user.info cmsrtp1 host:server: INFO : API trace 6634: content data size 54, type "application/x-www-form-urlencoded":
Apr 18 17:03:20 user.info cmsrtp1 host:server: INFO : API trace 6634: name=001036050001&
Apr 18 17:03:20 user.info cmsrtp1 host:server: INFO : API trace 6634: uri=001036050001&
Apr 18 17:03:20 user.info cmsrtp1 host:server: INFO : API trace 6634: callId=001036050001
Apr 18 17:03:20 local0.info cmsrtp1 host:server: INFO : 127.0.0.1:57730: API user "admin" created new space afb9676e-55f8-482b-8b71-2e4a21873090 (001036050001)
Apr 18 17:03:20 user.info cmsrtp1 host:server: INFO : API trace 6634: sending 200 response, size 0
Apr 18 17:03:20 user.info cmsrtp1 host:server: INFO : API trace 6634: Location: /api/v1/coSpaces/afb9676e-55f8-482b-8b71-2e4a21873090
Apr 18 17:03:20 user.info cmsrtp1 2017-04-18 17:03:20: [INFO] cucm-esc.CmsApi - Successful Response for POST request to http://127.0.0.1:8080/api/v1/coSpaces
Apr 18 17:03:20 user.info cmsrtp1 2017-04-18 17:03:20: [INFO] cucm-esc.CodianToCms - Creating call in space afb9676e-55f8-482b-8b71-2e4a21873090
Apr 18 17:03:20 user.info cmsrtp1 2017-04-18 17:03:20: [INFO] cucm-esc.CmsApi - POST request to "http://127.0.0.1:8080/api/v1/calls", "{'coSpace': 'afb9676e-55f8-482b-8b71-2e4a21873090'}"
Apr 18 17:03:20 user.info cmsrtp1 host:server: INFO : API trace 6635: POST for "/api/v1/calls" (from 127.0.0.1:57736)
Apr 18 17:03:20 user.info cmsrtp1 host:server: INFO : API trace 6635: content data size 44, type "application/x-www-form-urlencoded":
Apr 18 17:03:20 user.info cmsrtp1 host:server: INFO : API trace 6635: coSpace=afb9676e-55f8-482b-8b71-2e4a21873090
Apr 18 17:03:20 user.info cmsrtp1 host:server: INFO : call create failed to find coSpace -- attempting to retrieve from database
Apr 18 17:03:20 local0.info cmsrtp1 host:server: INFO : 127.0.0.1:57736: API user "admin" created new call 957755f2-7d08-4b04-834e-53fbb91ca0df
Apr 18 17:03:20 user.info cmsrtp1 host:server: INFO : API trace 6635: sending 200 response, size 0
Apr 18 17:03:20 user.info cmsrtp1 host:server: INFO : API trace 6635: Location: /api/v1/calls/957755f2-7d08-4b04-834e-53fbb91ca0df
Apr 18 17:03:20 user.info cmsrtp1 2017-04-18 17:03:20: [INFO] cucm-esc.CmsApi - Successful Response for POST request to http://127.0.0.1:8080/api/v1/calls
Apr 18 17:03:20 user.info cmsrtp1 2017-04-18 17:03:20: [INFO] cucm-esc.CodianToCms - Response to conference.create: operation successful.
Apr 18 17:03:20 user.info cmsrtp1 2017-04-18 17:03:20: [INFO] cucm-esc.Cache - Added call: 001036050001, cospace guid: afb9676e-55f8-482b-8b71-2e4a21873090, start: 2017-04-18 17:03:20.643
Apr 18 17:03:20 user.info cmsrtp1 2017-04-18 17:03:20: [INFO] cucm-esc.CallClearingThread - Clearing expired conferences
Apr 18 17:03:20 user.info cmsrtp1 2017-04-18 17:03:20: [INFO] cucm-esc.CallClearingThread - Sleeping for 3599.997589 s, before removing conferences.
Apr 18 17:03:20 user.info cmsrtp1 host:server: INFO : SIP trace: connection 1: incoming SIP TLS data from 192.168.1.11:34784 to 192.168.1.20:5061, size 1000:
Apr 18 17:03:20 user.info cmsrtp1 host:server: INFO : SIP trace: INVITE sip:001036050001@join.vdepee.com:5061 SIP/2.0

Apr 18 17:03:20 user.info cmsrtp1 host:server: INFO : call 4: incoming encrypted SIP call from "sip:3001@vdepee.com" to local URI "sip:001036050001@join.vdepee.com:5061" / "sip:001036050001@
Apr 18 17:03:20 user.info cmsrtp1 host:server: INFO : call 4: ending; local teardown, destination URI not matched - not connected after 0:00

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 75
WebBridge Login Failure – Bad Password
Apr 18 19:33:45 user.info cmsrtp1 webbridge: INFO : login request received - session:0 user:1 password:1
Apr 18 19:33:45 user.info cmsrtp1 webbridge: INFO : Session 7F89E4004DB0 activated - 1 currently active
Apr 18 19:33:45 user.info cmsrtp1 webbridge: INFO : Attempting XMPP connection to 192.168.1.20:5222
Apr 18 19:33:45 user.info cmsrtp1 webbridge: INFO : XMPP connected to 192.168.1.20:5222
Apr 18 19:33:45 user.info cmsrtp1 xmppd[7]: D-MLink-Auth SID-10 New client connection from 192.168.1.20
Apr 18 19:33:45 user.info cmsrtp1 xmppd[7]: I-MLink-Info TLS conn IP=192.168.1.20 version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES256-GCM-
SHA384 secret-bits=256 processed-bits=256 compression="(None)" no certificate provided
Apr 18 19:33:45 user.info cmsrtp1 authp: Using authentication server join.vdepee.com to authenticate user vdepee@vdepee.com (index: 1/2,
reason: first match)
Apr 18 19:33:45 user.info cmsrtp1 host:server: INFO : LDAP authorisation failed for user 'vdepee@vdepee.com'
Apr 18 19:33:45 user.info cmsrtp1 host:server: INFO : LDAP failure 49 (invalidCredentials) server diagnostics message: 80090308: LdapErr: DSID-
0C0903A9, comment: AcceptSecurityContext error, data 52e, v1db0
Apr 18 19:33:45 user.info cmsrtp1 host:server: INFO : unsuccessful login request from vdepee@vdepee.com
Apr 18 19:33:46 user.info cmsrtp1 host:server: INFO : web bridge link 1: get connection 7F356428F6A0 to join.vdepee.com lost rx_num -1
Apr 18 19:33:46 user.info cmsrtp1 host:server: INFO : web bridge link 1: get connection lost, last socket error code: 0
Apr 18 19:33:46 user.info cmsrtp1 host:server: INFO : DNS trace: resolving "join.vdepee.com" for web bridge connection
Apr 18 19:33:46 user.info cmsrtp1 host:server: INFO : DNS trace: resolution of "join.vdepee.com" for web bridge connection succeeded; results: 1
Apr 18 19:33:47 user.info cmsrtp1 host:server: INFO : web bridge link 1: transactions on send list: 0, send successful
Apr 18 19:33:50 user.info cmsrtp1 webbridge: INFO : XMPP handshake failed for reason 7
Apr 18 19:33:50 user.info cmsrtp1 webbridge: INFO : XMPP Thread state failure 1
Apr 18 19:33:50 user.info cmsrtp1 webbridge: INFO : Session 7F89E4004DB0 moving from state idle to disconnected
Apr 18 19:33:50 user.info cmsrtp1 webbridge: INFO : XMPP connection dropped while session was live for reason 0
Apr 18 19:33:50 user.info cmsrtp1 webbridge: INFO : Session 7F89E4004DB0 moving from state disconnected to disconnected
Apr 18 19:33:50 user.info cmsrtp1 webbridge: INFO : Session 7F89E4004DB0 destroyed (0 active, cumulative 4)
Apr 18 19:33:50 user.info cmsrtp1 xmppd[7]: D-MLink-Auth SID-10 Closed C2S connection from none

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Call processing works like this on CMS:
• SIP or Lync call comes in, server looks at called domain to see if it matches Incoming Call Match Domains. If it does, it
searches for URI User Part based on that call match rule. If does not match, server looks in Forwarding rules for domain
match. If matched, either with exact domain or wild card, call is forwarded based on rule. If not, call is rejected right
there. If it is forwarded, call is sent to Outbound Dial Rules for placing of outbound call. Call is placed based on dial rules
and settings for those rules, where first match is used and call is placed.

Some things to note about all of this:

If there are no incoming call match rules, server defaults to match all domains. If 1 rule is added, server changes to only
match what is defined there, anything else is rejected or forwarded.

For Forward Rules, you can create nasty call loops, so using * is not the best approach unless too many domains to add,
if you must use *, make sure to add local CallBridge IP and local CMS Domain as higher priority forward rules with Reject
as action, so call loops do not exist.

For outbound calls, leaving domain blank will match all domains, leaving Proxy blank will result in None, Call Directly,
meaning DNS lookups will be used (_sips._tcp, _sip._tcp and _sip._udp if SIP call and _sipinternaltls._tcp and
_sipfederationtls._tcp if Lync rule, if called using a Domain. If called using something@ip.address (i.e.
test@192.168.10.10), no DNS will be used and will simply send SIP call to the IP entered in URI.

This is where the call loops occur, someone dial a wrong URI at CallBridge IP, call is not matched, forwarded, sent out
on a Match All, None Call Directly rule and is looped back as the IP is its own local IP. call comes in, repeat, over and
over till 10,000 calls flood the system. Same if is @domain, if SRV exists for that domain that result in CallBridge IP
again.
BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 77
Multistream SDP Lines that are important
CMS INVITE
a=sprop-source:1 count=2;policies=cs:1
a=sprop-simul:1 1 *
a=rtcp-fb:* ccm cisco-scr

EP 200 OK:
Contact: <sip:1234@10.0.0.6:5061;transport=tls>;sip.cisco.multistream;x-cisco-
multiple-screen=2
a=rtcp-fb:* ccm cisco-scr
a=sprop-simul:1 1 98 max-br=6000;max-fs=8160;max-mbps=245000
a=sprop-simul:1 2 98 max-br=6000;max-fs=3600;max-mbps=108000
a=sprop-simul:1 3 98 max-br=3000;max-fs=2304;max-mbps=70000
a=sprop-source:1 csi=162577409

BRKCOL-3110 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 78