SIP Tracing and Call Processing logs

Last Updated : Oct 21, 2022 |
Prolog information
Metadata
Keywords:
The traceSM and sipTrace commands trace SIP messages of the Session Manager and display the Session Manager routing decisions and internal call processing. To view sipTrace output files, use the traceSM utility. The output from the traceSM command displays the following:
  • A ladder diagram of all the SIP messages.
  • A summary of each call.
  • Call flow representations.
Other:
  • Use SIP tracing tools for troubleshooting only since it can potentially cause SIP traffic disruptions.
  • Monitor the volume of SIP traffic handled by Session Manager (megabytes per minute) since a greater volume, including metadata, is captured in the log files. The data volume captured increases the disk I/O significantly and can exhaust disk space if the log file rotation is not appropriately configured.
  • Running the traceSM command to view the SIP traffic adversely affects CPU utilization since it needs to uses several regular expressions to read the traces and pull them on the console continually.
  • When SIP traces are captured through syslog (using traceSM), some messages may be dropped. The messages are dropped because the processes prioritize handling the call traffic to serialize it for troubleshooting. Lesser messages are dropped when the processes are directed to write the SIP traces directly to the files (using the sipTrace tool).

traceSM command

The traceSM command toggles between enabling and disabling the traces.
When enabled, the traceSM command generates the following log files:
  • tracer_asset.log for SIP messages.
  • call_proc.log for call processing messages.
  • trace_ppm.log for PPM.
  • tls_handshake.log for TLS.
  • pushnotification.log for push notifications.
The log files are located in the /var/log/Avaya/trace directory.

sipTrace command

When enabled, the sipTrace command generates the sip_trace.log. The log file is located in the /var/log/Avaya/sip_trace directory.

Command-line options

When you run the traceSM command, use the following command-line options to perform the required task:
# traceSM -h 

traceSM V10.1.0.1.003

Usage: traceSM [-startsip] [-startppm] [-starttls] [-stopasm] [-stopsip] [-stopppm] [-stoptls] [-stopasm] [-statussip] [-statusppm] [-statustls] [-statusasm] [-h] [-u <URI|NUMBER>] [-i <IP>] [-c <CALL-ID>][-s <GSID>][-g <HEA>=<VALUE>] [-o <REGEX>] [-or] [-nr] [-ns] [-no] [-na] [-np] [-nt] [-nd] [nu] [-uni] [-nouni] [-a] [-r] [-m] [-max] [-k] [-sc <NUM_FILES>] [<LOG_FILE>...]
Where:
     -startsip        Start SIP tracing in background

     -stopsip         Stop SIP tracing

     -statussip       Check status of SIP tracing

     -starttls        Start TLS tracing in background

     -stoptls         Stop TLS tracing

     -statustls       Check status of TLS tracing

     -startppm        Start PPM tracing in background

     -stopppm         Stop PPM tracing

     -statuspppm      Check status of PPM tracing

     -startasm        Start ASM/CALLP tracing in background

     -stopasm         Stop ASM/CALLP tracing

     -statusasm       Check status of Stop ASM/CALLP tracing

     -u <URI|NUMBER>  Filter calls that contain (URI|NUMBER) in the 'From' or 'To' field.

     -i <IP>          Filter SIP messages from/to <IP> address.

     -c <CALL-ID>     Filter based on the SIP 'Call-ID' header field.

     -s <GSID>        Filter based on the SIP 'Av-Global-Session-ID' header.

     -g <HEA>=<VALUE> Filter SIP packets with header field <HEA> and value <VALUE>.

     -o <REGEX>       Filter SIP sessions that contain <REGEX> on any part of the SIP message. If the matching message has previously captured message, you need to reapply the filter to display all the messages for that Call-ID

     -or              Use a logical OR operator instead of the implicit AND when

                      using multiple filter options.

     -nr              Do not display REGISTER messages.

     -ns              Do not display SUBSCRIBE/NOTIFY messages.

     -no              Do not display OPTIONS messages.

     -na              Do not display SM Call processing messages.

     -np              Do not display PPM messages.

     -nt              Do not display TLS messages.

     -nd              Do not lookup the Postgres DB to resolve names (Locations,

                      SIP Entities, Route Patterns, etc.)

     -nu              Do not display TLS messages.

     -ni              Do not display SIP PING/PONG messages (RFC5626)

     -uni             Use Unicode/UTF-8 characters. Display the arrows and other lines

                      in 'graphic' mode. Your terminal client has to support Unicode

                      to display this correctly. For example, PuTTY using unicode

                      fonts (like 'Lucida Console'). Default is on.

     -nouni           Disable Unicode/UTF-8 characters.

     -a               Display internal SIP messages between SM100 and SM.

     -r               Display the RTP simulation. This is not based on the actual

                      RTP packets but using just the SDP content.

     -m               Allows to run multiple instances of traceSM. Do not use this

                      option in a production system as it may cause performance issues.

     -max <NUM_MSG>   Limit the number of captured message in memory to <NUM_MSG>.

                      By default this value is 10000. Extending the default

                      could cause performance issues (CPU and Memory). Do not use

                      this option in a production system

     -k               Kill other traceSM instances.

     -sc <NUM_FILES>  Change the number of SIP files (tracer_asset.log) to retain.

                      Default is 60, changing this will persist so make sure to put

                      it back as needed. Files rotate every minute when they are

                      over 20MB, but under heavy traffic each file could be larger.

                      Make sure there is enough disk space.

     <LOG_FILE>       One or more files previously captured with traceSM.

                      traceSM automatically detects the file format for:

                        - Call Processing (e.g. call_proc.log)

                        - SIP (e.g. tracer_asset.log)

                        - TLS (e.g. tls_handshake.log)

                        - PPM (e.g. trace_ppm.log)

                        - Push Notifications (e.g. pushnotification.log)

                        - The .tgz generated by traceSM

                      If no file is specified, then you can start/stop the capture 

                      using the 's' key. OneX Communicator SIP log file

                      (e.g. SIPMessages.txt) could be open with traceSM as well.

  The Filter options can take a regular expression. Filters are also available by pressing 'f' in the application.

  WARNING: traceSM may use high CPU and memory in a busy Session Manager. traceSM stops displaying packets if captured 10000 packets.
 Usage examples:

    To start a new capture, run 'traceSM' without arguments and then press 's':

       $ traceSM

    To filter SIP messages from/to 1.1.1.1 and 2.2.2.2:

       $ traceSM -i "1.1.1.1|2.2.2.2"

    To analyze previously captured files named call_proc.log and tracer_asset.log:

       $ traceSM call_proc.log tracer_asset.log

    To filter SIP messages containing 'Avaya' in the 'User-Agent' header field:

       $ traceSM -g "User-Agent=Avaya"

    To filter SIP sessions that got a '487 Request Terminated' response:

       $ traceSM -o "487 Request Terminated"    
When you run the sipTrace command, use the following command line options to perform the required task:
# sipTrace
Usage: sipTrace [start | stop | status | resume | logs]

     start       Enable always on SIP traces.

     stop        Stop SIP trace.

     status      Check status of SIP trace.

     resume      Resume SIP trace on restarts.

     logs        Update log rotation params, n:sM where:

                   -n is number of files

                   -s is size in MB

                 If no values are provided, reset to default 64:16M (1GB).

                 eg: sipTrace logs 200:20M

Ensure sufficient partition space availability for new values.

NOTE: Enabling 24x7 SIP trace using this tool will:

       -Allow you to use traceSM to view these SIP traces.

       -Prevent traceSM from enabling/disabling/writing SIP traces,
        (since they are already enabled, and being captured).