Configuration in IS-IS is fairly simple and straightforward. In the two-stage process discussed in Chapter 10, the routing process is enabled globally, and IS-IS adjacency formation and LSP flooding is enabled on an interface by applying the command ip router isis. This command also puts the IP subnet information of the interface into the router’s LSP that is generated and flooded to adjacent neighbors. This section covers IS-IS routing update problems on the premise that there are no adjacency problems. This essentially implies that troubleshooting any routing update problems should start with verifying the appropriate adjacencies. Adjacency problems and related troubleshooting methodology are discussed extensively in the previous sections.
The following routing update problems are covered in this section:
Route advertisement problems
Route redistribution problems
One important method for troubleshooting routing problems in IS-IS is by direct inspection of the contents of link-state packets (LSPs). Depending on its configuration, an IS-IS router generates an LSP for each of the levels of routing that it participates in—Level 1 LSP, Level 2 LSP, or both. Inspection of the LSP contents of the expected source of a route can help diagnose routing advertisement problems in cases when no obvious adjacency problems exist. The show isis database detail command displays the contents of a specific LSP. Example 11-25 shows sample output from this command.
RT1#show isis database level-1 RT2.00-00 detail IS-IS Level-2 LSP RT2.00-00 LSPID LSP Seq Num LSP Checksum LSP Holdtime ATT/P/OL RT2.00-00 0x00001C9C 0x5F3E 1015 0/0/0 Area Address: 49.0002 NLPID: 0xCC Hostname: RT2 IP Address: 220.127.116.11 Metric: 10 IS-Extended RT1.00 Metric: 10 IP 10.1.2.0/24 Metric: 0 IP 18.104.22.168/32 Metric: 10 IP 22.214.171.124/32 Metric: 10 IP 192.168.1.0/30
RT2.00-00 is the LSP ID of RT2. Detail output of the LSP, with ID RT2.00-00, shows the IP subnets for directly connected links, together with their metric information.
The backbone (Level 2) is the shaded area. Example 11-26 shows the Level 1 and Level 2 data-bases of RT1. The Level 1 database features RT3 and RT5, confirming that these routers are indeed in the same area as RT1. The Level 2 database describes the backbone, which features RT2 and RT3. RT2 is not in the same area as RT1, but it is connected to the backbone, whereas RT3 is a Level 1–2 router in the same area as RT1.
RT1#show isis topology IS-IS paths to level-1 routers System Id Metric Next-Hop Interface SNPA RT1 -- RT3 10 RT3 Et0/0 00d0.58eb.f841 RT5 10 RT5 Et0/0 00d0.58eb.ff01 IS-IS paths to level-2 routers System Id Metric Next-Hop Interface SNPA RT1 -- RT2 10 RT2 Se0/0 *HDLC* RT3 10 RT3 Et0/0 00d0.58eb.f841
Route Advertisement Problems
Suppose that there is concern that a specific route is missing on RT5 (see Figure 11-6). You can start troubleshooting the problem by obtaining more information on the topology of the network. This should help you determine which router is the original source of the route. Suppose that the route 126.96.36.199/32 turns out to be the loopback address of RT2; the flowchart in Figure 11-7 can be followed to narrow the cause of the problem.
Using knowledge about the topology, you know that RT2 and RT5 are not in the same area. In this case, if route leaking is not enabled in the network, RT5, which is a Level 1–only router, should not see any route from RT2. Hence, tracking the problem takes you along the flowchart through boxes 0-1-7-10-5, where you determine that this is normal behavior. As a Level 1–only router, RT5 should follow a default router to the nearest Level 1 router to get to remote areas.
The procedure covered by the flowchart in Figure 11-7 provides a generic method for trouble-shooting missing route problems. The following sections discuss procedures for specific scenarios.
Local Routes Not Being Advertised to Remote
Because IS-IS is a link-state protocol, IS-IS routers depend on LSP flooding to obtain topology and routing information. For example, during stable conditions, each IS-IS router in an area will have the same Level 1 link-state database, which contains the LSPs generated by every router in the area. Dijkstra’s algorithm is run over the LS database to obtain the best path to every advertised route. Therefore, if a route is missing at a section of the area, it’s because the routers in that section did not receive the original LSP, or the LSP was received corrupted and, there-fore, was purged. An even simpler reason could be that the route was not even put into the LSP at the source. The flowchart in Figure 11-8 provides the troubleshooting methodology for such problems. The output of debug isis update-packets and debug isis snp-packets, as demon-strated in Example 11-27, could help decipher this sort of problem if it is related to LSP flooding or issues with link-state database synchronization.
Step 8 of the flowchart for troubleshooting route advertisement problems calls for enabling the debugging of LSP updates if it is determined that an LSP is not making it to certain remote locations of the IS-IS domain. Example 11-27 shows an output of the debug isis update-packets command. The highlighted lines show RT1 flooding its LSP and also receiving an LSP from RT2. Because the adjacency was just brought up, the output also shows the one-time exchange of CSNPs on point-to-point links between RT1 and RT2.
RT1#debug isis update-packets Mar 2 23:25:02: %LINEPROTO-5-UPDOWN: Line protocol on Interface Serial0/0, chp Mar 2 23:25:02: ISIS-Update: Building L2 LSP Mar 2 23:25:02: ISIS-Update: No change, suppress L2 LSP 0000.0000.0001.00-00,0 Mar 2 23:25:03: %CLNS-5-ADJCHANGE: ISIS: Adjacency to RT2 (Serial0/0) Up, newy Mar 2 23:25:07: ISIS-Update: Building L2 LSP Mar 2 23:25:07: ISIS-Update: TLV contents different, code 16 Mar 2 23:25:07: ISIS-Update: Full SPF required Mar 2 23:25:07: ISIS-Update: Sending L2 LSP 0000.0000.0001.00-00, seq 160, ht0 Mar 2 23:25:07: ISIS-Update: Rec L2 LSP 0000.0000.0002.00-00, seq 1D16, ht 11, Mar 2 23:25:07: ISIS-Update: from SNPA *HDLC* (Serial0/0) Mar 2 23:25:07: ISIS-Update: LSP newer than database copy Mar 2 23:25:07: ISIS-Update: No change Mar 2 23:25:08: ISIS-SNP: Rec L2 CSNP from 0000.0000.0002 (Serial0/0) Mar 2 23:25:08: ISIS-SNP: Rec L2 PSNP from 0000.0000.0002 (Serial0/0) Mar 2 23:25:08: ISIS-SNP: PSNP entry 0000.0000.0001.00-00, seq 160, ht 1197 Mar 2 23:25:08: ISIS-Update: Sending L2 CSNP on Serial0/0 Mar 2 23:25:08: ISIS-Update: Build L2 PSNP entry for 0000.0000.0002.00-00, se6 Mar 2 23:25:08: ISIS-Update: Build L2 PSNP entry for 0000.0000.0006.00-00, se2 Mar 2 23:25:08: ISIS-Update: Sending L2 PSNP on Serial0/0 Mar 2 23:25:09: ISIS-Update: Building L1 LSP Mar 2 23:25:09: ISIS-Update: Important fields changed Mar 2 23:25:09: ISIS-Update: Important fields changed Mar 2 23:25:09: ISIS-Update: Full SPF required Mar 2 23:25:09: ISIS-Update: Sending L1 LSP 0000.0000.0001.00-00, seq 15A, ht0 Mar 2 23:25:09: ISIS-Update: Sending L1 CSNP on Ethernet0/0 _____________________________________________________________________________________ RT5#debug isis snp-packets IS-IS CSNP/PSNP packets debugging is on RT5# Mar 6 20:02:28: ISIS-SNP: Rec L1 CSNP from 0000.0000.0001 (Ethernet0/0) Mar 6 20:02:28: ISIS-SNP: CSNP range 0000.0000.0000.00-00 to FFFF.FFFF.FFFF.FFF Mar 6 20:02:28: ISIS-SNP: Same entry 0000.0000.0001.00-00, seq 15D Mar 6 20:02:28: ISIS-SNP: Same entry 0000.0000.0001.01-00, seq 104 Mar 6 20:02:28: ISIS-SNP: Same entry 0000.0000.0005.00-00, seq FEA
The highlighted line indicates receipt of a CSNP by RT5 from the DIS (RT1). By comparing the contents of the CSNP with the local Level 1 database, RT5 determines that no changes occurred in all known LSPs.
As depicted by the flowchart in Figure 11-8, there could be many potential causes for problems in which routes are not reaching remote locations in the network. In the extreme case, the route might not be making it into the routing table because of a software glitch relating to IS-IS data structures (Steps 4 and 5). Such situations might need to be reported to the Cisco Technical Assistance Center (TAC) for further assistance.
In other cases, the LSPs might not be reaching some parts of the network because they are getting corrupted in flight (Step 9). Such problems can be determined by a combination of activities, such as debugging the update process and monitoring router logs for LSP errors. If the culprit device is located, it can be isolated or replaced to address the problem.
In most cases, however, the problem could be caused by a trivial configuration error, such as IS-IS not being enabled on an interface (Step 11). Applying the appropriate interface command (ip router isis) to the interface should be sufficient to address the problem.
Route Redistribution and Level 2-to-Level 1 Route-Leaking Problems
Cisco IOS Software allows routes from other routing sources, such as static, connected interfaces, and other routing protocols, to be redistributed into IS-IS Level 1 routing, Level 2 routing, or both as external routes. Technically, external routes should exist only in the Level 2 routing environment, but Cisco provides a configuration attribute that allows redistribution into Level 1 for practical operational purposes. For example, service providers using IS-IS as IGP with flat Level 1-only domains might need this capability. The existence of the IP External Reachability TLV in a Level 1 LSP should not pose any interoperability issues because IS-IS routers are required to skip TLVs that they don’t understand when they parse an IS-IS packet.
Redistribution in IS-IS is straightforward with hardly any potential pitfalls except occasional, unexpected software bugs. Sometimes, effective metric assignment becomes a challenge in redistribution scenarios. When configuring redistribution, the operator is asked to specify internal or external metrics. The default is the external metric type, which bumps up the metric by 128 on Cisco routers. The increase by 128 instead of 64 is the result of an incorrect bit setting in the default metric field when using narrow metrics. This issue is discussed in the configur-ation section as a Cisco IOS Software implementation issue that has been retained for backward compatibility.
The rather simple flowchart in Figure 11-9 should provide guidance for troubleshooting redistribution problems.
Route flaps normally are caused by unstable links between the source of the route and the location where the flap is being observed. Typically, multiple routes are impacted at the same time because of an adjacency change affecting many LSPs, all of which might carry numerous IP prefixes. Also, route flaps could induce consistent running of the SPF process, causing potentially dangerous high CPU utilization on route processors that might crash affected routers. If the advertised LSP changes affect only IP prefixes, only partial route calculation (PRC) is run instead of full SPF calculations. PRC is less costly in terms of CPU cycles than full SPF runs. The flowchart in Figure 11-10 provides guidance for troubleshooting route-flapping problems.
Apart from certain destinations not being reachable—obviously implying routing problems—high CPU utilization by the SPF process certainly should flag instabilities in the network. High CPU utilization can be observed through the IOS command line interface (CLI), network-management applications, or special network-performance analysis tools, such as CiscoWorks 2000, HP Openview, and so on.
From the Cisco IOS Software CLI, the show process cpu command can obtain CPU utilization information. If any indication of high CPU utilization caused by the SPF process is determined, the show isis spf-log command can determine SPF-related events that might cause the high CPU utilization. Example 11-28 provides sample output of this command.
RT1#show isis spf-log Level 1 SPF log When Duration Nodes Count Last trigger LSP Triggers 03:40:08 0 3 1 PERIODIC 03:25:08 0 3 1 PERIODIC 03:10:07 0 3 1 PERIODIC 02:55:07 0 3 1 PERIODIC 02:40:07 0 3 1 PERIODIC 02:25:06 0 3 1 PERIODIC 02:10:06 0 3 1 PERIODIC 01:56:08 0 2 1 RT1.01-00 TLVCONTENT 01:55:06 0 2 1 PERIODIC 01:40:06 0 2 1 PERIODIC 01:36:31 0 2 1 RT5.00-00 LSPEXPIRED 01:28:31 0 2 2 RT1.01-00 NEWADJ TLVCONTENT 01:28:25 0 3 1 RT5.00-00 NEWLSP 01:25:06 0 3 1 PERIODIC 01:10:06 0 3 1 PERIODIC
The output in Example 11-28 lists SPF process runs by time, trigger, and duration. You might recall that LSPs are refreshed every 15 minutes, triggering periodic SPF calculations. Such events are labeled periodic in the show isis spf-log output. It also shows that at 01:25:25, the process was run over an insignificant period of time because of receipt of a new LSP from RT5. Table 11-2 lists and describes common SPF triggers.
Table 11-2. SPF Process Triggers
Attached bit setting change
End system information change
Overload bit state change
Default information change
Connected IP prefix down
Route redistribution change
Interarea route change
Connected IP prefix up
New neighbor adjacency up
IS-IS process level changed
New metric assigned to an interface
New system ID assigned
Periodic SPF process (LSPDB refresh interval)
LSP with a new TLV code field received
LSP with changed TLV contents received
debug isis spf-events
debug isis spf-triggers
debug isis spf-statistics
debug isis update-packets
debug isis adj-packets
Use caution when enabling debugging in situations in which the route processor’s CPU already is overtaxed. Example 11-29 provides sample output of the debug isis spf-events command, which displays the events following an interface shut to simulate a link flap. Lines indicating LSPs flagged for recalculation as a result of this event are highlighted. Also, events flagging computation of the Level 1 and Level 2 SPF trees are highlighted.
RT1(config-if)#debug isis spf-events Mar 6 20:17:26: ISIS-SPF: L1 LSP 1 (0000.0000.0001.00-00) flagged for recalculC Mar 6 20:17:26: ISIS-SPF: L1 LSP 5 (0000.0000.0005.00-00) flagged for recalculC Mar 6 20:17:28: %LINK-5-CHANGED: Interface Ethernet0/0, changed state to admin down Mar 6 20:17:28: ISIS-SPF: Compute L1 SPT Mar 6 20:17:28: ISIS-SPF: 3 nodes for level-1 Mar 6 20:17:28: ISIS-SPF: Move 0000.0000.0001.00-00 to PATHS, metric 0 Mar 6 20:17:28: ISIS-SPF: Add 0000.0000.0001.01-00 to TENT, metric 10 Mar 6 20:17:28: ISIS-SPF: Add 0000.0000.0001 to L1 route table, metric 0 Mar 6 20:17:28: ISIS-SPF: Move 0000.0000.0001.01-00 to PATHS, metric 10 Mar 6 20:17:28: ISIS-SPF: Aging L1 LSP 1 (0000.0000.0001.00-00), version 214 Mar 6 20:17:28: ISIS-SPF: Aging L2 LSP 2 (0000.0000.0001.00-00), version 208 Mar 6 20:17:28: ISIS-SPF: Aging L1 LSP 3 (0000.0000.0001.01-00), version 207 Mar 6 20:17:28: ISIS-SPF: Aging L2 LSP 4 (0000.0000.0002.00-00), version 209 Mar 6 20:17:28: ISIS-SPF: Aging L1 LSP 5 (0000.0000.0005.00-00), version 207 Mar 6 20:17:28: ISIS-SPF: Aging L2 LSP 6 (0000.0000.0006.01-00), version 112 Mar 6 20:17:28: ISIS-SPF: Aging L2 LSP 7 (0000.0000.0006.00-00), version 114 Mar 6 20:17:28: ISIS-SPF: Aging L2 LSP 8 (0000.0000.0001.01-00), version 1 Mar 6 20:17:29: %LINEPROTO-5-UPDOWN: Line protocol on Interface Ethernet0/0 Mar 6 20:17:33: ISIS-SPF: Compute L2 SPT Mar 6 20:17:33: ISIS-SPF: 5 nodes for level-2 Mar 6 20:17:33: ISIS-SPF: Move 0000.0000.0001.00-00 to PATHS, metric 0 Mar 6 20:17:33: ISIS-SPF: Add 49.0001 to L2 route table, metric 0 Mar 6 20:17:33: ISIS-SPF: Add 0000.0000.0001.01-00 to TENT, metric 10 Mar 6 20:17:33: ISIS-SPF: considering adj to 0000.0000.0002 (Serial0/0) metric Mar 6 20:17:33: ISIS-SPF: (accepted) Mar 6 20:17:33: ISIS-SPF: Add 0000.0000.0002.00-00 to TENT, metric 10 Mar 6 20:17:33: ISIS-SPF: Next hop 0000.0000.0002 (Serial0/0) Mar 6 20:17:33: ISIS-SPF: Move 0000.0000.0001.01-00 to PATHS, metric 10 Mar 6 20:17:33: ISIS-SPF: Move 0000.0000.0002.00-00 to PATHS, metric 10 Mar 6 20:17:33: ISIS-SPF: Add 49.0002 to L2 route table, metric 10 Mar 6 20:17:33: ISIS-SPF: Redundant IP route 10.1.2.0/255.255.255.0, metric 20d Mar 6 20:17:33: ISIS-SPF: Redundant IP route 188.8.131.52/255.255.255.255, metric d Mar 6 20:17:33: ISIS-SPF: Redundant IP route 184.108.40.206/255.255.255.255, metric d Mar 6 20:17:33: ISIS-SPF: Add 192.168.1.0/255.255.255.252 to IP route table, m0 Mar 6 20:17:33: ISIS-SPF: Next hop 0000.0000.0002/192.168.1.2 (Serial0/0) (rej)
As shown in the flowchart for troubleshooting route flapping problems (see Figure 11-10), most route-flapping problems can be traced to unstable links in the network (see Step 2). Physical connectivity problems are addressed by troubleshooting the physical and data link layers.
In more complicated scenarios, however, the flaps could be caused by LSP corruption storms or even a routing loop. This might be the case when there are no unstable links in the network but CPU utilization is high, indicating continuous running of the SPF process (see Step 4). The show isis spf-log command can provide indication of which LSPs are changing the most frequently and are triggering the SPF calculations. Similar clues can be gleaned by enabling debugging of the update process with debug isis update-packets. This should be done with care so that the CPU is not overloaded. The logs also can be observed for LSP error loggings, which could give an indication of packet corruption by a culprit device (see Step 7). Zeroing in on any continuously changing LSPs is critical for determining and addressing the problem.
As CCIE instructors we provide this free offer to help any one who is interested in being a CCIE certificate engineer .
All the below tips are FREE!!!.
- Latest CCIE certification information.
- Free advice for any CCIE exam.
- Free tips on how to become a CCIE network engineer.