ACES
Systems Operational
Saturday, December 6, 2025 at 03:53:46

BFD & ISIS Adjacency Flap

Major
Resolved

Incident INC-000000694 • Created 2025-06-24 01:20 UTC

SLA DEADLINE
Met
ASSIGNED TO
Alex Rodriguez
CUSTOMERS AFFECTED
2
CORRELATED TICKETS
1
Network Topology
100G40G10G1GBNE BorderBNE BorderBNE CoreBNE CoreExternal PeerExternalSYD CoreSYD CoreMEL CoreMEL Core
Operational
Degraded
Down
Fiber
Ethernet
MPLS

Affected Components

BNE Border
BNE Border
operational
Primary Affected Component
Component Name
bdr03-ipt-20wharfs-bne
Type
Border Router
Location
BNE Border
Current Status
operational
Incident Information

Description

Bidirectional Forwarding Detection (BFD) and ISIS adjacency experiencing intermittent flapping on BNE border router, causing routing instability.

Root Cause Analysis

Interface buffer overflow during traffic bursts caused BFD packet drops. ISIS hello packets were also affected, triggering adjacency flaps and route reconvergence.

Estimated Resolution

Resolved at 2025-06-24 02:00 UTC
Incident Analysis
System Logs
2025-06-19 13:46:12.345
ERROR
cor01-etp-454stpau-bne: BFD session removed on interface FourHundredGigE0/0/0/3
BFD neighbor 10.1.1.2 state changed from Up to Down
2025-06-19 13:46:15.123
WARN
monitoring-system: Interface FourHundredGigE0/0/0/3 operationally down
Link state change detected, triggering alarm
2025-06-19 13:46:18.567
ERROR
cor01-etp-47bourke-syd: Loss of Signal detected on optical interface
Optical power level dropped below threshold (-20.5 dBm)
2025-06-19 13:46:22.890
INFO
mpls-engine: TE tunnel Tunnel-te421 state changed to down
Tunnel rerouting initiated via backup path
2025-06-19 13:46:25.234
CRITICAL
service-monitor: Customer EVC services impacted
3 customer services affected by link failure
Resolution Steps
4/4 Complete

Identify BFD session flaps and ISIS adjacency changes

Assigned to: Alex Rodriguez

Completed: 2025-06-24 01:30 UTC

Analyze interface buffer utilization and packet drops

Assigned to: Alex Rodriguez

Completed: 2025-06-24 01:40 UTC

Increase interface buffer size and adjust BFD timers

Assigned to: Alex Rodriguez

Completed: 2025-06-24 01:50 UTC

Monitor adjacency stability and routing convergence

Assigned to: Alex Rodriguez

Completed: 2025-06-24 02:00 UTC

Quick Actions
Emergency Contacts
NOC Manager
David Wilson
On-Call Engineer
Available 24/7
Impact Metrics
bfd Flaps8 events
isis Adjacency Flaps5 events
route Convergence< 2 seconds
error Rate4.1%