[go: nahoru, domu]
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Navigation Failure for LTE devices #15389

Open
asantos-vk opened this issue Mar 6, 2024 · 3 comments
Open

Navigation Failure for LTE devices #15389

asantos-vk opened this issue Mar 6, 2024 · 3 comments
Labels
type: bug Something isn't working

Comments

@asantos-vk
Copy link
asantos-vk commented Mar 6, 2024 •
edited
Loading

Your Environment

  • 1.8:
  • Affected Component: Access Gateway
  • Affected Subcomponent: AGW Datapath
  • Deployment Environment: bare-metal

Describe the Issue

I am troubleshooting a navigation issue, and here are the outputs. In short, packets arrive on GTP_BR0 but do not appear on SGi interface.

To Reproduce

After a period of operation, the devices stop navigating. The UE shows a signal, but the device can't pass data. One way to reproduce is resetting the gtpu_sys_2152 interface.

Expected behavior

Device is connected and passing data.

UL/DL STATS

root@access-gateway:/home/magma# dp_probe_cli.py -i 724900000000008 -d UL stats
IMSI: 724900000000008, IP: 192.168.128.13
UL stats: n_packets=1287, n_bytes=97239
root@access-gateway:/home/magma# dp_probe_cli.py -i 724900000000008 -d UL stats
IMSI: 724900000000008, IP: 192.168.128.13
UL stats: n_packets=1302, n_bytes=98147
root@access-gateway:/home/magma# dp_probe_cli.py -i 724900000000008 -d DL stats
IMSI: 724900000000008, IP: 192.168.128.13
LOCAL: n_packets=0, n_bytes=0
mtr0: n_packets=0, n_bytes=0
root@access-gateway:/home/magma#

TCPDUMP GTPU_SYS WITH DST 142.250.79.206

root@access-gateway:/home/magma# tcpdump -eni gtpu_sys_2152 dst 142.250.79.206
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on gtpu_sys_2152, link-type RAW (Raw IP), capture size 262144 bytes
10:32:47.435036 ip: 192.168.128.13 > 142.250.79.206: ICMP echo request, id 69, seq 1, length 8
10:32:48.455133 ip: 192.168.128.13 > 142.250.79.206: ICMP echo request, id 69, seq 2, length 8
10:32:49.475068 ip: 192.168.128.13.47450 > 142.250.79.206.7: Flags [S], seq 2878851933, win 65535, options [mss 1300,sackOK,TS val 159437123 ecr 0,nop,wscale 9], length 0
10:32:50.505082 ip: 192.168.128.13.47450 > 142.250.79.206.7: Flags [S], seq 2878851933, win 65535, options [mss 1300,sackOK,TS val 159438145 ecr 0,nop,wscale 9], length 0
10:32:52.535047 ip: 192.168.128.13 > 142.250.79.206: ICMP echo request, id 70, seq 1, length 8
10:32:53.645077 ip: 192.168.128.13 > 142.250.79.206: ICMP echo request, id 70, seq 2, length 8
10:32:54.568173 ip: 192.168.128.13.47454 > 142.250.79.206.7: Flags [S], seq 2541092784, win 65535, options [mss 1300,sackOK,TS val 159442212 ecr 0,nop,wscale 9], length 0
10:32:55.585104 ip: 192.168.128.13.47454 > 142.250.79.206.7: Flags [S], seq 2541092784, win 65535, options [mss 1300,sackOK,TS val 159443234 ecr 0,nop,wscale 9], length 0
10:32:57.615059 ip: 192.168.128.13 > 142.250.79.206: ICMP echo request, id 71, seq 1, length 8
^C
9 packets captured
9 packets received by filter
0 packets dropped by kernel

OVS-VSCTL SHOW

root@access-gateway:/home/magma# ovs-vsctl show
bc6f8bd1-fae8-48b3-b5bb-d36cf9b9ef8c
Manager "ptcp:6640"
Bridge gtp_br0
Controller "tcp:127.0.0.1:6633"
is_connected: true
Controller "tcp:127.0.0.1:6654"
is_connected: true
fail_mode: secure
Port patch-up
Interface patch-up
type: patch
options: {peer=patch-agw}
Port mtr0
Interface mtr0
type: internal
Port ipfix0
Interface ipfix0
type: internal
Port li_port
Interface li_port
type: internal
Port gtp0
Interface gtp0
type: gtpu
options: {key=flow, remote_ip=flow}
Port g_6402000a
Interface g_6402000a
type: gtpu
options: {csum="false", key=flow, remote_ip="10.0.2.100"}
bfd_status: {diagnostic="Neighbor Signaled Session Down", flap_count="0", forwarding="true", remote_diagnostic="No Diagnostic", remote_state=up, state=up}
Port gtp_br0
Interface gtp_br0
type: internal
Port proxy_port
Interface proxy_port
Bridge uplink_br0
Port dhcp0
Interface dhcp0
type: internal
Port uplink_br0
Interface uplink_br0
type: internal
Port patch-agw
Interface patch-agw
type: patch
options: {peer=patch-up}
ovs_version: "2.15.4"
root@access-gateway:/home/magma#
@asantos-vk asantos-vk added the type: bug Something isn't working label Mar 6, 2024
@asantos-vk
Copy link
Author

updating:
ovs flows keeps being modified while UE tries to pass data. When running watch -n0 ovs-ofctl -O OpenFlow13 dump-flows gtp_br0 table=0 we can see that the flows are not persisting.

image
image

@asantos-vk
Copy link
Author
asantos-vk commented Apr 2, 2024 •
edited
Loading

Also, there are some logs that ovsdb connection is being dropped.
root@magma-agw:/home/magma# tail -f /var/log/openvswitch/ovsdb-server.log
2024-04-02T19:56:12.728Z|00444|reconnect|WARN|unix#840: connection dropped (Connection reset by peer)
2024-04-02T19:56:12.761Z|00445|reconnect|WARN|unix#842: connection dropped (Connection reset by peer)
2024-04-02T19:56:12.777Z|00446|reconnect|WARN|unix#843: connection dropped (Connection reset by peer)
2024-04-02T19:56:12.794Z|00447|reconnect|WARN|unix#844: connection dropped (Connection reset by peer)
2024-04-02T19:56:12.814Z|00448|reconnect|WARN|unix#845: connection dropped (Connection reset by peer)
2024-04-02T19:56:12.828Z|00449|reconnect|WARN|unix#846: connection dropped (Connection reset by peer)
2024-04-02T19:56:12.844Z|00450|reconnect|WARN|unix#847: connection dropped (Connection reset by peer)
2024-04-02T19:56:12.861Z|00451|reconnect|WARN|unix#848: connection dropped (Connection reset by peer)
2024-04-02T19:56:12.880Z|00452|reconnect|WARN|unix#849: connection dropped (Connection reset by peer)
2024-04-02T19:56:12.896Z|00453|reconnect|WARN|unix#850: connection dropped (Connection reset by peer)
^C

@sineer
Copy link
sineer commented Jun 10, 2024

Hello! I think I have the same problem with the virtualbox dev env..

I ping UE and I see ping reply from UE on gtpu_sys_2152 but not on the gtp_br0...

root@magma-dev-focal:/home/magma# tail -f /var/log/openvswitch/ovsdb-server.log
2024-06-10T18:34:28.555Z|00041|jsonrpc|WARN|unix#129: receive error: Connection reset by peer
2024-06-10T18:34:28.555Z|00042|reconnect|WARN|unix#129: connection dropped (Connection reset by peer)
2024-06-10T18:34:28.643Z|00043|jsonrpc|WARN|unix#133: receive error: Connection reset by peer
2024-06-10T18:34:28.643Z|00044|reconnect|WARN|unix#133: connection dropped (Connection reset by peer)
2024-06-10T18:34:28.715Z|00045|jsonrpc|WARN|unix#137: receive error: Connection reset by peer
2024-06-10T18:34:28.715Z|00046|reconnect|WARN|unix#137: connection dropped (Connection reset by peer)
2024-06-10T18:51:15.408Z|00047|jsonrpc|WARN|unix#147: receive error: Connection reset by peer
2024-06-10T18:51:15.409Z|00048|reconnect|WARN|unix#147: connection dropped (Connection reset by peer)
2024-06-10T19:12:13.953Z|00049|ovsdb_jsonrpc_server|INFO|ptcp:6640: remote deconfigured

root@magma-dev-focal:/etc# ovs-vsctl show
8edc4ff5-fb78-4722-8479-32c0a243a6ab
Bridge gtp_br0
Port ipfix0
Interface ipfix0
type: internal
Port mtr0
Interface mtr0
type: internal
Port proxy_port
Interface proxy_port
Port gtp_br0
Interface gtp_br0
type: internal
Port gtp0
Interface gtp0
type: gtpu
options: {key=flow, remote_ip=flow}
Port g_350aa8c0
Interface g_350aa8c0
type: gtpu
options: {csum="false", key=flow, remote_ip="192.168.10.53"}
bfd_status: {diagnostic="Neighbor Signaled Session Down", flap_count="0", forwarding="true", remote_diagnostic="No Diagnostic", remote_state=up, state=up}
Port patch-up
Interface patch-up
type: patch
options: {peer=patch-agw}
Port li_port
Interface li_port
type: internal
Bridge uplink_br0
Port dhcp0
Interface dhcp0
type: internal
Port patch-agw
Interface patch-agw
type: patch
options: {peer=patch-up}
Port uplink_br0
Interface uplink_br0
type: internal
ovs_version: "2.15.4"

I don't udnerstand that Interface g_350aa8c0
            type: gtpu
            options: {csum="false", key=flow, remote_ip="192.168.10.53"}
            bfd_status: {diagnostic="Neighbor Signaled Session Down", flap_count="0", forwarding="true",

I don't know why it is using 192.168.10.53 is IP from ohh yeah it's the IP from the box that the ENB is connected from.

Hmm yeah I could never get it to work. I tried same dev env on linux box, intel mac, reverted to release v1.8 and also using HEAD ... I always end up in the same spot where UE attach, gets an IP (NAT mode) and receive ping but the egress from UE doesn't get routed :(

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
type: bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@sineer @asantos-vk