[go: nahoru, domu]

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

UDP crash #21

Closed
K-Mikonos opened this issue Jul 23, 2022 · 16 comments
Closed

UDP crash #21

K-Mikonos opened this issue Jul 23, 2022 · 16 comments
Labels
bug Something isn't working

Comments

@K-Mikonos
Copy link

hello it seems that UDP issue still crashes server.
to reproduce, try any udp connection (nmap, nc, etc..) through ligolo

@K-Mikonos
Copy link
Author

i've forgotten the most important part......
proxy crash:
panic: PullUp failed

goroutine 76 [running]:
github.com/nicocha30/gvisor-ligolo/pkg/tcpip/stack.(*PacketBuffer).headerView(0xc00011ac20?, 0x85a094?)
/root/go/pkg/mod/github.com/nicocha30/gvisor-ligolo@v0.0.0-20220722133119-738ce3d82e3a/pkg/tcpip/stack/packet_buffer.go:356 +0x145
github.com/nicocha30/gvisor-ligolo/pkg/tcpip/stack.PacketHeader.Slice({0xc0003ba3c0?, 0x400000?})
/root/go/pkg/mod/github.com/nicocha30/gvisor-ligolo@v0.0.0-20220722133119-738ce3d82e3a/pkg/tcpip/stack/packet_buffer.go:492 +0x25
github.com/nicocha30/gvisor-ligolo/pkg/tcpip/stack.(*PacketBuffer).Network(0xc0000f2e00?)
/root/go/pkg/mod/github.com/nicocha30/gvisor-ligolo@v0.0.0-20220722133119-738ce3d82e3a/pkg/tcpip/stack/packet_buffer.go:403 +0x5d
github.com/nicocha30/gvisor-ligolo/pkg/tcpip/transport/udp.(*ForwarderRequest).CreateEndpoint(0xc0001362c0, 0x8000?)
/root/go/pkg/mod/github.com/nicocha30/gvisor-ligolo@v0.0.0-20220722133119-738ce3d82e3a/pkg/tcpip/transport/udp/forwarder.go:77 +0xdc
github.com/nicocha30/ligolo-ng/pkg/proxy/netstack.HandlePacket.func1()
/root/opt/oscp3/Tools/Pivot/ligolo-ng/pkg/proxy/netstack/handlers.go:141 +0x453
created by github.com/nicocha30/ligolo-ng/pkg/proxy/netstack.HandlePacket
/root/opt/oscp3/Tools/Pivot/ligolo-ng/pkg/proxy/netstack/handlers.go:128 +0x885

@nicocha30
Copy link
Owner

That's a weird issue. Maybe related to Gvisor bufferv2 changes.

Right now, I recommend using the 0.3.3 release (https://github.com/nicocha30/ligolo-ng/releases/tag/v0.3.3), I will investigate this issue.

@nicocha30 nicocha30 added the bug Something isn't working label Aug 31, 2022
@AdnaneKhan
Copy link

@nicocha30 I've spent a few hours trying to hunt this down. It looks like the panic happens because the UDP forwarder endpoint tried to read the network header from the UDP packet buffer, but it is not set.

Do you know where the UDP packet is created in the code after being decoded from the ligolo-ng protocol envelope?

@nicocha30
Copy link
Owner

Hi @AdnaneKhan , thanks for trying to hunt this issue!

All the logic is done in the HandlePacket function https://github.com/nicocha30/ligolo-ng/blob/master/pkg/proxy/netstack/handlers.go#L65

What is strange is that it worked with the previous version of Gvisor (the one used with Ligolo-ng v0.3.3).

@AdnaneKhan
Copy link

I'm noticing that while the Windows proxy code has a call to stack.NewPacketBuffer, the Linux code uses the gvisor tun and fdbased packages.

The actual call to create the PacketBuffer happens within that code, right? I'm wondering if that code expects the data to be in a particular format to create the packet with the correct NetworkHeader

@AdnaneKhan
Copy link
AdnaneKhan commented Jan 27, 2023

I added the following "hack" to stack.go in the handler that is passed to udp.NewForwarder, and it prevented the crash:

		ptr_struct := unsafe.Pointer(request)
		ptr_struct = unsafe.Pointer(uintptr(ptr_struct) + uintptr(56))
		ptrTobuf := (*stack.PacketBufferPtr)(ptr_struct)
		ptrTobuf.IncRef()

Essentially I am forcing the PktBufferPtr out of the request struct, and incrementing the reference count. This means that somewhere in the code, the reference count is going to 0, and the packet's buffer is no longer valid. The puzzling part is that this isn't handled by ligolo-code, but instead the gvisor fdbased code.

@virusvfv
Copy link
virusvfv commented Jan 28, 2023

@AdnaneKhan can U tell please more about you patch.. I'm confused about *stack.PacketBufferPtr - there is no such var in stack.go. Maybe U mean *stack.PacketBuffer ??
Or can U please post full code of your stack.go file or maybe create pull request to ligolo-ng with your code...
Thanks !

@AdnaneKhan
Copy link

@virusvfv stack.PacketBufferPtr refers to the PacketBufferPtr object defined in https://github.com/google/gvisor/blob/master/pkg/tcpip/stack/packet_buffer.go, within the stack gvisor package. I forgot to mention that in my attempts to fix I switched the gvisor to the latest instead of the fork. In the fork's case the patch code would have a cast to (**stack.PacketBuffer) (they updated the code to use a wrapper object within the struct instead of a pointer).

You might have to update the memory offset, since the 56 was on my 64-bit system to get to the pointer within the struct.

@virusvfv
Copy link
virusvfv commented Jan 29, 2023

Ok. this works in may case (proxy in Linux x64). @AdnaneKhan Thanks.
So result code for gvisor-ligolo stack.go:

        ....
        // PacketBufferPtr is a pointer to a PacketBuffer.
	type PacketBufferPtr struct {
		// packetBuffer is the underlying packet buffer.
		*stack.PacketBuffer
	}
	// Forward UDP connections
	udpHandler := udp.NewForwarder(ns, func(request *udp.ForwarderRequest) {

		//mem hack by  AdnaneKhan
		ptr_struct := unsafe.Pointer(request)
		ptr_struct = unsafe.Pointer(uintptr(ptr_struct) + uintptr(56))
		ptrTobuf := (*PacketBufferPtr)(ptr_struct)
		ptrTobuf.IncRef()
        ....

@nicocha30 I think U can update your gvisor-ligolo repo with this patch...

@nicocha30
Copy link
Owner

Might be related:
google/gvisor#8025
I will try to update the gvisor stack and see if the issue persist.

@virusvfv @AdnaneKhan have you tested this code on other platforms?

@nicocha30
Copy link
Owner

Tried with the latest gvisor go branch, same issue.

panic: PullUp failed

goroutine 96 [running]:
github.com/nicocha30/gvisor-ligolo/pkg/tcpip/stack.PacketBufferPtr.headerView({0xc00006fc20?}, 0x877005?)
	/home/nicocha30/go/pkg/mod/github.com/nicocha30/gvisor-ligolo@v0.0.0-20230130124148-564eb86f0b49/pkg/tcpip/stack/packet_buffer.go:369 +0x145
github.com/nicocha30/gvisor-ligolo/pkg/tcpip/stack.PacketHeader.Slice({{0xc0000dd400?}, 0x400000?})
	/home/nicocha30/go/pkg/mod/github.com/nicocha30/gvisor-ligolo@v0.0.0-20230130124148-564eb86f0b49/pkg/tcpip/stack/packet_buffer.go:514 +0x25
github.com/nicocha30/gvisor-ligolo/pkg/tcpip/stack.PacketBufferPtr.Network({0xc0000af800?})
	/home/nicocha30/go/pkg/mod/github.com/nicocha30/gvisor-ligolo@v0.0.0-20230130124148-564eb86f0b49/pkg/tcpip/stack/packet_buffer.go:418 +0x5d
github.com/nicocha30/gvisor-ligolo/pkg/tcpip/transport/udp.(*ForwarderRequest).CreateEndpoint(0xc0002e8cc0, 0x2?)
	/home/nicocha30/go/pkg/mod/github.com/nicocha30/gvisor-ligolo@v0.0.0-20230130124148-564eb86f0b49/pkg/tcpip/transport/udp/forwarder.go:77 +0xdc
github.com/nicocha30/ligolo-ng/pkg/proxy/netstack.HandlePacket.func1()
	/home/nicocha30/go/src/ligolo-ng/pkg/proxy/netstack/handlers.go:141 +0x45b
created by github.com/nicocha30/ligolo-ng/pkg/proxy/netstack.HandlePacket
	/home/nicocha30/go/src/ligolo-ng/pkg/proxy/netstack/handlers.go:128 +0x885

@AdnaneKhan
Copy link

@nicocha30 The "hack" code? No, just Linux x64. It definitely would need to be improved in order to be more generic (also not sure if the struct size would be different in case of ipv6, etc.)

I think the logic would need to be:

  1. Get size of the UDP ForwarderRequest struct in bytes (using request_struct_size = unsafe.Sizeof(*request))
  2. Get size of ptr (8 bytes on 64 bit, 4 on 32) (few different ways to do this)
  3. Subtract the ptr size from the struct size, this should get you the packetbuffer ptr value, since it is the last entry in the struct
  4. Increment the reference

Of course, this avoids the question of why the reference count is off in the first place. It works for TCP, so it makes me think that there is some behavior where the underlying gvisor code is freeing that UDP forwarder request and, as a result, the packet buffer after the ligolo code encodes the data and sends it to agent.

@AdnaneKhan
Copy link

I made a PR with the "temporary fix" that should prevent the crash until the root cause can be identified and fixed. #28

@nicocha30
Copy link
Owner
nicocha30 commented Feb 1, 2023

Hey, looks like gvisor may have fixed the issue:
google/gvisor#8458
Edit:
Tested and it works! Pushing right now.

@nicocha30
Copy link
Owner

I released v0.4.2 which fix the issue, thanks a lot @AdnaneKhan and @virusvfv for hunting this issue.

https://github.com/nicocha30/ligolo-ng/releases/tag/v0.4.2

@AdnaneKhan
Copy link

Wow, this is awesome; pretty cool that Google fixed it!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

4 participants