What problem are you facing?

Currently, it's possible to set nil for a field in fieldpath library but that still leaves the map with that key whose value is nil. Later on, when this map is sent to api-server, it doesn't pass the validation since it should've been omitted if it was nil.

How could Crossplane help solve your problem?

Here is code snippet from unstructured.Unstructured:

func (u *Unstructured) SetLabels(labels map[string]string) {
	if labels == nil {
		RemoveNestedField(u.Object, "metadata", "labels")
		return
	}
	u.setNestedMap(labels, "metadata", "labels")
}

As you can see, in case of nil value, they remove the field completely instead of setting a nil value. I'd suggest we do the same thing in SetValue function in fieldpath.