[go: nahoru, domu]

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bug: dns service discovery upstreams are not working in GKE #6732

Closed
ukanwat opened this issue Mar 28, 2022 · 5 comments · Fixed by #6739 or #6764
Closed

bug: dns service discovery upstreams are not working in GKE #6732

ukanwat opened this issue Mar 28, 2022 · 5 comments · Fixed by #6739 or #6764

Comments

@ukanwat
Copy link
ukanwat commented Mar 28, 2022

Current Behavior

APISIX is throwing an error related to port being 0 when upstream has discovery_type "dns".
same exact setup works fine with clusters from other kubernetes providers.

Expected Behavior

Should not throw the error http_access_phase(): failed to set upstream: invalid nodes format: failed to validate item 1: property "port" validation failed: expected 0 to be at least 1

Error Logs

in web response -
503 Service Temporarily Unavailable
in apisix pod -
init.lua:520: http_access_phase(): failed to set upstream: invalid nodes format: failed to validate item 1: property "port" validation failed: expected 0 to be at least 1

Steps to Reproduce

  1. Create a standard Google Kubernetes Engine cluster.
  2. Install APISIX with ingress controller and dashboard via api7 helm chart version 0.8.4 using this values.yaml file (in ingress-apisix namespace).
  3. create this upstream via admin api - { "id": "398808046194655761", "create_time": 1647237719, "update_time": 1647237719, "timeout": { "connect": 6, "send": 6, "read": 6 }, "type": "roundrobin", "scheme": "http", "discovery_type": "dns", "pass_host": "node", "name": "apisix-dashboard", "service_name": "apisix-dashboard.ingress-apisix.svc.cluster.local", "keepalive_pool": { "idle_timeout": 60, "requests": 1000, "size": 320 } }
    4. create this route via admin api - {"id":"398808266596942353","create_time":1647237850,"update_time":1647441243,"uri":"/*","name":"apisix-dashboard","priority":50,"methods":["GET","POST","PUT","DELETE","PATCH","HEAD","OPTIONS","CONNECT","TRACE"],"host":"apisix-dashboard.example.com","plugins":{"redirect":{"http_to_https":true}},"upstream_id":"398808046194655761","labels":{"API_VERSION":"v1"},"status":1}
    5. visit the host in the route and then check the logs in the apisix pod.

Environment

  • APISIX version (run apisix version): 2.12.1
  • Operating system (run uname -a): kubernetes (GKE)
    Rest of the values are whatever is bundled in apisix/apisix version 0.8.4 helm chart.
  • OpenResty / Nginx version (run openresty -V or nginx -V):
  • etcd version, if relevant (run curl http://127.0.0.1:9090/v1/server_info):
  • APISIX Dashboard version, if relevant:
  • Plugin runner version, for issues related to plugin runners:
  • LuaRocks version, for installation issues (run luarocks --version):
@spacewander
Copy link
Member

I guess GKE's SRV record is an SRV with port zero. Would you provide a packet capture file to prove it? Thanks!

@ukanwat
Copy link
Author
ukanwat commented Mar 28, 2022

In the GKE dashboard, Target Port is shown as 0 and in the service config it is http. Could this mean that SRV record has port zero?
Also what should I do to get the packet capture file?

@tzssangglass
Copy link
Member

Also what should I do to get the packet capture file?

tcpdump : https://www.tcpdump.org/manpages/tcpdump.1.html

@shuaijinchao
Copy link
Member

Checking only SRV records can also use nslookup to query the target DNS server.

@spacewander
Copy link
Member

As a workaround, you can add the port to the "service_name" ("apisix-dashboard.ingress-apisix.svc.cluster.local").

spacewander added a commit to spacewander/incubator-apisix that referenced this issue Mar 31, 2022
It makes the workaround in apache#6732 (comment)
work.
Signed-off-by: spacewander <spacewanderlzx@gmail.com>
spacewander added a commit to spacewander/incubator-apisix that referenced this issue Mar 31, 2022
It makes the workaround in apache#6732 (comment)
work.
Signed-off-by: spacewander <spacewanderlzx@gmail.com>
spacewander added a commit that referenced this issue Apr 1, 2022
It makes the workaround in #6732 (comment)
work.
Signed-off-by: spacewander <spacewanderlzx@gmail.com>
Liu-Junlin pushed a commit to Liu-Junlin/apisix that referenced this issue May 20, 2022
…he#6764)

It makes the workaround in apache#6732 (comment)
work.
Signed-off-by: spacewander <spacewanderlzx@gmail.com>
spacewander added a commit that referenced this issue Jun 30, 2022
It makes the workaround in #6732 (comment)
work.
Signed-off-by: spacewander <spacewanderlzx@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
4 participants