[go: nahoru, domu]

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

help request: limit-req #11288

Open
dimon-v opened this issue May 24, 2024 · 8 comments
Open

help request: limit-req #11288

dimon-v opened this issue May 24, 2024 · 8 comments

Comments

@dimon-v
Copy link
dimon-v commented May 24, 2024

Description

May I ask if the implementation principles of apisix limit req and nginx are the same? According to the official documentation, the leaky bucket algorithm is used. If the rate is 1000/s burst=0 nodelay, The number of access entries for nginx through access log grep 200 is less than the rate value, while apisix is much greater than the rate value

Referring to:
alibaba/tengine#855
https://github.com/openresty/lua-resty-limit-traffic/blob/master/lib/resty/limit/req.md
https://groups.google.com/g/openresty/c/VY-LdQaEyDA/m/skf86NDHAAAJ?pli=1

route

{
  "uri": "/rate",
  "name": "rate-limit",
  "priority": 100,
  "host": "my-yace.com",
  "plugins": {
    "limit-req": {
      "burst": 1,
      "disable": false,
      "nodelay": true,
      "key": "server_addr",
      "rate": 1000,
      "rejected_code": 503
    }
  },
  "upstream": {
    "nodes": [
      {
        "host": "10.120.0.104",
        "port": 80,
        "weight": 1
      }
    ],
    "timeout": {
      "connect": 6,
      "send": 300,
      "read": 300
    },
    "type": "roundrobin",
    "scheme": "http",
    "pass_host": "pass",
    "keepalive_pool": {
      "idle_timeout": 300,
      "requests": 1000,
      "size": 1000
    }
  },
  "status": 1
}

1、Using WRK testing

Running 1m test @ http://10.120.0.113:9080/rate

2 threads and 2000 connections

Thread calibration: mean lat.: 5041.922ms, rate sampling interval: 21381ms

Thread calibration: mean lat.: 5870.648ms, rate sampling interval: 21889ms

Thread Stats   Avg      Stdev     Max   +/- Stdev

Latency    33.93s    12.77s    0.96m    57.86%

Req/Sec    10.22k    38.16    10.26k    50.00%

1250705 requests in 1.00m, 313.49MB read

Socket errors: connect 0, read 3, write 628, timeout 2567

Non-2xx or 3xx responses: 421753

Requests/sec:  20844.28

Transfer/sec:      5.22MB

2、Confirm that the request has arrived at Apisix

\* About to connect() to 10.120.0.113 port 9080 (#0)

\*   Trying 10.120.0.113...

\* Connected to 10.120.0.113 (10.120.0.113) port 9080 (#0)

\> GET /rate HTTP/1.1

\> User-Agent: curl/7.29.0

\> Accept: */*

\> Host: my-yace.com

\> 

< HTTP/1.1 200 OK

< Content-Type: text/plain; charset=utf-8

< Content-Length: 0

< Connection: keep-alive

< Date: Fri, 24 May 2024 06:48:09 GMT

< Server: APISIX/3.2.2

< 

\* Connection #0 to host 10.120.0.113 left intact

3 Number of grep log entries

grep 'GET /rate HTTP/1.1' access.log | grep 200 | grep '14:4' | awk '{print $4}' | sort | uniq -c



9334 [24/May/2024:14:40:00

9833 [24/May/2024:14:40:01

14500 [24/May/2024:14:40:02

14340 [24/May/2024:14:40:03

14648 [24/May/2024:14:40:04

14445 [24/May/2024:14:40:05

14849 [24/May/2024:14:40:06

13449 [24/May/2024:14:40:07

13938 [24/May/2024:14:40:08

13796 [24/May/2024:14:40:09

14089 [24/May/2024:14:40:10

14876 [24/May/2024:14:40:11

14499 [24/May/2024:14:40:12

12827 [24/May/2024:14:40:13

14015 [24/May/2024:14:40:14

14330 [24/May/2024:14:40:15

12880 [24/May/2024:14:40:16

13783 [24/May/2024:14:40:17

13856 [24/May/2024:14:40:18

14909 [24/May/2024:14:40:19

10330 [24/May/2024:14:40:20

10198 [24/May/2024:14:40:21

13245 [24/May/2024:14:40:22

13757 [24/May/2024:14:40:23

13998 [24/May/2024:14:40:24

10176 [24/May/2024:14:40:25

1478 [24/May/2024:14:40:26
  1. Log fragments

4.1 access

10.120.0.104 - - [24/May/2024:14:39:34 +0800] my-yace.com "GET /rate HTTP/1.1" 503 269 0.000 "-" "-" - - - "http://my-yace.com"
10.120.0.104 - - [24/May/2024:14:39:34 +0800] my-yace.com "GET /rate HTTP/1.1" 200 0 0.027 "-" "-" 10.120.0.104:80 200 0.027 "http://my-yace.com"
10.120.0.104 - - [24/May/2024:14:39:34 +0800] my-yace.com "GET /rate HTTP/1.1" 503 269 0.000 "-" "-" - - - "http://my-yace.com"
10.120.0.104 - - [24/May/2024:14:39:34 +0800] my-yace.com "GET /rate HTTP/1.1" 503 269 0.000 "-" "-" - - - "http://my-yace.com"
10.120.0.104 - - [24/May/2024:14:39:34 +0800] my-yace.com "GET /rate HTTP/1.1" 503 269 0.000 "-" "-" - - - "http://my-yace.com"
10.120.0.104 - - [24/May/2024:14:39:34 +0800] my-yace.com "GET /rate HTTP/1.1" 200 0 0.027 "-" "-" 10.120.0.104:80 200 0.027 "http://my-yace.com"
10.120.0.104 - - [24/May/2024:14:39:34 +0800] my-yace.com "GET /rate HTTP/1.1" 200 0 0.027 "-" "-" 10.120.0.104:80 200 0.027 "http://my-yace.com"
10.120.0.104 - - [24/May/2024:14:39:34 +0800] my-yace.com "GET /rate HTTP/1.1" 200 0 0.027 "-" "-" 10.120.0.104:80 200 0.027 "http://my-yace.com"
10.120.0.104 - - [24/May/2024:14:39:34 +0800] my-yace.com "GET /rate HTTP/1.1" 200 0 0.027 "-" "-" 10.120.0.104:80 200 0.027 "http://my-yace.com"
10.120.0.104 - - [24/May/2024:14:39:34 +0800] my-yace.com "GET /rate HTTP/1.1" 200 0 0.027 "-" "-" 10.120.0.104:80 200 0.027 "http://my-yace.com"

4.2、 error

2024/05/24 14:40:26 [warn] 6913#6913: *83358 [lua] plugin.lua:1100: run_plugin(): limit-req exits with http status code 503, client: 10.120.0.104, server: _, request: "GET /rate HTTP/1.1", host: "my-yace.com"
2024/05/24 14:40:26 [warn] 6913#6913: *92234 [lua] plugin.lua:1100: run_plugin(): limit-req exits with http status code 503, client: 10.120.0.104, server: _, request: "GET /rate HTTP/1.1", host: "my-yace.com"
2024/05/24 14:40:26 [warn] 6913#6913: *80673 [lua] plugin.lua:1100: run_plugin(): limit-req exits with http status code 503, client: 10.120.0.104, server: _, request: "GET /rate HTTP/1.1", host: "my-yace.com"
2024/05/24 14:40:26 [warn] 6912#6912: *82365 [lua] plugin.lua:1100: run_plugin(): limit-req exits with http status code 503, client: 10.120.0.104, server: _, request: "GET /rate HTTP/1.1", host: "my-yace.com"
2024/05/24 14:40:26 [warn] 6913#6913: *86209 [lua] plugin.lua:1100: run_plugin(): limit-req exits with http status code 503, client: 10.120.0.104, server: _, request: "GET /rate HTTP/1.1", host: "my-yace.com"
2024/05/24 14:40:26 [warn] 6913#6913: *80674 [lua] plugin.lua:1100: run_plugin(): limit-req exits with http status code 503, client: 10.120.0.104, server: _, request: "GET /rate HTTP/1.1", host: "my-yace.com"
2024/05/24 14:40:26 [warn] 6913#6913: *83544 [lua] plugin.lua:1100: run_plugin(): limit-req exits with http status code 503, client: 10.120.0.104, server: _, request: "GET /rate HTTP/1.1", host: "my-yace.com"
2024/05/24 14:40:26 [warn] 6913#6913: *83847 [lua] plugin.lua:1100: run_plugin(): limit-req exits with http status code 503, client: 10.120.0.104, server: _, request: "GET /rate HTTP/1.1", host: "my-yace.com"

Environment

  • APISIX version (run apisix version): 3.2.2

  • Operating system (run uname -a): Linux nginx-gray02 3.10.0-1160.31.1.el7.x86_64 change: added doc of how to load plugin. #1 SMP Thu Jun 10 13:32:12 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

  • OpenResty / Nginx version (run openresty -V or nginx -V):

  • nginx version: openresty/1.21.4.2
    built by gcc 9.3.1 20200408 (Red Hat 9.3.1-2) (GCC)
    built with OpenSSL 1.1.1s 1 Nov 2022 (running with OpenSSL 1.1.1w 11 Sep 2023)
    TLS SNI support enabled
    configure arguments: --prefix=/usr/local/openresty/nginx --with-cc-opt='-O2 -DAPISIX_BASE_VER=1.21.4.2.2 -DNGX_GRPC_CLI_ENGINE_PATH=/usr/local/openresty/libgrpc_engine.so -DNGX_HTTP_GRPC_CLI_ENGINE_PATH=/usr/local/openresty/libgrpc_engine.so -DNGX_LUA_ABORT_AT_PANIC -I/usr/local/openresty/zlib/include -I/usr/local/openresty/pcre/include -I/usr/local/openresty/openssl111/include' --add-module=../ngx_devel_kit-0.3.2 --add-module=../echo-nginx-module-0.63 --add-module=../xss-nginx-module-0.06 --add-module=../ngx_coolkit-0.2 --add-module=../set-misc-nginx-module-0.33 --add-module=../form-input-nginx-module-0.12 --add-module=../encrypted-session-nginx-module-0.09 --add-module=../srcache-nginx-module-0.33 --add-module=../ngx_lua-0.10.25 --add-module=../ngx_lua_upstream-0.07 --add-module=../headers-more-nginx-module-0.34 --add-module=../array-var-nginx-module-0.06 --add-module=../memc-nginx-module-0.19 --add-module=../redis2-nginx-module-0.15 --add-module=../redis-nginx-module-0.3.9 --add-module=../ngx_stream_lua-0.0.13 --with-ld-opt='-Wl,-rpath,/usr/local/openresty/luajit/lib -Wl,-rpath,/usr/local/openresty/wasmtime-c-api/lib -L/usr/local/openresty/zlib/lib -L/usr/local/openresty/pcre/lib -L/usr/local/openresty/openssl111/lib -Wl,-rpath,/usr/local/openresty/zlib/lib:/usr/local/openresty/pcre/lib:/usr/local/openresty/openssl111/lib' --add-module=/tmp/tmp.2MSCN59HKX/openresty-1.21.4.2/../mod_dubbo-1.0.2 --add-module=/tmp/tmp.2MSCN59HKX/openresty-1.21.4.2/../ngx_multi_upstream_module-1.1.1 --add-module=/tmp/tmp.2MSCN59HKX/openresty-1.21.4.2/../apisix-nginx-module-1.15.0 --add-module=/tmp/tmp.2MSCN59HKX/openresty-1.21.4.2/../apisix-nginx-module-1.15.0/src/stream --add-module=/tmp/tmp.2MSCN59HKX/openresty-1.21.4.2/../apisix-nginx-module-1.15.0/src/meta --add-module=/tmp/tmp.2MSCN59HKX/openresty-1.21.4.2/../wasm-nginx-module-0.6.5 --add-module=/tmp/tmp.2MSCN59HKX/openresty-1.21.4.2/../lua-var-nginx-module-v0.5.3 --add-module=/tmp/tmp.2MSCN59HKX/openresty-1.21.4.2/../grpc-client-nginx-module-v0.4.4 --with-poll_module --with-pcre-jit --with-stream --with-stream_ssl_module --with-stream_ssl_preread_module --with-http_v2_module --without-mail_pop3_module --without-mail_imap_module --without-mail_smtp_module --with-http_stub_status_module --with-http_realip_module --with-http_addition_module --with-http_auth_request_module --with-http_secure_link_module --with-http_random_index_module --with-http_gzip_static_module --with-http_sub_module --with-http_dav_module --with-http_flv_module --with-http_mp4_module --with-http_gunzip_module --with-threads --with-compat --with-stream --with-http_ssl_module

  • etcd version, if relevant (run curl http://127.0.0.1:9090/v1/server_info): {"id":"66356a45-bcc5-4a5a-9318-210342728ebd","boot_time":1716530994,"version":"3.2.2","etcd_version":"3.4.0","hostname":"nginx-gray02"}

  • APISIX Dashboard version, if relevant:

  • Plugin runner version, for issues related to plugin runners:

  • LuaRocks version, for installation issues (run luarocks --version):

@dimon-v
Copy link
Author
dimon-v commented May 24, 2024

By adding logs in/openresty/liulib/resty/limit/req.lua, the calculated elapsed has a complex number when the request is fast, After calculating the ABS function, Excess is 0

image
1 grep error log
image

@dimon-v
Copy link
Author
dimon-v commented May 24, 2024

image

@dimon-v
Copy link
Author
dimon-v commented May 27, 2024

When using OpenResty Limit Req, Set the rate to 200 Burst setting 1, after using WRK stress testing, I calculated that the number of 200 status entries in the access log is less than the rate ( see https://github.com/bingoohuang/blog/issues/209)

lua_shared_dict my_limit_req_store 100m;

server {
listen 8700;
default_type text/html;

location / {
    return 200 OK;
}

location /limit {
    access_by_lua_block {
        local limit_req = require "resty.limit.req"

        -- 限制每秒 200 个请求 (rate),以及 100 的等待队列 (burst), 超过每次 300,直接拒绝
        local rate = tonumber(ngx.var.arg_rate or 200)
        local burst = tonumber(ngx.var.arg_burst or 100)
        local lim, err = limit_req.new("my_limit_req_store", rate, burst)
        if not lim then
            ngx.log(ngx.ERR, "failed to instantiate a resty.limit.req object: ", err)
            return ngx.exit(500)
        end

        -- 以远端IP为限制 key
        local delay, excess = lim:incoming(ngx.var.binary_remote_addr, true)
        if excess == "rejected" then
            ngx.log(ngx.ERR, "rejected")
            -- ngx.say, ngx.print 发送消息包,必然要先发送消息头,所以需要提前设置 response status (默认为200)
            ngx.status = 503
            ngx.say("XX delay:", delay, ", rate: ", rate, ", burst: ", burst)
            return ngx.exit(503)
        end

        ngx.say("OK delay: ", delay, ", rate: ", rate, ", burst: ", burst, ", excess: ", excess)
        return ngx.exit(200)
    }
}

}

@dimon-v dimon-v closed this as completed May 27, 2024
@dimon-v dimon-v reopened this May 27, 2024
@zsmlinux

This comment was marked as resolved.

@zsmlinux

This comment was marked as resolved.

@rahulracker7539
Copy link

Hi, any update ? I have the same problem where limit-req plugin in not working as intended and not able to block the request.

@ShenFeng312
Copy link
Contributor
ShenFeng312 commented Jul 12, 2024

@rahulracker7539
Copy link

it was silly mistake from my side, had add my plugin in the plugin section of configmap so it can install and load to the apisix instance. I was just doing from apisix dashboard which not able to install automatically . thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: 📋 Backlog
Development

No branches or pull requests

4 participants