[go: nahoru, domu]

WO2006032028A2 - Metric-based monitoring and control of a limited resource - Google Patents

Metric-based monitoring and control of a limited resource Download PDF

Info

Publication number
WO2006032028A2
WO2006032028A2 PCT/US2005/033163 US2005033163W WO2006032028A2 WO 2006032028 A2 WO2006032028 A2 WO 2006032028A2 US 2005033163 W US2005033163 W US 2005033163W WO 2006032028 A2 WO2006032028 A2 WO 2006032028A2
Authority
WO
WIPO (PCT)
Prior art keywords
metric
threshold
recited
value
event
Prior art date
Application number
PCT/US2005/033163
Other languages
French (fr)
Other versions
WO2006032028A3 (en
Inventor
Jared Smith-Mickelson
Maxim Zhilyaev
Original Assignee
Reactivity, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Reactivity, Inc. filed Critical Reactivity, Inc.
Publication of WO2006032028A2 publication Critical patent/WO2006032028A2/en
Publication of WO2006032028A3 publication Critical patent/WO2006032028A3/en

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00Network architectures or network communication protocols for network security
    • H04L63/14Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
    • H04L63/1408Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
    • H04L63/1416Event detection, e.g. attack signature detection
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00Arrangements for monitoring or testing data switching networks
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00Network architectures or network communication protocols for network security
    • H04L63/14Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
    • H04L63/1408Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
    • H04L63/1425Traffic logging, e.g. anomaly detection
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00Network architectures or network communication protocols for network security
    • H04L63/14Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
    • H04L63/1441Countermeasures against malicious traffic
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00Network architectures or network communication protocols for network security
    • H04L63/14Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
    • H04L63/1441Countermeasures against malicious traffic
    • H04L63/1458Denial of Service
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00Arrangements for monitoring or testing data switching networks
    • H04L43/16Threshold monitoring
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00Network architectures or network communication protocols for network security
    • H04L63/02Network architectures or network communication protocols for network security for separating internal from external traffic, e.g. firewalls

Definitions

  • the present invention relates generally to monitoring and controlling access to a limited resource. More specifically, a metric-based approach to monitoring and controlling access to a limited resource is disclosed.
  • a limited resource is a network resource, such as a server or other system accessed via a network.
  • Network- connected systems have a limited ability to process and exchange (e.g., send, receive) data. Errors or other failures can occur when the processing and/or communication capacity of such a system is exceeded, either as the result of high legitimate demand or malicious attack (e.g., so-called "denial of service” attacks). It is desirable to have an efficient way to detect when such an error or failure condition may occur and, if desired, to limit access or usage to a level that will enable such errors and/or failures to be avoided.
  • One prior art approach to detecting and/or preventing a condition that might result in an error or failure, or that might result in a quality of service guarantee or some other relevant threshold or level of use being exceeded e.g., has been to define a "sliding window" and track the demand made on the system (e.g., the number or cumulative size of the messages or other communications received) during a period defined by the window. For example, under one typical approach one might track how many messages were received in the last X seconds. In some cases, an alert may be sent or other responsive action taken if more than N messages were received in a period of X seconds.
  • a similar approach might be used to track how many transactions were charged to the account in a given hour, week, day, etc., with an alert being generated if the use exceeds a prescribed threshold.
  • such an approach consumes a lot of processing and memory resources, as it is necessary to keep track of lots of data, such as which messages have been received (or transactions completed in the credit card example) and at what time, and continually (or at least periodically) perform computations on such information to determine the number of messages received within the sliding analysis window.
  • Figure IA illustrates a client-server environment.
  • Figure IB illustrates a client-server environment that includes a firewall.
  • Figure 1C illustrates yet another illustrative client-server environment.
  • Figure 2A illustrates a process used in one embodiment to implement a metric-based approach to detecting and/or preventing demand for a resource, such as a server, from exceeding a prescribed or otherwise applicable limit.
  • Figure 2B illustrates a process used in some embodiments to implement a metric-based approach to monitoring and/or controlling access to a resource.
  • Figure 3 illustrates a process used in some embodiments to block messages for a period in response to a threshold leaving for a metric being exceeded.
  • Figure 4 shows a plot 400 of the value of the metric m over time in an embodiment in which different weights may be assigned to different messages or message types.
  • the invention can be implemented in numerous ways, including as a process, an apparatus, a system, a composition of matter, a computer readable medium such as a computer readable storage medium or a computer network wherein program instructions are sent over optical or electronic communication links.
  • these implementations, or any other form that the invention may take, may be referred to as techniques.
  • the order of the steps of disclosed processes may be altered within the scope of the invention.
  • a detailed description of one or more embodiments of the invention is provided below along with accompanying figures that illustrate the principles of the invention. The invention is described in connection with such embodiments, but the invention is not limited to any embodiment. The scope of the invention is limited only by the claims and the invention encompasses numerous alternatives, modifications and equivalents.
  • a metric-based approach to detecting when conditions are such that demand, such as for a network-accessible resource, may exceed the bandwidth available and/or allotted to satisfy the demand, or when actual use of a resource deviates from an expected, normal, and/or permitted level of use, is disclosed.
  • a metric is incremented each time use event occurs, e.g., each time a message is received by a network based resource such as a firewall, gateway, server, etc., and decayed over time.
  • a responsive action is taken if the metric exceeds a threshold. In some embodiments, if the metric exceeds the threshold value subsequent attempts to use the resource, e.g., network communications from a particular offending source, may be blocked for a period.
  • FIG. IA illustrates a client-server environment.
  • a client 102 has a connection 104 to a server 106.
  • the client 102 may be a computer or other system configured to communicate with server 106 via connection 104, e.g., to request data from server 106.
  • Server 106 may be a computer or other system or process configured to receive data from and/or provide data to a client such as client 102 via connection 104. While a single client 102 and server 106 are shown, a typical commercial application may involve many clients and/or more than one server.
  • the connection 104 may be a direct connection, but more typically it is a network connection over a private network (e.g., a LAN or WAN) and/or a public network, such as the Internet.
  • the server 106 is configured to regulate messages, either globally or based on source, class, size, destination, user, account, data included in the message, or other criteria, using the metric-based approach described herein.
  • Figure IB illustrates a client-server environment that includes a firewall.
  • the client 102 is connected via a network 108 to a firewall 110.
  • Firewall 110 is configured to receive messages sent by client 102 to server 106 via network 108 and forward such messages to server 106 if forwarding criteria are satisfied.
  • the firewall 110 is configured to apply the metric-based approach described herein to prevent the client 102, or clients generally, from exceeding the bandwidth on server 106 that is available and/or has been allotted to it/them.
  • the firewall 110 blocks subsequent messages associated with the metric that has exceeded its associated threshold, e.g., messages from the affected source(s) and/or of the affected type.
  • FIG. 1C illustrates yet another illustrative client-server environment.
  • a plurality of clients Cl to Cn represented in Figure 1C by clients 120 and 122, are connected via network 124 to firewall 126.
  • Clients Cl to Cn send , messages to one or more of a plurality of backend servers Sl to Sm associated with firewall 126, represented in Figure 1C by servers 128, 130, and 132.
  • firewall 126 e.g., to detect and/or prevent the demand (e.g., messages) made on one or more of the servers Sl to Sm from exceeding an applicable limit, such as to detect and/or avoid a condition that may be associated with and/or similar in its consequences to a denial of service type attack.
  • the demand e.g., messages
  • each of servers Sl to Sm has associated with it a corresponding metric and threshold, not necessarily the same as the corresponding metric and threshold for one or more other of the servers Sl to Sm, and firewall 126 uses the metric for each server to throttle, i.e., control the rate of, data transmission to that server so as to not exceed a communication and/or processing bandwidth and/or other capacity constraint of the server.
  • throttle i.e., control the rate of, data transmission to that server so as to not exceed a communication and/or processing bandwidth and/or other capacity constraint of the server.
  • an enterprise may have two or more servers configured to perform a particular service, e.g., to host a particular application, but due to differences in hardware, configuration, etc. one server may have a higher capacity than the other(s).
  • the firewall tracks incoming connections/transmissions on a per client basis on the client side of firewall 126 (connected to network 124) and uses a per-server metric (for all clients, classes, etc.) on the server side (connected to servers 128-132) to throttle traffic to the servers. So long as the metric associated with a client has not been exceeded, it is able to connect to those of servers 128-132 that have not had traffic to them blocked as a result of their respective metrics exceeding their associated thresholds.
  • the firewall 126 blocks traffic to the affected server(s) for a time, and directs/redirects connections that otherwise would have been made to the affected server(s) to one or more other, unaffected servers for which the associated metric has not exceeded its applicable threshold.
  • the firewall uses a round robin approach to direct connection requests to servers Sl to Sm and skips any server for which the new connection would result in the server's metric exceeding its threshold.
  • the firewalls shown in Figures IB and 1C may comprise an XML firewall configured to validate and/or otherwise process requests or messages comprising XML documents, such as SOAP or other messages being sent to a backend server for processing in a web services context.
  • the firewall may monitor use of a financial or other account, or other data usable to detect a use pattern that deviates from an expected, normal, and/or permitted level of use, such as may indicate unauthorized use such as may occur in the case of theft (including identity theft) or fraud.
  • data transmissions associated with account use such as charge authorizations, debit transactions, etc., are monitored by maintaining a metric on a per user and/or per account basis. For each authorization/transaction, user and/or account identifier data, as applicable, is determined and an associated metric is updated. If the metric exceeds an associated threshold, e.g., as a result of an unusual number of transactions in a short period and/or transactions involving unusually large amounts, a responsive action (block further use, alert, etc.) is taken.
  • an associated threshold e.g., as a result of an unusual number of transactions in a short period and/or transactions involving unusually large amounts
  • the value of the metric is incremented on occurrence of each event by a weighted amount determined based on data associated with the event, e.g., by the size of a received data message in bytes, the dollar amount of a transaction, etc.
  • a weighted amount determined based on data associated with the event, e.g., by the size of a received data message in bytes, the dollar amount of a transaction, etc.
  • Figure 2A illustrates a process used in one embodiment to implement a metric-based approach to detecting and/or preventing demand for a resource, such as a server, from exceeding a prescribed or otherwise applicable limit.
  • a message is received (202).
  • an event other than receiving a message such as receiving an indication that a credit card has been used, occurs at 202.
  • the value of a metric used to monitor and/or limit messages e.g., to detect or prevent a denial of service attack or similar condition, is decayed (i.e., decremented) by an amount associated with the amount of time that elapsed between the message currently being processed and the next most recently received message (204).
  • the metric is decayed by an amount determined at least in part on data other than the amount of time that elapsed between the message currently being processed and the next most recently received message, such as the time of day, the size of the message, the nature/type of message, use conditions generally (i.e., across users), etc.
  • a separate metric is maintained for separate classes or types of message and in such embodiments each respective metric is decayed in 204 by an amount associated with the amount of time that has elapsed between the message currently being processed and the next most recently received message of the class/type with which the particular metric being decayed is associated.
  • an instance of the metric is used to limit the number of unauthorized messages to three messages per minute.
  • the time considered in calculating the amount of decay is the time that has elapsed since receipt of the unauthorized message that is currently being processed and the last unauthorized message, although one or more (even many) authorized messages may have been received in the time in between.
  • the value of the metric is decayed in 204 by multiplying the value of the metric by a base "b" raised to an exponent equal to the time "dt" that elapsed between the current and next most recently received messages associated with the metric, such that the decay is exponential over time.
  • the decay is linear, e.g., the metric is decayed by subtracting from the current value of the metric the product of a base "b" and the elapsed time "dt".
  • a hybrid approach is used, e.g., by decaying the value of the metric exponentially above the threshold and linearly below. The latter approach is used in some embodiments to avoid false detections in environments, such as test environments, in which periodic use patterns may occur.
  • exponential decay at least for the region above the threshold, is used to enable a block (or alert, etc.) state to be cleared more quickly after a detection event caused by a single large (i.e., heavily weighted) event and/or a burst of smaller events causes the metric value to spike.
  • the value of the metric is incremented by an amount associated with the message currently being processed.
  • the value of each message is the same and in such embodiments the same amount (e.g., 1) is added to the metric in step 206 regardless of the nature or source of the message being processed.
  • a weighted value associated with some attribute of the message is added to the metric in step 206.
  • Equation (1) summarizes the result of performing steps 204 and 206 in the general case in which a weighted value "W k " associated with the "k-th" received message, in an embodiment in which exponential decay is used.
  • m k is the value of the metric after receipt of the k-th message (i.e., the message currently being processed)
  • ni k -i is the value of the metric at the time of receipt of the message received just prior to the k-th message
  • b is a based between 0 and 1, determined in some embodiments as described below
  • 'V k " is the weight value of the k-th message.
  • Equation (1) While in Equation (1) the amount the metric is decayed is determined at least in part by the time elapsed between messages of the type (e.g., from the source, to the server, etc.) with which the metric is associated, in various embodiments other and/or different data is used to determine the amount the value of the metric is decayed, e.g., the time of day of the last previous message, the time of receipt of the current message, the day of the week (e.g., whether it is a weekday, weekend, holiday, etc.), a weight value associated with the last previous message, a weight value associated with the current message, the threshold value for the metric, the proximity of the metric to the threshold, the value of the metric relative to the threshold (e.g., greater or lesser), one or more constants (e.g., the time of day of the last previous message, the time of receipt of the current message, the day of the week (e.g., whether it is a weekday, weekend, holiday, etc.),
  • a threshold or maximum permissible value "M" of the metric is established and it is determined in step 208 that the threshold has been exceeded if the value of the metric "m" is greater than the threshold "M” (i.e., m > M).
  • the value of the threshold "M” for a given instance of a metric "m” may vary, depending on such factors as the time of day, day of the week, season, external conditions (e.g., environmental or weather conditions), status and availability of related resources (e.g., other servers able to process the message).
  • a credit card transaction (or a closely spaced burst of transactions) may be afforded greater weight at 4 a.m. than at 4 p.m., for a metric used to detect unauthorized use, e.g., on the theory that use at an unexpected level at a time when most users are asleep and most businesses closed is more suspicious than an increased use of similar proportions that occurs during the business day.
  • the equivalent of exponential decay is implemented by incrementing the value of the metric logarithmically at 208 and then applying linear decay at 206.
  • the process returns to step 202 and the next message (or other event) is received and processed. If an applicable threshold has been exceeded, responsive action is taken (210). In some embodiments, the responsive action comprises generating an alert. In some embodiments, the responsive action comprises blocking messages from one or more sources associated with the metric and/or messages of one or more classes or types of message associated with the metric, if the threshold is exceeded. In some such embodiments, the blocking continues until the value of the metric has decayed to a level such that at least one message of a prescribed weight (or at least one message of an expected or typical type or size, in an embodiment in which weights are not assigned) could be received without the value of the metric exceeding the threshold.
  • the responsive action may include notifying an owner and/or provider of the account (such as a bank), notifying public or private enforcement authorities, freezing or otherwise limiting further access to an account, and alerting other institutions, third party clearing houses, etc. so that they may inspect more carefully transactions associated with the same data, e.g., the same user as identified by name, SSN, and/or other identity information that may have been stolen.
  • 210 may include directing/redirecting requests to use the resource to other resources, e.g., other servers, able to process the request; and/or sending to a source of the request an indication that the resource is not currently available.
  • FIG. 2B illustrates a process used in some embodiments to implement a metric-based approach to monitoring and/or controlling access to a resource.
  • An event is received (240). Examples of events include a data or other message; a debit or other transaction and/or charge authorization request; data requiring processing by a server or other resource; etc.
  • a metric is updated based on data associated with the event received at 240 and/or the most recent prior event associated with the metric (242). In some embodiments, data associated with events previous to the most recent prior event is not used at 242 to update the metric and as such data for such prior events is not required to be retained for purposes of monitoring and/or controlling access to the resource. If the updated metric exceeds a prescribed threshold (246), responsive action is taken (248).
  • Figure 3 illustrates a process used in some embodiments to block messages for a period in response to a threshold leaving for a metric being exceeded.
  • the process of Figure 3 comprises at least part of the responsive action taken in step 210 of Figure 2A.
  • messages from one or more affected sources, and/or messages of a particular type, size, class, etc. regardless of source, are blocked (302).
  • a firewall such as firewall 110 of Figure IB or firewall 126 of Figure 1C is configured to block the messages by not forwarding them to the server(s) to which they are addressed.
  • a firewall is configured to reject connections from an offending source. The length of the blocking period is calculated (304).
  • messages are blocked for the period required for the value of the applicable metric to decay to a level such that a prescribed number of messages (e.g., one message) could be received without the metric being incremented to a level greater than the threshold.
  • the length of the period is determined by calculating the time differential "dt" required for the value of the metric to decay to the prescribed level. Assuming each message has a weight of "1", the period "dt" can be determined by solving equation (2):
  • a minimum blocking period of 15 seconds is implemented so that the overhead of blocking and unblocking does not become significant. While in the example shown above the blocking period is determined based on exponential decay, a similar approach may be used to determine a blocking period in implementations in which linear, combined linear and exponential, and/or other types of decay are used. In various embodiments, the blocking period is determined such that the metric is decayed to a level sufficiently below the threshold that a resumption of normal, expected, and/or permitted use would not result in the metric exceeding the threshold. [0035] Figure 4 shows a plot 400 of the value of the metric m over time in an embodiment in which different weights may be assigned to different messages or message types and in which exponential decay is used.
  • Equation (1) was applied to determine the decayed value of the metric m at time t 2 and the updated value m 2 that resulted once the weighted value W 2 was added.
  • a third message arrived at time t 3 and resulted in the updated value of the metric, m 3 , exceeding the threshold M.
  • messages associated with the metric the value of which over time is shown in Figure 4 such as those from an offending IP address associated with the metric, were blocked until time t u , determined by applying a generalized form of equation (3):
  • t u is the "unblock" time and w u is the prescribed weight of the hypothetical message(s) the system should be able to receive once messages are no longer being blocked without the value of the metric exceeding the threshold.
  • a separate metric m is maintained for each source of messages, e.g., one metric per source IP address in an embodiment in which the Internet protocol is used to send and receive messages.
  • Using such an approach would, e.g., enable an offending IP address, such as one from which a denial of service attack is being launched, to be blocked while continuing to allow requests from other sources.
  • a separate metric is maintained for each type of condition, e.g., each type of attack, the metric is being maintained to detect and/or avoid.
  • a separate metric is maintained for all messages, unauthorized messages, messages that generate an internal error (e.g., on a firewall such as firewall 126 of Figure 1C), messages that generate a backend error (e.g., on one of the backend servers Sl to Sm of Figure 1C); messages that require or are expected to require CPU usage above a prescribed threshold; and messages that result or are expected to result in backend latency (i.e., the time it takes to get a response back from a backend server such as servers Sl to Sm of Figure 1C) above a prescribed threshold.
  • backend latency i.e., the time it takes to get a response back from a backend server such as servers Sl to Sm of Figure 1C
  • different messages may be assigned different weights.
  • the weight assigned to each message may be based at least in part on the actual or expected CPU usage. For example, if the threshold were 250 milliseconds (ms), messages that consume 250 ms of CPU usage might be assigned a weight of 1, those requiring 500 ms a weight of 2, those requiring 2.5 sec a weight of 10, etc.
  • the metric-based approach described herein is applied to regulate the demand placed on one or more backend systems, as opposed to or in addition to ensuring that no source and/or type or class of message(s) exceeds the capacity available and/or allotted to it.
  • a user interface is provided to enable a system administrator or other user to specify, with respect to each backend system to be protected, a maximum simultaneous "burst" of messages the system should receive (e.g., 10 messages); a maximum sustained message rate for the system (e.g., 20 messages per second); an expected/typical/threshold latency period associated with the system (e.g., 50 ms); and an expected/typical/threshold message size associated with the system (e.g., 2 kilobytes).
  • the expected/threshold latency and expected message size are used to determine the weight to be assigned to each message. For example, messages of which the latency and size are less than or equal to the corresponding threshold may be assigned a weight of 1, those that exceed either the threshold latency or the threshold size a weight of 2, and those that exceed both thresholds a weight of 3.
  • the values of the coefficient "b" in equation (1) and the maximum/threshold metric value "M" may be predetermined or preconfigured by a provider of a system configured to apply the metric-based approach described herein.
  • a user interface may be provided to enable a user to specify a value either directly for the coefficient "b" and the threshold "M", or indirectly by specifying other parameters.
  • a user interface is provided to enable a user to specify a maximum sustained message rate (e.g., a maximum number of messages of weight 1 that can be received in a given period or unit time) and a maximum number of messages (e.g., of weight 1, or of cumulative weight W) that may be received instantaneously (or nearly so), i.e., in a single "burst".
  • the maximum burst maps directly to the threshold M for the metric.
  • the metric-based approach described herein requires that only data associated with a current and most recent prior event be stored and/or processed to monitor and/or control access to a limited resource.
  • Using the metric-based approach described herein as applied to a resource access via a network it is not necessary to track extensive information about the messages that have been received within a sliding window to enable denial of service attacks and/or other conditions to be detected and/or prevented. Instead, in a basic implementation it is only necessary to track the current value of the metric and the time of receipt of the last most recently received message associated with the metric.
  • further features such as assigning different weights to different messages based on relevant criteria, may be implemented.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Security & Cryptography (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Computer Hardware Design (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)
  • Error Detection And Correction (AREA)

Abstract

Detecting unauthorized or excessive use of a resource is disclosed. The value of a metric (242) is updated based at least in part on a first data associated with a current event associated with the metric and a second data associated with a most recent prior event associated with the metric. Responsive action is taken if the updated value of the metric (242) exceeds a threshold (246).

Description

METRIC-BASED MONITORING AND CONTROL OF A LIMITED RESOURCE
CROSS REFERENCE TO OTHER APPLICATIONS
[0001] This application claims priority to U.S. Provisional Patent Application No. 60/609,730 entitled DETECTION OF DENIAL OF SERVICE ATTACKS filed 09/13/2004 which is incorporated herein by reference for all purposes.
FIELD OF THE INVENTION
[0002] The present invention relates generally to monitoring and controlling access to a limited resource. More specifically, a metric-based approach to monitoring and controlling access to a limited resource is disclosed.
BACKGROUND OF THE INVENTION
[0003] It is useful to be able to monitor and/or control access to a limited resource, especially in contexts in which placing more demands on the resource than the resource can meet may result in the resource becoming unavailable to operate even at the limited level it can support. One example of a limited resource is a network resource, such as a server or other system accessed via a network. Network- connected systems have a limited ability to process and exchange (e.g., send, receive) data. Errors or other failures can occur when the processing and/or communication capacity of such a system is exceeded, either as the result of high legitimate demand or malicious attack (e.g., so-called "denial of service" attacks). It is desirable to have an efficient way to detect when such an error or failure condition may occur and, if desired, to limit access or usage to a level that will enable such errors and/or failures to be avoided.
[0004] Even in contexts in which there is no particular risk that the processing and/or communication capacity of a system will be exceeded, it may be desirable to limit access by a particular user, process, or system, e.g. to implement a quality of service or other guarantee made with respect to that or some other user, process, or system. In other contexts, it would be useful to have an efficient way to detect use patterns that deviate from historic use and/or use patterns otherwise determined to be associated with normal or authorized use, such as detecting an unusual pattern of use of a credit card or other financial account, which may indicate the credit card and/or account information has been stolen.
[0005] One prior art approach to detecting and/or preventing a condition that might result in an error or failure, or that might result in a quality of service guarantee or some other relevant threshold or level of use being exceeded, e.g., has been to define a "sliding window" and track the demand made on the system (e.g., the number or cumulative size of the messages or other communications received) during a period defined by the window. For example, under one typical approach one might track how many messages were received in the last X seconds. In some cases, an alert may be sent or other responsive action taken if more than N messages were received in a period of X seconds. Or, in the case of a credit card or other account, a similar approach might be used to track how many transactions were charged to the account in a given hour, week, day, etc., with an alert being generated if the use exceeds a prescribed threshold. However, such an approach consumes a lot of processing and memory resources, as it is necessary to keep track of lots of data, such as which messages have been received (or transactions completed in the credit card example) and at what time, and continually (or at least periodically) perform computations on such information to determine the number of messages received within the sliding analysis window.
[0006] Therefore, there is a need for an efficient way to detect when demand, either collectively or from a particular source, exceeds the bandwidth or capacity available or allotted to satisfy the demand, such as from a network-connected system, and to detect patterns that may indicate unauthorized use. BRIEF DESCRIPTION OF THE DRAWINGS
[0007] Various embodiments of the invention are disclosed in the following detailed description and the accompanying drawings.
[0008] Figure IA illustrates a client-server environment.
[0009] Figure IB illustrates a client-server environment that includes a firewall.
[0010] Figure 1C illustrates yet another illustrative client-server environment.
[0011] Figure 2A illustrates a process used in one embodiment to implement a metric-based approach to detecting and/or preventing demand for a resource, such as a server, from exceeding a prescribed or otherwise applicable limit.
[0012] Figure 2B illustrates a process used in some embodiments to implement a metric-based approach to monitoring and/or controlling access to a resource.
[0013] Figure 3 illustrates a process used in some embodiments to block messages for a period in response to a threshold leaving for a metric being exceeded.
[0014] Figure 4 shows a plot 400 of the value of the metric m over time in an embodiment in which different weights may be assigned to different messages or message types.
DETAILED DESCRIPTION
[0015] The invention can be implemented in numerous ways, including as a process, an apparatus, a system, a composition of matter, a computer readable medium such as a computer readable storage medium or a computer network wherein program instructions are sent over optical or electronic communication links. In this specification, these implementations, or any other form that the invention may take, may be referred to as techniques. In general, the order of the steps of disclosed processes may be altered within the scope of the invention. [0016] A detailed description of one or more embodiments of the invention is provided below along with accompanying figures that illustrate the principles of the invention. The invention is described in connection with such embodiments, but the invention is not limited to any embodiment. The scope of the invention is limited only by the claims and the invention encompasses numerous alternatives, modifications and equivalents. Numerous specific details are set forth in the following description in order to provide a thorough understanding of the invention. These details are provided for the purpose of example and the invention may be practiced according to the claims without some or all of these specific details. For the purpose of clarity, technical material that is known in the technical fields related to the invention has not been described in detail so that the invention is not unnecessarily obscured.
[0017] A metric-based approach to detecting when conditions are such that demand, such as for a network-accessible resource, may exceed the bandwidth available and/or allotted to satisfy the demand, or when actual use of a resource deviates from an expected, normal, and/or permitted level of use, is disclosed. A metric is incremented each time use event occurs, e.g., each time a message is received by a network based resource such as a firewall, gateway, server, etc., and decayed over time. A responsive action is taken if the metric exceeds a threshold. In some embodiments, if the metric exceeds the threshold value subsequent attempts to use the resource, e.g., network communications from a particular offending source, may be blocked for a period.
[0018] Figure IA illustrates a client-server environment. A client 102 has a connection 104 to a server 106. The client 102 may be a computer or other system configured to communicate with server 106 via connection 104, e.g., to request data from server 106. Server 106 may be a computer or other system or process configured to receive data from and/or provide data to a client such as client 102 via connection 104. While a single client 102 and server 106 are shown, a typical commercial application may involve many clients and/or more than one server. The connection 104 may be a direct connection, but more typically it is a network connection over a private network (e.g., a LAN or WAN) and/or a public network, such as the Internet. As described more fully below, in one embodiment the server 106 is configured to regulate messages, either globally or based on source, class, size, destination, user, account, data included in the message, or other criteria, using the metric-based approach described herein.
[0019] Figure IB illustrates a client-server environment that includes a firewall. The client 102 is connected via a network 108 to a firewall 110. Firewall 110 is configured to receive messages sent by client 102 to server 106 via network 108 and forward such messages to server 106 if forwarding criteria are satisfied. In one embodiment, the firewall 110 is configured to apply the metric-based approach described herein to prevent the client 102, or clients generally, from exceeding the bandwidth on server 106 that is available and/or has been allotted to it/them. If a message would exceed an associated limit, e.g., a threshold value for the metric would be exceeded, or in some embodiments if a prior message caused the threshold to be exceeded resulting in a blocking period being imposed, the firewall 110 blocks subsequent messages associated with the metric that has exceeded its associated threshold, e.g., messages from the affected source(s) and/or of the affected type.
[0020] Figure 1C illustrates yet another illustrative client-server environment. In this example, a plurality of clients Cl to Cn, represented in Figure 1C by clients 120 and 122, are connected via network 124 to firewall 126. Clients Cl to Cn send , messages to one or more of a plurality of backend servers Sl to Sm associated with firewall 126, represented in Figure 1C by servers 128, 130, and 132. As in the example shown in Figure IB, the metric-based approach described herein is implemented in one embodiment on firewall 126, e.g., to detect and/or prevent the demand (e.g., messages) made on one or more of the servers Sl to Sm from exceeding an applicable limit, such as to detect and/or avoid a condition that may be associated with and/or similar in its consequences to a denial of service type attack. In some embodiments, each of servers Sl to Sm has associated with it a corresponding metric and threshold, not necessarily the same as the corresponding metric and threshold for one or more other of the servers Sl to Sm, and firewall 126 uses the metric for each server to throttle, i.e., control the rate of, data transmission to that server so as to not exceed a communication and/or processing bandwidth and/or other capacity constraint of the server. For example, an enterprise may have two or more servers configured to perform a particular service, e.g., to host a particular application, but due to differences in hardware, configuration, etc. one server may have a higher capacity than the other(s). In some embodiments, the firewall tracks incoming connections/transmissions on a per client basis on the client side of firewall 126 (connected to network 124) and uses a per-server metric (for all clients, classes, etc.) on the server side (connected to servers 128-132) to throttle traffic to the servers. So long as the metric associated with a client has not been exceeded, it is able to connect to those of servers 128-132 that have not had traffic to them blocked as a result of their respective metrics exceeding their associated thresholds. In some embodiments, if the metric for a particular one or more of servers 128-132 has been exceeded, the firewall 126 blocks traffic to the affected server(s) for a time, and directs/redirects connections that otherwise would have been made to the affected server(s) to one or more other, unaffected servers for which the associated metric has not exceeded its applicable threshold. In some embodiments, the firewall uses a round robin approach to direct connection requests to servers Sl to Sm and skips any server for which the new connection would result in the server's metric exceeding its threshold.
[0021] The examples shown in Figure 1A-1C are for purposes of illustration only and do not limit the generality of the approach described herein, nor the scope of the appended claims. In particular, while Figures 1A-1C show client-server environments, the approach described herein may be applied as well to other environments. In one embodiment, the firewalls shown in Figures IB and 1C may comprise an XML firewall configured to validate and/or otherwise process requests or messages comprising XML documents, such as SOAP or other messages being sent to a backend server for processing in a web services context.
[0022] In some embodiments, the firewall may monitor use of a financial or other account, or other data usable to detect a use pattern that deviates from an expected, normal, and/or permitted level of use, such as may indicate unauthorized use such as may occur in the case of theft (including identity theft) or fraud. In some embodiments, data transmissions associated with account use, such as charge authorizations, debit transactions, etc., are monitored by maintaining a metric on a per user and/or per account basis. For each authorization/transaction, user and/or account identifier data, as applicable, is determined and an associated metric is updated. If the metric exceeds an associated threshold, e.g., as a result of an unusual number of transactions in a short period and/or transactions involving unusually large amounts, a responsive action (block further use, alert, etc.) is taken.
[0023] hi various embodiments, the value of the metric is incremented on occurrence of each event by a weighted amount determined based on data associated with the event, e.g., by the size of a received data message in bytes, the dollar amount of a transaction, etc. Single events that consume a large amount of a limited resource are weighted more heavily in this approach than events that consume relatively fewer resources.
[0024] Figure 2A illustrates a process used in one embodiment to implement a metric-based approach to detecting and/or preventing demand for a resource, such as a server, from exceeding a prescribed or otherwise applicable limit. A message is received (202). In some embodiments, an event other than receiving a message, such as receiving an indication that a credit card has been used, occurs at 202. The value of a metric used to monitor and/or limit messages, e.g., to detect or prevent a denial of service attack or similar condition, is decayed (i.e., decremented) by an amount associated with the amount of time that elapsed between the message currently being processed and the next most recently received message (204). In various embodiments, the metric is decayed by an amount determined at least in part on data other than the amount of time that elapsed between the message currently being processed and the next most recently received message, such as the time of day, the size of the message, the nature/type of message, use conditions generally (i.e., across users), etc. In one embodiment, a separate metric is maintained for separate classes or types of message and in such embodiments each respective metric is decayed in 204 by an amount associated with the amount of time that has elapsed between the message currently being processed and the next most recently received message of the class/type with which the particular metric being decayed is associated. By way of example, in one embodiment an instance of the metric is used to limit the number of unauthorized messages to three messages per minute. When an unauthorized message is received the time considered in calculating the amount of decay is the time that has elapsed since receipt of the unauthorized message that is currently being processed and the last unauthorized message, although one or more (even many) authorized messages may have been received in the time in between. In one embodiment, the value of the metric is decayed in 204 by multiplying the value of the metric by a base "b" raised to an exponent equal to the time "dt" that elapsed between the current and next most recently received messages associated with the metric, such that the decay is exponential over time. In some embodiments, the decay is linear, e.g., the metric is decayed by subtracting from the current value of the metric the product of a base "b" and the elapsed time "dt". In some embodiments, a hybrid approach is used, e.g., by decaying the value of the metric exponentially above the threshold and linearly below. The latter approach is used in some embodiments to avoid false detections in environments, such as test environments, in which periodic use patterns may occur. In some embodiments, exponential decay, at least for the region above the threshold, is used to enable a block (or alert, etc.) state to be cleared more quickly after a detection event caused by a single large (i.e., heavily weighted) event and/or a burst of smaller events causes the metric value to spike.
[0025] In 206, the value of the metric is incremented by an amount associated with the message currently being processed. In some embodiments, the value of each message is the same and in such embodiments the same amount (e.g., 1) is added to the metric in step 206 regardless of the nature or source of the message being processed. In other embodiments, a weighted value associated with some attribute of the message is added to the metric in step 206.
[0026] Equation (1) summarizes the result of performing steps 204 and 206 in the general case in which a weighted value "Wk" associated with the "k-th" received message, in an embodiment in which exponential decay is used.
(1) mk = mk_lbc" + wk
where "mk" is the value of the metric after receipt of the k-th message (i.e., the message currently being processed), "nik-i" is the value of the metric at the time of receipt of the message received just prior to the k-th message, "b" is a based between 0 and 1, determined in some embodiments as described below, "dt" represents the time that elapsed between receipt of the message k and k-1 (i.e., dt = tk - tk-i), and 'Vk" is the weight value of the k-th message.
[0027] While exponential decay is shown in Equation (1), as noted other approaches, such as linear decay, a combination of linear and exponential decay, or any other approach suitable to the circumstances and/or requirements of a particular embodiment may be used. While in Equation (1) the amount the metric is decayed is determined at least in part by the time elapsed between messages of the type (e.g., from the source, to the server, etc.) with which the metric is associated, in various embodiments other and/or different data is used to determine the amount the value of the metric is decayed, e.g., the time of day of the last previous message, the time of receipt of the current message, the day of the week (e.g., whether it is a weekday, weekend, holiday, etc.), a weight value associated with the last previous message, a weight value associated with the current message, the threshold value for the metric, the proximity of the metric to the threshold, the value of the metric relative to the threshold (e.g., greater or lesser), one or more constants (e.g., in addition to and/or instead of the base "b"), and message details such as the source, destination, size, and/or data included and/or associated with the message (e.g., a user identity, account number, etc.). The precise formula used to decay the value of the metric in any given implementation will depend on such factors as the nature and capacity of the resource being monitored and/or controlled, the requirements of and/or commitments made to respective users of the resource, and the characteristics of normal and anticipated permitted use of the resource.
[0028] Referring further to Figure 2 A, it is determined whether the value of the metric is greater than an associated threshold (208). hi some embodiments, a threshold or maximum permissible value "M" of the metric is established and it is determined in step 208 that the threshold has been exceeded if the value of the metric "m" is greater than the threshold "M" (i.e., m > M). In some embodiments, the value of the threshold "M" for a given instance of a metric "m" may vary, depending on such factors as the time of day, day of the week, season, external conditions (e.g., environmental or weather conditions), status and availability of related resources (e.g., other servers able to process the message). For example, a credit card transaction (or a closely spaced burst of transactions) may be afforded greater weight at 4 a.m. than at 4 p.m., for a metric used to detect unauthorized use, e.g., on the theory that use at an unexpected level at a time when most users are asleep and most businesses closed is more suspicious than an increased use of similar proportions that occurs during the business day.
[0029] In some embodiments in which the value of the metric is decayed at least in part exponentially, the equivalent of exponential decay is implemented by incrementing the value of the metric logarithmically at 208 and then applying linear decay at 206. In some embodiments in which a combination of exponential (e.g., for values of "m" greater than "M") and linear (e.g., for values of "m" below "M") is desired, partially exponential decay is implemented by incrementing the value of m at 208 in full increments up to the level m=M and logarithmically above the threshold "M". For example, if the current value of m=8, the threshold M=IO, and an event of weight w=4 occurred, in some embodiments the value of the metric m would be incremented to m = 10 + hi 2.
[0030] If no applicable threshold has been exceeded (208), the process returns to step 202 and the next message (or other event) is received and processed. If an applicable threshold has been exceeded, responsive action is taken (210). In some embodiments, the responsive action comprises generating an alert. In some embodiments, the responsive action comprises blocking messages from one or more sources associated with the metric and/or messages of one or more classes or types of message associated with the metric, if the threshold is exceeded. In some such embodiments, the blocking continues until the value of the metric has decayed to a level such that at least one message of a prescribed weight (or at least one message of an expected or typical type or size, in an embodiment in which weights are not assigned) could be received without the value of the metric exceeding the threshold. Further and/or different responsive actions may be taken in other embodiments. For example, in the case of a metric used to detect unauthorized use of personal data and/or property, such as a bank or credit card account, the responsive action may include notifying an owner and/or provider of the account (such as a bank), notifying public or private enforcement authorities, freezing or otherwise limiting further access to an account, and alerting other institutions, third party clearing houses, etc. so that they may inspect more carefully transactions associated with the same data, e.g., the same user as identified by name, SSN, and/or other identity information that may have been stolen. In an embodiment in which a metric is being used to throttle traffic to a backend server and/or other resource, 210 may include directing/redirecting requests to use the resource to other resources, e.g., other servers, able to process the request; and/or sending to a source of the request an indication that the resource is not currently available.
[0031] Figure 2B illustrates a process used in some embodiments to implement a metric-based approach to monitoring and/or controlling access to a resource. An event is received (240). Examples of events include a data or other message; a debit or other transaction and/or charge authorization request; data requiring processing by a server or other resource; etc. A metric is updated based on data associated with the event received at 240 and/or the most recent prior event associated with the metric (242). In some embodiments, data associated with events previous to the most recent prior event is not used at 242 to update the metric and as such data for such prior events is not required to be retained for purposes of monitoring and/or controlling access to the resource. If the updated metric exceeds a prescribed threshold (246), responsive action is taken (248). If the updated metric does not exceed an applicable threshold (246), data associated with events other than the event just processed is deleted (250) and the process of Figure 2B is repeated upon receipt of a next applicable event (240). Deleting old data and updating the metric based only on data associated with the current event and/or the most recent prior event limits the memory required to store data to be used to monitor/control access and also the processing resources required to determine upon occurrence of an event whether use has exceeded an applicable limit.
[0032] Figure 3 illustrates a process used in some embodiments to block messages for a period in response to a threshold leaving for a metric being exceeded. In some embodiments, the process of Figure 3 comprises at least part of the responsive action taken in step 210 of Figure 2A. Depending on the configuration, messages from one or more affected sources, and/or messages of a particular type, size, class, etc. regardless of source, are blocked (302). In some embodiments, a firewall such as firewall 110 of Figure IB or firewall 126 of Figure 1C is configured to block the messages by not forwarding them to the server(s) to which they are addressed. In some embodiments, a firewall is configured to reject connections from an offending source. The length of the blocking period is calculated (304). In some embodiments, messages are blocked for the period required for the value of the applicable metric to decay to a level such that a prescribed number of messages (e.g., one message) could be received without the metric being incremented to a level greater than the threshold. The length of the period is determined by calculating the time differential "dt" required for the value of the metric to decay to the prescribed level. Assuming each message has a weight of "1", the period "dt" can be determined by solving equation (2):
(2) mkbdt = M -l
[0033] where M is the threshold and nik is the value of the metric upon receipt of the k-th message, i.e. the one that caused the value to exceed the threshold. Solving equation (2) yields the time period "dt" required before a message k+1 of weight "1" can be received without the threshold M being exceeded, i.e., mk+i <= M. Solving equation (2) for "dt" yields:
In(M - I) - In mk
(3) dt =
[0034] In some embodiments, a minimum blocking period of 15 seconds is implemented so that the overhead of blocking and unblocking does not become significant. While in the example shown above the blocking period is determined based on exponential decay, a similar approach may be used to determine a blocking period in implementations in which linear, combined linear and exponential, and/or other types of decay are used. In various embodiments, the blocking period is determined such that the metric is decayed to a level sufficiently below the threshold that a resumption of normal, expected, and/or permitted use would not result in the metric exceeding the threshold. [0035] Figure 4 shows a plot 400 of the value of the metric m over time in an embodiment in which different weights may be assigned to different messages or message types and in which exponential decay is used. In the example shown, a first message having weight W1 and received at time t\ resulted in the metric m having a value Hi1. At time t2, a second message having weight W2 arrived. Equation (1) was applied to determine the decayed value of the metric m at time t2 and the updated value m2 that resulted once the weighted value W2 was added. A third message arrived at time t3 and resulted in the updated value of the metric, m3, exceeding the threshold M. In the example shown, messages associated with the metric the value of which over time is shown in Figure 4, such as those from an offending IP address associated with the metric, were blocked until time tu, determined by applying a generalized form of equation (3):
(4) dt = tu - t3 = HM - W") - lΑm
\nb
[0036] where tu is the "unblock" time and wu is the prescribed weight of the hypothetical message(s) the system should be able to receive once messages are no longer being blocked without the value of the metric exceeding the threshold.
[0037] In some embodiments, a separate metric m is maintained for each source of messages, e.g., one metric per source IP address in an embodiment in which the Internet protocol is used to send and receive messages. Using such an approach would, e.g., enable an offending IP address, such as one from which a denial of service attack is being launched, to be blocked while continuing to allow requests from other sources.
[0038] In some embodiments, a separate metric is maintained for each type of condition, e.g., each type of attack, the metric is being maintained to detect and/or avoid. For example, in one embodiment a separate metric is maintained for all messages, unauthorized messages, messages that generate an internal error (e.g., on a firewall such as firewall 126 of Figure 1C), messages that generate a backend error (e.g., on one of the backend servers Sl to Sm of Figure 1C); messages that require or are expected to require CPU usage above a prescribed threshold; and messages that result or are expected to result in backend latency (i.e., the time it takes to get a response back from a backend server such as servers Sl to Sm of Figure 1C) above a prescribed threshold. For example, one may wish to configure a system to allow an overall message rate of 120 per minute without triggering any responsive action but to take responsive action, such as by blocking an offending source or generating an alert, if 3 unauthorized messages were received from the same source within a minute.
[0039] hi some embodiments, as described above, different messages may be assigned different weights. For example, in an embodiment in which a separate metric is maintained for messages that exceed a prescribed threshold for CPU usage, the weight assigned to each message may be based at least in part on the actual or expected CPU usage. For example, if the threshold were 250 milliseconds (ms), messages that consume 250 ms of CPU usage might be assigned a weight of 1, those requiring 500 ms a weight of 2, those requiring 2.5 sec a weight of 10, etc.
[0040] hi some embodiments, the metric-based approach described herein is applied to regulate the demand placed on one or more backend systems, as opposed to or in addition to ensuring that no source and/or type or class of message(s) exceeds the capacity available and/or allotted to it. Such an approach may be useful, e.g., to ensure that a legacy backend system used in connection with more modern protocols and/or systems will not be overloaded by the additional processing required by, for example, the larger messages and more extensive processing that may be required to operate with systems and/or protocols that are more advanced than those the legacy system was designed to handle, hi one embodiment, a user interface is provided to enable a system administrator or other user to specify, with respect to each backend system to be protected, a maximum simultaneous "burst" of messages the system should receive (e.g., 10 messages); a maximum sustained message rate for the system (e.g., 20 messages per second); an expected/typical/threshold latency period associated with the system (e.g., 50 ms); and an expected/typical/threshold message size associated with the system (e.g., 2 kilobytes). In one embodiment, the expected/threshold latency and expected message size are used to determine the weight to be assigned to each message. For example, messages of which the latency and size are less than or equal to the corresponding threshold may be assigned a weight of 1, those that exceed either the threshold latency or the threshold size a weight of 2, and those that exceed both thresholds a weight of 3.
[0041] The values of the coefficient "b" in equation (1) and the maximum/threshold metric value "M" may be predetermined or preconfigured by a provider of a system configured to apply the metric-based approach described herein. Alternatively, a user interface may be provided to enable a user to specify a value either directly for the coefficient "b" and the threshold "M", or indirectly by specifying other parameters. In one embodiment, a user interface is provided to enable a user to specify a maximum sustained message rate (e.g., a maximum number of messages of weight 1 that can be received in a given period or unit time) and a maximum number of messages (e.g., of weight 1, or of cumulative weight W) that may be received instantaneously (or nearly so), i.e., in a single "burst". The maximum burst maps directly to the threshold M for the metric. This relationship is evident upon examination of the plot shown in Figure 4 and considering the case in which the value of the metric m has been initialized or has decayed to a value of 0, such that messages resulting in a value of m = M can be received at any time t at which the value of m has decayed to (or been initialized at) a value of 0. The value of the coefficient "b" can be determined by considering the case in which the value of the metric m has been increased up to the threshold M, and determining the value of the coefficient b that would be required to ensure that the threshold M would not be exceeded if message were to be received at the user-indicated maximum rate "r" under such conditions. Applying equation (1), and assuming for simplicity that all messages have a weight of 1 (such that w^ = 1 ; an average or typical or expected or threshold weight other than 1 could be used instead in an embodiment in which different messages may have different weights) and assuming that each successive message received at the maximum rate r restores the value of the metric m to the maximum permitted value M, yields:
(5) M = Mbdt + l
[0042] The time interval dt between messages is the inverse of the rate r when messages are received at the maximum rate. Substituting 1/r for dt in equation (5) yields: (6) M = MbUr +l
which, solving for b, yields:
(7) b = \ l--L
M
[0043] The metric-based approach described herein requires that only data associated with a current and most recent prior event be stored and/or processed to monitor and/or control access to a limited resource. Using the metric-based approach described herein as applied to a resource access via a network, it is not necessary to track extensive information about the messages that have been received within a sliding window to enable denial of service attacks and/or other conditions to be detected and/or prevented. Instead, in a basic implementation it is only necessary to track the current value of the metric and the time of receipt of the last most recently received message associated with the metric. By tracking and/or determining other basic data, further features such as assigning different weights to different messages based on relevant criteria, may be implemented.
[0044] Although the foregoing embodiments have been described in some detail for purposes of clarity of understanding, the invention is not limited to the details provided. There are many alternative ways of implementing the invention. The disclosed embodiments are illustrative and not restrictive.
[0045] WHAT IS CLAIMED IS :

Claims

1. A method for detecting unauthorized or excessive use of a resource, comprising: updating the value of a metric based at least in part on a first data associated with a current event associated with the metric and a second data associated with a most recent prior event associated with the metric; and taking responsive action if the updated value of the metric exceeds a threshold.
2. A method as recited in claim 1 , wherein the current event comprises a message sent to a network-accessible resource
3. A method as recited in claim 2, wherein updating the value of the metric includes: for each message received: decrementing the value of a metric by a first amount associated with the amount of time that elapsed between receipt of the message and receipt of a last message received immediately prior to the message; and incrementing the decremented metric value by a second amount associated with the message.
4. A method as recited in claim 1 , wherein the first data includes a first time associated with the current event and the second data includes a second time associated with the most recent prior event and updating the value of the metric includes decaying the metric by an amount based at least in part on a difference in time between the first time and the second time.
5. A method as recited in claim 4, wherein the decay is at least partly exponential.
6. A method as recited in claim 4, wherein the decay is at least partly linear.
7. A method as recited in claim 4, wherein the decay is exponential for values of the metric above the threshold and linear for values of the metric below the threshold.
8. A method as recited in claim 1, wherein the responsive action includes sending an alert.
9. A method as recited in claim 1, wherein the responsive action includes blocking the current event.
10. A method as recited in claim 1 , wherein the responsive action includes blocking a subsequent event.
11. A method as recited in claim 1 , wherein the responsive action includes blocking subsequent events for a period of time.
12. A method as recited in claim 11 , wherein the length of the period of time is selected such that the value of the metric is decayed to a level sufficiently below the threshold to ensure that resumption of a normal, expected, or permitted event activity will not cause the metric to exceed the threshold.
13. A method as recited in claim 1 , wherein taking responsive action includes directing or redirecting a request to a resource not associated with the metric the value of which has exceeded, or would exceed if the event were permitted, the threshold.
14. A method as recited in claim 1, wherein the metric and the threshold are selected to detect use of the resource at a level that deviates from a normal, expected, or permitted use of the resource.
15. A method as recited in claim 1 , wherein the metric and the threshold are selected to detect use of the resource at a level that exceeds a capacity of the resource.
16. A method as recited in claim 1, wherein the metric comprises a first metric associated with a first resource, the threshold comprises a first threshold associated with the first resource, and the first resource comprises one of a plurality of resources, each having associated with it a resource-specific metric and a corresponding resource-specific threshold.
17. A method as recited in claim 1 , wherein the current event and the most recent prior event share an attribute and the metric and threshold are specific to events having the attribute.
18. A method as recited in claim 17, wherein the attribute includes one or more of the following: a source of the event; a destination or target of the event; and user, account, or other identity-related data comprising or associated with the event.
19. A method as recited in claim 1 , further comprising deleting the second data and storing the first data based at least in part on a determination that the updated value of the message does not exceed the threshold.
20. A method as recited in claim 1 , wherein updating the value of the metric includes adding to the metric a weighted amount determined at least in part by an attribute of the current event.
21. A system for determining that an event has caused, or if permitted would cause, a limit established to detect an undesirable condition to be exceeded, comprising: a processor configured to: update the value of a metric based at least in part on a first data associated with a current event associated with the metric and a second data associated with a most recent prior event associated with the metric; and take responsive action if the updated value of the metric exceeds a threshold; and a memory configured to store the second data and provide instructions to the processor.
22. A computer program product for determining that an event has caused, or if permitted would cause, a limit established to detect an undesirable condition to be exceeded, the computer program product being embodied on a computer readable medium and comprising computer instructions for: updating the value of a metric based at least in part on a first data associated with a current event associated with the metric and a second data associated with a most recent prior event associated with the metric; and taking responsive action if the updated value of the metric exceeds a threshold.
PCT/US2005/033163 2004-09-13 2005-09-13 Metric-based monitoring and control of a limited resource WO2006032028A2 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US60973004P 2004-09-13 2004-09-13
US60/609,730 2004-09-13

Publications (2)

Publication Number Publication Date
WO2006032028A2 true WO2006032028A2 (en) 2006-03-23
WO2006032028A3 WO2006032028A3 (en) 2006-07-13

Family

ID=36060728

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2005/033163 WO2006032028A2 (en) 2004-09-13 2005-09-13 Metric-based monitoring and control of a limited resource

Country Status (2)

Country Link
US (1) US8255532B2 (en)
WO (1) WO2006032028A2 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TWI386917B (en) * 2010-06-29 2013-02-21 Tung Fang Inst Of Technology Find the same language of the same language and the method of grouping
TWI386918B (en) * 2010-06-29 2013-02-21 Tung Fang Inst Of Technology Sound recognition method

Families Citing this family (47)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7324922B2 (en) * 2005-10-26 2008-01-29 International Business Machines Corporation Run-time performance verification system
US7596670B2 (en) * 2005-11-30 2009-09-29 International Business Machines Corporation Restricting access to improve data availability
US7616624B2 (en) * 2006-07-20 2009-11-10 Avaya Inc. Determining user availability based on the expected duration of a new event
US7680480B2 (en) * 2006-07-20 2010-03-16 Avaya Inc. Determining user availability based on a past event
US8453165B2 (en) * 2008-01-22 2013-05-28 International Business Machines Corporation Distributing event processing in event relationship networks
US8255994B2 (en) * 2008-08-20 2012-08-28 Sprint Communications Company L.P. Detection and suppression of short message service denial of service attacks
US20110161987A1 (en) * 2009-12-30 2011-06-30 Anqi Andrew Huang Scaling notifications of events in a social networking system
WO2012054646A2 (en) * 2010-10-19 2012-04-26 The 41St Parameter, Inc. Variable risk engine
US9148376B2 (en) * 2010-12-08 2015-09-29 AT&T Intellectual Property I, L.L.P. Method and system for dynamic traffic prioritization
US9843488B2 (en) * 2011-11-07 2017-12-12 Netflow Logic Corporation Method and system for confident anomaly detection in computer network traffic
US8447851B1 (en) * 2011-11-10 2013-05-21 CopperEgg Corporation System for monitoring elastic cloud-based computing systems as a service
US20130159497A1 (en) * 2011-12-16 2013-06-20 Microsoft Corporation Heuristic-Based Rejection of Computing Resource Requests
US9612866B2 (en) * 2012-08-29 2017-04-04 Oracle International Corporation System and method for determining a recommendation on submitting a work request based on work request type
US9426020B2 (en) 2013-03-15 2016-08-23 Cisco Technology, Inc. Dynamically enabling selective routing capability
GB2515778A (en) 2013-07-03 2015-01-07 Ibm Measuring robustness of web services to denial of service attacks
US9705896B2 (en) * 2014-10-28 2017-07-11 Facebook, Inc. Systems and methods for dynamically selecting model thresholds for identifying illegitimate accounts
US11165812B2 (en) 2014-12-03 2021-11-02 Splunk Inc. Containment of security threats within a computing environment
US11153183B2 (en) * 2015-06-11 2021-10-19 Instana, Inc. Compacted messaging for application performance management system
WO2017039506A1 (en) * 2015-09-03 2017-03-09 Telefonaktiebolaget Lm Ericsson (Publ) Method and network node for localizing a fault causing performance degradation of service
US9762610B1 (en) 2015-10-30 2017-09-12 Palo Alto Networks, Inc. Latency-based policy activation
RU2649793C2 (en) 2016-08-03 2018-04-04 ООО "Группа АйБи" Method and system of detecting remote connection when working on web resource pages
FR3061570B1 (en) * 2016-12-29 2020-11-27 Bull Sas MECHANISM FOR MONITORING AND ALERT OF THE APPLICATIONS OF THE COMPUTER SYSTEM
RU2637477C1 (en) 2016-12-29 2017-12-04 Общество с ограниченной ответственностью "Траст" System and method for detecting phishing web pages
RU2671991C2 (en) 2016-12-29 2018-11-08 Общество с ограниченной ответственностью "Траст" System and method for collecting information for detecting phishing
RU2689816C2 (en) 2017-11-21 2019-05-29 ООО "Группа АйБи" Method for classifying sequence of user actions (embodiments)
RU2676247C1 (en) 2018-01-17 2018-12-26 Общество С Ограниченной Ответственностью "Группа Айби" Web resources clustering method and computer device
RU2680736C1 (en) 2018-01-17 2019-02-26 Общество с ограниченной ответственностью "Группа АйБи ТДС" Malware files in network traffic detection server and method
RU2677368C1 (en) 2018-01-17 2019-01-16 Общество С Ограниченной Ответственностью "Группа Айби" Method and system for automatic determination of fuzzy duplicates of video content
RU2668710C1 (en) 2018-01-17 2018-10-02 Общество с ограниченной ответственностью "Группа АйБи ТДС" Computing device and method for detecting malicious domain names in network traffic
RU2677361C1 (en) 2018-01-17 2019-01-16 Общество с ограниченной ответственностью "Траст" Method and system of decentralized identification of malware programs
RU2681699C1 (en) * 2018-02-13 2019-03-12 Общество с ограниченной ответственностью "Траст" Method and server for searching related network resources
RU2708508C1 (en) 2018-12-17 2019-12-09 Общество с ограниченной ответственностью "Траст" Method and a computing device for detecting suspicious users in messaging systems
RU2701040C1 (en) 2018-12-28 2019-09-24 Общество с ограниченной ответственностью "Траст" Method and a computer for informing on malicious web resources
WO2020176005A1 (en) 2019-02-27 2020-09-03 Общество С Ограниченной Ответственностью "Группа Айби" Method and system for identifying a user according to keystroke dynamics
US11023896B2 (en) * 2019-06-20 2021-06-01 Coupang, Corp. Systems and methods for real-time processing of data streams
RU2728497C1 (en) 2019-12-05 2020-07-29 Общество с ограниченной ответственностью "Группа АйБи ТДС" Method and system for determining belonging of software by its machine code
RU2728498C1 (en) 2019-12-05 2020-07-29 Общество с ограниченной ответственностью "Группа АйБи ТДС" Method and system for determining software belonging by its source code
RU2743974C1 (en) 2019-12-19 2021-03-01 Общество с ограниченной ответственностью "Группа АйБи ТДС" System and method for scanning security of elements of network architecture
SG10202001963TA (en) 2020-03-04 2021-10-28 Group Ib Global Private Ltd System and method for brand protection based on the search results
BR112023000159A2 (en) 2020-07-09 2023-01-31 Featurespace Ltd METHODS TO TRAIN A MACHINE LEARNING SYSTEM AND TO DETECT ANOMALIES WITHIN TRANSACTION DATA
US11475090B2 (en) 2020-07-15 2022-10-18 Group-Ib Global Private Limited Method and system for identifying clusters of affiliated web resources
RU2743619C1 (en) 2020-08-06 2021-02-20 Общество с ограниченной ответственностью "Группа АйБи ТДС" Method and system for generating the list of compromise indicators
US11947572B2 (en) 2021-03-29 2024-04-02 Group IB TDS, Ltd Method and system for clustering executable files
NL2030861B1 (en) 2021-06-01 2023-03-14 Trust Ltd System and method for external monitoring a cyberattack surface
RU2769075C1 (en) 2021-06-10 2022-03-28 Общество с ограниченной ответственностью "Группа АйБи ТДС" System and method for active detection of malicious network resources
US20230262077A1 (en) * 2021-11-15 2023-08-17 Cfd Research Corporation Cybersecurity systems and methods for protecting, detecting, and remediating critical application security attacks
GB2618317A (en) * 2022-04-28 2023-11-08 Featurespace Ltd Machine learning system

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6334124B1 (en) * 1997-10-06 2001-12-25 Ventro Corporation Techniques for improving index searches in a client-server environment
US6832239B1 (en) * 2000-07-07 2004-12-14 International Business Machines Corporation Systems for managing network resources

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5160901A (en) * 1990-09-13 1992-11-03 Frequency Electronics, Inc. Multimode crystal oscillator
US5708422A (en) * 1995-05-31 1998-01-13 At&T Transaction authorization and alert system
US6223985B1 (en) * 1998-06-10 2001-05-01 Delude Bethany J. System and method for protecting unauthorized access into an access-controlled entity by an improved fail counter
US6728955B1 (en) * 1999-11-05 2004-04-27 International Business Machines Corporation Processing events during profiling of an instrumented program
US7058708B2 (en) * 2001-06-12 2006-06-06 Hewlett-Packard Development Company, L.P. Method of and apparatus for managing predicted future user accounts assigned to a computer
WO2003005279A1 (en) * 2001-07-03 2003-01-16 Altaworks Corporation System and methods for monitoring performance metrics
US6993686B1 (en) * 2002-04-30 2006-01-31 Cisco Technology, Inc. System health monitoring and recovery
US6961562B2 (en) * 2002-06-19 2005-11-01 Openwave Systems Inc. Method and apparatus for acquiring, processing, using and brokering location information associated with mobile communication devices
JP4080911B2 (en) * 2003-02-21 2008-04-23 株式会社日立製作所 Bandwidth monitoring device
WO2005004370A2 (en) * 2003-06-28 2005-01-13 Geopacket Corporation Quality determination for packetized information
DE10337144A1 (en) * 2003-08-11 2005-03-17 Hewlett-Packard Company, Palo Alto Method for recording event logs
US7360102B2 (en) * 2004-03-29 2008-04-15 Sony Computer Entertainment Inc. Methods and apparatus for achieving thermal management using processor manipulation

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6334124B1 (en) * 1997-10-06 2001-12-25 Ventro Corporation Techniques for improving index searches in a client-server environment
US6832239B1 (en) * 2000-07-07 2004-12-14 International Business Machines Corporation Systems for managing network resources

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TWI386917B (en) * 2010-06-29 2013-02-21 Tung Fang Inst Of Technology Find the same language of the same language and the method of grouping
TWI386918B (en) * 2010-06-29 2013-02-21 Tung Fang Inst Of Technology Sound recognition method

Also Published As

Publication number Publication date
US8255532B2 (en) 2012-08-28
WO2006032028A3 (en) 2006-07-13
US20060059568A1 (en) 2006-03-16

Similar Documents

Publication Publication Date Title
US8255532B2 (en) Metric-based monitoring and control of a limited resource
US9325725B2 (en) Automated deployment of protection agents to devices connected to a distributed computer network
US6816910B1 (en) Method and apparatus for limiting network connection resources
US20020184362A1 (en) System and method for extending server security through monitored load management
US7308714B2 (en) Limiting the output of alerts generated by an intrusion detection sensor during a denial of service attack
US7707637B2 (en) Distributed threat management
US8015414B2 (en) Method and apparatus for providing fraud detection using connection frequency thresholds
US8844035B2 (en) Techniques for network protection based on subscriber-aware application proxies
US7039950B2 (en) System and method for network quality of service protection on security breach detection
US8108923B1 (en) Assessing risk based on offline activity history
US7171688B2 (en) System, method and computer program for the detection and restriction of the network activity of denial of service attack software
US8387144B2 (en) Network amplification attack mitigation
US20110035496A1 (en) Automatic hardware failure detection and recovery for distributed max sessions server
KR20070092656A (en) System for stabilizing of web service and method thereof
EP2492837A1 (en) Network communication system, server system and terminals
JP5173388B2 (en) Information processing apparatus and information processing method
MXPA05000901A (en) Mailbox pooling pre-empting criteria.
US20120072591A1 (en) Method and System To Optimize Efficiency When Managing Lists of Untrusted Network Sites
US7464410B1 (en) Protection against flooding of a server
Walfish et al. Distributed Quota Enforcement for Spam Control.
WO2002084512A1 (en) Method and system for restricting access from external
CN114338177B (en) Directional access control method and system for Internet of things
Iyengar et al. An effective layered load balance defensive mechanism against DDoS attacks in cloud computing environment
Bellaïche et al. SYN flooding attack detection by TCP handshake anomalies
JP2001265678A (en) Method for preventing flood attack using connectionless protocol

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A2

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BW BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE EG ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KM KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NA NG NI NO NZ OM PG PH PL PT RO RU SC SD SE SG SK SL SM SY TJ TM TN TR TT TZ UA UG US UZ VC VN YU ZA ZM ZW

AL Designated countries for regional patents

Kind code of ref document: A2

Designated state(s): GM KE LS MW MZ NA SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LT LU LV MC NL PL PT RO SE SI SK TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase