Skip to content

Commit 2e1f9c4

Browse files
author
charles_hsiao
committed
Init repo by add README.md
0 parents commit 2e1f9c4

File tree

1 file changed

+31
-0
lines changed

1 file changed

+31
-0
lines changed

README.md

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
# AWS Lambda Elasticsearch LogMonitor
2+
Monitor specific log pattern/aggregations by Elasticsearch API. <br>
3+
Leverage AWS Lambda+DynamoDB to achieve serverless log monitoring solution.
4+
5+
## Introduction
6+
ElasticSearch provide [Search API](https://www.elastic.co/guide/en/elasticsearch/reference/current/search.html), [Aggregations API](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations) to query logs query/aggregations easily.
7+
By leverage these API, we can create a monitoring method by aggragate specific log pattern volume/aggregations. And send notification.
8+
In this project, we use Python to create AWS Lambda+DynamoDB to achieve serverless and flexible configuration log monitoring solution.
9+
(Lambda function create by [python-lambda](https://github.com/nficano/python-lambda))
10+
11+
## Get Started
12+
### Create Monitor Item
13+
```
14+
1. Using HTTP request tool like Postman to valid your log filter patterns, the post body will be parameters we use later
15+
2. git clone
16+
3. Create monitor item json content (Please see paragraph "Monitor Item Parameter" or reference "/tools/sample.json")
17+
4. Use /tools/validation.py ${json_file} to valid the monitor item json content that you just created
18+
5. If validation passed, put this json content to DynamoDB table "lambda.elk.monitor"
19+
```
20+
21+
### Monitor Item Parameter
22+
Parameters | Description | Optional Value | Example
23+
----------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------- | ---------------
24+
monitor_id | The monitor name, will use for notification | - | Cyclops-Beta-SignIn_FailedRate
25+
enable | Whether enable this monitor or not | 1. true <br> 2. false | true
26+
elk_env | The elasticsearch environment to connect | 1. staging <br>2. production | staging
27+
formula | The formula for calculation, parameter: <br> 1. ${}:Log count (\$\{([A-Za-z0-9_.]+)\}) <br> 2. %{}:Json path with "." split (\%\{([A-Za-z0-9_.]+)\}) | - | 1. ${SignInFailed_Count}/${SignInTotal_Count} <br>2. %{aggregations.upstream_time.avg_value.value}
28+
parameters | The filter json content on Create Monitor Item - step1 | - | -
29+
alarms | The alarm criteria, will trigger by calculation result of formula | 1. alert <br>2. warning | {"alert": ">=0.1","warning": ">=0.05"}
30+
notify | The notification method | 1. slack | {"slack": "${SLACK_WebHook_URL}"}
31+

0 commit comments

Comments
 (0)