DualSpark/ops-elkstack

语言: Python

git: https://github.com/DualSpark/ops-elkstack

Elk Stack演示和部署
Elk Stack demo and deploy
README.md (中文)

每个堆栈

使用对流层生成的Cloudformation模板进行Elasticsearch,Logstash和Kibana(ELK堆栈)部署。

入门

使用Python虚拟环境隔离依赖项:

git clone git@github.com:dualspark/ops-elkstack
cd ops-elkstack
pip install virtualenvwrapper
mkvirtualenv elkstack
python setup.py develop

在将来处理此项目时,请使用elk stack上的工作来激活已配置的环境,并在完成后停用。

配置AWS身份验证

如果您安装并配置了AWS CLI,则无需执行任何其他操作。

如果您未安装AWS CLI,请按照AWS CLI页面上的说明进行操作。然后运行aws configure来设置您的凭据。

您还可以手动创建以下两个文件:

〜/ .aws /凭证

[default]
aws_access_key_id = ACCESS_KEY
aws_secret_access_key = SECRET_KEY

〜/ .aws /配置

[default]
output = json
region = us-west-2

为SSH创建EC2密钥对

您需要生成一个EC2密钥对,用于SSH到实例。此密钥的名称是任意的,但需要配置网络部署过程。如果需要,您可以重用现有密钥对。

使用ops-elk堆栈

运行elkstack.py文件:

./src/elkstack.py create

这将从您的AWS账户收集信息,以了解可用的区域和可用区域。然后,它将创建一个默认的config.json文件和一个名为environmentbase.template的Cloudformation模板。

自定义config.json

以下是验证所需的最小项目集:

  • 在“模板”部分中:
  • 将ec2_key_default的值设置为上面提到的密钥对名称。
  • 将remote_access_cidr的值设置为您希望能够从中访问堡垒主机的CIDR范围。这是一个单一的CIDR范围(目前),可能是公司办公室的网络出口CIDR等。
  • 在“网络”部分内:
  • 根据您的喜好/需要设置网络大小和CIDR基础的值。请注意,此过程将在配置的每个AWS可用区中创建公共和私有子网(按顺序,最多3个)。

运行ops-elk堆栈

./src/elkstack.py create
./src/elkstack.py deploy

创造了什么

通过使用cloudformation-environmentbase,VPC包含在生成的Cloudformation模板中。检查该项目的自述文件以查看有关VPC的更多信息。

为ELK堆栈创建子Cloudformation堆栈。这样可以更灵活地更新堆栈或将其删除,但保留VPC完好无损。此子堆栈存储在config.json中引用的S3存储桶中。父堆栈通过其S3 URL引用堆栈。显示参考的片段:

"ElkStackStack":{
"Type":"AWS::CloudFormation::Stack",
"Properties":{
"TemplateURL":"https://BUCKETNAME.s3.amazonaws.com/devtools/cloudformation/ElkStack.1438189565.template"

通过Elastic Load Balancer URL访问

Kibana ELB从ELK Cloudformation堆栈输出。可以使用端口5601上的URL访问Kibana。

本文使用googletrans自动翻译,仅供参考, 原文来自github.com

en_README.md

elkstack

Elastichsearch, Logstash and Kibana (ELK stack) deployment using troposphere-generated Cloudformation templates.

Getting Started

Using a Python virtual environment to isolate dependencies:

git clone git@github.com:dualspark/ops-elkstack
cd ops-elkstack
pip install virtualenvwrapper
mkvirtualenv elkstack
python setup.py develop

When working on this project in the future, use workon elkstack to activate the configured environment and deactivate when you're finished.

Configuring AWS authentication

If you have the AWS CLI installed and configured, there's nothing else to do.

If you do not have the AWS CLI installed, follow the instructions on the AWS CLI page. Then run aws configure to set up your credentials.

You can also manually create the following two files:

~/.aws/credentials

[default]
aws_access_key_id = ACCESS_KEY
aws_secret_access_key = SECRET_KEY

~/.aws/config

[default]
output = json
region = us-west-2

Creating an EC2 key pair for SSH

You need to generate an EC2 Key Pair to use to SSH into instances. The name of this key is arbitrary, but it is needed to configure the network deployment process. You can reuse an existing key pair if desired.

Using ops-elkstack

Run the elkstack.py file:

./src/elkstack.py create

This will gather information from your AWS account to know what regions and availability zones are available. It will then create a default config.json file and a Cloudformation template named environmentbase.template.

Customizing config.json

The following are the minimal set of items that are necessary to validate:

  • within the 'template' section:
  • Set the value of ec2_key_default to the key pair name mentioned above.
  • Set the value of remote_access_cidr to a CIDR range that you want to be able to access the bastion host from. This is a single CIDR range (for now) and could be the network egress CIDR for a corporate office, etc.
  • within the 'network' section:
  • Set the values of the network size and CIDR base to your liking/needs. Note that this process will create a public and private subnet in each of the AWS Availability Zones configured (in order, up to 3).

Running ops-elkstack

./src/elkstack.py create
./src/elkstack.py deploy

What's created

By using cloudformation-environmentbase, a VPC is included in the generated Cloudformation template. Check that project's Readme to see more information about the VPC.

A child Cloudformation stack is created for the ELK stack. This allows more flexibility for updating the stack or removing it, but leaving the VPC intact. This child stack is stored in the S3 bucket referenced in config.json. The parent stack refers to the stack by its S3 URL. Snippet showing the reference:

"ElkStackStack":{
"Type":"AWS::CloudFormation::Stack",
"Properties":{
"TemplateURL":"https://BUCKETNAME.s3.amazonaws.com/devtools/cloudformation/ElkStack.1438189565.template"

Access via Elastic Load Balancer URLs

The Kibana ELB is outputted from the ELK Cloudformation stack. Kibana can be accessed by using that URL on port 5601.