TEMP

The provided YAML file is a Docker Compose configuration for setting up a complete Elastic Stack environment. To convert it into individual Docker commands, we need to break down each service and translate its configuration into corresponding Docker run arguments.

Environment Variables (.env file):

Before running the Docker commands, you'll need an .env file in the same directory with the following variables (replace placeholders with your actual values):

STACK_VERSION=8.7.0 
ES_PORT=9200
CLUSTER_NAME=elastic-cluster
ELASTIC_PASSWORD=your_elastic_password
KIBANA_PASSWORD=your_kibana_password
LICENSE=trial
ES_MEM_LIMIT=2g
KB_MEM_LIMIT=1g
APM_MEM_LIMIT=1g
KIBANA_PORT=5601
REGISTRY_PORT=8080
APM_PORT=8200
ENCRYPTION_KEY=your_encryption_key
content_copy
download
Use code with caution.

Docker Commands:

Here's the breakdown of the Docker commands for each service:

1. Elasticsearch (elastic):

# Create a named volume for Elasticsearch data
docker volume create elastic-data

# Run Elasticsearch container
docker run -d \
  --name elastic \
  -p 9200:9200 \
  -v elastic-data:/usr/share/elasticsearch/data \
  -e "node.name=elastic" \
  -e "discovery.type=single-node" \
  -e "ELASTIC_PASSWORD=HFuALIS6bHZjre9KOIId" \
  -e "xpack.security.enabled=true" \
  -e "xpack.license.self_generated.type=basic" \
  --health-cmd='curl -s http://localhost:9200 | grep -q "missing authentication credentials"' \
  --health-interval=10s \
  --health-timeout=10s \
  --health-retries=120 \
  docker.elastic.co/elasticsearch/elasticsearch:8.7.1
content_copy
download
Use code with caution.
Bash

2. Setup (setup):

This service runs once and then exits. It configures the kibana_system user's password.

docker run -d --rm \
  --network=host \
  --name setup \
  -e "ELASTIC_PASSWORD=HFuALIS6bHZjre9KOIId" \
  -e "KIBANA_PASSWORD=cZzEtfPI4TbT" \
  docker.elastic.co/elasticsearch/elasticsearch:8.7.1 \
  bash -c '
    if [ x${ELASTIC_PASSWORD} == x ]; then
      echo "Set the ELASTIC_PASSWORD environment variable in the .env file";
      exit 1;
    elif [ x${KIBANA_PASSWORD} == x ]; then
      echo "Set the KIBANA_PASSWORD environment variable in the .env file";
      exit 1;
    fi;
    echo "Waiting for Elasticsearch availability";
    until curl -s http://localhost:9200 | grep -q "missing authentication credentials"; do sleep 30; done;
    echo "Setting kibana_system password";
    until curl -s -X POST -u "elastic:${ELASTIC_PASSWORD}" -H "Content-Type: application/json" http://localhost:9200/_security/user/kibana_system/_password -d "{\"password\":\"${KIBANA_PASSWORD}\"}" | grep -q "^{}"; do sleep 10; done;
    echo "All done!";
  '
content_copy
download
Use code with caution.
Bash

Important Note: Replace <your_network_name> with the name of the network that you create below for connecting these containers. The setup container must be on the same network as the elastic container.

3. Kibana (kibana):

# Create a kibana.yml file (see below)

# Run Kibana container
docker run -d \
  --name kibana \
  -p 5601:5601 \
  -v ./kibana.yml:/usr/share/kibana/config/kibana.yml \
  -e "SERVER_NAME=kibana" \
  -e "ELASTICSEARCH_HOSTS=http://localhost:9200" \
  -e "ELASTICSEARCH_USERNAME=kibana_system" \
  -e "ELASTICSEARCH_PASSWORD=cZzEtfPI4TbT" \
  -e "XPACK_SECURITY_ENABLED=true" \
  -e "XPACK_MONITORING_ENABLED=true" \
  -e "XPACK_APM_ENABLED=true" \
  -e "XPACK_SECURITY_ENCRYPTIONKEY=c34d38b3a14956121ff2170e5030b471551370178f43e5626eec58b04a30fae2" \
  -e "XPACK_ENCRYPTEDSAVEDOBJECTS_ENCRYPTIONKEY=c34d38b3a14956121ff2170e5030b471551370178f43e5626eec58b04a30fae2" \
  -e "XPACK_REPORTING_ENCRYPTIONKEY=c34d38b3a14956121ff2170e5030b471551370178f43e5626eec58b04a30fae2" \
  -e "xpack.fleet.registryUrl=http://localhost:8080" \
  --health-cmd='curl -s http://localhost:5601/login | grep -q "Kibana"' \
  --health-interval=10s \
  --health-timeout=10s \
  --health-retries=120 \
  --network=host \
  docker.elastic.co/kibana/kibana:8.7.1
content_copy
download
Use code with caution.
Bash

kibana.yml:

Create a file named kibana.yml with the following content:

server.host: "0" 
elasticsearch.hosts: [ "http://elastic:9200" ]
elasticsearch.username: "kibana_system"
elasticsearch.password: "${KIBANA_PASSWORD}"
xpack.fleet.registryUrl: http://registry:8080
content_copy
download
Use code with caution.
Yaml

4. Package Registry (registry):

docker run -d \
  --name registry \
  --network=host \
  -p 8080:8080 \
  --health-cmd='curl -s http://localhost:8080 | grep -q "Package Registry"' \
  --health-interval=10s \
  --health-timeout=10s \
  --health-retries=120 \
  docker.elastic.co/package-registry/distribution:lite-8.7.1
content_copy
download
Use code with caution.
Bash

5. APM Server (apm):

docker run -d \
  --name apm \
  -p 8200:8200 \
  --network=host \
  docker.elastic.co/apm/apm-server:8.7.1 \
  apm-server -e \
    -E output.elasticsearch.hosts=["localhost:9200"] \
    -E output.elasticsearch.protocol=http \
    -E output.elasticsearch.username=elastic \
    -E output.elasticsearch.password=HFuALIS6bHZjre9KOIId \
    -E apm-server.kibana.enabled=true \
    -E apm-server.kibana.host="localhost:5601" \
    -E apm-server.kibana.protocol="http" \
    -E apm-server.kibana.username="elastic" \
    -E apm-server.kibana.password="HFuALIS6bHZjre9KOIId" \
    -E apm-server.anonymous_access.enabled=true \
    -E apm-server.auth.anonymous=true \
    -E apm-server.auth.anonymous.allow_agent=["*"] \
    -E apm-server.auth.anonymous.allow_service=["*"] \
    -E apm-server.auth.anonymous.rate_limit.event_limit=10000 \
    -E apm-server.auth.anonymous.rate_limit.ip_limit=1000 \
    -E apm-server.auth.anonymous.rate_limit.window=1m \
    -E apm-server.rum.enabled=true \
    -E apm-server.rum.event_rate.limit=300 \
    -E apm-server.rum.event_rate.lru_size=300 \
    -E apm-server.rum.allow_origins=["*"] \
    -E apm-server.rum.library_pattern="node_modules|bower_components|~" \
    -E apm-server.rum.source_mapping.cache.expiration=5m \
    -E apm-server.rum.source_mapping.index_pattern="apm-*-sourcemap*" \
    -E apm-server.rum.source_mapping.enabled=true
content_copy
download
Use code with caution.
Bash

6. Create a Docker network:

docker network create <your_network_name>
content_copy
download
Use code with caution.
Bash

Execution Order:

Create the Docker network: Run the docker network create command.

Elasticsearch: Run the docker run command for Elasticsearch.

Setup: Run the docker run command for setup. Make sure to use the same network name you created in step 1. Wait for it to complete successfully.

Kibana, Registry, and APM Server: You can start these services in any order using their respective docker run commands. Use the same network name for all of them.

Important Considerations:

Network: All the services need to be on the same Docker network to communicate with each other. I've added --network=<your_network_name> to the commands, but you need to create the network first using docker network create <your_network_name>.

Dependencies: The depends_on option in Docker Compose is not directly translatable to docker run. You need to ensure the services are started in the correct order manually, as described above.

Healthchecks: The health checks are translated using the --health-* options in the docker run commands.

Resource Limits: The mem_limit is translated using the --memory option.

Ulimits: The ulimits are translated using the --ulimit option.

Error Handling: This conversion provides a basic translation. For production environments, you should add more robust error handling and potentially use scripts to manage the startup and shutdown of these containers.

Security: Review the security configurations for each service and adjust them according to your needs, especially in a production environment.

This detailed breakdown should help you run your Elastic Stack using individual Docker commands instead of Docker Compose. Remember to replace the placeholders with your actual values and adapt the commands as needed for your specific environment.

server.host: "0.0.0.0"
server.shutdownTimeout: "5s"
monitoring.ui.container.elasticsearch.enabled: true

xpack.fleet.packages:
  - name: apm
    version: "8.7.1"

xpack.fleet.registryUrl: "http://registry:8080"

# Password for the 'elastic' user (at least 6 characters)
ELASTIC_PASSWORD=HFuALIS6bHZjre9KOIId
# Password for the 'kibana_system' user (at least 6 characters)
KIBANA_PASSWORD=cZzEtfPI4TbT
# SAMPLE Predefined Key only to be used in POC environments
ENCRYPTION_KEY=c34d38b3a14956121ff2170e5030b471551370178f43e5626eec58b04a30fae2



# Version of Elastic products
STACK_VERSION=8.7.1
LICENSE=basic
CLUSTER_NAME=ecv-elastic-cluster

# port number for various services
ES_PORT=9200
KIBANA_PORT=5601
APM_PORT=8200
REGISTRY_PORT=8080

# Increase or decrease based on the available host memory (in bytes)
ES_MEM_LIMIT=1073741824
KB_MEM_LIMIT=1073741824
APM_MEM_LIMIT=1073741824

version: "3.8"

volumes:
  elastic-data:

services:
  elastic:
    image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}
    volumes:
      - elastic-data:/usr/share/elasticsearch/data
    ports:
      - ${ES_PORT}:9200

    environment:
      - node.name=elastic
      - cluster.name=${CLUSTER_NAME}
      - discovery.type=single-node
      - ELASTIC_PASSWORD=${ELASTIC_PASSWORD}
      - bootstrap.memory_lock=true
      - xpack.security.enabled=true
      - xpack.license.self_generated.type=${LICENSE}

    mem_limit: ${ES_MEM_LIMIT}
    ulimits:
      memlock:
        soft: -1
        hard: -1

    healthcheck:
      test: ["CMD-SHELL", "curl -s http://localhost:9200 | grep -q 'missing authentication credentials'"]
      interval: 10s
      timeout: 10s
      retries: 120

  setup:
    image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}
    depends_on:
      - elastic
    command: >
      bash -c '
        if [ x${ELASTIC_PASSWORD} == x ]; then
          echo "Set the ELASTIC_PASSWORD environment variable in the .env file";
          exit 1;
        elif [ x${KIBANA_PASSWORD} == x ]; then
          echo "Set the KIBANA_PASSWORD environment variable in the .env file";
          exit 1;
        fi;
        echo "Waiting for Elasticsearch availability";
        until curl -s http://elastic:9200 | grep -q "missing authentication credentials"; do sleep 30; done;
        echo "Setting kibana_system password";
        until curl -s -X POST -u "elastic:${ELASTIC_PASSWORD}" -H "Content-Type: application/json" http://elastic:9200/_security/user/kibana_system/_password -d "{\"password\":\"${KIBANA_PASSWORD}\"}" | grep -q "^{}"; do sleep 10; done;
        echo "All done!";
      '

  kibana:
    image: docker.elastic.co/kibana/kibana:${STACK_VERSION}
    volumes:
      - ./kibana.yml:/usr/share/kibana/config/kibana.yml
    ports:
      - ${KIBANA_PORT}:5601
    depends_on:
      - elastic
      - setup
    mem_limit: ${KB_MEM_LIMIT}
    environment:
      - SERVER_NAME=kibana
      - ELASTICSEARCH_HOSTS=http://elastic:9200
      - ELASTICSEARCH_USERNAME=kibana_system
      - ELASTICSEARCH_PASSWORD=${KIBANA_PASSWORD}
      - XPACK_SECURITY_ENABLED=true
      - XPACK_MONITORING_ENABLED=true
      - XPACK_APM_ENABLED=true
      - XPACK_SECURITY_ENCRYPTIONKEY=${ENCRYPTION_KEY}
      - XPACK_ENCRYPTEDSAVEDOBJECTS_ENCRYPTIONKEY=${ENCRYPTION_KEY}
      - XPACK_REPORTING_ENCRYPTIONKEY=${ENCRYPTION_KEY}
      - xpack.fleet.registryUrl=http://registry:8080

    healthcheck:
      test: ["CMD-SHELL", "curl -s http://localhost:5601/login | grep -q 'Kibana'"]
      interval: 10s
      timeout: 10s
      retries: 120

  registry:
    image: docker.elastic.co/package-registry/distribution:lite-${STACK_VERSION}
    ports:
      - ${REGISTRY_PORT}:8080
    healthcheck:
      test: ["CMD-SHELL", "curl -s http://localhost:8080 | grep -q 'Package Registry'"]
      interval: 10s
      timeout: 10s
      retries: 120

  apm:
    image: docker.elastic.co/apm/apm-server:${STACK_VERSION}
    depends_on:
      - elastic
      - setup
    ports:
      - ${APM_PORT}:8200
    mem_limit: ${APM_MEM_LIMIT}

    command: >
      apm-server -e
        -E output.elasticsearch.hosts=["elastic:${ES_PORT}"]
        -E output.elasticsearch.protocol=http
        -E output.elasticsearch.username=elastic
        -E output.elasticsearch.password=${ELASTIC_PASSWORD}

        -E apm-server.kibana.enabled=true
        -E apm-server.kibana.host="kibana:5601"
        -E apm-server.kibana.protocol="http"
        -E apm-server.kibana.username="elastic"
        -E apm-server.kibana.password="${ELASTIC_PASSWORD}"

        -E apm-server.anonymous_access.enabled=true
        -E apm-server.auth.anonymous=true
        -E apm-server.auth.anonymous.allow_agent=["*"]
        -E apm-server.auth.anonymous.allow_service=["*"]
        -E apm-server.auth.anonymous.rate_limit.event_limit=10000
        -E apm-server.auth.anonymous.rate_limit.ip_limit=1000
        -E apm-server.auth.anonymous.rate_limit.window=1m

        -E apm-server.rum.enabled=true 
        -E apm-server.rum.event_rate.limit=300 
        -E apm-server.rum.event_rate.lru_size=300 
        -E apm-server.rum.allow_origins=["*"] 
        -E apm-server.rum.library_pattern="node_modules|bower_components|~" 
        -E apm-server.rum.source_mapping.cache.expiration=5m 
        -E apm-server.rum.source_mapping.index_pattern="apm-*-sourcemap*" 
        -E apm-server.rum.source_mapping.enabled=true

listeners.tcp = none
listeners.ssl.default = 5672

ssl_options.cacertfile = /etc/rabbitmq/ca_certificate.pem
ssl_options.certfile = /etc/rabbitmq/server_certificate.pem
ssl_options.keyfile = /etc/rabbitmq/server_key.pem
ssl_options.verify = verify_peer
ssl_options.depth  = 2
ssl_options.fail_if_no_peer_cert = false

#!/bin/bash

set -e

# Get hostname from enviromant variable
HOSTNAME=$(hostname)
if [ $? -ne 0 ]; then
  echo "Error: Could not determine hostname."
  exit 1
fi

echo "Starting RabbitMQ Server For host: " $HOSTNAME

if [ ! -z "$JOIN_CLUSTER_HOST" ]; then
    echo "Starting as clıster node" $HOSTNAME "at" $JOIN_CLUSTER_HOST
    rabbitmq-server -detached
    sleep 5
    rabbitmqctl wait /var/lib/rabbitmq/mnesia/rabbit\@$HOSTNAME.pid
    rabbitmqctl stop_app
    rabbitmqctl join_cluster rabbit@$JOIN_CLUSTER_HOST
    rabbitmqctl stop
    sleep 2
fi

# Keep foreground process active ...
rabbitmq-server

version: "3"
services:
  rabbitmq1:
    image: masstransit/rabbitmq
    hostname: rabbitmq1
    environment:
      - RABBITMQ_DEFAULT_USER=${RABBITMQ_DEFAULT_USER}
      - RABBITMQ_DEFAULT_PASS=${RABBITMQ_DEFAULT_PASS}
      - RABBITMQ_DEFAULT_VHOST=${RABBITMQ_DEFAULT_VHOST}
    volumes:
      - ./.erlang.cookie:/var/lib/rabbitmq/.erlang.cookie
      - ./rabbitmq-entrypoint.sh:/usr/local/bin/rabbitmq-entrypoint.sh
      - ./rmq1ssl/ca_certificate.pem:/etc/rabbitmq/ca_certificate.pem
      - ./rmq1ssl/server_certificate.pem:/etc/rabbitmq/server_certificate.pem
      - ./rmq1ssl/server_key.pem:/etc/rabbitmq/server_key.pem
      - ./rabbitmq.conf:/etc/rabbitmq/rabbitmq.conf
      - rmq1data:/var/lib/rabbitmq/mnesia
    entrypoint: /usr/local/bin/rabbitmq-entrypoint.sh

  rabbitmq2:
    image: masstransit/rabbitmq
    hostname: rabbitmq2
    environment:
      - JOIN_CLUSTER_HOST=rabbitmq1
      - RABBITMQ_DEFAULT_USER=${RABBITMQ_DEFAULT_USER}
      - RABBITMQ_DEFAULT_PASS=${RABBITMQ_DEFAULT_PASS}
      - RABBITMQ_DEFAULT_VHOST=${RABBITMQ_DEFAULT_VHOST}
    volumes:
      - ./.erlang.cookie:/var/lib/rabbitmq/.erlang.cookie
      - ./rabbitmq-entrypoint.sh:/usr/local/bin/rabbitmq-entrypoint.sh
      - ./rmq2ssl/ca_certificate.pem:/etc/rabbitmq/ca_certificate.pem
      - ./rmq2ssl/server_certificate.pem:/etc/rabbitmq/server_certificate.pem
      - ./rmq2ssl/server_key.pem:/etc/rabbitmq/server_key.pem
      - ./rabbitmq.conf:/etc/rabbitmq/rabbitmq.conf
      - rmq2data:/var/lib/rabbitmq/mnesia
    entrypoint: /usr/local/bin/rabbitmq-entrypoint.sh

  rabbitmq3:
    image: masstransit/rabbitmq
    hostname: rabbitmq3
    environment:
      - JOIN_CLUSTER_HOST=rabbitmq1
      - RABBITMQ_DEFAULT_USER=${RABBITMQ_DEFAULT_USER}
      - RABBITMQ_DEFAULT_PASS=${RABBITMQ_DEFAULT_PASS}
      - RABBITMQ_DEFAULT_VHOST=${RABBITMQ_DEFAULT_VHOST}
    volumes:
      - ./.erlang.cookie:/var/lib/rabbitmq/.erlang.cookie
      - ./rabbitmq-entrypoint.sh:/usr/local/bin/rabbitmq-entrypoint.sh
      - ./rmq3ssl/ca_certificate.pem:/etc/rabbitmq/ca_certificate.pem
      - ./rmq3ssl/server_certificate.pem:/etc/rabbitmq/server_certificate.pem
      - ./rmq3ssl/server_key.pem:/etc/rabbitmq/server_key.pem
      - ./rabbitmq.conf:/etc/rabbitmq/rabbitmq.conf
      - rmq3data:/var/lib/rabbitmq/mnesia
    entrypoint: /usr/local/bin/rabbitmq-entrypoint.sh

  # load balance with ngnix
  ngnix:
    image: nginx:latest
    ports:
      - 80:80
      - 5672:5672
      - 15672:15672
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf

volumes:
  rmq1data:
  rmq2data:
  rmq3data: