It is assumed that Docker is already installed and running correctly on the installation machine. If not, please follow the appropriate instructions to install Docker on your system: Install Docker.
docker --version Docker version 17.12.0-ce, build c97c6d6
Please ensure you are using a reasonably up to date version, at least 17.x or above. If this is not the case then please visit Docker Get Started for more information.
The TAG Docker Image, includes a ready-to-run standalone instance of the Tribestream API Gateway and an instance of the NOSQL datastore Cassandra used by TAG for persistence.
Starting a TAG instance is simple:
docker run -it --name tag -e LICENSE=accept tomitribe/tribestream-api-gateway
This will run both TAG and Cassandra in a single Docker container. The environment variable LICENSE=accept will accept the End-User Licence Agreement. Please, read it before you start here: Tomitribe End-User License Agreement.
By default, the TAG runs HTTP on port 8080. If you need to expose the port in your Docker container, you can use the following command:
docker run -it -p 8080:8080 --name tag -e LICENSE=accept tomitribe/tribestream-api-gateway
Running both TAG and Cassandra in the same image is convenient for evaluation, development, and low-volume environments. For critical production workloads, we recommended you run Cassandra in a separate Docker Container. Use the Tomitribe specific Cassandra Docker Image for the TAG. Since the TAG requires Cassandra to startup, the Cassandra Image should be started first.
Starting Cassandra for TAG is simple:
docker run -it -p 9042:9042 --name cassandra -e CASSANDRA_REPLICATION="'dc1':1" tomitribe/tribestream-cassandra
By default, Cassandra runs on port 9042. You need to expose the port to connect to it. The variable CASSANDRA_REPLICATION sets Cassandra with a Replication factor of 1, meaning that only a single Cassandra node is required to start TAG. The default is 2. For more information about Cassandra Replication, please check Cassandra Data Distribution and Replication
Run TAG:
docker run -it -p 8080:8080 --name tag \
-e LICENSE=accept \
-e CASSANDRA_AUTHORITIES=[host]:9042 \
tomitribe/tribestream-api-gateway
Where host is the hostname of the container running Cassandra.
The Tomitribe specific Cassandra Docker Image for the TAG, extends from the official Cassandra Docker Image. All configuration supported in the Cassandra Docker Image is also supported by the Cassandra Docker Image for the TAG.
A TAG and Cassandra cluster always require a Master TAG and a Cassandra Seed node. These are only required at the Cluster startup to initialize data and settings. Afterwards, all nodes in the Cluster will behave in the same way, without any difference between them. The following instructions must be used to run Cassandra on separate machines.
Start Cassandra Seed node:
docker run -it -p 9042:9042 -p 7000:7000 --name cassandra-seed \
-e CASSANDRA_BROADCAST_ADDRESS=[cassandra-seed-ip]
tomitribe/tribestream-cassandra
Where cassandra-seed-ip is the ip of the machine running the Cassandra Seed Node. Additionally, you can use the environment variable CASSANDRA_REPLICATION to set up Cassandra replication factor.
The default replication factor is 2, if not specified.
Start additional Cassandra nodes:
docker run -it -p 9042:9042 -p 7000:7000 --name cassandra-node-[number] \
-e CASSANDRA_SEEDS=[cassandra-seed-ip] \
-e CASSANDRA_BROADCAST_ADDRESS=[cassandra-node-ip] \
tomitribe/tribestream-cassandra
Where cassandra-seed-ip is the ip of the machine running the Cassandra Seed node and cassandra-node-ip the ip address of the Cassandra Node (since the address of the container is behind the docker bridge). Use the same command to run multiple nodes, only by changing the number and the ip to represent the node instance being executed.
Start a TAG Master node:
docker run -it -p 8080:8080 --name tag-master \
-e LICENSE=accept \
-e CASSANDRA_AUTHORITIES=[cassandra-cluster-hosts] \
tomitribe/tribestream-api-gateway
Where cassandra-cluster-hosts is a comma separated list of hosts and ports for all Cassandra nodes, in the format cassandra-node1:port,cassandra-node2:port.
Start a TAG node:
docker run -it -p 8080:8080 --name tag-node-[number] \
-e LICENSE=accept \
-e TAG_MASTER_HOST:[tag-master-host] \
-e TAG_MASTER_PORT:[tag-master-port] \
-e CASSANDRA_AUTHORITIES=[cassandra-cluster-hosts] \
tomitribe/tribestream-api-gateway
Where tag-master-host is the hostname of the container running the TAG Master and tag-master-port the port of the container running TAG Master. Use the same command to run multiple nodes, only by changing the number to represent the node instance being executed.
If you wish to run all the containers in the same machine, we recommend to use a Docker Compose file:
version: '3'
services:
cassandra-node-1:
image: tomitribe/tribestream-cassandra
container_name: tag-cassandra-node-1
environment:
- JVM_OPTS=-Xms256m -Xmx512m -Xmn128m
cassandra-node-2:
image: tomitribe/tribestream-cassandra
container_name: tag-cassandra-node-2
environment:
- CASSANDRA_SEEDS=cassandra-node-1
- JVM_OPTS=-Xms256m -Xmx512m -Xmn128m
depends_on:
- cassandra-node-1
cassandra-node-3:
image: tomitribe/tribestream-cassandra
container_name: tag-cassandra-node-3
environment:
- CASSANDRA_SEEDS=cassandra-node-1
- JVM_OPTS=-Xms256m -Xmx512m -Xmn128m
depends_on:
- cassandra-node-1
tag-node-1:
image: tomitribe/tribestream-api-gateway
container_name: tag-node-1
ports:
- 8080:8080
- 8443:8443
environment:
- LICENSE=accept
- CASSANDRA_AUTHORITIES=cassandra-node-1:9042,cassandra-node-2:9042,cassandra-node-3:9042
- HAZELCAST_MEMBERS=tag-node-1,tag-node-2
depends_on:
- cassandra-node-1
- cassandra-node-2
- cassandra-node-3
tag-node-2:
image: tomitribe/tribestream-api-gateway
container_name: tag-node-2
ports:
- 8081:8080
- 8444:8443
environment:
- LICENSE=accept
- CASSANDRA_AUTHORITIES=cassandra-node-1:9042,cassandra-node-2:9042,cassandra-node-3:9042
- HAZELCAST_MEMBERS=tag-node-1,tag-node-2
- TAG_MASTER_HOST=tag-node-1
depends_on:
- cassandra-node-1
- cassandra-node-2
- cassandra-node-3
Just save the snippet into a docker composer file tag-cassandra-cluster.yaml and you can run it with:
docker-compose -f tag-cassandra-cluster.yaml up
It will start 3 Cassandra nodes and 2 TAG nodes for you automatically.
Elasticsearch is a distributed, RESTful search and analytics engine used by the TAG to store Metrics data. Kibana provides visualization capabilities on top of the content indexed on the Elasticsearch engine, where Dashboards build by TAG can be seen.
We use the popular community version of Elasticsearch, Logstash and Kibana (ELK) Docker Image. For more information please check Elasticsearch, Logstash, Kibana (ELK) Docker Image
You can run it with:
docker run -it --name elk -p 5601:5601 -p 9200:9200 -p 5044:5044 sebp/elk:564
At this time, the TAG only supports version 5.x of the ELK stack.
Then run the TAG and pass in Elasticsearch and Kibana configuration:
docker run -it --name tag -p 8080:8080 \
-e LICENSE=accept \
-e ELASTICSEARCH_HOST=[elasticsearch-host] \
-e KIBANA_HOST=[kibana-host] \
tomitribe/tribestream-api-gateway
Where elasticsearch-host and kibana-host is the hostname of the container running Elasticsearch and Kibana.
For TAG Image:
LICENSE
This variable is used to Accept the Tomitribe End-User License Agreement (EULA) to be able to run the TAG. A value of accept will accept the EULA. Any other value will print the EULA and the server won’t start.
TAG_MASTER_HOST
This variable is used to specify the hostname of the master TAG when running in Cluster mode. This is required for the initial TAG data to be seed to the other nodes.
TAG_MASTER_PORT
This variable is used to specify the port of the master TAG when running in Cluster mode. This is required for the initial TAG data to be seeded to the other nodes. You must also set TAG_MASTER_HOST, otherwise, this variable will have no effect. Defaults to 8080.
CASSANDRA_AUTHORITIES
This variable is used to specify the list of the Cassandra nodes that the TAG will use to connect to, in a comma-separated format cassandra-node1:port,cassandra-node2:port.
HAZELCAST_MEMBERS
This variable is used to specify the list of the Hazelcast members that the TAG will use in Cluster mode. This is only required if the TAG is unable to auto-discover members, due to network settings. Use a comma-separated format to specify members member1[:port],member2[:port]; port is optional and the default is Hazelcast default 5701.
ELASTICSEARCH_HOST
This variable is used to specify the hostname of an Elasticsearch instance that the TAG will use to connect to, and publish Metrics Data.
ELASTICSEARCH_PORT
This variable is used to specify the port of an Elasticsearch instance that the TAG will use to connect to, and publish Metrics Data. You must also set ELASTICSEARCH_HOST, otherwise this variable will have no effect. The default is to 9200.
KIBANA_HOST
This variable is used to specify the port of a Kibana instance that the TAG will use to publish Metrics Dashboards.
KIBANA_PORT
This variable is used to specify the port of a Kibana instance that the TAG will use to connect. You must also set KIBANA_HOST, otherwise, this variable will have no effect. The default is 5601.
For Cassandra Image:
CASSANDRA_SEEDS
This variable is used to specify the hostname of the Cassandra Seed in Cluster mode.
CASSANDRA_REPLICATION
This variable is to specify the replication settings in Cassandra in Cluster mode. Use a comma separated format 'datacenter-name-1':replication,'datacenter-name-2':replication, where datacenter-name is the name of the Cassandra Datacenter and replication the number of replicas for the Cassandra Cluster. Sample: 'dc1':3,'dc2':2. The default is 'dc1':3.
More variables are available from the official Cassandra Docker Image.
The TAG and Cassandra configuration files can be overridden directly using the Docker run command.
The TAG configuration files are stored inside the container in the /opt/tag/conf folder. The most common files that contain configuration suitable to be overridden are:
tomee.xml
server.xml
logging.properties
system.properties
hazelcast.xml
The Cassandra configuration files are stored inside the container in multiple locations:
/etc/cassandra/cassandra.yaml
/etc/cassandra/jvm.options
/etc/cassandra/logback.xml
/etc/cassandra/cassandra-rackdc.properties
/usr/share/cassandra/cassandra.in.sh
To override any of these files, just add -v source-file:/target-file:ro to your Docker command, where source-file is the file you want to use as the new configuration in the Docker container, and target-file if the file inside the Docker container that you want to override.
If the TAG is to control the authentication and authorization (recommended), then an LDAP server may be used for the authentication part.
It likely that an existing LDAP server will be used, but for evaluation, development, or low volume environments it may be beneficial to run an embedded server.
Using LDAP is not a requirement, as the TAG can actually manage users internally as "gateway accounts". User identification can be defined in such a way that escalation occurs; for example, if the user is not found in the gateway, then it will check LDAP, and then maybe a database.
A basic Docker OpenLDAP server can be started using the following command:
docker run -p 389:389 -it
-e LDAP_ORGANISATION="your.org.com" \
-e LDAP_DOMAIN="your.domain.com" \
-e LDAP_ADMIN_PASSWORD="a-secret" \
--name ldap osixia/openldap:1.1.9
This will provide a really simple setup to test against, but can also be configured for production use. For more information on OpenLDAP, please read the official documentation here: OpenLDAP Docker Documentation.