Introducation of Microservices Consul and MCQs

What is Consul?

Consul is the one stop solution for microservices (self) registration, discovery, health checks, key-value store and load balancing. Consul is multi-data center aware by default, which means we can connect consul clusters from multiple DCs. Consul works as a cluster of client and server processes.

Service registration and discovery

A service can be registered using a service definition, or by using the HTTP API. The microservice can register itself with Consul using the HTTP APIs during start up. Once the service is registered with Consul, any service which is in the same Consul cluster can discover and consume it.

De-registering services

Consul also allows to de-register a service via its exposed HTTP APIs. De-registration APIs allow the services to remove itself from the Consul registry. Combining de-registration with the self registration during server startup can help us achieve zero downtime during deployments. This means, during sequential/rolling deployment of services while one node is under deployment, the other node(s) will remain available. 

Load balancing & DNS interface

Consul provides a DNS interface. Every service is registered with the ‘service.consul’ domain. This is discoverable within the cluster (and outside, if it is registered as an external service). A service is registered with the same service name in all the instances it is deployed. When multiple instances are serving APIs – containerized or clustered, load balancing is crucial. Consul’s default load balancing – a randomized round robin response – is suitable for most of the scenarios. This means that when a service is deployed in two nodes under the same name, each time  a request comes to Consul servers, the nodes are hit in a random fashion similar to round robin (It is not guaranteed that the nodes are hit one after the other). However, there is a little latency associated with DNS interface which is fine for most of our use cases. For latency sensitive applications, we can always plug in a load balancer within  Consul like HA Proxy. 

Why Consul?

Consul set up is straight forward. It hardly takes half a day to set up the cluster for a couple of dozen nodes. Installation of the client and server agents are as simple as downloading and unzipping a single binary file, for both clients and servers. The DNS interface resolves the service names to different instances each time a service is called(in a round robin fashion). This helps removing one layer of infrastructure, i.e. a load balancer with static configuration. The clients will have to be Consul-aware, though. This means instead of mapping the proxy, we will be mapping the Consul service names.

The most important of all, is the extensive health check mechanism to monitor the service instances. The default health check script keeps checking the aliveness of registered services, with an easy way to integrate any number of custom health checks. depending on the time-out configured, the health checks are triggered periodically. Health check scripts should return PASSING, WARNING or CRITICAL response. Consul redirects requests only to healthy service instances – PASSING or WARING – using its DNS interface therebyimproving the availability of the services. If a service runs in critical health longer than the configured deregistration timeout (parameter: deregisterCriticalServicesAfter) Consul deregisters the service. Once the service is back available, since the health check is registered already, it is added back to the list of healthy service instances. Please find a set of basic health checks with the attachments.

The next useful feature is the hierarchical key value store. The KV store helps storing the data, such as ports, service parameters, just to name a few. Consul picks the service instances in a round-robin fashion by default. Like the health checks, this can also be extended to use other load balancing solutions. In other words, Consul is a ready-to-use solution with scope for a variety of customisation. A Consul agent is a long running daemon process with a rather simple 3-step set up, as explained in the set-up section. There can be thousands of client agents running within a Consul cluster. These agents are responsible for checking the health of the node and/or the service. However, we need only a few server agents within a Consul cluster. Though a cluster can function with a single server agent, it is highly recommended to run 3 or 5 server agents to avoid Consul becoming the single point of failure for the application. Below is the picture depicting the Consul architecture (from Consul official documentation).

Service Directory (GSD) > Microservices registration and discovery using Consul > consul-architecture.png

Clients and servers:

Each node which is part of the cluster is required to run a client agent. The Consul client agent registers the services, checks the health of the services or the node itself, but forwards all the service discovery requests to the servers. A client is stateless. Apart from the frequent gossiping, these daemon processes are not running anything in the background. This design helps maintain the client agents lightweight, there by supporting rapid horizontal scalability of thousands of clients with 3 or 5 server agents.  The clients communicate to each other using the gossip protocol, and use UDP pings most of the time.

A server agent can be either a leader, follower, or a candidate. Each server starts out as a follower, and once all the servers are started a leader is elected. We can also specify which server to be elected as leader at first, by bootstrapping. The leader handles all the requests from the client agents, and replicates the state to its peers (follower server nodes). If a follower receives a request, it is forwarded to the leader. The leader election is based on a consensus protocol called Serf. Underlying Consul is the Raft db which stores the state and logs the changes. The data is replicated between the servers. 

Consul servers from different DCs can talk over the WAN using WAN gossip using TCP/UDP . Click here to read more.

How to set up Consul

Below is an example of a Cluster set up with 3 server nodes and a couple of clients.

This section talks specifically about setting up Consul agents in Linux VMs. Here, we will set up a 3-server and n-client cluster. Let’s say, each micro service (of a particular version) is deployed to two VMs. The client agents will be installed onto each of these VMs and will be joined to the cluster. This section talks only about registering the internal Java Play micro services, which are running on VMs and accessible via VPN.

Installation instructions:

1. Download and unzip:

              sudo wget https://releases.hashicorp.com/consul/0.7.2/consul_0.7.2_linux_amd64.zip

              sudo unzip consul_0.7.2_linux_amd64.zip

       2. Start the agent in server mode in all the VMs which are going to be running as server agents:

               consul agent -server  -data-dir /tmp/consul -join=<IP.of.any.vm.in.the.cluster>

               Alternatively, the configuration can be specified in a config file, which is recommended for production use.

              consul agent -server -config-file=./consul-config.json

      3. Start the agents in client mode on all the VMs which are going to register/use the services:

           consul agent -data-dir /tmp/consul -config-dir /etc/consul.d  -join=<IP.of.any.vm.in.the.cluster>

By now, the clients and servers are running and are part of a Consul cluster.

Service Registration

The consul agent must be running on the VM where the service is deployed.

       1. Manual registration using consul agent’s REST API during Service startup

          a) In the start.sh, register the service to Consul using:

               curl -X PUT -d ‘{“ID”: <ServiceID>, “Name”: <ServiceName>, “Check”: {“id”: <CheckID>, “script”: <custom script to check the service health>, “interval”: <e.g. 10s>}}’ http://localhost:8500/v1/agent/service/register

          b) Upload the port number of the service to the consul key value store:

              curl -X PUT -d $PORT http://localhost:8500/v1/kv/<ServiceID>

       2. Self-Registration of Service – Java Spring boot

            a) Add the following dependency in pom.xml

<dependency>

<groupId>org.springframework.cloud</groupId>

<artifactId>spring-cloud-starter-consul-discovery</artifactId>

</dependency>

  <dependencyManagement>

               <dependencies> 

  <dependency> 

    <groupId>org.springframework.cloud</groupId> 

    <artifactId>spring-cloud-consul- dependencies</artifactId> 

    <version>1.1.2.RELEASE</version> 

    <type>pom</type> 

    <scope>import</scope> 

  </dependency> 

</dependencies>

</dependencyManagement>

         b) Add the following configuration in application.yml file.

spring:

   application:

      name: <Service name>

  cloud:

     consul:

        discovery:

           preferAgentAddress: true

           tags: <API Version>

           healthCheckPath: /management/health

           healthCheckInterval: 5s

In short, the service registers itself to Consul. When the service is registered consul stores it as <ServiceID>.service.consul. This is accessible to any applications in VMs running agents of the same cluster.

Service Discovery and Verification

Any agent that is part of the same cluster can discover and use the registered service. 

              On the client machine, ping the Service (<ServiceID>.service.consul). You will see response coming from different VMs, where the Services are running

               ping  -c1 <ServiceID>.service.consul

               Then hit the Microservice from the client machine using the “curl” command. Here, you will use Consul ServiceID instead of Hostname. If this works, you application should have no problem in calling the Microservice

               curl -X GET http://<ServiceId>.service.consul:<port>/index

 There is also a Java API Client available for Service Discovery, which will return list of Healthy Microservices.

System Configuration

Linux specific configurations:

Install dnsmasq to forward and resolve consul domain queries from consul DNS port to default DNS port of the VM.

Dnsmasq configuration is under /etc/dnsmasq.d by default.

Create a file which will have the below content: 

server=/consul/127.0.0.1#8600

Edit /etc/dnsmasq.conf and uncomment listen-address.

listen-address=127.0.0.1

Network Manager util to add ‘Nameserver 127.0.0.1’ to /etc/resolv.conf

References
  1. https://www.consul.io
  2. https://github.com/hashicorp/consul
  3. https://github.com/consul/consul/issues
  4. https://www.consul.io/docs/guides/consul-containers.html

TOP MCQs for Consul

Q.1 Consul supports 3rd party proxy integrations

       A. True

       B. False

Ans : True


Q.2 While configuring an Agent, using an external configuration file will override the environment variables defined to configure an agent.

       A. True

       B. False

Ans : False


Q.3 Connect feature is enabled by default and only needs to be configured in both the Servers and the Clients.

       A. True

       B. False

Ans : False


Q.4 Service Mesh is made up of

       A. Service Discovery

       B. Configuration

       C. Segmentation

       D. All the options

Ans : All the options


Q.5 Which command is used to query for the network round trip time between any two nodes?

       A. consul round_time

       B. consul ttr

       C. consul rtt

       D. consul rrt

Ans : consul rtt


Q.6 Applications can discover the services which they require using

       A. Either HTTP or DNS Interface

       B. HTTP Interface

       C. TCP Interface

       D. UDP interface

       E. DNS Interface

Ans : Either HTTP or DNS Interface


Q.7 With which community can Consul be linked the closest

       A. Full-Stack Community

       B. Web Development Community

       C. Back-end Development Community

       D. DevOps Community

Ans : DevOps Community


Q.8 How many possible states does a Raft node have?

       A. 3

       B. 1

       C. 2

       D. 4

       E. There is nothing such as state of Node

Ans : 3


Q.9 Which tag is used in the service configuartion optionally identify the service as a Connect proxy instance?

       A. kind

       B. proxy

   C. config

       D. type

Ans : kind


Q.10 Which command is used for DNS queries?

       A. agent

       B. join

       C. dig

       D. query_get

Ans : dig


Q.11 The Consul Intentions can be managed via

       A. CLI

       B. API

       C. UI

       D. All the options

Ans : All the options


Q.12 The resposibilitirs of the server include

       A. participating in the Raft quorum

       B. maintaining cluster state

       C. responding to RPC queries

       D. forwarding queries to leaders

       E. exchanging WAN gossip with other datacenters

       F. All the options

Ans : All the options


Q.13 To receive Telemetry data, what is the code that needs to be passed for Windows Platform.

       A. DATA

       B. BREAK

C. STDERR

       D. DUMP

Ans : BREAK


Q.14 The connect proxies can be registered via API only.

       A. True

       B. False

Ans : False


Q.15 Each datacenter has its own LAN Gossip pool

       A. True

       B. False

Ans : False

Q.16 Consul intentions are defined using static Firewall rules.

       A. True

       B. False

Ans : False


Q.17 All nodes within a Consul cluster must share the same encryption key to send and receive cluster information.

       A. True

       B. False

Ans : True


Q.18 Choose the correct Statement

A. A cluster receives new log entries inspite of having a leader

       B. A cluster with zero nodes receives log entries

       C. A cluster receives new log entries only when it has a leader

       D. A cluster never receives new log entries

Ans : A cluster receives new log entries only when it has a leader


Q.19 What does ACL stand for?

       A. Access Control List

       B. Anti-Entropy Control List

       C. Agent Control List

       D. API Control List

Ans : Access Control List

Q.20 The system continues to operate despite an arbitrary number of messages being dropped (or delayed) by the network between nodes.Which aspect of CAP theorem does this statement highlight?

       A. vailability

       B. Consistency

       C. Partition Tolerance

       D. CAP theorem is not concerned here

Ans : Partition Tolerance


Q.21 Choose the correct Statement

       A. Only the servers participate in the Gossip pool

       B. All Nodes in one Datacenter participate in the Gossip pool

       C. Only the clients participate in the Gossip pool

       D. The Agents across multiple datacenters participate in the Gossip pool

Ans : The Agents across multiple datacenters participate in the Gossip pool


Q.22 How many prevailing types of Gossip protocol are there?

       A. 2

       B. 3

       C. 4

       D. 5

Ans : 3


Q.23 Any member in the peer set can acknowledge log entries

       A. True

       B. False

Ans : False


Q.24 The gossip protocol uses symmetric key cryptosystem.

A. True

       B. False

Ans : True


Q.25 Stale Reads can be used to __________ . Complete the sentence

       A. Increase latency and increase the throughput of DNS queries

       B. Reduce latency and increase the throughput of DNS queries

       C. Reduce latency and Reduce the throughput of DNS queries

       D. Increase latency and reduce the throughput of DNS queries

Ans : Reduce latency and increase the throughput of DNS queries


Q.26 Datacenter has the following attributes

       A. high latency and high bandwidth

       B. high latency and low bandwidth

       C. low latency and low bandwidth

       D. low latency and high bandwidth

Ans : low latency and low bandwidth


Q.27 Choose the precedence of configuring an Agent.

       A. Configuration Files>Command Line Arguments>Environment Variables

       B. Command Line Arguments<Environment Variables<Configuration Files

       C. Command Line Arguments>Configuration Files>Environment Variables

       D. Command Line Arguments<Configuration Files<Environment Variables

Ans : Command Line Arguments>Configuration Files>Environment Variables


Q.28 Keeping Health checks in mind, choose the correct one

       A. Exit Code 0: Check is Failing Exit Code 1: Error Any other exit code: Check is passing

       B. Exit Code 0: Error Exit Code 1: Check is passing Any other exit code: Check is Failing

       C. Exit Code 0: Check is passing Exit Code 1: Check is Failing Any other exit code: Error

       D. Exit Code 0: Check is failing Exit Code 1: Check is passing Any other exit code: Error

Ans : Exit Code 0: Check is failing Exit Code 1: Check is passing Any other exit code: Error


Q.29 What is the format of the DNS query for Connect-capable Service Lookups?

       A. <service>.connect.<domain>

       B. <service>.node.<connec>.<domain>

       C. <node>.<service>.connect.<domain>

       D. <service>.<node>.connect.<domain<

Ans : <service>.connect.<domain>


Q.30 How many processor roles does PAXOS have?

       A. 3

       B. 4

       C. 5

       D. 6

Ans : 5


Q.31 How many types of Network Tomography are there?

       A. 2

       B. 3

       C. 4

       D. 5

Ans : 2


Q.32 Consul automatically removes all the dead Nodes and deregistered services.This process is called

       A. Autocleaning

       B. Dumping

       C. Reaping

       D. Cleaning

Ans : Reaping


Q.33 Which port is used by HTTP interface by default?

       A. 8080

       B. 8600

       C. 8500

       D. 8000

Ans : 8500


Q.34 How many types of RPS requests are there?

       A. 2

       B. 3

       C. 4

       D. 5

Ans : 2


Q.35 Each datacenter has its own WAN Gossip pool

       A. True

       B. False

Ans : True


Q.36 A session is used to build distributed locks.

       A. True

       B. False

Ans : True


Q.37 All Nodes are initially in the follower state.

       A. True

       B. False

Ans : True

Click here to read more bolgs of us

Click here to read more technology QA

Click here to read more technologies technical Q&A

About Author


After years of Technical Work, I feel like an expert when it comes to Develop wordpress website. Check out How to Create a Wordpress Website in 5 Mins, and Earn Money Online Follow me on Facebook for all the latest updates.

Leave a Comment