Observability in Microservices Built with Express.js

Introduction

In the world of modern software development, especially when building microservices architectures, ensuring observability is crucial. Observability refers to the ability to understand what's happening inside a system by collecting and analyzing data from various sources. This tutorial will guide you through setting up observability for your microservices built with Express.js.

Concept

Observability in microservices involves monitoring metrics, logs, and traces. Metrics help you measure the performance of your services, logs provide insights into what's happening at runtime, and traces allow you to understand the flow of requests across different services.

For this tutorial, we'll focus on setting up basic observability using popular tools like Prometheus for metrics, ELK Stack (Elasticsearch, Logstash, Kibana) for logs, and Jaeger for distributed tracing. We'll also integrate these with Express.js applications.

Examples

1. Setting Up Metrics with Prometheus

Prometheus is a powerful open-source monitoring system that collects metrics from your application.

Step 1: Install Dependencies

First, install the necessary dependencies:

You can now access the metrics endpoint at http://localhost:3000/metrics.

2. Setting Up Logging with ELK Stack

The ELK Stack is a popular choice for centralized logging.

Step 1: Install Dependencies

Install the necessary dependencies:

Step 2: Configure Jaeger Client

Create a file named tracer.js and add the following code:

1const initTracer = require('jaeger-client');
2const { FORMAT_HTTP_HEADERS } = require('opentracing');
3 
4// Configuration for Jaeger
5const config = {
6serviceName: 'express-service',
7sampler: {
8  type: 'const',
9  param: 1,
10},
11reporter: {
12  logSpans: true,
13},
14};
15 
16const options = {
17tags: {
18  [initTracer.TRACER_TAGS.COMPONENT]: 'my-express-app',
19},
20};
21 
22// Initialize the tracer
23const tracer = initTracer(config, options);
24 
25module.exports = tracer;

Step 3: Integrate with Your Express App

In your main app.js file, integrate the Jaeger middleware:

1const express = require('express');
2const metricsApp = require('./metrics');
3const logger = require('./logger');
4const tracer = require('./tracer');
5const { TracerMiddleware } = require('express-jaeger-middleware');
6 
7const app = express();
8 
9// Use the Jaeger middleware
10app.use(TracerMiddleware({
11tracer,
12serviceName: 'express-service',
13}));
14 
15// Use the metrics app
16app.use(metricsApp);
17 
18// Define a simple route
19app.get('/', (req, res) => {
20const span = tracer.startSpan('handle-request');
21logger.info('Handling request', { method: req.method, url: req.url });
22span.finish();
23res.send('Hello World!');
24});
25 
26// Start the server
27const PORT = process.env.PORT || 3000;
28app.listen(PORT, () => {
29console.log(`Server is running on port ${PORT}`);
30});

What's Next?

Now that you have set up basic observability for your Express.js microservices, you can explore more advanced topics such as:

• Advanced Testing Techniques for Express.js
• Integrating with other monitoring and logging tools like Datadog or Splunk
• Implementing distributed tracing across multiple services

By following this tutorial, you should have a solid foundation for ensuring observability in your microservices built with Express.js.