Enable Structured JSON Logging for Observability (SZ-6)
rk@tigase.net opened 2 hours ago

Enable Structured JSON Logging for Observability

Goal:
Configure structured, machine-readable logging in JSON format to support integration with observability tools (e.g., Prometheus, Grafana, Loki, ELK).

Estimated Time: 3–4 hours


Task Breakdown

1. Choose Logging Framework

  •  Use logback (default in Spring Boot) or switch to log4j2 if needed
  •  Evaluate compatibility with JSON encoders (e.g., logstash-logback-encoder)

2. Configure JSON Log Output

  •  Add dependency: net.logstash.logback:logstash-logback-encoder
  •  Define logback-spring.xml for JSON formatting
  •  Include fields like timestamp, level, logger, message, thread, and MDC context (e.g., userId, requestId)

3. Enable MDC Context Logging

  •  Use Spring’s HandlerInterceptor or OncePerRequestFilter to populate MDC with request metadata (e.g., userId, correlationId)

4. Profile-Based Logging Levels

  •  Configure verbose logging for dev, structured minimal logging for prod
  •  Add application-dev.yml and application-prod.yml overrides

5. Output Verification

  •  Write sample logs to console and/or file
  •  Ensure structured format using JSON linters
  •  Include log samples in docs or README

🕒 Estimated Time: 3–4 hours

Subtask Estimates

TaskEstimated Time
Add dependencies + configure JSON format~1 hour
Enable MDC and structured fields~1 hour
Add profile-based overrides~30 mins
Validate and sample logs~1 hour

Output Artifacts

  • logback-spring.xml with JSON encoder
  • Sample output logs (dev and prod)
  • MDC fields (e.g., correlationId, userId) in logs
  • Logging behavior described in README.md
  • rk@tigase.net commented 2 hours ago

    This will generate JSon-formatted logs using logstash-logback-encoder, which plugs into Logback (Spring Boot’s default logging framework). This enables logs to be machine-parsable JSON — perfect for tools like Loki, Elasticsearch, Prometheus (via exporters), or FluentBit. Example:

    {
      "@timestamp": "2025-10-02T13:45:00.123Z",
      "level": "INFO",
      "logger": "com.sztab.service.IssueService",
      "thread": "http-nio-8080-exec-3",
      "message": "Updated issue status to IN_PROGRESS",
      "context": "default",
      "mdc": {
        "userId": "rk@tigase.net",
        "requestId": "f3a9c8d0-4b5e-11ee-be56-0242ac120002",
        "projectId": "sztab-core"
      }
    }
    
  • rk@tigase.net commented 2 hours ago

    Required dependencies and configuration:

    <dependency>
      <groupId>net.logstash.logback</groupId>
      <artifactId>logstash-logback-encoder</artifactId>
      <version>7.4</version>
    </dependency>
    
  • rk@tigase.net commented 2 hours ago

    Custom logback-spring.xml config:

    <configuration>
      <include resource="org/springframework/boot/logging/logback/base.xml"/>
    
      <appender name="JSON_CONSOLE" class="net.logstash.logback.appender.LoggingEventCompositeJsonEncoder">
        <encoder class="net.logstash.logback.encoder.LoggingEventCompositeJsonEncoder">
          <providers>
            <timestamp>
              <fieldName>@timestamp</fieldName>
            </timestamp>
            <logLevel />
            <loggerName />
            <threadName />
            <message />
            <mdc />
          </providers>
        </encoder>
      </appender>
    
      <root level="INFO">
        <appender-ref ref="JSON_CONSOLE" />
      </root>
    </configuration>
    
issue 1 of 1
Type
New Feature
Priority
Normal
Assignee
Version
none
Sprints
n/a
Customer
n/a
Issue Votes (0)
Watchers (3)
Reference
SZ-6
Please wait...
Page is in error, reload to recover