README
“The
REFCODES.ORG
codes represent a group of artifacts consolidating parts of my work in the past years. Several topics are covered which I consider useful for you, programmers, developers and software engineers.”
Update [2018-06-09@10:41]: The RuntimeLogger
is now configured with a runtimelogger.*
file (where *
stands for the ini
, toml
, yaml
, json
, xml
or properties
suffixes) instead of the runtimelogger-config.xml
files used by previous versions of the refcodes-logger
artifact. This applies when using the RuntimeLoggerFactoryImpl
or the RuntimeLoggerFactorySingleton
factory to create your RuntimeLogger
instances. The article has been updated accordingly.
What is this repository for?
“… An introduction to the refcodes-logger framework; let’s start with giving your logs some color and tidy them up! Then let’s take over spring-boot’s log output …” [Logging like the nerds log, 04/04/2015]
The refcodes-logger
artifact provides the refcodes-logging
framework for flexible logging of any data to any data sink (including files, databases or the console provided as alternate implementations).
The refcodes-logger
artifact supports straight forward, composite (clustering) or partitioning functionality as implementations of the Logger
type. The REFCODES.ORG
runtime logger RuntimeLogger
integrates with SLF4J
seamlessly and also acts as an alternative to to SLF4J
.
The RuntimeLogger
implementations are being configured with a Logger
implementation. In your code, you use the RuntimeLogger
type which, depending on you configuring it, logs to a SimpleDB
cluster, the console (with ANSI
escape sequence support) or any I/O device as well as to an SLF4J
logger.
Being an alternative to SLF4J
, the RuntimeLogger
’s architecture settles upon the much more generic Logger
; which actually can be used to log high volume logs of any data type and not being restricted to runtime logs. The RuntimeLogger
implementations add functionality not found in other logging frameworks (logging out the class- and method-names of the logging methods without any configuration or additional lines of code)
How do I get set up?
To get up and running, include the following dependency (without the three dots “…”) in your pom.xml
:
1
2
3
4
5
6
7
8
9
<dependencies>
...
<dependency>
<artifactId>refcodes-logger</artifactId>
<groupId>org.refcodes</groupId>
<version>2.1.1</version>
</dependency>
...
</dependencies>
The artifact is hosted directly at Maven Central. Jump straight to the source codes at Bitbucket. Read the artifact’s javadoc at javadoc.io.
How do I get started?
You configure your RuntimeLoggerFactoryImpl
(and the RuntimeLoggerFactorySingleton
sub-class) by providing a runtimelogger.ini
file (also supported are *.toml
, *.yaml
, *.xml
, *.properties
or *.json
notations) in one of those locations relative to your main class’s location:
.
./config
./etc
./settings
./.config
./.settings
../config
../etc
../settings
../.config
../.settings
Given your main class’s JAR file resides in the folder /opt/app/lib
, then the valid locations for the runtimelogger.ini
file are:
/opt/app/lib
/opt/app/lib/config
/opt/app/lib/etc
/opt/app/lib/settings
/opt/app/lib/.config
/opt/app/lib/.settings
/opt/app/config
/opt/app/etc
/opt/app/settings
/opt/app/.config
/opt/app/.settings
(see also the class ConfigLocator.ALL
which is used to identify the configuration’s location)
In case you pass a JVM argument via -Dconfig.dir=path_to_your_config_dir
(where path_to_your_config_dir
stands for the path to the directory where you placed configuration files such as the runtimelogger.ini
file), then your path_to_your_config_dir
is placed first in the list of configuration directories to look at (in case the directory exists). See SystemProperty.CONFIG_DIR
as well as ConfigLocator.ALL
.
Use the JVM argument
-Dconfig.dir=path_to_your_config_dir
for your customruntimelogger.ini
configuration folder.
The runtimelogger.ini
configuration is deadly simple:
1
2
3
4
5
6
7
8
9
10
11
12
[root]
runtimelogger=org.refcodes.logger.RuntimeLoggerImpl
runtimelogger/logPriority=INFO
runtimelogger/logger=org.refcodes.logger.SystemLogger
[com.acme]
runtimelogger=org.refcodes.logger.RuntimeLoggerImpl
runtimelogger/logPriority=INFO
runtimelogger/name=com.acme
runtimelogger/logger=org.refcodes.logger.SystemLogger
The sections in squared braces (e.g. [com.acme]
) represent the Java package for which the afterwards configured RuntimeLogger
is responsible; e.g. a log issued from inside a class located in the package com.acme
(or in one of its sub-packages) will be handled by the com.acme
logger. In case no logger is found for a given package, then the root logger found below the [root]
section is used. Useful to know that a logger for a package namespace is only created once.
If you like logs colored nicely with ANSI escape codes, then you will love the ConsoleLoggerSingleton
:
1
2
3
4
5
[root]
runtimelogger=org.refcodes.logger.RuntimeLoggerImpl
runtimelogger/logPriority=INFO
runtimelogger/logger=org.refcodes.logger.alt.cli.ConsoleLoggerSingleton
Make sure to include the refcodces-logger-alt-console
dependency in your build setup to include the ConsoleLoggerSingleton
logger.
Supported notations
Below find the various supported notations by example. We configure a root
logger as well as a logger for the package com.acme
, both using the fancy ConsoleLoggerSingleton
logger.
Please note that some notations use the
this
keyword to denote the value of the enclosing element as some notations do not support (or make it hard) to assign a value to an element which has child elements.
TOML or INI
Using TOML
or INI
based runtimelogger.toml
or runtimelogger.ini
properties:
1
2
3
4
5
6
7
8
9
10
11
[root]
runtimelogger=org.refcodes.logger.RuntimeLoggerImpl
runtimelogger/logPriority=INFO
runtimelogger/logger=org.refcodes.logger.alt.cli.ConsoleLoggerSingleton
[com.acme]
runtimelogger=org.refcodes.logger.RuntimeLoggerImpl
runtimelogger/logPriority=ERROR
runtimelogger/logger=org.refcodes.logger.alt.cli.ConsoleLoggerSingleton
An advanced example setting the row width
, disabling ANSI escape codes
and setting an ASCII
table style looks as follows :
1
2
3
4
5
6
7
8
9
10
11
12
13
14
[root]
runtimelogger=org.refcodes.logger.RuntimeLoggerImpl
runtimelogger/logPriority=INFO
runtimelogger/logger=org.refcodes.logger.alt.cli.ConsoleLoggerSingleton
runtimelogger/logger/rowWidth=120
runtimelogger/logger/escapeCodes=false
runtimelogger/logger/tableStyle=ASCII
[com.acme]
runtimelogger=org.refcodes.logger.RuntimeLoggerImpl
runtimelogger/logPriority=ERROR
runtimelogger/logger=org.refcodes.logger.alt.cli.ConsoleLoggerSingleton
Java properties
Using Java
based runtimelogger.properties
properties:
1
2
3
4
5
6
7
root/runtimelogger=org.refcodes.logger.RuntimeLoggerImpl
root/runtimelogger/logPriority=INFO
root/runtimelogger/logger=org.refcodes.logger.alt.cli.ConsoleLoggerSingleton
com/acme/runtimelogger=org.refcodes.logger.RuntimeLoggerImpl
com/acme/runtimelogger/logPriority=ERROR
com/acme/runtimelogger/logger=org.refcodes.logger.alt.cli.ConsoleLoggerSingleton
An advanced example setting the row width
, disabling ANSI escape codes
and setting an ASCII
table style looks as follows :
1
2
3
4
5
6
7
8
9
10
root/runtimelogger=org.refcodes.logger.RuntimeLoggerImpl
root/runtimelogger/logPriority=INFO
root/runtimelogger/logger=org.refcodes.logger.alt.cli.ConsoleLoggerSingleton
root/runtimelogger/logger/rowWidth=120
root/runtimelogger/logger/escapeCodes=false
root/runtimelogger/logger/tableStyle=ASCII
com/acme/runtimelogger=org.refcodes.logger.RuntimeLoggerImpl
com/acme/runtimelogger/logPriority=ERROR
com/acme/runtimelogger/logger=org.refcodes.logger.alt.cli.ConsoleLoggerSingleton
XML
Using XML
based runtimelogger.xml
properties:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
<config>
<root>
<runtimelogger this="org.refcodes.logger.RuntimeLoggerImpl">
<logPriority>INFO</logPriority>
<logger this="org.refcodes.logger.alt.cli.ConsoleLoggerSingleton" />
</runtimelogger>
</root>
<com>
<acme>
<runtimelogger this="org.refcodes.logger.RuntimeLoggerImpl">
<logPriority>ERROR</logPriority>
<logger this="org.refcodes.logger.alt.cli.ConsoleLoggerSingleton" />
</runtimelogger>
</acme>
</com>
</config>
Please note that this notation uses the
this
attribute to denote the value of the enclosing element as this notations makes it hard to assign a value viamixed content
to an element which has child elements.
An advanced example setting the row width
, disabling ANSI escape codes
and setting an ASCII
table style looks as follows :
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
<config>
<root>
<runtimelogger this="org.refcodes.logger.RuntimeLoggerImpl">
<logPriority>INFO</logPriority>
<logger this="org.refcodes.logger.alt.cli.ConsoleLoggerSingleton" />
<logger>
<rowWidth>120</rowWidth>
<escapeCodes>false</escapeCodes>
<tableStyle>ASCII</tableStyle>
</logger>
</runtimelogger>
</root>
<com>
<acme>
<runtimelogger this="org.refcodes.logger.RuntimeLoggerImpl">
<logPriority>ERROR</logPriority>
<logger this="org.refcodes.logger.alt.cli.ConsoleLoggerSingleton" />
</runtimelogger>
</acme>
</com>
</config>
YAML
Using YAML
based runtimelogger.yaml
properties:
1
2
3
4
5
6
7
8
9
10
11
12
root:
runtimelogger:
this: org.refcodes.logger.RuntimeLoggerImpl
logPriority: INFO
logger: org.refcodes.logger.alt.cli.ConsoleLoggerSingleton
com:
acme:
runtimelogger:
this: org.refcodes.logger.RuntimeLoggerImpl
logPriority: ERROR
logger: org.refcodes.logger.alt.cli.ConsoleLoggerSingleton
Please note that this notations uses the
this
keyword to denote the value of the enclosing element as this notations do not support to assign a value to an element which has child elements.
An advanced example setting the row width
, disabling ANSI escape codes
and setting an ASCII
table style looks as follows :
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
root:
runtimelogger:
this: org.refcodes.logger.RuntimeLoggerImpl
logPriority: INFO
logger:
this: org.refcodes.logger.alt.cli.ConsoleLoggerSingleton
rowWidth: 120
escapeCodes: false
tableStyle: ASCII
com:
acme:
runtimelogger:
this: org.refcodes.logger.RuntimeLoggerImpl
logPriority: ERROR
logger: org.refcodes.logger.alt.cli.ConsoleLoggerSingleton
JSON
Using JSON
based properties runtimelogger.json
properties:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
{
"root": {
"runtimelogger": {
"this": "org.refcodes.logger.RuntimeLoggerImpl",
"logPriority": "INFO",
"logger": "org.refcodes.logger.alt.cli.ConsoleLoggerSingleton"
}
},
"com": {
"acme": {
"runtimelogger": {
"this": "org.refcodes.logger.RuntimeLoggerImpl",
"logPriority": "ERROR",
"logger": "org.refcodes.logger.alt.cli.ConsoleLoggerSingleton"
}
}
}
}
Please note that this notations uses the
this
keyword to denote the value of the enclosing element as this notations do not support to assign a value to an element which has child elements.
An advanced example setting the row width
, disabling ANSI escape codes
and setting an ASCII
table style looks as follows :
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
{
"root": {
"runtimelogger": {
"this": "org.refcodes.logger.RuntimeLoggerImpl",
"logPriority": "INFO",
"logger": {
"this": "org.refcodes.logger.alt.cli.ConsoleLoggerSingleton",
"rowWidth": "120",
"escapeCodes": "false",
"tableStyle": "ASCII"
}
}
},
"com": {
"acme": {
"runtimelogger": {
"this": "org.refcodes.logger.RuntimeLoggerImpl",
"logPriority": "ERROR",
"logger": "org.refcodes.logger.alt.cli.ConsoleLoggerSingleton"
}
}
}
}
Runtime logging
Using the refcodes-logging
framework in your code to do runtime logging (such as LOGGER.debug("Starting ...")
or LOGGER.error("Earth control, we have a problem ...")
is also deadly simple. Just declare a static member variable to your class where you want to do your logging:
private static RuntimeLogger LOGGER = RuntimeLoggerFactorySingleton.getInstance().createInstance();
The usage is very similar to a logger such as Log4j
or SLF4J
:
1
2
3
LOGGER.info( "Starting the application ..." );
LOGGER.error( "Failed to read file as of an IO exception!", e );
LOGGER.trace( "Start time is {0}, duration is {1} ms, end time is {2}", theStartTime, theDurationMs, theEndTime );
By default, the refcodes-logger
logs out the additional following information:
- The overall
line number
of the current logged line - The
class
and themethod
from which you produced a log line
Console logging
The artifact refcodes-logger-alt-console
provides a good starting point playing around with the refcodes-logger
tool-box. With the refcodes-logger-alt-console
you can print out colorful console logs into your terminal. It auto-detects the features of your terminal such as the number of chars per row, whether you use a shell
on a Unix
alike operating system or CMD.EXE
on Windows
and determines whether your terminal supports coloring of the output (ANSI
escape codes).
Add the following dependency to your pom.xml
:
1
2
3
4
5
6
7
8
9
<dependencies>
...
<dependency>
<artifactId>refcodes-logger-alt-console</artifactId>
<groupId>org.refcodes</groupId>
<version>2.1.1</version>
</dependency>
...
</dependencies>
It transitively pulls the refcodes-logger
artifact. Then create a runtimelogger.ini
similar to the one above:
1
2
3
4
5
6
7
8
9
10
11
12
13
[root]
runtimelogger=org.refcodes.logger.RuntimeLoggerImpl
runtimelogger/logPriority=INFO
runtimelogger/logger=org.refcodes.logger.alt.cli.ConsoleLoggerSingleton
runtimelogger/logger/tableStyle=ASCII
[com.acme]
runtimelogger=org.refcodes.logger.RuntimeLoggerImpl
runtimelogger/logPriority=DEBUG
runtimelogger/name=com.acme
runtimelogger/logger=org.refcodes.logger.alt.cli.ConsoleLoggerSingleton
There are some attributes which can be (optionally) configured when declaring the ConsoleLoggerSingleton
in your runtimelogger.ini
:
- tableStyle: Valid values for the
tableStyle
attribute are as of theTableStyle
enumeration: - SINGLE
- DOUBLE
- DOUBLE_SINGLE
- SINGLE_DOUBLE
- SINGLE_DASHED
- ASCII
- BLANK
- SINGLE_BLANK
-
ASCII_BLANK
- rowWidth: In case the row width cannot be determined automatically, you can either set the system property
-Dconsole.width=n
(see below) or you can provide the number of chars you desire per row with therowWidth
attribute, e.g.rowWidth=160
.
The ConsoleLoggerImpl
(and in turn as being its sub-class the ConsoleLoggerSingleton
) by default uses the SystemUtility.getConsoleWidth()
method, which determines the width in characters of the system’s console in use. In case you pass a -Dconsole.width=n
JVM argument (where n
stands for the number of chars you desire for your console row width), then your width is taken, else the actual console’s width is being tried to be determined. See SystemConsts.SYS_PROP_CONSOLE_WIDTH
.
A fancy output could look as follows:
The output as a console copy’n’paste dump (note that the tilde character
~
is used to truncate long lines; all aboveWARNING
is not truncated):
steiner@proteus:~/Workspaces/org.refcodes/refcodes-runtime-ext/refcodes-runtime-ext-console$ java -jar target/refcodes-runtime-ext-console-0.1.1-SNAPSHOT.jar ───────┬───────────────────┬───────┬───────────────┬──────────────────────────────┬──────────────────────┬──────────────────────────────────────────────────── 0000001│2015-04-04T11:21:19│INFO │main │~.console.SystemInfoConsoleApp│printInfo() │~12 12:19:35 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux 0000002│2015-04-04T11:21:19│INFO │main │~.console.SystemInfoConsoleApp│printInfo() │Console width := 158 0000003│2015-04-04T11:21:19│INFO │main │~.console.SystemInfoConsoleApp│printInfo() │Console height := 24 0000004│2015-04-04T11:21:19│INFO │main │~.console.SystemInfoConsoleApp│printInfo() │ANSI support := true 0000005│2015-04-04T11:21:19│INFO │main │~.console.SystemInfoConsoleApp│printInfo() │Operating system := UNIX 0000006│2015-04-04T11:21:19│INFO │main │~.console.SystemInfoConsoleApp│printInfo() │CLI := SHELL 0000007│2015-04-04T11:21:19│INFO │main │~.console.SystemInfoConsoleApp│printInfo() │Encoding := UTF8 0000008│2015-04-04T11:21:19│INFO │main │~.console.SystemInfoConsoleApp│printInfo() │Process (PID) := 13082 0000009│2015-04-04T11:21:19│INFO │main │~.console.SystemInfoConsoleApp│printInfo() │~rg.refodes.runtime.ext.console.SystemInfoConsoleApp 0000010│2015-04-04T11:21:19│INFO │main │~.console.SystemInfoConsoleApp│printInfo() │stty size := 0000011│2015-04-04T11:21:19│INFO │main │~.console.SystemInfoConsoleApp│printInfo() │tput cols := 80 0000012│2015-04-04T11:21:19│INFO │main │~.console.SystemInfoConsoleApp│printInfo() │tput lines := 24 steiner@proteus:~/Workspaces/org.refcodes/refcodes-runtime-ext/refcodes-runtime-ext-console$
SLF4J support
Instead of logging to the console via refcodes-logger-alt-console
, you can use the existing SLF4J
binding refcodes-logger-alt-slf4j
and make your refcodes-logger
log via SLF4J
just by including the below dependency (instead of the above one) in your pom.xml
:
1
2
3
4
5
6
7
8
9
<dependencies>
...
<dependency>
<artifactId>refcodes-logger-alt-slf4j</artifactId>
<groupId>org.refcodes</groupId>
<version>2.1.1</version>
</dependency>
...
</dependencies>
This will cause your …
private static RuntimeLogger LOGGER = RuntimeLoggerFactorySingleton.getInstance().createInstance();
… to do logging with your SLF4J
binding, which could be a Log4J
binding.
SLF4J binding
Via the refcodes-logger-ext-slf4j
artifact, you can use the refcodes-logger
framework itself as an SLF4J
binding - in order to make, for example, your Tomcat
or spring-boot
log to the fancy refcodes-logger-alt-console
or a NoSQL
DB cluster (as of refcodes-logger-alt-simpledb
):
1
2
3
4
5
6
7
8
9
<dependencies>
...
<dependency>
<artifactId>refcodes-logger-ext-slf4j</artifactId>
<groupId>org.refcodes</groupId>
<version>2.1.1</version>
</dependency>
...
</dependencies>
In addition you have to add a dependency of one of the refcodes-logger-alt-*
artifacts to your pom.xml
and an according runtimelogger.ini
pulling that logger implementation you want your SLF4J
logs to log to …
Attention: Never use the
SLF4J
bindingrefcodes-logger-ext-slf4j
together with therefcodes-logger-alt-slf4j
logger implementation, else the one will look for the other, the other for the one … until you get anStackOverflowException
:D
With the SLF4J
binding, you can log all your SLF4J
logs (also the ones of the libs you included in your app) to any of the refcodes-logger-alt-*
implementations:
Make your
microservices
log into aNoSQL
database cluster with therefcodes-logger-alt-simpledb
artifact providing support for Amazon’sSimpleDB
.
Scalability
Just some keywords on scalability: The refcodes-logger
framework implements the composite pattern
and various partitioning
and composition
strategies for high scalability and therewith high throughput.
-
The
CompositeLoggerImpl
is an asynchronous logger with a log lines queue. Depending on the availability of the encapsulated loggers, the one or the other encapsulated logger takes the next waiting log line in the queue and processes it. The encapsulatedrefcodes-logger
s can be any instance implementing theLogger
interface (e.g. one of the loggers found in therefcodes-logger
artifact and therefcodes-logger-alt-*
artifacts). -
The
PartedLoggerImpl
implements partitioning functionality. You define a partitioning criteria for your log lines, e.g. a tenant’s ID (or the date or in case of therefcodes-logger
a package, class name or method) and theparted-logger
takes care to address the encapsulated responsiblelogger
(which in turn can be anotherparted-logger
or acomposite-logger
or any otherrefcodes-logger
). The encapsulatedrefcodes-logger
s can be any instance implementing theLogger
interface (e.g. one of the loggers found in therefcodes-logger
artifact and therefcodes-logger-alt-*
artifacts).
Querying
As the data sink to which a refcodes-logger
writes its output can be write only (e.g. a terminal window) or write, read and delete (e.g. a NoSQL
database such as the SimpleDB
logger), there are three flavors of the Logger
interface. For example the composite-logger
and the parted-logger
provides them three flavors:
Logger
: Write only logging, just like you know it fromLog4J
orSLF4J
…QueryLogger
: Use a logical querying language (as of therefcodes-criteria
artifact) to retrieve logs from yourrefcodes-logger
logging setup.TrimLogger
: Use a logical querying language (as of therefcodes-criteria
artifact) to trim (delete) logs from yourrefcodes-logger
data sinks; most important when doing big-data processing to clean up afterwards …- The
CompositeLoggerImpl
is an asynchronous logger with a log lines queue. Depending on the availability of the encapsulated loggers, the one or the other encapsulated logger takes the next waiting log line in the queue and processes it. The encapsulatedrefcodes-logger
s can be any instance implementing theLogger
interface (e.g. one of the loggers found in therefcodes-logger
artifact and therefcodes-logger-alt-*
artifacts). - The
PartedLoggerImpl
implements partitioning functionality. You define a partitioning criteria for your log lines, e.g. a tenant’s ID (or the date or in case of therefcodes-logger
a package, class name or method) and theparted-logger
takes care to address the encapsulated responsiblelogger
(which in turn can be anotherparted-logger
or acomposite-logger
or any otherrefcodes-logger
). The encapsulatedrefcodes-logger
s can be any instance implementing theLogger
interface (e.g. one of the loggers found in therefcodes-logger
artifact and therefcodes-logger-alt-*
artifacts).
Querying
As the data sink to which a refcodes-logger
writes its output can be write only (e.g. a terminal window) or write, read and delete (e.g. a NoSQL
database such as the SimpleDB
logger), there are three flavors of the Logger
interface. For example the composite-logger
and the parted-logger
provides them three flavors:
Logger
: Write only logging, just like you know it fromLog4J
orSLF4J
…QueryLogger
: Use a logical querying language (as of therefcodes-criteria
artifact) to retrieve logs from yourrefcodes-logger
logging setup.TrimLogger
: Use a logical querying language (as of therefcodes-criteria
artifact) to trim (delete) logs from yourrefcodes-logger
data sinks; most important when doing big-data processing to clean up afterwards …
Contribution guidelines
- Writing tests and logger implementations for other data sinks.
- Implement an observable implementation of the log line queue based Logger implementations (e.g. in a
refcodes-logger-ext
multi module project. - Extending the
refcodes-logger
to contain a correlation ID (useful when working in distributed cloud environments or with microservices) - Code review
Who do I talk to?
- Siegfried Steiner (steiner@refcodes.org)
Terms and conditions
The REFCODES.ORG
group of artifacts is published under some open source licenses; covered by the refcodes-licensing
(org.refcodes
group) artifact - evident in each artifact in question as of the pom.xml
dependency included in such artifact.