README
The
REFCODES.ORG
codes represent a group of artifacts consolidating parts of my work in the past years. Several topics are covered which I consider useful for you, programmers, developers and software engineers.
What is this repository for?
This artifact provides flexible logging of any data to any data sink (including files, databases or the console). It supports straight forward, composite (clustering) or partitioning functionality provided by different implementations of the Logger
type. The RuntimeLogger
type harnesses the Logger
type for logging your runtime log messages and integrates with SLF4J
seamlessly (and may also act as an alternative data sink to log to when using SLF4J
.
How do I get set up?
To get up and running, include the following dependency (without the three dots “…”) in your pom.xml
:
1
2
3
4
5
6
7
8
9
<dependencies>
...
<dependency>
<artifactId>refcodes-logger</artifactId>
<groupId>org.refcodes</groupId>
<version>3.3.8</version>
</dependency>
...
</dependencies>
The artifact is hosted directly at Maven Central. Jump straight to the source codes at Bitbucket. Read the artifact’s javadoc at javadoc.io.
Introduction
The RuntimeLogger
implementations are being configured with a Logger
implementation. You may use the RuntimeLogger
type, which, depending on you configuring it, logs to a SimpleDB
cluster, the console (with ANSI
escape sequence support) or any I/O device or to an SLF4J
logger. Being an alternative data sink for SLF4J
, the RuntimeLogger
type’s architecture settles upon a generic Logger
type, which actually can be used to log high volume logs of any data type and not being restricted to runtime logs. Furthermore, the RuntimeLogger
type adds functionality not found in other logging frameworks such as logging out the class- and method-names of the logging source without any configuration or additional lines of code.
See also Logging like the nerds log for a quick start on how to do fancy console logging of your application’s runtime logs!
How do I get started?
You configure your RuntimeLoggerFactoryImpl
(and the RuntimeLoggerFactorySingleton
sub-class) by providing a runtimelogger.ini
file (also supported are *.toml
, *.yaml
, *.xml
, *.properties
or *.json
notations) in one of those locations relative to your main class’s location:
.
./config
./etc
./settings
./.config
./.settings
../config
../etc
../settings
../.config
../.settings
Given your main class’s JAR file resides in the folder /opt/app/lib
, then the valid locations for the runtimelogger.ini
file are:
/opt/app/lib
/opt/app/lib/config
/opt/app/lib/etc
/opt/app/lib/settings
/opt/app/lib/.config
/opt/app/lib/.settings
/opt/app/config
/opt/app/etc
/opt/app/settings
/opt/app/.config
/opt/app/.settings
(see also the class ConfigLocator.ALL
which is used to identify the configuration’s location)
In case you pass a JVM argument via -Dconfig.dir=path_to_your_config_dir
(where path_to_your_config_dir
stands for the path to the directory where you placed configuration files such as the runtimelogger.ini
file), then your path_to_your_config_dir
is placed first in the list of configuration directories to look at (in case the directory exists). See SystemProperty.CONFIG_DIR
as well as ConfigLocator.ALL
.
Use the JVM argument
-Dconfig.dir=path_to_your_config_dir
for your customruntimelogger.ini
configuration folder.
The runtimelogger.ini
configuration is deadly simple:
1
2
3
4
5
6
7
8
9
10
11
12
[root]
runtimelogger=org.refcodes.logger.RuntimeLoggerImpl
runtimelogger/logPriority=INFO
runtimelogger/logger=org.refcodes.logger.SystemLogger
[com.acme]
runtimelogger=org.refcodes.logger.RuntimeLoggerImpl
runtimelogger/logPriority=INFO
runtimelogger/name=com.acme
runtimelogger/logger=org.refcodes.logger.SystemLogger
The sections in squared braces (e.g. [com.acme]
) represent the Java package for which the afterwards configured RuntimeLogger
is responsible; e.g. a log issued from inside a class located in the package com.acme
(or in one of its sub-packages) will be handled by the com.acme
logger. In case no logger is found for a given package, then the root logger found below the [root]
section is used. Useful to know that a logger for a package namespace is only created once.
If you like logs colored nicely with ANSI escape codes, then you will love the ConsoleLoggerSingleton
:
1
2
3
4
5
[root]
runtimelogger=org.refcodes.logger.RuntimeLoggerImpl
runtimelogger/logPriority=INFO
runtimelogger/logger=org.refcodes.logger.alt.console.ConsoleLoggerSingleton
Make sure to include the refcodces-logger-alt-console
dependency in your build setup to include the ConsoleLoggerSingleton
logger.
Supported notations
Below find the various supported notations by example. We configure a root
logger as well as a logger for the package com.acme
, both using the fancy ConsoleLoggerSingleton
logger.
Please note that some notations use the
this
keyword to denote the value of the enclosing element as some notations do not support (or make it hard) to assign a value to an element which has child elements.
TOML or INI
Using TOML
or INI
based runtimelogger.toml
or runtimelogger.ini
properties:
1
2
3
4
5
6
7
8
9
10
11
[root]
runtimelogger=org.refcodes.logger.RuntimeLoggerImpl
runtimelogger/logPriority=INFO
runtimelogger/logger=org.refcodes.logger.alt.console.ConsoleLoggerSingleton
[com.acme]
runtimelogger=org.refcodes.logger.RuntimeLoggerImpl
runtimelogger/logPriority=ERROR
runtimelogger/logger=org.refcodes.logger.alt.console.ConsoleLoggerSingleton
An advanced example setting the row width
, disabling ANSI escape codes
and setting an ASCII
table style looks as follows :
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
[root]
runtimelogger=org.refcodes.logger.RuntimeLoggerImpl
runtimelogger/logPriority=INFO
runtimelogger/logger=org.refcodes.logger.alt.console.ConsoleLoggerSingleton
runtimelogger/logger/rowWidth=120
runtimelogger/logger/escapeCodes=false
runtimelogger/logger/style=ASCII
runtimelogger/logger/layout=HACKER
[com.acme]
runtimelogger=org.refcodes.logger.RuntimeLoggerImpl
runtimelogger/logPriority=ERROR
runtimelogger/logger=org.refcodes.logger.alt.console.ConsoleLoggerSingleton
Java properties
Using Java based runtimelogger.properties
properties:
1
2
3
4
5
6
7
root/runtimelogger=org.refcodes.logger.RuntimeLoggerImpl
root/runtimelogger/logPriority=INFO
root/runtimelogger/logger=org.refcodes.logger.alt.console.ConsoleLoggerSingleton
com/acme/runtimelogger=org.refcodes.logger.RuntimeLoggerImpl
com/acme/runtimelogger/logPriority=ERROR
com/acme/runtimelogger/logger=org.refcodes.logger.alt.console.ConsoleLoggerSingleton
An advanced example setting the row width
, disabling ANSI escape codes
and setting an ASCII
table style looks as follows :
1
2
3
4
5
6
7
8
9
10
11
root/runtimelogger=org.refcodes.logger.RuntimeLoggerImpl
root/runtimelogger/logPriority=INFO
root/runtimelogger/logger=org.refcodes.logger.alt.console.ConsoleLoggerSingleton
root/runtimelogger/logger/rowWidth=120
root/runtimelogger/logger/escapeCodes=false
root/runtimelogger/logger/style=ASCII
root/runtimelogger/logger/layout=HACKER
com/acme/runtimelogger=org.refcodes.logger.RuntimeLoggerImpl
com/acme/runtimelogger/logPriority=ERROR
com/acme/runtimelogger/logger=org.refcodes.logger.alt.console.ConsoleLoggerSingleton
XML
Using XML
based runtimelogger.xml
properties:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
<config>
<root>
<runtimelogger this="org.refcodes.logger.RuntimeLoggerImpl">
<logPriority>INFO</logPriority>
<logger this="org.refcodes.logger.alt.console.ConsoleLoggerSingleton" />
</runtimelogger>
</root>
<com>
<acme>
<runtimelogger this="org.refcodes.logger.RuntimeLoggerImpl">
<logPriority>ERROR</logPriority>
<logger this="org.refcodes.logger.alt.console.ConsoleLoggerSingleton" />
</runtimelogger>
</acme>
</com>
</config>
Please note that this notation uses the
this
attribute to denote the value of the enclosing element as this notations makes it hard to assign a value viamixed content
to an element which has child elements.
An advanced example setting the row width
, disabling ANSI escape codes
and setting an ASCII
table style looks as follows :
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
<config>
<root>
<runtimelogger this="org.refcodes.logger.RuntimeLoggerImpl">
<logPriority>INFO</logPriority>
<logger this="org.refcodes.logger.alt.console.ConsoleLoggerSingleton" />
<logger>
<rowWidth>120</rowWidth>
<escapeCodes>false</escapeCodes>
<style>ASCII</tableStyle>
<layout>HACKER</layout>
</logger>
</runtimelogger>
</root>
<com>
<acme>
<runtimelogger this="org.refcodes.logger.RuntimeLoggerImpl">
<logPriority>ERROR</logPriority>
<logger this="org.refcodes.logger.alt.console.ConsoleLoggerSingleton" />
</runtimelogger>
</acme>
</com>
</config>
YAML
Using YAML
based runtimelogger.yaml
properties:
1
2
3
4
5
6
7
8
9
10
11
12
root:
runtimelogger:
this: org.refcodes.logger.RuntimeLoggerImpl
logPriority: INFO
logger: org.refcodes.logger.alt.console.ConsoleLoggerSingleton
com:
acme:
runtimelogger:
this: org.refcodes.logger.RuntimeLoggerImpl
logPriority: ERROR
logger: org.refcodes.logger.alt.console.ConsoleLoggerSingleton
Please note that this notations uses the
this
keyword to denote the value of the enclosing element as this notations do not support to assign a value to an element which has child elements.
An advanced example setting the row width
, disabling ANSI escape codes
and setting an ASCII
table style looks as follows :
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
root:
runtimelogger:
this: org.refcodes.logger.RuntimeLoggerImpl
logPriority: INFO
logger:
this: org.refcodes.logger.alt.console.ConsoleLoggerSingleton
rowWidth: 120
escapeCodes: false
style: ASCII
layout: HACKER
com:
acme:
runtimelogger:
this: org.refcodes.logger.RuntimeLoggerImpl
logPriority: ERROR
logger: org.refcodes.logger.alt.console.ConsoleLoggerSingleton
JSON
Using JSON
based properties runtimelogger.json
properties:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
{
"root": {
"runtimelogger": {
"this": "org.refcodes.logger.RuntimeLoggerImpl",
"logPriority": "INFO",
"logger": "org.refcodes.logger.alt.console.ConsoleLoggerSingleton"
}
},
"com": {
"acme": {
"runtimelogger": {
"this": "org.refcodes.logger.RuntimeLoggerImpl",
"logPriority": "ERROR",
"logger": "org.refcodes.logger.alt.console.ConsoleLoggerSingleton"
}
}
}
}
Please note that this notations uses the
this
keyword to denote the value of the enclosing element as this notations do not support to assign a value to an element which has child elements.
An advanced example setting the row width
, disabling ANSI escape codes
and setting an ASCII
table style looks as follows :
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
{
"root": {
"runtimelogger": {
"this": "org.refcodes.logger.RuntimeLoggerImpl",
"logPriority": "INFO",
"logger": {
"this": "org.refcodes.logger.alt.console.ConsoleLoggerSingleton",
"rowWidth": "120",
"escapeCodes": "false",
"style": "ASCII"
"layout": "HACKER"
}
}
},
"com": {
"acme": {
"runtimelogger": {
"this": "org.refcodes.logger.RuntimeLoggerImpl",
"logPriority": "ERROR",
"logger": "org.refcodes.logger.alt.console.ConsoleLoggerSingleton"
}
}
}
}
Snippets of interest
Below find some code snippets which demonstrate the various aspects of using the refcodes-logger
artifact (and , if applicable, its offsprings). See also the example source codes of this artifact for further information on the usage of this artifact.
Runtime logging
Using the refcodes-logging
framework in your code to do runtime logging (such as LOGGER.debug("Starting ...")
or LOGGER.error("Earth control, we have a problem ...")
is also deadly simple. Just declare a static member variable to your class where you want to do your logging:
The usage is very similar to a logger such as Log4j
or SLF4J
:
1
2
3
LOGGER.info( "Starting the application ..." );
LOGGER.error( "Failed to read file as of an IO exception!", e );
LOGGER.trace( "Start time is {0}, duration is {1} ms, end time is {2}", theStartTime, theDurationMs, theEndTime );
By default, the refcodes-logger
logs out the additional following information:
- The overall
line number
of the current logged line - The
class
and themethod
from which you produced a log line
Console logging
The artifact refcodes-logger-alt-console
provides a good starting point playing around with the refcodes-logger
tool-box. With the refcodes-logger-alt-console
you can print out colorful console logs into your terminal. It auto-detects the features of your terminal such as the number of chars per row, whether you use a shell
on a Unix
alike operating system or CMD.EXE
on Windows
and determines whether your terminal supports coloring of the output (ANSI
escape codes).
Add the following dependency to your pom.xml
:
1
2
3
4
5
6
7
8
9
<dependencies>
...
<dependency>
<artifactId>refcodes-logger-alt-console</artifactId>
<groupId>org.refcodes</groupId>
<version>3.3.8</version>
</dependency>
...
</dependencies>
It transitively pulls the refcodes-logger
artifact. Then create a runtimelogger.ini
similar to the one above:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
[root]
runtimelogger=org.refcodes.logger.RuntimeLoggerImpl
runtimelogger/logPriority=INFO
runtimelogger/logger=org.refcodes.logger.alt.console.ConsoleLoggerSingleton
runtimelogger/logger/style=ASCII
runtimelogger/logger/layout=HACKER
[com.acme]
runtimelogger=org.refcodes.logger.RuntimeLoggerImpl
runtimelogger/logPriority=DEBUG
runtimelogger/name=com.acme
runtimelogger/logger=org.refcodes.logger.alt.console.ConsoleLoggerSingleton
There are some attributes which can be (optionally) configured when declaring the ConsoleLoggerSingleton
in your runtimelogger.ini
:
style
: The table’s style, values as of theTableStyle
enumeration:- ASCII
- ASCII_BLANK_HEADER_ASCII_BLANK_BODY
- ASCII_HEADER_ASCII_BODY
- BLANK
- BLANK_HEADER_BLANK_BODY
- BOLD
- BOLD_HEADER_BOLD_BODY
- BOLD_HEADER_SINGLE_BODY
- DOUBLE
- DOUBLE_HEADER_DOUBLE_BODY
- DOUBLE_HEADER_DOUBLE_SINGLE_BODY
- DOUBLE_SINGLE_HEADER_DOUBLE_SINGLE_BODY
- DOUBLE_SINGLE_HEADER_SINGLE_BODY
- DOUBLE_SINGLE_HEADER_SINGLE_DASHED_BODY
- HYBRID_BOLD_HEADER_SINGLE_BODY
- SINGLE
- SINGLE_BLANK_HEADER_SINGLE_BLANK_BODY
- SINGLE_DOUBLE_HEADER_SINGLE_BODY
- SINGLE_DOUBLE_HEADER_SINGLE_DOUBLE_BODY
- SINGLE_HEADER_SINGLE_BODY
-
rowWidth
: In case the row width cannot be determined automatically, you can either set the system property-Dconsole.width=n
(see below) or you can provide the number of chars you desire per row with therowWidth
attribute, e.g.rowWidth=160
. layout
: The setup of the rows to be displayed, values as of theColumnLayout
enumeration:- BASIC
- ENDUSER
- DEVELOPER
- GRANDPA
- HACKER
- DEVOPS
- SUPERUSER
- ANALYST
The ConsoleLogger
(and in turn as being its sub-class the ConsoleLoggerSingleton
) by default uses the Terminal.getConsoleWidth()
method, which determines the width in characters of the system’s console in use. In case you pass a -Dconsole.width=n
JVM argument (where n
stands for the number of chars you desire for your console row width), then your width is taken, else the actual console’s width is being tried to be determined (see also SystemProperty.CONSOLE_WIDTH
).
The output as a console copy’n’paste dump (note that the tilde character
~
is used to truncate long lines; all aboveWARNING
is not truncated):
steiner@proteus:~/Workspaces/org.refcodes/refcodes-runtime-ext/refcodes-runtime-ext-console$ java -jar target/refcodes-runtime-ext-console-0.1.1-SNAPSHOT.jar ───────┬───────────────────┬───────┬───────────────┬──────────────────────────────┬──────────────────────┬──────────────────────────────────────────────────── 0000001│2015-04-04T11:21:19│INFO │main │~.console.SystemInfoConsoleApp│printInfo() │~12 12:19:35 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux 0000002│2015-04-04T11:21:19│INFO │main │~.console.SystemInfoConsoleApp│printInfo() │Console width := 158 0000003│2015-04-04T11:21:19│INFO │main │~.console.SystemInfoConsoleApp│printInfo() │Console height := 24 0000004│2015-04-04T11:21:19│INFO │main │~.console.SystemInfoConsoleApp│printInfo() │ANSI support := true 0000005│2015-04-04T11:21:19│INFO │main │~.console.SystemInfoConsoleApp│printInfo() │Operating system := UNIX 0000006│2015-04-04T11:21:19│INFO │main │~.console.SystemInfoConsoleApp│printInfo() │CLI := SHELL 0000007│2015-04-04T11:21:19│INFO │main │~.console.SystemInfoConsoleApp│printInfo() │Encoding := UTF8 0000008│2015-04-04T11:21:19│INFO │main │~.console.SystemInfoConsoleApp│printInfo() │Process (PID) := 13082 0000009│2015-04-04T11:21:19│INFO │main │~.console.SystemInfoConsoleApp│printInfo() │~rg.refodes.runtime.ext.console.SystemInfoConsoleApp 0000010│2015-04-04T11:21:19│INFO │main │~.console.SystemInfoConsoleApp│printInfo() │stty size := 0000011│2015-04-04T11:21:19│INFO │main │~.console.SystemInfoConsoleApp│printInfo() │tput cols := 80 0000012│2015-04-04T11:21:19│INFO │main │~.console.SystemInfoConsoleApp│printInfo() │tput lines := 24 steiner@proteus:~/Workspaces/org.refcodes/refcodes-runtime-ext/refcodes-runtime-ext-console$
SLF4J support
Instead of logging to the console via refcodes-logger-alt-console
, you can use the existing SLF4J
binding refcodes-logger-alt-slf4j
and make your refcodes-logger
log via SLF4J
just by including the below dependency (instead of the above one) in your pom.xml
:
1
2
3
4
5
6
7
8
9
<dependencies>
...
<dependency>
<artifactId>refcodes-logger-alt-slf4j</artifactId>
<groupId>org.refcodes</groupId>
<version>3.3.8</version>
</dependency>
...
</dependencies>
This will cause your …
Now logs are printed with your
SLF4J
binding, which could be aLog4J
binding.
SLF4J binding
Via the refcodes-logger-ext-slf4j
artifact, you can use the refcodes-logger
framework itself as an SLF4J
binding - in order to make, for example, your Tomcat
or spring-boot
log to the fancy refcodes-logger-alt-console
or a NoSQL
DB cluster (as of refcodes-logger-alt-simpledb
):
1
2
3
4
5
6
7
8
9
<dependencies>
...
<dependency>
<artifactId>refcodes-logger-ext-slf4j</artifactId>
<groupId>org.refcodes</groupId>
<version>3.3.8</version>
</dependency>
...
</dependencies>
In addition you have to add a dependency of one of the refcodes-logger-alt-*
artifacts to your pom.xml
and an according runtimelogger.ini
pulling that logger implementation you want your SLF4J
logs to log to …
Attention: Never use the
SLF4J
bindingrefcodes-logger-ext-slf4j
together with therefcodes-logger-alt-slf4j
logger implementation, else the one will look for the other, the other for the one … until you get anStackOverflowException
:D
With the SLF4J
binding, you can log all your SLF4J
logs (also the ones of the libs you included in your app) to any of the refcodes-logger-alt-*
implementations:
Make your
microservices
log into aNoSQL
database cluster with therefcodes-logger-alt-simpledb
artifact providing support for Amazon’sSimpleDB
.
Scalability
Just some keywords on scalability: The refcodes-logger
framework implements the composite pattern
and various partitioning
and composition
strategies for high scalability and therewith high throughput.
-
The
CompositeLoggerImpl
is an asynchronous logger with a log lines queue. Depending on the availability of the encapsulated loggers, the one or the other encapsulated logger takes the next waiting log line in the queue and processes it. The encapsulatedrefcodes-logger
s can be any instance implementing theLogger
interface (e.g. one of the loggers found in therefcodes-logger
artifact and therefcodes-logger-alt-*
artifacts). -
The
PartedLoggerImpl
implements partitioning functionality. You define a partitioning criteria for your log lines, e.g. a tenant’s ID (or the date or in case of therefcodes-logger
a package, class name or method) and theparted-logger
takes care to address the encapsulated responsiblelogger
(which in turn can be anotherparted-logger
or acomposite-logger
or any otherrefcodes-logger
). The encapsulatedrefcodes-logger
s can be any instance implementing theLogger
interface (e.g. one of the loggers found in therefcodes-logger
artifact and therefcodes-logger-alt-*
artifacts).
Querying
As the data sink to which a refcodes-logger
writes its output can be write only (e.g. a terminal window) or write, read and delete (e.g. a NoSQL
database such as the SimpleDB
logger), there are three flavors of the Logger
interface. For example the composite-logger
and the parted-logger
provides them three flavors:
Logger
: Write only logging, just like you know it fromLog4J
orSLF4J
…QueryLogger
: Use a logical querying language (as of therefcodes-criteria
artifact) to retrieve logs from yourrefcodes-logger
logging setup.TrimLogger
: Use a logical querying language (as of therefcodes-criteria
artifact) to trim (delete) logs from yourrefcodes-logger
data sinks; most important when doing big-data processing to clean up afterwards …
Under the hood
Logger
The Logger
is the most plain logger definition allowing you to log any kinds of data provided in a Record
and which can contain any number of
Column
elements, so that you are enabled to log big data of any structured information (e.g. high volume HTTP-Requests entering your
RESTful microservice).
RuntimeLogger
The RuntimeLogger
defines
a plain simple interface for logging out runtime information generated by software systems. The RuntimeLoggerImpl
implementation takes care of logging out the class and the method generating a log line.
The RuntimeLoggerImpl
actually takes a Logger
instance, whichs implementation to take is up to you:
Use the RuntimeLoggerFactorySingleton
factory (the factory is implemented as a singleton) to obtain RuntimeLogger
instances configured by a
runtimelogger.ini
file.
Composite logger
The CompositeLogger
uses the composite patter to forward Logger
functionality to a number encapsulated logger instances. Depending on the performance (and availability) of an encapsulated logger, the calls to the composite’s CompositeLogger.log(org.refcodes.tabular.Record)
method are executed by the next encapsulated logger ready for execution. An invocation of the CompositeLogger.log(org.refcodes.tabular.Record)
method is forwarded to exactly one of the encapsulated Logger
instances. The actual instance being called depends on its availability (in case, partitioning is needed, take a look at the PartedLogger
and its sub-classes).
Using the CompositeLogger
, a huge number of Record
instances can be logged in parallel by logging them to different physical data sinks (represented by the encapsulated logger instances), thereby avoiding a bottleneck which a single physical data sink would cause for logging.
Internally a log line queue (holding Record
instances to be logged) as well as a daemon thread per encapsulated logger (taking elements from the log line queue) are used to decouple the encapsulated Logger
instances from the CompositeLogger
. A given number of retries are approached in case there is an overflow of the log line queue; this happens when the queue is full and there are none encapsulated Logger
instances to take the next Record
. To avoid a building up of the log line queue, eventually causing an out of memory, log lines not being taken into the log line queue (as it is full) within the given number of retries, them log lines are dismissed. In such a
case a warning with a log-level WARN is printed out.
Query logger
The QueryLogger
type extends the CompositeLogger
type with query functionality as of QueryLogger.findLogs(org.refcodes.criteria.Criteria)
. In contrast to the CompositeLogger.log(org.refcodes.tabular.Record)
method, which is forwarded to exactly one of the encapsulated Logger
instances, the QueryLogger.findLogs()
(and the loke) method calls are forwarded to all of the encapsulated Logger
instances (in case, partitioning is needed, take a look at the PartedQueryLogger
type).
Trim logger
The CompositeTrimLogger
extends the CompositeLogger
with trim functionality. In contrast to the CompositeLogger.log(org.refcodes.tabular.Record)
method, which is forwarded to exactly one of the encapsulated Logger
instances, the CompositeTrimLogger.deleteLogs(org.refcodes.criteria.Criteria)
and CompositeTrimLogger.clear()
method calls are forwarded to all of the encapsulated Logger
instances (in case, partitioning is needed, take a look at the PartedTrimLogger
type).
Parted logger
The PartedLogger
is a partitioning Logger
which encapsulates Logger
instances or CompositeLogger
instances (or sub-classes of it) representing partitions.
This means: A partition is regarded to be a dedicated physical data sink or a CompositeLogger
containing Logger
instances attached to physical data sinks. A physical data sink may be a database (SQL or NoSQL), a files-system or volatile memory (in memory). To be more concrete: A physical data sink may be a domain when using Amazon’s SimpleDB, it may be a database table when using MySQL, it may be a HashMap or a List when using in memory storage.
The Record
instances as managed by the Logger
instances are mapped to the fields of the physical data sink (e.g. table columns regarding databases).
The Record
instances are stored to, retrieved from or deleted from dedicated partitions depending on partitioning Criteria
contained in the Record
instances (or the query Criteria
instances). The Criteria
(e.g. the column partition Criteria
in a Record
) as provided to the #log(Record) method is used by the PartedLogger
to select the partition to be addressed. In case of query operations the query Criteria
is used to determine the targeted partition (in case no partition can be determined and a fallback Logger
has been configured, then data may get logged to the fallback Logger
)
In practice there can be several (composite) Logger
instances being the partitions of the PartedLogger
, each individually addressed by the partitioning Criteria
. This approach a) helps us scale horizontally per partition when using CompositeLogger
instances per partition and b) helps limiting the traffic on those horizontally scaling (composite) Logger
instances by partitioning the data per Criteria
using the parted Logger
(or its sub-classes): Partitioning simply means switching to the partition defined by the Criteria
to perform the according Logger
operation.
Not having the PartedLogger
(or a sub-class
of it) would cause all the traffic for all Criteria
to hit just a single (composite) Logger
, limiting the possibility to scale endlessly (this one Logger
would be the bottleneck, even when being massively scaled horizontally): In particular this applies when looking at the extended versions of the PartedLogger
such as the PartedQueryLogger
and the PartedTrimLogger
where query requests are passed only to the partition which contains the required data: Increasing query traffic is parted and does not hit increasingly a single (composite) Logger
.
A Record
instance to be assigned to a partition must provide a so called partition column, whose value is used to determine which partition is to be addressed. The partition identifying column is passed upon construction of the PartedLogger
type. Specializations may hide this parameter from their constructors and pass their partitioning column from inside their constructor to the super constructor.
The PartedQueryLogger
extends the PartedLogger
with the functionality of a query Logger
. Any query operations, such as PartedQueryLogger.findLogs(org.refcodes.criteria.Criteria)
, are targeted at that partition containing the queried data. For this to work, the query must obey some rules:
The query is to contain an EqualWithCriteria
instance addressing the partition in an unambiguous way, AndCriteria
instances as being part of the root Criteria
hierarchy or an unambiguously nested AndCriteria
element hierarchy. More than one partition gets detected when unambiguous OrCriteria
instances are applied to the partition criteria. In such cases, the query is addressed to all potential partitions. If it was not possible to identify any partitions, then, as a fallback, all partitions are queried.
Query results are taken from from the invoked partitions (in normal cases this would be a single partition) round robin. the first result is taken from
the first queried partition’s result set (Record
instances), the next result from the next queried partition and so on to start over again with the first queried partition. Round robin has proven useful to prevent invalidation of physical data sinks’s result sets as of timeouts.
The PartedTrimLogger
type extends the PartedQueryLogger
type with the functionality of a TrimLogger
type. Delete operations with a
query such as PartedTrimLogger.deleteLogs(org.refcodes.criteria.Criteria)
are applied to the partitions in the same manner as done for PartedTrimLogger.findLogs(org.refcodes.criteria.Criteria)
.
Contribution guidelines
- Report issues
- Finding bugs
- Helping fixing bugs
- Making code and documentation better
- Enhance the code
Who do I talk to?
- Siegfried Steiner (steiner@refcodes.org)
Terms and conditions
The REFCODES.ORG
group of artifacts is published under some open source licenses; covered by the refcodes-licensing
(org.refcodes
group) artifact - evident in each artifact in question as of the pom.xml
dependency included in such artifact.