Chicken/Egg configuration problem

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Chicken/Egg configuration problem

Adam Gent
I accidentally unsubscribed but I figured I would start a new thread
on what I mean by the Chicken/Egg configuration problem with SL4FJ.

The problem is you often in cloud environments want a fully resolved
configuration with out doing logging.
.... But you can't have fully correctly configured logging with out
configuration being complete.... but you can't a proper configuration
with out using logging.

I'm not talking about some reentrant or endless recursive calls or
infinite loop that the event queue in LogFactory prevents.

Lets go over this with a configuration initialization that is agnostic
of logging backend.

1. static main - initialization
2. configuration code begins
3. configuration needs to set some system properties and various
singletons required by a backend logging framework and various other
downstream libraries (the best way to configure old log4j is still
through system properties)
4. configuration code needs to get external configuration to set these
properties
5. configuration code uses a library that has SLF4J in it (lets say
Zookeeper but it could be any sort of configuration library like
Archaius).
5. SLF4J initialization begins
6. Backend logging initialization begins but is missing required
system properties and/or singletons.
7. Backend logging finishes initialization but is incorrect
8. Logging events do not have correct MDC attributes but worse Logging
appenders may not be setup correctly.

1-8 is actually what happens with dropwizard, and spring. What do they
do? They refresh logback (last time I checked). That is not an option
with all logging backends. And No Joachim I'm not using multiple
backends in the same application. I'm using multiple applications that
have different backends (WARs for example). I'd rather not have to do
special things for each logging backend.

Now you can fix the above problem by using a logging framework that
allows initialization interception... Problem is not many do. In fact
Logback didn't even have an option till I pushed for the ServiceLoader
SPI (ignoring hacks with Groovy or Appenders).  On a related note that
ServiceLoader patch now offered a way to save initialization time for
logback users...particularly android users. Brought to you by the guy
that apparently has no idea of the internals of SLF4J.

If you can participate in the backends logging initialization then yes
the replay event queue does help but no entirely as events may not
have proper MDC information.
That is if you want to set MDC information on the initialization
thread with something the configuration library is going to provide
you might be too late. Not a big deal. Pretty minor concern.

Now I don't have a really good solution other than either 1. don't use
libraries that use SL4J during initialization (which is hard with
things like Spring Cloud config and Archaius) or 2. call your
initialization code in the logging backend. I prefer 1 but it is
getting harder and harder with everyone using logging facades. I
prefer 1 so much it is the reason I was pushing the wrapper mentioned
earlier today.

Now going back to the blessed temporary event queue (that Joachim
thinks I don't understand) in LoggerFactory I think that could also be
used to capture logging messages while you are configuring before
kicking of the binding (bind()).

Something like:

1. SFL4J LoggerFactory.startConfiguration()
2. You the application developer do all your config loading free to
use SLF4J dependent libraries. The messages will be captured and
initialization will not happen.
3. SLF4J LoggerFactory.endConfiguration() // initialize backend and
replay events

You could even do the above with a temp autocloseable try {} style or
maybe something like LogFactory.doConfiguration(Runnable).

The semantics of calling the above methods multiple times I'll leave
up in the air but I would assume IllegalState.

I just don't think I can explain this issue any better or if is even
worth it... I'm just not looking forwarding to wasting time getting
trashed with various technicalities, semantics, jargon, and missing
details due to my ineptness to explain things (or lack of
intelligence).

All I wanted to know was how you get remote configuration loaded
before (and for) your logging framework with so many things implicitly
depending on your logging framework.

--
CTO
SnapHop (snaphop.com)
(twitter) @agentgt (linkedin) http://www.linkedin.com/in/agentgt
(cell) 781-883-5182
_______________________________________________
slf4j-user mailing list
[hidden email]
http://mailman.qos.ch/mailman/listinfo/slf4j-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Chicken/Egg configuration problem

Joachim Durchholz
Am 19.03.2017 um 05:50 schrieb Adam Gent:

> I accidentally unsubscribed but I figured I would start a new thread
> on what I mean by the Chicken/Egg configuration problem with SL4FJ.
>
> The problem is you often in cloud environments want a fully resolved
> configuration with out doing logging.
> .... But you can't have fully correctly configured logging with out
> configuration being complete.... but you can't a proper configuration
> with out using logging.
>
> I'm not talking about some reentrant or endless recursive calls or
> infinite loop that the event queue in LogFactory prevents.
>
> Lets go over this with a configuration initialization that is agnostic
> of logging backend.
>
> 1. static main - initialization
> 2. configuration code begins
> 3. configuration needs to set some system properties and various
> singletons required by a backend logging framework and various other
> downstream libraries (the best way to configure old log4j is still
> through system properties)
> 4. configuration code needs to get external configuration to set these
> properties
> 5. configuration code uses a library that has SLF4J in it (lets say
> Zookeeper but it could be any sort of configuration library like
> Archaius).
> 5. SLF4J initialization begins
> 6. Backend logging initialization begins but is missing required
> system properties and/or singletons.

How this?
Step 3 is supposed to provide all this.

> 7. Backend logging finishes initialization but is incorrect
> 8. Logging events do not have correct MDC attributes but worse Logging
> appenders may not be setup correctly.

Since we disagree about what problems can happen, I'm leaving discussion
of the solutions to another thread.

(I do believe you that you did have problems, I just do not agree with
the problem description because it doesn't make sense to me - yet.)
_______________________________________________
slf4j-user mailing list
[hidden email]
http://mailman.qos.ch/mailman/listinfo/slf4j-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Chicken/Egg configuration problem

Matt Sicker
You could always write your configuration bootstrapping another programming language!

In my experience, what I've usually done or seen done for managing this sort of configuration is through application startup scripts more so than pure Java. For example, using Mesos, I can set environment variables ahead of time. Using config management, I can have a program to configure my tasks before starting a Java executable entirely. It all really depends on the deployment environment you use. I'm also curious as to what other things you're doing before logging initialization is wanted.

Also, Commons Logging has been using the ServiceLoader method for a while, though again, as with all the logging frameworks, initialization happens as soon as you use a logger.

On 19 March 2017 at 06:08, Joachim Durchholz <[hidden email]> wrote:
Am 19.03.2017 um 05:50 schrieb Adam Gent:
I accidentally unsubscribed but I figured I would start a new thread
on what I mean by the Chicken/Egg configuration problem with SL4FJ.

The problem is you often in cloud environments want a fully resolved
configuration with out doing logging.
.... But you can't have fully correctly configured logging with out
configuration being complete.... but you can't a proper configuration
with out using logging.

I'm not talking about some reentrant or endless recursive calls or
infinite loop that the event queue in LogFactory prevents.

Lets go over this with a configuration initialization that is agnostic
of logging backend.

1. static main - initialization
2. configuration code begins
3. configuration needs to set some system properties and various
singletons required by a backend logging framework and various other
downstream libraries (the best way to configure old log4j is still
through system properties)
4. configuration code needs to get external configuration to set these
properties
5. configuration code uses a library that has SLF4J in it (lets say
Zookeeper but it could be any sort of configuration library like
Archaius).
5. SLF4J initialization begins
6. Backend logging initialization begins but is missing required
system properties and/or singletons.

How this?
Step 3 is supposed to provide all this.

7. Backend logging finishes initialization but is incorrect
8. Logging events do not have correct MDC attributes but worse Logging
appenders may not be setup correctly.

Since we disagree about what problems can happen, I'm leaving discussion of the solutions to another thread.

(I do believe you that you did have problems, I just do not agree with the problem description because it doesn't make sense to me - yet.)

_______________________________________________
slf4j-user mailing list
[hidden email]
http://mailman.qos.ch/mailman/listinfo/slf4j-user



--
Matt Sicker <[hidden email]>

_______________________________________________
slf4j-user mailing list
[hidden email]
http://mailman.qos.ch/mailman/listinfo/slf4j-user
Loading...