[Solved] Correctly log protobuf messages as unescaped JSON with zap logger

VHristov Asks: Correctly log protobuf messages as unescaped JSON with zap logger
I have a Go project where I’m using Zap structured logging to log the contents of structs. That’s how I initialise the logger:

Code:
zapLog, err := zap.NewProductionConfig().Build()
if err != nil {
    panic(err)
}

Initially I started with my own structs with json tags and it all worked perfectly:

Code:
zapLog.Info("Event persisted", zap.Any("event", &event))

Result:

Code:
{"level":"info","ts":1626448680.69099,"caller":"persisters/log.go:56",
 "msg":"Event persisted","event":{"sourceType":4, "sourceId":"some-source-id", 
 "type":"updated", "value":"{...}", "context":{"foo":"bar"}}}

I now switched to protobuf and I’m struggling to achieve the same result. Initially I just got the “reflected map” version, when using zap.Any():

Code:
zapLog.Info("Event persisted", zap.Any("event", &event))

Code:
{"level":"info","ts":1626448680.69099,"caller":"persisters/log.go:56",
 "msg":"Event persisted","event":"sourceType:TYPE_X sourceId:"some-source-id", 
 type:"updated" value:{...}, context:<key: foo, value:bar>}

I tried marshalling the object with the jsonpb marshaller, which generated the correct output on itself, however, when I use it in zap.String(), the string is escaped, so I get an extra set of ” in front of each quotation mark. Since there’s processing of the logs at a later point, this causes problems there and hence I want to avoid it:

Code:
m := jsonpb.Marshaler{}
var buf bytes.Buffer
if err := m.Marshal(&buf, msg); err != nil {
    // handle error
}
zapLog.Info("Event persisted", zap.ByteString("event", buf.Bytes()))

Result:

Code:
{"level":"info","ts":1626448680.69099,"caller":"persisters/log.go:56",
 "msg":"Event persisted","event":"{"sourceType":"TYPE_X", "sourceId":"some-source-id", 
 "type":"updated", "value":"{...}", "context":{"foo":"bar"}}"}

I then tried using zap.Reflect() instead of zap.Any() which was the closest thing I could get to what I need, except that enums are rendered as their numerical values (the initial solution did not have enums, so that didn’t work in the pre-protobuf solution either):

Code:
zapLog.Info("Event persisted", zap.Reflect("event", &event))

Result:

Code:
{"level":"info","ts":1626448680.69099,"caller":"persisters/log.go:56",
 "msg":"Event persisted","event":{"sourceType":4, "sourceId":"some-source-id", 
 "type":"updated", "value":"{...}", "context":{"foo":"bar"}}}

The only option I see so far is to write my own MarshalLogObject() function:

Code:
type ZapEvent struct {
    event *Event
}

func (z *ZapEvent) MarshalLogObject(encoder zapcore.ObjectEncoder) error {

  encoder.AddString("sourceType", z.event.SourceType.String()
  // implement encoder for each attribute

}

func processEvent(e Event) {
   ...
   zapLog.Info("Event persisted", zap.Object("event", &ZapEvent{event: &e}))
}

But since it’s a complex struct, I would rather use a less error prone and maintenance heavy solution. Ideally, I would tell zap to use the jsonpb marshaller somehow, but I don’t know if that’s possible.

Ten-tools.com may not be responsible for the answers or solutions given to any question asked by the users. All Answers or responses are user generated answers and we do not have proof of its validity or correctness. Please vote for the answer that helped you in order to help others find out which is the most helpful answer. Questions labeled as solved may be solved or may not be solved depending on the type of question and the date posted for some posts may be scheduled to be deleted periodically. Do not hesitate to share your response here to help other visitors like you. Thank you, Ten-tools.