APM logs in context connects your logs with all of your telemetry data for your apps, hosts, and other entities. Bringing all of this data together in a single tool helps you quickly:
Cut through the noise of thousands of logs when troubleshooting time-critical issues, so you automatically see only the most relevant logs.
Navigate within multiple types of telemetry data, and have the data correlate back to the original issue.
Easily drill down into more detailed information from the same place in the UI.
Find the log lines that you need to identify and resolve a problem.
For more information, including examples, learn how to get started with APM logs in context.
Automatic logs in context options
If you are using a supported framework, you have two options to configure APM logs in context to send your app's logs and linking metadata automatically to New Relic. Supported frameworks for automatic logs in context include:
Log4j 2 2.6 or higher
Logback 1.1 or higher
Important
Agent releases 7.7.0 and higher have this feature enabled in the agent configuration file by default.
Agent log forwarding will cause an increase in the consumption of data when a supported framework is detected. The amount depends on the application and amount of logs it produces. This feature can be disabled. See Disable automatic logging for more information about your options.
If you already have a log forwarding solution in place, you should disable this feature.
If you are using a different logging framework, our manual logs in context solution might be right for you.
This is the simplest approach, and it's a great choice for developers who may not have the access or interest in setting up a log forwarder, or for accounts that want to see the power of logs and other linking metadata in context of their apps, without a lot of overhead.
Using this option, your logs will include span.id, trace.id, hostname, entity.guid, and entity.name. These attributes automatically link your logs to spans, traces, and other telemetry, making it easier to troubleshoot.
This feature may be enabled by default in a future agent version.
All you need to do is install an agent version with log forwarding capabilities (Java agent 7.6.0 or higher), and update your configuration to enable forwarding.
Configuration file (newrelic.yml), indented 2 spaces under the common stanza:
Once this feature is enabled, you also have control over the maximum number of log lines sent to New Relic every minute. The default value is 10,000. If more than 10,000 log lines are seen by the agent in a 60-second window, your logs will begin to be sampled.
Set it to a higher number to receive more logs. Set it to a lower number to receive fewer logs. Any non-negative integer is valid.
If you have an existing log forwarding solution and are updating your agent to use automatic logs in context, be sure to disable your manual log forwarder. Otherwise, your app will be sending double log lines. Depending on your account, this could result in double billing. For more information, follow the procedures to disable your specific log forwarder.
Already have a log forwarder you like? We've got you covered! The Java agent can decorate your logs with the linking metadata needed to provide access to automatic logs-in-context features, and you can send them to New Relic using your manual log forwarder.
This option should not be used with in-agent forwarding. Using an external log forwarder to send logs to New Relic while in-agent forwarding is enabled will cause your logs to be sent up twice to New Relic. Depending on your account, this may result in double billing.
If you want to use this option, make sure you have the in-agent forwarding configuration option disabled.
Configuration file (newrelic.yml), indented 2 spaces under the common stanza:
Our decorator adds five attributes to every log message in your application log file: entity.guid, hostname, trace.id, span.id and entity.name. Example:
This is my log message. NR-LINKING|{entity.guid}|{hostname}|{trace.id}|{span.id}|{entity.name}|
Some attributes may be empty if the log occurred outside a transaction or if they are not applicable to your application's context.
Secure your data
Your logs may include sensitive information protected by HIPAA or other compliance protocols. By default we obfuscate number patterns that appear to be for items such as credit cards or Social Security numbers, but you may need to hash or mask additional information.
For more information, see our documentation about obfuscation expressions and rules. You can hash or mask your log data by using the New Relic UI or by using NerdGraph, our GraphQL API.
Explore your data
To make the most of your logging data:
On the APM Summary page, click your Web transaction time chart to view logs associated with a specific point in time.
Check your app's Errors inbox to view the logs associated with your errors.
Typically your logs will start to appear less than a minute after you enable APM logs in context. Check your app's Triage > Logs section. You will also start seeing log patterns related to the error there.
If you don't see any logs for errors or traces, there may not be any for your app. Try refreshing the UI page, or change the selected time period.
Disable automatic logging
APM logs in context automatically forwards APM agent log data and is enabled by default. This can have a negative impact on your security, compliance, billing, or system performance. For more information, or if you need to adjust the default setting, follow the procedures to disable automatic logging.
Manual logs in context option
Before language agents had the ability to forward and decorate logs, you could use our manual solutions to send linking metadata.
This option is still supported, but is no longer encouraged. Also, this method requires that you install a log forwarder before enabling logs in context. If you do not have a log forwarder, the New Relic UI will prompt you to use our infrastructure agent.
If you decide to use your existing log forwarding solution and later decide to update your agent to use automatic logs in context, be sure to disable your manual log forwarder. Otherwise, your app will be sending double log lines. Depending on your account, this could result in double billing. For more information, follow the procedures to disable your specific log forwarder.
If you need to use the manual process to set up logs in context for Java, follow these steps:
Make sure you have already set up logging in New Relic. This includes configuring a supported log forwarder that collects your application logs and extends the metadata that is forwarded to New Relic.
Configure logs in context for Java to enrich your log data, using any of the following extensions as applicable. If you use Spring or Spring Boot and aren't sure which extension you need, see our Spring documentation.
We offer a Dropwizard extension for logs in context with the Java agent. To get started, review the code and an example application on GitHub.
To configure logs in context for your Java app with the DropWizard extension:
Make sure you have the Dropwizard 1.3 or higher package installed and working on your application. Use the original Dropwizard appenders and logging factory installed and working on the application.
Make sure you have the New Relic Java agent version 5.6.0 or higher installed on your application, and that you have enabled the JVM argument -javaagent.
Update your project's dependencies to include the Dropwizard 1.3 extension as applicable:
Gradle: Add the following to your build.gradle file:
Update your Dropwizard .yaml configuration file with a newrelic-json layout, replacing the currently used type: console or type: file with either type: newrelic-console or type: newrelic-file as appropriate. For example:
logging:
appenders:
- type: newrelic-console
# Add the two lines below if you don't have a layout specified on the appender.
# If you have a layout, remove all parameters to the layout and set the type.
layout:
type: newrelic-json
The New Relic Dropwizard extension also supports a log-format layout type that uses the standard Dropwizard logging. For testing purposes, you can change the type of the layout with a one-line change:
logging:
appenders:
- type: newrelic-file
# This format will be ignored by the newrelic-json layout, but used by the log-format layout.
To configure logs in context for your Java app with with the java.util.logging extension:
Make sure you have the java.util.logging package installed and working on the application.
Make sure you have the New Relic Java agent version 5.6.0 or higher installed on your application, and that you have enabled the JVM argument -javaagent.
Gradle: Add the following to your build.gradle file:
dependencies {
implementation("com.newrelic.logging:jul:2.0")
}
Maven: Add the following to your pom.xml file:
<dependencies>
<dependency>
<groupId>com.newrelic.logging</groupId>
<artifactId>jul</artifactId>
<version>2.0</version>
</dependency>
</dependencies>
Check if your logging file's handlers property is set to something other than NewRelicMemoryHandler. Look for a line listing the root logger's handlers, like this:
handlers = java.util.logging.FileHandler
Update your logging properties file to set the root logger's handler to NewRelicMemoryHandler so it intercepts messages destined for another handler:
Configure the NewRelicMemoryHandler by setting the target to the handler that was previously assigned to the root logger, so it captures data New Relic needs on the thread the log message is coming from:
Use a NewRelicFormatter for the final handler. Update your logging properties file to set the formatter property like the following example. Make sure the handler where you set the formatter is the target handler from the previous step (java.util.logging.FileHandler in this example).
The New Relic log format is JSON with telemetry metadata we use to correlate transactions and logs together. Currently we do not support any customization of that format.
Once complete, JSON is logged instead of text. The JSON should be formatted as single objects, one per line, and should contain fields like log.level and thread.name. The trace.id, which is required for logs in context, should only have a value for log messages that occur within a transaction.
The most direct way to get the logs-in-context extensions is to add these dependencies to Maven's pom.xml or Gradle's build.gradle. This allows the packaging tools to pick up the correct dependencies.
If you can't edit these files, you can instead add the jars directly to the application classpath for your logging framework's configuration.
Verify which logging framework the application is using.
Make sure you are able to change your logging framework's configuration.
Add the following three jars to the classpath if they aren't already present. Generally, we recommend taking the latest versions published on Maven Central.
Group ID com.fasterxml.jackson.core and Artifact ID: Use jackson-core.
Group ID com.newrelic.agent.java and Artifact ID: Use newrelic-api.
Tip
Take advantage of our automatic logs in context solution for Java. No need to use this manual process!
We offer a Log4j 1.x extension for manual logs in context with the Java agent. To get started, review the code and an example application on GitHub.
To configure manual logs in context for your Java app with with the Log4j 1.x extension, you must configure the Log4j extension in code or via XML. Properties files are not supported because AsyncAppender instances can only be automatically configured via XML.
Make sure you have the Log4j 1.x package installed and working on the application.
Make sure you have the New Relic Java agent version 5.6.0 or higher installed on your application, and that you have enabled the JVM argument -javaagent.
Update your project's dependencies to include the Log4j 1.x extension as applicable.
Gradle: Add the following to your build.gradle file:
dependencies {
implementation("com.newrelic.logging:log4j1:2.0")
}
Maven: Add the following to your pom.xml file:
<dependencies>
<dependency>
<groupId>com.newrelic.logging</groupId>
<artifactId>log4j1</artifactId>
<version>2.0</version>
</dependency>
</dependencies>
In your logging configuration XML file, update your <appender> element with a NewRelicLayout, adding <layout class="com.newrelic.logging.log4j1.NewRelicLayout"/>:
<appender name="TypicalFile" class="org.apache.log4j.FileAppender">
<param name="file" value="logs/log4j1-app.log"/>
<param name="append" value="false"/>
<layout class="com.newrelic.logging.log4j1.NewRelicLayout"/> <!-- only this line needs to be added -->
</appender>
Use NewRelicAsyncAppender to wrap any appenders that will target New Relic's log forwarder. For example:
Make sure you have the New Relic Java agent version 5.6.0 or higher installed on your application, and that you have enabled the JVM argument -javaagent.
Update your project's dependencies to include the Log4j 2.x extension as applicable:
Gradle: Add the highlighted section to your build.gradle file:
If you only modified an existing appender, skip this step. If you added a new appender, add <AppenderRef/> within <Root> to use this appender. Use the ref attribute to refer to appender name you created in the previous step. For example:
If you're using a properties file and added a new appender, add:
rootLogger.level = info
rootLogger.appenderRef.stdout.ref = STDOUT
Tip
Take advantage of our automatic logs in context solution for Java. No need to use this manual process!
We offer a Logback extension for logs in context with the Java agent. To get started, review the code and an example application on GitHub.
To configure logs in context for your Java app with Logback:
Make sure you have Logback version 1.2.0 or higher and the New Relic Java agent version 5.6.0 or higher installed and working on your application.
Make sure you have the New Relic Java agent version 5.6.0 or higher installed on your application, and that you have enabled the JVM argument -javaagent.
Update your project's dependencies to include the Logback extension as applicable:
Gradle: Add the highlighted section to your build.gradle file:
Update your logging configuration xml with the NewRelicAsyncAppender. To ensure that NewRelicAsyncAppender wraps any appenders that will target New Relic's log forwarder, add the following section. Change "LOG_FILE" to the name of the appender you updated in the previous step.
Make sure NewRelicAsyncAppender is the first appender used in your logger. Replace your root logger’s appenders with the ASYNC appender created in the previous step. Then list any other appenders after the NewRelicAsyncAppender in the <root> list.
<root>
<appender-ref ref="ASYNC" />
</root>
Here are examples of an updated logging .xml file for the Logback extension. You can also see a working example in GitHub.
Single console appender example
Example configuration file after adding in the logging extension information:
<!-- Send every message to normal console logging, as well. -->
<appender-ref ref="STDOUT" />
</root>
</configuration>
We offer extensions for current versions of Spring and Spring Boot. If you already know the logging library, you can skip directly to that documentation:
Spring implements a bridge to other logging libraries that will automatically find them. However, those individual libraries must be configured and explicitly included in your project dependencies.
To identify your logging dependency, consult your Gradle, Maven, or other build tool's dependency tree. Then follow the procedures to configure logs in context for your Java app with that extension.