.NET: Configure logs in context

This is the simplest approach, and it's a great choice for developers who may not have the access or interest in setting up a log forwarder, or for accounts that want to see the power of logs and other linking metadata in context of their apps, without a lot of overhead.

Using this option, your logs will include span.id, trace.id, hostname, entity.guid, and entity.name. These attributes automatically link your logs to spans, traces, and other telemetry, making it easier to troubleshoot.

All you need to do is install an agent version with log forwarding capabilities (.NET agent 9.7.0 or higher), and update your configuration to enable forwarding:

<applicationLogging enabled="true">
  <forwarding enabled="true" />
</applicationLogging>

This option can also be enabled by setting the environment variable NEW_RELIC_APPLICATION_LOGGING_FOWARDING_ENABLED=true. The environment variable value, if present, takes precedence over the config file value.

This option may be on by default in a future agent version.

Optional adjustments:

Once this is enabled, you also have control over the maximum number of logs sent to New Relic every minute. The default value is 10,000. If more than 10,000 logs are received in a 60-second window, your logs will begin to be sampled.

Set it to a higher number to receive more logs. Set it to a lower number to receive fewer logs. Any integer is valid.

<applicationLogging enabled="true">
  <forwarding enabled="true" maxSamplesStored="10000" />
</applicationLogging>

This setting can also be adjusted by setting the environment variable NEW_RELIC_APPLICATION_LOGGING_FORWARDING_MAX_SAMPLES_STORED. The environment variable value, if present, takes precedence over the config file value.

If you have an existing log forwarding solution and are updating your agent to use automatic logs in context, be sure to disable your manual log forwarder. Otherwise, your app will be sending double log lines. Depending on your account, this could result in double billing. For more information, follow the procedures to disable your specific log forwarder.

Already have a log forwarder you like? We've got you covered! Language agents can decorate your logs with the linking metadata needed to provide access to automatic logs-in-context features.

This option should not be used with in-agent forwarding. Using an external log forwarder to send logs to New Relic while in-agent forwarding is enabled will cause your logs to be sent up twice to New Relic. Depending on your account, this may result in double billing.

1. If you want to use this option, make sure you have the in-agent forwarding configuration option disabled.

2. Enable log decorating in your configuration, then relaunch the agent to start decorating your logs.

   <applicationLogging enabled="true">
     <localDecorating enabled="true" />
   </applicationLogging>

   Copy

   This option can also be enabled by setting the environment variable NEW_RELIC_APPLICATION_LOGGING_LOCAL_DECORATING_ENABLED=true. The environment variable value, if present, takes precedence over the config file value.

   Our decorator adds five attributes to every log message: entity.guid, entity.name, hostname, trace.id, and span.id. Example:

   This is my log message. NR-LINKING|{entity.guid}|{hostname}|{trace.id}|{span.id}|{entity.name}|

   Copy

   Some attributes may be empty if the log occurred outside a transaction or if they are not applicable to your application's context. We recommend this option over the manual process to use a decorating formatter.

   Important

   The key used to read out the NR-LINKING token from a logging framework is NR_LINKING not NR-LINKING in order to support logging frameworks that do not allow hyphens in key names.

   log4net configuration

   For log4net, the .NET agent stores the NR-LINKING token as a property called NR_LINKING in the log event object. The following common layout examples demonstrate how to configure log4net to write out out the NR-LINKING token. Other layouts are supported, but not listed below. SimpleLayout is not supported since it cannot be altered.

   PatternLayout/DynamicPatternLayout: Update the conversionPattern to include the NR_LINKING property at the end of the line.

   <layout type="log4net.Layout.PatternLayout">
       <conversionPattern value="%date %5level %logger.%method [%line] - MESSAGE: %message %property{NR_LINKING}%newline %exception" />
   </layout>

   Copy

   log4net.Ext.Json: Update the layout to include a member called NR_LINKING.

   <layout type='log4net.Layout.SerializedLayout, log4net.Ext.Json'>
       <decorator type='log4net.Layout.Decorators.StandardTypesFlatDecorator, log4net.Ext.Json' />
       ...
       <member value='NR_LINKING' />
   </layout>

   Copy

   log4net.Appender.AdoNetAppender: This appender has a slightly different configuration than other layouts. You will need to update the "message" parameter to include the metadata. The following example demonstrates this using a PatternLayout.

   <parameter>
       <parameterName value="@message" />
       <dbType value="String" />
       <size value="4000" />
       <layout type="log4net.Layout.PatternLayout">
           <conversionPattern value="%message %property{NR_LINKING}" />
       </layout>
   </parameter>

   Copy

   Serilog configuration

   For Serilog, the .NET agent stores the NR-LINKING token as a property called NR_LINKING in the log event object. The following common formatter and sink examples demonstrate how to configure Serilog to write out out the NR-LINKING token. Other sinks and formatters are supported, but not listed below.

   Plain text using an outputTemplate Update the outputTemplate to include the NR_LINKING property at the end of the line.

   Log.Logger = new LoggerConfiguration()
     .MinimumLevel.Override("Microsoft", LogEventLevel.Information)
     .Enrich.FromLogContext()
     .WriteTo.File(
       path: @"plaintext_log.txt",
       fileSizeLimitBytes: 1_000_000,
       rollOnFileSizeLimit: true,
       shared: true,
       flushToDiskInterval: TimeSpan.FromSeconds(1),
       outputTemplate: "{Timestamp:yyyy-MM-dd HH:mm:ss.fff zzz} [{Level:u3}] {Message:lj} {NR_LINKING} {NewLine}{Exception}"
     )
     .CreateLogger();

   Copy

   JSON Formatter with automatic property output By default, the JsonFormatter will write out every property on a message.

   Log.Logger = new LoggerConfiguration()
     .MinimumLevel.Override("Microsoft", LogEventLevel.Information)
     .Enrich.FromLogContext()
     .WriteTo.Console(new JsonFormatter());
     .CreateLogger();

   Copy

   Microsoft.Extensions.Logging configuration

   For Microsoft.Extensions.Logging, the .NET agent stores the NR-LINKING token with the message in a newly created a Scope. Since Microsoft.Extensions.Logging relies on other frameworks for output, please see either the log4net or Serilog configuration above for details.

Before language agents had the ability to forward and decorate logs, you could enable logs in context by updating your application to send linking metadata.

This option is still supported, but is no longer encouraged. For instructions to use this approach, see Manual logs in context option.

Also, this method requires that you install a log forwarder before enabling logs in context. If you do not have a log forwarder, the New Relic UI will prompt you to use our infrastructure agent.

If you decide to use your existing log forwarding solution and later decide to update your agent to use automatic logs in context, be sure to disable your manual log forwarder. Otherwise, your app will be sending double log lines. Depending on your account, this could result in double billing. For more information, follow the procedures to disable your specific log forwarder.

The following diagram illustrates the flow of log messages through Apache log4net, highlighting specific components of the New Relic log4net extension. Many log forwarders are available. This example uses Fluentd.

Appender: The NewRelicAppender adds contextual information from the .NET agent (using the API) to the log events generated by the application. This contextual information, known as linking metadata, is used by New Relic to link log messages to the transactions and spans from which they were created. This appender will pass the enriched log events to downstream appenders for further processing.

Since the NewRelicAppender is ForwardingAppender type, it needs to be the first appender in the chain. It also requires another appender that can write to an actual output destination as its child in order to work.

Layout: The NewRelicLayout formats the enriched log events into the JSON format expected by New Relic. The appender, which this layout is assigned to, instructs log4net to output the JSON to a file in the location that the log forwarder expects.

Log Forwarder: The log forwarder monitors an output folder and incrementally sends the properly formatted and enriched log information to the New Relic logging endpoint.

Log4net uses appender and layout to store and format log messages. NewRelicAppender enriches log messages with contextual information from the New Relic .NET agent if it is attached to your application. The appender passes enriched log messages to downstream appenders to handle specific use cases for log messages.

For more information about logging with log4net, see the Apache log4net Getting started documentation.

To configure logs in context with the log4net extension:

1. Using the Visual Studio NuGet Package Manager, locate and install the NewRelic.LogEnrichers.Log4Net package.

2. In your log4net configuration file, update your logging configuration to use the NewRelicAppender as the first level appender, and reference your existing appenders as its children. Also replace the layout of the appender that writes log messages to an output destination with the NewRelicLayout.

   The following log4net configuration example enriches log events with New Relic linking metadata. In addition to the existing log files, it outputs new log files in a specific JSON format at C:\logs\log4netExample.log.json for consumption by the log forwarder:

   <log4net>
     <root>
       <level value="ALL" />
       <appender-ref ref="NewRelicAppender" />
     </root>
   
     <appender name="NewRelicAppender" type="NewRelic.LogEnrichers.Log4Net.NewRelicAppender, NewRelic.LogEnrichers.Log4Net" >
       <threshold value="ALL"/>
       <appender-ref ref="FileAppender" />
     </appender>
   
     <appender name="FileAppender" type="log4net.Appender.FileAppender">
       <file value="C:\logs\log4netExample.log.json" />
       <param name="AppendToFile" value="true" />
       <layout type="NewRelic.LogEnrichers.Log4Net.NewRelicLayout, NewRelic.LogEnrichers.Log4Net">
       </layout>
     </appender>
   </log4net>

   Copy

   After you configure the log4net extension and update your logging file, you can configure your extension to send data to New Relic. There are several options for forwarding your logs. Here is an example configuration using the infrastructure agent for logs in context:

   logs:

   Copy

• name: application-log file: C:\logs\log4netExample.log.json # Path to a single log file

  #Documentation for using the infrastructure agent forwarder

  Copy

The New Relic NLog extension provides a NewRelicJsonLayout that formats a log event in the way required by the New Relic logging endpoint. Next, it adds contextual information from the .NET agent when attached to your application. Then, a target can be configured to write logging data to an output folder. The log forwarder can monitor this folder and incrementally send log information to New Relic.

The following diagram illustrates the flow of log messages through NLog, highlighting specific components of the New Relic NLog extension.

New Relic JSON Layout: The NewRelicJsonLayout adds contextual information from the .NET agent (using the API) to the log events generated by the application, and outputs log messages in the JSON format expected by New Relic. This contextual information, known as linking metadata, is used by New Relic to link log messages to the transactions and spans where they were created.

Since the NewRelicAppender is ForwardingAppender type, it needs to be the first appender in the chain. It also requires another appender that can write to an actual output destination as its child in order to work.

File Target: A FileTarget defines a file on disk where log messages are written. Adding the NewRelicJsonLayout to that target allows the output to be formatted correctly for forwarding to New Relic.

Log Forwarder: The log forwarder is configured to send the properly formatted and enriched log data from the FileTarget output to the New Relic logging endpoint.

For more information about logging with NLog, see the nlog-project.org documentation.

Use our NLog 4.5 or higher extension to link to your log data with related data across the rest of the New Relic platform.

1. Using the Visual Studio NuGet Package Manager, locate and install the NewRelic.LogEnrichers.NLog package.

2. In your application code, update your logging configuration to add the NewRelicJsonLayout and decide if you want to collect MappedDiagnosticsContext (MDC) or the MappedDiagnosticsLogicalContext (MDLC) data.

   The following configuration examples result in new JSON files that are written to disk. Some of these configuration options may be useful for managing the amount of disk space used and/or the performance of the target.

• archiveAboveSize

• maxArchiveFiles

• bufferSize

• enableArchiveFileCompression

• autoFlush

• concurrentWrites

  Although the NLog AsyncWrapper Target is not required, it may help improve performance by performing formatting and output of log files on a different thread.

  Don't collect MDC or the MDLC data (default):

  The following code example enriches log events with New Relic linking metadata, but not with MDC or the MDLC data. In addition to the existing log files, it outputs new log files in a specific JSON format at C:\logs\NLogExample.log.json for consumption by the log forwarder:

  var loggerConfig = new LoggingConfiguration();
  var newRelicFileTarget = new FileTarget("NewRelicFileTarget");
  newRelicFileTarget.Layout = new NewRelicJsonLayout();
  newRelicFileTarget.FileName = "C:\logs\NLogExample.json";
  loggerConfig.AddTarget(newRelicFileTarget);
  loggerConfig.AddRuleForAllLevels("NewRelicFileTarget");
  LogManager.Configuration = loggerConfig;
  var logger = LogManager.GetLogger("Example");

  Copy

  Collect MDC or the MDLC data:

  If your application uses the MDC or the MDLC, you can configure the NewRelicJsonLayout to include items in those collections. The following code example adds the additional configuration to enable collecting MDC and MDLC data. As in the previous example, it outputs new log files in a specific JSON format at C:\logs\NLogExample.log.json for consumption by the log forwarder:

  var loggerConfig = new LoggingConfiguration();
  var newRelicFileTarget = new FileTarget("NewRelicFileTarget");
  var newRelicLayout = new NewRelicJsonLayout
  {
    IncludeMdc = `true,`
    IncludeMdlc = `true`
  };
  
  newRelicFileTarget.Layout = newRelicLayout;
  newRelicFileTarget.FileName = "C:\logs\NLogExample.json";
  loggerConfig.AddTarget(newRelicFileTarget);
  loggerConfig.AddRuleForAllLevels("NewRelicFileTarget");
  LogManager.Configuration = loggerConfig;
  var logger = LogManager.GetLogger("Example");

  Copy

  Once you have configured the NLog extension and updated your logging file, you can configure your extension to send data to New Relic. There are several options for forwarding your logs. Here is an example configuration using the infrastructure agent for logs in context:

  logs:

  Copy

• name: application-log file: C:\logs\log4netExample.log.json # Path to a single log file

#Documentation for using the infrastructure agent forwarder

You can also configure the New Relic NLog extension with file-based configuration providers. The folowing example code creates a logger based on settings contained in an App.config file.

Instantiating Logger using .config file

var logger = LogManager.GetLogger("NewRelicLog");
logger.Info("Hello, New Relic!");

Sample App.config file

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <configSections>
    <section name="nlog" type="NLog.Config.ConfigSectionHandler, NLog"/>
  </configSections>
  <startup>
    <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.5" />
  </startup>
  <nlog xmlns="http://www.nlog-project.org/schemas/NLog.xsd"
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <extensions>
      <add assembly="NewRelic.LogEnrichers.NLog" />
    </extensions>
    <targets>
      <target name="NewRelicLogFile" xsi:type="File" fileName="C:/path/to/NewRelicLog.json">
        <layout xsi:type="newrelic-jsonlayout">
        </layout>
      </target>
    </targets>
    <rules>
      <logger name="NewRelicLog" minlevel="Info" writeTo="newRelicLogFile" />
    </rules>
  </nlog>
</configuration>

Serilog is a structured logging framework that records log messages from your application and creates a LogEvent to store the message data. Using Enrichers, you can add additional information to the log events. Sinks and Formatters allow you to format and output those log events for downstream consumption and viewing.

The following diagram illustrates the flow of log messages through Serilog, highlighting specific components of the New Relic Serilog extension. Many log forwarders are available. This example uses Fluentd.

New Relic Enricher: The NewRelicEnricher adds contextual information from the .NET agent (using the API) to the log events generated by the application. This contextual information, called linking metadata, is used by New Relic to link log messages to the transactions and spans where they were created.

New Relic Formatter: The NewRelicFormatter translates enriched log events into the JSON format expected by New Relic. A sink instructs Serilog to output the JSON to a file in the location that the log forwarder expects.

New Relic Log Forwarder: The log forwarder is configured to send the properly formatted and enriched log data from the FileTarget output to the New Relic logging endpoint.

For more information about Serilog log events, see the Serilog documentation on GitHub.

To configure logs in context with the Serilog extension:

1. Use the Visual Studio NuGet Package Manager to locate and install the NewRelic.LogEnrichers.Serilog package.

2. In your application code, update your logging configuration to add the NewRelicEnricher and NewRelicFormatter.

   The following code example enriches log events with New Relic linking metadata. In addition to the existing log files, it outputs new log files in a specific JSON format at C:\logs\SerilogExample.log.json for consumption by the log forwarder:

   var loggerConfig = new LoggerConfiguration()
   
     loggerConfig
        .Enrich.WithThreadName()
        .Enrich.WithThreadId()    
        .Enrich.WithNewRelicLogsInContext()
        .WriteTo.File( path: @"C:\logs\ExistingLoggingOutput.txt")
        .WriteTo.File(
            formatter: new NewRelicFormatter(), 
            path: @"C:\logs\SerilogExample.log.json");
   
   var log = loggerConfig.CreateLogger();

   Copy

   This configuration results in new JSON files that are written to disk. Some of these configuration options may be useful for managing the amount of disk space used and/or the performance of the sink.

• restrictedToMinimumLevel

• buffered

• rollingInterval

• rollOnFileSizeLimit

• retainedFileCountLimit

  Although not required, using the Serilog Asynchronous Sink Wrapper may help improve the performance by performing formatting and output of log files on a different thread.

3. Once you have configured the Serilog extension and updated your logging file, there are several options for forwarding your logs. Here is an example configuration using the infrastructure agent for logs in context:

   logs:

   Copy

• name: application-log file: C:\logs\log4netExample.log.json # Path to a single log file

#Documentation for using the infrastructure agent forwarder

You can also configure the New Relic Serilog extension with file-based configuration providers.The following additional NuGet Packages are required:

• Microsoft.Extensions.Configuration

• Serilog.Settings.Configuration

  The following example code creates a logger based on settings contained in an appSettings.json file.

  Instantiating logger using appsettings.json

  var builder = new ConfigurationBuilder()
          .AddJsonFile("appsettings.json");
  
  var configuration = builder.Build();
  
  var logger = new LoggerConfiguration()
          .ReadFrom.Configuration(configuration)
          .CreateLogger();

  Copy

  Sample appsettings.json file

  {
    "Serilog": {
      "Using": [ 
            "Serilog.Sinks.Console",
            "Serilog.Sinks.File",
            "NewRelic.LogEnrichers.Serilog" 
      ],
      "MinimumLevel": "Debug",
      "Enrich": [ "WithNewRelicLogsInContext" ],
      "WriteTo": [
        {
          "Name": "File",
          "Args": {
            "path": "C:\\Logs\\SerilogExample.log.json",
            "formatter": "NewRelic.LogEnrichers.Serilog.NewRelicFormatter, NewRelic.LogEnrichers.Serilog"
            }
        }
      ],
  
      "Properties": {
        "Application": "NewRelic Logging Serilog Example"
      }
    }
  }

  Copy

  The following example code creates a logger based on settings contained in a web.config file. The Serilog.Settings.AppSettings NuGet Package is required.

  Instantiating logger using .config file

  var logger = new LoggerConfiguration()
      .ReadFrom.AppSettings()
      .CreateLogger();

  Copy

  Sample web.config file

  <?xml version="1.0" encoding="utf-8"?>
  <configuration>
    <appSettings>
      <add key="serilog:using:NewRelic" value="NewRelic.LogEnrichers.Serilog" />
      <add key="serilog:using:File" value="Serilog.Sinks.File" />
      <!--Add other enrichers here-->
      <add key="serilog:enrich:WithNewRelicLogsInContext" />
      <add key="serilog:write-to:File.path" value="C:\logs\SerilogExample.log.json" />
      <add key="serilog:write-to:File.formatter" value="NewRelic.LogEnrichers.Serilog.NewRelicFormatter, NewRelic.LogEnrichers.Serilog" />
    </appSettings>

  Copy