• /
  • Log in
  • Free account

Forward your logs using the infrastructure agent

You can forward your logs to New Relic using our infrastructure monitoring agent. This makes all of your logging data available in one location and provides deeper visibility into both your application and your platform performance data.

Forwarding your logs to New Relic will give you enhanced log management capabilities to collect, process, explore, query, and alert on your log data. To see how logs in context of your apps and hosts can help you find the root cause of an issue, watch this short video (approx. 3:40 minutes):

Basic process

You can use our guided install process to quickly and easily install log management and infrastructure monitoring together! To get learn how the guided install process works and how to use the logging data you see in New Relic, watch this Nerdlog video on YouTube (14:46 minutes):

To forward your logs through our infrastructure monitoring agent:

  1. If you haven't already, create a New Relic account. It's free, forever.
  2. Verify the system requirements needed for configuring logs.
  3. Ensure you have installed the infrastructure agent, version 1.11.4 or higher. (If you use our guided install in the UI, skip the following steps.)
  4. Create a logging.yml configuration file in the infrastructure agent's logging.d directory.
  5. Configure your log sources and other parameters.
  6. Generate some traffic and wait a few minutes, then check your account for data.
  7. Explore your log data in the Logs UI and benefit from the log attributes automatically inserted by the infrastructure agent.

Here is an example of logs for your host's UI. You can see logs in context of events for the selected time period, and drill down into detailed data for any of the highlighted attributes. To examine even more detailed data, run a query, or click Open in logs.

Screenshot of logs in context for a host

Here is an example of a host's logs in context related to an event.

Enable logging for your on-host integrations

With the infrastructure agent installed, you can enable automatic log parsing and forwarding for our most popular on-host integrations with one step. To enable this feature, rename the on-host-log.yml.example file to on-host-log.yml. Once done, your integration's logs are automatically parsed and sent to New Relic.

This option is available for our supported Linux platforms.

To enable the on-host integration log forwarding feature:

System requirements

To use the log forwarder of the infrastructure agent, make sure you meet the following requirements:

  • Infrastructure agent version 1.11.4 or higher.
  • Fluent Bit. The infrastructure agent already installs the latest version for you. To update or downgrade it to a specfic version, refer to the Fluent Bit installation procedures.
  • OpenSSL library 1.1.0 or higher is required by the infrastructure agent starting from version 1.16.4.
  • Built-in support for ARM64 architecture on Linux systems (for example, AWS Graviton architecture) added in infrastructure agent 1.20.6.

Important

The log forwarding feature is not supported with the Docker container for infrastructure monitoring agents.

The log forwarding feature is compatible with the following operating systems:

Operating system

Supported version

Amazon Linux

Amazon Linux 2

CentOS

Version 7 or higher

Debian

Version 9 ("Stretch") or higher

Exception: Version 11 is not supported.

Red Hat Enterprise Linux (RHEL)

Version 7 or higher

SUSE Linux Enterprise Server (SLES)

Version 12

Ubuntu

Versions 16.04.x, 18.04.x and 20.04.x (LTS versions)

Windows

Windows Server 2012, 2016, 2019, and 2022, and their service packs.

Windows 10

Install the infrastructure agent

Starting with version 1.11.4, the infrastructure agent can forward logs to New Relic. To install and run the agent, use a package manager (Linux) or the MSI installer (Windows).

Important

The log forwarding feature is not included when the infrastructure agent is implemented using Linux tarball or Windows ZIP installations.

To use the following links, make sure you are logged to your New Relic account.

Amazon Linux
Amazon Linux
CentOS
CentOS
Debian
Debian
Red Hat
RHEL
SLES
SLES
Ubuntu
Ubuntu
Windows
Windows

If you don't have a New Relic account yet, or if you prefer to follow the procedure manually, see our tutorial to install the package manager.

Enable log forwarding on agent installed using Linux tarball

Our custom Linux installation process for infrastructure monitoring allows you to tailor all aspects of the installation process, and to place files and folders on your machine. If you choose the assisted or manual tarball installation process, follow these steps to implement the log forwarder feature:

  1. Create the following directories:
  • /var/db/newrelic-infra/newrelic-integrations/logging
  • /etc/newrelic-infra/logging.d
  1. Download and install New Relic's fluent-bit-package (RPM) by running a command similar to:

    bash
    $
    yum localinstall td-agent-bit-<some-version>.rpm`
  2. Download New Relic's fluentbit plugin and save it as /var/db/newrelic-infra/newrelic-integrations/logging/out_newrelic.so.

  3. Dowload or copy the parsers.conf file from this Github repository, and save it as /var/db/newrelic-infra/newrelic-integrations/logging/parsers.conf.

Configure the infrastructure agent

Configuration files describe which log sources are forwarded. Our infrastructure agent uses .yml files to configure logging. You can add as many config files as you want.

To add a new configuration file for the log forwarding feature:

  1. Navigate to the log forwarder configuration folder:

    • Linux: /etc/newrelic-infra/logging.d/
    • Windows: C:\Program Files\New Relic\newrelic-infra\logging.d\
  2. Create a logging.yml configuration file, and add the parameters you need. The logging.d directory has various .yml.example files you can use as a reference or starting point.

The agent automatically processes new configuration files without having to restart the infrastructure monitoring service. The only exception to this is when configuring a custom Fluent Bit configuration.

Log forwarding parameters

The infrastructure log forwarding .yml config supports the following parameters:

Name (required)

To start, define a name of the log or logs you want to forward to New Relic.

Log source (required)

What you use for the log source will depend on where you want to forward your logs from. Available options include:

Optional configuration

The following configuration parameters are not required but are still recommended.

Sample configuration file

Here is an example of a logging.d/ configuration file in YAML format. For more configuration examples, see the infrastructure agent repository.

View your log data

If everything is configured correctly and your data is being collected, you should see logs and related telemetry data in these places:

  • The selected host's Summary page in the New Relic UI: Go to one.newrelic.com > Explorer or Infrastructure > Hosts > (select an entity) > Logs.
  • New Relic's Logs UI
  • New Relic tools for running NRQL queries. For example, you can execute a query like this:
SELECT * FROM Log

Troubleshooting

If you encounter problems with configuring your log forwarder, try these troubleshooting tips.

What's next?

Explore logging data across your platform with our Logs UI.

Disable log forwarding

To disable log forwarding capabilities, go to your logging.d directory, and remove files with the .yml extension that were originally added during the configuration process.

  • Linux: /etc/newrelic-infra/logging.d/
  • Windows: C:\Program Files\New Relic\newrelic-infra\logging.d\
Create issueEdit page
Copyright © 2022 New Relic Inc.