Jump to content

logger Re-starting zrd


Cowboy Denny
 Share

Recommended Posts

This forum topic is my journey figuring out why one of our BIG-IP GTM's (BIG-IP DNS) was disconnected from the sync group

These broadcast messages were blowing up the cli

Broadcast message from systemd-journald@usfnt1slbgtm99.thezah.corp (Wed 2022-02-23 12:18:52 CST):

logger[16928]: Re-starting zrd

The BIG-IP GTM system may occasionally log the following error message to the console and to the /var/log/daemon.log file:

Re-starting zrd
 

tmsh show sys service zrd
zrd                   finish (pid 26149) 0 seconds, 21 starts, 6209 restarts 

The zrd process is the ZoneRunner daemon. The ZoneRunner utility is used to create, manage, and maintain DNS files on the BIG-IP GTM system.

Various issues may cause the zrd process to restart. The most common issues include the following:

  • Syntax errors in the BIND configuration file
  • Syntax errors in the BIND zone files
  • ZoneRunner records do not match BIND zone files

To determine why the zrd process is restarting, perform the following procedure:

  1. Log in to the command line.
  2. If the Re-starting zrd error message is repeatedly logged to the console, you can prevent the zrd process from attempting to restart by typing the following command:

    bigstart stop zrd

  3. After the zrd process halts, use a text reader such as the less command to review the the /var/log/gtm file for error messages related to the named.conf file, or specific zone file.

    For example:

    less /var/log/gtm

    For example, the following log entry indicates the journal file for the test.com zone is out of sync with the zone file. The error message also indicates that the test.com zone was not found. This behavior is caused by the errant space in the zone name in the named.conf file:

    zrd[26232]: 0115020b:3: Errors in config file zone siterequest.com/IN: loaded serial 55. zone m1.test.net/IN: loaded serial 2009020301. db.external.test.com.:3: ignoring out-of-zone data (test.com). zone test.\032com/IN: journal rollforward failed: journal out of sync with zone. externa. l/test. com./IN: not found.
  4. Depending on the error message found in the /var/log/gtm file, you may need to refer to one of the following articles:
  5. After you have fixed the source of the error, restart the zrd process by typing the following command:

    bigstart start zrd

error: zone domain.com/IN/external: journal rollforward failed: journal out of sync with zone

The zrd process exits when it encounters this error message. Additionally, you cannot restart the zrd process until you resolve the error condition.

 

Manually deleting the zone journal files from the BIG-IP DNS/GTM system

FIRST identify what zones you have issues with:

grep named /var/log/daemon.log

Feb 23 12:00:49 usfnt1slbgtm99.thezah.corp err named[13617]: 23-Feb-2022 12:00:49.713 general: error: zone thezah.com/IN/external: journal rollforward failed: journal out of sync with zone Feb 23 12:00:49 usfnt1slbgtm99.thezah.corp err named[13617]: 23-Feb-2022 12:00:49.720 general: error: zone thezah.com/IN/external: not loaded due to errors. 

Feb 23 12:00:49 usfnt1slbgtm99.thezah.corp err named[13617]: 23-Feb-2022 12:00:49.723 general: error: zone int.thezah.com/IN/external: journal rollforward failed: journal out of sync with zone Feb 23 12:00:49 usfnt1slbgtm99.thezah.corp err named[13617]: 23-Feb-2022 12:00:49.723 general: error: zone int.thezah.com/IN/external: not loaded due to errors.

NOTE: You may have too many errors that filled up the latest daemon.log so you may have to search the older files which you can do with: zgrep named /var/log/daemon*

In the above example you can see the zones that are having issues are thezah.com and int.thezah.com

 

Location zone files: ls -ltrh /var/named/config/namedb

ls -ltrh /var/named/config/namedb
total 128K
-rw-r--r--. 1 named named  337 Oct 12  2014 db.external.abc.
-rw-r--r--. 1 named named  350 Sep 12  2018 db.external.int.thezah.com.https.
-rw-r--r--. 1 named named  14K Nov 15 19:48 db.external.int.thezah.com.~
-rw-r--r--. 1 named named 3.3K Nov 15 19:49 db.external.thezah.com.~
-rw-r--r--. 1 named named  984 Nov 15 19:53 db.external.eventguyz.com..jnl
-rw-r--r--. 1 named named 7.5K Nov 15 19:53 db.external.thezah.corp..jnl
-rw-r--r--. 1 named named 1.7K Nov 15 19:53 db.external.cowboydenny.com..jnl
-rw-r--r--. 1 named named 4.1K Nov 15 20:04 db.external.thezah.corp.
-rw-r--r--. 1 named named  469 Nov 15 20:05 db.external.eventguyz.com.
-rw-r--r--. 1 named named  881 Nov 15 20:05 db.external.cowboydenny.com.
-rw-r--r--. 1 named named  22K Nov 16 21:22 db.external.int.thezah.com..jnl
-rw-r--r--. 1 named named  11K Nov 16 21:22 db.external.thezah.com..jnl
-rw-r--r--. 1 named named 3.4K Nov 16 21:33 db.external.thezah.com.
-rw-r--r--. 1 named named  14K Nov 16 21:34 db.external.int.thezah.com.

 

Stop ZonerRunner: tmsh stop /sys service zrd

Stop named: tmsh stop /sys service named

Remove the journal files (.jnl) for the affected zones

rm /var/named/config/namedb/db.external.thezah.com..jnl

rm /var/named/config/namedb/db.external.int.thezah.com..jnl

Start services backup

tmsh start /sys service named
tmsh start /sys service zrd

Link to comment
Share on other sites

 Share

×
×
  • Create New...