Your Ad Here

IBM AIX/UNIX system storage administration ksh/perl scripting

Tuesday, March 16, 2010

Command Respawning Too Rapidly From init, wtmp Filling Up




 Technote (FAQ)
Question
How can I find out what command is respawning too rapidly and filling up my wtmp file?

Answer
A common situation is that the init process is repeatedly attempting to start a failing process. The init man page describes what happens when init finds an entry is being respawned:

If the init command finds that it is continuously running an entry in

the /etc/inittab file (more than five times in 225 seconds), it assumes

that an error in the entry command string exists. It then prints an

error message to the console and logs an error in the system error log.

After the message is sent, the entry does not run for 60 seconds. If

the error continues to occur, the command will respawn the entry only

five times every 240 seconds. The init command continues to assume an

error occurred until the command does not respond five times in the

interval, or until it receives a signal from a user. The init command

logs an error for only the first occurrence of the error.
To find out what is being respawned use the steps below.

1. Check the console or console logs

Check on the console to see if init is writing an error message similar to the one below:

0 Thu Jan 22 10:16:27 EST 2009

INIT: Command is respawning too rapidly. Check for possible errors.

id:  xvfb "/usr/bin/X11/X -force -vfb -x abx -x dbe -x GLX :1 &"


Or search through the console log using the alog command:
# alog -t console -o | more
2. Check errpt
Next there may be an entry in the errpt output with the label "INIT_RAPID" like below:

LABEL: INIT_RAPID

IDENTIFIER: 3A30359F


Date/Time:       Wed Jan 28 10:14:17 2009

Sequence Number: 1789

Machine Id:      00CC2F914C00

Node Id:         libgng

Class:           S

Type:            TEMP

Resource Name:   init


Description

SOFTWARE PROGRAM ERROR


Probable Causes

SOFTWARE PROGRAM


User Causes

PERFORMANCE DEGRADED


Recommended Actions

REVIEW DETAILED DATA


Detail Data

SOFTWARE ERROR CODE

Command is respawning too rapidly. Check for possible errors.

COMMAND

id:  xvfb "/usr/bin/X11/X -force -vfb -x abx -x dbe -x GLX :1 &"

Both messages clearly identify the failing command that is being run out of the /etc/inittab file.


3. Check the wtmp file
If the warning messages are not noticed on the system console or in errpt, the next indication of he problem may be that the /var file system is filling up. This is a result of init creating an entry in /var/adm/wtmp file during each attempt to start the problem process. See below for a procedure to format the wtmp file in readable characters for diagnosis.


This technique makes use of the fwtmp utility which is part of the bos.acct fileset.

# lslpp -w /usr/sbin/acct/fwtmp

File                                    Fileset               Type

------------------------------------------------------------------

/usr/sbin/acct/fwtmp                    bos.acct              File



What's In The wtmp File


The actual content of the wtmp is not viewable, as the wtmp entries are written as binary structures (see /usr/include/utmp.h for the format). The fwtmp utility can be used to extract the contents into a human readable format.


For example we redirect the the contents of the /var/adm/wtmp file:


# /usr/sbin/acct/fwtmp < /var/adm/wtmp--> /tmp/wtmp_readable


At quick cat of the /tmp/wtmp_readable file and we find that the

file mainly consists of the following entries:

xvfb   xvfb   5 319596 0000 0000 1078170250    Mon Mar  1 11:44:10 2004

      xvfb   8 319596 0000 0001 1078170250    Mon Mar  1 11:44:10 2004

xvfb   xvfb   5 319598 0000 0000 1078170250    Mon Mar  1 11:44:10 2004

      xvfb   8 319598 0000 0001 1078170250    Mon Mar  1 11:44:10 2004

The first numeric column shows us the ut_type of entry, as defined in the utmp.h header file. The interesting types in our case are:


#define INIT_PROCESS    5    /* Process spawned by "init" */

#define LOGIN_PROCESS   6    /* A "getty" process waiting for login */

#define USER_PROCESS    7    /* A user process */

#define DEAD_PROCESS    8

In this example the "xvfb" entry is being started by init (signified by the "5" in column 3) and in the next line it's dying (ut_type = 8)


A quick check of the inittab file we find our problem:

# grep xvfb /etc/inittab

xvfb:2:respawn:/usr/bin/X11/X -force -vfb -x abx -x dbe -x GLX :1 &



In this case, the xvfb entry was starting an X terminal server daemon.


SOLUTION

The solution would then to resolve the command problem or change the entry in inittab from respawn to off using the chitab utility:

# chitab xvfb:2:off:'/usr/bin/X11/X -force -vfb -x abx -x dbe -x GLX :1 &'
In this specific case the trailing "&" sign was removed from the X Server command and it started up normally.
Historical Number
aix/tty/8


4 comments:

Tekslate Trainings said...

Looking for training on IBM AIX, Check this site for trainings on emerging IT courses . Go here if you’re looking for training on IBM Aix.

黃文彥 said...

真愛旅舍視頻聊天室
真愛旅舍視頻聊天室
真愛旅舍視頻聊天室
成熟女人社區
風情天下社區
戀母論壇
真實媽媽論壇
徐娘社區
柘城QQ交友
全球倫理片網
無碼一級片
中國性健康網
藍玫瑰求包養網
藍玫瑰包養網
都市包養網
三陪美女
三陪女大學生
學生包養網
精英包養網
求包養網
高端交友包養美女網站
心儀交友網
55包養網
紅玫瑰求包養網
九八情侶網
真實富婆包養網
富婆網
久久求包養網
重慶包養網
廣西包養網
柳州求包養
桂林包養學生
愛上求包養網
漾美眉視訊交友
Q女孩真人視訊
寂寞交友富婆聊天室
台灣甜心辣妹視訊
台灣視訊聊天室破解
影音視訊聊天室
真人美女視頻交友
一對一免費視訊聊天室
真愛旅舍視頻聊天室
真愛旅舍視頻聊天室
真愛旅舍視頻聊天室
真愛旅舍視頻聊天室
同城聊天室視頻聊
真人秀視頻聊天室
173免費視訊聊天
s383s 視訊
視訊聊天交友
視訊聊天交友
視訊美女
live173影音視訊live秀
173 live 視訊
金瓶梅影音視訊聊天室
金瓶梅影音視訊聊天室
真愛旅舍視頻聊天室
真愛旅舍視頻聊天室
寂寞交友富婆聊天室
寂寞交友富婆聊天室
寂寞交友富婆聊天室
同城聊天室視頻聊
QQ樂真人視訊交友網

Suseela Susiee said...


This information is impressive; I am inspired with your post writing style & how continuously you describe this topic. After reading your post, thanks for taking the time to discuss this, I feel happy about it and I love learning more about this topic.

Digital Marketing Company in Chennai

Digital Marketing Services in Chennai

Dwarakesh babu said...


Ya that's great to have blog like this it may clear the many thought from this

Pega Training
PlSQL Training
AWS Training
Teradata Training
Qlikview Training
Informatica MDM Training

Labels

BlogCatalog