No Bottlenecks

5 Simple Commands to Remove DB2 Cleanly

2009-03-02T22:45:00.000-08:00

Before you install new version of DB2, if you need to remove the old version, please perform the following steps:

1. Log in as root.
2. List all DB2 instances:
bash# //instance/db2ilist

3. Drop each instance listed in the previous step with the following command:
bash# //instance/db2idrop instance-name

4. Drop the DB2 administration server:
bash# //instance/dasidrop `//instance/dasilist`

5. Uninstall all of the DB2 packages on your system using the db2deinstall command on your DB2 CD-ROM or from the unpackaged directory:
bash# /mnt/cdrom/db2/db2_deinstall -n

Actual example:

staff1@pw70:/opt/IBM/db2/V9.5/instance>./db2idrop db2inst1
DBI1070I Program db2idrop completed successfully.

staff1@pw70:/opt/IBM/db2/V9.5/instance>./db2idrop db2ins91
DBI1070I Program db2idrop completed successfully.

staff1@pw70:/opt/IBM/db2/V9.5/instance>daslist
dasusr

staff1@pw70:/opt/IBM/db2/V9.5/instance>dasdrop dasusr
SQL4410W The DB2 Administration Server is not active.
DBI1070I Program dasdrop completed successfully.

staff1@pw70:/data/software/db2v953a/server>./db2_deinstall -a

Enter full path name for the install directory -
------------------------------------------------
/opt/IBM/db2/V9.5
DBI1016I Program db2_deinstall is performing uninstallation. Please
wait.
The execution completed successfully.

.pw.

How to kill hanging Windows remote desktop connection

2009-02-07T15:59:00.000-08:00

When you don't log off Windows Remote Desktop (or terminal service client), the connection becomes
orphaned. You may get the following screen:

Too many sessions prevents future logins.  Do the following to clean up connections (on Windows 2003 server).
From a client machines CMD windows:

D:>query session /SERVER:myserver.myco.com
SESSIONNAME       USERNAME                 ID  STATE   TYPE        DEVICE
console                                     0  Conn    wdcon
rdp-tcp                                 65536  Listen  rdpwd
rdp-tcp#11        Administrator             1  Active  rdpwd
rdp-tcp#4         Administrator             2  Active  rdpwd

The above show 2 connections to “myserver”. You can kill the sessions with ID 1, 2, 3, so on.

tsdiscon 1 /SERVER:myserver.myco.com

where 1  rdp-tcp#11        Administrator             1  Active  rdpwd “ID” col
Also, can tskill to remotely stop a process.

Scheduling Linux Tasks

2009-02-07T15:48:00.000-08:00

automation and scheduling:

The at command enables the simplest form of job scheduling. It runs a command or a series of commands once only at a specified time, and mails any output to the user. The jobs it runs are processed in the background. You use this syntax for the at command. In the syntax, you can specify the time in a number of different ways. For example, you can specify the time as 01:30 or as 1:30 am.

[root@host11 pwstaff]# at -m 01:30

You now want to schedule a series of commands that you’ve saved in a file named “system_maintenance” to run at 12:00 pm. To do this, you use the -f option to Specify the file in an at command.

[root@host11 pwstaff]# at -f system_maintenance 12:00

Suppose that it is the end of June and you want to schedule a job to run four weeks from today, without entering the exact date on which it must run. Which command do you enter before specifying the job that must run to do this?

at now + 4 weeks

Ordinary users use the crontab command to create or edit their crontab files. To use the crontab command, users must be listed in the file /etc/cron.allow. If that file doesn’t exist, the users must not be listed in /etc/cron.deny. If neither file exists, all users can use the crontab command unless some other configuration prevents this.

[root@samwise root]# crontab -l

no crontab for root

[root@samwise root]# crontab -e to create

An entry of 00 in the minute field, 20 in the hour field, and 1-5 in the day of week field specifies that a command must run at 8:00 pm every day, from Monday to Friday.

00 20 * * 1-5 sort/usr/sales

In order, the fields that make up the time pattern in a crontab file entry include minute, hour, day of month, month of year, and day of week. Examples:

0 22 * * 1-5 10:00 pm on weekdays
15 14 1 * * 2:15 pm on the first of every month
30 /2 * * * Half past the hour every two hours
30 19 * * 1 7:30 pm every Monday

The /etc/at.allow and /etc/at.deny files enable you to control which users can run the at command. When a user tries to run an at job, the at command checks to see if the /etc/at.allow file exists. If it does and the user is listed in the file, at schedules the job.

If the user is not listed in /etc/at.allow, they will not be able to schedule the job, irrespective of whether they are listed in the /etc/at.deny file.If the /etc/at.allow file does not exist, at checks the /etc/at.deny file. If the user’s name does not appear in this file, the job will be scheduled. If the /etc/at.allow file doesn’t exist and /etc/at.deny is empty, any user can run at jobs. Alternatively, if /etc/at.allow exists and is empty, or if neither file exists, no user except root can run at jobs.

In some Linux distributions, such as Red Hat 7, any user can run cron jobs if neither the /etc/cron.allow or /etc/cron.deny files exist. In other distributions, only root can run cron jobs if neither file exists.

How to export NFS share and mount it on a client (AIX)

2009-02-05T18:04:00.000-08:00

bash-3.00# exportfs
/data/fs/fs1 -rw=pw41,access=pw41.test.pw.com,root=pw41.test.pw.com

NOTE: the exported directory (e.g., /data/fs/fs1) must be a file system itself, not just any directory.
In this case, /data/fs/fs1 is a file system itself, but /data is not. So if you export /data, when you
mount it from client, it will not work (you will get an error).

bash-3.00# exportfs -a

This commands exports (or freshes) the list again from the server. Man page for exportfs shows this:
Flags

-a Exports all directories listed in the exports file.
-v Prints the name of each directory as it is exported or unexported.

To verify what’s been exported, run:
bash-3.00# exportfs
/cesa/fs/fs1 -rw=pw41,access=pw41.test.pw.com,root=pw41.test.pw.com

From client side, you can run showmount to list what’s exported:
bash-3.00# showmount -a pw40
pw41.test.pw.com:/data/fs/fs1

Then you mount it to an existing directory (create it if it doesn’t exist yet):

bash-3.00# mount pw40:/data/fs/fs1 /data/clientfs
bash-3.00# cd /data/clientfs
bash-3.00# ls

DB2 V9.1: How to Re-create a Instance

2009-02-05T18:00:00.000-08:00

Like the Nissan Pathfinder’s ad: “not that you would, but you could”. In certain performance test conditions, you may have to drop the current DB2 V8 instance, install V9.1, and re-create a instance. Here is a short list of helpful commands - run as root unless specifically sudo’ed:

1. Drop DB2 instance

/opt/IBM/db2/V9.1/instance/db2idrop -f db2inst1
OR version 8
/usr/opt/db2_08_01/instance/db2idrop -f db2inst1

2. Uninstall previous db2

Stop and drop the DB2 Admin server first.

bash-3.00# su dasusr
bash-3.00$ pwd
/home/dasusr/das/bin
bash-3.00$ ./db2admin stop
SQL4407W The DB2 Administration Server was stopped successfully.

Drop the DAS server as root:

bash-3.00# pwd
/opt/IBM/db2/V9.1/instance
bash-3.00# ./dasdrop
SQL4410W The DB2 Administration Server is not active.
DBI1070I Program dasdrop completed successfully.

Then db2_deinstall UDB:

bash-3.00# pwd
/opt/IBM/db2/V9.1/install
bash-3.00# ./db2_deinstall -a
DBI1016I Program db2_deinstall is performing uninstallation.
Please wait.
The execution completed successfully.

3. Run db2 V9.1 install, db2_install will install at console (non-GUI).

4. After install, create an instance

Make sure db2fence1 and db2inst1 directories created under /home and with correct
permissions:
drwxr-xr-x 2 db2fenc1 db2admi 256 May 04 15:19 db2fenc1
drwxr-xr-x 2 db2inst1 db2admi 256 May 04 15:19 db2inst1
then

/opt/IBM/db2/V9.1/instance/db2icrt -u db2fenc1 db2inst1

bash-3.00# ./db2icrt -u db2fenc1 db2inst1
DBI1070I Program db2icrt completed successfully.

ORA-12705: Cannot access NLS data files or invalid environment specified

2009-02-05T17:59:00.000-08:00

This tip resolves Oracle client error: “ORA-12705: Cannot access NLS data files or invalid environment specified“. According to Oracle documentation, “ORA-12705″ can happen under the following conditions:

1. ORA-12705 with incorrect NLS_LANG parameter
2. ORA-12705 with incorrectly specified ORA_NLSx
3. ORA-12705 from incorrect Oracle install or configuration
4. ORA-12705 when using Special Character sets
5. ORA-12705 when connecting with SQL*Net
6. ORA-12705 during migrations
7. ORA-12705 when connecting with SQL*Plus
8. ORA-12705 when connecting with language pre-compilers
9. ORA-12705 during Export/Import

Try doing the following to fix this error on the client side. On the client Windows machine (this tip
is only for Windows), The NLS_LANG must be unset in the Windows registry (re-naming works the best because you know what you changed). Look for the NLS_LANG subkey in the registry at \HKEY_LOCAL_MACHINE\SOFTWARE\ORACLE, and rename it. Then try SQLPlus again.

How to "tar" Only Directories, Not Files

2009-02-05T17:57:00.000-08:00

Sometimes you want to preserve only a directory structure into a tar ball without having to archive the files themselves. This can happen if you want to backup a pre-defined directory structure, run some tests, and do not really care about the files created during the test. You can use the UNIX commands ‘find’ and ‘tar’ to do that.

# find bwtest -type d > dirlist.txt
# more dirlist.txt
bwtest
bwtest/dd1
bwtest/dd1/1
bwtest/dd1/458986
bwtest/dd1/217128
bwtest/dd1/217136
bwtest/dd1/217182
bwtest/dd2
bwtest/dd2/2
bwtest/dd3
bwtest/dd3/3
bwtest/reports
# tar -cvf ./bwtest.tar -L dirlist.txt

content.tar will contain only the directories, no files. This should work on
all UNIX and Linux platforms.

Simple Commands To Backup and Restore a Oracle Database

2009-02-05T17:55:00.000-08:00

Below shows simple steps in Oracle 9i and 10g to backup and restore Oracle databases; the goal is for the tasks to be called remotely so they can be automated as part of a performance test execution (i.e., always restores a known baseline before start a test!) Applies to Windows (steps apply to other platform with minor modifications).

1) Shuts down the listener:

F:\_backup>lsnrctl stop

LSNRCTL for 32-bit Windows: Version 10.2.0.3.0 - Production on 23-MAY-2007 09:49:20

Copyright (c) 1991, 2006, Oracle. All rights reserved.

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=pw213)(PORT=1521)))
The command completed successfully

2) Shuts down the instance:

F:\_backup>sqlplus /nolog

SQL*Plus: Release 10.2.0.3.0 - Production on Wed May 23 09:49:27
2007

Copyright (c) 1982, 2006, Oracle. All Rights Reserved.

SQL> connect sys as sysdba
Enter password:
Connected.
SQL> shutdown immediate
Database closed.
Database dismounted.
ORACLE instance shut down.
SQL> quit
Disconnected from Oracle Database 10g Enterprise Edition Release 10.2.0.3.0 - Production
With the Partitioning, OLAP and Data Mining options

3) Use operating system “copy” command to copy the entire “oradata” directory to your backup location:

F:\_backup>xcopy /Y/E D:\oradata oradata_2007_05_21

Copies all oradata to the
“oradata_2007_05_21″ directory.

4) To restore, repeat steps 1) and 2), then copy back the files from “oradata_2007_05_21″ to your run-time “oradata” directory.

5) Start up listener:

F:\_backup>lsnrctl start

6) Start up instance:

F:\_backup>sqlplus /nolog

SQL*Plus: Release 10.2.0.3.0 - Production on Wed May 23 09:49:27 2007

Copyright (c) 1982, 2006, Oracle. All Rights Reserved.

SQL> connect sys as sysdba
Enter password:
Connected to an idle instance.
SQL> startup

5 Steps To Increase Storage for Your Existing Logical Volume (AIX)

2009-02-05T17:48:00.000-08:00

Steps taken to to increase the storage for an existing logical volume. The following commands
are for AIX. The steps, however, are universal. You only need to substitute the commands for other platforms.

1) How big is the vg?

/home/oracle $ lsvg oradata_vg
VOLUME GROUP: oradata_vg VG IDENTIFIER: 00224ada00004c000000010f64352701
VG STATE: active PP SIZE: 128 megabyte(s)
VG PERMISSION: read/write TOTAL
PPs: 1086 (139008 megabytes)
MAX LVs: 256 FREE PPs: 685 (87680 megabytes)
LVs: 2 USED PPs: 401 (51328 megabytes)
OPEN LVs: 2 QUORUM: 2
TOTAL PVs:
2 VG DESCRIPTORS: 3
STALE PVs: 0 STALE PPs: 0
ACTIVE PVs: 2 AUTO ON: yes
MAX PPs per VG: 32512
MAX PPs per PV: 1016 MAX PVs: 32
LTG size (Dynamic): 128
kilobyte(s) AUTO SYNC: no
HOT SPARE: no BB POLICY: relocatable

2) Use lspv to find out which hdisk belong to which vg:

/home/oracle $ lspv
hdisk2 00224ada0abc41b2 rootvg active
hdisk0 00224adaee4d5b0e None
hdisk1 00224adaa0d19b45 None
hdisk3 00224ada432ae745 oradata_vg active
hdisk4 00224adaf645417c oradata_vg active
hdisk5 00224adafc227f17 udbdata_vg active
hdisk6 00224adafc22d1ed udbdata_vg active
hdisk7 00224adafc3ab232 udbdata_vg active
hdisk8 00224adafc3ab64b udbdata_vg active
hdisk9 0003321524243690 udbdata_vg active
hdisk10 00033215242458d4 udbdata_vg active
hdisk11 00224adab4c1999c pagevg active
hdisk12 00224adab4c199c9 oracle_vg active
hdisk13 00224adafc20bbc5 oracle_vg active
hdisk14 00224adafc20bed6 oracle_vg active

hdisk3 and hdisk4 belong to oradata_vg.

3) how large is each hdisk?

/home/oracle $ lspv hdisk3
PHYSICAL VOLUME: hdisk3 VOLUME GROUP: oradata_vg
PV IDENTIFIER: 00224ada432ae745 VG IDENTIFIER 00224ada00004c000000010f64352701
PV STATE: active
STALE PARTITIONS:
0 ALLOCATABLE: yes
PP SIZE: 128 megabyte(s) LOGICAL VOLUMES: 2
TOTAL PPs: 543 (69504 megabytes) VG DESCRIPTORS: 2
FREE PPs: 342 (43776 megabytes) HOT SPARE: no
USED PPs: 201 (25728 megabytes) MAX REQUEST: 128 kilobytes
FREE DISTRIBUTION: 108..00..17..108..109
USED DISTRIBUTION: 01..109..91..00..00
/home/oracle $
lspv hdisk4
PHYSICAL VOLUME: hdisk4 VOLUME GROUP: oradata_vg
PV IDENTIFIER: 00224adaf645417c VG IDENTIFIER 00224ada00004c000000010f64352701
PV STATE: active
STALE PARTITIONS: 0 ALLOCATABLE: yes
PP SIZE: 128 megabyte(s) LOGICAL VOLUMES: 1
TOTAL PPs: 543 (69504 megabytes) VG DESCRIPTORS: 1
FREE
PPs: 343 (43904 megabytes) HOT SPARE: no
USED PPs: 200 (25600 megabytes) MAX REQUEST: 128 kilobytes
FREE DISTRIBUTION: 109..00..17..108..109
USED DISTRIBUTION: 00..109..91..00..00

so total about 69G _ 69G = 140G.

4) now, how large is the oradata_lv currently?

/home/oracle $ lslv oradata_lv
LOGICAL VOLUME: oradata_lv VOLUME GROUP: oradata_vg
LV IDENTIFIER: 00224ada00004c000000010f64352701.1 PERMISSION: read/write
VG
STATE: active/complete LV STATE: opened/syncd
TYPE: jfs2 WRITE VERIFY: off
MAX LPs: 512 PP SIZE: 128 megabyte(s)
COPIES: 1 SCHED POLICY:
striped
LPs: 400 PPs: 400
STALE PPs: 0 BB POLICY: relocatable
INTER-POLICY: maximum RELOCATABLE: no
INTRA-POLICY: middle UPPER BOUND: 2
MOUNT POINT:
/oradata LABEL: /oradata
MIRROR WRITE CONSISTENCY: on/ACTIVE
EACH LP COPY ON A SEPARATE PV ?: yes (superstrict)
Serialize IO ?: NO
STRIPE WIDTH: 2
STRIPE SIZE: 128K

looks like LPs=400 each is 128M = 51GB.
Still available 140G - 51GB = 90GB.

and from the lsvg, we know that:

VG PERMISSION: read/write TOTAL PPs: 1086 (139008 megabytes)
MAX LVs: 256 FREE
PPs: 685 (87680 megabytes)
LVs: 2 USED PPs: 401 (51328 megabytes)

5a) unmount the fs first
5b) change the logical volume in smitty - increase MAXIMUM NUMBER OF LOGICAL PARTITIONS to 1086 - ran OK
5c) increase size of lv, enter number of ADDITIONAL LPs: to 684 (not 685 because stripped volumes require even numbers; also don’t choose PVs) - ran OK
5d) change file system, Change/Show characteristics of an Enhanced JFS; set the new
“number of units (of 512)” to 283978437 - ran OK
5e) check the new fs:

6) Validate everything:

bash-2.05b# df -k
Filesystem 1024-blocks Free %Used Iused %Iused Mounted on
/dev/oradata_lv 142082048 106387720 26% 22 1% /oradata

the logical volume is now 142GB! It worked.

A Picture of the typical IO Stack

2009-02-05T17:44:00.000-08:00

I/O stack components Overview

Typically the stack consists of the following
• Database
• Character or block device driver
• Multipathing driver
• SCSI driver
• HBA
• System Bus
• Switch/iSCSI routers
• Storage Array
• Disks

Example of vendors:

ASM-Database IBM Oracle
Character or block device driver
Multipathing driver – PowerPath, Windows MPIO
SCSI driver
HBA – Host Bus Adapter, Qlogic (IBM), Emulex
System Bus - PCI
Switch/iSCSI routers – Brocade, McData
Storage Array – IBM DS8100, EMC, HDS, HP
Disks - Seagate

The database processes makes I/O calls to the block device. The I/O scheduler in Linux forms the
interface between the generic block layer and the low level device drivers. The block layer provides
functions that are utilized by the file systems, databases (10gR2) and the virtual memory manager to submit I/O requests to block devices. These requests are transformed by the I/O scheduler and made available to the low-level device drivers. The device drivers consume the transformed requests and forward them (by using device specific protocols) to the actual device controllers that perform the I/O operations.

IOPS vs. Mbytes/sec

2009-02-05T17:39:00.000-08:00

IOPS

• The standard unit of measurement for I/O operations per second. Includes all reads and writes.
• This is how you rate a I/O stack components ability to process small block random I/O requests.
• Used to describe I/O rate driven applications (OLTP, random I/O ) Mbytes/s

Mega”Bytes” per sec

• Used to measure large block sequential transfer rates, with no response time characterization
• Used to describe data rate driven applications (DSS, OLAP)
• Transaction/s – its anything you claim it to be.

Oracle query to gather this data from your database:

set lines 250
set pages 9999
spool sysmetric_outp.log
alter session set nls_date_format=’dd-mm-yyyy hh24:mi’;
select min(begin_time), max(end_time),
sum(case metric_name when ‘Physical Read Total Bytes Per Sec’ then average end)
Physical_Read_Total_Bps,
sum(case metric_name when ‘Physical Write Total Bytes Per Sec’ then average end)
Physical_Write_Total_Bps,
sum(case metric_name when ‘Redo Generated Per Sec’ then average end)
Redo_Bytes_per_sec,
sum(case metric_name when ‘Physical Read Total IO Requests Per Sec’ then average end)
Physical_Read_IOPS,
sum(case metric_name when ‘Physical Write Total IO Requests Per Sec’ then average end)
Physical_write_IOPS,
sum(case metric_name when ‘Redo Writes Per Sec’ then average end) Physical_redo_IOPS,
sum(case metric_name when ‘Current OS Load’ then average end) OS_LOad,
sum(case metric_name when ‘CPU Usage Per Sec’ then average end)
DB_CPU_Usage_per_sec,
sum(case metric_name when ‘Host CPU Utilization (%)’ then average end) Host_CPU_util, –
NOTE 100% = 1 loaded RAC node
sum(case metric_name when ‘Network Traffic Volume Per Sec’ then average end)
Network_bytes_per_sec,
snap_id
from dba_hist_sysmetric_summary
group by snap_id
order by snap_id;
spool off

[source: CERN]

How Much IO Does Your Application Generate?

2009-02-05T17:31:00.000-08:00

IO rate of 1+ Gigabytes/sec is rarely achievable, but hundreds of Megabytes/sec is achieved in lots of high-end production environment. Keep those numbers in mind when planning for what’s possible.

In Oracle, use AWR reports to determine current I/O metrics (Instance Activity Stats per sec).

• IOPS = “physical reads total I/O requests” + “physical writes total I/O requests”
• MBytes/s = “physical reads total bytes” + physical writes total bytes”
• For RAC environments - aggregate IOPS or MBytes/s for all nodes

Now the tricky part – estimating expected I/O throughput rates

• Establish what is being expanded – more users, new reports
• Pull I/O stats for a given session or transaction and extrapolate to the required number of users.
• Ensure data points are representative of the new workload
• Know when your Web users get on your system (e.g., peak hour transactions)

For new or non-existing applications, use business rules or data model transaction profiles flow to understand “what is a transaction”, and then extrapolate for transactions/s or per hour. Optionally you can use the numbers we have seen in our consulting gigs. Note that these are just guideline values.
Use the following as basic guidelines for OLTP (Online transaction processing) systems :

Low transaction system – 1000 IOPS or 200MBytes/s
Medium transaction system – 5000 IOPS or 600 Mbytes/s
High-end transaction system – 10,000 IOPS or 1Gbytes/s (almost rarely achievable and usually TPC-C type workloads)

Use the following as basic guidelines for DSS (Decision Support Systems) (units are in Gig = Gigabytes/Sec):

2 Gig I/O entry level DW
4-8 Gig I/O High End DW
10+ Gig I/O Very High END DW not yet seen in production !
40+ Gig I/O High End TPC-H

Common Linux Commands By Examples

2009-02-05T17:24:00.000-08:00

These are usually invoked by root and used for system maintenance or emergency filesystem repairs. Use with caution, as some of these commands may damage your system if misused.

Users and Groups

users

Show all logged on users. This is the approximate equivalent of who -q.

groups

Lists the current user and the groups she belongs to. This corresponds to the $GROUPS internal variable, but gives the group names, rather than the numbers.

bash$ groups

bozita cdrom cdwriter audio xgrp

bash$ echo $GROUPS

chown, chgrp

The chown command changes the ownership of a file or files. This command is a useful method that root can use to shift file ownership from one user to another. An ordinary user may not change the ownership of files, not even her own files.

root# chown bozo *.txt

The chgrp command changes the group ownership of a file or files. You must be owner of the file(s) as well as a member of the destination group (or root) to use this operation.

chgrp –recursive dunderheads *.data

#  The “dunderheads” group will now own all the “*.data” files

#+ all the way down the $PWD directory tree (that’s what “recursive” means).

useradd, userdel

The useradd administrative command adds a user account to the system and creates a home directory for that particular user, if so specified. The corresponding userdel command removes a user account from the system and deletes associated files.

NOTE	The adduser command is a synonym for useradd and is usually a symbolic link to it.

The id command lists the real and effective user IDs and the group IDs of the current user. This is the counterpart to the $UID, $EUID, and $GROUPS internal Bash variables.

bash$ id

uid=501(bozo) gid=501(bozo) groups=501(bozo),22(cdrom),80(cdwriter),81(audio)

bash$ echo $UID

who

Show all users logged on to the system.

bash$ who

bozo  tty1     Apr 27 17:45

 bozo  pts/0    Apr 27 17:46

 bozo  pts/1    Apr 27 17:47

 bozo  pts/2    Apr 27 17:49

The -m gives detailed information about only the current user. Passing any two arguments to who is the equivalent of who -m, as in who am i or who The Man.

bash$ who -m

localhost.localdomain!bozo  pts/2    Apr 27 17:49

whoami is similar to who -m, but only lists the user name.

bash$ whoami

bozo

Show all logged on users and the processes belonging to them. This is an extended version of who. The output of w may be piped to grep to find a specific user and/or process.

bash$ w | grep startx

bozo  tty1     -                 4:22pm  6:41   4.47s  0.45s  startx

logname

Show current user’s login name (as found in /var/run/utmp). This is a near-equivalent to whoami, above.

bash$ logname

bozo

bash$ whoami

bozo

However…

bash$ su

Password: ……

bash# whoami

root

bash# logname

bozo

Runs a program or script as a substitute user. su rjones starts a shell as user rjones. A naked su defaults to root.

sudo

Runs a command as root (or another user). This may be used in a script, thus permitting a regular user to run the script.

#!/bin/bash

# Some commands.

sudo cp /root/secretfile /home/bozo/secret

# Some more commands.

The file /etc/sudoers holds the names of users permitted to invoke sudo.

passwd

Sets or changes a user’s password.

The passwd can be used in a script, but should not be.

#!/bin/bash

#  set-new-password.sh: Not a good idea.

#  This script must be run as root,

#+ or better yet, not run at all.

ROOT_UID=0         # Root has $UID 0.

E_WRONG_USER=65    # Not root?

if [ “$UID” -ne “$ROOT_UID” ]

then

  echo; echo “Only root can run this script.”; echo

  exit $E_WRONG_USER

else

  echo; echo “You should know better than to run this script, root.”

fi

username=bozo

NEWPASSWORD=security_violation

echo “$NEWPASSWORD” | passwd –stdin “$username”

#  The ‘–stdin’ option to ‘passwd’ permits

#+ getting new password from stdin (or a pipe).

echo; echo “User $username’s password changed!”

# Using the ‘passwd’ command in a script is dangerous.

exit 0

Show users’ logged in time, as read from /var/log/wtmp. This is one of the GNU accounting utilities.

bash$ ac

        total       68.08

last

List last logged in users, as read from /var/log/wtmp. This command can also show remote logins.

newgrp

Change user’s group ID without logging out. This permits access to the new group’s files. Since users may be members of multiple groups simultaneously, this command finds little use.

Terminals

tty

Echoes the name of the current user’s terminal. Note that each separate xterm window counts as a different terminal.

bash$ tty

/dev/pts/1

stty

Shows and/or changes terminal settings. This complex command, used in a script, can control terminal behavior and the way output displays. See the info page, and study it carefully.

Example 13-1. setting an erase character

#!/bin/bash

# erase.sh: Using “stty” to set an erase character when reading input.

echo -n “What is your name? “

read name                      # Try to erase characters of input.

                               # Won’t work.

echo “Your name is $name.”

stty erase ‘#’                 # Set “hashmark” (#) as erase character.

echo -n “What is your name? “

read name                      # Use # to erase last character typed.

echo “Your name is $name.”

exit 0

Example 13-2. secret password: Turning off terminal echoing

#!/bin/bash

echo

echo -n “Enter password “

read passwd

echo “password is $passwd”

echo -n “If someone had been looking over your shoulder, “

echo “your password would have been compromised.”

echo && echo  # Two line-feeds in an “and list”.

stty -echo    # Turns off screen echo.

echo -n “Enter password again “

read passwd

echo

echo “password is $passwd”

echo

stty echo     # Restores screen echo.

exit 0

A creative use of stty is detecting a user keypress (without hitting ENTER).

Example 13-3. Keypress detection

#!/bin/bash

# keypress.sh: Detect a user keypress (”hot keyboard”).

echo

old_tty_settings=$(stty -g)   # Save old settings.

stty -icanon

Keypress=$(head -c1)          # or $(dd bs=1 count=1 2> /dev/null)

                              # on non-GNU systems

echo

echo “Key pressed was ”"$Keypress””.”

echo

stty “$old_tty_settings”      # Restore old settings.

# Thanks, Stephane Chazelas.

exit 0

terminals and modes Normally, a terminal works in the canonical mode. When a user hits a key, the resulting character does not immediately go to the program actually running in this terminal. A buffer local to the terminal stores keystrokes. When the user hits the ENTER key, this sends all the stored keystrokes to the program running. There is even a basic line editor inside the terminal.

bash$ stty -a

speed 9600 baud; rows 36; columns 96; line = 0;

 intr = ^C; quit = ^; erase = ^H; kill = ^U; eof = ^D; eol = ; eol2 = ;

 start = ^Q; stop = ^S; susp = ^Z; rprnt = ^R; werase = ^W; lnext = ^V; flush = ^O;

…

 isig icanon iexten echo echoe echok -echonl -noflsh -xcase -tostop -echoprt

Using canonical mode, it is possible to redefine the special keys for the local terminal line editor.

bash$ cat > filexxx

whaIfoo barhello world

bash$ cat filexxx

hello world

bash$ bash$ wc -c <>

The process controlling the terminal receives only 13 characters (12 alphabetic ones, plus a newline), although the user hit 26 keys.

In non-canonical (“raw”) mode, every key hit (including special editing keys such as ctl-H) sends a character immediately to the controlling process.

The Bash prompt disables both icanon and echo, since it replaces the basic terminal line editor with its own more elaborate one. For example, when you hit ctl-A at the Bash prompt, there’s no ^A echoed by the terminal, but Bash gets a \1 character, interprets it, and moves the cursor to the begining of the line.

tset

Show or initialize terminal settings. This is a less capable version of stty.

bash$ tset -r

Terminal type is xterm-xfree86.

 Kill is control-U (^U).

 Interrupt is control-C (^C).

setserial

Set or display serial port parameters. This command must be run by root user and is usually found in a system setup script.

# From /etc/pcmcia/serial script:

IRQ=`setserial /dev/$DEVICE | sed -e ’s/.*IRQ: //’`

setserial /dev/$DEVICE irq 0 ; setserial /dev/$DEVICE irq $IRQ

getty, agetty

The initialization process for a terminal uses getty or agetty to set it up for login by a user. These commands are not used within user shell scripts. Their scripting counterpart is stty.

mesg

Enables or disables write access to the current user’s terminal. Disabling access would prevent another user on the network to write to the terminal.

TIP	It can be very annoying to have a message about ordering pizza suddenly appear in the middle of the text file you are editing. On a multi-user network, you might therefore wish to disable write access to your terminal when you need to avoid interruptions.

wall

This is an acronym for “write all”, i.e., sending a message to all users at every terminal logged into the network. It is primarily a system administrator’s tool, useful, for example, when warning everyone that the system will shortly go down due to a problem (see Example 17-2).

bash$ wall System going down for maintenance in 5 minutes!

Broadcast message from bozo (pts/1) Sun Jul  8 13:53:27 2001…

 System going down for maintenance in 5 minutes!

NOTE	If write access to a particular terminal has been disabled with mesg, then wall cannot send a message to it.

dmesg

Lists all system bootup messages to stdout. Handy for debugging and ascertaining which device drivers were installed and which system interrupts in use. The output of dmesg may, of course, be parsed with grep, sed, or awk from within a script.

bash$ dmesg | grep hda

Kernel command line: ro root=/dev/hda2

 hda: IBM-DLGA-23080, ATA DISK drive

 hda: 6015744 sectors (3080 MB) w/96KiB Cache, CHS=746/128/63

 hda: hda1 hda2 hda3 <> hda4

Information and Statistics

uname

Output system specifications (OS, kernel version, etc.) to stdout. Invoked with the -a option, gives verbose system info (see Example 12-4). The -s option shows only the OS type.

bash$ uname -a

Linux localhost.localdomain 2.2.15-2.5.0 #1 Sat Feb 5 00:13:43 EST 2000 i686 unknown

bash$ uname -s

Linux

arch

Show system architecture. Equivalent to uname -m. See Example 10-25.

bash$ arch

i686

bash$ uname -m

i686

lastcomm

Gives information about previous commands, as stored in the /var/account/pacct file. Command name and user name can be specified by options. This is one of the GNU accounting utilities.

lastlog

List the last login time of all system users. This references the /var/log/lastlog file.

bash$ lastlog

root          tty1                      Fri Dec  7 18:43:21 -0700 2001

 bin                                     **Never logged in**

 daemon                                  **Never logged in**

…

 bozo          tty1                      Sat Dec  8 21:14:29 -0700 2001

bash$ lastlog | grep root

root          tty1                      Fri Dec  7 18:43:21 -0700 2001

CAUTION

This command will fail if the user invoking it does not have read permission for the /var/log/lastlog file.

lsof

List open files. This command outputs a detailed table of all currently open files and gives information about their owner, size, the processes associated with them, and more. Of course, lsof may be piped to grep and/or awk to parse and analyze its results.

bash$ lsof

COMMAND    PID    USER   FD   TYPE     DEVICE    SIZE     NODE NAME

 init         1    root  mem    REG        3,5   30748    30303 /sbin/init

 init         1    root  mem    REG        3,5   73120     8069 /lib/ld-2.1.3.so

 init         1    root  mem    REG        3,5  931668     8075 /lib/libc-2.1.3.so

 cardmgr    213    root  mem    REG        3,5   36956    30357 /sbin/cardmgr

…

strace

Diagnostic and debugging tool for tracing system calls and signals. The simplest way of invoking it is strace COMMAND.

bash$ strace df

execve(”/bin/df”, [”df”], [/* 45 vars */]) = 0

 uname({sys=”Linux”, node=”bozo.localdomain”, …}) = 0

 brk(0)                                  = 0×804f5e4

…

This is the Linux equivalent of truss.

free

Shows memory and cache usage in tabular form. The output of this command lends itself to parsing, using grep, awk or Perl. The procinfo command shows all the information that free does, and much more.

bash$ free

                total       used       free     shared    buffers     cached

   Mem:         30504      28624       1880      15820       1608       16376

   -/+ buffers/cache:      10640      19864

   Swap:        68540       3128      65412

To show unused RAM memory:

bash$ free | grep Mem | awk ‘{ print $4 }’

procinfo

Extract and list information and statistics from the /proc pseudo-filesystem. This gives a very extensive and detailed listing.

bash$ procinfo | grep Bootup

Bootup: Wed Mar 21 15:15:50 2001    Load average: 0.04 0.21 0.34 3/47 6829

lsdev

List devices, that is, show installed hardware.

bash$ lsdev

Device            DMA   IRQ  I/O Ports

 ————————————————

 cascade             4     2

 dma                          0080-008f

 dma1                         0000-001f

 dma2                         00c0-00df

 fpu                          00f0-00ff

 ide0                     14  01f0-01f7 03f6-03f6

…

Show (disk) file usage, recursively. Defaults to current working directory, unless otherwise specified.

bash$ du -ach

1.0k    ./wi.sh

 1.0k    ./tst.sh

 1.0k    ./random.file

 6.0k    .

 6.0k    total

Shows filesystem usage in tabular form.

bash$ df

Filesystem           1k-blocks      Used Available Use% Mounted on

 /dev/hda5               273262     92607    166547  36% /

 /dev/hda8               222525    123951     87085  59% /home

 /dev/hda7              1408796   1075744    261488  80% /usr

stat

Gives detailed and verbose statistics on a given file (even a directory or device file) or set of files.

bash$ stat test.cru

  File: “test.cru”

   Size: 49970        Allocated Blocks: 100          Filetype: Regular File

   Mode: (0664/-rw-rw-r–)         Uid: (  501/ bozo)  Gid: (  501/ bozo)

 Device:  3,8   Inode: 18185     Links: 1

 Access: Sat Jun  2 16:40:24 2001

 Modify: Sat Jun  2 16:40:24 2001

 Change: Sat Jun  2 16:40:24 2001

If the target file does not exist, stat returns an error message.

bash$ stat nonexistent-file

nonexistent-file: No such file or directory

vmstat

Display virtual memory statistics.

bash$ vmstat

   procs                      memory    swap          io system         cpu

 r  b  w   swpd   free   buff  cache  si  so    bi    bo   in    cs  us  sy id

 0  0  0      0  11040   2636  38952   0   0    33     7  271    88   8   3 89

netstat

Show current network statistics and information, such as routing tables and active connections. This utility accesses information in /proc/net.

netstat -r is equivalent to route.

uptime

Shows how long the system has been running, along with associated statistics.

bash$ uptime

10:28pm  up  1:57,  3 users,  load average: 0.17, 0.34, 0.27

hostname

Lists the system’s host name. This command sets the host name in an /etc/rc.d setup script (/etc/rc.d/rc.sysinit or similar). It is equivalent to uname -n, and a counterpart to the $HOSTNAME internal variable.

bash$ hostname

localhost.localdomain

bash$ echo $HOSTNAME

localhost.localdomain

hostid

Echo a 32-bit hexadecimal numerical identifier for the host machine.

bash$ hostid

7f0100

NOTE

This command allegedly fetches a “unique” serial number for a particular system. Certain product registration procedures use this number to brand a particular user license. Unfortunately, hostid only returns the machine network address in hexadecimal, with pairs of bytes transposed.The network address of a typical non-networked Linux machine, is found in /etc/hosts.

bash$ cat /etc/hosts

127.0.0.1               localhost.localdomain localhost

As it happens, transposing the bytes of 127.0.0.1, we get 0.127.1.0, which translates in hex to 007f0100, the exact equivalent of what hostid returns, above. There exist only a few million other Linux machines with this identical hostid.

sar

Invoking sar (system activity report) gives a very detailed rundown on system statistics. This command is found on some commercial UNIX systems, but is not part of the base Linux distribution. It is contained in the sysstat utilities package, written by Sebastien Godard.

bash$ sar

Linux 2.4.7-10 (localhost.localdomain)        12/31/2001

 10:30:01 AM       CPU     %user     %nice   %system     %idle

 10:40:00 AM       all      1.39      0.00      0.77     97.84

 10:50:00 AM       all     76.83      0.00      1.45     21.72

 11:00:00 AM       all      1.32      0.00      0.69     97.99

 11:10:00 AM       all      1.17      0.00      0.30     98.53

 11:20:00 AM       all      0.51      0.00      0.30     99.19

 06:30:00 PM       all    100.00      0.00    100.01      0.00

 Average:          all      1.39      0.00      0.66     97.95

readelf

Show information and statistics about a designated elf binary. This is part of the binutils package.

bash$ readelf -h /bin/bash

ELF Header:

   Magic:   7f 45 4c 46 01 01 01 00 00 00 00 00 00 00 00 00

   Class:                             ELF32

   Data:                              2’s complement, little endian

   Version:                           1 (current)

   OS/ABI:                            UNIX - System V

   ABI Version:                       0

   Type:                              EXEC (Executable file)

   . . .

size

The size [/path/to/binary] command gives the segment sizes of a binary executable or archive file. This is mainly of use to programmers.

bash$ size /bin/bash

   text    data     bss     dec     hex filename

  495971   22496   17392  535859   82d33 /bin/bash

System Logs

logger

Appends a user-generated message to the system log (/var/log/messages). You do not have to be root to invoke logger.

logger Experiencing instability in network connection at 23:10, 05/21.

# Now, do a ‘tail /var/log/messages’.

By embedding a logger command in a script, it is possible to write debugging information to /var/log/messages.

logger -t $0 -i Logging at line “$LINENO”.

# The “-t” option specifies the tag for the logger entry.

# The “-i” option records the process ID.

# tail /var/log/message

# …

# Jul  7 20:48:58 localhost ./test.sh[1712]: Logging at line 3.

logrotate

This utility manages the system log files, rotating, compressing, deleting, and/or mailing them, as appropriate. Usually crond runs logrotate on a daily basis.

Adding an appropriate entry to /etc/logrotate.conf makes it possible to manage personal log files, as well as system-wide ones.

Job Control

Process Statistics: lists currently executing processes by owner and PID (process id). This is usually invoked with ax options, and may be piped to grep or sed to search for a specific process.

bash$  ps ax | grep sendmail

295 ?      S      0:00 sendmail: accepting connections on port 25

pstree

Lists currently executing processes in “tree” format. The -p option shows the PIDs, as well as the process names.

top

Continuously updated display of most cpu-intensive processes. The -b option displays in text mode, so that the output may be parsed or accessed from a script.

bash$ top -b

  8:30pm  up 3 min,  3 users,  load average: 0.49, 0.32, 0.13

 45 processes: 44 sleeping, 1 running, 0 zombie, 0 stopped

 CPU states: 13.6% user,  7.3% system,  0.0% nice, 78.9% idle

 Mem:    78396K av,   65468K used,   12928K free,       0K shrd,    2352K buff

 Swap:  157208K av,       0K used,  157208K free                   37244K cached

   PID USER     PRI  NI  SIZE  RSS SHARE STAT %CPU %MEM   TIME COMMAND

   848 bozo      17   0   996  996   800 R     5.6  1.2   0:00 top

     1 root       8   0   512  512   444 S     0.0  0.6   0:04 init

     2 root       9   0     0    0     0 SW    0.0  0.0   0:00 keventd

…

nice

Run a background job with an altered priority. Priorities run from 19 (lowest) to -20 (highest). Only root may set the negative (higher) priorities. Related commands are renice, snice, and skill.

nohup

Keeps a command running even after user logs off. The command will run as a foreground process unless followed by &. If you use nohup within a script, consider coupling it with a wait to avoid creating an orphan or zombie process.

pidof

Identifies process id (pid) of a running job. Since job control commands, such as kill and renice act on the pid of a process (not its name), it is sometimes necessary to identify that pid. The pidof command is the approximate counterpart to the $PPID internal variable.

bash$ pidof xclock

Example 13-4. pidof helps kill a process

#!/bin/bash

# kill-process.sh

NOPROCESS=2

process=xxxyyyzzz  # Use nonexistent process.

# For demo purposes only…

# … don’t want to actually kill any actual process with this script.

# If, for example, you wanted to use this script to logoff the Internet,

#     process=pppd

t=`pidof $process`       # Find pid (process id) of $process.

# The pid is needed by ‘kill’ (can’t ‘kill’ by program name).

if [ -z “$t” ]           # If process not present, ‘pidof’ returns null.

then

  echo “Process $process was not running.”

  echo “Nothing killed.”

  exit $NOPROCESS

fi

kill $t                  # May need ‘kill -9′ for stubborn process.

# Need a check here to see if process allowed itself to be killed.

# Perhaps another ” t=`pidof $process` “.

# This entire script could be replaced by

#    kill $(pidof -x process_name)

# but it would not be as instructive.

exit 0

fuser

Identifies the processes (by pid) that are accessing a given file, set of files, or directory. May also be invoked with the -k option, which kills those processes. This has interesting implications for system security, especially in scripts preventing unauthorized users from accessing system services.

crond

Administrative program scheduler, performing such duties as cleaning up and deleting system log files and updating the slocate database. This is the superuser version of at (although each user may have their own crontab file which can be changed with the crontab command). It runs as a daemon and executes scheduled entries from /etc/crontab.

Process Control and Booting

init

The init command is the parent of all processes. Called in the final step of a bootup, init determines the runlevel of the system from /etc/inittab. Invoked by its alias telinit, and by root only.

telinit

Symlinked to init, this is a means of changing the system runlevel, usually done for system maintenance or emergency filesystem repairs. Invoked only by root. This command can be dangerous - be certain you understand it well before using!

runlevel

Shows the current and last runlevel, that is, whether the system is halted (runlevel 0), in single-user mode (1), in multi-user mode (2 or 3), in X Windows (5), or rebooting (6). This command accesses the /var/run/utmp file.

halt, shutdown, reboot

Command set to shut the system down, usually just prior to a power down.

Network

ifconfig

Network interface configuration and tuning utility. It is most often used at bootup to set up the interfaces, or to shut them down when rebooting.

# Code snippets from /etc/rc.d/init.d/network

# …

# Check that networking is up.

[ ${NETWORKING} = “no” ] && exit 0

[ -x /sbin/ifconfig ] || exit 0

# …

for i in $interfaces ; do

  if ifconfig $i 2>/dev/null | grep -q “UP” >/dev/null 2>&1 ; then

    action “Shutting down interface $i: ” ./ifdown $i boot

fi

# The GNU-specific “-q” option to “grep” means “quiet”, i.e., producing no output.

# Redirecting output to /dev/null is therefore not strictly necessary.

# …

echo “Currently active devices:”

echo `/sbin/ifconfig | grep ^[a-z] | awk ‘{print $1}’`

#                            ^^^^^  should be quoted to prevent globbing.

#  The following also work.

#    echo $(/sbin/ifconfig | awk ‘/^[a-z]/ { print $1 })’

#    echo $(/sbin/ifconfig | sed -e ’s/ .*//’)

#  Thanks, S.C., for additional comments.

route

Show info about or make changes to the kernel routing table.

bash$ route

Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface

 pm3-67.bozosisp *               255.255.255.255 UH       40 0          0 ppp0

 127.0.0.0       *               255.0.0.0       U        40 0          0 lo

 default         pm3-67.bozosisp 0.0.0.0         UG       40 0          0 ppp0

chkconfig

Check network configuration. This command lists and manages the network services started at bootup in the /etc/rc?.d directory.

Originally a port from IRIX to Red Hat Linux, chkconfig may not be part of the core installation of some Linux flavors.

bash$ chkconfig –list

atd             0:off   1:off   2:off   3:on    4:on    5:on    6:off

 rwhod           0:off   1:off   2:off   3:off   4:off   5:off   6:off

…

tcpdump

Network packet “sniffer”. This is a tool for analyzing and troubleshooting traffic on a network by dumping packet headers that match specified criteria.

Dump ip packet traffic between hosts bozoville and caduceus:

bash$ tcpdump ip host bozoville and caduceus

Of course, the output of tcpdump can be parsed, using certain of the previously discussed text processing utilities.

Filesystem

mount

Mount a filesystem, usually on an external device, such as a floppy or CDROM. The file /etc/fstab provides a handy listing of available filesystems, partitions, and devices, including options, that may be automatically or manually mounted. The file /etc/mtab shows the currently mounted filesystems and partitions (including the virtual ones, such as /proc).

mount -a mounts all filesystems and partitions listed in /etc/fstab, except those with a noauto option. At bootup, a startup script in /etc/rc.d (rc.sysinit or something similar) invokes this to get everything mounted.

mount -t iso9660 /dev/cdrom /mnt/cdrom

# Mounts CDROM

mount /mnt/cdrom

# Shortcut, if /mnt/cdrom listed in /etc/fstab

This versatile command can even mount an ordinary file on a block device, and the file will act as if it were a filesystem. Mount accomplishes that by associating the file with a loopback device. One application of this is to mount and examine an ISO9660 image before burning it onto a CDR.

Example 13-5. Checking a CD image

# As root…

mkdir /mnt/cdtest  # Prepare a mount point, if not already there.

mount -r -t iso9660 -o loop cd-image.iso /mnt/cdtest   # Mount the image.

#                  “-o loop” option equivalent to “losetup /dev/loop0″

cd /mnt/cdtest     # Now, check the image.

ls -alR            # List the files in the directory tree there.

                   # And so forth.

umount

Unmount a currently mounted filesystem. Before physically removing a previously mounted floppy or CDROM disk, the device must be umounted, else filesystem corruption may result.

umount /mnt/cdrom

# You may now press the eject button and safely remove the disk.

NOTE	The automount utility, if properly installed, can mount and unmount floppies or CDROM disks as they are accessed or removed. On laptops with swappable floppy and CDROM drives, this can cause problems, though.

sync

Forces an immediate write of all updated data from buffers to hard drive (synchronize drive with buffers). While not strictly necessary, a sync assures the sys admin or user that the data just changed will survive a sudden power failure. In the olden days, a sync; sync (twice, just to make absolutely sure) was a useful precautionary measure before a system reboot.

At times, you may wish to force an immediate buffer flush, as when securely deleting a file or when the lights begin to flicker.

losetup

Sets up and configures loopback devices.

Example 13-6. Creating a filesystem in a file

SIZE=1000000  # 1 meg

head -c $SIZE < /dev/zero > file  # Set up file of designated size.

losetup /dev/loop0 file           # Set it up as loopback device.

mke2fs /dev/loop0                 # Create filesystem.

mount -o loop /dev/loop0 /mnt     # Mount it.

# Thanks, S.C.

mkswap

Creates a swap partition or file. The swap area must subsequently be enabled with swapon.

swapon, swapoff

Enable / disable swap partitition or file. These commands usually take effect at bootup and shutdown.

mke2fs

Create a Linux ext2 filesystem. This command must be invoked as root.

Example 13-7. Adding a new hard drive

#!/bin/bash

# Adding a second hard drive to system.

# Software configuration. Assumes hardware already mounted.

# From an article by the author of this document.

# in issue #38 of “Linux Gazette”, http://www.linuxgazette.com.

ROOT_UID=0     # This script must be run as root.

E_NOTROOT=67   # Non-root exit error.

if [ “$UID” -ne “$ROOT_UID” ]

then

  echo “Must be root to run this script.”

  exit $E_NOTROOT

fi

# Use with extreme caution!

# If something goes wrong, you may wipe out your current filesystem.

NEWDISK=/dev/hdb         # Assumes /dev/hdb vacant. Check!

MOUNTPOINT=/mnt/newdisk  # Or choose another mount point.

fdisk $NEWDISK

mke2fs -cv $NEWDISK1   # Check for bad blocks & verbose output.

#  Note:    /dev/hdb1, *not* /dev/hdb!

mkdir $MOUNTPOINT

chmod 777 $MOUNTPOINT  # Makes new drive accessible to all users.

# Now, test…

# mount -t ext2 /dev/hdb1 /mnt/newdisk

# Try creating a directory.

# If it works, umount it, and proceed.

# Final step:

# Add the following line to /etc/fstab.

# /dev/hdb1  /mnt/newdisk  ext2  defaults  1 1

exit 0

tune2fs

Tune ext2 filesystem. May be used to change filesystem parameters, such as maximum mount count. This must be invoked as root.

WARNING

This is an extremely dangerous command. Use it at your own risk, as you may inadvertently destroy your filesystem.

dumpe2fs

Dump (list to stdout) very verbose filesystem info. This must be invoked as root.

root# dumpe2fs /dev/hda7 | grep ‘ount count’

dumpe2fs 1.19, 13-Jul-2000 for EXT2 FS 0.5b, 95/08/09

 Mount count:              6

 Maximum mount count:      20

hdparm

List or change hard disk parameters. This command must be invoked as root, and it may be dangerous if misused.

fdisk

Create or change a partition table on a storage device, usually a hard drive. This command must be invoked as root.

WARNING

Use this command with extreme caution. If something goes wrong, you may destroy an existing filesystem.

fsck, e2fsck, debugfs

Filesystem check, repair, and debug command set.

fsck: a front end for checking a UNIX filesystem (may invoke other utilities). The actual filesystem type generally defaults to ext2.

e2fsck: ext2 filesystem checker.

debugfs: ext2 filesystem debugger. One of the uses of this versatile, but dangerous command is to (attempt to) recover deleted files. For advanced users only!

CAUTION

All of these should be invoked as root, and they can damage or destroy a filesystem if misused.

badblocks

Checks for bad blocks (physical media flaws) on a storage device. This command finds use when formatting a newly installed hard drive or testing the integrity of backup media. As an example, badblocks /dev/fd0 tests a floppy disk.

The badblocks command may be invoked destructively (overwrite all data) or in non-destructive read-only mode. If root user owns the device to be tested, as is generally the case, then root must invoke this command.

mkbootdisk

Creates a boot floppy which can be used to bring up the system if, for example, the MBR (master boot record) becomes corrupted. The mkbootdisk command is actually a Bash script, written by Erik Troan, in the /sbin directory.

chroot

CHange ROOT directory. Normally commands are fetched from $PATH, relative to /, the default root directory. This changes the root directory to a different one (and also changes the working directory to there). This is useful for security purposes, for instance when the system administrator wishes to restrict certain users, such as those telnetting in, to a secured portion of the filesystem (this is sometimes referred to as confining a guest user to a “chroot jail”). Note that after a chroot, the execution path for system binaries is no longer valid.

A chroot /opt would cause references to /usr/bin to be translated to /opt/usr/bin. Likewise, chroot /aaa/bbb /bin/ls would redirect future instances of ls to /aaa/bbb as the base directory, rather than / as is normally the case. An alias XX ‘chroot /aaa/bbb ls’ in a user’s ~/.bashrc effectively restricts which portion of the filesystem she may run command “XX” on.

The chroot command is also handy when running from an emergency boot floppy (chroot to /dev/fd0), or as an option to lilo when recovering from a system crash. Other uses include installation from a different filesystem (an rpm option) or running a readonly filesystem from a CD ROM. Invoke only as root, and use with care.

CAUTION

It might be necessary to copy certain system files to a chrooted directory, since the normal $PATH can no longer be relied upon.

lockfile

This utility is part of the procmail package (www.procmail.org). It creates a lock file, a semaphore file that controls access to a file, device, or resource. The lock file serves as a flag that this particular file, device, or resource is in use by a particular process (“busy”), and this permits only restricted access (or no access) to other processes.

Lock files are used in such applications as protecting system mail folders from simultaneously being changed by multiple users, indicating that a modem port is being accessed, and showing that an instance of Netscape is using its cache. Scripts may check for the existence of a lock file created by a certain process to check if that process is running. Note that if a script attempts create a lock file that already exists, the script will likely hang.

Normally, applications create and check for lock files in the /var/lock directory. A script can test for the presence of a lock file by something like the following.

appname=xyzip

# Application “xyzip” created lock file “/var/lock/xyzip.lock”.

if [ -e “/var/lock/$appname.lock ]

then

…

mknod

Creates block or character device files (may be necessary when installing new hardware on the system).

tmpwatch

Automatically deletes files which have not been accessed within a specified period of time. Usually invoked by crond to remove stale log files.

MAKEDEV

Utility for creating device files. It must be run as root, and in the /dev directory.

root# ./MAKEDEV

This is a sort of advanced version of mknod.

Backup

dump, restore

The dump command is an elaborate filesystem backup utility, generally used on larger installations and networks. It reads raw disk partitions and writes a backup file in a binary format. Files to be backed up may be saved to a variety of storage media, including disks and tape drives. The restore command restores backups made with dump.

fdformat

Perform a low-level format on a floppy disk.

System Resources

ulimit

Sets an upper limit on system resources. Usually invoked with the -f option, which sets a limit on file size (ulimit -f 1000 limits files to 1 meg maximum). The -t option limits the coredump size (ulimit -c 0 eliminates coredumps). Normally, the value of ulimit would be set in /etc/profile and/or ~/.bash_profile.

umask

User file creation MASK. Limit the default file attributes for a particular user. All files created by that user take on the attributes specified by umask. The (octal) value passed to umask defines the file permissions disabled. For example, umask 022 ensures that new files will have at most 755 permissions (777 NAND 022). [6] Of course, the user may later change the attributes of particular files with chmod. The usual practice is to set the value of umask in /etc/profile and/or ~/.bash_profile.

rdev

Get info about or make changes to root device, swap space, or video mode. The functionality of rdev has generally been taken over by lilo, but rdev remains useful for setting up a ram disk. This is another dangerous command, if misused.

Modules

lsmod

List installed kernel modules.

bash$ lsmod

Module                  Size  Used by

 autofs                  9456   2 (autoclean)

 opl3                   11376   0

 serial_cs               5456   0 (unused)

 sb                     34752   0

 uart401                 6384   0 [sb]

 sound                  58368   0 [opl3 sb uart401]

 soundlow                 464   0 [sound]

 soundcore               2800   6 [sb sound]

 ds                      6448   2 [serial_cs]

 i82365                 22928   2

 pcmcia_core            45984   0 [serial_cs ds i82365]

NOTE	Doing a cat /proc/modules gives the same information.

insmod

Force installation of a kernel module (use modprobe instead, when possible). Must be invoked as root.

rmmod

Force unloading of a kernel module. Must be invoked as root.

modprobe

Module loader that is normally invoked automatically in a startup script. Must be invoked as root.

depmod

Creates module dependency file, usually invoked from startup script.

Miscellaneous

env

Runs a program or script with certain environmental variables set or changed (without changing the overall system environment). The [varname=xxx] permits changing the environmental variable varname for the duration of the script. With no options specified, this command lists all the environmental variable settings.

NOTE

In Bash and other Bourne shell derivatives, it is possible to set variables in a single command’s environment.

var1=value1 var2=value2 commandXXX

# $var1 and $var2 set in the environment of ‘commandXXX’ only.

TIP

The first line of a script (the “sha-bang” line) may use env when the path to the shell or interpreter is unknown.

#! /usr/bin/env perl

print “This Perl script will run,n”;

print “even when I don’t know where to find Perl.n”;

# Good for portable cross-platform scripts,

# where the Perl binaries may not be in the expected place.

# Thanks, S.C.

ldd

Show shared lib dependencies for an executable file.

bash$ ldd /bin/ls

libc.so.6 => /lib/libc.so.6 (0×4000c000)

/lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0×80000000)

strip

Remove the debugging symbolic references from an executable binary. This decreases its size, but makes debugging it impossible.

This command often occurs in a Makefile, but rarely in a shell script.

List symbols in an unstripped compiled binary.

rdist

Remote distribution client: synchronizes, clones, or backs up a file system on a remote server.

Using our knowledge of administrative commands, let us examine a system script. One of the shortest and simplest to understand scripts is killall, used to suspend running processes at system shutdown.

Example 13-8. killall, from /etc/rc.d/init.d

#!/bin/sh

# –> Comments added by the author of this document marked by “# –>”.

# –> This is part of the ‘rc’ script package

# –> by Miquel van Smoorenburg,

# –> This particular script seems to be Red Hat specific

# –> (may not be present in other distributions).

# Bring down all unneeded services that are still running (there shouldn’t

# be any, so this is just a sanity check)

for i in /var/lock/subsys/*; do

        # –> Standard for/in loop, but since “do” is on same line,

        # –> it is necessary to add “;”.

        # Check if the script is there.

        [ ! -f $i ] && continue

        # –> This is a clever use of an “and list”, equivalent to:

        # –> if [ ! -f “$i” ]; then continue

        # Get the subsystem name.

        subsys=${i#/var/lock/subsys/}

        # –> Match variable name, which, in this case, is the file name.

        # –> This is the exact equivalent of subsys=`basename $i`.

        # –>  It gets it from the lock file name (if there is a lock file,

        # –>+ that’s proof the process has been running).

        # –>  See the “lockfile” entry, above.

        # Bring the subsystem down.

        if [ -f /etc/rc.d/init.d/$subsys.init ]; then

            /etc/rc.d/init.d/$subsys.init stop

        else

            /etc/rc.d/init.d/$subsys stop

        # –> Suspend running jobs and daemons

        # –> using the ’stop’ shell builtin.

fi

done

More at http://www.die.net/doc/linux/man/

2 Gigabit FC Card - How Fast is It, Really?

2009-02-05T17:22:00.000-08:00

Common misunderstandings:

“If I buy 2 Gigabit HBAs I will get 2 Gigabytes of throughput, that’s more than enough throughput for my 400 MB/s application”

“I only need 2 disks to store my 1 TB database, now that we have 500GB disk drives are available!”

The first misunderstanding is not just about bits vs bytes, its understanding the context of when and where the [unit of measurement] bits vs bytes is used. For example, in the world of networks
(Ethernet, Fibre Channel, etc.) bits is a common unit of measurement, whereas bytes is used to describe throughput outside the network; i.e., disks or HBAs. Just be aware of the unit measurement and the context of its use.

To convert 2Gbits into Mbytes :

2Gbits = (2* 1024 * 1024 * 1024) / (1024* 1024) / 8bits
= 2147483648 / 1048576
= 2048 / 8
= 256Mbytes

The second misunderstanding has to do with sizing for capacity instead of sizing for throughput. Always size for throughput.

3 Easy Steps to Use 'sar' to Log and Analyze System Performance

2009-02-04T23:58:00.001-08:00

Commands below are for Linux, but the idea works everywhere ’sadc’ is available.

1) Start

/usr/lib/sa/sadc

Example

machine1:~ # /usr/lib/sa/sadc 2 7200 sadc.outfile

this logs SAR data for 4 hours.

2) Stop logging, do

‘killall sadc’ or ‘kill’

3) Usage of sar. Creating report with all options

sar -A -f

Example

machine1:~ # sar -A -f sadc.outfile > sar.outfile
machine1:~ # cat sar.outfile
Linux 2.4.21-75-default (machine1) 12/03/03

13:35:34 proc/s
Average: 0.00

13:35:34 cswch/s
Average: 38.50

13:35:34 CPU %user %nice %system %idle
Average: all 0.06 0.00 0.00 99.94

13:35:34 INTR intr/s
Average: sum 0.00

13:35:34 pgpgin/s pgpgout/s activepg inadtypg inaclnpg inatarpg
Average: 0.00 16.00 9931 0 0 0

13:35:34 pswpin/s pswpout/s
Average: 0.00 0.00

13:35:34 tps rtps wtps bread/s bwrtn/s
Average: 0.75 0.00 0.75 0.00 32.00

13:35:34 CPU %user %nice %system %idle
Average: 0 0.00 0.00 0.00 100.00
Average: 1 0.00 0.00 0.00 100.00
Average: 2 0.00 0.00 0.00 100.00
Average: 3 0.25 0.00 0.00 99.75

13:35:34 frmpg/s shmpg/s bufpg/s campg/s
Average: -0.50 0.00 0.50 0.00

13:35:34 kbmemfree kbmemused %memused kbmemshrd kbbuffers kbcached kbswpfree kbswpused %swpused
Average: 8860 114108 92.79 0 21996 42764 2097112 0 0.00

13:35:34 dentunusd file-sz %file-sz inode-sz super-sz %super-sz dquot-sz %dquot-sz rtsig-sz %rtsig-sz
Average: 10558 719 5.49 7376 0 0.00 0 0.00 0 0.00

13:35:34 Iface rxpck/s txpck/s rxbyt/s txbyt/s rxcmp/s txcmp/s rxmcst/s
Average: lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Average: sit0 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Average: eth1 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Average: eth0 2.00 0.50 381.50 66.00 0.00 0.00 0.00

13:35:34 Iface rxerr/s txerr/s coll/s rxdrop/s txdrop/s txcarr/s rxfram/s rxfifo/s txfifo/s
Average: lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Average: sit0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Average: eth1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Average: eth0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

13:35:34 totsck tcpsck udpsck rawsck ip-frag
Average: 116 4 2 0 0

13:35:34 runq-sz plist-sz ldavg-1 ldavg-5
Average: 0 43 0.00 0.01

13:35:34 DEV tps blks/s
Average: dev94-0 0.75 32.00

Check Web Server Status from Browser

2009-02-04T23:54:00.000-08:00

Sometimes it is easier and quick to get status on a Web Server from a browser - you can be anywhere and know what is going on with the server. Of course, this works only if you are on a firewall protected network.

The IBM HTTP Server server-status page is available on all supported IBM HTTP Server platforms. It shows performance data on a Web page in HTML format. Statistics include total accesses, requests/sec, data(KB)/sec, total traffic (MB) since last restart, etc.

To make this work, perform the following steps to activate the server-status page:
1. Open the IBM HTTP Server file httpd.conf in an editor.
2. Remove the comment character “#” from the following lines:

#LoadModule status_module modules/mod_status.so
#
# SetHandler server-status
# Order deny,allow
# Deny from all
# Allow from .example.com
#
3. Customize the “.example.com” in the sample configuration to match your source workstation reverse DNS entry so you’re allowed to access the page. In our example this must be changed to:

Allow from .pw.com [replace with your own domain name]
4. Save the changes and restart the IBM HTTP Server.
5. Open the URL http:///server-status in a Web browser, and click Refresh to update the status.

If your browser supports refresh, you can also use the URL http:///server-status?refresh=5 to refresh every 5 seconds.

As shown in the following image, you can see (along with additional information) the number of requests currently being processed, and the number of idle servers.

How much IO is Imposed by System (Oracle)

2009-02-04T23:50:00.000-08:00

Don’t forget to add extra IO imposed by users of the database. For example, to determine how much extra I/O will be imposed on the system when the new load is put into place. For example, if more users or reports will be added, determine how much I/O these reports (or a typical user) currently does, then extrapolate for the estimated growth. The query below will assist in getting some of this data.

SELECT a.sid,
a.statistic#,
SUBSTR(b.name,1,40) name,
a.value
FROM v$sesstat a,
v$statname b,
v$session se
WHERE se.audsid = (select userenv(’sessionid’) from dual)
AND a.statistic# = b.statistic#
AND se.sid = a.sid
AND b.name in (’physical read total IO requests’,'physical read total bytes’,'physical write total IO
requests’,'physical write total bytes’)
ORDER BY b.class, b.name

What is Throughput?

2009-02-04T23:44:00.000-08:00

Saw this definition from one of the Oracle OpenWorld sessions, thought it was pretty clear and concise:
Throughput

The measure of the transfer of bits across the media over a given period of time. Commonly used in discussing data transfer rates
Due to a number of factors, throughput usually does not match the specified bandwidth. Factors include:

The amount and type of traffic on the network
The number of network devices encountered in the network path being measured (path latency).

In any network (including FC networks), throughput cannot be faster than the slowest link of the path from source to destination. Even if all or most of the segments have high bandwidth, it will only take one segment in the path with low throughput to create a bottleneck to the throughput of the entire network. Bandwidth is the amount of information it is physically possible to send through the media of choice.

Slow response time? Start here!

2009-02-04T23:34:00.000-08:00

Simple & Effective Performance Tips - Your end users complain that your Web pages load slowly and they don’t intend to wait! So you need to dust off your system topology diagram and understand which servers user traffic travels. Then you need to determine if CPUs are busy on these systems, if they are busy (i.e., above 85%), you need to track down which process is using the CPU cycles; if CPU seems fine (i.e., under 85%), then ask yourself if memory is running out? If your system is memory-bound, you could be paging to disks - disk I/O is a performance killer. If memory is the issue, find out which process(es) are hogging memory - and they could be abusing memory too (e.g., memory leak, or loading large Word files into memory for processing); if your systems have pletty of free memory, then you need to check if your disk I/O is high. If disk I/O is high, you need to find out which process or thread is writing to and reading from the disk! Finally, if all the above checked out fine, ask yourself, if network bandwidth is maxed out: traffic is so high that responses from your Web site simply cannot push through the thin network “pipe”. But ask yourself, is your application supposed to push that much data?

Tuning AIX systems for WebSphere Application Server 6.0.x

2009-02-04T23:31:00.000-08:00

IBM recommends a number of configuration changes and variables you can set to tune the performance of Websphere to suit your needs. Because the AIX operating system is not a WebSphere Application Server product, be aware that it can change and results can vary.

Procedure

Change the following configuration settings or variables according to your needs:

TCP_TIMEWAIT
- Description: Determines the time that must elapse before TCP/IP can release a closed connection and reuse its resources. This interval between closure and release is known as the TIME_WAIT state or twice the maximum segment lifetime (2MSL) state. During this time, reopening the connection to the client and server costs less than establishing a new connection. By reducing the value of this entry, TCP/IP can release closed connections faster, providing more resources for new connections. Adjust this parameter, if the running application requires rapid release or the creation of new connections, or if a low throughput occurs due to many connections sitting in the TIME_WAIT state.
- How to view or set:
  Issue the following command to set TCP_TIMEWAIT state to 15 seconds:
```
/usr/sbin/no –o tcp_timewait =1
```
AIX operating systems with DB2
- Description: Separating your DB2 log files from the physical database files can boost performance. You can also separate the log and the database files from the drive that contains the Journaled File System (JFS) service. AIX uses specific volume groups and file systems for the JFS logging.
- How to view or set: Use the AIX filemon utility to view all the file system input and output and to strategically select the file system for the DB2 log files. Set the DB2 log location according to the DB2 tuning parameters topic.
- Default value: The default location for the DB2 log files is often the same disk drive where the database tables are stored.
- Recommended value: Move the files to a disk that is separate from the DB2 data and has the minimum input or output activity.
AIX file descriptors (ulimit)
- Description: Specifies the various restrictions on resource usage on the user account. The ulimit -a command displays all the ulimit limits. The ulimit -a command specifies only the number of open files that are permitted. The default number of open files setting (2000) is typically sufficient for most applications. If the value set for this parameter is too low, errors might occur when opening files or establishing connections. Because this value limits the number of file descriptors that a server process might open, a value that is too low prevents optimum performance.
- How to view or set: Perform the following steps to change the open file limit to 10,000 files:
  1. Open the command window.
  1. Edit the /etc/security/limits file. Add the following lines to the user account that the WebSphere Application Server process runs on:
```
nofiles =  10000
nofiles_hard = 10000
```
  2. Save the changes.
  3. Restart your AIX system.
  4. To verify the result, type the ulimit -a command on the command line. For example, type # ulimit -a.
- Default value: For the AIX operating system, the default setting is 2000.
- Recommended value: The value is application dependent. Increasing the ulimit file descriptor limits might improve performance. Increasing some of the other limits might be needed depending on your application. Any changes to the data or stack ulimits should ensure that data+stack <>
AIX TCP_KEEPIDLE
- Description: The keepAlive packet ensures that a connection stays in an active/ESTABLISHED state.
- How to view or set: Use the no command to determine the current value or to set the value. The change is effective until the next time you restart the machine. To permanently change the value, add the no command to the /etc/rc.net directory. For example:
```
no -o tcp_keepidle=600
```
- Default value: 14400 half seconds (2 hours).
- Recommended value: 600 half seconds (5 minutes).
TCP_KEEPINTVL
- Description: Specifies the interval between packets that are sent to validate the connection.
- How to view or set: Use the following command to set the value to 5 seconds:
```
no -o tcp_keepintvl=10
```
- Default value: 150(1/2 seconds)
- Recommended value: 10(1/2 seconds)
TCP_KEEPINIT
- Description: Specifies the initial timeout value for TCP connection.
- How to view or set: Use the following command to set the value to 20 seconds:
```
no -o tcp_keepinit=40
```
- Default value: 150(1/2 seconds)
- Recommended value: 40(1/2 seconds)
Allocating large pages (16 MB) for Java virtual machines heap
Some applications require a very large heap for optimal performance. Reduce the CPU overhead of managing a large heap by using large page support that is provided by the CPU and the operating system. The following steps allocate 4 GB of RAM as large pages (16 MB):
1. As root user, run the following commands to reserve 4 GB of large page:
```
vmo  -r  -o  lgpg_regions=256  -o lgpg_size=16777216
bosboot  -ad  /dev/ipldevice
reboot   -q
```
2. After reboot, run the following command to enable large page support on the AIX operating system:
```
vmo -p -o  v_pinshm=1
```
3. As root user, add the following capabilities for the user:
```
chuser  capabilities=CAP_BYPASS_RAC_VMM,CAP_PROPAGATE  $USER
```
4. Add the -Xlp Java options to the Java command.
  1. Click Servers > Application Servers > server_name.
  2. Under Server Infrastructure, click Java and Process Management > Process Definition > Java Virtual Machine.
  3. In the Generic JVM Argument field, add -Xlp.
5. Add the EXTSHM custom property and set to OFF.
  1. Click Servers > Application Servers > server_name.
  2. Under Server Infrastructure, click Java and Process Management > Process Definition > Custom properties > New.
  3. In the Name field, enter EXTSHM.
  4. In the Value field, enter OFF.
6. Validate large page support is used with the following command:
```
vmstat  -l  1
```
  Note: The “alp” column is non-zero when the application is running.
There are several concerns when enabling large pages, which can cause serious events to occur on the machine when large pages are enabled. For more information on AIX large pages, see the “Considerations for using large pages” section at the following address: http://publib.boulder.ibm.com/infocenter/pseries/v5r3/index.jsp?topic=/com.ibm.aix.prftungd/doc/prftungd/large_page_ovw.htm.

If you do not want to use the large pages option, there is also a medium page option. The medium page size option, which is similar, and has close to the same performance gains as large pages. However, it does not involve the problems of reserving physical memory for a specific user or process. For more information, see the -Xlp64k option in the Tuning Java virtual machines topic.
Other AIX information
Consider the other AIX operating system settings that are not within the scope of this document. You can adjust the following additional settings:
- Adapter transmit and receive queue
- TCP/IP socket buffer
- IP protocol mbuf pool performance
- Update file descriptors
- Update the scheduler

Tuning Linux systems for WebSphere Application Server 6.0.x

2009-02-04T23:28:00.000-08:00

When you have a performance concern, check the operating system settings to determine if these settings are appropriate for your application. Because the Linux operating system is not a WebSphere Application Server product, be aware that it can change and results can vary.

Procedure

Configure the following settings and variables according to your tuning needs:

timeout_timewait parameter
- Description: Determines the time that must elapse before TCP/IP can release a closed connection and reuse its resources. This interval between closure and release is known as the TIME_WAIT state or twice the maximum segment lifetime (2MSL) state. During this time, reopening the connection to the client and server cost less than establishing a new connection. By reducing the value of this entry, TCP/IP can release closed connections faster, providing more resources for new connections. Adjust this parameter if the running application requires rapid release, the creation of new connections, and a low throughput due to many connections sitting in the TIME_WAIT state.
- How to view or set:Issue the following command to set the timeout_timewait parameter to 30 seconds:
```
echo 30 > /proc/sys/net/ipv4/tcp_fin_timeout
```
SUSE Linux Enterprise Server 8 (SLES SP2A - sched_yield_scale tuning
- Description: The Linux scheduler is very sensitive to excessive context switching, so fixes are integrated into the SLES 8 kernel distribution to introduce delay when a thread yields processing. This fix is automatically enabled in SLES 8 SP3, but must be enabled explicitly in SLES 8 SP2A or later.
- How to view or set:
  1. Upgrade your SLES 8 service pack to SP2A.
  2. Issue the sysctl -w sched_yield_scale=1 command .
- Default value: 0
- Recommended value: 1
RedHat Advanced Server 2.1 kernel update
- Description: Kernel updates for RedHat Advanced Server 2.1 implemented changes that affect WebSphere Application Server performance, especially memory-to-memory HTTP session replication.
- How to view or set:
  1. Issue the uname -a command
  2. If you are running any kernel prior to 2.4.9-e.23, upgrade at least to the RedHat Advanced Server 2.1 kernel, but preferably to the latest supported.
- Default value: 2.4.9-e.3
- Recommended value: 2.4.9-e.23
Linux file descriptors (ulimit)
- Description: Specifies the number of open files that are supported. The default setting is typically sufficient for most applications. If the value set for this parameter is too low, a file open error, memory allocation failure, or connection establishment error might be displayed.
- How to view or set: Check the UNIX reference pages on the ulimit command for the syntax of different shells. To set the ulimit command to 8000 for the KornShell shell (ksh), issue the ulimit -n 8000 command. Use the ulimit -a command to display the current values for all limitations on system resources.
- Default value: For SUSE Linux Enterprise Server 9 (SLES 9), the default is 1024.
- Recommended value: 8000
Connection backlog
- Description: Change the following parameters when a high rate of incoming connection requests result in connection failures:
```
echo 3000 > /proc/sys/net/core/netdev_max_backlog
echo 3000 > /proc/sys/net/core/somaxconn
```
TCP_KEEPALIVE_INTERVAL
- Description: Determines the wait time between isAlive interval probes.
- How to view or set: Issue the following command to set the value:
```
echo 15 > /proc/sys/net/ipv4/tcp_keepalive_intvl
```
- Default value: 75 seconds
- Recommended value: 15 seconds
TCP_KEEPALIVE_PROBES
- Description: Determines the number of probes before timing out.
- How to view or set: Issue the following command to set the value:
```
echo 5  > /proc/sys/net/ipv4/tcp_keepalive_probes
```
- Default value: 9 seconds
- Recommended value: 5 seconds
Allocating large pages for Java virtual machine (JVM) heap (tested with SLES 9)Some applications require a very large heap for optimal performance. The CPU overhead of managing a large heap can be reduced by using the “large page” support that is provided by the CPU and operating system. The following example assumes a large page size of 4MB and a desired heap size of 2300MB.
1. Set the following three settings by a sysctl.conf file, typically located at /etc/sysctl.conf.
  Note: You must have root privilege access to modify this file. Also, verify the file is not marked as read-only before attempting to make changes.
  1. Set the number of large pages (2300MB = 575 * 4MB) by issuing the following command:
```
vm.nr_hugepages = 575
```
  2. Set the maximum shared segment size to 2300MB plus a little more (about 95MB) (2511724800 = 2300MB * 1048576 bytes/MB + 100000000 bytes) by issuing the following command:
```
kernel.shmmax = 2511724800
```
  3. Set the total amount of memory to be shared by issuing the following command:
```
kernel.shmall = 2511724800
```
2. Specify the -Xlp JVM option so the JVM heap can utilize large pages.
3. Set the Xmx JVM option to 2300MB.
4. Relocate the program text to a lower virtual memory address (0×10000000) to provide more address space for a larger heap. On SUSE Linux Enterprise Server 9 , run the following command to relocate the text in the script that invokes the JVM or in a .profile file:
```
echo "0x10000000" > /proc/self/mapped_base
```

Tuning Windows systems for WebSphere Application Server 6.0.x

2009-02-04T23:06:00.000-08:00

IBM describes how to tune Windows 2000, Windows XP, and Windows 2003 operating systems to optimize the performance of WebSphere Application Server. Because Windows operating systems are not WebSphere Application Server products, be aware that the products can change and results can vary.

Procedure

Configure the following settings or variables according to your specific tuning needs:

TcpTimedWaitDelay
- Description: Determines the time that must elapse before TCP/IP can release a closed connection and reuse its resources. This interval between closure and release is known as the TIME_WAIT state or twice the maximum segment lifetime (2MSL) state. During this time, reopening the connection to the client and server costs less than establishing a new connection. By reducing the value of this entry, TCP/IP can release closed connections faster and provide more resources for new connections. Adjust this parameter if the running application requires rapid release, the creation of new connections, or an adjustment because of a low throughput caused by multiple connections in the TIME_WAIT state.
- How to view or set:
  1. Use the regedit command, access the HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\ Services\TCPIP\Parameters registry subkey, and create a new REG_DWORD value named TcpTimedWaitDelay.
  2. Set the value to decimal 30, which is Hex 0×0000001e. This value sets the wait time to 30 seconds.
  3. Stop and restart the system.
- Default value: 0xF0, which sets the wait time to 240 seconds (4 minutes).
- Recommended value: A minimum value of 0×1E, which sets the wait time to 30 seconds.
MaxUserPort
- Description: Determines the highest port number that TCP/IP can assign when an application requests an available user port from the system.
- How to view or set:
  1. Use the regedit command, access the HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\ Services\TCPIP\Parameters registry subkey, and create a new REG_DWORD value named MaxUserPort.
  2. Set this value to at least decimal 32768.
  3. Stop and restart the system.
- Default value: None
- Recommended value: At least decimal 32768.
MaxConnect Backlog
- Description: If many connection attempts are received simultaneously, increase the default number of pending connections that are supported by the operating system.
- How to view or set:
  1. Use the regedit command and access the HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\AFD\Parameters registry subkey
  2. Create and set (and create if necessary) the following values:
```
"EnableDynamicBacklog"=dword:00000001

"MinimumDynamicBacklog"=dword:00000020

"MaximumDynamicBacklog"=dword:00001000

"DynamicBacklogGrowthDelta"=dword:00000010
```
  3. These values request a minimum of 20 and a maximum of 1000 available connections. The number of available connections is increased by 10 each time that there are fewer than the minimum number of available connections.
  4. Stop and restart the system.
KeepAliveInterval
- Description: Determines how often TCP repeats keep-alive transmissions when no response is received.
- How to view or set:
  1. Use the regedit command, access the HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\AFD\Parameters registry subkey, and create a new REG_DWORD value named KeepAliveInterval.
  2. Set this value to 1 second.
  3. Stop and restart the system.
- Default value: 1 second
- Recommended value: None
TcpMaxDataRetranmission
- Description: Determines how many times TCP retransmits an unacknowledged data segment on an existing connection.
- How to view or set:
  1. Use the regedit command, access the HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters registry subkey, and edit or create a new REG_DWORD value named TcpMaxDataRetransmissions.
  2. Set this value to 5 seconds
  3. Stop and restart the system.
- Default value: 5 second
- Recommended value: None
TPC/IP acknowledgements
- TCP/IP can be the source of some significant remote method delays. You can increase TCP performance by immediately acknowledging incoming TCP segments, in all situations.Complete the following steps to immediately acknowledge incoming TCP segments on a server that runs a Microsoft Windows 2000 operating system:
  1. Start the Registry Editor (regedit.exe).
  2. Locate and click the following registry subkey:
```
HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServicesTcpipParametersInterfaces
```
  3. On the Edit menu, click Add Value, and create the following registry value:
```
Value name: TcpDelAckTicks
Data type: REG_DWORD
Value data: 0
Quit Registry Editor.
```
  4. Restart your Windows operating system.
  Similarly, to immediately acknowledge incoming TCP segments on a server that runs a Microsoft Windows XP or Windows Server 2003 operating system:
  1. Start the Registry Editor (regedit.exe).
  2. Locate and then click the following registry subkey:
```
HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServicesTcpipParametersInterfaces
```
  3. On the Edit menu, click New > DWORD Value.
  4. Name the new value, TcpAckFrequency, and assign it a value of 1.
  5. Close the Registry Editor.
  6. Restart your Windows operating system.
Large page support
- Description: Using large pages can reduce the CPU overhead of managing a large JVM heap.
- How to view or set: The Windows operating system provides large page support by default. Use the -Xlp JVM option to make use of this support.