A cute little Case study

Qouted from :

http://raymondtay.blogspot.com/2008/09/j2eeweblogic-performance-tuning-with.html.

One of the excellent posts I have ever read.

his article contains a series of investigations for a customer of mine where the environment is running a WebLogic cluster of 20 machines in round-robin on HP-UX to service a global J2EE application and it performed slowly during peak periods and occasional hangs. The application was a typical 3-tier architecture whereby web relegates requests to the middle-tier (EJBs, MQs, MDBs) and this middleware goes to the Database-tier (SQL inserts, updates, deletes, stored procedures etc). The application was found to be experiencing heavy load during peak periods everyday.

There were a couple of issues related to poor performing SQLs, poorly designed middleware apps, WebLogic cluster design and runtime issues, JVM memory consumption and frequent garbage collections. Let me try to detail them a bit without giving away too much customer information. Hopefully, it can help you in your investigations in your environment.

During the peak period, the major contributing factors of the apps slow-ness were:

The heap size was 1.5GB (min,max), 512MB for Eden and the PermGen was 192MB. The minor GC kicked in frequently releasing approximately 60MB on average; the major GC kicked in twice every minute (avg. 3-5s on average, 40s on max) releasing 400 – 500 MB each time and reverse engineering the figures reveals that the object creation rate was roughly 800 M – 1.0 GB per minute. As GC is primarily a CPU-intensive operation (with saving state, freeing memory, compacting the heap etc). The large object creation rate combined with the relatively long pauses GCs occurences suggests that the application are creating objects in an in-efficient manner and that created problems with the cluster’s session replication mechanism as the users of the system would see stale data – due to long pauses in GC, the data in the session was not replicated *properly* across to the other servers.

Applications were attached to the WebLogic system classpath which meant that the Java classes were never unloaded from memory and combined with the fact that there are ALWAYS classloader leaks meant that whenever the operation team redeploy a.k.a “hot”-redeployment the apps, it worsens the memory footprint since the previous memory was never release due to this leakage. If you keep hot-deploying these stuff you will almost certainly get an OutOfMemory Error: PermGen out of space.

EJBs (4,000+ EJBs deployed, in my opinion too many) were utilizing remote interfaces when there was no need as those apps were not doing cross-vm operations and based on my previous experimentation, you would get a 3-fold runtime improvement when you convert the EJBs to local interfaces. This improvement is because there is less object marshalling/unmarshalling via RMI since everything is on the same JVM heap and consumes less system resources like file descriptors/socket & memory since local interfaces implies a local/normal Java call.

As i mentioned previously, the apps were deployed in the cluster and that meant that all persistent objects (e.g. session data, user preferences etc) must be Serializable (i.e. persistent objects need to implement java.lang.Serializable) since there would be session replication across the servers in the cluster which further degraded the performance as the cluster needs to maintain state across all 20 machines. Source code analysis found that user’s were keeping results of database fetches in session data! You can imagine the pressure faced by the JVM memory subsystem + WebLogic cluster replication.

WebLogic cluster was also malfunctioning during peak periods throwing an exception message like <WorkManager> <BEA-002911> <WorkManager weblogic.kernel.System failed to schedule a request due to weblogic.utils.UnsyncCircularQueue$FullQueueException: Queue exceed maximum capacity of: ‘65536′ elements and this is an critical error thrown from the Work Manager which replaced the BEA traditional thread pool. What this meant was that the WebLogic cluster could no longer handle user’s requests and hanged. *I plan to unravel this mystery in a while to understand why this is happening*

The hardware loadbalancer was in “sticky” mode even though the WebLogic cluster was in round-robin mode which negated this round-robin-ness and resulted in certain servers encountering more stress than others and this was made worse by the long session timeout of 20+ hrs. That’s the cost of doing business….

After tracing the SQL statements execution times, it was found that they were causing alot of problems from missing indexes, lack of functional indexes, improper SQL statements which causes large database table joins and many “select count(*)…” from large table joins statements contributed to this object creation rate.

When i looked at these issues, the first couple of items i advised my customer was to do the following:
(1) Convert the EJBs to use local interfaces i.e. call-by-reference
(2) Tune the SQL statements via SQL reordering, indexes etc
(3) Tune the JVM heap to use more aggressive + parallel heap collectors via -XX:+UseParallelGC -XX:+UseConcMarkSweepGC (We are still experimenting this portion)
(4) Do not use system classpath to load application classes
(5) Review source codes to remove known classloader leaks

 

Part 2:

In my previous article, my tuning project with a customer ran into some trouble with WebLogic’s Work Manager and in particular, on the Java exception weblogic.utils.UnsyncCircularQueue$FullQueueException where the WebLogic server indicated that the queue where the server works on submitted requests. Checked the WebLogic docs and accordingly, the server will automatically resize the queue to fit the requests but what the docs didn’t mention was that the resize will fail to work if the size of the queue equals the capacity which happens to be 65536 and that’s the reason why it threw the error message “Queue exceeded the maximum capacity of ‘65536′ elements”.

However, checking the code reveals something quite peculiar and that is the constructor suggests that only queues of sizes exceeding 1 GB will throw this error but the default capacity is always 256 and reaching a maximum of 65536 elements and btw, its WebLogic 9.2 ; so my guess is that source codes need to be cleaned up ? If anybody has any idea why, do drop me a comment, thanks in advanced.

 

Part 3:

 

 

analyzing heap dumps

jmap -histo <pid>   num     #instances         #bytes  class name
———————————————-
   1:         41202       27502288  [C
   2:          2304        1332520  [I
   3:         37934         910416  java.lang.String
   4:          5130         503632  <constMethodKlass>
   5:          5130         412968  <methodKlass>
   6:         12348         395136  fr.xebia.memory.bean.Person
   7:         12846         308304  java.util.concurrent.ConcurrentHashMap$HashEntry
   8:          7650         285656  <symbolKlass>
   9:         12518         200288  java.lang.Long
  10:           349         189184  <constantPoolKlass>
  11:           349         138704  <instanceKlassKlass>
  12:           327         132144  <constantPoolCacheKlass>
  13:            38         115456  [Ljava.util.concurrent.ConcurrentHashMap$HashEntry;
  14:           414          76392  [B
  15:           422          40512  java.lang.Class
  16:           556          32656  [[I
  17:           518          32056  [S
  18:            44          14080  <objArrayKlassKlass>
  19:           315          13480  [Ljava.lang.Object;
  20:           576           9216  java.lang.StringBuffer
…..
 184:             1              8  java.lang.System$2
 185:             1              8  java.io.File$1
 186:             1              8  java.lang.Terminator$1
Total        142211       32693376

 

First places are often trustees by the basic objects (java.lang.String, java.lang.Object, java.lang.Class) and declination in collections (java.util .* and tables, symbolized by [ ‘). If you find classes in your application in the top 20, they become suspicious. In our example, the histogram shows that the class’ fr.xebia.memory.bean.Person is instantiated 12,348 times for a memory footprint of 308,304 bytes. It is closely followed by entries from a ConcurrentHashMap (java.util.concurrent.ConcurrentHashMap $ HashEntry) Obtaining this histogram is very fast and somewhat disrupts the FMV (Full GC to close;)). It is possible to do regularly (eg every ‘n’ seconds) and thus deduce trends: number of instances of a class increases, size of instances of a class that decreases .. . Note: with JVM version 1.4.2, it is also possible to obtain a histogram for each thread dump by adding the option-XX: + PrintClassHistogram start of the JVM. As histo-jstat, this option automatically implies a Full GC memory of the JVM. The histogram lets you know the classes that occupy the most memory, it does not indicate who was allocated. The dump The histogram is good, but not sufficient: it is sometimes necessary to obtain a complete dump of the memory to determine accurately the chain of referral. Jmap The tool also allows a complete dump and binary memory of a JVM. Two important notes: during the dump, nothing running in the JVM, all threads are stopped, whether applications or systems. the duration of the dump is proportional to the size of the memory used and depending on the size of the remaining memory (it takes a bit of free memory to run the treatment of the dump!). Example: the duration of a dump of a JVM with a heap of 1 Go to the saturation limit of memory is counted in minutes! Example: Creating a dump

Node managers

Node Managers Weblogic Server utility to start, stop and restartAdministration and Managed Server Instances from remote location (There are other ways as well to start/stop Weblogic check here – Node Manager is optional component).

1. Node Manager Process is associated with a Machine and NOT with specific Weblogic Domain (i.e. Use one node manager for multiple domains on same machine)

2. There are two versions of Node Manager – Java-based and Script-based

Java-based node manager – runs with in JVM (Java Virtual Machine) Process and more secure than script-based node manager. Configuration for java-based node manager are stored in nodemanager.properties

Script-based node manager – is available for Linux and Unix systems only and is based on shell script.

3. There are multiple ways to access Node Manager
- From Administration Console : Environments -> Machines -> Configuration -> Node Manager
- JMX utilities (Java Management eXtension) more here
- WLST commands (WebLogic Scripting Tool)

4.Default port on which node manager listen for requests is localhost:5556, When you configure Node Manager to accept commands from remote systems, you must uninstall the default Node Manager service, then reinstall it to listen on a non-localhost (IP’s other than 127.0.0.1) listen address.

5. Any domain created before creation of Node Manager Service will not be accessible via node Manager(even after restarting node manager), solution is to run the WLST command “nmEnroll” to enroll that domain with the Node Manager.

6. Any domains created after the Node Manager service has been installed should not have to be enrolled against the Node Manager. The Node Manager should automatically be ‘reachable‘ by the domain.

.
How to Configure Node Manager ?

1.Configure each computer (on which you wish to use Node Manager) as a Machine in WebLogic Server
Environments -> Machines -> New (Add Machine) Use Link here
Environments -> Machines -> Machine Name (created above) -> Configuration -> Node Manager

2. Assign each server instance (Admin or Managed that you wish to control with Node Manager) to Machine.
Environments -> Machines -> Machine Name (created above) -> Configuration -> Servers -> Add (Add Server running on this node which you would like to monitor using Node Manager) for more info Click here

3. Enroll domain (created before installation of Node Manager) to Node Manager

Windows
cd $BEA_HOME\user_projects\domains\\bin
setDomainEnv.cmd
java weblogic.WLST
wls> connect(’weblogic’,’weblogic’, ‘t3://mymachine.mydomain:7001′)
wls> nmEnroll(’C:\bea\user_projects\domains/’, ‘C:\bea\wlserver_/common/nodemanager’)

.
Unix /Linux
cd $BEA_HOME/user_projects/domains//bin/
. setDomainEnv.sh
java weblogic.WLST
wls> connect(’weblogic’,’weblogic’, ‘t3://mymachine.mydomain:7001′)
wls> nmEnroll(’$BEA_HOME/user_projects/domains/’, ‘$BEA_HOME/wlserver_/common/nodemanager’)

where “mymachine.mydomain:7001″ is the reference to the Admin Server of the domain to which the server and machine definition belongs

.
How to start Node Manager ?

$WL_HOME\server\bin\startNodeManager.sh (startNodeManager.cmd on Windows)

How to install Node Manager as Service on Windows ?

Use $WLS_HOME\server\bin\installNodeMgrSvc.cmd (Where default WLS_HOME location is c:\bea\wlserver_)

To uninstall Node Manager Service on windows use $WLS_HOME\server\bin\uninstallNodeMgrSvc.cmd

installNodeMgrSvc.cmd will create Windows server with name as Oracle WebLogic NodeManager (C_bea_wlserver_)
.
Important Configuration files

– $WL_HOME/common/nodemanager/ nodemanager.properties, nodemanager.domains, nm_data.properties

–$DOMAIN_HOME/config/nodemanager/nm_password.properties

–$DOMAIN_HOME/servers//data/nodemanager/ boot.properties, startup.properties, server_name.addr, server_name.lck, server_name.pid, server_name.state

“check-passthrough” for defining cache ..

check-passthrough

The check-passthrough ObjectType SAF checks to see if the requested resource (for example, the HTML document or GIF image) is available on the local server. If the requested resource does not exist locally, check-passthrough sets the type to indicate that the request should be passed to another server for processing by service-passthrough.

The check-passthrough SAF accepts the following parameters:

*

type – (Optional) The type to use for files that do not exist locally. If not specified, type defaults to magnus-internal/passthrough.

All About Session

What’s a Session?

SSL makes a distinction between a connection and a session. A connection represents one specific communications channel (typically mapped to a TCP connection), along with its keys, cipher choices, sequence number state, etc. A session is a virtual construct representing the negotiated algorithms and the master_secret . A new session is created every time a given client and server go through a full key exchange and establish a new master_secret.

Multiple connections can be associated with a given session. Although all connections in a given session share the same master_secret, each has its own encryption keys. This is absolutely necessary for security reasons because reuse of bulk keying material can be extremely dangerous. Resumption allows the generation of a new set of bulk keys and IVs from a common master_secret because the keys depend on the random values, which are fresh for each connection. The new random values are combined with the old master_secret to produce new keys.
How It Works

The first time a client and server interact, they create both a new connection and a new session. If the server is prepared to resume the session, it assigns the session a session_id and transmits the session_id to the client during the handshake. The server caches the master_secret for later reference. When the client initiates a new connection with the server, it provides the session_id to the server. The server can choose to either resume the session or force a full handshake. If the server chooses to resume the session, the rest of the handshake is skipped, and the stored master_secret is used to generate all the cryptographic keys.

When Runtime.exec() won’t

As part of the Java language, the java.lang package is implicitly imported into every Java program. This package’s pitfalls surface often, affecting most programmers. This month, I’ll discuss the traps lurking in the Runtime.exec() method.

Pitfall 4: When Runtime.exec() won’t
The class java.lang.Runtime features a static method called getRuntime(), which retrieves the current Java Runtime Environment. That is the only way to obtain a reference to the Runtime object. With that reference, you can run external programs by invoking the Runtime class’s exec() method. Developers often call this method to launch a browser for displaying a help page in HTML.

There are four overloaded versions of the exec() command:

public Process exec(String command);
public Process exec(String [] cmdArray);
public Process exec(String command, String [] envp);
public Process exec(String [] cmdArray, String [] envp);

For each of these methods, a command — and possibly a set of arguments — is passed to an operating-system-specific function call. This subsequently creates an operating-system-specific process (a running program) with a reference to a Process class returned to the Java VM. The Process class is an abstract class, because a specific subclass of Process exists for each operating system.

You can pass three possible input parameters into these methods:

A single string that represents both the program to execute and any arguments to that program
An array of strings that separate the program from its arguments
An array of environment variables

Pass in the environment variables in the form name=value. If you use the version of exec() with a single string for both the program and its arguments, note that the string is parsed using white space as the delimiter via the StringTokenizer class.

Stumbling into an IllegalThreadStateException

The first pitfall relating to Runtime.exec() is the IllegalThreadStateException. The prevalent first test of an API is to code its most obvious methods. For example, to execute a process that is external to the Java VM, we use the exec() method. To see the value that the external process returns, we use the exitValue() method on the Process class. In our first example, we will attempt to execute the Java compiler (javac.exe):

Listing 4.1 BadExecJavac.java

import java.util.*;
import java.io.*;
public class BadExecJavac
{
public static void main(String args[])
{
try
{
Runtime rt = Runtime.getRuntime();
Process proc = rt.exec(“javac”);
int exitVal = proc.exitValue();
System.out.println(“Process exitValue: ” + exitVal);
} catch (Throwable t)
{
t.printStackTrace();
}
}
}

InfoWorld: Modernizing IT
JavaWorld: Solutions for Java Developers

Research Centers
Core Java
Enterprise Java
Mobile Java
Tools & Methods
JavaWorld Archives
Site Resources
Featured Articles
News & Views
Community
Java Q&A
JW Blogs
Podcasts
Site Map
Careers
Newsletters
Whitepapers
RSS Feeds
About JavaWorld
Advertise
Write for JW

JW’s Most Read
Recent:

Java Tips: The Serialization algorithm revealed
Know your Oracle application server
SwingX, JRuby: Survivors?
Cloud-ready, multicore-friendly apps, Part 2
Lean service architectures with Java EE 6
Archives:

Hello, OSGi: Bundles for beginners (2008)
Sockets programming in Java: A tutorial (1996)
Understanding JPA, Part 1 (2008)
Smartly load your properties (2003)
REST for Java developers, Part 1 (2008)

Featured White Papers

How to Choose a Web Application Monitoring Solution: Evaluation Guide & Checklist
Newsletter sign-upView all newsletters
Sign up for our technology specific newsletters.

Enterprise Java
Email Address:
Sponsored Links
Free Access to SAP BI Resource Center
Empower your users to discover how to explore business at the speed of thought.
Buy a link now

When Runtime.exec() won’t
Navigate yourself around pitfalls related to the Runtime.exec() method
By Michael C. Daconta, JavaWorld.com, 12/29/00

PrintEmailFeedbackResourcesDiscuss (25)DiggRedditSlashDotStumbledel.icio.usTechnoratidzone As part of the Java language, the java.lang package is implicitly imported into every Java program. This package’s pitfalls surface often, affecting most programmers. This month, I’ll discuss the traps lurking in the Runtime.exec() method.

Pitfall 4: When Runtime.exec() won’t
The class java.lang.Runtime features a static method called getRuntime(), which retrieves the current Java Runtime Environment. That is the only way to obtain a reference to the Runtime object. With that reference, you can run external programs by invoking the Runtime class’s exec() method. Developers often call this method to launch a browser for displaying a help page in HTML.

There are four overloaded versions of the exec() command:

public Process exec(String command);
public Process exec(String [] cmdArray);
public Process exec(String command, String [] envp);
public Process exec(String [] cmdArray, String [] envp);

For each of these methods, a command — and possibly a set of arguments — is passed to an operating-system-specific function call. This subsequently creates an operating-system-specific process (a running program) with a reference to a Process class returned to the Java VM. The Process class is an abstract class, because a specific subclass of Process exists for each operating system.

You can pass three possible input parameters into these methods:

A single string that represents both the program to execute and any arguments to that program
An array of strings that separate the program from its arguments
An array of environment variables

Pass in the environment variables in the form name=value. If you use the version of exec() with a single string for both the program and its arguments, note that the string is parsed using white space as the delimiter via the StringTokenizer class.

Stumbling into an IllegalThreadStateException

The first pitfall relating to Runtime.exec() is the IllegalThreadStateException. The prevalent first test of an API is to code its most obvious methods. For example, to execute a process that is external to the Java VM, we use the exec() method. To see the value that the external process returns, we use the exitValue() method on the Process class. In our first example, we will attempt to execute the Java compiler (javac.exe):

Listing 4.1 BadExecJavac.java

import java.util.*;
import java.io.*;
public class BadExecJavac
{
public static void main(String args[])
{
try
{
Runtime rt = Runtime.getRuntime();
Process proc = rt.exec(“javac”);
int exitVal = proc.exitValue();
System.out.println(“Process exitValue: ” + exitVal);
} catch (Throwable t)
{
t.printStackTrace();
}
}
}

A run of BadExecJavac produces:

E:\classes\com\javaworld\jpitfalls\article2>java BadExecJavac
java.lang.IllegalThreadStateException: process has not exited
at java.lang.Win32Process.exitValue(Native Method)
at BadExecJavac.main(BadExecJavac.java:13)

If an external process has not yet completed, the exitValue() method will throw an IllegalThreadStateException; that’s why this program failed. While the documentation states this fact, why can’t this method wait until it can give a valid answer?

A more thorough look at the methods available in the Process class reveals a waitFor() method that does precisely that. In fact, waitFor() also returns the exit value, which means that you would not use exitValue() and waitFor() in conjunction with each other, but rather would choose one or the other. The only possible time you would use exitValue() instead of waitFor() would be when you don’t want your program to block waiting on an external process that may never complete. Instead of using the waitFor() method, I would prefer passing a boolean parameter called waitFor into the exitValue() method to determine whether or not the current thread should wait. A boolean would be more beneficial because exitValue() is a more appropriate name for this method, and it isn’t necessary for two methods to perform the same function under different conditions. Such simple condition discrimination is the domain of an input parameter.

Therefore, to avoid this trap, either catch the IllegalThreadStateException or wait for the process to complete.

Now, let’s fix the problem in Listing 4.1 and wait for the process to complete. In Listing 4.2, the program again attempts to execute javac.exe and then waits for the external process to complete:

Listing 4.2 BadExecJavac2.java

import java.util.*;
import java.io.*;
public class BadExecJavac2
{
public static void main(String args[])
{
try
{
Runtime rt = Runtime.getRuntime();
Process proc = rt.exec(“javac”);
int exitVal = proc.waitFor();
System.out.println(“Process exitValue: ” + exitVal);
} catch (Throwable t)
{
t.printStackTrace();
}
}
}

Unfortunately, a run of BadExecJavac2 produces no output. The program hangs and never completes. Why does the javac process never complete?

Why Runtime.exec() hangs

The JDK’s Javadoc documentation provides the answer to this question:

Because some native platforms only provide limited buffer size for standard input and output streams, failure to promptly write the input stream or read the output stream of the subprocess may cause the subprocess to block, and even deadlock.

Is this just a case of programmers not reading the documentation, as implied in the oft-quoted advice: read the fine manual (RTFM)? The answer is partially yes. In this case, reading the Javadoc would get you halfway there; it explains that you need to handle the streams to your external process, but it does not tell you how.

Another variable is at play here, as is evident by the large number of programmer questions and misconceptions concerning this API in the newsgroups: though Runtime.exec() and the Process APIs seem extremely simple, that simplicity is deceiving because the simple, or obvious, use of the API is prone to error. The lesson here for the API designer is to reserve simple APIs for simple operations. Operations prone to complexities and platform-specific dependencies should reflect the domain accurately. It is possible for an abstraction to be carried too far. The JConfig library provides an example of a more complete API to handle file and process operations (see Resources below for more information).

Now, let’s follow the JDK documentation and handle the output of the javac process. When you run javac without any arguments, it produces a set of usage statements that describe how to run the program and the meaning of all the available program options. Knowing that this is going to the stderr stream, you can easily write a program to exhaust that stream before waiting for the process to exit. Listing 4.3 completes that task. While this approach will work, it is not a good general solution. Thus, Listing 4.3’s program is named MediocreExecJavac; it provides only a mediocre solution. A better solution would empty both the standard error stream and the standard output stream. And the best solution would empty these streams simultaneously (I’ll demonstrate that later).

Listing 4.3 MediocreExecJavac.java

import java.util.*;
import java.io.*;
public class MediocreExecJavac
{
public static void main(String args[])
{
try
{
Runtime rt = Runtime.getRuntime();
Process proc = rt.exec(“javac”);
InputStream stderr = proc.getErrorStream();
InputStreamReader isr = new InputStreamReader(stderr);
BufferedReader br = new BufferedReader(isr);
String line = null;
System.out.println(“”);
while ( (line = br.readLine()) != null)
System.out.println(line);
System.out.println(“”);
int exitVal = proc.waitFor();
System.out.println(“Process exitValue: ” + exitVal);
} catch (Throwable t)
{
t.printStackTrace();
}
}
}

A run of MediocreExecJavac generates:

E:\classes\com\javaworld\jpitfalls\article2>java MediocreExecJavac

Usage: javac
where includes:
-g Generate all debugging info
-g:none Generate no debugging info
-g:{lines,vars,source} Generate only some debugging info
-O Optimize; may hinder debugging or enlarge class files
-nowarn Generate no warnings
-verbose Output messages about what the compiler is doing
-deprecation Output source locations where deprecated APIs are used
-classpath Specify where to find user class files
-sourcepath Specify where to find input source files
-bootclasspath Override location of bootstrap class files
-extdirs Override location of installed extensions
-d Specify where to place generated class files
-encoding Specify character encoding used by source files
-target Generate class files for specific VM version

Process exitValue: 2

So, MediocreExecJavac works and produces an exit value of 2. Normally, an exit value of 0 indicates success; any nonzero value indicates an error. The meaning of these exit values depends on the particular operating system. A Win32 error with a value of 2 is a “file not found” error. That makes sense, since javac expects us to follow the program with the source code file to compile.

Thus, to circumvent the second pitfall — hanging forever in Runtime.exec() — if the program you launch produces output or expects input, ensure that you process the input and output streams.

Assuming a command is an executable program

Under the Windows operating system, many new programmers stumble upon Runtime.exec() when trying to use it for nonexecutable commands like dir and copy. Subsequently, they run into Runtime.exec()’s third pitfall. Listing 4.4 demonstrates exactly that:

Listing 4.4 BadExecWinDir.java

import java.util.*;
import java.io.*;
public class BadExecWinDir
{
public static void main(String args[])
{
try
{
Runtime rt = Runtime.getRuntime();
Process proc = rt.exec(“dir”);
InputStream stdin = proc.getInputStream();
InputStreamReader isr = new InputStreamReader(stdin);
BufferedReader br = new BufferedReader(isr);
String line = null;
System.out.println(“”);
while ( (line = br.readLine()) != null)
System.out.println(line);
System.out.println(“”);
int exitVal = proc.waitFor();
System.out.println(“Process exitValue: ” + exitVal);
} catch (Throwable t)
{
t.printStackTrace();
}
}
}

A run of BadExecWinDir produces:

E:\classes\com\javaworld\jpitfalls\article2>java BadExecWinDir
java.io.IOException: CreateProcess: dir error=2
at java.lang.Win32Process.create(Native Method)
at java.lang.Win32Process.(Unknown Source)
at java.lang.Runtime.execInternal(Native Method)
at java.lang.Runtime.exec(Unknown Source)
at java.lang.Runtime.exec(Unknown Source)
at java.lang.Runtime.exec(Unknown Source)
at java.lang.Runtime.exec(Unknown Source)
at BadExecWinDir.main(BadExecWinDir.java:12)

As stated earlier, the error value of 2 means “file not found,” which, in this case, means that the executable named dir.exe could not be found. That’s because the directory command is part of the Windows command interpreter and not a separate executable. To run the Windows command interpreter, execute either command.com or cmd.exe, depending on the Windows operating system you use. Listing 4.5 runs a copy of the Windows command interpreter and then executes the user-supplied command (e.g., dir).

Listing 4.5 GoodWindowsExec.java

import java.util.*;
import java.io.*;
class StreamGobbler extends Thread
{
InputStream is;
String type;

StreamGobbler(InputStream is, String type)
{
this.is = is;
this.type = type;
}

public void run()
{
try
{
InputStreamReader isr = new InputStreamReader(is);
BufferedReader br = new BufferedReader(isr);
String line=null;
while ( (line = br.readLine()) != null)
System.out.println(type + “>” + line);
} catch (IOException ioe)
{
ioe.printStackTrace();
}
}
}
public class GoodWindowsExec
{
public static void main(String args[])
{
if (args.length < 1)
{
System.out.println("USAGE: java GoodWindowsExec “);
System.exit(1);
}

try
{
String osName = System.getProperty(“os.name” );
String[] cmd = new String[3];
if( osName.equals( “Windows NT” ) )
{
cmd[0] = “cmd.exe” ;
cmd[1] = “/C” ;
cmd[2] = args[0];
}
else if( osName.equals( “Windows 95″ ) )
{
cmd[0] = “command.com” ;
cmd[1] = “/C” ;
cmd[2] = args[0];
}

Runtime rt = Runtime.getRuntime();
System.out.println(“Execing ” + cmd[0] + ” ” + cmd[1]
+ ” ” + cmd[2]);
Process proc = rt.exec(cmd);
// any error message?
StreamGobbler errorGobbler = new
StreamGobbler(proc.getErrorStream(), “ERROR”);

// any output?
StreamGobbler outputGobbler = new
StreamGobbler(proc.getInputStream(), “OUTPUT”);

// kick them off
errorGobbler.start();
outputGobbler.start();

// any error???
int exitVal = proc.waitFor();
System.out.println(“ExitValue: ” + exitVal);
} catch (Throwable t)
{
t.printStackTrace();
}
}
}

Running GoodWindowsExec with the dir command generates:

E:\classes\com\javaworld\jpitfalls\article2>java GoodWindowsExec “dir *.java”
Execing cmd.exe /C dir *.java
OUTPUT> Volume in drive E has no label.
OUTPUT> Volume Serial Number is 5C5F-0CC9
OUTPUT>
OUTPUT> Directory of E:\classes\com\javaworld\jpitfalls\article2
OUTPUT>
OUTPUT>10/23/00 09:01p 805 BadExecBrowser.java
OUTPUT>10/22/00 09:35a 770 BadExecBrowser1.java
OUTPUT>10/24/00 08:45p 488 BadExecJavac.java
OUTPUT>10/24/00 08:46p 519 BadExecJavac2.java
OUTPUT>10/24/00 09:13p 930 BadExecWinDir.java
OUTPUT>10/22/00 09:21a 2,282 BadURLPost.java
OUTPUT>10/22/00 09:20a 2,273 BadURLPost1.java
… (some output omitted for brevity)
OUTPUT>10/12/00 09:29p 151 SuperFrame.java
OUTPUT>10/24/00 09:23p 1,814 TestExec.java
OUTPUT>10/09/00 05:47p 23,543 TestStringReplace.java
OUTPUT>10/12/00 08:55p 228 TopLevel.java
OUTPUT> 22 File(s) 46,661 bytes
OUTPUT> 19,678,420,992 bytes free
ExitValue: 0

Running GoodWindowsExec with any associated document type will launch the application associated with that document type. For example, to launch Microsoft Word to display a Word document (i.e., one with a .doc extension), type:

>java GoodWindowsExec “yourdoc.doc”

Notice that GoodWindowsExec uses the os.name system property to determine which Windows operating system you are running — and thus determine the appropriate command interpreter. After executing the command interpreter, handle the standard error and standard input streams with the StreamGobbler class. StreamGobbler empties any stream passed into it in a separate thread. The class uses a simple String type to denote the stream it empties when it prints the line just read to the console.

Thus, to avoid the third pitfall related to Runtime.exec(), do not assume that a command is an executable program; know whether you are executing a standalone executable or an interpreted command. At the end of this section, I will demonstrate a simple command-line tool that will help you with that analysis.

It is important to note that the method used to obtain a process’s output stream is called getInputStream(). The thing to remember is that the API sees things from the perspective of the Java program and not the external process. Therefore, the external program’s output is the Java program’s input. And that logic carries over to the external program’s input stream, which is an output stream to the Java program.

Runtime.exec() is not a command line

One final pitfall to cover with Runtime.exec() is mistakenly assuming that exec() accepts any String that your command line (or shell) accepts. Runtime.exec() is much more limited and not cross-platform. This pitfall is caused by users attempting to use the exec() method to accept a single String as a command line would. The confusion may be due to the fact that command is the parameter name for the exec() method. Thus, the programmer incorrectly associates the parameter command with anything that he or she can type on a command line, instead of associating it with a single program and its arguments. In listing 4.6 below, a user tries to execute a command and redirect its output in one call to exec():

Listing 4.6 BadWinRedirect.java

import java.util.*;
import java.io.*;
// StreamGobbler omitted for brevity
public class BadWinRedirect
{
public static void main(String args[])
{
try
{
Runtime rt = Runtime.getRuntime();
Process proc = rt.exec(“java jecho ‘Hello World’ > test.txt”);
// any error message?
StreamGobbler errorGobbler = new
StreamGobbler(proc.getErrorStream(), “ERROR”);

// any output?
StreamGobbler outputGobbler = new
StreamGobbler(proc.getInputStream(), “OUTPUT”);

// kick them off
errorGobbler.start();
outputGobbler.start();

// any error???
int exitVal = proc.waitFor();
System.out.println(“ExitValue: ” + exitVal);
} catch (Throwable t)
{
t.printStackTrace();
}
}
}

Running BadWinRedirect produces:

E:\classes\com\javaworld\jpitfalls\article2>java BadWinRedirect
OUTPUT>’Hello World’ > test.txt
ExitValue: 0

The program BadWinRedirect attempted to redirect the output of an echo program’s simple Java version into the file test.txt. However, we find that the file test.txt does not exist. The jecho program simply takes its command-line arguments and writes them to the standard output stream. (You will find the source for jecho in the source code available for download in Resources.) In Listing 4.6, the user assumed that you could redirect standard output into a file just as you could on a DOS command line. Nevertheless, you do not redirect the output through this approach. The incorrect assumption here is that the exec() method acts like a shell interpreter; it does not. Instead, exec() executes a single executable (a program or script). If you want to process the stream to either redirect it or pipe it into another program, you must do so programmatically, using the java.io package. Listing 4.7 properly redirects the standard output stream of the jecho process into a file.

Listing 4.7 GoodWinRedirect.java

import java.util.*;
import java.io.*;
class StreamGobbler extends Thread
{
InputStream is;
String type;
OutputStream os;

StreamGobbler(InputStream is, String type)
{
this(is, type, null);
}
StreamGobbler(InputStream is, String type, OutputStream redirect)
{
this.is = is;
this.type = type;
this.os = redirect;
}

public void run()
{
try
{
PrintWriter pw = null;
if (os != null)
pw = new PrintWriter(os);

InputStreamReader isr = new InputStreamReader(is);
BufferedReader br = new BufferedReader(isr);
String line=null;
while ( (line = br.readLine()) != null)
{
if (pw != null)
pw.println(line);
System.out.println(type + “>” + line);
}
if (pw != null)
pw.flush();
} catch (IOException ioe)
{
ioe.printStackTrace();
}
}
}
public class GoodWinRedirect
{
public static void main(String args[])
{
if (args.length < 1)
{
System.out.println("USAGE java GoodWinRedirect “);
System.exit(1);
}

try
{
FileOutputStream fos = new FileOutputStream(args[0]);
Runtime rt = Runtime.getRuntime();
Process proc = rt.exec(“java jecho ‘Hello World’”);
// any error message?
StreamGobbler errorGobbler = new
StreamGobbler(proc.getErrorStream(), “ERROR”);

// any output?
StreamGobbler outputGobbler = new
StreamGobbler(proc.getInputStream(), “OUTPUT”, fos);

// kick them off
errorGobbler.start();
outputGobbler.start();

// any error???
int exitVal = proc.waitFor();
System.out.println(“ExitValue: ” + exitVal);
fos.flush();
fos.close();
} catch (Throwable t)
{
t.printStackTrace();
}
}
}

Running GoodWinRedirect produces:

E:\classes\com\javaworld\jpitfalls\article2>java GoodWinRedirect test.txt
OUTPUT>’Hello World’
ExitValue: 0

After running GoodWinRedirect, test.txt does exist. The solution to the pitfall was to simply control the redirection by handling the external process’s standard output stream separately from the Runtime.exec() method. We create a separate OutputStream, read in the filename to which we redirect the output, open the file, and write the output that we receive from the spawned process’s standard output to the file. Listing 4.7 completes that task by adding a new constructor to our StreamGobbler class. The new constructor takes three arguments: the input stream to gobble, the type String that labels the stream we are gobbling, and the output stream to which we redirect the input. This new version of StreamGobbler does not break any of the code in which it was previously used, as we have not changed the existing public API — we only extended it.

Since the argument to Runtime.exec() is dependent on the operating system, the proper commands to use will vary from one OS to another. So, before finalizing arguments to Runtime.exec() and writing the code, quickly test the arguments. Listing 4.8 is a simple command-line utility that allows you to do just that.

Here’s a useful exercise: try to modify TestExec to redirect the standard input or standard output to a file. When executing the javac compiler on Windows 95 or Windows 98, that would solve the problem of error messages scrolling off the top of the limited command-line buffer.

Listing 4.8 TestExec.java

import java.util.*;
import java.io.*;
// class StreamGobbler omitted for brevity
public class TestExec
{
public static void main(String args[])
{
if (args.length java TestExec “e:\java\docs\index.html”
java.io.IOException: CreateProcess: e:\java\docs\index.html error=193
at java.lang.Win32Process.create(Native Method)
at java.lang.Win32Process.(Unknown Source)
at java.lang.Runtime.execInternal(Native Method)
at java.lang.Runtime.exec(Unknown Source)
at java.lang.Runtime.exec(Unknown Source)
at java.lang.Runtime.exec(Unknown Source)
at java.lang.Runtime.exec(Unknown Source)
at TestExec.main(TestExec.java:45)

Our first test failed with an error of 193. The Win32 error for value 193 is “not a valid Win32 application.” This error tells us that no path to an associated application (e.g., Netscape) exists, and that the process cannot run an HTML file without an associated application.

Therefore, we try the test again, this time giving it a full path to Netscape. (Alternately, we could add Netscape to our PATH environment variable.) A second run of TestExec produces:

E:\classes\com\javaworld\jpitfalls\article2>java TestExec
“e:\program files\netscape\program\netscape.exe e:\java\docs\index.html”
ExitValue: 0

This worked! The Netscape browser launches, and it then loads the Java help documentation.

One additional improvement to TestExec would include a command-line switch to accept input from standard input. You would then use the Process.getOutputStream() method to pass the input to the spawned external program.

To sum up, follow these rules of thumb to avoid the pitfalls in Runtime.exec():

You cannot obtain an exit status from an external process until it has exited
You must immediately handle the input, output, and error streams from your spawned external process
You must use Runtime.exec() to execute programs
You cannot use Runtime.exec() like a command line

Correction to Pitfall 3
In the discussion of Pitfall 3 (“Don’t mix floats and doubles when generating text or XML messages”) in my last column, I incorrectly stated that the different string representation of a decimal number after casting it from a float to a double was a bug. While this is a pitfall, its cause is not a bug, but the fact that the decimal numbers in question — 100.28 and 91.09 — do not represent precisely in binary. I’d like to thank Thomas Okken and the others who straightened me out. If you enjoy discussing the finer points of numerical methods, you can email Thomas.

The combination of forgetting my numerical methods class, the numerous bug reports on the bug parade, and the automatic rounding of floats and doubles when printing (but not after casting a float to a double) threw me. I apologize for confusing anyone who read the article, especially to new Java programmers. I present two better solutions to the problem:

The first possible solution is to always specify the desired rounding explicitly with NumberFormat. In my case, I use the float and double to represent dollars and cents; therefore, I need only two significant digits. Listing C3.1 demonstrates how to use the NumberFormat class to specify a maximum of two fraction digits.

Listing C3.1 FormatNumbers.java

import java.text.*;
public class FormatNumbers
{
public static void main(String [] args)
{
try
{
NumberFormat fmt = NumberFormat.getInstance();
fmt.setMaximumFractionDigits(2);
float f = 100.28f;
System.out.println(“As a float : ” + f);
double d = f;
System.out.println(“Cast to a double : ” + d);
System.out.println(“Using NumberFormat: ” + fmt.format(d));
} catch (Throwable t)
{
t.printStackTrace();
}
}
}

When we run the FormatNumbers program, it produces:

E:\classes\com\javaworld\jpitfalls\article2>java FormatNumbers
As a float : 100.28
Cast to a double : 100.27999877929688
Using NumberFormat: 100.28

As you can see — regardless of whether we cast the float to a double — when we specify the number of digits we want, it properly rounds to that precision — even if the number is infinitely repeating in binary. To circumvent this pitfall, control the formatting of your doubles and floats when converting to a String.

A second, simpler solution would be to not use a float to represent cents. Integers (number of pennies) can represent cents, with a legal range of 0 to 99. You can check the range in the mutator method.

Qouted from an Article Whose author is :
Michael C. Daconta is the director of Web and technology services for McDonald Bradley, where he conducts training seminars and develops advanced systems with Java, JavaScript, and XML. Over the past 15 years, Daconta has held every major development position, including chief scientist, technical director, chief developer, team leader, systems analyst, and programmer. He is a Sun-certified Java programmer and coauthor of Java Pitfalls (John Wiley & Sons, 2000), Java 2 and JavaScript for C and C++ Programmers (John Wiley & Sons, 1999), and XML Development with Java 2 (Sams Publishing, 2000). In addition, he is the author of C++ Pointers and Dynamic Memory Management (John Wiley & Sons, 1995).

WebLogic 10.3 JMX

I have been using Oracle WebLogic 10.3 for a while in my current assignment. I wanted to publish my own custom MBeans and be able to see and edit properties and invoke methods on them. However, the built-in management console does not support custom MBeans (without adding custom configuration). I ended up using JManage which is an open source management console implemented as a web application.

The problem was that JManage 2.0-RC1 does not support WebLogic 10.3 and their last release was made two years ago. However, since they support version 9 I figured it might be possible to upgrade the jar files to make it work with 10.3. This is how you do it:

In the folder $JMANAGE_HOME/modules/weblogic9 remove wljmxclient.jar and add wlfullclient.jar and com.bea.core.descriptor.wl_1.1.0.0.jar instead. Here are the instructions how to create the wlfullclient.jar. The other jar is included in $BEA_HOME/modules.

I have also tried jconsole, the built-in WLDF console extension and wlnav but all of them had problems:
# To make jconsole work remotely you must change how the WebLogic server is started. As I really want to use the standard installation package this won’t work
# The WLDF console does not allow method invocations and I can only get it to show the built-in MBeans
# wlnav is built on wlst and seemed very powerful, but was quite ugly and I couldn’t find my custom MBeans

Although JManage is good enough for my current purposes I am interested in other alternatives for remote JMX management of WebLogic!

Creating SSL enabled origin server

If you already have an SSL enabled origin server you can skip this.

For ease of use I have used SJS Web Server 7.0 as origin server also.
Start the administration server, and go to wadm
>./wadm –user=admin
Please enter admin-user-password> ****

1. Create a self signed certificate
wadm>create-selfsigned-cert –config=test.sun.com –server-name=test.sun.com –nickname=My-CA-Cert
2. Create a HTTP listener
wadm> create-http-listener –listener-port=8888 –config=test.sun.com –server-name=test.sun.com –default-virtual-server-name=test.sun.com mylistener
3. Enable SSL for this listener and set the server certificate
wadm> set-ssl-prop –config=test.sun.com –http-listener=mylistener server-cert-nickname=My-CA-Cert enabled=true
4. Deploy the changes
wadm> deploy-config test.sun.com
5. Start this origin server instance.

Settings in Web Server 7.0 instance
Lets say we want to forward all requests to /xyz to the origin server. Go the Web Server instance config directory and modify the obj.conf as given below

AuthTrans fn=”match-browser” browser=”*MSIE*” ssl-unclean-shutdown=”true”
NameTrans fn=”ntrans-j2ee” name=”j2ee”
NameTrans fn=”pfx2dir” from=”/mc-icons” dir=”/export2/mv/lib/icons” name=”es-internal”
NameTrans fn=”map” from=”/xyz” name=”reverse-proxy-/” to=”/xyz”
PathCheck fn=”uri-clean”
…

Service fn=”proxy-retrieve” method=”*”

Route fn=”set-origin-server” server=”https://test.sun.com:8888″

**Note that I have given manual steps here. In my last blog I have given Administration CLI steps.

Lets say for this instance server.xml has 8080.
Make sure that the origin server is up and running.
Start the server and access http://test.sun.com:8080/xyz/ should show you xyz directory in the docroot of the origin server.
Troubleshooting
In case we get a Gateway Timeout error and in error logs we see some error like

[23/Jul/2007:16:44:11] failure (27927): for host …. trying to GET …., service-http reports: HTTP7758: error sending request (SEC_ERROR_UNTRUSTED_ISSUER: Client certificate is signed by an untrusted issuer.)

We get this error because the origin server’s certificate was not issued by a trusted CA. It means we need to export CA certificate of the origin server instance and import it into Web Server instance.
Export the origin server’s CA certificate
Go to /config directory of the origin server, and list certificates and then use pk12util to export the certificate.
>../../bin/certutil -L -d .
My-CA-Cert CTu,CTu,CTu

>../../bin/pk12util -o /tmp/exported.crt -n My-CA-Cert -d .
Import the origin server CA certificate in server instance config directory
Initialize NSS Database
To import certificate in server instance config directory you have to first initialize the NSS database. Note that if this Web Server instance is SSL enabled you can skip this NSS database intialization part.
>../../bin/certutil -N -d .
Enter Password or Pin for “NSS Certificate DB”:
Re-enter password:
Password changed successfully.
Import the certificate
Lets say the file /tmp/exported.crt contained the CA cert of the origin server, import that to NSS database.
> ../../bin/pk12util -i /tmp/exported.crt -d . -v
Enter Password or Pin for “NSS Certificate DB”:
Enter password for PKCS12 file:
pk12util: PKCS12 IMPORT SUCCESSFUL

Confirm by listing certs
>../../bin/certutil -L -d .
My-CA-Cert u,u,u
Modify trust flags if required
You can see that the certificates imported doesn’t contain ‘CT’ trust flags.
>../../bin/certutil -M -n My-CA-Cert -t ‘CTu,CTu,CTu’ -d .

Now u can see it is now a proper CA cert
>../../bin/certutil -L -d .
My-CA-Cert CTu,CTu,CTu

Restart the server instance and things should work fine now.

Cron jobs

The basic format of a crontab schedule consists of 6 fields, separated by spaces, formatted as follows:

minute hour day month day-of-week command-line-to-execute

The acceptable values for each of the 6 fields are:
Field Range of values
minute 0-59
hour 0-23
day 1-31
month 1-12
day-of-week 0-7 (where both 0 and 7 mean Sun, 1 = Mon, 2 = Tue, etc)
command-line-to-execute the command to run along with the parameters to that command if any

The fields have to be in that exact order, with no empty or missing fields.

“Minute” is a number from 0 to 59. “Hour” is a number from 0 to 23. They represent the time of the day in a 24-hour day format, so for example, if you want a certain command to run at 5.30 am, you will have to code it as:

30 5

If you want something run at 8 pm, it has to be coded as

0 20

since 2000 hours is 8 pm in the 24-hour time format.

“Day” and “month” refer to dates. “Day” takes a value between 1 and 31, and “month”, as you may have already guessed, can take any value between 1 and 12. So if you want a command run on 5th January at 9.15 am, your schedule should begin with the following:

15 9 5 1

“Day-of-week” means basically which day you want your command to run. If you want your command to run on Sundays, use either 0 or 7 here. If you want it on Monday, use 1. (Note: if you are getting worried at this point how to combine all the various fields, some of which seem to contradict the other, don’t worry. We’re getting to that.)

The trick to scheduling things, say, once a day, or once in 2 hours or the like, is to use a wildcard character. A wildcard character is like the Joker in a pack of playing cards. It can represent any card in the pack. Similarly, in a crontab file, the wildcard character “*” (the asterisk, without the quotes), represents every possible value for the field.

If you want a particular program to run, say, once every day at 10.45 am, the time portion of the cron schedule should read:

45 10 * * *

Here’s how to read the above line.

The first two fields “45 10″ means that you want it to run at 10.45. The next field, the day field, is set to * (the asterisk character) to show that we’re talking about 10.45 every day, not just the 1st of the month (which would be “1″) or the 30th of the month (“30″) or some other number.

The month field is set to the asterisk as well. If we set some number in the month field, say “2″, we will be saying that we only want the command to run at 10.45 in the month of February (“2″). Since that’s not what we need, we put the asterisk to mean every month.

Similarly, the day-of-week field is set to the asterisk, because we want the command to run whether it’s Sunday (“0″) or Monday (“1″) or whatever day.
More Examples: An Hourly Schedule

Now if you want a job to run every hour on the hour, you will have to set the time component of the crontab line as follows:

0 * * * *

Can you see why? The “0″ means at the top of the hour, that is, when the minute readout on a digital clock shows “00″. The asterisk in the hour field means every single hour. In other words, every hour, on the hour.
Alternate Hour or 3 Hourly Schedule

If you want something to run once every two hours, you will have to use the slash, “/”, character in your field. The slash character is the “step” character. In the case of a two hourly schedule, your time component of your cron file will read:

* */2 * * *

The second field, “*/2″, means every alternate hour.

Similarly, if you want something to run every 3 hours, you can change that field to “*/3″, and so on.
Other Examples

If you want a particular command to run only at 8.00am on the 1st and 20th of every month, you should code the time as:

0 8 1,20 * *

The comma, “,”, means “and”. If you are confused by the above line, remember that spaces are the field separators, not commas.

What does the following schedule mean?

2 3 4,5 6 7

Decoded, the above line says at 3:02 am on the 4th and 5th of June (6) and on every Sunday (7), run your program.

There are other possibilities in the time fields, and I won’t go through all of them, since you already know enough to be able to construct whatever schedule you need.
3.
How to specify the command line

The command-line-to-execute portion of the schedule is basically the command you want run at the specified time. For example, if you have a Perl script called “whatever.pl” that you want run every day at 11.30 pm, your crontab schedule might read as follows:

30 11 * * * /your/directory/whatever.pl

If your script is one of those that must be called from a web browser, like “cron.php” on a Drupal installation, you will need to use a command called “wget”. Technically, wget is not really a browser, but it works adequately like one for our purpose here, which is to simply get the web server to run the script called “cron.php”.

30 11 * * * /usr/bin/wget http://www.example.com/cron.php

In this case, you will need to specify the full path to wget since, for security reasons, the cron program (not the cron.php script) usually executes in a sort of clean environment, which means that it won’t know how to find the command “wget” unless you tell it where it is.

Another important thing to note about this crontab line that we’re constructing is that the entire line, schedule and command to execute, must fit into one line. You cannot put it into two lines for aesthetic reasons even if your command is very long.
4.
How to be notified of errors

Since you are running your script as a scheduled task, there will be nobody there to view its output. By default, cron will send any output from the script in an email to you, if it knows your email address.

If your script is very talkative, and issues all sort of information when it executes, you’ll probably want to shut it up (unless you are starved for email messages). To do this, we need to send all the normal output to a place called “/dev/null” which is basically like a black hole. It accepts anything you dump there, but you will never see it again. In the case of our first example, modify the command line to read:

30 11 * * * /your/directory/whatever.pl >/dev/null

The “>” sign means to redirect every normal message sent to screen to whatever is next in the command line, which, in our case, is /dev/null. If your script is designed to work correctly in a Unix environment, only the normal output will be swallowed up. Error messages will still be processed by the cron program. This is desirable, since you will want to informed when something is wrong so that you can fix the problem.

To receive the remaining unredirected messages, you will need to add another line to your crontab schedule to specify your email address. Use the following format:

MAILTO=myemailaddress@example.com
30 11 * * * /your/directory/whatever.pl >/dev/null

The MAILTO line must be on a separate line. It is optional. That is, you don’t have to specify it if you don’t want to. Depending on how your web host has set up the system, cron might still be able to successfully send you error messages. If you really don’t want to hear from cron at all, you will need to make your MAILTO line look like this:

MAILTO=”"

5.
Create a text file with all the necessary lines

There are many ways to feed all the lines you’ve constructed so far to crontab. I think the least problematic way for newcomers is to create an ASCII text file on your own computer containing all the needed lines. Use an ASCII text editor to do this. If you use Windows, use the program called Notepad, found in the Accessories folder of your Start menu. Do not use Microsoft Office or Word or some such fancy word processor program.

Open a new document with your text editor, and type in the MAILTO line and your schedule (or just your schedule alone if you don’t want to be notified when an error occurs). Remember, everything must fit into one line. Do not allow your editor to wrap the line. After you have typed in your line, hit the ENTER (or RETURN key on the Mac) so that your cursor is on a new blank line just after your schedule. Read that last sentence again; it’s important.

Save the file using any name, but with a “.txt” extension. You especially need to do this if you are using a system like Windows which use a different line ending from Unix systems. The “.txt” extension will cause your FTP program to translate the line ending to a Unix line endingwhen it uploads the file. If you’re not sure, just name the file “crontab.txt” (whatever system you happen to be using), and you should be fine.
6.
Upload the file to your web server using an FTP program

Now upload the file to your site using your FTP program. The file must be uploaded in ASCII mode, which your FTP program will probably automatically do when it sees that you’re uploading a file with a “.txt” extension.
7.
Connect to your web server’s shell account

Connect to your web host’s shell using your telnet or SSH software. If you don’t have one, download one from the Free SSH (Secure Shell) and Telnet Clients page. I will not go into the details of how you should use the software to log into your shell account, since the method varies according to the program you have downloaded. Ask your web host if you have problems.
8.
Invoke crontab to set your cron job

From the shell command line, go to where you uploaded your crontab.txt file. You can use “cd” (which means “change directory”) to change to the appropriate directory. Then type in the line below followed by the ENTER key (or RETURN key on the Mac).
crontab crontab.txt

where crontab.txt is the name of the file you created and uploaded earlier.

That command will cause the program “crontab” to install your cronjob. At the appropriate time, a separate program called “cron” will execute your job. If you are returned back to the shell prompt with no messages, it means all was well, and you can check that your schedule was correctly set by using the following command:
crontab -l

where the “-l” means list all the schedules or cronjobs that you have set so far. If you find you made a mistake, or if you wish to reschedule your tasks, you can remove the existing schedules with the following command:
crontab -r

You can add more schedules the same way. Put all your different schedules into a single crontab file. The first line should be your MAILTO line, followed by each schedule/command on a separate line. When you run “crontab crontab.txt” later, crontab will replace your existing schedule with the schedules in your new crontab.txt.

Once you’ve completed everything, you can delete your crontab.txt file. It’s no longer needed.

Dealing with Error Messages from Crontab

If you received an error message when you used “crontab crontab.txt”, you probably made a mistake creating your crontab file. Here are the common errors:

1.

You split your schedule/command line into more than one line. Every scheduled task must fit into a single line. Do not allow your text editor to wrap it to another line. Switch off the word wrap feature in your editor if it has one.
2.

You did not end your file in a new line. That is, you failed to hit the ENTER (or RETURN) key after your last command line in the crontab.txt file.
3.

You did not upload the file in ASCII mode. Name your file with a .txt extension to have your FTP program automatically convert it. Alternatively, learn how to change the FTP upload mode to ASCII.
4.

You used a word processor to create your crontab file. If the editor you were using allows you to do things like underline text, make nice bullet points, or do anything fancy besides typing raw text, you’re probably using the wrong program.

Clustering at the Apache end

Clustering of services is done to avoid down time. Before that there are few things that you need to check if you really need a clustered service.

Cluster considerations:
How stateful is your application?
Do you really need sessions?
– If yes, use sticky sessions
Can a client afford to lose a session on
node failure / shut down?
– It not, use session replication
How to connect front-end to Tomcat?
– mod_proxy_http recommended

The stateless approach :
Sessions will not work
• Requests will alternate between Tomcat
instances
• Multiple Tomcat instances
– same machine (must use different IP / port)
– different machines
• Required httpd modules:
– mod_proxy, mod_proxy_balancer,
mod_proxy_http

• httpd.conf
# Cluster definition

BalancerMember http://192.168.0.31:8080 disablereuse=On
BalancerMember http://192.168.0.32:8080 disablereuse=On

# Pass all requests except the manager to the cluster
ProxyPass /balancer-manager !
ProxyPass / balancer://devcluster/
# Configure the manager

SetHandler balancer-manager
Order Deny,Allow
Deny from all
Allow from 127.0.0.1

Add sticky session support:
• Tomcat configuration
– server.xml
– Set
– jvmRoute must be unique for each instance

• httpd.conf
# Cluster definition

BalancerMember http://192.168.0.31:8080 route=tc01
disablereuse=On
BalancerMember http://192.168.0.32:8080 route=tc02
disablereuse=On

# Pass all requests except the manager to the cluster
ProxyPass /balancer-manager !
ProxyPass / balancer://devcluster/
nofailover=On stickysession=JSESSIONID|jsessionid
# Balancer as before

Add session replication
• Application configuration
– WEB-INF/web.xml
– Add the element
• Keep the session as small as possible
• Session attributes must implement
Serializable
• Tomcat configuration – server.xml
– Uncomment element under

• The same element can also
be used under <Host …
• Defaults to get you started
• Overview:
–/docs/cluster-howto.html
• Details:
–/docs/config/cluster.html

• httpd.conf
# Cluster definition

BalancerMember http://192.168.0.31:8080 route=tc01
disablereuse=On
BalancerMember http://192.168.0.32:8080 route=tc02
disablereuse=On

# Pass all requests except the manager to the cluster
ProxyPass /balancer-manager !
ProxyPass / balancer://devcluster/
nofailover=Off stickysession=JSESSIONID|jsessionid
# Balancer as before

How to avoid down time:

• Use manager to disable route to tc01
• Wait for current requests to finish
• Upgrade / reconfigure instance tc01
• Use manager to enable route to tc01
• Repeat for tc02

Reliability
• Redeployment can cause memory leaks
• Caused by static references to the class
loader
– shared libraries
• Include redeployment in testing

Concurrent requests:
• Tomcat maintains a thread pool
• Incoming request is assigned to a free
thread
• If all threads are busy, request is queued
• If queue is full, request is rejected
• Running out of threads is usually a
symptom of an application issue
• If you do need more threads – server.xml
– maxThreads
• For highly concurrent environments
– maxKeepAliveRequests=”1”
– connectionTimeout=”3000”

Quoted from a PPT:”http://www.springsource.com/files/OptimizingAndTuningApacheTomcat.pdf”