I recently came across Colm MacCarthaigh’s Apache tuning presentation and technical white paper:
Tuning Apache and Linux for performance presentation:
The presentation and white paper cover the entire software stack, which includes kernel, file system and of course Apache web server tuning
Posted in Apache | Leave a Comment »
Apache allows you to create hundreds of virtual host containers. Each container is required to have a ServerName directive, which contains the domain name associated with the virtual host. In addition to a server name, one ore more aliases can be associated with the virtual host with the ServerAlias directive. Aliases can contain a domain, or a regex that allows you to match based on some specific criteria. This is super useful, and allows you to do things like this:
NameVirtualHost 192.168.1.18:8080
ServerName foo.com
ServerAlias *.foo.com
ServerName bar.com
ServerAlias *.bar.com
Posted in Apache | Leave a Comment »
a reference to mod_substitute. This nifty module allows you to substitute text in the HTTP request body, which provides an easy way to do things similar to the following:
AddOutputFilterByType SUBSTITUTE text/html
Substitute s/SECRET/XXXXX/ni
Posted in Apache | Leave a Comment »
Jmap is a very usefull tool to take heap dump for analysis on the memory part.
Java has become one of the most successful languages to hit the IT industry. One of the reasons behind it’s high rate of adoption is that fact that Java manages memory resources for the programmer. This makes programming significantly easier, but introduces additional complexity, since engineers need to size the Java heap and pick a garbage collection algorithm (or in the case of generational collectors, more than one) that best matches an application’s workload. Fortunately, two extremely useful tools allow you to observe the Java heap: the DTrace hotspot provider and the Java jmap utility. In this blog post, I will discuss the jmap utility.
The Java jmap utility provides a number of useful options to summarize heap usage, and get a break down of objects in the new and old generations. To summarize the new and old generations, the jmap utility can be run with the “-heap” option, and the PID of the JVM to summarize (the PID can be acquired by running the jps utility, or by reviewing the ps output):
$ jmap -heap `pgrep java`
Attaching to process ID 5365, please wait…
Debugger attached successfully.
Server compiler detected.
JVM version is 1.6.0_01-b06
using parallel threads in the new generation.
using thread-local object allocation.
Concurrent Mark-Sweep GC
Heap Configuration:
MinHeapFreeRatio = 40
MaxHeapFreeRatio = 70
MaxHeapSize = 1073741824 (1024.0MB)
NewSize = 268435456 (256.0MB)
MaxNewSize = 268435456 (256.0MB)
OldSize = 805306368 (768.0MB)
NewRatio = 7
SurvivorRatio = 6
PermSize = 21757952 (20.75MB)
MaxPermSize = 88080384 (84.0MB)
Heap Usage:
New Generation (Eden + 1 Survivor Space):
capacity = 234881024 (224.0MB)
used = 102062424 (97.33431243896484MB)
free = 132818600 (126.66568756103516MB)
43.452818053109304% used
Eden Space:
capacity = 201326592 (192.0MB)
used = 94318424 (89.94905853271484MB)
free = 107008168 (102.05094146728516MB)
46.84846798578898% used
From Space:
capacity = 33554432 (32.0MB)
used = 7744000 (7.38525390625MB)
free = 25810432 (24.61474609375MB)
23.07891845703125% used
To Space:
capacity = 33554432 (32.0MB)
used = 0 (0.0MB)
free = 33554432 (32.0MB)
0.0% used
concurrent mark-sweep generation:
capacity = 805306368 (768.0MB)
used = 15032688 (14.336288452148438MB)
free = 790273680 (753.6637115478516MB)
1.8667042255401611% used
Perm Generation:
capacity = 50577408 (48.234375MB)
used = 30285240 (28.88225555419922MB)
free = 20292168 (19.35211944580078MB)
59.87898786746842% used
The jmap output contains the size of eden (the place where new objects are created), each survivor space, the old generation and the permanent generation. If you aren’t familiar with the generations that are used in modern JVMs, you might want to take a look at Tony Printezis’ article on this topic.
In addition to printing summary data, jmap can also be used to display the objects in the heap.
$ jmap -histo `pgrep java` |more
num #instances #bytes class name
————————————–
1: 224437 27673848 [C
2: 38611 23115312 [B
3: 47801 12187536 [I
4: 208624 8344960 java.lang.String
5: 45332 6192904
6: 45332 5450864
7: 3889 4615536
8: 45671 4193136 [Ljava.lang.Object;
9: 66203 3222312
10: 3889 3192264
11: 3455 2999296
12: 19754 1106224 java.nio.HeapCharBuffer
This summary contains the type of each object (e.g., “[C” would refer to arrays of characters), the number of objects of each type as well as the total number of bytes these objects take up in the heap. Jmap additionally allows you to filter the output to display only the live objects:
$ jmap -histo:live 698
num #instances #bytes class name
————————————–
1: 45256 6185464
2: 45256 5441744
3: 6899 5201176 [B
4: 3851 4577776
5: 29133 4259592 [C
6: 66069 3213448
7: 3851 3169464
8: 3417 2970112
9: 29714 1188560 java.lang.String
10: 1553 745224
11: 4161 732336 java.lang.Class
There are also jmap options to dump the heap to a file, display the contents of the permanent generation and view obejcts that are awaiting finalization. Jmap is an incredibly useful tool, and in my next blog post I will show how the DTrace hotspot provider can be used to dig into object allocation in much more detail.
Source : http://prefetch.net/blog/index.php/2007/10/27/summarizing-java-heap-utilization-with-jmap/
Posted in Uncategorized | Leave a Comment »
Perl has been called the glue that holds
the Internet together because it is an extremely powerful text
processing and CGI programming language. Although Perl was designed
in the beginning to be a text manipulation language, it has evolved
into a potent multi-purpose programming language. One area in which
Perl has shown its power is that of network programming.
Perl makes network programming easy by providing built-in
functions that can be used to create low-level client/server
programs from scratch. Also, many modules are freely available that
make programming common networking tasks simple and quick. These
tasks include pinging remote machines, TELNET and FTP sessions.
This article presents examples of each of these types of network
programs.
Client/server network programming requires a server running
on one machine to serve one or more clients running on either the
same machine or different machines. These different machines can be
located anywhere on the network.
To create a server, simply perform the following steps using
the built-in Perl function indicated:
Establishing a client is even easier:
socket.
Several other required functions and variables are defined in
the Socket.pm module. This module
is probably already installed on your machine, but if not, it is
available at the Comprehensive Perl Archive Network (CPAN), the
official Perl source code repository (see Resources). To use this
module in our programs, the following statement is required at the
top of the program:
use Socket;
This statement will locate the file Socket.pm and import all
of its exported functions and variables.
All examples in this article use modules that are available
at no cost from CPAN.
Perl modules are usually self-documenting. If the author of
the module follows the generally accepted rules of creating a Perl
module, they will add Plain Old Documentation (POD) to the module’s
.pm file. One way to view the POD for the Socket module (assuming
Perl and Socket.pm were installed correctly) is to execute the
following at the shell:
perldoc Socket
This command displays Socket.pm’s POD converted to a man
page. The output is a relatively thorough discussion of the
functions and variables defined in this module.
Another way to view the documentation is to convert the POD
to text using:
pod2text \ /usr/lib/perl5/i686-linux/5.00404/Socket.pm | more
The program pod2text is
included in the Perl distribution, as are the programs
pod2html,
pod2man,
pod2usage and
pod2latex.
Our first programming example is a simple server running on
one machine that can service only one client program at a time
connecting from the same or a different machine. Recall that the
steps for creating a server were to create a socket, bind it to a
port, listen at the port and accept client connections.
Listing 1, server1.pl, is the source code for this simple
server. First, it is generally a good idea to compile using Perl’s
strict rules:
use strict;
This requires all variables to be declared with the
my function before they are used.
Using my may be inconvenient, but it can catch many common
syntactically correct yet logically incorrect programming bugs.
The variable $port is assigned the first
command-line argument or port 7890 as the default. When choosing a
port for your server, pick one that is unused on your machine. Note
that the only way to ensure you select a port that does not have a
predefined use is to look at the appropriate RFC (see
Resources).
Next, the socket is created using the socket function. A
socket is like a file handle—it can be read from, written to or
both. The function setsockopt is
called to ensure that the port will be immediately reusable.
The sockaddr_in function
obtains a port on the server. The argument
INADDR_ANY chooses one of the server’s virtual
IP addresses. You could instead decide to bind only one of the
virtual IP addresses by replacing INADDR_ANY
with
inet_aton("192.168.1.1")
or
gethostbyname("server.onsight.com")
The bind function binds the socket to the port, i.e., plugs the
socket into that port. Then, the listen function causes the server
to begin listening at the port. The second argument to the listen
function is the maximum queue length or the maximum number of
pending client connections. The value SOMAXCONN
is the maximum queue length for the machine being used.
Once the server begins listening at the port, it can accept
client connections using the accept function. When the client is
accepted, a new socket is created named CLIENT
which can be used like a file handle. Reading from the socket reads
the client’s output and printing to the socket sends data to the
client.
To read from a file handle or socket in Perl, wrap it in
angle brackets (<FH>). To write to it,
use the print function:
print SOCKET;
The return value of the accept function is the Internet
address of the client in a packed format. The function sockaddr_in
takes that format and returns the client’s port number and the
client’s numeric Internet address in a packed format. The packed
numeric Internet address can be converted to a text string
representing the numeric IP using
inet_ntoa (numeric to ASCII). To
convert the packed numeric address to a host name, the function
gethostbyaddr is used.
Let’s assume all of the servers referred to in this article
are started on the machine named server.onsight.com. To start the
server on this machine, execute:
[james@server networking]$ server1.pl SERVER started on port 7890
The server is now listening at port 7890 on
server.onsight.com, waiting for clients to connect.
Listing 2, client1.pl, shows a simple client. The first
command-line argument to this program is the host name to which it
should connect, which defaults to server.onsight.com. The second
command-line argument is the port number which defaults to
7890.
The host name and the port number are used to generate the
port address using inet_aton
(ASCII to numeric) and sockaddr_in. A socket is then created using
socket and the client connects the socket to the port address using
connect.
The while loop then reads the data the
server sends to the client until the end-of-file is reached,
printing this input to STDOUT. Then the socket is closed.
Let’s assume all of the clients are started on the the
machine named client.avue.com, although they could be executed from
any machine on the network. To execute the client, type:
[james@client networking]$ client1.pl server.onsight.com Hello from the server: Tue Oct 27 09:48:40 1998
The following is the standard output from the server:
got a connection from: client.avue.com [192.168.1.2]
Creating sockets using the functions described above is good
when you want to control how the socket is created, the protocol to
be used, etc. But using the functions above is too hard; I prefer
the easy way—IO::Socket.
The module IO::Socket provides an easy way to create sockets
which can then be used like file handles. If you don’t have it
installed on your machine, it can be found on CPAN. To see this
module’s POD, type:
perldoc IO::Socket
Listing 3, serverIO.pl, is a simple server using IO::Socket.
A new IO::Socket::INET object is created using
the new method. Note that the
arguments to the method include the host name, port number,
protocol, queue length and an option indicating we want this port
to be immediately reusable. The new method returns a socket that is
assigned to $sock. This socket can be used like
a file handle—we can either read the client output from it, or
write to it by sending data to the client.
A client connection is accepted using the accept method. Note
the accept method returns the client socket when evaluated in
scalar context:
$new_sock = $sock->accept()
and returns the client’s socket and the client’s IP address
when evaluated in list context:
($new_sock, $client_addr) = $sock->accept()
The client address is computed and printed the same as in Listing
1, server1.pl. Then the socket associated with that client is read
until end-of-file. The data read is printed to STDOUT. This example
illustrates that the server can read from a client using < >
around the socket variable.
Listing 4, clientIO.pl, is a simple client using
IO::Socket. This time, a new object is created
that connects to a host at a port using the TCP protocol. Ten
strings are then printed to that server, then the socket is
closed.
If the server in Listing 3, serverIO.pl, is executed and then
the client Listing 4, clientIO.pl, connects, the output would
be:
[james@server networking]$ serverIO.pl
got a connection from: client.avue.com [192.168.1.2] hello, world: 1 hello, world: 2 hello, world: 3 hello, world: 4 hello, world: 5 hello, world: 6 hello, world: 7 hello, world: 8 hello, world: 9 hello, world: 10
It is possible to create servers and clients that communicate
with one another in both directions. For instance, the client may
send information to the server, then the server may send
information back to the client. Therefore, network programs can be
written so that the server and client follow some predetermined
protocol.
Listing 5, server2way.pl, shows how a simple server can be
created to read a command from a client, then print out an
appropriate response to the client. The module
Sys::Hostname provides a function
named hostname that returns the
host name of the server. To insure output is seen as we print, IO
buffering is turned off for the STDOUT file handle using the
autoflush function. Then a
while loop is executed that accepts connections.
When a client connects, the server reads a line from the client,
chopping off the newline character. Then a
switch statement is executed. (The
switch is cleverly disguised as a
foreach loop, which happens to be one of my
favorite ways of writing a switch.) Depending on
the input entered by the client, the server outputs an appropriate
response. All lines from the client are read until
end-of-file.
Listing 6, client2way.pl, shows the companion client. A
connection to the server is made, then the client prints a few
commands to the server reads the response and prints the response
to STDOUT.
The following is the output of the client code in Listing
6:
[james@client networking]$ client2way.pl server.onsight.com Hi server.onsight.com Tue Oct 27 15:36:19 1998 DEFAULT
If you want to write a client that accepts commands from
STDIN and sends them to the server, the easiest solution is to
write a client that forks a child. (A solution can be written using
select that does not fork, but it
is more complicated.) The client’s parent process will read the
commands from the user through STDIN and print them to the server.
The client’s child process will then read from the server and print
the responses to STDOUT.
Listing 7, clientfork.pl, is an example of a client that
forks.
To fork in Perl, call the cleverly named
fork function. It returns
undef if the fork fails. If it succeeds, it
returns 0 to the child, non-zero (the child’s
pid) to the parent. In clientfork.pl, an if
statement checks the value of $kid, the return
value from the fork. If $kid is true (non-zero,
the child’s pid), parent executes reading from STDIN printing to
the server. If $kid is false (zero), the child
executes reading from the server printing to STDOUT.
The following is the example session executing the client
code in Listing 7, clientfork.pl connecting to the code in Listing
5, server2way.pl:
[james@client networking]$ clientfork.pl server.onsight.com NAME server.onsight.com DATE Tue Oct 27 15:42:58 1998 HELP DEFAULT HELLO Hi
When the parent process is finished reading from STDIN, it
executes the kill function to kill
the child process. It is very important the parent reap its child
so that the child does not outlive the parent and become a zombie.
Servers usually don’t handle only one client at a time. One
approach to a server that can handle more than one client is a
server that forks a child process to handle each client connection.
Listing 8, serverfork.pl, is an example of a forking server.
One way for the parent process to reap its children is to
define a subroutine and assign a reference to that subroutine to
$SIG{CHLD}. (The hash %SIG is
Perl’s way of handling signals.) In this example, a subroutine
named REAP is defined and a
reference to this subroutine is assigned to
$SIG{CHLD}. When the parent receives the
CHLD (child terminated) signal, the REAP
subroutine will be invoked.
Within the while loop that accepts all the
client connections, the server forks. If the fork returns true, the
parent is running and it executes the next
statement which immediately transfers control to the
continue block, performs the housecleaning step
of closing the child socket and waits for the next client to
connect. If the fork returns undef, then the
fork failed, so the server dies. If the fork returns neither true
nor undef, then the child is running, so the
parent socket is closed and the child reads from the client and
processes the client. When the child is finished processing the
client, the child exits and is reaped by the parent.
Perl version 5.005 supports thread programming. This means a
threaded networking program can be created to be either a server or
a client.
Listings 9, 10, and 11 are three different versions of a
client that logs into several web servers and determines the type
of server being used (Apache, Netscape, etc).
Listing 9, getservertype1.pl, shows a non-forking,
non-threaded client. First, an array of hosts is created and
initialized to a few web sites. The subroutine
doit is defined to receive the web
server name as an argument, open a client connection to that server
at port 80 (the HTTP port), send the HTTP request and read each
line of response. When the line starting Server:
is read, it will extract the server name and store it in
$1. Then the host name and web server name are
printed. This subroutine is called for each host in the array of
hosts.
Here is the output of getservertype1.pl:
processing www.ssc.com... www.ssc.com: Stronghold/2.2 Apache/1.2.5 PHP/FI-2.0b12 processing www.linuxjournal.com... www.linuxjournal.com: Stronghold/2.2 Apache/1.2.5 PHP/FI-2.0b12 processing www.perl.com... www.perl.com: Apache/1.2.6 mod_perl/1.11 processing www.perl.org... www.perl.org: Apache/1.2.5 processing www.nytimes.com... www.nytimes.com: Netscape-Enterprise/2.01 processing www.onsight.com... www.onsight.com: Netscape-Communications/1.12 processing www.avue.com... www.avue.com: Netscape-Communications/1.12
Note that the hosts are processed in the same order as stored
in @hosts.
Listing 10, getservertype2.pl, is a forking version of
getservertype1.pl. The forking occurs within the
foreach loop. The fork is executed and if it
returns true, the parent then executes the next
statement to the next host name. If the fork returns
undef, then the program dies. Otherwise, the
child calls the doit function passing in the host, then exits.
After the parent completes its work in the while
loop, it waits for all child processes to finish, then
exits.
Here is the output of getservertype2.pl:
processing www.ssc.com... processing www.linuxjournal.com... processing www.perl.com... processing www.perl.org... processing www.nytimes.com... processing www.onsight.com... processing www.avue.com... www.onsight.com: Netscape-Communications/1.12 www.nytimes.com: Netscape-Enterprise/2.01 www.avue.com: Netscape-Communications/1.12 www.linuxjournal.com: Stronghold/2.2 Apache/1.2.5 PHP/FI-2.0b12 www.perl.com: Apache/1.2.6 mod_perl/1.11 www.ssc.com: Stronghold/2.2 Apache/1.2.5 PHP/FI-2.0b12 www.perl.org: Apache/1.2.5 Parent exiting...
Note that the hosts are not printed in the order stored in
@hosts. They are printed in the order processed,
the slower hosts taking longer than the faster ones.
Listing 11, getservertype3.pl, is a threaded version. In the
loop through the host names, a new Thread object
is created. When creating the Thread, the new
method is passed a reference to a subroutine that the thread will
execute, as well as the arguments passed into that subroutine. The
thread then executes its subroutine and when the subroutine
returns, the thread is destroyed. Here is the output of
getservertype3.pl:
processing www.ssc.com... processing www.linuxjournal.com... processing www.perl.com... processing www.perl.org... processing www.nytimes.com... processing www.onsight.com... processing www.avue.com... www.nytimes.com: Netscape-Enterprise/2.01 www.onsight.com: Netscape-Communications/1.12 www.avue.com: Netscape-Communications/1.12 www.linuxjournal.com: Stronghold/2.2 Apache/1.2.5 PHP/FI-2.0b12 www.perl.com: Apache/1.2.6 mod_perl/1.11 www.ssc.com: Stronghold/2.2 Apache/1.2.5 PHP/FI-2.0b12 www.perl.org: Apache/1.2.5
The Net::Ping module makes
pinging hosts easy. Listing 12, ping.pl, is a program similar to a
program on my server that pings my ISP to keep my connection alive.
First, a new Net::Ping object is created. The
protocol chosen is tcp (the choices are
tcp, udp and
icmp; the default is udp).
The second argument is the timeout (two seconds). Then an infinite
loop is executed, pinging the desired host. The
ping() method returns true if the host responds,
false otherwise, and an appropriate message is printed. Then the
program sleeps ten seconds and pings again.
An example output of Listing 12, ping.pl, is:
Success: Wed Nov 4 14:47:58 1998 Success: Wed Nov 4 14:48:08 1998 Success: Wed Nov 4 14:48:18 1998 Success: Wed Nov 4 14:48:28 1998 Success: Wed Nov 4 14:48:38 1998 Success: Wed Nov 4 14:48:48 1998
The Net::Telnet module makes
automating TELNET sessions easy. Listing 13, telnet.pl, is an
example of connecting to a machine, sending a few system commands
and printing the result.
First, a server and a user name are used. The user name
defaults to the user running the script by assigning to
$user the value $ENV{USER}.
(The hash %ENV contains all of the environment
variables the script inherits from the shell.)
Next, the password is requested, then read in. Note that
turning off the stty echoing is done through a
system call. It can also be done
using the Term::ReadKey
module.
Then, a Net::Telnet object is created. To
log in to the server using this object, the
login method is called. Several
system commands are executed using the
cmd method which returns the
STDOUT of the system command which is then printed. Note that part
of that output is the system prompt, which is printed along with
the output of the command.
Also note that the code
$tn->cmd(‘/usr/bin/who’) is evaluated in list
context and stored in @who, which is an array
that contains all the lines of ouptut of that command, one line of
output per array element.
After all of the system commands are executed, the TELNET
session is closed.
Here is an example output of Listing 13, telnet.pl:
Enter password:
Hostname: server.onsight.com [james@server james] Here's who: james tty1 Oct 24 21:07 james ttyp1 Oct 27 20:59 (:0.0) james ttyp2 Oct 24 21:11 (:0.0) james ttyp6 Oct 28 07:16 (:0.0) james ttyp8 Oct 28 19:02 (:0.0) [james@server james] What is your command: date Thu Oct 29 14:39:57 EST 1998 [james@server james]
The Net::FTP module makes
automating FTP sessions easy. Listing 14, ftp.pl, is an example of
connecting and getting a file.
A Net::FTP object is created, the login is
called to log in to the machine, the
cwd changes the working directory
and the get method gets the file.
Then the session is terminated with
quit.
There are methods to do many common FTP operations:
put,
binary,
rename,
delete, etc. To see a list of all
the available methods, type:
perldoc Net::FTP
Here is an example output of Listing 14, ftp.pl:
[james@k2 networking]$ ftp.pl server.onsight.com Enter your password: Before ---------------------------------------- /bin/ls: *.gz: No such file or directory ---------------------------------------- After ---------------------------------------- perl5.005_51.tar.gz ----------------------------------------
Using both Net::Telnet and Net::FTP, a very simple script can
be created that can archive a directory structure on a remote
machine.
Listing 15, taritup.pl, is a Perl program that is similar to
a program I use that logs in to my ISP and archives my web
site.
The steps this program follows are:
cd.
This program outputs text to let the user know how the script
is progressing.
Posted in Uncategorized | Leave a Comment »
Logging is an important part of any application, the only mean to know what’s happened in a production environment before that nasty error. The Java virtual machine took a while to provide a proper logging system (no, System.out is not a proper logging system) and in the meantime an Apache library, Log4J, had become the “de facto” standard for this process.
Although one could think the release of a logging tool in the JDK would start the demise of log4J, it’s not been that way. One reason is that what the JDK provides is a Java Logging API, a layer to link to a real log system (being the Sun system the default one) in a structure similar to Apache Commons Logging, which seems powerful until you realize you won’t often change the chosen log system of a project. The other reason is when you check the documentation you find it lacks some of the ease of use and capabilities that have made Log4J what it is.
Log4J is really easy to set up, at least for the most common usage: writing a rolling file that store the daily output of your application. Thing is I always find myself “Googling” for those configuration files and the pages with relevant information, so I’ve thought it would save me some time to collect here all that information. Here I would talk about two tools, Log4j and Chainsaw.
Log4J
Log4J is the logging library from Apache. Current version is 1.2, with version 1.3 no longer being developed and version 2.0 being an experimental library to use Java 5 capabilities within logging. This section will talk about log4J 1.2 and will focus in a simple scenario: logging into a daily log in the same file system.
First a bit of theory. Log4J has two main components: loggers and appenders. A logger is responsible of logging the events. There’s always a rootLogger and we can define as many loggers as we need, usually on a package basis. Loggers accept inheritance (although out of scope of this post).
An appender is a destination of the messages sent to the logger. One logger can have several appenders related to it, and all of them will receive the messages. An appender can write the messages into a file, send them through the network or show then in the console. All appenders have a related layout that tells the class how to format the message. You can find the format configuration keys here.
Log4J has 6 logging levels: TRACE, DEBUG, INFO, WARN, ERROR and FATAL. Each one has a priority so when you set up a logging level of TRACE you will see all messages while a logging level of WARN will only show WARN, ERROR and FATAL.
Onto the usage part: first of all you need to copy the Log4J jar (available here) on the lib folder (or any other folder in the Classpath) of your application. The configuration file (log4j.properties) must also be placed in the Classpath of the application.
You can use this as an example of a properties (log4j.properties) configuration file:
# Root logger uses debug level and 2 appenders named stdout and R
log4j.rootLogger=debug, stdout, R
# Print only messages of level WARN or above in the package com.foo, using root logger
log4j.logger.com.foo=WARN
# Declaring first appender
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
# Pattern to output the caller’s file name and line number.
# sample output: INFO [main] (MyApp2.java:12) – Entering application.
log4j.appender.stdout.layout.ConversionPattern=%5p [%t] (%F:%L) – %m%n
# Declaring second appender
log4j.appender.R=org.apache.log4j.RollingFileAppender
log4j.appender.R.File=example.log
log4j.appender.R.MaxFileSize=100KB
# Keep one backup file
log4j.appender.R.MaxBackupIndex=1
log4j.appender.R.layout=org.apache.log4j.PatternLayout
log4j.appender.R.layout.ConversionPattern=%p %t %c – %m%n
# Declaring a third appender, not used right now
log4j.appender.dest1=org.apache.log4j.DailyRollingFileAppender
log4j.appender.dest1.DatePattern=’.'yyyy-MM-dd
log4j.appender.dest1.Threshold=info
log4j.appender.dest1.File=info.log
log4j.appender.dest1.layout=org.apache.log4j.PatternLayout
log4j.appender.dest1.layout.ConversionPattern=%d [%t: %C.%M:%L] %p: %m%n
# Declaring a new logger for the package com.myco that uses the third appender
log4j.logger.com.myco=DEBUG, dest1
To make the logger available to your class you should declare it as shown here:
import org.apache.log4j.Logger;
public class MyClass {
final static Logger logger = Logger.getLogger(MyClass.class.getName());
MyClass() {
}
public void foo() {
logger.debug(“Hello world.”);
…
}
}
As you can see the logger variable (which I usually name “log”) is a final static variable that will be shared between all the object. Don’t worry, Log4J is thread safe. You can notice we pass the name of the class as parameter to the getLogger method. This allows us to identify the origin of the message quite easily in the log file, and ensures each log object is unique.
To add a new entry to the log file, you can use:
log.debug(“Message”);
log.debug(“Message”, new Throwable());
You may change debug by any of the other levels of logging as appropriate. Every level has two versions of the method, one that accepts only a message and one that also accepts a Throwable object, so you can pass exceptions into the log file.
A recommended way to improve the performance of the system is to check if the log level is enabled before calling the corresponding method, as you can see next:
if(log.isDebugEnabled()){
log.debug(“Message”);
}
if(log.isInfoEnabled()){
log.info(“Message”);
}
if(log.isTraceEnabled()){
log.trace(“Message”);
}
This avoid the system to build a log message that later on will be discarded by the logger due to the level not being logged, saving several cpu cycles.
You have a free small manual of Log4J 1.2 here. If you need more information on Log4J 1.2 or need to use it in a more complex scenario check this commercial book that provides a detailed explanation.
Chainsaw
Chainsaw is a companion application to Log4j written by members of the Log4j development community. It provides a GUI interface that parses Log4J logs and allows you to see the most relevant lines of it, like rrors, etc. The latest release if from 2006, but it should still work with your log files.
Posted in JBoss | Leave a Comment »
Sun Java System Web Server 7.0 Update 1 is enhanced to support regular expressions (also known as Patterns) and request time parameter interpolation in configuration files. In addition, wildcard pattern matching support is extended to server.xml. URL redirecting is implemented as a SAF. The redirect SAF lets you redirect URIs that match a certain prefix. You can specify the prefix using the from parameter and the URL to redirect to using the url or url-prefix parameters. In Sun Java System Web Server Sun Java System Web Server, the from parameter is optional. If from is omitted, all URIs are redirected.
In the obj.conf file, SAF parameters are supported with new , and tags. These tags contain directives. Using these tags, you can define conditions under which the directives are executed. These tags can also be used to dynamically generate SAF parameters.
Sun Java System Web Server 7.0 Update 1 offers URL rewrite capability that is a super set of Apache HTTP server’s mod_rewrite module. Unlike Apache’s mod_rewrite function, tag provides the following functionality:
*
It can manipulate URI, path, header fields and response bodies.
*
It works at any stage of request processing.
*
It works with any SAF, including 3rd party plug-ins.
Consider the following directive:
NameTrans fn=”redirect”
from=”/site1″
url=”http://site1.mycompany.com”
The above directive can be rewritten using regular expression as follows:
NameTrans fn=”redirect”
url=”http://site1.mycompany.com”
In the above snippet, note the usage of regular expression instead of the from parameter. If you need to redirect all requests for /site1/* to http://site1.mycompany.com/*/index.html note this technique:
NameTrans fn=”redirect”
url=”http://site1.mycompany.com/$1/index.html”
Here, the tag assigns whatever value matches (.*) to the variable $1. The $1 in the url parameter is dynamically replaced with the value from the original request. That means the above obj.conf snippet will cause a request for /site1/download to be redirected to http://site1.mycompany.com.com/download/index.html.
Combination of and redirect offers some of the flexibility of mod_rewrite. However, unlike mod_rewrite, can be used for things other than redirecting and rewriting URLs. can also be used in conjunction with any SAF, including third party plug-ins.
The above method configures a 302 Moved Temporarily redirect. In Sun Java System Web Server 7.0 Update 1, you can also add a status=”301″ parameter to indicate that you need a 301 Moved Permanently redirect instead
NameTrans fn=”redirect” from=”/path” url=”http://server.example.com” status=”301″
Posted in Uncategorized | Leave a Comment »
Two JVM options are often used to tune JVM heap size: -Xmx for maximum heap size, and -Xms for initial heap size. Here are some common mistakes I have seen when using them:
* Missing m, M, g or G at the end (they are case insensitive). For example,
java -Xmx128 BigApp
java.lang.OutOfMemoryError: Java heap space
The correct command should be: java -Xmx128m BigApp. To be precise, -Xmx128 is a valid setting for very small apps, like HelloWorld. But in real life, I guess you really mean -Xmx128m
* Extra space in JVM options, or incorrectly use =. For example,
java -Xmx 128m BigApp
Invalid maximum heap size: -Xmx
Could not create the Java virtual machine.
java -Xmx=512m HelloWorld
Invalid maximum heap size: -Xmx=512m
Could not create the Java virtual machine.
The correct command should be java -Xmx128m BigApp, with no whitespace nor =. -X options are different than -Dkey=value system properties, where = is used.
* Only setting -Xms JVM option and its value is greater than the default maximum heap size, which is 64m. The default minimum heap size seems to be 0. For example,
java -Xms128m BigApp
Error occurred during initialization of VM
Incompatible initial and maximum heap sizes specified
The correct command should be java -Xms128m -Xmx128m BigApp. It’s a good idea to set the minimum and maximum heap size to the same value. In any case, don’t let the minimum heap size exceed the maximum heap size.
* Heap size is larger than your computer’s physical memory. For example,
java -Xmx2g BigApp
Error occurred during initialization of VM
Could not reserve enough space for object heap
Could not create the Java virtual machine.
The fix is to make it lower than the physical memory: java -Xmx1g BigApp
* Incorrectly use mb as the unit, where m or M should be used instead.
java -Xms256mb -Xmx256mb BigApp
Invalid initial heap size: -Xms256mb
Could not create the Java virtual machine.
* The heap size is larger than JVM thinks you would ever need. For example,
java -Xmx256g BigApp
Invalid maximum heap size: -Xmx256g
The specified size exceeds the maximum representable size.
Could not create the Java virtual machine.
The fix is to lower it to a reasonable value: java -Xmx256m BigApp
* The value is not expressed in whole number. For example,
java -Xmx0.9g BigApp
Invalid maximum heap size: -Xmx0.9g
Could not create the Java virtual machine.
The correct command should be java -Xmx928m BigApp
NOTE:
How to set java heap size in Tomcat?
Stop Tomcat server, set environment variable CATALINA_OPTS, and then restart Tomcat. Look at the file tomcat-install/bin/catalina.sh or catalina.bat for how this variable is used. For example,
set CATALINA_OPTS=”-Xms512m -Xmx512m” (Windows)
export CATALINA_OPTS=”-Xms512m -Xmx512m” (ksh/bash)
setenv CATALINA_OPTS “-Xms512m -Xmx512m” (tcsh/csh)
In catalina.bat or catallina.sh, you may have noticed CATALINA_OPTS, JAVA_OPTS, or both can be used to specify Tomcat JVM options. What is the difference between CATALINA_OPTS and JAVA_OPTS? The name CATALINA_OPTS is specific for Tomcat servlet container, whereas JAVA_OPTS may be used by other java applications (e.g., JBoss). Since environment variables are shared by all applications, we don’t want Tomcat to inadvertently pick up the JVM options intended for other apps. I prefer to use CATALINA_OPTS.
How to set java heap size in JBoss?
Stop JBoss server, edit $JBOSS_HOME/bin/run.conf, and then restart JBoss server. You can change the line with JAVA_OPTS to something like:
JAVA_OPTS=”-server -Xms128m -Xmx128m”
How to set java heap size in Eclipse?
You have 2 options:
1. Edit eclipse-home/eclipse.ini to be something like the following and restart Eclipse.
-vmargs
-Xms64m
-Xmx256m
2. Or, you can just run eclipse command with additional options at the very end. Anything after -vmargs will be treated as JVM options and passed directly to the JVM. JVM options specified in the command line this way will always override those in eclipse.ini. For example,
eclipse -vmargs -Xms64m -Xmx256m
How to set java heap size in NetBeans?
Exit NetBeans, edit the file netbeans-install/etc/netbeans.conf. For example,
netbeans_default_options=”-J-Xms512m -J-Xmx512m -J-XX:PermSize=32m -J-XX:MaxPermSize=128m -J-Xverify:none
How to set java heap size in Apache Ant?
Set environment variable ANT_OPTS. Look at the file $ANT_HOME/bin/ant or %ANT_HOME%\bin\ant.bat, for how this variable is used by Ant runtime.
set ANT_OPTS=”-Xms512m -Xmx512m” (Windows)
export ANT_OPTS=”-Xms512m -Xmx512m” (ksh/bash)
setenv ANT_OPTS “-Xms512m -Xmx512m” (tcsh/csh)
How to set java heap size in jEdit?
jEdit is a java application, and basically you need to set minimum/maximum heap size JVM options when you run java command. jEdit by default runs with a default maximum heap size 64m. When you work on large files, you are likely to get these errors:
java.lang.OutOfMemoryError: Java heap space
at java.lang.String.concat(String.java:2001)
at org.gjt.sp.jedit.buffer.UndoManager.contentInserted(UndoManager.java:160)
at org.gjt.sp.jedit.Buffer.insert(Buffer.java:1139)
at org.gjt.sp.jedit.textarea.JEditTextArea.setSelectedText(JEditTextArea.java:2052)
at org.gjt.sp.jedit.textarea.JEditTextArea.setSelectedText(JEditTextArea.java:2028)
at org.gjt.sp.jedit.Registers.paste(Registers.java:263)
How to fix it? If you click a desktop icon, or Start menu item to start jEdit: right-click the icon or menu item, view its property, and you can see its target is something like:
C:\jdk6\bin\javaw.exe -jar “C:\jedit\jedit.jar”
You can change that line to:
C:\jdk6\bin\javaw.exe -Xmx128m -Xms128m -jar “C:\jedit\jedit.jar”
If you run a script to start jEdit: just add these JVM options to the java line inside the script file:
java -Xmx128m -Xms128m -jar jedit.jar
If you start jEdit by running java command: just add these JVM options to your java command:
java -Xmx128m -Xms128m -jar jedit.jar
Note that when you run java with -jar option, anything after -jar jar-file will be treated as application arguments. So you should always put JVM options before -jar. Otherwise, you will get error:
C:\jedit>java -jar jedit.jar -Xmx128m
Unknown option: -Xmx128m
Usage: jedit [] []
How to set java heap size in JavaEE SDK/J2EE SDK/Glassfish/Sun Java System Application Server?
Stop the application server, edit
$GLASSFISH_HOME/domains/domain1/config/domain.xml, search for XML element name java-config and jvm-options. For example,
-Xmx512m
-XX:NewRatio=2
-XX:MaxPermSize=128m
…
You can also change these settings in the web-based admin console, typically at http://localhost:4848/, or https://localhost:4848/. Go to Application Server near the top of the left panel, and then on the right panel, click JVM Settings -> JVM Options, and you will see a list of existing JVM options. You can add new ones and modify existing ones there.
Yet another option is to use its Command Line Interface (CLI) tool command, such as:
./asadmin help create-jvm-options
./asadmin help delete-jvm-options
They may be a bit hard to use manually, but are well suited for automated scripts.
Posted in JAVA | Leave a Comment »
jFM is a file manager written as a java web application. It can be downloaded from http://jfm.dev.java.net/.
jGallery uses some Jakarta Libraries.
jGallery itself is licensed under the Gnu Public License (GPL).
What you need
JDK 1.4 or later, available from http://java.sun.com
Version 5.0 or later of Jakarta Tomcat, available from http://jakarta.apache.org or any other JSP 2.0 compatible web container
if you downloaded jFM-nolibs-xxx.zip you’ll also need these libs from http://jakarta.apache.org
commons-fileupload
commons-io
jakarta-taglibs-standard-1.1.0
Installation and Configuration
For Tomcat 5:
Unzip the downloaded jFM-xxx.zip to the Tomcat webapps folder.
If you downloaded jFM-nolibs-xxx.zip, copy the commons-fileupload, commons-io and jakarta-taglibs-standard-1.1.0 to jFM/WEB-INF/lib
Finding Folders
jFM always displays one folder’s contents.
There are two ways to specify the folder.
folder address as real path (path addressing)
enter a folder address as real path in the file system
http://yourhost.com/jFM/path/a/path/
will access
$filebase/a/path/
Where filebase is a context-param in WEB-INF/web.xml, it will always be prepended to entered paths. So you can restrict path addressing to a subdirectory of the filesystem.
e.g. if filebase is c:/
http://localhost:8080/jFM/path/jakarta-tomcat-5.0.19/webapps/tomcat-docs/
will access a file system folder c:/jakarta-tomcat-5.0.19/webapps/tomcat-docs/
Files will be downloadable as application/x-binary via jFM.
folder addresses as virtual path (context adressing)
enter a folder address as a context address like
http://localhost:8080/jFM/ctx/tomcat-docs/
to access a folder containing the context /tomcat-docs (this might be the real file system folder /jakarta-tomcat-5.0.19/webapps/tomcat-docs/)
This will only work, if jFM is allowed to access other contexts. For this the jfm.xml file must be copied to $CATALINA_BASE/conf/Catalina/localhost and Tomcat must be restarted.
An alternative to this is to specify in server.xml the line
If you cannot enable crossContext=”true”, there is also the possibility of specifying a mapping between context name and real path via the dirmappings context-param in WEB-INF/web.xml
Files will be downloadable as usual via Tomcat, with exception of files in WEB-INF which will be downloadable as application/x-binary via jFM.
Security
jFM can be secured with the standard web application security.
For this, remove the comment around the entry in jFM/WEB-INF/web.xml and only users in role manager will have access.
If your webmaster insists on running Tomcat with a security manager, he should add the following entry to $CATALINA_BASE/conf/catalina.policy
grant codeBase “file:${catalina.home}/webapps/jFM/WEB-INF/-” {
permission java.io.FilePermission “d:${/}-”, “read,write,delete”;
permission java.net.SocketPermission “*:80″, “connect”;
};
FilePermission allows access to the filesystem part below the given path.
The SocketPermission is needed for the Get file from URL feature of jFM.
Posted in Tomcat | 1 Comment »
