Saturday, March 21, 2009

Fix bugs with core dump

A perplexity developers usually meet is they release the product to qa team and get a feedback of occasional crash. And the testers don't have a solid reproduction steps. In this case, it's a time-consuming task to find out the cause and fix it. What we need is an efficient postmortem debugging method.

Core dump is a mechanism provided by operating system to automatically capture the address apace of a crashed process into a dump file that can be used to help us debugging.

How to enable it
In most linux distributions, core dump is disabled by default. This can be validated by running "ulimit -a" command from shell. We are very likely to see the results below:
core file size (blocks, -c) 0

It indicates core dump is diabled. To enble it, the simpliest way is invoking "ulimit -c unlimited" command. Ulimit is a bash builtin command, and it only takes effect for current shell and all porcesses spawned in this shell.
If we want to enable it permanently for a user, we can add the command to the ~/.bashrc file.
If we want to enable it globally, we can edit the /etc/security/limits.conf file by setting
* soft core unlimited

Customize core dump name
By default, the core dump file will be created in the same directory that the application is running in with fixed file name "core". The name conforms to the definition in /proc/sys/kernal/core_pattern file.
This pattern can be changed via "sysctl -w kernel.core_pattern=DesiredValue" command. Again, this setting is't persisted. To make it permanent, we can edit the /etc/sysctl.conf file and add the line "kernel.core_pattern=DesiredValue" to it.
The list below is some place holder can be used in the naming pattern to make it more flexible.

%% A single % character
%p PID of dumped process
%u real UID of dumped process
%g real GID of dumped process
%s number of signal causing dump
%t time of dump (seconds since 0:00h, 1 Jan 1970)
%h hostname (same as ’nodename’ returned by uname(2))
%e executable filename

And here is an exmple.

EXE = test
CC = g++
#CFLAGS += -Os
# add debug information
CFLAGS += -g
CFLAGS += -Wall

$(CC) $(CFLAGS) test.cpp -o $(EXE)

.PHONY: clean

rm $(EXE) -rf

// source code

using namespace std;

void foo()
char *buf = "aa";
cout << "before exception" <<>
buf[0] = 'b';// invalid code
cout << "after exception" <<>

int main()
return 0;

Every time we run the application, it will crash and generate a core dump file. We can analysis the dump file with gdb (gdb test core_dump_file).

Capture live core dump
It's also desirable to capture core dump of a process when it's still running. Such dump is useful for trouble shooting contention and dead lock issues. gcore is the right tool for this purpose.


Sunday, March 15, 2009

Extensive usage of Make

Many guys have used make utility with makefile to compile source code. The official introduction of gnu make also introduces it as "Make is a tool which controls the generation of executables and other non-source files of a program from the program's source files".
If we look inside how does make utility works, we'll find it can do much more than compiling code. It's so powerful to make our life much easier, and by our, i mean ordinary people, not just programmers.
The essential point is when combined with shell script, make can assist you doing a sequence of actions to perform automatically.

Typical scenario
Suppose we're writing a book "How to win lottery" which will surely be the best sell all around the world on after it's available. Because everyone in different countries would like to have a copy of it, we also need to translate it into different languages.
And a generous, smart programmer provides us a super translation tool that is capable of translating all languages, at no cost.
The last thing is we need to share new chapters to our kind editor by placing documents at //ipaddress/book/(Sorry, I can't share the address with you since it's confidential). She will have some guys to proof reading them.

So, here is our typical working flow:
1. Write / update english version draft
2. Run the translation tool to generate a draft for a different language
3. Save the file according to its language
4. Upload the file to //ipaddress/book
5. Send a email to notify editor

A little bit boring, right ? We need to repeat this again and again when we have new chapter available or the editor asks us to correct errors. Can't we just focus on the writing the book itself ? We have two options, hire a guy to do steps two to five for us, or use make utility.

How to do?
We can define a makefile according to the steps above.

all: generateDraft translate upload sendMail

draftGenerator.exe -o draft.pdf #this line generate a draft pdf file -o is the argument passed to draftGenerator

superTranslator.exe --lan $(lan) -o draft_$(lan).pdf #this line translate the draft to specified language and save it as a copy with language in filename

cp draft_$(lan).pdf //ipaddress/book

#send mail to editor

Then, each time after we update the draft, we can simply use "make lan=chinese"command to ask the compter do the rest for us.

The make utility is actually a parser that read the makefile and perform every actions defined there. So what we need to do is carefully design our makefile.

Well, to sum up, make utility if useful when we have:
  1. A sequence of steps to perform
  2. Frequent update
make will keep our working process consistent without forgetting to do several steps.

For example, we can define a make file to run all unit tests before comit new code to code respository.

Gnu make manual
Compile Apps Your Way With Custom Tasks For The Microsoft Build Engine

Tuesday, March 10, 2009

Enable SSL in goAhead web server

goAhead web server comes with internal ssl support. It's disabled by default, and I haven't seen a tutorial around this topic on the web. So here is the my adventure of enabling it.

Difficulties with goAhead
According to the goAhead's feature page, ssl is fully supported. But it's not so convienent to enable it. The downloaded source package doesn't include the source code of the ssl library that it depends it. Even the distribution package (header file and library file) isn't there.
If we trun the macro WEBS_SSL_SUPPORT definition on, what we get eventually is compilcation error. You may see the error of type SSL isn't defined.
Then we can see there is a mocana project file, so we guess go ahead is using mocana library. And now problem comes, mocana is a comerical product and it's not freely available.

Use openssl instead
Luckily, we found there was a macro named OPENSSL. It's a symbol of openssl can be an alternative option here.
Looking deeply inside the code, we can see that goAhead provides a abstract layer above the underlying ssl library. So that we can change the implementaion easily. See? It's a typical usage of adapter design pattern.
We downloaded the openssl source, compile a linux version binary set. The openssl is a powerful ssl tool set. In addition to the ssl library itself, it also contains several utility tools. Within those tools, openssl is a useful terminal tool. It can act as a ssl server, ssl client, and certificate file generator.
What's specifically useful is we can generate and sign certificate file with it and use them to test our server. This page tells how to generate and sign certificates.
In goAhead's code, it uses three certificate files.
privkey.pem : the private key
cacert.pem : the certificate
server.pem : A combination of private key and cacert. The first part of its content is privkey.pem and the second part is cacert.pem.

Having got those files, we changed the makefile to define WEBS_SSL_SUPPORT and OPENSSL macro to enable ssl. Compiled again and run.

How to visit
The goAhead web server listens on a differnt port for incoming secure connection. The port is defined as SSL_PORT macro n in websSSL.c with default value 433. To test, open the browser and visit https://address:SSL_PORT .
Since we used a self signed certificates, the firefox won't allow access to it. We have to add our site to firefox exception list from "tools - advanced - encryption - view certificates - add exception" .