Start blogging

Hi there! This is my latest blog, where I am going to "log" things that interest me the most and may be useful not only for myself :)

Eclipse with CUDA

Working with CUDA code in Eclipse is easy. There are some steps we need to setup in our IDE to start developing applications in pure C/C++ with CUDA support.

Wednesday 26 September 2012

JavaFX 2.1 examples on Linux x64

JavaFX 2.1 Developer Preview for Linux has been released recently, unfortunately only for 32bit architecture. But what about x64? Well, I have Ubuntu x64 10.04 LTS and it also works :D! Here are the tips to run BrickBreaker example, which comes with JavaFX SDK examples.

To download SDK preview we should look at proper download section here. Also we need to download latest JavaSE JDK 7 but for Linux x86 (despite the fact we are using 64-bit Linux). After downloading all necessary packages, just unpack them wherever you want.

Right now JavaFX SDK examples are distributed in separate package. It is available at the bottom of this page (notice that we need examples for Linux!). Inside package we may find src directory with BrickBreaker sources, a simple game which we will compile and run.

The easiest choice of IDE for latest JavaFX release is Netbeans 7. To be able to run and compile FX projects we need to setup Java platforms. Just select "Tools->Java Platforms" from the menu. Click "Add platform..." and select directory when we have extracted JavaSE JDK 7, then select "Finish". We have added new JVM, but we need to enable FX. Select new added Java platform in "Java Platform Manager" window and navigate to "JavaFX" tab. Enable JavaFX and point to JavaFX SDK directory by clicking "Browse..." next to "JavaFX SDK:" input.

Now we can open BrickBreaker project in NetBeans and open its properties window. Navigating to "Libraries" section in "Project Properties" window we should select Java platform with enabled JavaFX libraries. Then we can compile and run project right from IDE.

JSF 1.2 download file link with managed bean


Recently I had to use in one of my JSF projects a download link for binary file from server. I didn't want to use my own servlet class, but instead just managed bean. I have done it several times before, but I always forget the proper code :) So here it is, maybe someone will find it useful ;)

public class DownloadManagedBean {

  /**
   * This is the action listener method
   */
  public void downloadAction(ActionEvent event) {
     // Get data from any source you want :)
     byte [] data = ...; 
     downloadAction("out.bin", data);  
  }

  /**
   * This is the core of download action
   */
  protected void downloadAction(String filename, byte[] data) {
        FacesContext faces = FacesContext.getCurrentInstance();
        HttpServletResponse response = (HttpServletResponse) faces.getExternalContext().getResponse();
        response.setContentLength(data.length);
        response.addHeader("Content-Type", "application/x-force-download");
        response.addHeader("Content-Disposition", "attachment; filename=\"" + filename + "\"");
        ServletOutputStream out = null;
        try {
            long length = data.length;
            out = response.getOutputStream();
            response.setContentLength((int) length);
            out.write(data, 0, (int) length);
        } catch (Exception ex) {
            System.err.println(ex);
        } finally {
            if (out != null) {
                try {
                    out.close();
                } catch (Exception ex) {
                    System.err.println(ex);
                }
            }
        }
        // The most important line - finish response as we have already
        // close output stream response writer
        faces.responseComplete();
    }
}

To use above bean, we just put the following code inside JSF page source:

<h:form>
  <h:commandlink actionlistener="#{downloadManagedBean.downloadAction}" immediate="true" target="_blank" value="Download">
  </h:commandlink>
</h:form>

We could change downloadAction method to be more flexible, by using buffered streams and files as data sources, that may be located anywhere in the server's local filesystem. Above example uses for simplicity byte array only to show the main idea how to make it work.

Automatically reload log4j configuration

I have been using Log4j for a while and IMO it is one of the greatest libraries to trace your application behaviour. Yet, I used to have this recurring problem, when I wanted to change logger configuration, my application was running and I couldn't restart it to reload loggers. But then one day I have found a solution, which turned out to be very simple and provided out of the box with the library.

Here is an example of a simple class, which can be used at the very beginning of any Java application to setup Log4j configuration. Configuration may be located wherever we want and it doesn't need to be named as default "log4j.properties".

public abstract class LoggerSetup {

    private static Logger logger;
    // Period in [ms] between checking for logger properties change
    private static final long CHECK_PERIOD = 20000;

    /**
     * Set logger file properties
     * @param filename - path to log4j properties file
     */
    public static void setLogger(String filename) {
        File f = new File(filename);
        if (f.exists()) {
            PropertyConfigurator.configureAndWatch(filename, 20000);
        }
        logger = Logger.getLogger(LoggerSetup.class);
        logger.debug("Setup logger");
    }

}

As we can see, every 20 seconds, Log4j check whether properties file has changed. If so, it will reload logger setup. It means that during application execution you may change logger levels, appenders, formats etc. without the need
for application restart.

Bash Shell in Java: error=24, Too many open files


Bash Shell in Java: “Cannot run program ”/bin/bash“: java.io.IOException: error=24, Too many open files”.
If you happen to see above error in your logs, there is a simple solution to fix this. It is only a matter of few lines in your code.

Recently i have faced a problem as in the title. I was using JAVA's ProcessBuilder class, which allows to use bash shell and its scripts. Unfortunately my program after a while started to log that it "Cannot run program".

If you encounter similar problem, googling a while will give you plenty of pages with some more or less accurate solutions. Most of them will tell to check how many files are currently open in the system (by unix command lsof) and how's that number related to your system limit (check by bash command ulimit -n). Increasing maximum number of open files at a time is short-term solution in my opinion.

Let's get back to the core of problem. In my code I was using below lines:

ProcessBuilder pb = new ProcessBuilder("bash", "-c", command);
pb.redirectErrorStream(true); // use this to capture messages sent to stderr
Process shell = pb.start();
InputStream shellIn = shell.getInputStream(); // this captures the output from the command
int shellExitStatus = shell.waitFor(); // wait for the shell to finish and get the return code
InputStreamReader reader = new InputStreamReader(shellIn);
BufferedReader buf = new BufferedReader(reader);
// Read lines using buf

After using new ProcessBuilder and reader of input stream we must assure that InputStreamReader will be closed after reading as it should release any associated files. So I have added following lines:

if(reader != null) {
  try {
    reader.close();
  } catch (IOException ex) {
    log.error(ex, ex);
  }
}
if(shellIn != null) {
  try {
    shellIn.close();
  } catch (IOException ex) {
    log.error(ex, ex);
  }
}

Above code should be used in finally block. This solution works but not always, because it will not close all pipes and subprocesses triggered by shell command. For example when I have used

ps aux | grep java | awk '{if($1=="root") print $11}' | grep test | wc -l

I was creating multiple unnamed pipes, which stayed open and were increasing the total number of open files. To cleanup after executing shell command all subprocesses have to be stopped:

if(shell != null) {
  try {
    shell.destroy();
  } catch (Exception ex) {
    log.error(ex, ex);
  }
}

Destroying all subprocesses will do the thing. It will not impact on the result of shell command as in the code we have following line

int shellExitStatus = shell.waitFor();

To summarize, if we want to avoid this problem we should use something similar to below function:

  public String bashCommand(String command) {
        ProcessBuilder pb = new ProcessBuilder("bash", "-c", command);
        pb.redirectErrorStream(true); // use this to capture messages sent to stderr
        Process shell = null;
        String result = "";
        InputStream shellIn = null;
        InputStreamReader reader = null;
        try {
            shell = pb.start();
            shellIn = shell.getInputStream(); // this captures the output from the command
            int shellExitStatus = shell.waitFor();
            reader = new InputStreamReader(shellIn);
            BufferedReader buf = new BufferedReader(reader);
            String line;
            while ((line = buf.readLine()) != null) {
                result += line + "\n";
            }
        } catch (InterruptedException ig) {
            // Handle error
        } catch (IOException ignoreMe) {
            // Handle error
        } finally {
            if(reader != null) {
                try {
                    reader.close();
                } catch (IOException ex) {
                    // Handle error
                }
            }
            if(shellIn != null) {
                try {
                    shellIn.close();
                } catch (IOException ex) {
                    // Handle error
                }
            }
            if(shell != null) {
                try {
                    shell.destroy();
                } catch (Exception ex) {
                    // Handle error
                }
            }
        }
        return result;
    }

That's all. Hope it will help :)

Eclipse with CUDA

Writing small programs in CUDA could be done simply in any text editor. But as I was working on slightly bigger application, I tried to find good IDE, which allows to develop applications with CUDA support. Unfortunately I was working on Ubuntu so I couldn't just use Parallel Nsight, which comes as a plugin for Visual Studio only. I have tried different IDEs, but Eclipse was the most flexible. So here are the hints which may be useful when we want to use CUDA in Eclipse.

First of all we should install CUDA SDK and download Eclipse with C/C++ support. After downloading and setting all up properly we should create just empty C/C++ project in Eclipse or just use existing one. I always try to separate CUDA source files from ordinary C/C++ sources in project.

The next steps will set necessary global properties for our project. We should create some build variables (just select "Project"->"Properties") and make new variables for selected build configuration as presented below:


Four variables should be set:
  1. CUDA_INCLUDE - path for CUDA header files, which are installed with CUDA SDK
  2. CUDA_LIB - path for libraries installed with CUDA SDK. Notice that you should point to directory with libraries compiled for our system (in my case 64bit architecture)
  3. NVCC - CUDA compiler which will be used to compile our CUDA sources
  4. NVCC_ARCH - this one is for additional compiler flags. We don't have to set this one. I have used it for setting target compute architecture (double precision)
Next we should use above variables for setting default compiler and linker options. Following screens should guide you through the basics.




Next we should tell Eclipse, that our CUDA source should be considered as C source files.



Now everything is ready to use. When we create new file with *.cu extension, in which we put all CUDA specific code, we need to compile this sources with nvcc compiler. In order to do that, we should select "Properties" for that specific file and set custom build step.


The last figure shows how to set custom build step for integrate_2d_cuda.cu example file. This build step creates integrate_2d_cuda.o file, by using nvcc compiler, which then will be used by gcc/g++ linker to build application.

There is one thing we should do to prevent Eclipse from complaining about unknown keywords in CUDA source files. At the very beginning of each *.cu we just add following lines:

//Only for eclipse parsers
#ifdef __CDT_PARSER__
#define __global__
#define __device__
#define __shared__
#define __const__
#endif

That is all. Now we can develop application as ordinary C/C++ project. We should only remember that for each CUDA source file we have to set custom build step. Happy coding!