Wednesday, June 27, 2012

How to create a web app using Java, MySQL, JDBC, Servlets, and Tomcat

If you ask developers what technologies are used to build modern web applications today, you would probably get something along the lines of PlaySpring and Hibernate.  These tools are great because they hide all the lower level implementation details such as servlets and the actual SQL itself, but as a developer, it is beneficial to understand what goes on beneath these layers of abstraction.  To fulfill my own curiosity, I created a simple web application using Java, MySQL, JDBC, Servlets, and Tomcat.  The rest of this post describes what I did.



Step 1: Download MySQL
The first step is to download MySQL and get it up and running.  Follow the installation instructions, start up the server, and make sure you can log in using "mysql -u root -p" in the terminal.   This link provides a good resource for a refresher on SQL.

Step 2: Download Connector/J
Connector/J can be downloaded here.  All we really need is mysql-connector-java-5.1.20-bin.jar in the src/lib folder.   This jar file is the JDBC driver that lets us communicate with MySQL using Java code.

Step 3: Establish a connection to MySQL using JDBC
Create a new Java project and add mysql-connector-java-5.1.20-bin.jar to the build path.  This link provides a good tutorial on JDBC.

Step 4: Download Tomcat
Tomcat 7 can be downloaded here.  Unpack and follow the instructions in RUNNING.txt.  Once Tomcat is running, you should be able to see the default web page at http://localhost:8080.   Add mysql-connector-java-5.1.20-bin.jar to apache-tomcat-7/lib.

Step 5: Understand the directory structure of a Java web application
Java web applications should have the following directory structure

Directory structure of a Java web app

The root folder is the name of your Java project. Under the root folder, there should be another folder called WEB-INF.  Under the WEB-INF folder, there should be a classes folder, a lib folder, and a web.xml file.  Java source files and compiled .class files belong in the classes folder.  Libraries belong in the lib folder.  The web.xml file is used by Tomcat to map a URL to a servlet.

Step 6: Write a simple Java application using JDBC and servlets
Create a new Java project using the directory structure above.  If you don't know where to begin with JDBC and servlets, make sure Tomcat is running and go to http://localhost:8080/examples/servlets.  Look at the code samples and execute them.  Don't forget to add apache-tomcat-7/lib/servlet-api.jar to your project build path.

I created a to-do list web application.  It queries a JDBC/MySQL backed database for a list of items, displays it in a table, and lets you add or delete items.  The full source code is available at my Github repository here.

Screenshot of my web app
Step 7: Configure web.xml
Configure web.xml to map the URL to your servlet class.  Below is my sample web.xml file.


When a client visits http://localhost:8080/simple-servlet/example, the request is handled by the application server.   Tomcat scans for the servlet-mapping containing the "/example" URL pattern, looks up the servlet-name which is "todo", matches the servlet-name to the right servlet , and routes the HTTP request to TodoServlet.class.  Then TodoServlet.class executes its logic and returns a response back to the client through Tomcat.

Step 8: Package web application into WAR file and deploy on Tomcat
First, compile all source files.  In a terminal/console, navigate to your project's root directory.  Then type in "jar cvf simple-servlet.war .".  This creates a WAR (Web application ARchive) which is simply a jar file with a .war extension.  Drop this war file into your apache-tomcat-7/webapps directory and test by going to http://localhost:8080/simple-servlet/example.


That's it!  Your database-backed web application is up and ready to go.

Note: Creating HTML in servlets is actually not a good practice.  Ideally, you should use some type of view technology like JavaServer Pages or JavaServer Faces to move data between the view and logic layers.  Also, the database login username and password should be passed in to the program as parameters instead of being hardcoded.   For the purpose of keeping it quick and simple, I did not follow these guidelines in my project.

Tuesday, March 20, 2012

A ToDoList for the Masses

I have started work on a new social media web app for sharing one's to-do list with friends and coworkers.
Initial proof of concept design.
My motivations for this project are numerous:
  1. To fulfill a personal need for a web based to-do list.
  2. To create something cool and learn some new skills along the way.
  3. To build a free, open-source platform that provides a more efficient way for people to work together.

Here are some use cases:
  1. People who need a generic to-do list for everyday tasks like feeding the dog, buying milk, paying the bills, etc.
  2. Students who want to sync up all class assignments and eliminate confusion on due dates.  Teachers/professors will be able to form groups and electronically push assignments to their students.
  3. People who need to work collaboratively in team projects.  Developers already have tools like issue trackers, but what about the non-developer masses?  Every single organization can benefit from project management tools.
  4. Friends who want to know what their friends are up to.

I found a great site that offers free icons here.

My project is hosted on GitHub here.

Stay tuned for more updates!

Thursday, March 15, 2012

How to Set Up Play Framework 2.0 Javadocs for Eclipse

Play framework 2.0 was released a few days ago and many developers are wondering how to set up the Javadocs for Eclipse.  The documentation is quite lacking so I hope this post will show up on Google and save some people a few hours.

The Problem:
No Javadocs!
The Solution:
  • In Eclipse, select your project in the Project Explorer
  • In the menu bar, select Project > Properties > Java Build Path > Libraries
  • Scroll down and look for play_2.9.1.jar
  • Click the arrow on play_2.9.1.jar and set the Source attachment and Javadoc location
  • Edit the Source attachment, click External Folder, and point to play-2.0/framework/src/play/src
  • Edit the Javadoc location, click Browse under Javadoc URL, and point to play-2.0/documentation/api/java

The Result:
Yay Javadocs!
Some other helpful tips:

Be sure to install the Scala plugin for Eclipse here.  Play 2.0 now uses Scala so the plugin will fix any pesky problems in Eclipse where it can't resolve some classes and methods provided by the framework.

If you're using Git to host your Play projects, check out the sample .gitignore file here.  Just stick it into your Play project folder.  It automatically excludes all Eclipse configuration files and makes it easy to work across multiple environments and developers.

Saturday, March 3, 2012

Tech News Roundup

Here are some great sites for the tech-minded reader.  They cover the entire gamut from software, hardware, startups, and science in general.  I have these all bookmarked myself.

Reddit - http://www.reddit.com/r/programming/

HackerNews - http://news.ycombinator.com/

TechCrunch - http://techcrunch.com/

DZone - http://www.dzone.com/links/index.html

DailyTech - http://www.dailytech.com/

TechReport - http://techreport.com/

CNNMoney - http://money.cnn.com/technology/

CNet News - http://news.cnet.com/

Sunday, January 15, 2012

Parallel Image Processing

The emergence of AJAX, Facebook, Youtube, iPhone, iPad, and cloud computing has created the perfect storm for the greatest societal changes in history-- social media.  We have become crazed consumers of content on the web.  This blog post will explain the basics behind a digital image.

Videos are made up of frames.  Think of the term 'frames per second'.  A frame is just a still image.  When you flash a bunch of frames at a rate of thirty per second, the human eye perceives it as video.  Each frame can be edited individually to achieve various visual effects such as inversion or smearing.

This frame uses smear + black-and-white image filters.

Let's break it down a bit further.  The first question is-- what is an image?  An image is made up of a bunch of pixels.  For example, a 100x100 resolution image has 100 rows and 100 columns for a total of 10,000 pixels.

Breaking it down even further, the second question is-- what is a pixel?  A pixel is a dot of color.  A pixel is a mixture of three different colors in the RGB model-- red, green, and blue.  Most images use 24-bit pixels with red, green, and blue taking 8 bits each.  This allows for a depth range between 0 and 255 (8 bits unsigned can go up to 255).  A depth range of 256 means that the color can have 256 different shades of intensity.  Since the final pixel color is the result of mixing red, green, and blue together, the total number of possible pixel colors is 256*256*256 = 16,777,216.  Many R,G,B charts like this can be found online.

Let's recap.  A video is made from images.  An image is made from pixels.  A pixel is made from three R,G,B values.  Now that we've broken down the process completely, it is easy to understand that image processing boils down to the manipulation of individual pixels via their R,G,B values.  For example, we can transform a black pixel into a white pixel by changing its R,G,B value from (0,0,0) to (255,255,255).

We can apply a visual effect to an entire image by using something called an image filter.  It is a fancy term for describing an algorithm that manipulates a set of pixels to achieve a certain effect.  To show what goes on behind the scenes in actual code, I wrote two Java programs that take an image file as input, applies an invert/oil filter, then outputs the new image file.  The full source code is available at my GitHub project site.

Original
Inverted
Oiled













The algorithm for inverting an image is

  • For each pixel (x,y) in the original image with (R,G,B) values:
  • Subtract 255 from the value of R
  • Do the same for the G and B values
  • For example, the inversion of a white pixel (255,255,255) is a black pixel (0,0,0)

The complete source code is available here in ConcurrentInvert.java

The algorithm for oiling an image is
  • For each pixel (x,y) in the original image with (R,G,B) values:
  • Replace the R value by the most common R value among all pixels with x-coordinates between (x - radius) and (x + radius) and with y-coordinates between (y - radius) and (y + radius)
  • Do the same for the G and B values
  • For example, if radius = 1, one must find the most common R,G,B values among 9 pixels (in a 3x3 grid).  If radius = 2, one must find the most common R,G,B values values among 25 pixels (in a 5x5 grid)
The complete source code is available here in ConcurrentOil.java

Inverting an image has a time complexity of O(n) so the program runs pretty quickly.  However, oiling an image has a time complexity of O((2n+1)*(2n+1)*n) = O(4n^3 + 4n^2 + n) so it runs slower by several orders of magnitude.

Luckily, we can achieve a big speedup by using parallelization, also known as multi-threading.  The concept is simple.  We create additional threads and split the pixels as evenly as possible across all cores.

Workload on a single thread 
Workload on two threads







The caveat is that parallelization only works when the order of execution does not matter, or when the result of a computation does not depend on the result of another computation.  For example, the following piece of code can be parallelized


The order in which this for loop executes does not matter.  We could parallelize this computation across 2 threads by having the first thread compute iterations i=0 to i=4, and the second thread compute iterations i=5 to i=9.

whereas the next piece of code cannot be parallelized

The order in which this for loop executes does matter.  We must compute iteration i=2 before i=3, i=3 before i=4, and so on.

Test results show that the algorithm for inverting and oiling an image is perfectly parallelizable and scalable to an infinite number of cores.

Inverting the cliff image above is twice as fast on 2 cores vs 1 core

Oiling the cliff image above is twice as fast on 2 cores vs 1 core

The concepts presented here form the basic backbone of all digital image and video processing in programs like Photoshop, Final Cut Pro, and even the camera in your smartphone.  In this age where image and video sharing is an epidemic on social media sites like Facebook and Youtube, I hope that readers have gained a better appreciation for how images and videos work.