Defend against SQL Injection using ActiveJDBC

I was asked on forums how ActiveJDBC defends against SQL Injection attacks. My first reaction was: hey, this is not an ORM problem, talk to web devs :)
Luckily, Lukas Eder wrote a nice blog that helped put things into perspective. While SQL Injection attacks are not the task of ORMs per se, ORMs still need to provide some level of advice on how to deal SQL injection pitfalls.
Lukas even offered some actual code snippets on his article. What is a developer to do to try things out?  HE/she writes tests, and so I proceeded to write a test using some of the examples of his code.
The first test had this code:

String id = "1';drop table users;select * from people where name = 'John";
List<User> users = User.where("id = '" + id + "'");
users.size();
User user1 = new User();
user1.set("first_name", "John", "last_name", "Doe", 
           "email", "john@doe.com").saveIt();
User.findAll().dump();

At first when I tried this against MySQL, I got "invalid statement" exception, apparently it could not handle the semicolon in the middle of the statement. After that, I ran this same code against H2 database, and sure enough, the SQL injection worked. This resulted in an exception:


Caused by: org.h2.jdbc.JdbcSQLException: Table "USERS" not found; SQL statement:
INSERT INTO users (email, first_name, last_name) VALUES (?, ?, ?) [42102-171]

Holy crap! My table USERS is gone!

This is bad news for those people who use a simple string concatenation to make SQL statements in public facing web projects.

However, I quickly wrote a second test that uses dynamic parameters passed to a model:

String id = "1';drop table users;select * from people where name = 'John";
List<User> users = User.where("id = ?", id);
users.size();
User user1 = new User();
user1.set("first_name", "John", "last_name", "Doe", 
        "email", "john@doe.com").saveIt();
User.findAll().dump();

As you can see here, the only difference is that I pass the "name" parameter as a dynamic one, instead of simply concatenating strings

The output from this test is:

Model: org.javalite.activejdbc.test_models.User, table: 'users', attributes: {EMAIL=john@doe.com, FIRST_NAME=John, ID=1, LAST_NAME=Doe}

As you can see, the table USERS is intact, no harm is done.

So, what is the take out from here? String concatenation to build dynamic queries in web applications is  evil!

But, since ActiveJDBC uses PreparedStatement, you are safe as long as you use dynamic parameters instead of splicing strings together. 

I actually never use string concatenation, not because I constantly worry about SQL Injection attacks, but simply because this makes for some ugly spaghetti code and this probably one of the reasons I did not pay much attention to it before.

Cheers to safe coding!

Just how thin can a framework be? ActiveJDBC vs Hibernate.

When people talk about thickness of a framework, and it being lightweight, what do they actually mean?

I think there needs to be a few parameters to be looked at, for instance:

• Level of intrusiveness into your code (subjective)
• Speed of execution (objective)
• Code elegance (subjective)
• Physical weight - total size of dependencies for "Hello world" (objective)
• Stack depth (objective) - depth of a stack in case of exception coming from a lower level technology

Lets consider the level of intrusiveness with Hibernate and ActiveJDBC. I personally of course prefer ActiveJDBC because it has almost no annotations, no configuration files, but most of all it has no third party object such as Hibernate Session that actually operates on entites.

Here is a code in Hibernate entity:

@Table(name = "employees")
public class Employee implements Serializable {
    public Employee() {}
    @Id
    @Column(name = "id")
    @GeneratedValue
    Integer id;
    @Column(name = "first_name")
    String firstName;
    public Integer getId() {
        return id;
    }
    public void setId(Integer id) {
        this.id = id;
    }
    public String getFirstName() {
        return firstName;
    }
    public void setFirstName(String firstName) {
        this.firstName = firstName;
    }
    @Override
    public String toString() {
        return "Employee{" +
                "id=" + id +
                ", first_name='" + firstName + '\'' +
                '}';
    }
}



Usage of entity:

SessionFactory sessionFactory = new Configuration().configure().buildSessionFactory();

Session session = sessionFactory.openSession();

List employees = session.createQuery("select e from Employee as e").list();
session.close();

And here is comparable code in ActiveJDBC:

Model:
public class Employee extends Model{}

That is right! One line of code, and nothing else.

Usage of model:
Base.open("com.mysql.jdbc.Driver", "jdbc:mysql://localhost/test_db", "user1", "*****");
List employees = Employee.findAll();
Base.close();

As you can see, ActiveJDBC uses a different paradigm: models (entities in AJ talk) operate
on self, without requiring a third party class.

I think that by code intrusiveness, ActiveJDBC is certainly thinner than Hibernate.

Speed of execution: I built a simple non-scientific test in both ActiveJDBC and Hibernate.
The code inserts 50 thousand records into MySQL table, then reads all 50K records from it.
This is performed on my laptop, which is: MBP 2009, Core 2 Duo CPU P8700 @ 2.53GHz,
with 8G RAM, 256G SSD and Ubuntu running natively.
Hibernate insert: 16057 milliseconds
ActiveJDBC insert: 9630 milliseconds

Hibernate select: 50000 records in: 1874 milliseconds
ActiveJDBC select: 50000 records in: 836 milliseconds

As you can see, ActiveJDBC is significantly faster given exactly the same task:
Insert: ActiveJDBC took 40% less time.
Select:ActiveJDBC took 54% less time.

Code elegance: while this is super subjective especially considering the fact that I'm the author of ActiveJDBC API, I hope most people will agree with me simply by looking at code above.

Physical weight: This is a total size of all dependencies a simple application requires.
Here ActiveJDBC wins hands down.
ActiveJDBC dependencies:

-rw-r--r-- 1 igor igor 137026 2012-04-05 12:03 activejdbc-1.2-SNAPSHOT.jar
-rw-r--r-- 1 igor igor 33795 2012-04-05 12:03 javalite-common-1.2-SNAPSHOT.jar
-rw-r--r-- 1 igor igor 495944 2012-04-05 12:03 mysql-connector-java-5.0.4.jar
-rw-r--r-- 1 igor igor 23659 2012-04-05 12:03 slf4j-api-1.5.10.jar
-rw-r--r-- 1 igor igor 7599 2012-04-05 12:03 slf4j-simple-1.5.10.jar

Total size: 704K

Hibernate dependencies:
-rw-r--r-- 1 igor igor 443432 2012-04-05 12:13 antlr-2.7.6.jar
-rw-r--r-- 1 igor igor 26361 2012-04-05 12:13 asm-1.5.3.jar
-rw-r--r-- 1 igor igor 16757 2012-04-05 12:13 asm-attrs-1.5.3.jar
-rw-r--r-- 1 igor igor 282338 2012-04-05 12:13 cglib-2.1_3.jar
-rw-r--r-- 1 igor igor 175426 2012-04-05 12:13 commons-collections-2.1.1.jar
-rw-r--r-- 1 igor igor 38015 2012-04-05 12:13 commons-logging-1.0.4.jar
-rw-r--r-- 1 igor igor 313898 2012-04-05 12:13 dom4j-1.6.1.jar
-rw-r--r-- 1 igor igor 208048 2012-04-05 12:13 ehcache-1.2.3.jar
-rw-r--r-- 1 igor igor 2321639 2012-04-05 12:13 hibernate-3.2.7.ga.jar
-rw-r--r-- 1 igor igor 365546 2012-04-05 12:13 hibernate-annotations-3.5.6-Final.jar
-rw-r--r-- 1 igor igor 66426 2012-04-05 12:13 hibernate-commons-annotations-3.0.0.ga.jar
-rw-r--r-- 1 igor igor 2566731 2012-04-05 12:13 hibernate-core-3.5.6-Final.jar
-rw-r--r-- 1 igor igor 100884 2012-04-05 12:13 hibernate-jpa-2.0-api-1.0.0.Final.jar
-rw-r--r-- 1 igor igor 8812 2012-04-05 12:13 jta-1.0.1B.jar
-rw-r--r-- 1 igor igor 495944 2012-04-05 12:13 mysql-connector-java-5.0.4.jar
-rw-r--r-- 1 igor igor 52150 2012-04-05 12:13 persistence-api-1.0.jar
-rw-r--r-- 1 igor igor 22338 2012-04-05 12:13 slf4j-api-1.5.6.jar
-rw-r--r-- 1 igor igor 7583 2012-04-05 12:13 slf4j-simple-1.5.6.jar

Hibernate requires a whopping 7424K, more than 10 times of ActiveJDBC! Even if I remove MySQL driver, this comparison is still not in favour of Hibernate.

Stack depth: this is simply counting methods on the stack trace, which gives you a pretty good idea of a framework depth. In order to see this for ORM, I mangled the SQL to cause an exception in the DB layer, and here are results:

Hibernate depth:
1276 [main] WARN org.hibernate.util.JDBCExceptionReporter - SQL Error: 1054, SQLState: 42S22
1276 [main] ERROR org.hibernate.util.JDBCExceptionReporter - Unknown column 'age' in 'where clause'
Exception in thread "main" org.hibernate.exception.SQLGrammarException: could not execute query
at org.hibernate.exception.SQLStateConverter.convert(SQLStateConverter.java:92)
at org.hibernate.exception.JDBCExceptionHelper.convert(JDBCExceptionHelper.java:66)
at org.hibernate.loader.Loader.doList(Loader.java:2536)
at org.hibernate.loader.Loader.listIgnoreQueryCache(Loader.java:2276)
at org.hibernate.loader.Loader.list(Loader.java:2271)
at org.hibernate.loader.hql.QueryLoader.list(QueryLoader.java:452)
at org.hibernate.hql.ast.QueryTranslatorImpl.list(QueryTranslatorImpl.java:363)
at org.hibernate.engine.query.HQLQueryPlan.performList(HQLQueryPlan.java:196)
at org.hibernate.impl.SessionImpl.list(SessionImpl.java:1268)
at org.hibernate.impl.QueryImpl.list(QueryImpl.java:102)
at hibernate_test.SelectAllHibernate.main(SelectAllHibernate.java:13)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)
Caused by: com.mysql.jdbc.exceptions.MySQLSyntaxErrorException: Unknown column 'age' in 'where clause'
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:936)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2870)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1573)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:1665)
at com.mysql.jdbc.Connection.execSQL(Connection.java:3176)
at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1153)
at com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:1266)
at org.hibernate.jdbc.AbstractBatcher.getResultSet(AbstractBatcher.java:208)
at org.hibernate.loader.Loader.getResultSet(Loader.java:1953)
at org.hibernate.loader.Loader.doQuery(Loader.java:802)
at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:274)
at org.hibernate.loader.Loader.doList(Loader.java:2533)
... 13 more


ActiveJDBC depth:
Exception in thread "main" org.javalite.activejdbc.DBException: com.mysql.jdbc.exceptions.MySQLSyntaxErrorException:
You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right
syntax to use near 'where age > 1' at line 1, Query: SELECT * FROM employees WHERE where age > 1
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:936)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2870)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1573)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:1665)
at com.mysql.jdbc.Connection.execSQL(Connection.java:3176)
at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1153)
at com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:1266)
at org.javalite.activejdbc.DB.find(DB.java:408)
at org.javalite.activejdbc.LazyList.hydrate(LazyList.java:304)
at org.javalite.activejdbc.LazyList.size(LazyList.java:454)
at activejdbc_test.SelectAllActiveJDBC.main(SelectAllActiveJDBC.java:13)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)
Caused by: com.mysql.jdbc.exceptions.MySQLSyntaxErrorException: You have an error in your SQL syntax; check the manual
that corresponds to your MySQL server version for the right syntax to use near 'where age > 1' at line 1
... 16 more


When you count the number of lines in exception stack trace betwen the client code (your code) and low level technology
(MySQL in this case), you will have an idea of a framework depth. For ActiveJDBC it is 3, for Hibernate it is 15.
So, Hibernate is about 5 times thicker than ActiveJDBC.

One might say: so what, why do I care about the size of dependencies, depth of stack trace, etc. I think a good developer
should care about these things. The thicker the framework, the more complex it is, the more memory it allocates,
the more things can go wrong. Besides, simply by requiring certain dependencies, it can limit your options
of using up to date versions of the same dependencies, creating headaches on classpath.

Bottom line is: choose your frameworks wisely

cheers

ActiveJDBC cascades deep and shallow

Each ORM adds functionality not found in standard DB access layer, and ActiveJDBC is no exception. Lets say delete with cascade. Method:

model.deleteCascade();

has been in existence for a while, but had somewhat limited functionality. It deleted the model and its immediate children in case One to Many and Polymorphic associations. In case of Many to Many, it was merely clearing links in a join table. The main reason for this was performance. In order to implement a true cascade delete, an ORM must follow all relationships until none left, but unfortunately in the process, it has to load every record instance into memory. This process might allocate huge chunks of memory and generate unexpected number of DELETE statements to the database.

So, initially performance considerations stopped me from implementing a true cascade delete. After all, deleting immediate children is very efficient: clean all of them in one SQL, then delete the parent.

However, once people started using ActiveJDBC, many asked a question: "this deleteCascade() is not really cascading, what the heck?" (well, they are all nice people, but I need to add some drama here keep you reading:)).
In any case, they pointed out inconsistencies of a name and the actual semantics. This prompted me to implement delete cascade true to its name. So, a new version just published to Sonatype will cascade like there is no tomorrow. It will navigate all child and many to many relationships of a model being deleted, find their children, grand children, grand-grand kinds, etc. No one walks out alive, if you know what I mean:)

Implications might be strange at first, but logical if you think about it. Imagine you have a relationship where doctors treat patients and patients visit doctors. In other words, this is a many to many relationship. If you delete a doctor, then all patients associated to that doctor are also deleted. But, what if a patient also visits another doctor? Guess what, that doctor is also deleted (because it is a dependency of a patient being deleted) and so are his/her patients, and so on. So, "deleteCascade()" really knows how to cascade!

But, what about the fast and efficient delete if all I want is to delete a model and immediate children (assuming no grand kids)? For that, there is a new method:

model.deleteCascadeShallow();

which retained the same functionality deleteCascade() had before.

So, deleting models in ActiveJDBC is an easy business, with methods:

delete();
deleteCascade();
deleteCascadeShallow();

For more detailed info, see this Wiki page: http://code.google.com/p/activejdbc/wiki/DeleteCascade

cheers..

Stop hating Java 2

This post is in response to Andrzej on Software/ Stop hating Java post. I support most of the ideas Andrzej expressed in that post. Think of this post as continuation. This is my brain dump on things that was sitting somewhere in the back of my mind for some time and Andrzej's post jolted it out.

Ruby developers are prone to cults (observation).
It is interesting to see that Ruby developers (most people I worked with are good smart guys)  seem to have all drank several cool-aids: Ruby/Rails/Apple MBPs/IPhones/TextMate. They are "green", look to be "democratic", wear sloppy clothes, and are "laid back". They certainly fall into a few stereotypical descriptions. They religiously follow another big gorilla (Apple), whose policies are even more tight than that of Microsoft. What is more interesting is that we were able to pinpoint a Ruby developer in a group photo shot! Overall, I'd say because Ruby developers are prone to cult-like behavior, they miss a bigger picture sometimes. For them if it is not Ruby, it must be crap. Ruby developers are sometimes categorical to the extend bordering adolescent behavior. I think it is generally a human thing to resists other peoples' opinions and change. (note to self: Ruby developers are humans :)).

Any platform will do
Most people dislike languages they do not use, and Ruby developers are not exception. However, any language/platform can be used to build excellent piece of software, and there are many examples of that. Most all (as someone mentioned on Andrzej's post comments) think that PHP is crap, but although I'm not a PHP developer, I enjoyed this post from MailChimp, who has proven that if you have a brain you can build a great system ... even in PHP:) - http://blog.mailchimp.com/ewww-you-use-php/

How Java screwed up royally

• Standards - the biggest flop in the Java history was Java Enterprise. This does not require any explanation, I hope. Standards are a plague of Java. They are designed to make different implementations together, but this is not happening. Instead, they take years to "standardize", when Internet years are even shorter than dog years. There is a handful of low level good standards: JDBC, Servlets, JMS, but the rest is just a waste of time.
• Trying to circumvent standards - Spring/XML mess. Spring came to mass market some time  in 2003, and spread like wild fire (due to complexity of JEE). I personally do not like Spring and try to avoid it at all costs (same goes for JEE). Spring projects are messy, impossible to debug, and tend to grow like a cactus: in all directions.
• Way of thinking that if you have a hammer(Spring/XML) in your hand, every problem looks like a nail (your project).
• Popularity of Java sucked a lot of people into the Java world who should not be there (maybe they should not even be in IT in general). I'd argue that when (if ever) Ruby becomes as popular, it will get all the problems Java has: boring business applications, millions of lines of unmaintainable legacy code, army of non-talented and non-passionate developers, corporate culture,  heavy management, etc.
• Java developers are ostriches - they keep their heads in the Java sand and are afraid to look around.

Ruby would be in obscurity if not for Rails
Ruby developers say they have things other than Rails. I'd say this is BS. All things non-Rails came about to support Rails in one form or another. I think that Rails undoubtedly made Ruby famous. Rails is a Ruby killer application. Ruby is actually older than Java by a year or two, but has been in obscurity all these years until Rails came along. If it were possible to predict a different past, I'd say that if DHH used PHP for his projects and never wrote Rails, the world as we know it would associate the word ruby with a precious stone, rather than a programming language.

My History of Rails experience
I worked on a website project for Sears that was all Java, but slowly became a blend of Java and Rails. When this was happening, the project was joined by a number of good Ruby developers. While we had disagreements and arguments, I adopted a strategy to learn as much as I could from these folks, and about Ruby/Rails thing. As it turned out, there was much to learn and so I did. In the process, I also saw that many good things in Rails can certainly be implemented in Java, to the benefit of Java community.

What I did to make Java developers happier
Needless to say, I like Rails for its productivity and think that the Rails way of conventions and style of web development is(was) better than anything I knew in Java. So, I waited for someone to do cross-pollination and implement these ideas in Java. And then I waited some more. After a 2 - 3 year wait, I realized this is not happening, and decided to take the initiative in my hands. I wrote JSpec with DSL similar that of  RSpec, ActiveJDBC - implementation of ActiveRecord in Java, and ActiveWeb - dynamic web framework similar to Rails. I manage a team of 10  developers and we have a mixed environment Rails/ActiveWeb/ActiveJDBC and about 10 commercial websites/batch applications. All new sites are built with Active* stuff, and I can attest that developer productivity in Java are the same as using Rails.

What I seriously miss in Java
Closures! Of all the stupid useless language features that Sun has been adding for years (generics for instance - only a madman can understand their syntax), they
missed the boat with closures, the one and only feature I genuinely miss. Closures certainly would make all the callbacks and stupid inner anonymous  classes go away - and this will be the biggest contribution to making Java more readable. IN addition, closures would make most Java APIs concise and easy to use, as Groovy has already done by adding a ton of methods to JDK classes.

Message to haters
It seems that people who start sentences with: "I hate..." have constipation or something. My advice: take some Metamucil, after that a few beers with friends, and then surprise people you know by always starting with: "I like ...".

Conclusion
Whew, if you are reading this sentence, you are one patient person! All these Java vs Ruby vs .NET vs PHP [plug your abbreviation here] discussions are water under bridge. People call Java Cobol of the day. I'd say that if a Java program runs a business for 30 - 40 years, it is a huge success. Who knows what languages we will be using 30 - 40 years from now? So far, we have a good selection, let's enjoy!

What is good for Ruby is good for Java: JSpec

People familiar with Ruby will invariably learn RSpec. RSpec is a great library for writing specifications, or specs as Ruby developers call them. Some time ago, I developed JSpec somewhat modeled after RSpec. I needed a better language for writing expectations. What is an expectation and how is it different from assertion?

In a Java tests, people usually use assertions to check conditions after some code is executed, such as:

assertEquals(stirng1, string1); 

In the case above if string1 and string2 are not equal, the assertion fails thus failing the test. In general, having a test is much better than not having one, but after learning RSpec, I really felt that the asserts are inadequate.
Asserts are an old way of saying: "I have developed code, and I will check that it works". I really prefer a more modern TDD/BDD approach that says: "I captured requirements of a system in test code, and will implement it after". This allows me to develop the implementation of my system after I write a specification. There is so much written on the virtues of good TDD/BDD development. Those interested should at least watch this: Dave Astels BDD presentation.

As part of the work I did while working on ActiveJDBC , I developed the JSpec library.
The main idea is to replace "assert" language with "should" language and make it as close to English as possible. This forces the brain to work in a different mode, writing a "specification of behavior" for your program rather than "assertion" that the program works. The difference might seem subtle, but requires a different style of thinking and promotes true TDD/BDD - when specifications are written before implementation, sometimes even by different people.

Here is an example of "standard" JUnit code:

@Test
public void testCalculator(){
   Calculator c = new Calculator();
   c.add(2, 2);
   assertEquals(c.result(), 4);
}

and here is the same written in JSpec (also with JUnit):

@Test
public void shouldAddNumbersCorrectly(){
   Calculator c = new Calculator();
   c.add(2, 2);
   a(c.result()).shouldBeEqual(4);
}

As you can see, the difference is subtle, but profound. More information on the use of this library can be found here: http://code.google.com/p/activejdbc/wiki/JSpec


happy coding!
igor

An easier GWT starter project

GWT is a powerful technology for Rich Internet Applications (RIA), hands down more powerful than anything else I know that can produce quality JavaScript. It comes with a simple script to generate new projects called webAppCreator. You would think that this will create a simple project structure you can then enhance and call your own, right? Kinda...

The product of this script is a working application with a server round trip that shows the power and simplicity of GWT RPC. Unfortunately, this project is chock full of comments and unnecessary code so much, that it is hard to see trees behind forest! I cleaned this many times to figure out simple things, but this time decided to drop it into the blogo-sphere so that other people could benefit as well.

I was able to reduce the amount of code to about 1/4th of the original, leaving the same functionality in place.

Here is the link: http://igorpolevoy.com/public/attach/GWT/test.zip
enjoy
igor

JavaLite HTTP gets Basic HTTP Authentication

While I'm not a big proponent of Basic HTTP Authentication, some people find it useful. Usually when you use internal services, you do not need any authentication - you use firewalls and other restrictions.

When we expose services to partners, we usually define some sort of token based authentication that is application - specific. However, the JavaLite HTTP package being general purpose HTTP client needed Basic Auth.

The new method I added is basic(user, password), here is a sample of code using basic authentication:

Get get = Http.get("http://localhost:8080/manager/html").basic("tomcat", "tomcat");
System.out.println(get.text());
System.out.println(get.headers());
System.out.println(get.responseCode());

I used Tomcat 6 to test against, and it worked just fine after simple Tomcat configuration:

<tomcat-users>
  <role rolename="tomcat"/>
  <role rolename="role1"/>
  <user username="tomcat" password="tomcat" roles="tomcat,manager"/>
  <user username="both" password="tomcat" roles="tomcat,role1"/>
  <user username="role1" password="tomcat" roles="role1"/>
</tomcat-users>

What you see above is tomcat-users.xml file, where I uncommented all XML and added a "manager" role to "tomcat" user.

As usual, I like it when: simple things are simple.

Enjoy,

Igor

Java: REST with ease :)

Update: you can get the latest Http client from Maven central:
http://search.maven.org/#search|gav|1|g%3A%22org.javalite%22%20AND%20a%3A%22javalite-common%22

Ever wanted to just send an HTTP request from Java? What should be easy is actually not. There are myriads of ways to do this in Java; you can write a half of page of ugly code with exceptions, use Apache HTTPClient library (which has its own dependencies), find another library, learn its API, etc. I told myself: "... but I just need to call a service and get a reply back!".
Eventually I got tired of this situation, and wrote my own tiny library for doing this: JavaLite.

Here is an example. If all you need is to send a GET request and get a response back, you could do this in one line:

Get get = Http.get("http://yahoo.com");
System.out.println(get.text());
System.out.println(get.headers());
System.out.println(get.responseCode());

The implementation has no dependencies and relies solely on standard Java API. The library even sets sensible defaults for timeouts. Also supports POST, DELETE and PUT.
The reason I developed it is because we are heavily relying on using REST services and a simple call to a service is really a must have.
So, an example of a service call would look like this:

if(Http.post("http://host/context/resources", postContent).text().equals("OK")){
    //..success
else{
    //failure
}

Eat that, SOAP! The code is so unobtrusive, sometimes it is hard to see. Oh, boy, I can breath now... head is cool, feet are warm...live is better now :)
Happy coding!
igor

Changing SSH port on Ubuntu Server

I followed so many online postings that did not work for me. Apparently Ubuntu has two files for SSHD configuration:
/etc/ssh/ssh_config
and
/etc/ssh/sshd_config
Both of these files have the port setting, but only the second one works for me.
Everyone is giving advice to change the port in the first file, however it just does not work.
After I changed the setting in the
/etc/ssh/sshd_config
and restarted the SSH daemon, all was fine!
Do not forget to restart the daemon:
sudo /etc/init.d/ssh restart

Cheers,
igor

ActiveJDBC has been released on Google code

Good news for those waiting for ActiveJDBC, it has been released under Apache 2 license. The project code can be found on Google Code: http://code.google.com/p/activejdbc/.
Going forward, I will be blogging about this project on a ProductiveEdge Java Blog.

Thank you for patience, enjoy!

igor

ActiveJDBC the Basics

The title of this post has a class name in it...almost. The real class name is activejdbc.Base

This is an interesting class, in a sense that it is a completely static class (all methods static) and it is designed to wrap standard JDBC functionality in the most simple and succinct way humanly possible.
It allows to open a connection, query DB and close a connection in 3 lines of code - you have to write a half of page for this in any Java technology!
Here are some examples:

Base.open("oracle.jdbc.driver.OracleDriver", "jdbc:oracle:thin:@localhost:1521:xe", "usr", "pwd1");
List<Map> records = Base.findAll("select * from people");
//..iterate over list
Base.close();

As you can see, this exposes all SQL that is interesting to a developer and hides all the ungodliness of JDBC, including driver, connection, exceptions, etc.
I literally bent backwards (well almost:)) to make this code as clear as possible.
In the example above, the Base.open() opens a connection and attaches it to a thread. This allows any subsequent call consume it, while Base.close() closes the connection, obviously.
Another example:

List<Map> records = Base.findAll("select * from people where last_name = ? and name = ?", "Smith", "John");

I think this one is self-explanatory...and another one:

Base.find("select * from people", new RowListenerAdapter() {
            public void onNext(Map record) {
                System.out.println(record);
            }
        });

In the former examples, the entire result set (findAll()) was read into a list, but in some cases you will need to read millions of records and process them as in the stream (SAX - style of sorts).
The latter example achieves this goal, and you do not have to write a loop, the Base class takes care of it.

The Base class is not a super class to anything. It is just a utility that can be used externally and of course is used internally by the framework.

cheers,
igor

ActiveJDBC Features - Birds View

August 12 update: it seems that some people link directly to this post and do not see in later posts that this project has been published on Google Code: http://code.google.com/p/activejdbc/

Original post text follows:

This blog is not really a tutorial, but rather a high level overview of some important features this framework has. As I stated in a previous post, I really bent backwards when implementing it, only to make it easier for developers to access persistent data.

I will present various use cases in a list format:

How to run a simple query

//find by id:
Person p = Person.findById(0);

//find first:
Person p = Person.first("name = ?", "John");

//simple select of multiples:
List<Person> people = 
Person.where("department = ? and hire_date > ? ", "IT", hireDate);
//...iterate

How to build pageable resultsets

List<Employee> people = 
Employee.where("department = ? and hire_date > ? ", "IT", hireDate)
              .offset(21)
                 .limit(10)
                    .orderBy("hire_date asc");
...iterate

This query will ensure that the returned result set will start at the 21st record and will return only 10 records, according to the "orderBy". The ActiveJDBC has a built in facility for various database flavors and it will generate appropriate SQL statement that is specific for a DB (Oracle, MySQL, etc) and is efficient. It will not fetch all records, starting with 1.
I tried these queries on tables with millions of records on Oracle and performance is flat.
In fact, you can learn how to create queries like this if ActiveJDBC logging is enabled.

How to create new records

Person p = new Person();
p.set("name", "Marilyn");
p.set("last_name", "Monroe");
p.set("dob", "1935-12-06");
p.saveIt();

This code should be self explanatory. As you can see, ActiveJDBC does not require to have getters and setters. You can write them, if you like, but IMHO, they are nothing but code pollution.

The set(name, value) method returns reference to the same model object, which makes it possible to string method calls like this:

Person p = new Person();
p.set("name", "Marilyn").set("last_name", "Monroe").set("dob", "1935-12-06").saveIt();

There is even a third way to set values into a model:

String[] names = {"first_name", "last_name", "dob"};
Object[] values = {"John", "Doe", new Date(johnsDobTime)}

Person john = new Person();
john.set(names, values).saveIt();

...and yet another way to set values into a model is with a use of a map:

Map values = ... initialize map
Person p = new Person();
p.fromMap(values);
p.saveIt();

I hope this was entertaining. I will write more about features of ActiveJDBC in future posts. Specifically how it handles relationships.

Constructive feedback is much appreciated!

Have fun :)

ActiveRecord in Java == ActiveJDBC

As a Java developer, I spent a a couple of years working in a mixed environment where Java and Ruby intermixed. Working side by side with Ruby developers made me aware of things outside the Java world. To some extend, this was my own fault for ignoring the Ruby world. I clearly remember beng at the SD West conference in 2005 when the Ruby frenzy began. I looked at what RoR can do and was unimpressed by it mostly because the presentations at the conference focused on generating an application from scratch using scaffolding. Having to implement web app generators for Hibernate/Tiles/Struts-Layout two years prior, this looked boring and I put my interest to Ruby on the back burner. I was wrong. A couple of years later, working in a mixed environment (Java/Ruby) I learned a whole a lot more about Ruby and Rails, and was pleasantly surprised by the level of innovation. Over the years I have developed somewhat a disgust for big frameworks and "architectures" using them. The biggest offender IMHO is Spring. I really, really, REALLY do not understand what value it provides. The Spring context files multiply like rabbits, making it extremely difficult to debug the application. As you can imagine, in a mixed environment, there were many heated discussions on the merits of different languages, and I tended to protect Java, stating that the Java environment was poisoned by large useless frameworks as well as a large number of corporate developer-drones, and really there is nothing wrong with the language. The argument that the Ruby folks had was that Ruby is more elegant, dynamic and powerful and therefore attracts a more sophisticated crowd. My arguments that there are plenty examples of outstanding software created in whatever language (including Java) were just brushed aside.  On the flip side, when Ruby becomes as successful and as prominent as Java, it too will be flooded by corporate bonehead developers. I think that some of the Ruby folks behave as Ruby were a cult, not another tool at developers' disposal. So, to make the story short, the desire to prove that it is totally possible to create a lightweight persistence layer in Java similar to ActiveRecord, and make it even simpler to use in some regard, as well as desire to roll up my sleeves and just do some coding (I have done too much architecture and management over the past few years), drove me to create another ORM framework in Java, named ActiveJDBC.
The idea is to model the behavior and feel similar that of ActiveRecord, but make it for the Java developer. It is amazing how little Java developers know of Ruby. RoR has some great features, and despite the fact that there is no method_missing in Java, many ideas are possible to borrow and implement in Java.
The ActiveJDBC is already used on one commercial project, and according to developers who are used to Hibernate, they "do not even notice it, it just works".
My design goals were:

• Should infer all metadata from DB (like ActiveRecord)
• Should be very easy to work with
• Should reduce amount of code to a minimum
• No configuration, just conventions
• Some conventions are overridable in code by simply calling methods (this will be aided by IDEs)
• No need to learn another language
• No need to learn another QL - SQL is sufficient
• Code must be lightweight and intuitive, should read like English
• No sessions, no "attaching, re-attaching" 
• No persistence managers. 
• No classes outside your own models.
• Models are lightweight, no transient fields
• No proxying. What you write is what you get (WYWIWYG :))
• Should have the least possible resistance to startup a project
• No useless getters and setters (they just pollute code). You can still write them if you like.
• No DAOs and DTOs  - this mostly junk code anyway

Well, enough suspense, I can say that I started development on my free time around September, and at the end of October I had something I could use in a real system.
Let's look at code examples.
Here is an example of a model:

public class Person extends Model {}

Despite the fact that there is no code in it, it is fully functional and will map to a table called PEOPLE automatically. Here is how to use it:

List<Person> people = Person.find("name = 'John'");
Person aJohn =  people.get(0);
String johnsLastName = aJohn.get("last_name");

As you can see, the amount of code is reduced to a level when it is actually readable.
Finder methods can also be parametrized like this:

List<Person> teenagers = Person.find("age > ? and age < ?", 10, 20);

ActiveJDBC supports many features and is great for building web applications as well. It has a nice validation mechanism similar that of ActiveRecord, automatic support of many to one and many to many relationships, batch deletes and updates (conditional too), etc. Too many to list. I hope this wets your appetite. Any feedback and suggestions are welcome.

cheers!

Daily VIM commands

These are commands I found to be using frequently when on command line:

Window Operations:

• :split - horizontal split
• :vs - vertical split
• :vsplit - vertical split
• Ctrl + W, Ctrl + W - will tab between windows
• Ctrl + W, _ - will maximize current window
• Ctrl + W, = - will resize all windows equally
• 4 CTRL-W + - will increase height of window by 4 lines

File Explorer:

• :e - will tab through the files available in current directory
• :cd <..> - will change to directory, just like system command
• :Ex - open file explorer in current window
• :Sex - split current window and open explorer in one of windows
• Hit a file to open that file in current window
• Ctrl + 6 to go back to explorer from opened file

Search/Replace

• :s/OLD/NEW - find and replace a first occurrence on current line
• :s/OLD/NEW/g - find and replace all occurrences on current line
• :%s/OLD/NEW/g - find and replace all occurrences in entire file
  

Diffing files

• From within VI: :vertical diffsplit file
• From command shell: vimdiff file1, file2

Buffer Operations

• :ls - will list currently open buffers
• :bn - will open a buffer n, where n is a number as reported by ls
• :b - tab through open buffers, select one to open with Enter

Tabs

• :tabe - opens a new tab (tab in edit mode)
• gt - advance to the next
• gT - advance to the previous
• {count}gt - go to the {count} tab
• :tabe - path/to/file - to open a file
• :tabn - go to next tab

External Commands

• :pwd - print working directory
• :cd - as usual - move to directory
• :cd D - will iterate through directories starting with D

Visual Selection Mode

• v - to put in the visual selection mode
• y - yank/copy
• x - cut selection
• p - paste at cursoe location

Autosuggest
Ctrl + N or Ctrl + P - this will show choices when typing partial working


Expand tabs with spaces(put these onto the .vimrc file):
:set tabstop=4
:set expandtab
:%retab

Mouse Support (in the .vimrc file)
:set mouse=a


This is mostly a cheet sheet for myself, ..but enjoy
igor

Ubuntu 9.04 on MacBook Pro 5.4

A few days ago I purchased a new shiny MacBook Pro, and decided to install Ubuntu on it. Why? because Linux is my habitat, and also because some software packages I use are readily available on Linux, but not on Mac OS. In any case, this blog is not about Mac OS vs Linux (more on this maybe one day; I can write a book on how Mac OS sucks :)), but rather some steps I had to overcome to complete the installation.

Out of the box, Jaunty works on MBP, with some exceptions. The trackpad is too slow be useful, the Fn key in in the reverse.
After installation using the Boot Camp, I followed the instructions on the Ubuntu forums: https://help.ubuntu.com/community/MacBookPro5-1_5-2/Jaunty, additionally used information here: https://help.ubuntu.com/community/AppleKeyboard
to adjust keyboard settings. All was fine, except the Trackpad refused to be configured. This has proven to be the biggest hurdle I spent the most time on... until I discovered this posting:
Configure ALPS (Synaptics) touchpad in Ubuntu 9.0.4 (Jaunty Jackalope)
Apparently, the fdi file must be mapped to the correct HAL device (kinda makes sense :)).
Using this command:

$lshal > hal.txt

, I was able to see that the name of my device was: "Apple Inc. Apple Internal Keyboard / Trackpad".
Armed with this knowledge, I edited my FDI file to my liking, and HAL was able to load it, and apply my settings to the right device.
The trick is to find the parameter: "info.product" in the lshal output and use it in the fdi file.
Here is my complete fdi file after adjusting speed and sensitivity:

http://igorpolevoy.com/public/attach/Ubuntu9.04OnMacBookPro/x11-synaptics-bcm5974.fdi

cheers,
igor