How to install the Filter That’s Supposed to Go Between your Brain and your Mouth

First, an analogy. Speaking when one shouldn’t is somewhat like being faced with a 400lb bench press when you can only bench 150. The emotion is quicker than your conscious mind. You wouldn’t just go up to the bench and try 400lbs. You would pick far easier weight and train.

This is the same way we should approach installing the filter that goes between the brain and the mouth. Simply pick lots of little phrases throughout the day that you’d normally mention in conversation, and don’t say them. That’s it! Pretty soon this becomes habit, and your natural reaction to a verbal thought is to keep your mouth closed.

Believe it or not, it’s really that easy. Create a new habit that works on autopilot, it’s faster than your emotions.

How to install the Filter That’s Supposed to Go Between your Brain and your Mouth

Test Driven Development: A Quick Introduction

What is Test Driven Development?

In simplest terms, TDD is a method of developing software that relies on writing a test first, then the code that makes the test pass. Some developers prefer to use the term Behavior Driven Development, because they feel that their tests are defining how their functional code should behave.

At a more conceptual level, TDD is often used to define the feature set that a product is supposed to contain. This is usually called Acceptance Test Driven Development.

  1. Write a failing test
  2. Write the simplest code that makes it pass
  3. Refactor – follow the rules of simple design

From http://securesoftwaredev.com/2011/12/05/practicing-tdd-using-the-roman-numerals-kata/

“The hardest part of TDD, however, is the final step. Some TDD novices skip it altogether, others have trouble evolving their design. Code that follows the rules of simple design

  1. Passes all the tests
  2. Contains no duplication
  3. Clearly expresses the programmer’s intent
  4. Minimizes code

For an excellent walk-through of how TDD works in practice, see the following article: http://securesoftwaredev.com/2011/12/05/practicing-tdd-using-the-roman-numerals-kata/

TDD is an investment that pays off most when followed by everyone, all the time. During the phase of a project when the code has evolved to a high level of complexity, the high degree of test coverage ensures that changes WILL be caught by the checks in the tests. Programmers don’t have to overthink every little decision. They can make a change, verify it and move on.

TDD also pays off early on. A developer may try to foresee how the code will be used in the most generic and reusable way possible. While this is absolutely an end goal, TDD allows the developer to reach that goal organically without having to keep it all in their head or write it all down on a napkin first. Of course, this may not apply to higher level design considerations. The lower level functional design and code is created along with the tests. As the virtual napkin is written on, so the system nears completion. Over-engineering can be avoided using this approach.

Best Practices

  • Treat test objects and test support objects as first class citizens. This means that you should not avoid writing code just for the sake of making functionality testable. Take the time to allow the injection of services and configuration whenever possible. This often pays off in other ways beyond testing.
  • Separate test setup and teardown into support services where practical.
  • Test the unhappy path of code execution as well as the expected happy results.

Anti-patterns

  • Do not write tests that depend on the result of previous tests
  • Slow running tests
  • Unit Tests that cover too many methods. Save that for integration tests.

Benefits (excerpt from http://en.wikipedia.org/wiki/Test-driven_development)

“Test-driven development offers more than just simple validation of correctness, but can also drive the design of a program.[15] By focusing on the test cases first, one must imagine how the functionality is used by clients (in the first case, the test cases). So, the programmer is concerned with the interface before the implementation. This benefit is complementary to Design by Contractas it approaches code through test cases rather than through mathematical assertions or preconceptions.

Test-driven development offers the ability to take small steps when required. It allows a programmer to focus on the task at hand as the first goal is to make the test pass. Exceptional cases and error handling are not considered initially, and tests to create these extraneous circumstances are implemented separately. Test-driven development ensures in this way that all written code is covered by at least one test. This gives the programming team, and subsequent users, a greater level of confidence in the code.

While it is true that more code is required with TDD than without TDD because of the unit test code, the total code implementation time could be shorter based on a model by Müller and Padberg.[16]Large numbers of tests help to limit the number of defects in the code. The early and frequent nature of the testing helps to catch defects early in the development cycle, preventing them from becoming endemic and expensive problems. Eliminating defects early in the process usually avoids lengthy and tedious debugging later in the project.

TDD can lead to more modularized, flexible, and extensible code. This effect often comes about because the methodology requires that the developers think of the software in terms of small units that can be written and tested independently and integrated together later. This leads to smaller, more focused classes, looser coupling, and cleaner interfaces. The use of the mock object design pattern also contributes to the overall modularization of the code because this pattern requires that the code be written so that modules can be switched easily between mock versions for unit testing and “real” versions for deployment.“

Test Driven Development: A Quick Introduction

For the People who want to learn how to code: How to Code

There are two main areas of concern for any new programmer. Those are; “How to think about writing code,” and “How code behaves.”

a. How to think about code

You have data (put into variables) and you have actions. It’s a good idea to decide what data you need before your action code happens. For example, if you’re going to loop over a list of something and build another list out of it, or a string, create the result object first.

String myResult = ""; // my result is a blank string, first.
for(String x: allMyStrings) {
    myResult += ","+x; 
    // make a string like this over time - ,Bob,Fred,Mary,Steam,Apple
}
System.out.println("My result is: " + myResult); // do something with what you made.

Data followed by actions is the basic pattern of all code. While you can certainly do it other ways, following this basic formula will keep things readable and logical. If something doesn’t work, you and anyone else will have an idea of the lay of the land.

b. How code behaves

Code does two things in respect to actions: it forks and it loops. A fork in the road is a decision. Do I go left or right? This is most often in the form of an IF/ELSE statement. The second thing code does is iterate or loop over a set of data. Data you can loop over is most often in an array (like a String[] args type thing,) or a List, like List<String> or List<Customer>.

For example, even if you have zero understanding of how to write database code, you can still do a lot and plan out 90% of the code it would take to find a customer with the name Robert Smith, and change his name to Bob Smith. For the following example, let’s assume we have a Customer object that we hope someone else will help us query the database to “populate.” To populate an object is to take an empty or non-existent object and fill it with data from something, like a table. We will assume that the following call will be figured out by someone else:

Customer myBobCustomer = someDataWizardObject.getCustomerByFirstAndLastName("Robert","Smith");
// it is now someone's responsibility to make the 
// above method exist, and give us back a customer object
System.out.println(myBobCustomer.getFirstName()); // should print out "Robert"
myBobCustomer.setFirstName("Bob"); 
// this changes the data of the customer object

someDataWizardObject.update(myBobCustomer); 
// someone else needs to make the SQL to make it really save

As you can see, we make a lot of assumptions, but that is ok. This is the idea and beauty of object oriented code; that you keep things simple and separate. (Look up SOLID as an acronym in OO programming.) The details of how something is populated or updated in a database are worth worrying about, but should be separate from what you need to do at a business level.

This illustrates a principle in all problem solving that is especially important when dealing with complex situations: keep in mind an idea of the big picture, but worry about one detail at a time. I’ve noticed that beginning programmers fall into paralysis because they feel the need to completely understand what they’ll need before beginning. Code evolves, and you can trust that. When testing your code, you will see that it either works or doesn’t. Make a change and check whether it works or not.

Check out “Test Driven Development” for a deep dive on how to think this way. Cheers and good luck!!

For the People who want to learn how to code: How to Code

Working as a Team

An admission: I drank the koolaid. I am currently coming to some realizations about being a member of a development team. The reason I’m validating these things at this moment in my career is that this is first time I’ve been able to work with a truly Agile team. Lately I’ve been encouraged to do the right things for development vs. working in a pressure cooker.

I know some of you learned and experienced what I’m about to say when you were possibly 5 years old. But, I didn’t. So this is important for me and perhaps others can benefit from my “epiphany.”

Here’s what seems to work really well for us:

  1. Be nice to your team members. I know this is obvious, but… these are people who have talents, and they’re not going to use those talents when they’re not happy. Simple.
  2. Voice your opinion in the most objective way you can, but voice it! It’s important for every member of a development team to get used to seeing problems from different perspectives. The synthesis of disparate perspectives and broad (or brand new,) concepts and ideas is what makes creativity bond with knowledge and skill. I don’t know what you know. Even if I know many of the same programming techniques and skills – to another team member those skills are mixed together in a different flavor and mean something a little different. Many times it’s that jungle of variety that solves a problem within a day vs. a week.
  3. Every opinion or idea has some value. It might be a lot of value or it might be 99% bad idea. Regardless of where it falls in the spectrum of usefulness, different ideas from different people (see #2!) are vital to coming up with the absolute best approaches. The beautiful thing about this is – if you’re a Mensa member or just the “average joe,” or maybe you know next to nothing about the subject – there is always something to be used to better the process. It just takes an open atmosphere and a receptive team approach to get the benefit.
  4. Invest in research time as a team. Get together about a new technology and say to each other, in summary: “This is new, let’s try some stuff out on a shared screen and see what works and how.” Google together!
  5. Invest in team programming.

That last point is a real problem for some developers. It used to be something I detested because… hey, I had my own ways of using an IDE, or a language. I felt like trying to collaborate (although it had benefits,) just slowed me down. It wasn’t that anyone knew less or preferred some approach I didn’t like. I just… move my fingers on the keyboard my way. I do things in a certain order and I’m comfortable with it. It’s kind of like being in the passenger seat and you can’t stand the way someone else drives.

Boy, was I wrong. There is a palatable way to go about team programming. Notice I don’t say “pair” programming because that might be too specific. Team programming might be many people, it might be two.

Why does team programming rub some developers the wrong way? For my own part, I eventually figured out that the real problem was in my unwillingness to relinquish control of my problem solving process. I thought my problem solving process was pretty good. I could, in isolation, crank out a lot of good code.

That wasn’t exactly wrong, but I have discovered that there is something more. You see, there are three levels of basic maturity in this context. These are:

  1. Dependence. You need a lot of help to get going, to get familiar with the technology, etc.
  2. Independence. (This is where I felt most comfortable.) You know how to get going pretty well, and when faced with the unknown, you know it’s only a matter of time until you figure it out. You have confidence.
  3. Interdependence. This is what I’d like to talk to you about now.

I think everyone is familiar with steps one and two. Interdependence always seemed to me like something that a people-person espoused as the best way because it made everybody happy, and “relationships are everything.” As an introvert, I mostly heard “blah blah blah,” when someone would describe this type of approach. But they’re right.

If I or any team member is willing to take an initially slower approach and clue each other in on their thought processes, then sure… at first it’s obviously slow going. It can be frustrating to get past the sharing of basics that each individual has taken for granted as part of their everyday thought process. It’s easy to want to wrest the keyboard from someone else and say “I already know this, just let me type it!” However – it’s vital to realize that this is part of what happens before teamwork is internalized. It’s part of the pain of moving up to the next level.

There will come a time if you invest in teamwork (real teamwork – slogging through code together,) when everything clicks, and that frustration disappears. Internalization is a big part of this process. At first, when you learn to walk, or speak a new language or write in a new programming language, everything is a chore. Everything is effort. But there comes a time in an individual’s path to learning when the subject matter becomes a part of them. Suddenly we don’t have to think about the language anymore. We don’t have to think about how to put one foot in front of the other and shift our weight just so. It just happens. When that line is crossed, a new world opens up and creativity is suddenly an option.

A team is an organism and just as each team member reaches a point where they can bob and weave in and out of group programming, the team itself also reaches a point of nirvana. When that happens, each member of the team not only knows how to develop code in a group, but they also know how to make the best use of each person’s talents without even having to think about it. That is a powerful thing and I didn’t really see it for myself until our development team spent weeks in one conference room, learning technologies and writing code together.

Without all the different kinds of input from every person on the team (this includes quality control engineers, product managers, scrum masters, sales people…. you name it!) we would fail to create something worthy of our collective.

I have to say that it’s been one of the most rewarding experiences of my career and has done wonders for my professional maturity. To the other members of our teams – thank you, guys and gals! I look forward to more of this.

Working as a Team

How Should We Create Software? (Why do I love what I’m doing?)

First of all… some love for where I work. I am not the type that has an emotional tie to most things, but for once I really love where I’m at. iSirona. Yes, I loved other companies, too… but they changed.

I’ve been in the deep end of systems integration – mobile Federal units uploading to Washington D.C. databases, VSAM to .NET, DB2 to Flex… all kinds of crazy combinations of architectures. However, some of the most fun I’ve had in the information technology field has been in Healthcare. In fact, my first job was managing an AS/400 for a medical supply company. $7 an hour. I’ve also “hand integrated” a few solutions for hospital systems to reliably store HL7 data, analyze revenue generators within their facilities and now… well I can’t tell you what I’m doing now. But it’s fun!

To write great code it helps to have a goal you care about. You have to know (and you should ideally care!) why you’re writing the code you’re writing. It helps to have people who understand the business or the reasons for creating the software in the first place. If you don’t have that kind of understanding, you might as well be writing gibberish.

To create useful software as you create code artifacts, you have to be willing to trash what you’ve got when it’s necessary. Good software evolves fast during development. People want their jobs to be easier and specifically for us at iSirona – they want to provide a path (one of many, of course,) to better patient care. Please keep in mind that I’m NOT saying to go out and waste a lot of time. Evolving code out of the chaos of R&D has to be directed properly. That’s a huge part of the right process.

To recap: as a software developer – you/I need business people. We need constant input/feedback from clients. There must be a continuous dialog. This continuous feedback loop is crucial (and is the core reason for,) the Agile software development methodology.

Often, this means that you have to be o.k. with internally producing something that you know, somehow deep down is flawed. Perhaps it’s flawed at many levels. Maybe the solution you have just isn’t elegant enough, yet. However – if you give the subject matter experts something – anything, even though it’s flawed, it can spark crucial conversations. Why does the client want what they want? Are there more direct or more accurate, elegant ways of providing what they want? Usually, a person doesn’t know quite exactly what they want until they’ve experienced a number of failures and successes. Only then do most people give developers and liaisons truly accurate information about what should be produced.

What this all means

It means that during research and development, we should forge ahead as quickly as possible. (Within reason.) If that means creating mock-ups only, so be it. If it means creating “throwaway” code, keep in mind that code isn’t the most valuable thing produced from writing software. The value produced from writing software is the accurate and proper expression of business needs. Even further, the expression of a need people have. 

At iSirona, that’s what we do. We spend time re-factoring everything. Will it ever be perfect? Nope! But with our processes, we’re seeing amazing things because people are willing to be Agile. We have excellent subject matter experts. We have passionate Scrum Masters and Developers. We have wonderful clients that are willing to invest in helping us create something great, something amazing. We have development talent that consistently surprises me with a veritable lust for attacking problems creatively. We have people that don’t see this stuff as a job – they see it as their life’s work and ambition. That is only possible in software development when you know you are producing something that is actually having an effect – making people’s lives better.

How Should We Create Software? (Why do I love what I’m doing?)

A Pattern for Development: ORM Reverse DTO Pattern

NEOS, LLC. (a previous employer,) specializes in modernizing business processes and technology. The following issues and solutions are fairly common in that kind of effort, especially when porting from an older system where complex queries already exist and no one wants to change them or tune for other scenarios. While this may not apply to your software effort, it does ease a lot of pain when solving certain niche problems. Also, see Java Hair, the technical blog of a few folks at NEOS. 

Let’s start with a disclaimer – there are few perfect things and I don’t claim this is one of them, I just know that this works really well and I hope it helps someone else. I also don’t know if anyone’s named this pattern already. If so, let me know. Also, if there are any suggestions for better names, like “Lightning Bolt,” I’m all ears.

The Problems this Pattern Solves

Typically in a J2EE project involving Hibernate, the developers will encounter a situation where there are two or three conflicting needs. If they’re not specifically in conflict, these needs at least make the job a little difficult.

On one hand, there is a need to tune the Hibernate settings of an Entity bean to perform well in queries and take up minimal memory while executing the least number of sub-queries. On the other hand, there is a need for the Hibernate object to be editable and serializable in its original form, meaning that in some cases all data must be available and non-lazy. A third possible need is that somewhere out there is an Oracle/SQL Server/MySQL etc. DBA who is going crazy trying to tune for all the different ways Hibernate is generating queries. While each of these problems is reasonable on their own, solving all of them simultaneously (while doable,) takes up more effort than it should.

In other words, the following pattern allows the developer to solve all three problems with minimal effort on his or her part and minimal effort on the part of the DBA. Everybody’s happy, though I have heard complaints from Java purists. My response to purism is usually the same, whether it’s over-normalizing a database or writing software: “Purism and strict adherence to design doctrines are a means to an end, they are NOT the end. The end result should be cost effective deployment of a working application that can be maintained.” Sometimes I don’t actually say all of that, I may just mutter under my breath and go to refill my coffee.

How this Pattern Came to Be

Having been the software architect on several efforts where the above problems manifested (including one where Hibernate staff were involved and liked the approach,) this really didn’t become a standard until I was exposed to Oracle ADF. I like ADF quite a bit, it is a very short path to delivery and to a developer or business owner that means some good things. What caught my eye was that the ADF framework separated Entity objects from View objects. In ADF, an Entity object represents a table just like a Hibernate POJO usually represents a table. However, Entity objects proper are hardly ever shown in Lists of Values, Search results and so on. Instead, ADF abstracts the Entity objects with View objects. An ADF View object allows a layer of complexity or simplicity to built on top of one or many Entity objects, or it can be built using only a query as its source.

I decided that if it were a good idea for a core product and I’d made a similar concept work in my past projects, then maybe it was time it became a well defined solution to the aforementioned problems. The solution below isn’t exactly like ADF’s pattern. With ADF, View objects make reference to the underlying Entity objects. In the following pattern, we make an effort NOT to.

What it Is

The concept is very simple in practice. Hibernate mappings and beans are created as usual. However, when it comes time to run queries or search for data, a separate object is used. This second object is a Hibernate object that is not mapped to a table. Rather, it’s an object that includes one or more native SQL queries which alias columns. This has several benefits.

  1. A DBA can easily tune queries that return more than one result
  2. The Hibernate persistence context doesn’t need to track changes to the objects
  3. The objects are immediately detachable without ever needing to worry about lazy loading.
  4. The original entity can be tuned for any kind of eager or lazy fetching. This is especially useful if one isn’t using (or doesn’t want to deal with,) an  OSIV pattern.

To get these benefits, a few simple rules must apply to the View/DTO object.

  1. Don’t map the object to a table
  2. Don’t create more than one hibernate object to represent query results -in other words, NO relationship objects. No many to ones, no one to many… you get the idea.

There may be obvious exceptions to these rules and these are just the broad strokes. Usually when I implement this pattern I create a common abstract method in the View object that will allow a developer to get back a valid Serializable key for use in opening up a real Entity object for editing. In my implementations I usually call that method “getEntityPrimaryKey()” which returns a Serializable. Then I just pass that result to a simple Hibernate get or load operation and work with the Entity.

To reiterate: the purpose here is allow for peaceful coexistence between DBA approved queries, possibly massive search result tuning for various displays and individual tuning for editable objects.

How you choose to implement the pattern is up to you, but if I receive a few requests for examples, I’ll gladly post them here.

A Pattern for Development: ORM Reverse DTO Pattern

Javaspaces Redux

UPDATED – I’ve restored this code that was previously missing. The project is open sourced and available at http://code.google.com/p/splitframe-jar-server. I welcome improvements and commentary, and I will try and respond to requests or problems with the code.

You can download the zip file with everything you need at http://splitframe-jar-server.googlecode.com/files/SplitFrameJarServer.zip

Anyone who has used Javaspaces has probably achieved a measure of success on one or two machines, then started to think more about how to deploy their solution. They might ask…

“How do the other computers get the libraries needed to execute work without my altering their classpath?”

This is where the “codebase” comes in. Javaspaces/JINI (and other Java/RMI technologies for that matter,) have the ability to query a url for a particular class or jar file, and receive bytes from that request. That downloaded bytecode can then be added to the ClassLoader of the running JVM. For further explanation, see this excellent article.

In short, an Http server can easily serve classes OR jar files. However, configuring the client computers to take advantage of the available code still exists as a task. The scenario can increase in complexity based on added jars and new tasks to execute. One day, three jars served up may be enough. A month from then, perhaps fifteen jars are needed.

If you didn’t read the article mentioned above – Note the following… If the codebase url ends in a slash – “/” then the url points to the root of a classpath. However, if the url is a list of jars, separated by spaces, then that specific jar is served by the Http server instead. It is possible to do both:

-Djava.rmi.server.codebase=”http://poobah:8080/MyClasses.jar http://poobah:8080/ http://poobah:8080/log4j1-X.jar&#8221;

So if we serve multiple jars to the network, then obviously this configuration path can become long. When mutliple machines start participating, maintaining the proper list can be a problem. Also, one hard coded Http server address is hardly fault tolerant. Some failover ability is needed and lord yes, some simplicity.

I ran into this problem myself while working with a couple Grid Computing frameworks and decided to make my life more simple. The code for my solution can be downloaded http://splitframe-jar-server.googlecode.com/files/SplitFrameJarServer.zip, free to anyone who’d like to use it. (See the link at the bottom of this post.) The code is simple in concept and involves three components:

1. A small Http Server to handle requests.

2. A multicast listener (my own grid client processes extend a multicast base class that finds the url automatically on a LAN)

3. A jar indexer. The indexer should be pointed to a directory, and will recursively index every jar found in its path (including subdirectories.) One url is then needed to serve any class file found in any of the jars.

What this gives me, is the ability to always treat my jar serving http address as if it’s pointing to the root of an unzipped classpath, even though I may be actually serving classes from fifty jar files. Further, my grid clients can automatically find new jar servers if one of the jar servers goes down, since the servers themselves listen for requests and answer with their own url. This means that none of my grid computing remote computers require any rmi codebase configuration parameter at all. All I have to do to serve a new jar is drop the jar file into the path my jar server is indexing and it will be picked up and its classes will be available.

I am currently working on a more robust solution with a visual administration interface coupled with the ability to manage participating grid client tasks.

(repeat…) http://splitframe-jar-server.googlecode.com/files/SplitFrameJarServer.zip



			
Javaspaces Redux