Agile Software Company From Scratch

Gr8 movie! I love their office.

In the first video I watched over the weekend, Damien Katz, the creator of CouchDB, gives some advices as about when one should use the document database, which CouchDB is. He suggests to look at the real world task that is being computerized. If one would use a number of pieces of paper that are passed around to different people than is a every good indication that document based database will suit the solution well. In addition, if the piece of information is to be taken offline to be edited and later on replicated to the server to reflect the most recent version of the document, CouchDB is something to consider. Katz said that with CouchDB he is trying to bring the Lotus Notes model of document centered database to the open source world. He touches a bit upon Google’s BigTable: he believes that it’s only benefit is scalability and that CouchDB is trying to address different problems. CouchDB allows to create a view of the data whereas BigTable if just “this big key value table store.”

By the way, I also learned that Exchange was suppose to be an MS version of IBM Lotus Notes document database but it did not work out quite that well.

In the 2nd video, the presentation given by Katz at RubyFringe, I did not expect to find the great presentation about doing what you like and enjoying life (reaally something I did not expect). The stuff was good but not much practical info about CouchDB.

While browsing through planet@CouchDB I have found an excellent post about how CouchDB can make all ORM/GORM stuff obsolete. Since CounchDB does not enforce particular schema, my understanding is that all the document that are stored can be quite different i.e. contain different set of attributes. That really makes sense but I am still not sure if this is good – for sure it makes things easier in the beginning. The question that bothers me is how that affects the system once its running. What if we store a number of documents with one set of attributes and then update the view/editor to include some more. Data will be inconsistent and may lead to unpredicted results. It requires some additional checking for the presence/notnull/whatever parameters when working on the view part. If we were to use traditional/relational/schema model It would be required to alter the schema before new version could be up and running. It would be an explicit step to make sure that all the old data is suitable for new view.

Resources
Measuring performance

Starting with GWT 1.5.3 it is not possible to read data from other servers than local. It makes development a bit tedious since if one was to do a test run of an app it would have to be first compiled (which takes sooo long) and then transferred to the server that all the XMLHttpRequest would be going to. Obviously we need some kind of proxy that will help us to fool the browser. Apache mod proxy fits great. Not to have repeat the same installation and configuration procedure on our all development machines I have created a VMWare vm that contains all the necessary things. Below are the steps that I followed.

1. Configure NAT interface for VMWare 192.168.100.1/255.255.255.0 (on host machine)
2. Install Ubuntu server 8.10
3. Configure /etc/network/interfaces
   # The loopback network interface
auto lo
iface lo inet loopback
# The primary network interface
auto eth0
iface eth0 inet static
address 192.168.100.3
netmask 255.255.255.0
gateway 192.168.100.2

4. sudo apt-get install apache2
5. Enable http proxy module
   a2enmod proxy_http
6. Enable virtual hosts
   a2enmod vhost_alias
7. Allow connections to proxy from all hosts (for convenience) in /etc/apache2/mods-enabled/proxy.conf
   AddDefaultCharset off
Order deny,allow
Deny from all
#Allow from .example.com
Allow from all

8. Enable proxy mapping by adding /etc/apache2/sites-enabled/010-proxy:

   <VirtualHost *:80>
       ServerAdmin spam@poland.pl
       ServerName gwtproxy.poland.pl
       ServerAlias gwtproxy
       ErrorLog /var/log/apache2/gwtproxy.poland.pl-error_log
       CustomLog /var/log/apache2/gwtproxy.poland.pl-access_log common
       ProxyPass /pl.poland.cube.Cube http://192.168.100.1:8888/pl.poland.cube.Cube
       ProxyPassReverse /pl.poland.cube.Cube http://192.168.100.1:8888/pl.poland.cube.Cube
       ProxyPass /alfresco http://repo.cube.poland.pl:1030/alfresco
       ProxyPassReverse /alfresco http://repo.cube.poland.pl:1030/alfresco
       DocumentRoot /var/www
   </VirtualHost>

9. Add to /etc/hosts (on host machine):
   192.168.100.3 gwtproxy gwtproxy.mxts.pl
10. Edit run configuration in Intellij Idea
    

When run the hosted browser should run app via proxy and it should be possible to read data from http://gwtproxy/alfresco/* which is actually on completely different server.

Sources
Solution on google groups

My database name is erace (not to have to type the actual name of the database each time I invoke psql).

1. Get JDBC Driver for PostgreSQL
2. Drop the jar in $project_home/lib
3. Change $project_home/conf/DataSource.groovy to this:

   dataSource {
   	pooled = true
   	driverClassName = "org.postgresql.Driver"
   	username = "erace"
   	password = "passw0rd"
   }
   hibernate {
       cache.use_second_level_cache=true
       cache.use_query_cache=true
       cache.provider_class='com.opensymphony.oscache.hibernate.OSCacheProvider'
   }
   
   environments {
   	development {
   		dataSource {
   			dbCreate = "update" // one of 'create', 'create-drop','update'
   			url = "jdbc:postgresql:erace"
   		}
   	}
   	test {
   		dataSource {
   			dbCreate = "update"
   			url = "jdbc:postgresql:erace"
   		}
   	}
   	production {
   		dataSource {
   			dbCreate = "update"
   			url = "jdbc:postgresql:erace"
   		}
   	}
   }
           

Docs:
Grails Ref Guide on DataSource

When reading about exodus of staff from Google to Microsoft I have found very valuable information about how to make a great connection between software team and the client.

It was my plan to create a software organization for some time now. I have been collecting interesting writings on the subject. About a year ago I have found a set of questions that every candidate should ask their potential employer. I knew it would be fun to answer them myself from the employer perspective . Here they come along with answers.

1. Explain the company’s development process at a high level.
   Phase 1 – collect underpants, phase 2 – ?, phase 3 – profit
2. How are releases scheduled/planned, do you utilize iterations
   We try to have releases every Thursday but so far that proved to be a bit problematic due to short deadline, lack of automated builds, lack of unified skills, too loose coupling of components. As for the 2nd part – sure we try to use iterations.
3. Do you have dedicated environments (development, qa, stage, production)
   Not really. We have a server dedicated for testing but it is not very formally defined.
4. Do you follow any formal methodology (agile (what flavor), waterfall, etc)
   I am trying to implement Scrum. Currently we were only able to implement morning Scrum meetings and they proved to be vary valuable.
5. What is the company’s philosophy on team member involvement in the SDLC process
   SDLC-what?
6. Do you have a continuous build environment setup
   Nope.
7. Do you follow some form of TDD
   We try to. Actually I have assigned the task of “getting through” the testing of components in Seam to 3 people on the team but none of them was able to complete task. It might be my fault since I tried to push quicker implementation of the components that we needed rather than exploration of tests under Seam.
8. How do you track/follow bugs/issues
   We have successfully implemented and used Jira. Not all team members had enough knowledge to us it but I believe that we are slowly getting there. I would like to push Mylyn.
9. Do you perform code reviews
   Not yet. Last Friday we have delivered so so ready first version of our product. This week we should be able to install it at the customer’s site and after that we will do formal code review.
10. What type of development tools do you have
    Apart from Jira we use SVN and Red Hat Developer Studio. I am really thinking about using Mingle but that costs quite a bit of money and I really should have time to investigate it a bit longer. Last week we have started to play with VMWare as we have to test everything on different database and directory server than in development environment.
11. What type of hardware for the developers to you have
    Dell Precision M4300 Core 2 Duo T7500 (2.2GHz,4MB) with Base Discrete FX360M Graphics with 4GB of RAM.
12. Is QA involved from the onset of development
    Yyyy not really?
13. Do you have a knowledge center for the team such as a team portal, wiki, etc
    Nope. We have setup document repository based on Alfresco but I did not have time to show it to the team.
14. Do you have up to date specs on the application
    I am not sure if we need it. We are working more in R&D kind of way. There is some initial, high level overview but that is about it.
15. What form of stake holder involvement is there in the development cycle
    They pay .
16. Do you have documented coding standards, are they followed
    One of the team members has actually created such a document but I need to look at it further and possibly make some adjustments.

Uff… that was rather painful reality check. Lets make things harder — in the comments to the original questions I have found some more suggestions about how to make a better code. Lets try to go through those as well.

1. Do you use source control?
   Yes.
2. Can you make a build in one step?
   Kind of. If exporting an EAR file from IDE counts as one step then yes.
3. Do you make daily builds?
   No.
4. Do you have a bug database?
   Yes.
5. Do you fix bugs before writing new code?
   Nope. We do not have very rigorous testing so we do not know what kind of bugs are there.
6. Do you have an up-to-date schedule?
   No.
7. Do you have a spec?
   No.
8. Do programmers have quiet working conditions?
   Yes – it could be better. Our next office is not going to be on the 2nd floor on the city’s main street. The building is nice but it is kind of distracting when one opens a window.
9. Do you use the best tools money can buy?
   Not yet.
10. Do you have testers?
    Kinda – rather not.
11. Do new candidates write code during their interview?
    No.
12. Do you do hallway usability testing?
    No.

4 out of 12 – ouch again. It looks like there are still quite a few things that should be improved . I guess I better prepare some plan with the things that should be done. See next posts.

Finally, after a bit less than 4 years from the moment in which I graduated from the university I am getting a chance to go back to the subjects that were so interesting back then. I am now fully responsible for creating a software organization in the company that I am a co owner of.
During those past four years I did a couple of things that I believe me gave me some experiencethat will make my task easier. I worked as a consultant/developer at IBM/Red Hat distributor in Poland. During that time I was given an opportunity to work with a number of companies that deal with software engineering and delivery IT products and services to the customers.
My findings are rather usual:

• it is not the actual need of the customer that drives the most of IT initiatives
• most of the business people do not have a clue why given project is to take place
• most of the spendings (especially in gov area) are a waste of money
• no control over budget – simply the profit margins are so big that nobody really cares about the productivity, the costs of maintenance, number of defects, end user satisfaction
• the only reason for some projects to start is because some “smart sales bonzo” persuaded some truly confused customer to implement the product of “this great company that will solve all the problems at once”
• and a lot of similar things.

Maybe that will sound a bit naive but I really believe that we are now (at least in Poland) reaching the moment in which the business needs, end user satisfaction, quality of the solution will drive more and more sales of the software.
My goal is to have a team, or a number of teams, that will be able to deliver and maintain number of products that will win customers because of quality, simplicity of use, great support, fast delivery, fast improvements. I want those teams to be efficient, I want my people to be satisfied and I want to make money (the good thing is that I am not the sales guy in our organization ).

As Nonaka and Takeuchi recognized (or at least I remember it was them) , externalization of ideas helps to systematize thoughts. This is the reason for this blog to exists. I want to have a personal notebook that will help me to think through the experiences, new knowledge, possible obstacles, new directions in the written form.