software development

A cross-language project structure

I currently find myself writing a plethora of code examples in C#, Java, Perl, and PowerShell. An issue that has cropped up is that there is not a good project directory structure that can be followed for multiple languages. At first I tried working with Maven’s (many thanks to Brian Carr for helping me understand Maven better than I ever could have on my own), but it proved too tied to Java, and not very useful outside of the Maven context anyway. I discovered Ant (not that it needed discovering) last week and have been using it intensely ever since, as well as NAnt, Ant for .NET. But again, Ant or NAnt do not really define a project structure. After looking at project structures for .NET, Java, Perl, Python, etc., I have come up with what I believe is a project structure that can satisfy all the languages I mentioned, and is flexible enough that it can be extended if an unknown requirement pops up.

The Project Structure

When researching project structures I came across Jean-Paul’s directory structure for projects, which obviously very heavily influences my own. Thanks JP! The following directory hierarchy is a proposed project structure for modern development languages.

root
|
|-- LICENSE.txt
|
|-- README.txt
|
|-- the build file
|
|-- lib
|
|-- tools
|
|-- build
|
|-- conf
| |
| |-- default.properties
| |
| |-- local.properties
|
|-- res
|
|-- docs
|
|-- dist
|
|-- src
|
|-- test

The project structure explained

  • root – this is the root of the project.
     
  • LICENSE.txt – the project’s license file
     
  • README.txt – the project’s readme file
     
  • the build file – this is the files used for automated builds. ex. an Ant or NAnt file
     
  • lib – this directory includes libraries the project depends upon during the build process and distribution
     
  • tools – this directory includes tools the project and its packages depend upon during the build process and for unit testing. ex. NAnt, NUnit
     
  • build – this directory contains the project’s build output
     
  • conf – this directory includes project configuration files
                

    • default.properties – the default properties for the project
       
    • local.properties – properties specific to individual build systems, ex. the location of a certain library that may not be installed in a default location
       
  • res – this directory includes resource localization files
     
  • docs – this directory includes the project’s documentation files
     
  • dist – this directory includes the project’s distribution files, ex. compressed versions of the project’s source code and binary build output
     
  • src – the project’s source code organized into packages
     
  • test – the project’s unit tests organized into packages

This is a recursive structure in that the top-level project structure can be repeated under the src or test directories per package. An example of this would be when a particular source package needs a particular tool or library that is not common to the entire project.

There is a saying, “In theory it works in theory.” The question is then, does this project structure survive in the real world?

.NET and Visual Studio

Top-level project structure for .NET CLI

To the right is a top-folder level view in Explorer of the .NET CLI project. Notice how it corresponds neatly to the project structure outlined above. If all.NET code was compiled with NAnt, then we could stop here. However, there is the small matter of ensuring compatibility with the world’s best, and sometimes most frustrating, integrated development environment (IDE).

Inside the IDE

The project structure described in this post works very well with other languages and IDEs. The real test is whether or not it works for Visual Studio (VS). Anyone who has programmed with VS knows that it likes to force certain a certain solution and project structure on you. For example, it creates a Properties folder per project that you cannot get rid of via the graphical user interface (GUI) (you can hand-edit the (cs|vb)proj file to get ride of it). VS also requires separate projects for unit testing, most likely because all the source code in a project is compiled into the final assembly (by default), and therefore the unit tests are relegated to a separate project so that your final binary won’t have the tests built in. VS installer projects are also required to be separate projects for various reasons. The point is that VS forces you to adhere to certain requirements if you want to take advantage of its integration features and other aspects that (in my opinion) make it the IDE there is available today (and I use NetBeans 6, Komodo IDE, and Eclipse 3.3 as well). All that said, the project structure detailed in this post does not work against VS, but as we will see, gets along with the IDE swimmingly.

The image to the left is of the same project as shown above, except it is a view of the project from inside VS. Although it may not be immediately apparent, VS is using the exact same project structure I have been discussing. I’m getting ahead of myself. First we need to talk about solutions, projects, and packages, and what they mean to you and me.

Solutions, Projects, Packages

Perhaps the best way to understand VS solutions and projects is to compare them with Java projects and packages. In Java the top-most, or root, level of any software development venture is called the Project, exactly like the project structure I have been describing. A Java project contains all of the configuration information, such as configuration files, build scripts, etc. However, there is still subdivision of source code within the project, and to achieve this something called packages are used. A package is a logical grouping of source code under a single namespace. For example, the original Java version of .NET CLI is organized under the package org.apache.commons.cli. There may be one package in a project that has a public static void main( String[] args) entry point for a command line application, and their may be another package that is a pure library. Different packages can be used for different purposes. Indeed one of the most common uses of package separation is to place packages with identical names under two separate folders, src and test, for the source code and the source code’s unit tests.

VS also has a parent-child folder structure for software development, but instead of the project acting as a root folder, the solution is at the top. A VS solution is the top-most folder for any development project you create with VS. A VS solution then contains VS projects (I know that a VS project doesn’t require a solution, but VS practically forces one on you, so I always create one, even for solutions with just one project). A VS project is akin to a Java package in that they:

  • Provide logical grouping of code
  • Can be compiled to different target types by VS (ex. libraries, executables

Because target type (what the source code gets compiled into, ex. library, executable) is specified at the VS project level, it is not possible to have a single source tree compiled into different targets using VS. For this purpose you create multiple projects under a single solution.

Remember: a Java project is the same as a VS solution and a Java package is the same as a VS project.

 

Making Nice with VS

So how does this all fit into the my project structure? Quite nicely actually. In the project structure I have described there are two folders, src and test. For .NET development the src folder should contain all of the VS projects that are not unit tests, such as libraries, executables, installers. The test folder should contain unit test projects for the projects in the src folder. A unit test project should be named the same as its corresponding source project with a suffix of .test. For example, here is what the folder structure looks like for .NET CLI.

|
|-- src
|   |
|   |-- net.sf.dotnetcli
|
|-- test
    |
    |-- net.sf.dotnetcli.test

In Java the unit test package is named the same as the corresponding source code package, and this is okay for Java because there is no restriction on having packages named the same if they are in different folders. However, in VS you cannot have two projects (remember: in VS projects are packages) that belong to the same solution named the same. Thus we append the suffix .test.

As we can see on the right, in VS the solution’s src and test folders are not visible. Instead it appears as the VS projects belong directly to the root of the solution. And for all intents and purposes, from VS’s perspective, they do. But we know better, don’t we? You can also see from the image on the right that each project, or package, contains its own src folder where its CSharp or Visual Basic (VB) source code files are stored. Remember what I said about this being a recursive project structure? The unit test project on the right illustrates my point by having its own conf folder. The unit test project’s conf folder contains the project’s application configuration file, a file that although important to the unit test does not need to belong at the solution level because other projects in the solution do not care about it.

.NET and VS Wrap-Up

Microsoft has clear ideas on structuring VS solutions and projects, and as long as you never venture beyond the warm fuzzy world of VS, they work just fine. However, if you start to explore other languages you may begin to realize that alternative project organizational structures work better for you. This is what happened to me, and so I have tried to come up with a project structure that works for .NET with and without VS.

Closing Remarks

I am not going to make this long post any longer by describing how this structure works well with languages like Java, Python, Perl, etc. Those languages have a long standing tradition of using similar project structures. Indeed, there is nothing revolutionary about my own, its special trait being that it works well with VS. The goal of this project structure is to enable developers to feel comfortable moving between many different languages, such as C#, VB.NET, Perl, Python, Java, etc.

What do you think?

Update 1 – 2008/06/02 14:00

An acquaintance of mine from VMware brought to light a possible point of confusion with regards to the project structure detailed in this post. He asked if this was a structure for source code repositories or local development, and if the former then the structure should not include directories such as tools, build, etc. He is exactly right, a source code repository should not include large binaries (if binaries at all). That is why you can set the ignore property on directories and files you don’t want to check into the repository. For example, I use Subversion and set the svn:ignore property on the lib, build, and dist directories so they are not uploaded to the repository.

Additionally, it is important to point out that the project structure discussed above does not forgo the usual trunk, branches, tags structure associated with source control, instead, it is what is used inside of the trunk or a branch.

Thanks Steve!

Advertisements

3 thoughts on “A cross-language project structure

  1. I think the overall structure looks good, particularly since (as you mention) it formalizes what is already in use in the field. Indeed…aside from small, mostly cosmetic differences it closely resembles the structure I’ve used before in projects, and I’m sure others would tend to agree. My structures do tend to be simpler, both because they are mostly in Python or C (and so typically don’t contain meta-data files) and because I don’t use IDEs (and so I don’t need to structure the code such that an IDE can locate stuff). Interesting work though.

  2. Nice article. I would just add that you may actually want to include the tools directory under your version control system so any developer who performs check-out and wants to create a “local” build would have everything ready. Your build script would refer to tools directory, instead of relying on global settings (env. variables, path, etc.). Of course, any other directories which would be result of the build or test processes would be created automatically by your build scripts and would not be kept in version control.

  3. Fadil,

    The reason I did not include the tools directory, or lib directory for that matter, under version control is due to the additional size. I always include the lib and tools dir in a source zip though. For an example of a project using the structure check out the SVMotion plug-in at http://vip-svmotion.sourceforge.net/. The source zip files on the File Release area include the tools and lib directories but the version control system does not.

    I agree, it would make things easier though.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s