I developed a Unit-Test-Framework for Natural applications as part of my Master’s thesis and I wanted to share my results with you.
With NUNIT you can write automated tests for your Natural modules (subprograms or subroutines) that can be run by the framework like you may know it from JUnit, NUnit etc. Here is an example TestCase (btw, is there any way to highlight the Natural source code? I could provide my GeSHi definition):
DEFINE DATA
PARAMETER USING NUTESTP /* parameters for a single test
LOCAL USING NUCONST /* NUNIT constants (error code etc.)
LOCAL USING NUASSP /* parameters for the assertions
END-DEFINE
*
NUTESTP.FIXTURE := 'Example TestCase 1' /* this TestCase's fixture
*
INCLUDE NUTCTEMP /* the template method (SETUP -> TEST -> TEARDOWN)
INCLUDE NUTCSTUB /* stub implementations of SETUP and TEARDOWN
*
DEFINE SUBROUTINE TEST /* the main test routine
*
**************************************************************
IF NUTESTP.TEST EQ 'comparing two equal numbers should pass' /* name of first test
**************************************************************
ASSERT-LINE := *LINE; PERFORM ASSERT-NUM-EQUALS NUASSP 2 2
END-IF
*
**************************************************************
IF NUTESTP.TEST EQ 'comparing two different numbers should fail' /* name of second test
**************************************************************
ASSERT-LINE := *LINE; PERFORM ASSERT-NUM-EQUALS NUASSP 1 2
END-IF
*
END-SUBROUTINE
*
END
NUNIT parses the TC’s source code and checks whether it provides the needed inclusions, subroutines etc. shown in the listing. The actual tests are defined in the IF-branches (over ten different assertions are already available). The tests’ “names” are plain text so it is possible to provide meaningful descriptions about what is tested.
The attached screenshots show the results of a test run including a “red bar”.
What are your experiences with automated tests in Natural? Does anyone do automated testing? How do you do it? If not, how do you test your applications? Has anyone tried TDD?
Great to hear that I’m not alone here. The last 10 years I have been developing with Java/C# and recently I started developing with Natural again. The first thing I missed were unit tests. I also created a test framework using Natural classes (Also inspired by JUnit. Yours looks better at the moment ;-). I will have a look and try to incorporate some of your ideas within my framework or vice versa.
The great thing with unit tests is that we, as Natural developers, can refactor our code without fear (I already enjoyed this privilege). TDD is the next step. The only downside is that it will take some time for the other developers to make this mind shift (they also created ‘unit’. but they throw it away. A Unit test is a proof that your ‘service module’ meets some quality standard. Why should you throw that away???).
What I also like to see (as a next step) is a way to perform CI (Continuous Integration). The customer I’m working for is thinking about using Agile development (SCRUM) and, as you might know, this requires a decent test harness. So if someone has worked on CI I would very much like to hear about it.
That looks great. But don’t expect that it will work on other machines. There is some hard coded stuff there (path to the unit tests).
That’s why it’s maybe a good idea to turn these unit tests into classes (my colleague suggests to use user exists), so that these parameters can be set (what about DI. Anyone interested in building Natural SPRING).
*TRIM() doesn’t work on mainframe. Personally I would like to see the same functionality between all platforms.
One of the pieces that are missing on the mainframe are functions. It makes programming much more interesting. I don’t know why it is not supported, but it should be possible to turn a subprogram into a function since one of its parameters returns a result. The rest is just syntactic sugar.
That was exactly my background, too. Developing C# for a while now and experimenting with Scrum there was simply a need for a Unit-Test-Framework in Natural. The original version was developed by a colleague of mine (see this thread) and I refactored it (God, I love to refactor code covered by tests ;-).
I also tried to use Natural classes for the framework but they have some drawbacks (e.g. nobody I know uses them ;-). My current approach marks the end of a long journey through the depths of Natural, especially when it comes to the issue of something like “interfaces” (which comes in very handy when writing a Unit-Test-Framework but isn’t supported by Natural). So I ended up writing my own parser for the test cases…
CI is an interesting feature, too. But what exactly do you have in mind when it comes to Natural sources? Something like automated tests running after each “commit”? We use Subversion for versioning control but the problem with that is, that the source code is already located on our Unix server and doesn’t have to be commited to a repository first (in fact, it is easy to forget a commit after you changed the sources).
What hard-coded part are you talking about exactly? I tried to move the OS-specific code into two modules (take a look at my component diagram in the upper right corner) that indeed need to be changed when porting the framework. Is there even more code that is platform-specific in other modules?
Mh… I didn’t know that *TRIM() doesn’t work But I don’t think that it’s that important because it should only affect the text output. Maybe you could just remove the calls!?
Anyways, thanks for the comments. If you have more issues I would like to hear about them. Maybe we could work together and fix them…
NatUnit 2.0 was released this week! We completely renewed the framework and removed any dependencies on the file system or other frameworks of ours. It’s written in 100% Natural and should run on any platform now (LUW as well as Mainframes). Please check it out and leave us a comment.
But I don’t think that it’s that important because it should only affect the text output. Maybe you could just remove the calls!?