An NUnitForms testing strategy

by Paul 3/12/2008 9:27:00 PM

These are basically some notes about to a GUI testing strategy I have adopted over the last year or so with NUnitForms. I am not going into what NUnitForms are etc here (because there an old NUnitForms post here) so I am assuming that you have at least some (or willing to get some) experience with the open source library prior to reading this post.

For a basic test fixture, typically you will inherit from the NUnitFormTest class overriding the Setup method:

[TestFixture]
public class FormXyzTests : NUnitFormTest{ 
 public override void Setup() 
 {
  new VesselDetailForm().Show();
  Application.DoEvents();
 }
 [Test] 
 public void CheckBla()
 {
  ButtonTester goButton = new ButtonTester("btnGo");
  LabelTester statusLabel = new LabelTester("btnGo");
  goButton.Click();
  Assert.That(statusLabel.Text, Is.EqualTo("Done."));
 }
}

When you test a button click you create a ButtonTester and initiate the Click method perform asserts and go home happy that your GUI is in a working state.

In the interests of refactoring GUI test that can quickly get bloated if you are not careful, I have ended up creating “form testers” that are in turn inherited by the test fixture. The form tester sets up a particular “scenario” and provides control testers and other helper methods through properties, for example:

public class FormXyzTester : NUnitFormTest
{
 public override void Setup()
 {
  VesselDetailForm frm = new VesselDetailForm();
  frm.Show();
  Application.DoEvents();
 }
 public ButtonTester GoButton
 {
  get { return new ButtonTester("btnGo"); } 
 }
 public LabelTester StatusLabel
 {
  get { return new LabelTester("lblStatus"); }
 }
}
 

Now the fixture inherits from the form tester and the tests are more readable.

[TestFixture]
public class FormXyzTests2 : FormXyzTester
{
 [Test]
 public void CheckBla()
 {
  GoButton.Click();
  Assert.That(StatusLabel.Text, Is.EqualTo("Done."));
 }
}

As other form specific related test helpers are required I add them to this base class. If needed, I also split this further with another base form tester class with common functionality. Other things I have included in these form testers are the message box responders, logging, even reporting and facilities to take screen shots of the application as its testing. These all help make for some very robust GUI testing classes.

PK Smile

 

Tip - Use failing Unit Tests to mark your TODO Items

by Paul 10/31/2007 8:40:00 PM

It's common practice for developers to make small "TODO" notes in code as they work for themselves or others to clarify at some time...

// TODO: confirm this business requirement...
// TODO: make this better!
// TODO: bread, butter, eggs and milk...

Issues can arise when the TODO's are not taken care of for whatever reason or get lost in the mayhem of meeting deadlines. A worst case scenario could arise when there is a production defect for an obscure situation and the maintenance programmer finds something like this:

// TODO: Not sure if there are any more response codes for this one, check before release.

Opps! Now that's expensive.
Now I am not saying this is good or bad (!) practice - but what I have started to do within my team is create either a failing or ignored unit test (depending on the importance) marking what would normally be an innocent "TODO" item. The unit test stays in the build as either a fail or ignore and does not drop off the radar.

[Test]
public void Need_to_confirm_foo()
{
  Assert.Fail();
}

Or...

[Test]
[Ignore("Confirmation required from the business")]
public void A_foo_only_has_a_bla()
{
}

It's much harder to miss a bunch of ignored or failing unit tests before that production release than some well hidden TODO comments!

 

An NUnitForms Note for the Form Shown Event

by Paul 6/13/2007 8:30:00 PM

Just a quick note that could drive you completely mad if you were not aware...

If you are using NUnitForms for testing your GUI and have code in the forms "Shown" event, it will not run unless you follow the Form.Show call with Application.DoEvents(), see sample code below

This example is just a Label (label1) dumped on a Form with the Load and Shown events updating the label with the respective event text:

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Text;
using System.Windows.Forms;

namespace NUnitFormsDemo1
{
    public partial class ShownTestForm : Form
    {
        public ShownTestForm()
        {
            InitializeComponent();
        }

        private void ShownTestForm_Load(object sender, EventArgs e)
        {
            label1.Text = "Load";
        }

        private void ShownTestForm_Shown(object sender, EventArgs e)
        {
            label1.Text = "Shown";
        }
    }
}

Here is some sample NUnitForms test code, the first test asserts that after the Form.Show call the label text is "Load" and the second test shows that the label text is "Shown".

using System;
using System.Windows.Forms;
using NUnit.Framework;
using NUnit.Extensions.Forms;

namespace NUnitFormsDemo1.UnitTests
{
    [TestFixture]
    public class TestFormShownIssue : NUnitFormTest
    {
        [Test]
        public void TestFormShow()
        {
            ShownTestForm frm = new ShownTestForm();
            LabelTester label1Tester = new LabelTester("label1");
            frm.Show();
            Assert.AreEqual("Load", label1Tester.Text);
        }

        [Test]
        public void TestFormShowWithDoEvents()
        {
            ShownTestForm frm = new ShownTestForm();
            LabelTester label1Tester = new LabelTester("label1");
            frm.Show();
            Application.DoEvents(); // allows the 'Shown' event to fire
            Assert.AreEqual("Shown", label1Tester.Text);
        }
    }
}

 

I will take an educated guess that this is due to the GUI message pump etc.

In general if I come across this any of this type of unexpected behavior with NUnitForms I will try a DoEvents before tearing my hair out.

Working with unsupported controls in NUnitForms

by Paul 3/20/2007 11:32:00 PM

Another sequel to the entry on "Getting Started with NUnitForms"

  http://www.pksoftware.net/devblog/post/2007/02/Getting-Started-With-NUnitForms.aspx

This post is focused on compiling the latest NUnitForms code from sourceforge.

I have seen/heard/had a few questions about testing windows forms controls that are not currently supported by NUnitForms (specifically ".Net 2.0 NUnitForms alpha 5 release" - http://sourceforge.net/project/showfiles.php?group_id=95656).

There are 2 answers...

  1. Cut your own (see the "How to add Control Testers" section at http://nunitforms.sourceforge.net/docs.html), or
  2. Get the latest source out of the subversion repository and use the generic tester class...
The subversion connection details are here: http://sourceforge.net/svn/?group_id=95656

You will need TortoiseSVN (http://tortoisesvn.net/) or similar to get the files.

 



Note that all related files for the build are also downloaded (nant, nunit, ncover and ndoc etc) so there won’t be any messing around trying to find the correct library dependencies (ahhh!) Of course, because of this the download is about 8.5mb...

Now... from here on in things could get a little messy. You could have problems with key containters, there may be a duplicate "ButtonTestser.cs" file... The list was getting a little long - the short answer to the "problems" were to compile the NUnitForms project after modifying the signing method of the 2 projects "NUnitForms" and "NUnitForms.ScreenCapture". You will also need to delete the AssemblyKeyName code references in the relevent AssemblyInfo.cs files.



Keep in mind that when you get the latest out of a code repository this sort of thing is not unexpected or "bad". Its a work in progress...

Now the latest build of the NUnitForms DLL will give you access to the generic control tester class. When you need to test a control that does not have its own tester class (e.g. ButtonTester) you can use the generic declaration and create your own tester class, e.g. for a picture box:


public class PictureBoxTester : ControlTester<PictureBox, PictureBoxTester>
{
  // Now implement each overloaded constructor calling the base class
  public PictureBoxTester() {}
  public PictureBoxTester(string name, Form form) : base(name, form) {}
  public PictureBoxTester(string name, string formName) : base(name, formName) {}
  public PictureBoxTester(string name) : base(name) {}
  public PictureBoxTester(ControlTester tester, int index) : base(tester, index) {}
}



Now in the tests you can create tester objects and access all the properties and perform clicks etc:

[Test]
public void ExamplePropertyCheckAndDoubleClickTest()
{
  PictureBoxTester picTester = new PictureBoxTester("pictureBox1");
  Assert.AreEqual(@"C:\dir\somePic.bmp", picTester.Properties.ImageLocation);
  picTester.DoubleClick();
}


Easy as pie right?! Well sort of. But much easier than writing your own windows forms GUI testing framework!!

PK  ;-)

Getting Started with NUnitForms - GUI Testing with Message Boxes

by Paul 2/12/2007 11:19:00 PM

A sequel to the entry on "Getting Started with NUnitForms"

  http://www.pksoftware.net/devblog/post/2007/02/Getting-Started-With-NUnitForms.aspx

This post is focused on handling the testing of message boxes in an application.

Another common test requirement that you will probably come across in the GUI world is the use of message boxes. To handle a message box with NUnitForms (i.e. simulate a user click or similar), use a "message box handler" method. Firstly set up the test to "expect" a message box and then supply the name of the method to handle the reaction:

[Test]
public void MessageBoxTest()
{
   base.ExpectModal("Info", "MessageBoxTestHandler");
  ButtonTester runButton = new ButtonTester("RunButton");
  runButton.Click();
}

This tells the test sub-system that a message box is expected and the title should be "Info". Also supplied is the name of a handling method - in this case "MessageBoxTestHandler". This method should create a "MessageBoxTester" and (most likely) click the OK button:

public void MessageBoxTestHandler()
{
  MessageBoxTester messageBox = new MessageBoxTester("No Item Selected");
  messageBox.ClickOk();
}

Other useful methods of the MessageBoxTester class are "ClickCancel" and "SendCommand(cmd)" where "cmd" is an enum value of type MessageBoxTester.Command:
  • OK
  • Cancel
  • Abort
  • Retry
  • Ignore
  • Yes
  • No
  • Close
  • Help
For example:

public void MessageBoxTestHandler()
{
  MessageBoxTester messageBox = new MessageBoxTester("Cancel, are you sure?");
  messageBox.SendCommand(Command.Yes);
}

A practical application of this could be tests where for example if search criteria is not supplied a "no criteria" message box is displayed. Boundary checks in methods are common places for bugs to occur so make sure you perform the same boundary check on the GUI layer!

Getting Started with NUnitForms

by Paul 2/7/2007 1:30:00 AM

An introduction to NUnitForms with a basic example of automated Microsoft .Net Windows Forms Testing using NUnit.

NUnitForms is a lesser known extension to the well known NUnit testing framework. See http://nunitforms.sourceforge.net/ for details and downloads. It is currently in the process of being upgraded from .Net Framework V1.1 to V2.0 and is only being distributed as an MSI with no source code (except via CVS as Adam pointed out below).

I have found it quite useful but the documentation was lacking and the learning curve a bit painful, hence this post! I will assume that you have NUnit and NUnitForms installed for the following code samples to run.

How can NUnitForms be used?

NUnitForms gives a developer the ability to approach forms development from a test first perspective, or to simply provide a reliable set of automated regression tests for a user interface. (For the record, I am not getting into concepts such as model-view-controller/presenter, interface coding or the like - all I am demonstrating is how to use NUnitForms for testing.)

NUnitForms replaces (or supplements) the traditional “GUI test harness” method commonly used for GUI development. Sometime the use of even a test harness is bypassed (because of time restrictions of course!) The problem with manual test harnesses are that the typically become a mess that only the original developer can even make sense of and, all the testing is done manually. NunitForms alleviates these issues by making things automated in the first place. I am not against the use of a manual GUI test harness - they still have their benefits - but compared to automated testing they lose out big time. A useful GUI test harness example that would be hard to test in an automated fashion is control resize behaviour and positioning.

How does it work?

Basically there is a lot of reflection and forms/controls parsing to locate controls and invoke the methods that are normally performed by the user, such as pushing a button. For example, a test uses NUnitForms code that fires the “OnClick” event for a button. You put together a series of these events simulating a user and you have an automated script. That’s putting it really simply!

A Simple Example

Get a new instance of VS.Net going. I created a basic form as below:

The textbox and button are named “NameTextbox” and “RunButon” respectively (more on that later).

Add a new project for the unit testing - add references to:

  • The windows application we are testing
  • NUnit
  • NUnitForms
  • System.Windows.Forms

See sample solution setup below:

Now define a test fixture as normal (with the addition of the NUnit.Extensions.Forms using directive) but there are 2 main differences:

  • the fixture needs to inherit from “NUnitFormTest” and to get the test to work at all.
  • we need to override the “Setup” method. In this setup method we create an instance of the form and show it.

Make sure you show it or you will go crazy trying to debug the problem (yes I forgot!) The setup method is called before each test so we start with a clean slate.

Now for a test. This is a dumb test, but remember we are looking at the concept!

When I push the “Run” button, I want whatever test is in the textbox to become the form title, sample test:

We need a TextBoxTester and a ButtonTester. This will give us access to the button and textbox on the form. Assign a value to the textbox and invoke a “click” on the button. Now for the test, we have a reference to the test form at the fixture level so we can get the form title from there.

using System;
using NUnit.Framework;
using NUnit.Extensions.Forms;

namespace NUnitFormsDemo1.UnitTests
{
    [TestFixture]
    public class SimpleFormTests : NUnitFormTest
    {
        SimpleForm _simpleForm;

        public override void Setup()
        {
            _simpleForm = new SimpleForm();
            _simpleForm.Show();
        }

        [Test]
        public void FillTextboxWithData()
        {
            TextBoxTester nameTextbox = new TextBoxTester("NameTextbox");
            ButtonTester runButton = new ButtonTester("RunButton");
            string expected = "From automated test.";
            
            Assert.AreEqual("Simple Form", _simpleForm.Text, "Initial value incorrect.");
            nameTextbox["Text"] = expected;
            runButton.Click();
            Assert.AreEqual(expected, _simpleForm.Text, "Title should be that textbox value.");
        }
    }
}

The test failed.

TestCase 'NUnitFormsDemo1.UnitTests.SimpleFormTests.FillTextboxWithData'
failed: Title should be that textbox value.
String lengths differ. Expected length=20, but was length=11.
Strings differ at index 0.
expected: <"From automated test.">
but was: <"Simple Form">

Now implement the code to pass the test. Here is the rocket science code back in the form:

private void RunButton_Click(object sender, EventArgs e)
{
    this.Text = NameTextbox.Text;
}

Hey - I can do that without NUnitForms!

That particular test yes - with some changes to the sample form itself. The controls would need to be exposed either via a property or similar and the button click can be simulated with the “PerformClick” method. However that calls for exposing controls via properties etc which means alot more coding and to be honest we are just scratching the surface of what NUnitForms can do. The use of NUnitForms allows us to avoid that sort of "coding simply to support testing".

I hope that can get you started, more to come... PK :-)

Demo Source: NUnitFormsDemo1.zip (12.83 KB)


More Forms Testing:

 

Powered by BlogEngine.NET 1.3.1.0
Theme by Mads Kristensen

About the author

Paul Kohler Paul Kohler
A .NET software developer living and working in Brisbane, Australia.

PK Software
E-mail me Send mail

Calendar

<<  August 2008  >>
MoTuWeThFrSaSu
28293031123
45678910
11121314151617
18192021222324
25262728293031
1234567

View posts in large calendar

Authors

Disclaimer

The opinions expressed herein are my own personal opinions and do not represent my employer's view in anyway.

© Copyright 2008