Model-based testing saves a lot of time and money. It’s true, and believe me, I’m a firm believer in it. Model-based testing allows you to generate better tests faster, with higher quality. Another thing that a test manager should know: creating models is fun! Who wouldn’t prefer creating models rather than creating tests manually? A person who has fun at work will always be more productive. I experienced this myself when my boss told me early in my career: “You should take a look at model-based testing!” “You should take a look at model-based testing” early in my career. Today, I can’t imagine going back and creating tests manually.

In my job, I look at models from a lot of different companies and application areas. I sometimes see a problem, especially with highly motivated people: the models start to get really complicated. I think this is because they spend too much time modeling details that don’t actually need to be modeled. When should you stop modeling?

I think it’s a question of black box versus white box testing. The literature talks about it a lot, but the reality is that it’s sometimes easy to go from black box to white box. It’s especially tempting when you have a powerful test generation engine and language. You may be doing white box testing without even realizing it.

Let me tell you a story about a case I recently encountered in enterprise IT. Someone had specified a very good model for testing a particular menu for a web application. In one part of this model, he wanted to model a context menu. In other words: « when my mouse moves over the menu, the menu should appear, and when my mouse moves out of the menu, the menu should disappear ». For reasons unknown to us, the user did not use a context menu in the model (although it was available), but modeled it literally in this detail: « when my mouse moves over the menu, the menu should appear, when my mouse moves down, left, and right, the menu should disappear » (and he forgot to specify that the menu should also disappear when the mouse moves up). From such a model, a test generation engine will create special tests for all special cases. So he got four test cases to test the menu, including a funny one: « When my mouse leaves the top of the screen, the menu should stay open. Imagine you’re a tester who’s just been given these tests to run manually. In fact, the tester won’t be able to tell that the tests were generated automatically. machine, and if you model at that level of detail, it will generate your tests and exploit that level of detail,
it will generate your tests and exploit that level of detail
A test generation engine will always assume that everything in a model is the . intended behavior and will generate tests for all of it. It can’t judge what is correct, or more or less important, unless you (the human user) specify it.

But how much time did he spend modeling the behavior of this menu, which didn’t include the proper behavior for functional testing? How much additional time did he spend figuring out why he got this weird test, and where and how to fix it? What did he gain from spending that time? How likely is it that the application under test will fail these detailed tests? What’s the value? When you model the functionality of your menu at this level of detail, you’re doing « white box » testing rather than « black box » testing . There’s a big difference between what you can model and what you should model. Remember, model-based testing is most effective when used for black box testing. If you model such details, they should be specified at this level in your specification.

To conclude, this is just one example, there are many more. When you enter the details of the feature,
always ask yourself
: Is this really relevant to my feature? Are these still black box tests? At some point, it becomes more about: How much time will I spend creating, fixing and maintaining models at this level of detail and what is the real value I will get from it? As I said before, we do not intentionally model this way. So if you notice that you are spending « too much time » making tests look better, take a step back and ask yourself: Am I doing something more complicated than it really is?

Thanks for reading! Please feel free to share your comments with us!

Read more at: https://www.conformiq.com/category/blogs/

If you have a specific topic, please let me know!