From Friday 27 January till Sunday 29 January 2017 the seventh annual peer conference of the Dutch Exploratory Workshop on Testing was held at Hotel Bergse Bossen in Driebergen, the Netherlands. The conference started with dinner, drinks and games in the bar on Friday evening. A number of the attendees also attended the first TestBash held in the Netherlands and so it took quite a while for all to get to Driebergen. Furthermore the fatigue of being at two conferences in a row forced several to go to bed at a reasonable hour. But despite this fatigue the conference saw an overall energetic crowd. The discussions were nicely spread over the experience reports and every topic got the attention it deserved.
Experience reports
The theme of DEWT7 was Lessons learned in software testing and around that topic a total of seven experience reports were presented. Rick Tracy kicked off the conference with a talk about how he unintentionally broke the test environment and found a number of bugs as a result.
He learned that we need fresh angles on software and that we cannot continue to use to same approach over and over again if we want to find bugs. Boris Beizer’s Pesticide Paradox was mentioned during the discussion.
Bas van Berkel was up next and he talked about his difficulties introducting the Heuristic Test Strategy Model (HTSM) as a means to diversify the test approach.
The HTSM approach did not fit well with the intentions (mental models) of the team that consisted mainly of developers. After that attempt Bas set up a risk based testing approach and succeeded in getting a conversation about risk started.
After lunch Gwen Diagram implored us all to introduce continuous delivery into our organisations. Her lesson learned was that continuous delivery very drastically reduces the pain of deployment. We were reminded of tedious but tricky manual deployments that lasted hours and took place in the evening or in the night. Gwen learned the technical aspects of continuous delivery in one assignment and was able to apply this knowledge in other projects.
Patrick Prill continued with a story about a software development effort in which the disciplines operated on separate ‘islands’.
He was able to explain his increasing frustration with this situation using the Cynefin model. This model allowed him to discuss the complexity of the organisation with other people on the project and to build bridges between the islands.
The last experience report on Saturday was presented by Joost van Wollingen. Joost approaches testing from a technical perspectives and this allows him to find technical failures that would be hidden to the eye of the functional tester. But in another project his focus on technology left him unaware of a functional defect. So his lesson learned was that we need different perspectives in order to find the defects that matter. Testers need to be aware of the technological implementation of the software and yet they also need to maintain a critical distance in order to bring new information to the table.
The second day of the conference started with a talk by Richard Sholtes.
https://twitter.com/j19sch/status/825631815465582596
He showed us a number of elaborate (Excel) reports that contained information about the progress of testing and the results. He used these reports to communicate with his manager. Gradually it dawned on him that the reports were not read as carefully as he thought and that the decision for releasing the software had become his own responsibility. His lesson learned was that his focus should have been more on finding problems than on making elaborate reports. A discussion about the role of the tester in making release decisions followed.
The last report of the conference was presented by Ash Winter.
Ash talked about the consulting work that he did in which he visited companies and advised them on the improvement of their test approach. He saw that the problems encountered in testing were influenced by what was going on in other parts of the organisation. So he took a look at the wider picture while studying systems thinking. He read An Introduction to General Systems Thinking by Gerald Weinberg which helped him to create this picture. His quest was driven by his belief that he could make a difference. During the discussion we touched upon the principles of a tester and on viewing the testing organisation as a part of a larger system.
Analysis of a lesson learned
Ruud Cox and Joris Meerts closed DEWT7 with a workshop. Its aim was to see how lessons can be learned in software testing. As we saw during the weekend, many of the lessons were learned by the personal motivation of the tester to change something in the way of working. Terms such as bravery and ownership were mentioned. Furthermore, the tester needs background information that allows her to see things in a different perspective. Patrick encountered the Cynefin model, Ash took to reading books by Weinberg and Gwen had the advantage of having been able to study an approach in detail early on.
Models
Every lesson learned is subjective but the question is how we deal with that subjectivity in order to be able to share experiences. We can share our experiences using models and an important part of the discussions during DEWT7 focused on models. During the talk by Bas we found out that we should not assume that our understanding of a certain approach (which is a model in our heads) is similar to how, for example, the developers see it. Richard challenged his own assumptions about what management needed by experimenting with different reports. We need to share our models and figure out what the model in the head of the other person looks like. Through the experience reports by Patrick and Ash we also learned that we can investigate our own subjective opinions by modeling them.
Bias and principles
Joost van Wollingen told us op front that he was biased toward technical testing. This is another form of subjectivity that we can compensate for and communicate to others once we are aware of it. Joost’s presentation also once again triggered the topic of diversification when it comes to the test approach. Rick already mentioned that it is nice to have a fresh perspective from time to time. Bas introduced his approach to take other dimensions of the software product into consideration. And both Patrick and Ash sought different perspectives by employing systems thinking.
Criteria
We saw many criteria by which the speakers judged the outcome of an approach. Many of those criteria were personal in nature and yet tied to the goals of the organisation. Gwen showed us a clear outcome; the reduction of the amount of frustration and the time it takes to deploy an application to production. Other criteria were less measurable, such as the ability to speak the language of the developers (mentioned by Joost), the degree to which a consultant makes a difference in the organisation (mentioned by Ash), or the ability of the team to look at the software from different perspectives (mentioned by Bas and Rick). Altogether we found that often we act in an organisation based on personal experiences and feelings and that criteria by which we judge our actions become clearer along the way.
Participants
DEWT7 was experienced in the gracious company of the following people.
Ash Winter (UK)
Bas van Berkel (NL)
Beren van Daele (BE)
Christopher Chant (UK)
Emma Preston (UK)
Gwen Diagram (UK)
Huib Schoots (NL) – Facilitator
Jean-Paul Varwijk (NL)
Joep Schuurkes (NL) – Content owner
Joost van Wollingen (NL)
Manon Penning (NL)
Marianne Duijst (NL)
Patrick Prill (DE)
Philip Hoeben (NL) – Conference chair
Pieter Withaar (NL)
Richard Scholtes (NL)
Rick Tracy (NL)
Rosie Sherry (UK)
Ruud Cox (NL)
Zeger van Hese (BE) – Facilitator
The DEWT7 attendees
Program
Rick Tracy – One minor test, one huge bug
Bas van Berkel – Introducing HTSM within a project
Patrick Prill – The reason for my grumpiness
Gwen Diagram – You do what manually?!
Joost van Wolingen – About being a technically oriented tester
Richard Scholtes – What to know and what to show?
Ash Winter – Inadvertent local optimisation
Ruud Cox, Joris Meerts – Analysis of a lesson learned (workshop)
Sponsorship
DEWT7 was sponsored by the Association for Software Testing. DEWT and the participants of DEWT7 thank them for their support!
DEWT 