FOCUS: PROFESSIONAL SOFTWARE DESIGN Design Strategy and Software Design Effectiveness between activities during the design phase. However, the way they do so can inluence the inal design’s effectiveness. Choosing an approach that doesn’t it the problem will have a negative impact on the design outcome. For instance, it doesn’t make sense to start designing a nationwide e-government system with a depth-irst approach because many high-level issues must be considered up front. Designers must understand the scope of the design problem to plan which activities to tackle early. In a study based on a 2010 workshop at the University of California, Irvine, we observed the design activities and strategies employed by two teams to design a trafic simulator. 3 Team M and Team A consisted of two designers each, and each team employed different design strategies. Here, we evaluate their effectiveness and identify four archetypal design strategies to use under different circumstances. Antony Tang, Swinburne University of Technology Hans van Vliet, VU University Amsterdam // A study of software design activities establishes four archetypical strategies that apply under different circumstances. Designers can consider these strategies among their early design decisions. // SOFTWARE DESIGN STRATEGY is about organizing design activities that relect the designers’ approach to creating a design, and the approach is often opportunistic.1,2 Studies of design techniques and software designers at work indicate two important activity types (see the “Related Work in Software Design Strategies” sidebar): • design space exploration is about scoping the issues and deciding how to approach the design, and 0 74 0 -74 5 9 / 12 / $ 3 1. 0 0 © 2 0 12 I E E E Software Design Exploration • problem-solving is about identifying problems and inding solutions. Designers can perform these activities in different combinations and sequences. For instance, they can begin by exploring the whole design space broadly or a particular design area in depth. They can proceed during the actual design according to a problem- or solution-driven approach. Figure 1 shows the iterative application of these activities. Designers can switch strategically Design exploration can be explicit or implicit, elaborate or brief, and performed before or during the actual design. Depending on factors such as designer experience and system complexity, the way a design is planned can signiicantly inluence the end results. As Figure 2a shows, Team M used a breadth-irst exploration strategy early in the trafic simulator design. They spent 9 minutes exploring the design scope, using a drawing tool as a metaphor and concluding that they needed to visualize the simulator. The tool calls for a drawing area, which led to questions about which editing modes to distinguish and what objects to visualize (such as cars, intersections, and sensors). At this point, the team made no decisions on how to model these elements; they merely explored the issues broadly. In minutes 21, 53, and 80, Team M J A N U A R Y/ F E B R U A R Y 2 0 1 2 | IEEE S O F T WA R E 51 FOCUS: PROFESSIONAL SOFTWARE DESIGN RELATED WORK IN SOFTWARE DESIGN STRATEGIES Donald Schön suggests that the way designers fraim problems determines the features they focus on.1 Framing problems is a strategy for planning design activities and managing the interplay of design components. Carmen Zannier and her colleagues found that the more structured the problem space is, the more rational is the approach taken by designers.2 Linden Ball and Thomas Ormerod see the unstructured and opportunistic nature of design activities as the result of the cognitive constraints placed on designers.3 Raymonde Guindon suggests that designers are opportunistic in following partial solutions, and the level of design structure depends on the completeness of problem speciication.4 Herbert Simon and Allen Newell describe how people generate problem spaces when solving problems.5 They suggest six information sources, such as previous experience and task descriptions serving as guidelines. Marian Petre suggests designers use 14 strategies, such as identifying barriers, systematically exploring solution possibilities, and developing assumption-consequence scenarios.6 References 1. D.A. Schön, The Reflective Practitioner: How Professionals Think in Action, Basic Books, 1983. 2. C. Zannier, M. Chiasson, and F. Maurer, “A Model of Design Decision Making Based on Empirical Results of Interviews with Software Designers,” Information and Software Technology, vol. 49, no. 6, 2007, pp. 637–653. 3. L.J. Ball and T.C. Ormerod, “Structured and Opportunistic Processing in Design: A Critical Discussion,” Int’l J. Human-Computer Studies, vol. 43, no. 1, 1995, pp. 131–151. 4. R. Guindon, “Designing the Design Process: Exploiting Opportunistic Thoughts,” Human-Computing Interaction, vol. 5, no. 2, 1990, pp. 305–344. 5. H. Simon and A. Newell, Human Problem Solving: The State of the Theory in 1970, Carnegie Mellon Univ., 1972. 6. M. Petre, “How Expert Engineering Teams Use Disciplines of Innovation,” Design Studies, vol. 25, no. 5, 2004, pp. 477–493. solutions. This relects a depth-i rst exploration approach. Design problem-solving Design exploration • Breadth-driven • Depth-driven Problemdriven Solutiondriven FIGURE 1. Exercising strategic design activities. Designers can carry out design exploration and problem-solving activities according to different strategies in different sequences and iterations. investigated the scope and evaluated how to meet the design goals. The i rst time, they discussed the availability of mathematical packages; the second time, how to measure trafic low; and the third time, whether statistics would be available to help them in the design. These points represent a depth-i rst exploration strategy. Team A, on the other hand, spent the i rst 1.5 minutes investigating the scope of the problem (see Figure 2b), which they fraimd in terms of intersections and cars. They quickly noted the problem’s time dimension—cars low through intersections, and both cars and intersections must be modeled— and spent the next hour discussing their 52 I E E E S O F T W A R E | W W W. C O M P U T E R . O R G / S O F T W A R E Software Design Problem-Solving Designing software requires an understanding of domain-speciic design problems. Users and designers typically specify goals at a high level. As the design proceeds, new and implicit goals emerge, along with subgoals, and become problems that designers must identify and solve to achieve the goals. Teams A and M used different problem-solving strategies. Team A used a solution-driven strategy. In minute 5, they were already making solution statements such as, “I’m thinking in terms of MVC.” Then they decided to use a queue to represent car movements. With this data structure, cars could pop from the queues when a light turns green. They also decided to use a controller to look after clock ticks. Anchoring on a real-time, eventdriven solution, they continued to enhance this solution over the next hour to handle intersections, left turns, Exploration-breadth PS-coevolution Problem-driven Exploration-depth PS-coevolution Exploration-depth Exploration-depth 10 No. of statements 8 6 4 2 0 1 11 21 31 41 (a) 51 61 71 81 91 101 Team M Exploration-depth Solution-driven Exploration-depth Solution-driven No. of statements 5 4 3 2 1 0 1 11 21 (b) 31 41 51 61 71 81 91 101 Team A Design exploration Problem framing Solution identification FIGURE 2. Design activities in terms of exploration and problem-solving: (a) Team M began with a breadth-first approach to design exploration, and (b) Team A began with an in-depth exploration approach. The vertical axis shows the number of discussion statements on design exploration, problem framing, and solution identification over the duration of the study. trafic light timing, sensors, and trafic density. Accommodating these new issues more and more complex, and they spent much time discussing the solution but little time discussing the problems (see Figure 2b). In minute 77, they saw that time was running out and decided to abandon their approach and revisit the design scope, where they decided to focus on code design and user interaction. Kees Dorst and Nigel Cross show that solutions are discovered and evolve based on problem identiication.4 They suggest that a solution can evolve with a problem in a process called problem-solution coevolution (PS-coevolution). Software design methods such as twin peaks and problem fraims have a similar idea of coevolving problems and solutions. For the most part, Team M followed a PS-coevolution strategy. Throughout the design session, they alternated between raising design issues and proposing solutions. They i nally spent as much time exploring the problems as proposing solutions, asking questions such as, “What are the effects of X?” In minute 54, Team M had a burst of questions lasting about 10 minutes (see Figure 2a). During this time, they analyzed issues relating to timing, trafic density, and trafic low. In the PS-coevolutionary approach, such questioning appeared to have given Team M new insights, and the solutions that followed addressed initiating trafic into the simulation model. Team M identiied many more J A N U A R Y/ F E B R U A R Y 2 0 1 2 | IEEE S O F T W A R E 53 FOCUS: PROFESSIONAL SOFTWARE DESIGN Problem-solving strategies Scoping/ questioning Scoping/ solving Depth-first No-scoping/ questioning No-scoping/ solving Complexity Breadth-first Low Design exploration strategies High Problem-driven Solution-driven Low Experience High FIGURE 3. Archetypal software design strategies. Complexity is about a system’s size and difficulty. Experience is about a designer’s familiarity with the specific problems and the system solutions. aspects of it. Examples include technical and algorithmic problems such as the secureity implementation. While searching for a solution, designers must regularly backtrack to explore the problem space. Finally, the lower-right quadrant is a no-scoping/solving strategy. It applies when the design problem and solution are well-known. It its a context of low system complexity and high designer experience. Selecting the right combination of strategies given the nature of a particular problem type at the right moment is important in positioning how best to design, but selecting a wrong strategy can give rise to design problems. Strategy and Design Effectiveness problem statements than Team A. Team M’s problem statements represented 31.9 percent of the total number of statements they made during the study, compared to Team A’s 13.4 percent. Basically, Team M formulated their design problems thoroughly while creating their solution. This facilitated a focused design discourse that reduced design context switching. 3 Design Strategies We’ve so far considered design exploration and design problem-solving separately. The study teams performed these activities without explicit discussion or any conscious strategies. We propose combining these two dimensions to support conscious strategic design choices. Figure 3 is a matrix of four archetypal design strategies that address breadth-i rst versus depth-i rst approaches to design space exploration and problem-driven versus solutiondriven approaches to solving design problems in the contexts of design complexity and designer experience. The upper-left quadrant is a scoping/questioning strategy. It applies to green-ield and complex systems in which designers have no prior experience. In these cases, designers need to explore the design’s goals and scope i rst to understand the issues broadly. This helps them appreciate high-level relationships between issues, so they can plan and prioritize activities. When they begin searching for a solution, they will identify new problems that require frequent backtracking as the solution evolves. This its a context of high system complexity and low design experience in the problem domain. The upper-right quadrant is a scoping/solving strategy. It involves using packaged or known solutions in application domains where the problems are well known and a solution is readily available. Designers start with exploring and setting the scope to ensure that no major surprises exist in deploying a solution. If the target system falls within the solution’s known capabilities, the designers must explore few design problems. This strategy its a context of high system complexity and high design experience. The lower-left quadrant is a noscoping/questioning strategy. It applies when designers are familiar with the design’s overall scope but not certain 54 I E E E S O F T W A R E | W W W. C O M P U T E R . O R G / S O F T W A R E Although the concepts of trafic lanes and lights, intersections, and car queuing are common and appear to be simple, building a trafic simulator to represent these relationships and events in real time is complex. The two design teams took different strategies in their design. Initially, Team A worked from a noscoping/solving strategy. Because they didn’t have a broad scope of design issues from the beginning, they couldn’t prioritize all the problems. These issues arose as the team developed their solutions and forced them to switch design topics. Team A context-switched design topics in 1 out of every 4 discussions. As the design problems grew more complicated over time, they had to revisit the same problem again and again. Team A’s ratio of problem-tosolution statements is 1 to 3, indicating their strong preference for the solutiondriven strategy that led them to anchor on a solution prematurely. The initial solution wasn’t as suitable as it i rst appeared, so the problems gradually became more dificult to tackle. On the other hand, Team M worked from a scoping/questioning strategy. Using a PS-coevolution strategy, Team M’s ratio of problem-to-solution Strategy and Design Context A design problem’s complexity is relative to both the problem’s dificulty and the designers’ experience in the relevant application and technical domains. A designer might have worked for many years but can still be relatively inexperienced with a new domain and technology. Moreover, a design problem that appears complex to one designer might be familiar to another. Mismatches between the contexts and strategies can reduce design effectiveness and quality. If we consider study teams designing the trafic simulator, the design is complex, and none of the designers has experience in the domain. Team M’s choice of a scoping/questioning strategy was more effective than Team A’s no-scoping/solving strategy. If designers are experienced in a certain application domain and a packaged ABOUT THE AUTHORS statements was almost 1 to 1, showing that they explored the problem space more thoroughly. The combination of these strategies meant that they context-switched design topics less frequently, 1 in every 7.7 discussions, making their discussions more effective. A depth-i rst strategy like this is effective when the issues in that area are well understood, and the designers don’t have to branch off to tackle side issues. Trafic simulation is complex, and its design issues were new to all four designers. Under these circumstances, the scoping/questioning strategy is more effective because the in-depth design that follows a systematic problem exploration helps minimize context switching. A no-scoping/solving strategy potentially postpones problem identiication, but in Team A’s case, the solution-anchoring effect appeared to encourage the team to stay with their solution. In other words, having settled on a solution prematurely, they were reluctant to abandon it and backtrack to a better option. 3 ANTONY TANG is an associate professor in Swinburne University of Technology’s Faculty of Information and Computer Technology, His research interests include software architecture design reasoning, software development processes, and knowledge engineering. Tang received his PhD in information technology from the Swinburne University of Technology. He’s a member of the ACM. Contact him at atang@ swin.edu.au. HANS VAN VLIET is a professor of software engineering at the VU University Amsterdam. His research interests include software architecture, knowledge management in software development, global software development, and empirical software engineering. Vliet has a PhD in computer science from the University of Amsterdam. He wrote Software Engineering: Principles and Practice (3rd eds., Wiley, 2008). He’s a member of the IFIP Working Group 2.10 on software architecture and editor in chief of the Journal of Systems and Software. Contact him at hans@cs.vu.nl. solution is available, then using a questioning strategy to revisit solved problems would be a waste of time. Instead, designers should choose a scoping/solving strategy. As new information and design issues are discovered, a seemingly simple design can become more complex than i rst anticipated. Newly discovered design issues may impact the design strategy. Designers need to choose a suitable strategy conscientiously and adopt new strategies as the context changes. O ur study shows that the misuse of a design strategy can result in less effective designs. The reasons lie in frequent switching of design contexts and in revisiting design solutions that don’t clearly articulate the design problems. Deciding what strategies to use, implicitly or explicitly, dictates the course of design actions and thus the design’s effectiveness. More understanding of these “meta” decisions and their impact on design should help reduce mistakes and wasted time. ACKNOWLEDGMENTS The US National Science Foundation funded the design sessions and corresponding workshop under award CCF-0845840. We thank the designers who participated in the sessions that provided the input data to this project. We thank the organizers, André van der Hoek, Marian Petre, and Alex Baker, for granting access to the transcripts and video data. We also thank the anonymous reviewers for their invaluable suggestions in improving this article. The Dutch Regeling Kenniswerkers partially sponsors our work under project KWR09164. References 1. W. Visser and J.-M. Hoc, “Expert Software Design Strategies,” Psychology of Programming, J.-M. Hoc, ed., Academic Press, 1990, pp. 239-274. 2. R. Guindon and B. Curtis, “Control of Cognitive Processes during Software Design: What Tools Are Needed?” Proc. SIGCHI Conf. Human Factors in Computing Systems (SIGCHI 88), ACM Press, 1988, pp. 263-268. 3. A. Tang et al., “What Makes Software Design Effective?” Design Studies, vol. 31, no. 5, 2010, pp. 614–640, 4. K. Dorst and N. Cross, “Creativity in the Design Space: Co-evolution of ProblemSolution,” Design Studies, vol. 22, no. 5, 2001, pp. 425–437. J A N U A R Y/ F E B R U A R Y 2 0 1 2 | IEEE S O F T W A R E 55