PPI SyEN 71

Read monthly Project Performance International's Systems Engineering Newsjournal, named "PPI SyEN". PPI SyEN presents for the engineering professional 30-60 pages of valuable technical articles on topical subjects, shorter technical pieces, in-depth coverage of the month's news in systems engineering and directly related fields, pointers to useful resources and relevant industry events, plus limited information on PPI's activities.

Welcome to PPI SyEN 71

In This Edition

1. Quotations to Open On

Read More…

2. Feature Article

2.1 Divergent Thinking in Systems Engineering Practice: Is There a Shortfall?

Read More…

3. Additional Article

3.1 The Afsluitdijk as a Complex System

Read More…

4. Systems Engineering News

4.1 29th Annual INCOSE International Symposium 2019 Call for Submissions

4.2 The US Department of Defense Announces Digital Engineering Strategy

4.3 From Grassroots to Classroom in Australia: How one Humble Event Turned into a State-wide Curriculum Resource

4.4 Cadets Lead Workshop in Detroit, Michigan

4.5 Airbus and Georgia Tech Open the Center for Model-Based Systems Engineering (MBSE)-enabled Overall Aircraft Design (OAD)

Read More…

5. Featured Organizations

5.1 Engineers Australia (EA)

5.2 Alpha Pi Mu

5.3 Macau Association of Systems Engineering (MASE)

Read More…

6. News on Software Tools Supporting Systems Engineering

6.1 Cameo Systems Modeler

Read More…

7. Systems Engineering Publications

7.1 IEEE Access

7.2 Industrial Dynamics

7.3 The Systems View of Life

7.4 Focusing Change Management Where It Counts

7.5 The Department of the Navy (USA) Systems Engineering Career Competency Model

(SECCM)

7.6 Risk Management for Project Driven Organizations: A Strategic Guide to Portfolio, Program, and Project Management Organization Success

Read More…

8. Education and Academia

8.1 Educating the Systems Engineers of the Future

8.2 Systems Engineering at Rochester Institute of Technology

8.3 Postdoctoral Fellowship: The Psychology of Decision Making in Systems Design

Read More…

9. Some Systems Engineering-Relevant Websites

Read More…

10. Standards and Guides

10.1 Best Practices for Using Systems Engineering Standards (ISO/IEC/IEEE 15288, IEEE 15288.1, and IEEE 15288.2) on Contracts for U.S. Department of Defense Acquisition Programs

Read More…

11. A Definition to Close On

11.1 Introduction to System Dynamics

Read More…

12. Conferences and Meetings

Read More…

13. PPI and CTI News

Read More…

14. PPI and CTI Events

Read More…

15. Upcoming PPI Participation in Professional Conferences

Read More…

1. Quotations to Open On

“The principles of systems engineering apply almost identically to non-systems. But their importance differs.”

Robert John Halligan

“Never express yourself more clearly than you are able to think.”

Niels Bohr

“The two forces that have done the most to shape who we are as human beings, both inside and out, and throughout history, are technology and revelation.”

Bill Viola, quoting Houston Smith in a 2009 interview

2. Feature Article

2.1 Divergent Thinking in Systems Engineering Practice: Is There a Shortfall?

by

Jim Armstrong

Email: Jim.Armstrong@stevens.edu

Stevens Institute of Technology

Copyright © 2018 by Jim Armstrong. All rights reserved.

Abstract

Divergent thinking expands both the definition of problem and solution space. Convergent thinking narrows the options and leads to a selected solution. A balance of the two is an essential element of effective systems engineering practice. A significant predominance of convergent thinkers and shortage of divergent thinkers has been noticed in a group of systems engineering students. A negative impact on exercise results was also noticed. If this is a valid observation of the general systems engineering field, there are serious implications that must be addressed in the education of systems engineers and in the workplace. This paper outlines the initial observations, impact, and potential for future research.

Introduction

After 20-plus years of teaching systems engineering courses that use reasonably complex problems for the in-class exercises, considerable experience has been gathered on the amount of time it takes to do each exercise and the types of results that should be expected. It was a surprising development when one class repeatedly finished the exercises in a quarter to half of the time expected. The results reflected the lack of time spent in analysis. As a result, the approach to reviewing the exercises changed from a short discussion of the various findings and alternatives to an instructor led addition of depth and breadth to the analysis and consideration of alternatives that had been missed.

Near the end of the course is a lesson on personality impacts on systems engineering using Myers-Briggs personality types. One of the factors in MBTI is related to the preference for convergent thinking or divergent thinking. It was discovered that this particular class was absent of any members preferring divergent thinking. This was certainly a potential factor in class behavior worth looking at.

Over several additional classes, the data was reviewed for this anomaly. In fact, there was a significant trend of convergent thinking in the students that far exceeds the overall population and even the subpopulation of engineers. This has potential impact on systems engineering performance in the work place that we should be alert to and be prepared to address.

Convergence and Divergence

While this particular discussion addresses convergence and divergence as indicated by the fourth preference in Myers-Briggs, the general intent is more in line with the definition below (encyclopedia.com, 2008). This definition is based on the description of the terms by J. P. Guilford.

“convergence-divergence n. A cognitive style defined by two radically different modes of thinking. At one extreme is convergent thinking, characterized by a tendency to home in on a unique solution to a problem, usually involving the synthesis of information, typified by analytical, deductive thinking in which formal rules are followed, as in arithmetic. It is logical, consciously controlled, reality-oriented, and largely dependent on previously learnt knowledge and skills, and it is measured by conventional IQ tests. At the opposite extreme is divergent thinking, characterized by the fluent production of a variety of novel ideas relevant to the problem in hand. Divergent thinkers prefer, and perform better at, open-ended problems that do not have unique solutions.”

This source also continues by relating divergence to creativity. While creativity is certainly of value in the practice of systems engineering, as a minimum, divergent thinking should be considered as looking beyond the given. A basic element of systems engineering should be the ability to expand beyond what is given rather than accept it as the valid outer limit of problem and solution space.

Either extreme in the choice of convergent and divergent thinking can result in problems. Too much divergence results in a continually expanding description of the problem and list of alternatives without arriving at a solution. Too much convergence leads to a hasty decision that misses critical requirements and many possible solution alternatives. A combination of convergence and divergence is depicted in Figure 1 (Pugh, 1991). Although the diagram begins with the front-end design work having been completed, the flow depicted can be applied throughout the systems engineering process. One concern with the figure shown is that it leaves the impression that the final solution is within the bounds of the initial number of concepts. In reality, the biggest value of divergence is when it expands the problem and solution space outside the originally provided bounds to identify requirements and solutions not initially included.

Figure 1: Combining convergence and divergence

A balance between the two extremes has been shown to produce better results. The following are specific instances encountered in program management and systems engineering training and consulting. In some cases, balance led to effective performance. In others, imbalance became problematic.

An example of the impact of the convergence and divergence balance and imbalance occurred during a class exercise at the Defense Systems Management College. Three classes of 30 students were starting the first session of an extended exercise to apply their learning to a simulated major procurement.

One class had a strong convergence tendency in the leadership for this problem. The leaders had made their decisions on the correct path before class started and quickly laid out the plan and handed out assignments. Within 30 minutes, the room was almost empty as the assignees took off on their assigned tasks. There was a lack of buy in and several students felt that they had good suggestions that they had not been given the opportunity to air. The group was headed down the wrong path and had to do considerable rework later.

A second class had leadership dominated by divergent behavior. After six hours, the faculty had to intervene. Not only had no decision been made, but the discussion was whether or not the list of possible issues was complete enough. Decision oriented members of this group were growing extremely frustrated.

A third group had a balanced approach. They spent a couple of hours discussing the case and looking at alternatives. Then they selected a course of action and proceeded with an effective approach that the class supported.

One real-world example of the impact of balance occurred in a contract protest case based on the performance of the contractor in systems engineering. After six months of preparation, the diverger of two investigators of the systems engineering aspects of the case suggested rereading the specification. The converger argued for several minutes that it was not necessary since they had read the specification thoroughly several months earlier. When it became apparent that it would be faster to read the document than continue the argument, the converger consented. In doing so, the “smoking gun” jumped off the pages when read in context. Both participants agreed that the balance of their abilities and thinking styles was critical to this and other consulting efforts they worked on.

Impact on Systems Engineering

As is mentioned above, the need for divergent, creative thinking is probably most critical in the early stages when the problem and solution spaces are being defined. Missing requirements or creative solutions can lead a project down an incorrect path that is difficult or impossible to recover from later. One of the primary objectives of early validation should be to find where the requirements do not match the actual environment and uncover the hidden needs and situations. There is a tendency to accept the requirements provided as set in stone. In one exercise, the students were provided with a problem including a customer-provided satellite data link and basic technical information. Analysis should have revealed that the polar orbiting satellites would have a potentially significant time delay between event observation and eventual transmission of the data to the user in a time-critical operation. In other cases, some interesting logic was applied to explain how two conflicting requirements are both valid rather than simply inquiring about them.

There is some rationale behind the lack of divergent treatment of requirements in that it is not a good idea to promote scope growth, particularly in a tightly negotiated fixed price contract. It is true that we do want engineers to focus on meeting the requirements. As systems engineers, we need to work with management to discover and address missing requirements and novel architectures in an orderly manner.

On the solution side, there is a tendency to continue to do what has worked well before. This can provide short term success, but will eventually result in not being competitive. Examples abound in many industries of excellent products that are no longer available due to creative technologies. Computers and phones are classic examples. At a more abstract level, Eberhardt Rechtin has referred to an analysis of system development awards where the significant factor in not selecting the incumbent was that they used the same architecture while the winner came up with a new approach. Certainly, at the system architecture level, telephone communications are nothing like what existed even twenty years ago.

While it might seem that the divergent thinking can be put aside at this point in the systems engineering process, that isn’t the case. In integration, a common error is to stop the thought process at the boundary of what is being developed. The result is products that work well in isolation but do not fit well in the intended environment. Again, this should be a focus of early validation. One communications site in Italy placed a steel lattice tower on top of the highest mountain between Pisa and Milan. The locals didn’t think it was a good idea and the first three winters resulted in a series of redesigns to be able to withstand the wind, snow, and ice. In another example of not seeing past the system of interest boundary, a study was being presented for new technologies for a tactical air command and control system. At the end of the briefing, a pilot asked where the system output was, and the presenter said that the entire scenario was presented to the commander in the command center. The pilot was not happy with the response and things got tense. He wanted to know when the information the system was working with would get to him to perform the mission. The presenter had completely lost focus on that because it was external to the system of interest. And, for that matter, attention to all enabling systems is also a weakness that many products have, particularly the maintenance interface.

Treatment in Systems Engineering Standards and References

The sources of definition for the expected practices of systems engineering include standards, handbooks, maturity models, and textbooks. A review of EIA 632, IEEE 1220, ISO 15288, the CMMI, and the INCOSE SE Handbook results in similar findings. Although the idea of divergent thinking is implicit in each, there is little that is explicit. The best evidence is in references to identification of relevant stakeholders and an emphasis on multiple alternatives in architectures, technologies, and designs. This also shows in the consideration of multiple alternatives in trade studies. However, the focus in each of them is the convergent path to select and define a single solution. The INCOSE handbook does use the word “creative” in regard to the systems engineering effort, but only in the discussion of the pre-concept phase.

Competency models have been developed to more specifically identify the practitioner skills that are needed to perform systems engineering tasks. There are several competency models publicly available and many others that corporations hold close. We can start with the INCOSE Systems Engineering Core Competencies Model (INCOSE, 2005). Most of the competency statements relate to specific tasks that do not rely on divergent thinking. However, in the section on architectural design, the competencies included are “able to generate alternative architectural designs” and “assess a range of architectural solutions” as examples of statements that do.

The National Academy of Public Administration included a systems engineering competency model in an analysis for the FAA (NAPA, 2008). It similarly focuses on typical systems engineering tasks without specific reference to extent of divergence or convergence in performance. There are references that sound divergent such as the plural of designs.

The Systems and Software Consortium’s competency model (Wells, 2007) does add some additional features that address how the tasks are carried out more than just the task itself. One such competency is defined as “negative thinking”, or the consideration of what is missing. This is the aspect of divergence that is emphasized in this paper, particularly as applied to defining problem space, operational concepts, and requirements.

While most of these sources do not provide obvious statements supporting the need for divergent thinking, they all have subtler implications. For the various functions of systems engineering to be effective and provide value added, they must expand both problem and solution space beyond what is given to discover what is not given. They must also be performed in an atmosphere that is acceptant of change in direction when the evidence indicates it is necessary. While decisions are necessary, and programs cannot be constantly changing direction, being stuck on an original wrong path is also not the right approach. To balance the two extremes in conduct, a balance in the mental approach is also necessary. To do this, the program will be better served by a balance of convergent and divergent thinking rather than one extreme or the other.

Systems Engineering Course Exercises

The exercises in the classes mentioned previously are fairly complex. The original system being addressed is a weather and tsunami warning system for a fictitious island nation (Armstrong, 2003). The location is in the Pacific Ocean at the equator, due south of Los Angeles. The students are given a request for a system of 1,000 fully instrumented weather buoys to be deployed over 4,000,000 square nautical miles of ocean. The purpose is both for weather prediction in shipping lanes leading to the islands and data for research. A tsunami alert capability is also required. A target budget of $250 million for 10 years is also defined.

The problem has many interfaces that must be analyzed and conflicting requirements. The complexities begin with the analysis of the multiple missions in the concept of operations and continue as the students define requirements, develop functional analysis, select an architecture, and complete a system design. Additional issues include a customer provided satellite communications service that will work for weather data, but does not address the critical timing requirements of tsunami warnings. Also, the initial scope of the system with 1,000 buoys exceeds the cost constraints. These problems and more must be addressed with an open mind to challenge the customer data and develop alternative solutions that meet the core needs. Hence, the need exists for divergent thinking.

In recent years, the problem has been tailored for a system integration course to only address a tsunami warning system in the Indian Ocean. Similar issues with convergent and divergent thinking have been observed in this more focused application.

Convergent/Divergent and the MBTI

Personality. There are multiple approaches to addressing the differences in individual behavior. Some of the more popular are Herrmann Brain Dominance, Kolb Learning Styles, and the Myers-Briggs Type Indicator (MBTI). The MBTI has been used by the author in systems engineering training for 20 years as a means of teaching systems engineers how to work better with people in leading the technical aspect of the program.

The MBTI is based on Carl Jung’s analysis of personality. Jung identified three characteristics of individuals that he observed. Myers and Briggs added a fourth which includes the convergent/divergent dimension. Each preference is described below along with the distribution ratio in the general population and the potential impact of that preference on systems engineering

The first characteristic is a preference for introversion (I) or extraversion (E), Figure 2. While we tend to view this as people being outgoing or reclusive, that is only a symptom of the meaning. The actual definition relates to whether a person is energized or drained by either group activity or solitude. The general population is split 50/50 on this preference. The benefits of extraversion in a systems engineer is the comfort level in working with groups on a project. This helps in participation in dealing with multiple stakeholders and team meetings, particularly with Integrated Product Teams. The strength of introversion is the ability to sit down and do analysis without feeling the need to go talk to someone every few minutes. Of course, the weakness of each preference is the lessened ability to do the opposite when the need for that behavior arises.

Figure 2: Introversion/Extraversion

The second characteristic addresses the type of input a person is more comfortable receiving. The intuitor (N) has an input preference for concepts and big picture. The sensor (S) prefers hands-on inputs and details as shown in Figure 3. The general population is 30% intuitor and 70% sensor. At first glance, the role of systems engineer would appear to favor the intuitor due to the focus on the higher level of abstraction of the system view and the need to be able to deal with more uncertainty. However, big picture concepts that look very good at the surface are too often found not to work when the detail analysis is performed. This ability to work well with the details of a thorough analysis and interest in getting hands dirty with the actual system as fielded is a systems engineering strength of the sensor.

Figure 3: Intuition/Sensing

The third characteristic is an output preference for the basis of judgment in making decisions. The two options are feeling (F) and thinking (T), Figure 4. The conflict between these two options is the basis for the character Spock in the Star Trek series. He is torn between the emotionally based decisions of his human mother side and the pure logical decisions of his father’s Vulcan side. This is the one preference that is gender biased. Males show a preference for thinking by two to one and females prefer feeling by the same margin. However, it must be noted that this bias is far from enough of a difference to form stereotypes or jump to conclusions based on gender. The strength of making logical decisions in systems engineering should be obvious. However, we need to recognize the considerable value in being able to empathize with others in the performance of systems engineering tasks. This is true both in working with the project team and in dealing with the various stakeholders.

Figure 4: Feeling/ Thinking

The fourth preference added by Myers and Briggs is whether a person is more comfortable with the input side or the output. Those preferring input are perceivers (P) and those preferring output are judgers (J), Figure 5. This is the preference that relates to divergent thinking or convergent thinking. In the extreme a perceiver never comes to a conclusion and the answer to every question is three more questions. This behavior is associated with laboratory scientists. At the other end of the spectrum is the total judger who has an answer before the question is even asked. A significantly high percentage of project managers exhibit this preference. In the general population, there is a slightly higher percentage of judgers than perceivers. This is the area in which an exceptional distribution was noticed in the students observed and is the focus of this analysis. The strength of the judging preference is the ability to make the decisions needed to execute the program in a timely fashion and not keep the rest of the technical team waiting for the decisions to be completed. The weakness is a tendency to rush to judgment and not look at enough alternatives. A particularly significant statement is made by (Kroeger, 1988) in discussing the judger, “Judgers, in contrast, have the tendency to judge – rather than to respond to new information, even (or perhaps especially) if that information might change their decision.” This tendency can be very dangerous in complex systems development. The value of the perceiver preference is the ability to identify multiple alternatives or issues that others have not recognized for consideration. The weakness is the lack of getting to a decision on time.

Figure 5: Perception/Judgment

The resulting four preference choices are Extraversion/Introversion, iNtuitor/Sensor, Feeling/Thinking, and Perception/Judgment. When each of the preferences is determined, the result is a four-letter preference type. The set of 16 combinations result in the 16 Myers-Briggs types. These are intended to be helpful in understanding differences in normal behavior and are not related to quality of performance or identification of psychological problems. The 16 types and the average percentage found in the general population are shown in Figure 6.

Figure 6: The 16 Types

The Data

The data from 109 students in a series of classes is shown in Table 1. The percentages for each preference, as compared to the general population are shown. The general population numbers for Feeling and Thinking are not included. As described above, this preference has a gender bias and the gender data for the classes was not retained. However, the numbers are consistent with a predominantly male population which is consistent with the overall class population.

Preference SE Students General Population Preference SE Students General Population
# % % # % %
Extraversion 53 48.6 50% Introversion 56 51.4 50%
iNtuition 66 60.6 30% Sensing 43 39.4 70%
Feeling 37 33.9   Thinking 72 66.1  
Perception 13 11.9 45% Judgment 96 88.1 55%

Table 1: SE Student Preferences and General Population

The predominance of judgers is considerably different from the general population. Engineers generally tend to lean towards the judgment preference. Table 2 shows the preferences for several categories of engineers and engineers in general based on early MBTI scoring returned to the Center for Application of Psychological Type (Briggs. 1988). The group of students that is the basis of this analysis has a considerably higher percentage of judgers than even the highest group, chemical engineers.

Type of Engineer % Judgment % Perception Number
Chemical 78.55 21.15 52
Electrical and Electronic 62.96 37.04 54
Mechanical 62.34 37.66 77
Engineer 60.45 39.55 986
Aeronautical 53.70 46.30 54
Mining 44.21 55.79 190

Table 2: Judgment/Perception Preference of Engineers

One factor that may play in the classroom situation is the group of personality types that exhibit both sensor and perceiver preferences, xSxP, as indicated by the dark border in the middle left of Figure 6 above. These are described (Kroeger) as people who prefer to learn by hands-on rather than classroom. They also are less likely to be interested in what would be perceived as a theoretical discussion. The absence of this group in significant numbers will have an influence on classroom behavior but may be a compensating factor in the workplace. However, even doubling the number of perceivers in the classes would not explain the shortage experienced.

While the volume of the data collected is limited, it matches observations in practice and classrooms over several decades for both engineers and project managers. This should be considered a significant risk that needs to be addressed in the training and education of systems engineers. Identifying and addressing the issues should result in an improvement in the practice leading to better results for the customer from the broader understanding of the problem in the requirements phase, an improved set of alternatives in the architecture, and a more thorough integration, verification, and final validation.

Other Areas of Concern

While the focus of this discussion has been the distribution of divergent and convergent thinkers in the systems engineering population, there are other potential preferences to address. For instance, the distribution of big picture (intuition) and detail (sensing) thinking was significantly tilted towards the big picture side in this sample population. At first look, it would seem appropriate that systems engineers have this tendency since the primary value in systems engineering is to coordinate the detail views of specialist in multiple disciplines to achieve the higher order purpose. However, extremes can have negative consequences even when they appear to be the right extremes. Sometimes those with an extreme preference for the big picture are lacking in attention to details and head down paths that later are discovered to be dead ends. At the other end is the tendency to be too much into the details and get lost in them. Or, as one engineer put it, it is more than not seeing the forest for the trees; it’s not seeing the trees for the bark. In each of the preferences, there is value in each aspect and it helps to know where the strengths and weaknesses are to apply them appropriately.

In one small software development organization, the manager was having difficulty in promoting from within for first level managers. A sample of 30 of the employees, about 15% of the organization, taking part in a class was 85% introverted, sensing, thinking, perceivers (ISTP). This type is only about 5% of the general population. However, the characteristics are positive for software developers. They are comfortable working at a computer station developing detailed logic and coming up with creative solutions to the technically challenging problems this particular group faced. However, when they moved to take on management responsibilities, they needed to get away from their own screen and interact with those they managed, see the big picture, deal well with people, and make decisions. It is not a case of not being able to do these tasks, but rather needing to understand that they were out of the comfort zones of most of the employees. Special care needed to be taken to aid in the transition. Knowledge of the situation would allow management to address the previously unrecognized issues and more effectively develop the leaders they needed

Conclusion

The initial observations of a bias towards convergent thinking in a student population of practicing systems engineers have potential impact on the overall practice of systems engineering. If this holds to be true in the general population of the systems engineering community, the result is likely to be a less effective definition of both problem and solution space and a sub-optimal end product. Awareness of the circumstance can lead to corrective actions. A balance should be sought in the use of both styles of thinking. Where practical, this can be established by understanding the strengths of potential systems engineering practitioners and assuring that both styles are represented. Some organizations do use instruments such as the MBTI to address the strengths of the workforce. Should an unbalanced situation exist, as has been seen in some cases documented here, extra effort can be applied to stimulate additional divergence. However, this is not an easy task, particularly when attempted by someone who is a strong convergent thinker. Further research into the validity of the observations, extent of the condition, impact, and mitigations is warranted. Meanwhile, we can all be aware of this potential issue in the practice of systems engineering.

List of Acronyms Used in this Paper

Acronym Explanation

CMMI Capability Maturity Model Integration

EIA Electronic Industries Alliance

FAA Federal Aviation Agency

IEEE Institute of Electrical and Electronics Engineers

INCOSE International Council on Systems Engineering

ISO International Organization for Standardization

MBTI Myers-Briggs Type Indicator

References

Armstrong and Bocast, The Integration of Systems Engineering and Software Development, Software Productivity Consortium, April 2003.

Bennet, E. A., What Jung Really Said, Schockten Books, New York, 1983.

Capability Maturity Model Integration, Development Version 1.2, Carnegie Mellon University, August, 2006.

EIA 632, Process for Engineering a System, Electronic Industries Alliance, Arlington, VA, January 1999.

Herrmann, The Theory behind the HBDI® and Whole Brain® Technology, August 1999.

IEEE 1220, IEEE Standard for Application and Management of the Systems Engineering Process, IEEE, 345 East 47th Street, New York, NY 10017, 1999. Available at https://standards.ieee.org/standard/1220-2005.html.

INCOSE, Systems Engineering Handbook, a Guide for System Life Cycle Processes and Activities, June 2006. Available through INCOSE at https://www.incose.org/products-and-publications/se-handbook.

ISO/IEC 15288, Systems engineering – System life cycle processes, ISO, 1 rue de Varembe, CH-1211 Geneva 20, Switzerland, 2002. Available through ISO at https://www.iso.org/standard/63711.html.

Kroeger and Thuesen, Type Talk, Delecorte Press, New York, 1988.

MIL-STD-499B, Systems Engineering, May 1, 1995.

Myers, Isabel Briggs and Mary H. McCaulley, Manual: A Guide to the Development and Use of the Meyers-Brigs Type Indicator, Consulting Psychology Press, Palo Alto, CA, 1988.

Naval Systems Engineering Guide, Naval Systems Engineering Steering Group, Washington, D.C., 2004. Available at http://www.dtic.mil/dtic/tr/fulltext/u2/a527494.pdf.

National Academy of Public Administration, Identifying the Workforce to Respond to a National Imperative – The Next Generation Air Transportation System (NextGen), 2008. Available at https://www.napawash.org/studies/academy-studies/identifying-the-workforce-to-respond-to-a-national-imperative-the-next-gene.

Pugh, Stuart, Total Design: Integrated Methods for Successful Product Engineering, Addison-Wesley Publishing Company, Wokingham England, 1991.

Wells, Systems Engineering Competency Model, Systems and Software Consortium, Inc., 2006.

About the Author

C:\Users\Ralph\Documents\Articles\Jim Armstrong\Jim_Armstrong.jpg

Jim Armstrong has practiced systems engineering for 51 years, performing various roles including configuration management, test, deployment, chief engineer, program manager, and program element monitor. For the last 30 years, he taught, consulted, and appraised systems engineering in industry and government. Also, he was on the author teams for several of the standards and models discussed in this paper. He is also certified in the use of the Myers-Briggs Type Indicator. He has a BS in Mechanical Engineering from Rensselaer Polytechnic Institute, an MS in Systems Management from the University of Southern California, and a PhD in Systems Engineering from Stevens Institute of Technology. He has an INCOSE Expert Systems Engineering Professional (ESEP) certification.

3. Additional Article

3.1 The Afsluitdijk as a Complex System

by

Berber de Liefde-Vogt

Systems and Integration Architect bij Rijkswaterstaat

Rotterdam Area, Netherlands

Introduction

In 1891, a young engineer by the name of Lely developed the first plans for the Afsluitdijk (Enclosure Dam). In 1913, serving as the Dutch Minister of Water Management, he made sure that the project was placed on the Cabinet agenda. One of the purposes of the enclosure was to reclaim land to improve food supply. The plans were labelled financially unfeasible and shelved. However, the 1916 flood disaster and the growing need for farmland during the First World War caused the National Government to change their minds. In 1918, the Dutch Parliament agreed to adopt the Zuyder Zee Act, soon followed by the start of construction of the Afsluitdijk, which was completed in 1932.

As well as providing better protection of the hinterland against flooding as well as an opportunity to reclaim new land from the sea, the damming off of the Zuiderzee by the Afsluitdijk created a freshwater reservoir. Furthermore, the construction of the Afsluitdijk enabled a traffic connection between the provinces of North Holland and Friesland.

Figure 1: Location of the Afsluitdijk

Figure 1 represents the main system elements:

  • Purple arrow: Primary water-retaining structures
  • Grey arrow: Main water system
  • Orange arrow: Main waterway network
  • Red arrow: Trunk road network
  • Green arrow: Cycleway and footpath network

At the time, the Afsluitdijk was constructed using the then available knowledge and understanding. The design process took into account factors such as the tides in the Wadden Sea, while at the same time taking into account that it had to be possible to discharge water brought in by the River IJssel[1] and the surrounding areas into the Wadden Sea. In the 21st century, these boundary conditions are still valid.

However, both the physical and the public environment of the Afsluitdijk have changed as a result of climate change. Over the years, as well as environmental protection and sustainability rising in importance the years are beginning to take their toll. These years of service life not only affect the physical design, but also the cultural experience. A system such as the Afsluitdijk should be adaptive and able to anticipate change. However, this has an impact on the original objectives, making it necessary to redefine these objectives. Not only will new objectives be added, such as contributing to environmental protection and sustainability, but also existing objectives, such as guaranteeing the safety of the hinterland, will be tightened.

In other words, this redefinition means that we need to make the Afsluitdijk system ‘Fit for the Future!’

The challenge to make the Afsluitdijk ‘Fit for the Future’ involves a large number of projects on and around the Afsluitdijk (www.deafsluitdijk.nl). One of them is the Afsluitdijk project, which was contracted out as a DBFM (design, build, finance, and maintain) contract. The most important theme within this project is to ensure effective flood management. An inspection in 2006 revealed that the Afsluitdijk no longer met the standards in terms of flood management, meaning it no longer complied with the function of ‘Defend high tide’. At the same time, the water discharge capacity, or the function of ‘Discharge water’ from the IJsselmeer, was not sufficient any more either.

This article addresses the manner in which systems engineering will make the Afsluitdijk future-resilient in a structured and controlled manner. As an example of this approach, we discuss the objective of ‘Guaranteeing the safety of the hinterland (keeping our feet dry)’. In order to achieve this objective, the Afsluitdijk as a system needs to have the necessary capabilities and the corresponding functions to be able to protect the hinterland.

In view of the large number of different projects that are taking place within the Afsluitdijk system – and, therefore, the large number of interactions between these projects – it is of the utmost importance to ensure that the various scopes and scope boundaries are properly monitored in order to ensure control over the complexity. This applies to geographical, organizational, process-related as well as technical influences (interfaces). The approach described below contributes to this goal by allowing future control of objectives, capabilities, and functions.

The Afsluitdijk System

The complexity of the Afsluitdijk system arises not only from diverse objectives, but also from the large number of stakeholders (over 250), each with their own needs. These stakeholders include members of national, local, and regional tiers of government; members of environmental protection groups; representatives of agriculture, fisheries, recreation, transport, et cetera. All of these stakeholders represent the interests of their respective fields, ranging from economic interests to environmental protection.

In order to bring the various stakeholders ‘on board’, we use a model known as SEM (Dutch: SOM), Strategic Environmental Management, as represented in the Figure 2. One of the elements of SEM is an issue monitor that will clearly outline the issues that need to be addressed. Another element of SEM is a regularly updated issue file. The position of a wind turbine, for instance, may be an issue in terms of whether it is located on or alongside the dike. A change of government at a regional level in itself may present an issue. Based on the issues, a strategy will be developed, as well as any measures necessary to tackle the issue.

Figure 2: SOM-Model (source: WesselinkVanZijst)

The stakeholders are involved at the outset of the project (plan development phase of the Multi-Year Program for Infrastructure, Space, and Transport; Dutch: MIRT), and a stakeholder/environmental analysis is performed to gather information about the various needs. Within the systems engineering procedure of Rijkswaterstaat (the Directorate-General for Public Works and Water Management), a prioritization process is carried out to determine which needs will and will not be addressed and included in the development of the system.

The interests of the various stakeholders are identified and laid down in several objectives, indicating the criteria that need to be met. Validation will take place based on these criteria. The law (Water Act) may be a criterion, for example, as can measures taken by the stakeholders.

Figure 3 provides a partial overview of the various objectives as well as their corresponding validation criteria.

During the various phases of the entire life cycle of the system, the objectives for the Afsluitdijk system may be inconsistent or contradictory. To name one example, at high tide in the Wadden Sea (Objective: Guaranteeing the safety of the hinterland), it is impossible to support the natural fish migration (Objective: Improving the natural environment, G.01.03). This means that at times of excessively high water levels, it is impossible to comply with both the ‘Guaranteeing the safety of the hinterland (G.01.01)’ and the ‘Improving the natural environment (G.01.02)’ objectives. In this situation, ‘Guaranteeing the safety of the hinterland’ will prevail over ‘Improving the natural environment’. In similar situations, the objectives will be prioritized based on the stakeholder analysis.

In order for the system to achieve the various objectives, the Afsluitdijk will need to have a number of capabilities.

  • Green are objectives
  • Blue are capabilities
  • Yellow are the functions that provide the above capability

Figure 3: Objectives and their validation criteria

Figure 4: A subset of objectives and capabilities of the Afsluitdijk

One of the most important capabilities that the Afsluitdijk system has to offer is protection of the hinterland. This requires a water defense structure for the exterior water of the Wadden Sea as well as a system to keep the inland water at the desired level.

Excessive water levels in the inland areas (IJsselmeer) may lead to floods in the provinces of North Holland, Friesland, and the IJsselmeer polders. A side effect of inadequate defense against the exterior Wadden Sea water is that salt intrusion prevents the intake of fresh water from the IJsselmeer. This has consequences for fields including agriculture, water distribution, and the ecology.

Figure 5: Capability ‘Protecting of the hinterland’ is provided by these two functions

Defense against exterior water requires dikes that are sufficiently strong and will not collapse under the (water) load, while at the same time they should prevent excessive (exterior) water from washing over the dike. Prevention of erosion both on the inside and the outside of the dike is a prerequisite. These are, in other words, the sub-functions of the function of ‘Exterior water defense’, for which a decomposition was made.

Figure 6: The function ‘Defense Water’ consists of Several Sub-functions

If we limited the system to retaining the water from the Wadden Sea, the IJsselmeer would fill up with fresh water carried in by the River IJssel. A simple solution for high water levels in the IJsselmeer would be to discharge the water into the Wadden Sea. However, the IJsselmeer is also a freshwater basin, which means that the water level must be kept above a certain minimum level. The desired level, therefore, will determine the quantity of water that needs to be discharged. In order to be able to determine this level, the water management department monitors the expected water levels in the river system. This is calculated based on factors including the quantity of river water entering our country from abroad, rainfall forecasts, and the wind. The measurement results are used to calculate a so-called discharge figure, which is communicated to the Afsluitdijk system as a ‘discharge set point’. Based on this figure, water is discharged into the Wadden Sea. Figure 7 below represents the sub-functions of the ‘Maintaining desired interior level’ function. This figure represents the information flow between the various functions.

Figure 7: ‘Maintaining desired interior level’ consists of several sub-functions

Figure 8: Information flow of the ‘Maintaining desired interior level’ function

In order to be able to deliver all of the desired functionality, a system architecture has been developed that contains the required system elements that can provide this functionality (Figure 8).

Figure 9 represents the various types of associations between the typical system elements. For example: one dike protects one or several areas, or the fish pass enables the actor Fish to pass through. This Architecture forms the basis for initiating changes.

The system architecture of a dike can be used as a basis for designing a system structure for the Afsluitdijk. In this process, each system element is an instance of an architectural element, and associations are translated into interfaces.

Examples:

The Afsluitdijk has two Discharge sluices, Lorentz and Stevin, both instanced from the architectural element of the Discharge sluice.

The Afsluitdijk consists of several dike segments, which were instanced from the architectural element of the Dike segment.

Figure 9: System Architecture of a Dike

Figure 10: System Structure of the Afsluitdijk

Conclusion

What makes a system such as the Afsluitdijk a complex system? Besides the many stakeholders with their often very diverse and sometimes even conflicting interests, it is particularly the required adaptivity of the Afsluitdijk system to a changing physical environment that brings complexity to the system. The application of the method as outlined, in which objectives such as ‘G.01.01 Guaranteeing the safety of the hinterland’ are traceable, and capabilities such as ‘Protection of the hinterland’ and ‘Control water level’, and functions such as ‘F.1.1 Retaining exterior water’ and ‘F.1.2 Maintaining desired interior level’, and architecture such as ‘Discharge sluice’ and solutions derived from this such as ‘Lorentz’ and ‘Stevin’ are guaranteed, forms the basis to be able to perform impact analyses. The impact analyses on the basis of changing objectives may lead to modifications that are deemed necessary, while impact analyses for failure of functionality can shed light on which capabilities or even objectives cannot, or not entirely, be guaranteed. The outlined method, therefore, enables control of the outlined complexity today as well as in the future.

An additional complexity is that over the years, the Afsluitdijk has been subjected to several maintenance projects involving renovations and innovations. The complexity is caused by the fact that there is a difference between contract limits and system limits, leading to sub-optimization and complexity at the interface between the system elements. One example of this effect is ship traffic light control based on the position of the gates of the tide lock and the ship lock, which are governed by two different contracts.

References

The Afsluitdijk, https://www.theAfsluitdijk.com

SOM mode, http://somset.nl/over-som/

SE Leidraad, https://www.leidraadse.nl

INCOSE Systems Engineering Handbook: A Guide for System Life Cycle Processes and Activities, 4th Edition, https://connect.incose.org/Pages/Product-Details.aspx?ProductCode=TechSEHandbookDiscountCode

RWS MIRT – https://www.government.nl/topics/spatial-planning-and-infrastructure

RWS internal – Ontwerpkader Keren Hoogwater Afsluitdijk (functionele Analyse) (Design Framework High Tide Defense Afsluitdijk, [functional analysis]), RWS-#2872234-v8

RWS internal – Afsluitdijk is een Icoon (Afsluitdijk is an Icon), Joost van Beek

RWS Internal – Functionele, aspect en raakvlakanalyse (Functional, aspect and interface analysis), RWS-#2749593-v5C-rapport

Editor’s note: The RWS Group provides intellectual property translation, filing and search services, and technical and commercial translation and localization. See https://www.rws.com/

About the Author

C:\Users\Ralph\Documents\Articles\Berber de Liefde-Vogt\th.jpg

Berber de Liefde-Vogt is a member of INCOSE and she is Assistant Director of Communications of the EMEA Sector. She is working as a Systems and Integration Architect at Rijkswaterstaat. Her working fields are model-based systems engineering, technical management and value management as part of asset lifecycle management for the Dutch Water Management, highway infrastructure, and waterway structures. Since August 2017 she is working as Systems and Integration Architect at the project DBFM Afsluitdijk. Berber has obtained a Bachelor of Software Engineering and a Bachelor of Electrical Engineering.

4. Systems Engineering News

4.1 29th Annual INCOSE International Symposium 2019

Call for Submissions

The INCOSE International Symposium is the premier international forum for Systems Engineering. Participants network, share ideas, knowledge and practices, and learn more about the most recent innovations, trends, experiences and issues in Systems Engineering from world-class thought leaders in the field. The theme for the INCOSE International Symposium 2019 is:

System Applications for Global Challenges

INCOSE appreciates all submissions on Systems Engineering, with particular emphasis on:

Developing and Evolving Complex Sociotechnical Systems

Domains include but are not limited to:

  • Automation & Smart Cities
  • Healthcare and Biomedical
  • Aerospace and Defense
  • Sustainable Technologies
  • Cybersecurity
  • Transportation & Automotive
  • Energy & Environment

Emphasis Areas include but are not limited to:

  • Critical Infrastructure
  • System of Systems (SoS)
  • Emerging Technologies
  • System Modernization
  • Engineering Management
  • System Resilience, Reliability, Safety
  • Innovative Approaches: Agile, Iterative and Lean
  • System Security Engineering
  • Internet of Things (IoT)/ Cyberphysical systems
  • System Sustainment

Key dates:

  • Paper, Panel, Tutorial Submission 16th November 2018
  • Notification of Acceptance 15th February 2019
  • Paperless Presentation Abstract Submission 15th February 2019
  • Paperless Presentation Acceptance 15th March 2019
  • Final Paper, Panel, Tutorial Submission 15th March 2019

More information

4.2 The US Department of Defense Announces Digital Engineering Strategy

https://www.israeldefense.co.il/sites/default/files/styles/full_article_image/public/_Uploads/dbsArticles/180410-Z-ZZ999-001_0.JPG?itok=9P2-HO9u

Dr Michael D. Griffin, U.S. DoD Secretary of Defense for Research and Engineering

Image source

Under Secretary of Defense for Research and Engineering Michael Griffin released the Department of Defense Digital Engineering Strategy on July 5.

The strategy promotes the use of digital representations of systems and components and the use of digital artifacts to design and sustain national defense systems.

The department’s five strategic goals for digital engineering are:

  • Formalize the development, integration, and use of models to inform enterprise and program decision making;
  • Provide an enduring, authoritative source of truth;
  • Incorporate technological innovation to improve the engineering practice;
  • Establish a supporting infrastructure and environment to perform activities, collaborate and communicate across stakeholders;
  • Transform the culture and workforce to adopt and support digital engineering across the lifecycle.

The DoD Digital Strategy is available in .pdf here.

4.3 From Grassroots to Classroom in Australia: How one Humble Event Turned into a State-wide Curriculum Resource

The first of its kind program helped teachers deliver STEM-related subjects and also foster an interest in engineering subjects

In what started as a fun way to engage local teachers in teaching engineering to their students in Newcastle has now turned into an accredited curriculum resource for educators across New South Wales (NSW).

Engineers Australia (EA) hosted an ‘Engineering Ingenuity’ workshop in Newcastle as a way for teachers to inspire their students to study STEM-related subjects.

The day was part of Engineering Week in Australia in 2014 and advertised as a way to “help your students achieve what’s needed to embark on a rewarding career in the engineering profession” with discussions and a site tour of electrical engineering firm Ampcontrol.

Engineers Australia Newcastle General Manager Helen Link says after the information day she was bombarded by teachers wanting more resources for their students. It didn’t take long for Ms. Link to realize her team was on to something great. She mobilized her small group of staff to speak to teachers to get to find out what they were missing in their STEM curriculums.

“After the discussions, we hosted a number of focus group meetings which found a real need for connecting our teachers in engineering Studies at our high schools with the engineering profession,” Ms. Link says.

Teachers not only needed help in delivering STEM-related subjects but also fostering an interest in subjects like engineering.

“What we found was that the teachers also needed help in contextualizing engineering theories and practice as a way to better explain the engineering studies syllabus,” Ms. Link says.

“They needed the resources to integrate the science and mathematics disciplines with the purpose of the science and practice of engineering.”

Volunteers and staff worked closely with engineering studies teachers to develop a pilot program where local EA volunteers hosted networking sessions focused on the modules in the current year 11 and 12 syllabus.

The program provided current industry information as well as practical application of aspects of engineering.

“Teachers attend one and a half hour sessions twice per school term, either face-to-face or by webinar, to develop an improved understanding and appreciation of the field of engineering.”

The content has been continuously improved over the years, with the number of schools taking part growing from six to 70 in just under three years.

“This year the course has been endorsed by NSW Education Standards Authority and we will start to deliver the program across NSW,” Ms. Link says.

The number of Year nine engineering studies classes at her school had “recently grown from one to three. The number of female students had grown from 20 per cent to 50 per cent”.

The amount of resources and modules is growing. Right now, teachers can start modules around learning the fundamentals of structures, driverless cars, and even how missiles are launched on Royal Australian Navy frigates.

Feedback from teachers has praised the practical nature of the course.

Teacher Michael Platt from Merewether High said that “being able to relate the content to real world practical situations and experiences (even anecdotal stories about different projects) helps to connect the students to the learning event. Helping us to broaden our experience can only improve the learning event.”

Speaking to the Newcastle Herald, teacher Lu Taylor said the program also “gave teachers who may not have received enough university-level training in engineering studies a deeper understanding of the field, which filtered down to students.”

Peter Cook from Kotara High School also only had good things to say.

“For teachers delivering the course in schools, the information presented is a critical component, and infinitely more useful when the discussion can be accessed later,” he said.

“It represents a wealth of experience and importantly a real-world wider perspective that resonates with students and that we otherwise can’t readily access.”

Article source

4.4 Cadets Lead Workshop in Detroit, Michigan

U.S. Military Academy cadets and staff joined the local area Army Reserve Officer Training Corps cadets to lead the 8th Annual West Point Leadership and Ethics in STEM Workshop on Friday, Oct. 26, at Marygrove College in Detroit, Michigan. The workshop featured a track for students and a track for educators, both focused on leadership, values and ethics, problem-solving, moral-reasoning and decision-making. The goals of the workshop were to provide participants with an opportunity to develop leadership and teambuilding skills; to reinforce the importance of ethical leadership in an increasingly technology-driven world; to promote ethics; to underscore the importance of competent and diverse leadership in the fields of science, technology, engineering and mathematics. Over 250 middle and high school students from the Detroit Metro Schools attended. Students were challenged to lead diverse groups, solve problems, and make values-based decisions in an atmosphere of friendly competition.

4.5 Airbus and Georgia Tech Open the Center for Model-Based Systems Engineering (MBSE)-enabled Overall Aircraft Design (OAD)

The Daniel Guggenheim School of Aerospace Engineering at Georgia Tech last week joined Airbus in officially opening the Airbus / Georgia Tech Center for Model-Based Systems Engineering (MBSE)-enabled Overall Aircraft Design (OAD). This Center will contribute to the development and demonstration of a concurrent overall aircraft design process – taking full advantage of MBSE, interactive, parametric design space exploration and digital enablers with a team of 30 Georgia Tech researchers, doctoral students and Airbus experts.

Joining AE School Chair, Mark Costello at the opening ceremony were Dr. Marc Fischer, senior vice president flight physics at Airbus, and Amanda Simpson, the vice president of research and technology for Airbus Americas.

“We are proud to take the partnership with Georgia Tech to the next level,” said Fischer.

“Our future aircraft developments will benefit from much more integrated and interdisciplinary processes. Model Based Systems Engineering is at the foundation of this ambition and allows us to cover not just the conventional OAD disciplines, but also trade opportunities between engineering and manufacturing. This partnership aims to build a much larger ecosystem within and beyond Airbus, bringing our European and international partners on board.”

“This collaboration between Airbus and our Aerospace Systems Design Lab provides a unique opportunity for our students and faculty to work on research projects that are defining the next generation aircraft,” Costello said.

“Model-based Systems Engineering is a specific strength of the Aerospace Engineering School, and this collaboration with Airbus will enable us to help develop the next generation of aircraft design thinking.”

The collaboration has moved from a framing and ideation phase into project mode – delivering along a multi-year roadmap. Beyond contractual research work in the traditional sense, Airbus and Georgia Tech have engaged in a truly collaborative effort, where industrial experience and state-of-the-art research are combined.

“This Center is further demonstration of the value our American education institutions contribute to our entire global enterprise,” said Simpson.

Article source

5. Featured Organizations

5.1 Engineers Australia (EA)

Engineers Australia is the largest body of engineers in Australia. EA serves and represents around 100,000 professionals at every level, across all fields of practice. EA is committed to advancing engineering and the professional development of its members. Multiple organizations, institutions and government agencies engage with EA to leverage their expertise to create, accredit and assess engineering programs and practitioners.

EA has a rich history and has created awards programs to highlight the dedication and accomplishments of engineers. EA has a diverse group of members and hosts thousands of events each year. The EA website contains information on the organization’s governance, advocacy and regional facets.

More information

5.2 Alpha Pi Mu

Alpha Pi Mu is an industrial engineering honor society that encourages any movement that advances the best interest of industrial engineering education. Alpha Pi Mu recognises industrial engineering students who show exceptional academic interests and abilities in their field. Part of Alpha Pi Mu’s mission is to unify the student body of Industrial Engineering departments and to cooperate with all organizations and persons working in the fields of industrial and systems engineering.

Goals of Alpha Pi Mu include increasing professional development among its members and promoting networking among faculty, graduate and undergraduate students.

More information

5.3 Macau Association of Systems Engineering (MASE)

The Macau Association of Systems Engineering is a non-profit organization supervised and formed by Macau New Technologies Incubation Centre. It is a neutral platform to connect systems engineering and technical units from around the globe to promote collaborations and exchange based on the practice of ‘Probity, Fairness, Objectivity, Practicality, Science and Effectiveness’. MASE aims to introduce advanced system knowledge, techniques and experience to Macau and promotes local research and development of system applications. Objectives of MASE include:

  • Cooperating with systems engineering-related institutions in China, Taiwan, Hong Kong and Macau to provide information about technologies, resources, property and opportunities
  • Pre-analyzing new System Engineering-related products and projects by adhering to the ‘scientific, fair, pragmatic and efficient’ approaches and a ‘coordinating and responsible’ working spirit
  • Analyzing and assessing each project and recommending projects to its members
  • Cooperating with foreign Systems Engineering organizations to provide sustainable training programs
  • Assisting enterprises in Greater China to adopt Systems Engineering techniques and enhance Macau’s technological influence in the region
  • Studying the specifications, standards of practice and code of conduct of Systems Engineering, thereby providing value to the local government

More information

6. News on Software Tools Supporting Systems Engineering

6.1 Cameo Systems Modeler

No Magic Cross-platform Collaborative

Model-Based Systems Engineering (MBSE) Environment

by

Alwyn Smit

Principal Consultant & Course Presenter, PPI

Email: asmit@ppi-int.com

Cameo Systems Modeler provides tools to define, track, and visualise all aspects of systems in SysML models and diagrams. The environment provides an extensive set of features, the following list being a short summary:

  • Requirements Management including traceability, automatic numbering and versioning.
  • Implementing traceability between different levels of abstraction customizable to users’ needs.
  • Reports supporting standard text, RTF, HTML, Spreadsheet template (need to be saved as HTML format), XML template (DocBook or FO), Microsoft Word and Excel 2007 files.
  • Parametric Models and System MoEs can be solved using the built-in math solver as well as interfaces to well-known solvers such as Matlab, Mathematica and OpenModelica.
  • Distributed Use / Parallel Development.
  • Configuration Management with all designs stored in a single place and, module level versioning control and visual model differencing.
  • Security with different people in the project having different access levels to the projects stored in the MagicDraw Teamwork Server repository.
  • Interoperability – No Magic is an official member of the OMG Model Interchange Working Group (MIWG) and UML Diagram Interchange (UMLDI) Group both formed to demonstrate and facilitate interoperability between UML®-based modeling tools. The environment further provides extensive import/export options between No Magic products and other vendors tools.
  • Adjustments / Tailoring is possible, adapting the tool to a domain-specific profile modeling domain, adding Object Constraint Language expressions to any model element, providing a scripting engine to create a custom action for repetitive tasks and the Report Wizard with the customizable WYSIWYG reports.

Much more detailed information is available from the No Magic website.

7. Systems Engineering Publications

7.1 IEEE Access

IEEE Access is an online only, open access journal of the Institute for Electrical and Electronics Engineers (IEEE). Established in May 2013, IEEE Access is a multidisciplinary journal that publishes original articles across all of the IEEE fields of interest. All IEEE Access published articles have a global reach via the IEEE Xplore digital library for free. The Editor-in-Chief is Michael Pecht, Ph.D.

Contents

  1. Peer Review Process
  2. Journal Content
  3. Special Sections
  4. Business Model
  5. Impact Factor
  6. Other Services
  7. References

Peer Review Process

IEEE Access has a rapid, binary peer review publishing system. The approximate time from submission to decision is 4–6 weeks. Articles are reviewed for technical substance and presentation quality by Associate Editors. The IEEE Access editors are responsible for adhering to the publication policies and procedures of IEEE and ensuring that the publication maintains the highest quality by selecting appropriate reviewers for each article in the peer review. After collecting all feedback from the reviewers, the final accept/reject decisions are made by the Associate Editor.

IEEE Access accepts the integration of multimedia within articles. Multimedia (video abstracts, graphics, and video demonstrations) is sent through peer review along with article submissions.

Journal Content

  • Interdisciplinary topics, or applications-oriented articles that do not fit in the scope of traditional journals.
  • Practical discussions of new experimental or measurement techniques, including negative results.
  • Practical articles that describe interesting solutions to engineering or information system design challenges.
  • Development of new or improved fabrication or manufacturing techniques.
  • Reviews of new or evolving fields oriented to assist others in understanding of these emerging topics.

Special Sections

IEEE Access hosts Special Sections that highlight a specific topic of general IEEE interest. Associate Editors propose a concentration area that emphasizes applications-oriented and interdisciplinary topics. The Associate Editor and the IEEE Access editorial staff then send out a “Call for Papers” to academic and industrial researchers to submit articles that identify and discuss technical challenges and recent results in the Special Section. Special Sections have a deadline for submission.

Business Model

IEEE Access is supported by article processing charges (APC) that are paid by the authors. The APC is $1,750 (US) per article.

Impact Factor

IEEE Access has been accepted into the Thomson Reuters Science Citation Index Expanded. Starting with Volume 1 of 2013, IEEE Access will be included in the Web of Science, Journal Citation Report, and Current Contents Engineering, Computing and Technology edition. According to IEEE Xplore, impact factor for IEEE Access in 2016 is 3.244.

Other Services

Authors of published material have the ability to track usage metrics and citations of published articles. IEEE Access also allows readers and authors to comment on articles.

IEEE Access has a LinkedIn page, Facebook page, Twitter, and G+ page where newly published articles, upcoming conferences, popular article highlights, industry news, and engineering facts and tips are announced.

IEEE Access has a YouTube channel where video abstracts, author testimonials, and informational videos are posted.

IEEE TV is an archival online video series where select multimedia for IEEE Access articles and informational videos are listed.

References

IEEE Access Website: https://en.wikipedia.org/wiki/IEEE_Access

7.2 Industrial Dynamics

by

Jay Wright Forrester

Image source

From the Amazon Website:

This is a 2013 Reprint of 1961 First Edition. It is a full facsimile of the original edition, now reproduced with Optical Recognition Software. This work has been cited as one of the most seminal works of the era. Forrester outlines industrial dynamics as an experimental, quantitative philosophy for designing corporate structure and policies that are compatible with an organization’s growth and stability objectives. Forrester believes that management systems possess an orderly and identifiable framework that determines the character of industrial and economic behavior. In this volume, he presents for the first time a methodology for detecting and exhibiting this structure for study.

Format: Hardcover, paperback

Publisher: Martino Fine Books (December 2, 2013)

ISBN-10: 1614275335

ISBN-13: 978-1614275336

More information

7.3 The Systems View of Life

A Unifying Vision

by

Fritjof Capra and Luigi Luisi

Image source

From the Amazon Website

Over the past thirty years, a new systemic conception of life has emerged at the forefront of science. New emphasis has been given to complexity, networks, and patterns of organization, leading to a novel kind of ‘systemic’ thinking. This volume integrates the ideas, models, and theories underlying the systems view of life into a single coherent framework. Taking a broad sweep through history and across scientific disciplines, the authors examine the appearance of key concepts such as autopoiesis, dissipative structures, social networks, and a systemic understanding of evolution. The implications of the systems view of life for health care, management and our global ecological and economic crises are also discussed. Written primarily for undergraduates, it is also essential reading for graduate students and researchers interested in understanding the new systemic conception of life and its implications for a broad range of professions – from economics and politics to medicine, psychology and law.

Format: Hardcover, paperback, eTextbook

Publisher: Cambridge University Press; Reprint edition (September 29, 2016)

ISBN-10: 1316616436

ISBN-13: 978-1316616437

More information

7.4 Focusing Change Management Where It Counts

by

Marge Combe

PMI White Paper, July 2014

Change management is becoming a ubiquitous concept in the management literature, and has a natural home in project management literature. It is reasonably being viewed as a competency worthy of research, study, and mastery as part of leadership development. Change management concepts and practices are also increasingly being highlighted as integral parts of many disciplines, including portfolio, program and project management, as PMI has articulated (PMI, 2013). It is increasingly the subject of academic research and post-graduate degree programs.

And yet, the value of change management thinking and integration may be limited by insufficient attention to a single question: How successful can any change management approach be if the organization is not ready for the change? Examples abound:

Numerous health care organizations in the United States have had to make changes to comply with new health care laws. Though it’s been a challenge for all, for some it has been all but impossible. One health care organization, despite an admirable change management plan, cannot do required reporting because employees are mislabeling coded entries into new systems. Re-training efforts have not eradicated the errors. The problems have been traced to nursing directors who fear being held accountable for misjudging treatment decisions and are insisting on nonspecific treatment codes. Their lack of confidence in their nursing decisions limits the effectiveness of the change.

A corporation opted to expand globally for all the right strategic reasons. It planned the change carefully, with strong management sponsorship and commitment, and a well-vetted selection of partners in the new marketplace. But its decision-making process for getting operations up and running ran into hurdle after hurdle, causing near-doubling of both time and costs to implement. In retrospect, it understood that its inclusive, consensus decision-making model was incompatible with the top-down hierarchical model prevalent in the country culture of their new base of operations. Furthermore, with only lip service paid to inclusivity by its global partner, decisions that were made were often inaccurate, ultimately resulting in its inability to fully realize the benefits of the global alliance.

There are numerous models and processes put forward for managing change. But many of them, even those by highly respected gurus, begin with an assumption that bringing enough sponsorship, vision, communication and resources can overcome any obstacles and make an organization and its people ready to successfully adopt the change. That is likely a faulty assumption, as evidenced by the examples above.

There are, to be fair, models and processes that attempt to assess how ready an organization is for the change. But as Weiner (2009) notes, “Unlike individual readiness for change, organizational readiness for change has not been subject to extensive theoretical development or empirical study.”

In other words, there has not been a lot of substantiated direction on how to judge at a point in time whether an organization is ready for a change. Much of what has been offered is, at best, an overview of areas to consider. A few assessment vehicles have been formulated. They are often of limited focus, most often on employee readiness. The maturity of this type of organizational change readiness assessment is still young and incomplete.

What we do know, however, are the factors that contribute vitally to organizational change success. We know what aids change agility (Combe, 2014a). This area has been better researched and theorized. With this understanding of what makes organizations better able to absorb frequent and complex change, we can begin to identify, when faced with a need for change, a robust means to ask ourselves the critical question, “Are we ready for this change?” The answer to that question informs such decisions as:

Whether to proceed with the change.

How to address risks.

What resources will be most valuable?

Judging readiness for a change is a critical step in the Change Life Cycle Framework presented in PMI’s (2013) Managing Change in Organizations: A Practice Guide. It is shown in Figure 1 below as the second consideration of formulate change in the process of moving a strategic priority to a successfully implemented and well-sustained new operational reality. In this process, the assessment of change readiness is carried out in the management of the program and project portfolio even before the change is fully scoped.

C:\Users\Ralph\Documents\SyEN 2018\SyEN 71 November 2018\Books\P4_1.jpg

Figure 1: Change Life Cycle Framework

Given the critical decisions that rest on the assessment of change readiness, this paper offers guidance to the portfolio, program and project management community for assessing readiness for a specific program or project. In doing so, it offers practical working knowledge for practitioners, and begins to add theoretical options to the scarcity identified by Weiner (2009).

Access the paper

7.5 The U.S. Department of the Navy Systems Engineering Career Competency Model

(SECCM)

The U.S. Deputy Assistant Secretary of the Navy for Research, Development, Test and Evaluation is sponsoring a strategic initiative led by a team from the Naval Postgraduate School Systems Engineering Department to implement an overall approach to study and develop a systems engineering career competency model. The Naval Postgraduate School is working in collaboration with the U.S. Office of Personnel Management, Navy, Army, Air Force, Marine Corps, and the Missile Defense Agency to develop and verify the competencies used by defense systems engineers. Verification of the competency model is critical to allow it to be used as a basis for “high stakes” human resource functions for all of the U.S. Department of Defense. This paper presents the development of the systems engineering career competency model, and the subsequent efforts to have the systems engineering competencies verified.

More information

7.6 Risk Management for Project Driven Organizations:

A Strategic Guide to Portfolio, Program, and Project Management Organization Success

C:\Users\Ralph\Documents\SyEN 2018\SyEN 71 November 2018\Books\51OoxFQ2T9L._SX331_BO1,204,203,200_.jpg

Image source

by

Andy Jordan, PMP

From the Amazon Website:

Organizations invest a lot of time, money, and energy into developing and utilizing risk management practices as part of their project management disciplines. Yet, when you move beyond the project to the program, portfolio, PMO and even organizational level, that same level of risk command and control rarely exists. With this in mind, well-known subject matter expert and author Andy Jordan starts where most leave off. He explores risk management in detail at the portfolio, program, and PMO levels. Using an engaging and easy-to-read writing style, Mr. Jordan takes readers from concepts to a process model, and then to the application of that customizable model in the user’s unique environment, helping dramatically improve their risk command and control at the organizational level. He also provides a detailed discussion of some of the challenges involved in this process. Risk Management for Project Driven Organizations is designed to aid strategic C-level decision makers and those involved in the project, program, portfolio, and PMO levels of an organization.

Format: Kindle, hardcover, paperback

Publisher: J. Ross Publishing (May 1, 2013)

ISBN-10: 1604270853

ISBN-13: 978-1604270853

More information

8. Education and Academia

8.1 Educating the Systems Engineers of the Future

Dr. Cecilia Haskins, ESEP

Norwegian University of Science and Technology

Email: cecilia.haskins@ntnu.no

Oli de Weck wrote in his final editorial for the 20th anniversary edition of Systems Engineering that

“Systems engineering is at the nexus of understanding real problems in society and in markets and clarifying what is needed, the channeling of creativity, the bundling of technical expertise, the expression of architecture, the emergence of inspiring and competitive design, the harmonization of humans and machines, and the sustainable transition to industrialization at scale. These are truly high stakes.[2]

I agree, the stakes are high and the consequences of failure will be felt in society as a whole. The challenge we face is that problem-solving at the societal scale is rarely thought of as ‘engineering’ and hence our discipline is rarely sought as an enabler for finding solutions. To be taken seriously, engineers need to come out of hiding behind the ‘introvert’ stereotype and assume positions of leadership.

Where do we teach systems engineering? Mostly we find our discipline in the engineering departments of major universities, which corresponds to the historical roots of systems engineering, but may exclude the talent we require to address de Weck’s analysis of what is needed.

It need not be so problematic if it were not for the single-minded tendency to fill engineering curriculum exclusively with engineering. The rare engineering student who takes philosophy, management, marketing and accounting is faced with very heavy workloads that may prepare them for the future but be difficult to achieve academically. Compounding this conundrum is the heavy emphasis on courses that teach only ‘hard engineering’ with a heavy reliance on optimization and ‘right’ answers. These students when faced with fuzzy requirements or the need to establish their own requirements are often frustrated, confused, and paralyzed. I speak from experience.

For the past seven years I have been teaching a course entitled Industrial System Design at NTNU with a laboratory component. I should set the stage by explaining that the course attracts Erasmus[3] students who come to Norway from all over the world, bringing diverse academic and cultural backgrounds to the classroom. The laboratory is an exercise in serious play using Lego ™ bricks as the building medium. The students are only given two requirements, work in teams to build an agreed upon product, and do so by exclusively using the bricks, motors, and sensors from the Lego Mindstorms ™ series.

To accomplish this goal, students learn and practice a set of practical methods, such as scenario writing for ConOps, prototyping of preliminary design ideas, establishing requirements and interfaces that help the various teams coordinate their activities, and the importance of communication and frequent testing. The activities are time-boxed within a single semester – approximately 8 labs of 3 hours each – with a video recording made on the final day. Persons interested may view these videos online[4]. While every year the students have succeeded, usually beyond any of my expectations, the path has been difficult for many of them. My one consolation is that I receive mails periodically from past students thanking me for their first introduction to an environment that gave some hints about their eventual real-world job experience.

Donaldson (2017)[5] wrote on the importance of the ‘ologies, and I see a real need for students to apply non-engineering skills to requirements elicitation for their theses and capstone projects. A recent example involved designing an unprecedented virtual, fully digitized, air traffic monitoring and control system. Who are you going to call to describe a job that does not yet exist?

Rouse (2013)[6] wrote eloquently on the importance of bringing systems engineering into healthcare and other public-private systems domains. The healthcare domain encompasses both the monitoring and control equipment for patients, as well as concurrently maintaining privacy and transparency for patient records – neither a trivial task.

Robotics specialist continue research to embed the machine with the human to address lost abilities, and to embed the human into the machine. Where are the systems engineers in these projects?

Piciaccia (2018)[7] delved into the realms of artificial intelligence and machine learning to create a more efficient requirements elicitation method for projects that depend on digesting and extracting system needs from a mountain of standards, regulations, and other documentation. The oil and gas industry is a typical example of this situation. When training professionals in this domain, I often challenge them with the end-of-life scenario for oil well abandonment. This type of problem is the consequence when systems engineering fails to consider that all-important disposal phase of the system. Thankfully, young engineers seem to grasp the importance of this phase very quickly when faced with this scenario.

Returning to my Lego labs – in anticipation of the challenges facing the students upon graduation, recent labs have focused on recycling plants and Greenfield development of self-sufficient urban environments. Tomorrow’s engineers need to be able to understand societal needs for housing, transportation, medical and other services and be able to create innovative solutions that also respect a planet facing resource depletion and other shortages. In addition to Greenfield[8] developments, engineers will also face the conundrum of aging infrastructures, brownfield[9] developments, and resource recovery beyond simply collecting paper, plastics and glass. I probably do not even have a full handle on the scope of these issues, which are important topics within industrial ecology.

Students rise to the challenge of the unfamiliar, as I have experienced in a recent EU-supported Erasmus project entitle European Engineering Teams (EET).[10] The project involved a transnational project-based learning course for students from four European universities. During this time, they tackled problems such as hospital-acquired illnesses, wind turbines as a source of electricity in developing nations, efficient refrigerated transportation, and recyclable pallets.

Concluding Thoughts

I believe it is essential that educators familiarize themselves with the INCOSE SE Vision 2025, loosely based on the 17 sustainable development goals from the UN (see Figure 1). These are the challenges your students must conquer as our generation retires. Some of these challenges may even be consequences of our own making, which is a sad reflection and renders it all the more compelling that we take on the perspective and the challenge of our colleague Oli de Weck to integrate a wider context for systems engineering into our education of systems engineers.

https://i2.wp.com/www.un.org/sustainabledevelopment/wp-content/uploads/2017/12/E_2018_SDG_Poster_without_UN_emblem_Letter-US.png?resize=1024%2C791&ssl=1

Figure 1: The 17 UN Sustainable Development Goals

Image source

About the Author

Cecilia Haskins is an industry Associate Professor of Systems Engineering at the Norwegian University of Science and Technology (NTNU) where she teaches courses in research methods, sustainable development, systems engineering, and project management. Cecilia entered academia after more than thirty years as an SE-professional. Her career spans large and small firms, commercial and government projects, as employee and entrepreneur. Her educational background includes a BSc in Chemistry from Chestnut Hill College, an MBA from Wharton, University of Pennsylvania, and a PhD in systems engineering from NTNU. This combination has contributed to her ability to understand issues with an insider’s view of both the business environments and the technical solution domains. She is recognized as a Certified Systems Engineering Professional (CSEP)[11] since 2004. Her research focuses on improvements to engineering higher education, and innovative applications of systems engineering to socio-technical problems such as those encountered in software development, sustainable development, and global production systems. She is also owner of Global Sustainability Consulting.

8.2 Systems Engineering at Rochester Institute of Technology

Rochester Institute of Technology (RIT) is a private doctoral university within the town of Henrietta in the Rochester, New York metropolitan area. RIT is composed of nine academic colleges, including the National Technical Institute for the Deaf. The Institute is one of a few engineering institutes in the State of New York, including New York Institute of Technology, SUNY Polytechnic Institute, and Rensselaer Polytechnic Institute. It is most widely known for its fine arts, computing, engineering, and imaging science programs; several fine arts programs routinely rank in the national “Top 10” according to US News & World Report.

Edward C. Hensel is the Associate Dean for Research and Graduate Studies.

The RIT Industrial and Systems Engineering (ISE) Department is globally recognized for graduates who are highly sought after due to their ability to solve problems and transform organizations.

Mission

Provide ISE education that integrates experiential learning and applied research, with a student-centered approach, resulting in graduates who make immediate and long-lasting contributions in manufacturing, service, government, and academia.

Vision

The RIT ISE Department is globally recognized for graduates who are highly sought after due to their ability to solve problems and transform organizations.  Our graduates, along with research performed by our students and faculty, positively impact the quality and competitiveness of manufacturing and logistics, the efficacy of health care, and the integration of sustainable practices into many settings.

Values

In addition to the overarching values of the Kate Gleason College of Engineering (KGCOE) and RIT, ISE is founded on the following values:

Student Centered:  The department makes decisions and behaves in a manner that demonstrates the primary importance of their students’ needs and interests.

Community:  The department is a close-knit community characterized by respect for differences, inclusion of a diverse set of ideas and people, and friendly collaboration among the faculty, staff, and students.

Teaching Excellence:  The department demonstrates continuous excellence and innovation in how they deliver classes to their students, and the support they provide to their students outside of class.

Experiential Learning: The department provides experiential learning throughout the undergraduate and graduate curricula via co-op, relevant projects, and practical experiences in their state-of-the-art labs.

Practical Research:  The department’s innovative research makes an impact on the outside world, both directly through its application, and for their students via project opportunities and incorporation into the courses on offer within the department.

Innovation:  The department’s teaching and research are characterized by new ideas and approaches, as well as a willingness to take risks.

A High-Tech Environment for Learning

The department has several specialized labs to aid students in their understanding of the fundamentals covered in lecture. These labs are made available to students in a variety of ways. First, during scheduled labs, the student has access to the lab and equipment with a faculty member present to answer any questions as well as observe demonstrations of assigned lab work. In addition, students have card-swipe access to the laboratories at posted times. The final form of lab access is for students in their final year working on their capstone design project, who have unlimited access to the real-time lab.

C:\Users\Ralph\Documents\SyEN 2018\SyEN 71 November 2018\Academia and Education\toyotalab.jpg

Figure 1: The Toyota Production Systems Lab at Rochester Institute of Technology, founded with support from Toyota Motor Engineering & Manufacturing in 2006

Image source

The Toyota Production Systems Lab, pictured in Figure 1 above, was founded in 2006 with support from the Toyota Motor Engineering & Manufacturing North America. The mission of this facility is to provide hands-on education in state-of-the-art production systems. Particular emphasis is placed on the concepts of teamwork, problem solving, and process improvement by studying the fundamental behavior of production lines. Residing in a 2,520 sq. ft. room in the Kate Gleason College of Engineering, this laboratory currently features two reconfigurable production lines with storage areas and kitting areas, conveyors, and conveyance operations. Over 25 activities have been designed to emphasize specific tools, concepts or techniques in an experiential way: line balancing, kanbans, heijunka, jidoka, takt time, among others.

This facility supports activities at varying levels for many different audiences, including: K-12 students, undergraduate students (both in engineering as well as other disciplines), and graduate students (both in engineering as well as other disciplines). The lab is an integral part of four courses in Industrial Engineering, integrated throughout the curriculum (one required freshman-level undergraduate course, one required senior-level undergraduate course, and two elective graduate courses).

More information

8.3 Postdoctoral Fellowship: The Psychology of Decision Making in Systems Design

At the University of Toronto, a postdoctoral position is available. The successful applicant will join the research group led by Prof. Christoph Becker to explore how decision making actually happens in systems development project teams. The initial appointment will be for one year, with the possibility of extension based on funding and the direction of the research.

The desired start date is between now and January 1st, 2019. Information on how to apply as well as guidelines can be found at the University of Toronto page for Postdoctoral Fellows, here.

Instead of studying how designers and engineers should make decisions according to engineering methods, this project examines their actual practices while developing a psychological understanding of their decision-making and use it to design practical interventions for sustainability design in software systems with a focus on requirements, the key to sustainability. In leading this work, the applicant will establish themselves in the emerging area of sustainability design. Depending on the candidate’s interests, the focus can be on design and human computer interaction, requirements engineering, software engineering, information systems, or other disciplines. The applicant must have completed (or are about to complete) a PhD with relevance to the empirical focus of this project. Relevant fields include (but are not limited to) the collaborative and human aspects of software engineering, studies of requirements engineering or project management practice, design studies, behavioral software engineering, the psychology of judgment and decision making, and others. Knowledge of these areas is much more relevant to this project than classical ‘software engineering’ work such as modelling, the development of tools or modelling languages, method development or quantitative analysis.

Article source

8.4 University Degree on Systems Engineering in Switzerland

Lucerne University of Applied Sciences and Arts is the first institution in Switzerland to offer a Certificate of Advanced Study (CAS) degree on Systems Engineering for Smart Industries. The postgraduate CAS degree further educates engineers so that they can competently define, develop and test complex systems. In addition, the degree prepares engineers to manage increasingly complex, interdisciplinary system tasks typical in technologically-advanced projects.

Overview of the course content:

• Systems & Requirements Engineering Basics
• Model Based Systems Engineering
• Digital Transformation and Systems Engineering
• Project Management & Lean Methods for Systems Engineering
• Intellectual Property Basics
• Quality Assurance and Systems Engineering
• Systems Engineering Project

More information can be found (in German) here.

9. Some Systems Engineering-Relevant Websites

APPEL Knowledge Services, Academy of Project/Program and Engineering Leadership (APPEL), National Aeronautics and Space Administration (NASA)

APPEL Knowledge Services unites the award-winning curriculum and career development tools from the Academy of Program/Project & Engineering Leadership (APPEL) with the critical knowledge sharing and knowledge management capabilities of the Chief Knowledge Officer (CKO) to create a comprehensive, knowledge-dedicated resource for NASA. A key goal of the organization is to better meet the requirements for developing the NASA technical workforce while enhancing the ability to manage and share the different types of knowledge needed for mission success. The extensive online tools and materials from APPEL and CKO are featured on the unified APPEL Knowledge Services website, which combines critical aspects of both original websites in a single location.

https://appel.nasa.gov/

The “How” of Transformation, by McKinsey & Company

A web page dedicated to helping organizations facilitate transformation. The web page explains the ‘what’ and ‘how’ of the McKinsey transformation approach and provides tips for leading companies out of a crisis as well as how to keep transformations on track. A valuable read for all companies seeking to execute a transformation process successfully by developing suitable ‘performance infrastructure’.

https://www.mckinsey.com/industries/retail/our-insights/the-how-of-transformation

Infrastructure and Projects Authority: assurance review toolkit

This web page by The Infrastructure and Projects Authority (IPA) of the UK Government’s contains a collection of general guidance on reviews such as project validation review, strategic assessments, business justification reviews and delivery strategies for projects. As the IPA arranges and manages more than 200 independent assurance reviews of major government projects each year, the documents are useful in doing IPA reviews, and non-IPA reviews such as government or medium risk reviews.

https://www.gov.uk/government/collections/infrastructure-and-projects-authority-assurance-review-toolkit

10. Standards and Guides

10.1 Best Practices for Using Systems Engineering Standards (ISO/IEC/IEEE 15288, IEEE 15288.1, and IEEE 15288.2) on Contracts for U.S. Department of Defense Acquisition Programs

April 2017

Prepared by:

Office of the Deputy Assistant Secretary of Defense for Systems Engineering

Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics

Background

The Department of Defense (DoD) and the defense industry have found that applying systems engineering (SE) processes and practices throughout the system life cycle improves project performance, as measured by the project’s ability to satisfy technical requirements within cost and schedule constraints. Simply put, projects that use effective SE processes perform better than those that do not. Given this knowledge, it is in the best interest of both acquirers and suppliers to ensure that defense acquisition projects use effective SE processes as the core of the technical management effort.

The purpose of this document is to assist:

  • Acquirers in tailoring the 15288 Standards to meet and communicate project needs.
  • Acquirers in incorporating appropriate language into a Request for Proposal (RFP) to invoke the standards and express relative importance of the standards in proposal evaluations.
  • Offerors in developing their proposals to leverage existing organizational processes, or propose alternative value-added tailoring, to support the RFP requirements and comply with the standards as tailored.
  • Acquirers in evaluating an offeror’s ability and commitment to effectively implement SE processes compliant with acquirer’s requirements based on the proposed Systems Engineering Management Plan (SEMP), project plan, master schedule, and past performance.
  • Acquirers in monitoring and enforcing a supplier’s compliance with the contract and delivery of the product/service/system.

Contents

Introduction

Background

Purpose

15288 Standards Overview

Overview

Systems Engineering Planning Prior to Request for Proposal

Planning for Use of Systems Engineering Standards

Use of 15288 and 15288.1 on Contract

Use of 15288.2 on Contract

Tailoring Considerations

Request for Proposal and Source Selection

Development of the Request for Proposal

Suggested Request for Proposal Language

Offeror Response to Request for Proposal

Proposal Evaluation and Source Selection

Contract Execution

Process Compliance

Output Compliance

Technical Review Compliance

Appendix A – How Project Characteristics May Drive Tailoring of the 15288 Standards

Appendix B – Work Aid for Definition of Outputs to Be Supplied

Appendix C – Example Tailoring Document Template

Appendix D – Definitions and Acronyms

Definitions

Acronyms

Appendix E – References

Table 1: Systems Engineering Life Cycle Processes

Source: ISO/IEC/IEEE 15288, “Systems and Software Engineering–System Life Cycle Processes”

11. A definition to close on

11.1 Introduction to System Dynamics

Overview

System Dynamics is a computer-aided approach to policy analysis and design.  It applies to dynamic problems arising in complex social, managerial, economic, or ecological systems — literally any dynamic systems characterized by interdependence, mutual interaction, information feedback, and circular causality.

The field developed initially from the work of Jay W. Forrester.  His seminal book Industrial Dynamics (Forrester, 1961) is still a significant statement of philosophy and methodology in the field.  Within ten years of its publication, the span of applications grew from corporate and industrial problems to include the management of research and development, urban stagnation and decay, commodity cycles, and the dynamics of growth in a finite world.   It is now applied in economics, public policy, environmental studies, defense, theory-building in social science, and other areas, as well as its home field, management.  The name industrial dynamics no longer does justice to the breadth of the field, so it has become generalized to System Dynamics. The modern name suggests links to other systems methodologies, but the links are weak and misleading.  System Dynamics emerges out of servomechanisms engineering, not general systems theory or cybernetics (Richardson 1991).

The System Dynamics Approach

The System Dynamics approach is founded on the scientific method.  The goal of a system dynamics project is sometimes to build theoretical understanding, sometimes to implement policies for improvement, and often both.  To do so, system dynamics modelers seek to: include a broad model boundary that captures important feedbacks relevant to the problem to be addressed; represent important structures in the system including accumulations and state variables, delays and nonlinearities; use behavioral decision rules for the actors and agents in the system that are grounded in first-hand study; and use the widest range of empirical data to formulate the model, estimate parameters, and build confidence in the conclusions.

Although the points below are presented as a list, modeling (and any scientific activity) is iterative – a continual process of formulating hypotheses, testing against data of all types, and revision of both formal and mental models.

The approach can be summarized as:

  • Beginning with a problem to focus systems thinking and modeling, involving the stakeholders whose understanding and action is required to implement change.
  • Defining problems dynamically, in terms of graphs over time (time series), employing actual data wherever possible.
  • Striving for an endogenous, behavioral view of the significant dynamics of a system, a focus inward on the structures and decision rules in a system that themselves generate or exacerbate the perceived problem.
  • Thinking of all concepts in the real system as quantities interconnected in loops of information feedback and circular causality, a consequence of the endogenous point of view.
  • Identifying the key variables essential to address the problem and deciding on an appropriate level of aggregation for them.  System dynamics models range from highly disaggregate representations such as individual items or agents to highly aggregated representations, and can be deterministic or stochastic, as needed to address the purpose of the study.
  • Formulating a richly explanatory behavioral model capable of reproducing, by itself, the dynamic problem of concern, drawing on all relevant evidence, including qualitative and quantitative data.  The model is usually a computer simulation model, but is occasionally left unquantified as a map capturing the important accumulations (stocks) in the system, the flows that alter them, and the causal feedback structure determining the flows.
  • Testing the structure and behavior of the model against all relevant evidence to deepen understanding and to build confidence in it, including the model’s ability to replicate historical data, ensuring the model is robust under extreme conditions, exploring the sensitivity of results to uncertainty in assumptions, and diagnosing the sources of unexpected model behavior.
  • Designing and testing policies to address the problem of concern, testing these against data and comparing to real-world policies that have been tried in the system or similar settings.
  • Documenting the model and its supporting sources so that it is as transparent as possible and enabling others to critique, use, and extend the work.
  • Working with stakeholders and others to help translate model-based insights into implementable policies, assist in implementation, assess the results, and improve both the model and policies.

Experiments conducted in the virtual world of the model inform the design and implementation of experiments in the real world. That experience then leads to changes and improvements in the virtual world, in participants’ mental models, and in actions taken in the real world, in an iterative process of continuous improvement.

Feedback Thinking

Conceptually, the feedback concept is at the heart of the System Dynamics approach.  Diagrams of loops of information feedback and circular causality are tools for conceptualizing the structure of a complex system and for communicating model-based insights.  Intuitively, a feedback loop exists when information resulting from some action travels through a system and eventually returns in some form to its point of origin, potentially influencing future action.  If the tendency in the loop is to reinforce the initial action, the loop is called a positive or reinforcing feedback loop; if the tendency is to oppose the initial action, the loop is called a negative or balancing feedback loop.  The sign of the loop is called its polarity. Balancing loops can be variously characterized as goal-seeking, equilibrating, or stabilizing processes.  They can sometimes generate oscillations, as when a pendulum seeking its equilibrium goal gathers momentum and overshoots it.  Reinforcing loops are sources of growth or accelerating collapse; they are disequilibrating and destabilizing.  Combined, reinforcing and balancing circular causal feedback processes can generate all manner of dynamic patterns.

Loop Dominance and Nonlinearity

The loop concept underlying feedback and circular causality by itself is not enough, however.  The explanatory power and insightfulness of feedback understandings also rest on the notions of active structure and loop dominance.  Complex systems change over time.  A crucial requirement for a powerful view of a dynamic system is the ability of a mental or formal model to change the strengths of influences as conditions change, that is to say, the ability to shift active or dominant structure.

In a system of equations, this ability to shift loop dominance comes about endogenously from nonlinearities in the system.  For example, the S-shaped dynamic behavior of the classic logistic growth model (dP/dt = aP – bP2) can be seen as the consequence of a shift in loop dominance from a positive, self-reinforcing feedback loop (aP) producing exponential-like growth to a negative balancing feedback loop (-bP2) that brings the system to its eventual goal.  Only nonlinear models can endogenously alter their active or dominant structure and shift loop dominance.  From a feedback perspective, the ability of nonlinearities to generate shifts in loop dominance and capture the shifting nature of reality is the fundamental reason for advocating nonlinear models of social system behavior.

The Endogenous Point of View

The concept of endogenous change is fundamental to the System Dynamics approach. It dictates aspects of model formulation:  exogenous disturbances are seen at most as triggers of system behavior (like displacing a pendulum); the causes are contained within the structure of the system itself (like the interaction of a pendulum’s position and momentum that produces oscillations).  Corrective responses are also not modeled as functions of time, but are dependent on conditions within the system.  Time by itself is not seen as a cause.

But more importantly, theory building and policy analysis are significantly affected by this endogenous perspective. Taking an endogenous view exposes the natural compensating tendencies in social systems that conspire to defeat many policy initiatives.  Feedback and circular causality are delayed, devious, and deceptive.  For understanding, System Dynamics practitioners strive for an endogenous point of view.  The effort is to uncover the sources of system behavior that exist within the structure of the system itself.

System structure

These ideas are captured in Forrester’s (1969) organizing framework for system structure:

  • Closed boundary
  • Feedback loops
  • Levels
  • Rates
  • Goal
  • Observed condition
  • Discrepancy
  • Desired action

The closed boundary signals the endogenous point of view.  The word closed here does not refer to open and closed systems in the general system sense, but rather refers to the effort to view a system as causally closed.  The modeler’s goal is to assemble a formal structure that can, by itself, without exogenous explanations, reproduce the essential characteristics of a dynamic problem.

The causally closed system boundary at the head of this organizing framework identifies the endogenous point of view as the feedback view pressed to an extreme.  Feedback thinking can be seen as a consequence of the effort to capture dynamics within a closed causal boundary.  Without causal loops, all variables must trace the sources of their variation ultimately outside a system.  Assuming instead that the causes of all significant behavior in the system are contained within some closed causal boundary forces causal influences to feed back upon themselves, forming causal loops.  Feedback loops enable the endogenous point of view and give it structure.

Levels and Rates

Stocks (levels) and the flows (rates) that affect them are essential components of system structure.  A map of causal influences and feedback loops is not enough to determine the dynamic behavior of a system.  A constant inflow yields a linearly rising stock; a linearly rising inflow yields a stock rising along a parabolic path, and so on.   Stocks (accumulations, state variables) are the memory of a dynamic system and are the sources of its disequilibrium and dynamic behavior.

Forrester (1961) placed the operating policies of a system among its rates (flows), many of which assume the classic structure of a balancing feedback loop striving to take action to reduce the discrepancy between the observed condition of the system and a goal.  The simplest such rate structure results in an equation of the form NETFLOW = (GOAL – STOCK)/(ADJTIM), where ADJTIM is the time over which the level adjusts to reach the goal.

Behavior is a Consequence of System Structure

The importance of levels and rates appears most clearly when one takes a continuous view of structure and dynamics.  Although a discrete view, focusing on separate events and decisions, is entirely compatible with an endogenous feedback perspective, the System Dynamics approach emphasizes a continuous view.  The continuous view strives to look beyond events to see the dynamic patterns underlying them.  Moreover, the continuous view focuses not on discrete decisions but on the policy structure underlying decisions.  Events and decisions are seen as surface phenomena that ride on an underlying tide of system structure and behavior.  It is that underlying tide of policy structure and continuous behavior that is the system dynamicist’s focus.

There is thus a distancing inherent in the System Dynamics approach — not so close as to be confused by discrete decisions and myriad operational details, but not so far away as to miss the critical elements of policy structure and behavior.  Events are deliberately blurred into dynamic behavior.  Decisions are deliberately blurred into perceived policy structures.  Insights into the connections between system structure and dynamic behavior, which are the goal of the System Dynamics approach, come from this particular distance of perspective.

Suggestions for Further Reading

The System Dynamics Review, the journal of the System Dynamics Society, is the best source of current activity in the field, including methodological advances and applications.  Other important journal sources include Management Science, the European Journal of Operational Research (EJOR), the Journal of the Operational Research Society (JORS), and Systems Research and Behavioral Science.  For texts on the modeling process in System Dynamics, see Sterman (2000), Maani and Cavana (2007), Ford (2009), Morecroft, (2007), Wolstenholme (1990), and Richardson and Pugh (1981).

An early, interesting collection of applications is Roberts (1978); Richardson (1996) is a more recent two-volume edited collection in the same spirit, containing prize-winning work in philosophical background, dynamic decision making, applications in the private and public sectors, and techniques for modeling with management.

A current direction within the field is the use of model-based insights for organizational learning, represented most forcefully in Senge (1990) and Morecroft and Sterman (1994).  The important new effort to build models with relatively large groups of experts and stakeholders, known as group model building, is described in Vennix (1996) and Richardson and Andersen (2010).

Richardson (1991/1999) puts the endogenous feedback perspective of the System Dynamics approach in its historical context and includes an extensive bibliography.

References

Ford, A. 2009. Modeling the Environment. Washington, DC: Island Press.

Forrester, J.W. 1961.  Industrial Dynamics. Cambridge, MA: The MIT Press.  Reprinted by Pegasus
Communications, Waltham, MA.

Forrester, J.W. 1969.  Urban Dynamics. Cambridge, MA: The MIT Press.  Reprinted by Pegasus Communications, Waltham, MA.

Maani, K. E. and R. Y. Cavana. 2007.  Systems Thinking, System Dynamics: Understanding Change and Complexity. Aukland: Prentice Hall.

Morecroft, J. D. W. 2007.  Strategic Modeling and Business Dynamics: a Feedback Systems Approach. Chichester: Wiley.

Morecroft, J. D. W. and J. D. Sterman, Eds. 1994. Modeling for Learning Organizations. System Dynamics Series. Cambridge, MA:  Pegasus Communications.

Richardson, G.P.  1991/1999.  Feedback Thought in Social Science and Systems Theory. Philadelphia: University of Pennsylvania Press; reprinted by Pegasus Communications, Waltham, MA.

Richardson, G.P., Ed. 1996.  Modelling for Management:  Simulation in Support of Systems Thinking.  International Library of Management.  Aldershot, UK:  Dartmouth Publishing Company.

Richardson, G.P. and D. F. Andersen. 2010. “Systems Thinking, Mapping, and Modeling for Group Decision and Negotiation.” Handbook for Group Decision and Negotiation, C. Eden and D. N. Kilgour, Eds.  Dordrecht: Springer, 2010, pp. 313-324.

Richardson, G.P. and A.L. Pugh III. 1981. Introduction to System Dynamics Modeling with DYNAMO. Cambridge, MA: The MIT Press.  Reprinted by Pegasus Communications, Waltham, MA.

Roberts, E.B. 1978, Ed.  Managerial Applications of System Dynamics. Cambridge, MA: The MIT Press.  Reprinted by Pegasus Communications, Waltham, MA.

Senge, P.M.  The Fifth Discipline:  The Art and Practice of the Learning Organization. New York:
Doubleday/Currency.

Sterman, J.D. 2000.  Business Dynamics: Systems Thinking and Modeling for a Complex World.  Boston: Irwin McGraw-Hill.

System Dynamics Review. 1985-present.  Chichester, U.K.:  Wiley, Ltd.

Vennix, J. A. M. 1996. Group Model Building: Facilitating Team Learning Using System Dynamics. Chichester: Wiley.

Wolstenholme, E.F. 1990.  System Enquiry:  a System Dynamics Approach.  Chichester, U.K.:  John Wiley & Sons, Ltd.

12. Conferences and Meetings

For more information on systems engineering related conferences and meetings, please proceed to our website.

The featured conference for this month is:

CIISE 2018 – Conferenza INCOSE Italia su Systems Engineering

28-30 Novembre 2018, Roma (Italia)

The INCOSE Italia Conference on Systems Engineering (CIISE 2018) aims to provide researchers, practitioners and Italian organizations, involved in the Systems Engineering field, with an opportunity of exchanging and discussing their experiences and thus facilitating possible future collaborations and synergies.

More information can be found on the conference website here.

13. PPI and CTI News

13.1 PPI Adds Five More Locations to its Worldwide Footprint

PPI may be a small company, but our impact in improving enterprises worldwide, and the quality of life of their engineering workforce, is large. Clients have supported PPI with three new locations over the last two months: Aberdeen, MD and Portland, OR in the United States, and Krakow, Poland, with Doha in Qatar to be added next week and Jeddah in Saudi Arabia to be added in January. The map shown in Figure 1 below highlights the locations where PPI courses have been delivered.

Figure 1: World map showing locations where PPI courses have been delivered

13.2 PPI Participates in EnergyTech 2018

PPI team members, Robert Halligan (managing director) and René King (senior engineer) enjoyed a wonderful week in Cleveland, Ohio at the EnergyTech conference from 22-25 October 2018. The conference kicked off with a PPI-presented Fundamentals in Systems Engineering Workshop held on Monday followed by participation by Robert Halligan in a panel on Advancing the Systems Engineering Practice on Wednesday and a panel and presentation on Integrating Project Management and Systems Engineering on Thursday. The conference brought a wonderful opportunity for PPI to connect with great thinkers and learn from the most experienced and knowledgeable in the infrastructure, energy and technology industries, whilst sharing PPI’s systems engineering expertise. Notable topics in the conference included: cyber security in infrastructure, applying systems engineering to megaprojects and the move towards digital engineering. PPI looks forward eagerly to EnergyTech 2019 which will be held in Cleveland,Ohio. The organizers guarantee it to be bigger and better than ever!

Left to right: John Johasz (EnergyTech conference chair), Robert Halligan and René King

13.3 PPI Gears Up for First Delivery of Medical Device Risk Management (MDRM) Course

Next week is an exciting time for PPI as risk management and medical devices expert, Bijan Elahi, presents the first ever public PPI course on this prominent topic in the medical industry. Bijan is the winner of the Educator of the Year Award from the International System Safety Society and is passionate about elevating global knowledge and proficiency in medical device risk management for the benefit of companies and society.

The three-day course will take place in Eindhoven in the Netherlands from the 20-22 November and will cover a comprehensive range of topics including the systematic analysis, estimation, evaluation and control of safety risks associated with medical devices. The content of the course includes management of risk associated with product development as well as post-market risk. The course will equip participants with tools needed for successful management of risk in order to predict and prevent serious harm to patients and loss of business. For more information, please see the MDRM course on the PPI website, here.

13.4 Bijan Elahi Presents Webinar on Medical Device Risk Management

https://www.ppi-int.com/wp-content/uploads/2018/09/Bijan-Elahi-Circle.jpg

PPI’s Bijan Elahi MSEE, BS AeroE presented a webinar, “Insights into Risk Management for Medical Devices” to members of INCOSE’s Biomedical Working Group on Tuesday, 14 November 2018. The webinar will be available on the Group’s website, and subsequently on YouTube.

Synopsys: “The word “risk” has many meanings, depending on the discipline within which it is applied.  For Medical Devices, risk is about the probability of causing harm to humans.  ISO 14971 is the central standard that governs risk management for medical devices.  This standard was first released in 2000 and offers a framework for the management of safety risks of medical devices.  In this presentation, we will talk about the requirements of ISO 14971, why we do risk management, and what the benefits and main challenges of performing risk management are.”

This webinar is potentially available to other societies having a direct interest in the development of medical devices – please contact PPI to engage in discussion if you are interested.

13.5 Michael Gainford Returns to the PPI Team

https://www.ppi-int.com/wp-content/uploads/2018/10/MG-Circle-150x150.png

We are thrilled to announce that Michael Gainford has returned to the PPI/CTI family, after some time spent pursuing other interests. Michael will be presenting CTI CSEP training in Madrid next month. We look forward to seeing the energy, enthusiasm and professionalism of Michael again benefit the clients of PPI and CTI.

13.6 PPI and CTI Leaders – the Faces Behind the Names

Knowing the face behind the name is always nice. Now you can see the faces behind the PPI/CTI names at https://www.ppi-int.com/about-ppi/our-people/, at least for our frontline people.

13.7 PPI is a Proud Sponsor of CIISE 2018

PPI is pleased to be a sponsor of the upcoming INCOSE Italia Conference on Systems Engineering (CIISE 2018) taking place 28-30 November 2018. The conference is dedicated to bringing researchers, organizations and practitioners of Systems Engineering together to promote collaboration and synergistic activity within the field. This conference is also the feature conference of the month (see Section 12: Conferences and Meetings).

14. PPI and CTI Events

On-site systems engineering training is being delivered worldwide throughout the year. An overview of public courses is below. For a full public training course schedule, please visit https://www.ppi-int.com/course-schedule/

Systems Engineering 5-Day Courses

Upcoming locations include:

  • Adelaide, Australia (P006-749)

19 Nov – 23 Nov 2018

  • Eindhoven, the Netherlands (P006-750)

26 Nov – 30 Nov 2018

Requirements Analysis and Specification Writing 5-Day Courses

Upcoming locations include:

  • Pretoria, South Africa (P007-474)

28 Jan – 01 Feb 2019

Systems Engineering Management 5-Day Courses

Upcoming locations include:

  • London, United Kingdom (P1135-155)

10 Dec – 14 Dec 2018

  • Ankara, Turkey (P1135-158)

07 Jan – 11 Jan 2019

Requirements, OCD and CONOPS in Military Capability Development 5-Day Courses

Upcoming locations include:

  • Amsterdam, the Netherlands (P958-56)

03 Dec – 07 Dec 2018

  • Melbourne, Australia (P958-57)

18 Feb – 22 Feb 2019

       Washington, D.C., United States of America (P958-59)

13 May – 17 May 2019

Architectural Design 5-Day Course

Upcoming locations include:

  • Pretoria, South Africa (P1768-19)

06 May – 10 May 2019

CSEP Preparation 5-Day Courses (Presented by Certification Training International, a PPI company)

Upcoming locations include:

      San Francisco, CA, United States of America (C002-81)

11 Feb – 15 Feb 2019

      Bristol, United Kingdom (C002-91)

04 Mar – 08 Mar 2019

Medical Device Risk Management 3-Day Course (Presented by Bijan Elahi)

Upcoming locations include:

  • Eindhoven, Netherlands

20 Nov – 22 Nov 2018 (P1848-1)

Other training courses available on-site only include:

  • Project Risk and Opportunity Management 3-Day
  • Managing Technical Projects 2-Day
  • Integrated Product Teams 2-Day
  • Software Engineering 5-Day.

15. Upcoming PPI Participation in Professional Conferences

PPI will be participating in the following upcoming events. We support the events that we are sponsoring and look forward to meeting old friends and making new friends at the events at which we will be exhibiting.

INCOSE UK Annual Systems Engineering Conference (ASEC 2018)

(Exhibiting)

Date: 20-21 November, 2018

Location: Bedfordshire, United Kingdom

INCOSE Italia Conference on Systems Engineering (CIISE 2018)

(Sponsoring)

Date: 28-30 November, 2018

Location: Rome, Italy

The INCOSE International Symposium 2019

(Exhibiting)

Date: 20-25 July, 2019

Location: Orlando, USA

EnergyTech Conference 2019

(Exhibiting)

Date: 21-25 October, 2019

Location: Cleveland, USA

The INCOSE International Symposium 2020

(Exhibiting)

Date: 18-23 July, 2020

Location: Cape Town, South Africa

Kind regards from the PPI SyEN team:

Robert Halligan, Editor-in-Chief, email: rhalligan@ppi-int.com

Ralph Young, Editor, email: ryoung@ppi-int.com

René King, Managing Editor, email: rking@ppi-int.com

Project Performance International

2 Parkgate Drive, Ringwood, Vic 3134 Australia

Tel: +61 3 9876 7345

Fax: +61 3 9876 2664

Tel Brasil: +55 12 9 9780 3490

Tel UK: +44 20 3608 6754

Tel USA: +1 888 772 5174

Tel China: +86 188 5117 2867

Web: www.ppi-int.com

Email: contact@ppi-int.com

Copyright 2012-2018 Project Performance (Australia) Pty Ltd, trading as
Project Performance International.

Tell us what you think of PPI SyEN. Email us at syen@ppi-int.info.

  1. IJssel River, river of the Netherlands, an important distributary of the Rhine River. It leaves the Lower Rhine (Neder Rijn) just southeast of Arnhem and flows northeastward for 70 miles (113 km) to enter the IJsselmeer (a lake formed from the old Zuiderzee) between the Northeast (Noordoost) and East Flevoland polders. Source: Britannica.com.
  2. de Weck, O. L., 2018. Systems engineering 20th anniversary special issue. https://doi.org/10.1002/sys.21443
  3. See http://erasmusprogramme.com/ for information concerning this student program
  4. https://mediasite.ntnu.no/Mediasite/Catalog/catalogs/tpk4185
  5. Donaldson, W. 2017. In Praise of the “Ologies”: A Discussion of and Framework for Using Soft Skills to Sense and Influence Emergent Behaviors in Sociotechnical Systems. https://doi.org/10.1002/sys.21408
  6. Rouse, W. B., 2013. Coping With Complexity – Understanding and managing our complex public-private systems. TechNews, www.njtc.org, July 2013
  7. Piciaccia, L. A., 2018. Systems Engineering for the Subsea Oil & Gas Industry – requirements elicitation through semantically aware technologies – a quantitative assessment. NTNU Thesis 2018:278.
  8. Greenfield: land (such as a potential industrial site) not previously developed or polluted (merriam-webster.com)
  9. Brownfield: a tract of land that has been developed for industrial purposes, polluted, and then abandoned (merriam-webster.com)
  10. See http://engineering-team.net/about-eet/ for more information.
  11. https://www.incose.org/systems-engineering-certification/certification
Scroll to Top