- In this perspective paper, we investigate the parallels between public policy and IT projects from the perspective of traditional RE practice. Using the mainstream media as an information source (as would an average citizen), over a period of approximately one year we captured documents that presented analyses of public policy issues. The documents were categorized into eight topic areas, then analyzed to identify patterns that RE practitioners would recognize. We found evidence of policy failures that parallel project failures traceable to requirements engineering problems. Our analysis revealed evidence of bias across all stakeholder groups, similar to the rise of the “beliefs over facts” phenomenon often associated with “fake news”. We also found substantial evidence of unintended consequences due to inadequate problem scoping, terminology definition, domain knowledge, and stakeholder identification and engagement. Further, ideological motivations were found to affect constraint definitions resulting in solution spaces that may approach locally optimal but may not be globally optimal. Public policy addresses societal issues; our analysis supports our conclusion that RE techniques could be utilized to support policy creation and implementation. (Abstract) Index Terms-Requirements engineering, public policy, bias, unintended consequences, mainstream media, ideology and belief, failure. (key words)
We believe that there is a strong parallel between crafting public policy in response to (societal needs to meet) citizen’s goals and software crafted (in response to requirements) to meet stakeholder goals. In this context, we define public policy as the mechanism through which societal challenges are identified and addressed by the creation of policies, laws and regulations as enacted by government. We see sustainability as a significant societal challenge that could be addressed by effective policy creation and implementation.
Requirements Engineering (RE) practices such as goal
identification and modeling, requirements analysis,
requirements negotiation, prioritization and triage have direct
correspondence with the political process of policy identification,
policy creation and with resolving challenges associated with
realizing policy goals [16]. What is not as clear is the
correspondence between RE practices associated with identifying
risks, threats and unintended consequences [
Given the perceived correspondence between the domains, we decided to investigate further. However, we are not public policy experts and we chose to investigate the issues just as members of the public would do, using the information source most readily available – the Main Stream Media (MSM), rather than using the (traditional) peer-reviewed literature. In other words, we wanted to know whether public policy initiatives that received MSM coverage appeared to have any characteristics revealed in their reporting that confirmed the analogy with RE for software artifacts. We observed evidence of bias in the reported positions, bias in those doing the reporting and even evidence of “fake news” effects.
Our initial investigations led to the following research questions: 1. Can we identify challenges associated with defining, formulating and realizing public policies? 1.1. Do the challenges have analogs in RE for software intensive systems? 2. How could RE techniques help mitigate the identified public policy challenges? 2.1. Can RE techniques be used to proactively identify possible public policy challenges during formulation and before enactment?
To answer these questions, we performed an explorative case study using North American mainstream media and categorized the motivating problem, goals and solutions for eight topics that received significant MSM coverage over the study period. The study materials were gathered by monitoring news feeds (e.g. Google News) for a period of approximately one year and capturing those documents that presented a public policy issue along with analysis or commentary. We reviewed the documents en masse, then categorized and coded them.
Our analysis revealed evidence of (apparently unintentional and often large-scale) side effects. These unintended artifacts appear to exhibit many of the classic RE problems that occur during the development of software-intensive systems.
The rest of this paper is organized as follows. In Section 2 we review prior and related work. Section 3 presents the research methodology, research design and discusses threats to validity. Section 4 describes the data collection and analysis efforts and Section 5 presents our observations. A supplementary discussion follows in Section 6 and Section 7 presents the conclusions and directions for future work.
We present related work from the topic areas of ideological biases in stakeholders, mainstream media as information source in RE, challenges of data mining versus humanism, and problem analysis in other domains using RE tools.
The works on ideological biases in stakeholders are
principally in the area of business policy. For example, in 1986,
Shrivastava [
Handelmann et al. [
Entine [16] critiques the myth of social investing based on
an analysis of the flaws of proclaimed objective ratings and of
‘socially responsible’ businesses and their strategies. He
concludes that social screening is highly anachronistic and based
on ideologically constructed notions of corporate social
responsibility. Taking a stance against Entine’s analysis, Waddock
[
Chomsky [14] discusses what makes mainstream media “mainstream”. He elaborates that most of mass media is intended to divert attention (consumers as spectators), the elite media is geared towards the educated, wealthy and powerful, and most academic articles are still within the boundaries of institutional obedience. He concludes that, from these characteristics, we can predict what we would expect to find in the current work – and we did.
Kwak et al. [
Newman [
Wright and Hinson [
Maalej [
Guzman and Maalei also investigated Twitter messages to understand their potential to help requirements engineers better understand user needs, using the micro-blogging system as an additional information source for RE. In contrast, our research uses RE analysis to understand parallels between RE for software intensive systems and crafting public policy.
Manovich [
Kirschenbaum [
Sculley and Pasanek [
Chandrasekaran [
The requirements engineering community has made
significant contributions in the area of legislative work, for
traceability and analysis [
Due to space restrictions, there are large areas of work within RE which this work has not referenced.
We conducted an exploratory case study over a period of approximately one year during which we investigated public policy topics where there was significant Main Stream Media (MSM) press coverage. The MSM was used as an information source, rather than the academic literature, because we were focused upon public policy and the MSM is the principal information source for members of the general public.
The MSM was monitored using news feeds such as Google News (https://news.google.com), content aggregators that can be trained (via click through on articles) to perform some degree of filtering upon the vast quantity of available material. Whenever we identified an article related in some way to announced public policy and the author’s commentary identified inadequate results or unintended consequences, we then captured that article to the document repository for later analysis.
The resulting dataset is a collection of 152 articles or documents on government policies, policy topics or policy initiatives, government procurement and policy implementation strategies. Sustainability was the primary focus of 37 of the articles or documents. In each case, the topic of the article was an initiative that was (seemingly) made with the best of intentions. Unfortunately, the results ranged from simply inadequate to outright failure and the incidence of (potentially large-scale) unintended consequences was high. We include in the category of unintended consequences, policies that even a superficial RE analysis would identify as probably not achievable given the solution constraints. The unintended consequences were either explicitly identified by the author of the article or they were identified after our own analytic efforts (e.g. diverging or contradicting policy goals) or prior domain experience.
As a counterpoint to the MSM sources, we also
investigated sustainability policies in California, USA [
At the end of the document collection phase, the documents were reviewed in their entirety in two sessions totaling approximately 12 hours. We used researcher triangulation to decrease the subjectivity bias, with the first two authors performing the analysis in discourse and the third author reviewing the coding and interpretation for consistency and correctness. The coding was emergent and led to the following eight categories. Given the topic areas, there is some potential that a document could be coded into more than one category; the final placement was based on discussion among the researchers.
• Algorithms (e.g. big data analysis, artificial intelligence) that have drawn sufficient attention to warrant public policy discussions • IT projects (e.g. large-scale publicly funded projects, generally in support of some policy goal) • Social (e.g. free speech, critical thinking, gender issues, fake news, radicalism) • Privacy (e.g. location data, social media, children’s self-determination) • Policy (e.g. cybersecurity, copyright, taxes, housing) • Climate change (e.g. resilience, carbon emissions, energy, electric vehicles, pipelines) • Controlled substances (e.g. state versus federal law, avoiding crime, licensing, taxes) • Equalization (e.g. income, taxes, resources, cost of living)
During the coding phase, we attempted to identify the challenges that the policies were meant to address and the subsequent problems that arose because of the policy implementation. We then mapped the results to traditional RE nomenclature (e.g. in some articles we found indicators of inadequate stakeholder identification). A sample of the coding sheet is presented in Table 1.
This study has several validity threats that need to be discussed. One of the significant construct validity threats is the assumption that RE processes and policy crafting processes share a strong parallel. We believe that the collected evidence and discussion presented in the paper provides sufficient evidence to support our claims. Still, further empirical validation of this assumption needs to be performed in the future.
The most significant threat to internal validity is that the observed unintentional effects and consequences have not been statistically analyzed or confirmed. We have not used experimental methods to study the effect of changes in the independent variables on the dependent variables (for example, involving a class of stakeholders in relation to unintended consequences). However, the study has an exploratory nature and we do not claim that the presented consequences are complete or true for all contexts.
Conclusion validity threats have limited impact on this study since we have not used statistical tests to obtain our results. At the same time, We made several efforts to minimize subjectivity and resolve potential conflicts when analyzing and categorizing qualitative evidence. We used researcher triangulation to decrease the subjectivity bias, with the first two authors performing the analysis in discourse and the third author reviewing the coding and interpretation for consistency and correctness.
Since the study is exploratory, external validity remains the main limitation of our work. We aim for analytical generalization rather than statistical generalization [16] and present the case and method details to enable replications and further studies. Still, we studied only a limited dataset of 152 articles on government policies and policy initiatives.
We note that we are taking a humanistic approach to our analysis. While there is a significant body of research in automated processing of news feeds and sources like Twitter, that work generally analyzes large corpuses. Unfortunately, that is not the way “the average person works”; they do not read hundreds or thousands of articles on an issue, they might read one or two. This is a substantial validity threat, but we mitigated this risk by individually reading every article and performing the final coding after discussion.
We retrieved and analyzed 152 articles and an excerpt of our analysis is presented in Table 1. The left column indicates the identifier of the news item, then the category into which we classified the article. The bottom three rows are summary rows of the categories Controlled Substances, Equalization, and California Sustainability Policy, as we found the results more insightful on the aggregated level. For each row, we identify the Goal as the original intention for the policy and the Solution that was chosen. We then identify the Unintended Consequence arising from that solution. We further tagged with Keywords and identified Problems of the scenario.
For example, the first row identifies the issue of the Reuters AI, in the category Algorithms, where the decision makers had the intention of making news faster, more accurate, and more resilient against fake news attempts (in response to public outcry and nascent public policy initiatives). The established solution was to deploy 13 AI algorithms that mine Twitter feeds to identify topics of interest. The (potentially) unintended consequence of the desire to more quickly react to current events is that the jobs of 2,500 highly educated and skilled reporters are potentially being eliminated. We associated the keywords news, media, AI and algorithms, and the main problem that all relevant stakeholders were not considered.
The retrieved articles are dominated by works wherein the author reflects upon some policy initiative and the associated successes and failures. These kinds of articles appear to be inherently biased towards negative critique (perhaps in an effort to generate more page views?) and they became a rich source, perhaps even a treasure trove, of unintended consequences. These articles all contain strong observer bias (they are opinion pieces), but we use them as data sources anyway – for these are the same data sources that shape their reader’s opinions and perceptions. After all, just because a source is biased it does not mean that the inherent message is not reality to the reader. We also gathered resources for two instances where policy was reduced to practice (sustainability and environmental policy initiatives at California State University Long Beach and the Port of Long Beach, including significant traditional engineering technology analyses and engineering economic analyses).
The first two authors coded the articles in discussion and we applied significant domain knowledge of their own to provide context for the observations and to enhance the richness of the conclusions. This technique has the potential to provide greater insight but is also a significant threat to validity. We are trying to be humanistic in this work, we are analytical but not coldly so. In other words, we are reacting as people, not as a machine algorithm. We are observing emotional content and there is the potential that we have introduced some of our own emotional bias on some of the topics. For each article, we read the content and (typically) the first 50 to 100 reader comments (assuming that comments are present).
Within the first few months of our study, we realized that our data set would have an inherent bias: it would not be unreasonable to assert that the MSM generally reports upon things in a negative manner, and the associated comments are often more extreme than the studied article.
With this realization we adjusted our research effort, focusing more of our efforts upon those reports wherein there appeared to be unintended consequences of some public policy initiative. We note that there were many, many reports of unintended consequences and not all were negative. We then refined our effort to identifying the unintended consequences and evaluated them using a system model. Our analysis was based on the question “If this was a software system and we were performing a post hoc evaluation using RE techniques, what observations and recommendations would we make?” The complete codebook is available on Google Drive [15]
Legislative Contradictions: We observed cases where
there are contradictions within legislation – how do the
individuals responsible create legislation that contradicts? Are
these conflicts deliberately created by those responsible or is
there something else influencing (and possibly corrupting) the
process? For example, on the topic of marijuana legalization,
individual states in the USA have decriminalized personal use
while federal law continues to make possession a crime.
Legislators in individual states have deliberately chosen to contradict
federal law. Would we tolerate conflicting, and unresolved,
requirements when designing a software intensive system?
Related work represented policy constraints as logic program
[
Same Old Problems: Despite 50 years of experience in IT systems, the last 30 years of (approximately) which RE has been a formal discipline, we observe that system after system continues to experience problems such as missing requirements and missing stakeholders. For example, the Government of Canada embarked upon the creation of a unified payroll system for all federal government employees. The system must manage the contracts for hundreds of different unions, each of which has their own pay scales, promotions, benefits packages and retirement plans. Individual employees could spend their entire career within a single union or change to a new union each time they change the position in which they are employed. RE practitioners would immediately recognize the likelihood of a combinatorial explosion in the business rules and data elements that must be managed and would identify the issue to the stakeholders. In this case, the issue appears to have been trivialized or ignored, and while we do not have any “insider information” that would allow us to elaborate further, we do note that the project is considered a near-complete disaster by all stakeholders and projected implementation and remediation costs are in excess of 400% of the original budget (the Government recently announced1 that cost estimates have exceeded $1B CDN and the creation of a task force to find a replacement before this system is even fully functional). Other government IT projects (especially those related to health care) do not seem to fare much better.
Holistic Perspectives: What is possible and highly desired
from a political perspective is often not possible from an
economic perspective and it seems that policy makers rarely take
this holistic view. For example, promoting the use of electric
cars should reduce CO2 emissions and is relatively easy to
justify if the only metric used is emissions per distance traveled.
However, electric vehicle production creates significant CO2
emissions [17] and the consumption of significant quantities of
rare metals. The electricity used must be generated by low
emission sources and (somehow) delivered to the vehicles. It is
well-known in electrical engineering practice that the North
1
http://www.ctvnews.ca/politics/minister-fixing-phoenix-pay-system-couldcost-1b-1.3672663
American electric distribution grid was not designed to deliver
this much energy and can only hope to do so with careful
demand management (e.g. only charge your vehicle after 9 p.m. if
you live in a residential neighborhood) unless significant
investments are made in infrastructure improvements. Despite
the challenges, to achieve the desired public policy objectives
systemic changes and early adopters will be necessary. For this,
we can use the foundations of systems thinking [13][
Identifying the “Right” problem: Our review identified
numerous cases where those directly involved with a policy or
project appear to believe that they have correctly identified the
problem, and that their proposed (or actual) solution addresses
the problem. However, when other parties look at the problem
they quite strongly disagree upon the problem definition –
consider the acrimony that exists between perspectives on climate
change challenges and proposed solutions. This pattern implies
that there is a class of problems where perspective is very
important. If that is the case, is our established body of RE
practices applicable to those problems? Does RE have to evolve to
be able to support these problems or do we just ignore that
class of problems? This concern is partly addressed by some
work in RE on viewpoints[
Side Effects: When reviewing the articles, we were
repeatedly given the impression that comprehensive analyses of
potential complicating factors is either performed badly or not
performed. This is an area where RE can significantly
contribute beyond the work in [
(Magnitude of) Unintended Consequences: The unintended consequences of the policies under investigation have a much larger scope and scale than we expected. And, larger than the original policy intervention necessarily would have made many people believe.
As an example, consider affirmative action policies whose goals are to “level the playing field” between disparate groups. Superficially, these policies obtain at least grudging acceptance by a majority of the populace in North American jurisdictions. However, we see evidence that different special interest groups “weaponize” these policies, in different ways, and use them to increase conflict rather than decrease conflict. As a result, positions become ever more polarized and compromise solutions become more difficult to achieve.
Affected Domains: Different regions within a state,
province or country tend to have different social mores and these
can translate to differences in local legislation and increased
potential for conflicts. In Canada, legislative powers are
deliberately split between the federal and provincial levels to help to
address these differences. Despite the well-established
principle that federal powers overrule provincial powers, individual
provinces that do not agree with federal policy on a given topic
can, and do, attempt to override the federal policy by crafting
confounding or competing legislation within those aspects of
exclusive provincial jurisdiction. For example, inter-provincial
and international pipelines are clearly placed under federal
jurisdiction. Environmental regulatory powers exist at both the
federal and provincial levels and anti-pipeline activists attempt
to use provincial environmental regulatory powers to impede or
completely block any federal approvals for such projects.
While conflict identification and resolution have been targeted
by [
Privacy: Even though users legally agree to having their
data tracked by many of the apps and services that they use,
most people grossly underestimate the magnitude of the data
trail that they create. While this data collection activity is
presented to the user as a way to improve the user experience,
more and more users are learning that the same data can also
have significant unintended consequences – especially when
that data is licensed to a third party. We routinely see reports of
individuals that use social media experiencing negative
consequences (e.g. denied insurance claims, denied bank financing,
inability to get job interviews, etc.) [
Public policy is responding to these reports, most notably in
the European Union, and there is increasing pressure on the
providers of these services to support correction and deletion of
data collected about individual users, including the “right to be
forgotten” [
Biases in AI and Data Mining: Bias in automated
decision-making systems is receiving ever-increasing public policy
attention. Closely related to privacy issues, deliberate or
unintentional bias has the potential for significant unintended
consequences. The specialists in the field don’t always know why
they are accepting the results they get out of the algorithms,
leading to backlash from observers (“Will anyone ever write
another positive story about a tech startup? I said probably not”
[
Significant elements of the technology sector could find themselves regulated, or at least required to justify or defend 2 https://www.theguardian.com/technology/2017/oct/30/facebook-russia-fakeaccounts-126-million their algorithms in ways that can be comprehended by policy makers and by the general public. What happens when the technology sector answers “We don’t know. We just know that it (seems to) work better than anything that we have done before.”? What will happen to “technological progress” if everyone believes that they have “the right” to provide input any time that they believe that this class of algorithms is affecting their lives (or they will threaten to claim some form of oppression or human rights violation)? How many companies that rely upon data mining would find their business models at risk in such a regulated environment?
Deployment of these technologies in support of public policy will require us to truly understand what is “going on inside.” Otherwise, how do we evaluate whether we are or not excluding people from fair treatment in our society based upon what some algorithm returns as a result. In the world of the movie “Minority Report”, precognition was combined with significant technical support to eliminate murder. What happens when we replace precognition with data mining and AI? The computational techniques may be mathematically accurate but how do we know if we are correctly interpreting the results? If, for example, ethnic heritage in combination with neighborhood, socioeconomic status and educational level, leads to a person being identified as a potential future criminal, does that mean the individual is indeed a criminal and should be treated as one? Does that mean that the analysis has identified fundamental flaws or failures in society’s structure? Even though these factors often correlate, they are not necessarily causal and, therefore, should we be working on fixing the cause of the problem and not the symptom? Finally, if the models are telling us things we don’t want to hear, then maybe the models are simply identifying opportunities for improvement.
Social Perspectives: We identified a pattern of hardening
of positions by factions interested in public policy topics.
Rather than looking for compromise, it appears that the factions
are treating issues as a zero-sum game: “we adopt my position,
or else…” Can RE techniques (especially those related to
conflict resolution and mediation) be used to find common ground
between these stakeholders? What does it mean to practitioners
(and society in general) when stakeholders tell us there can be
no validity in a common ground? As Brown points out, in our
current society there is a “phenomenon of you are either with
us or against us.” [
For example, this pattern is very evident in people’s positions about climate change. Do you believe humans contribute to something that is called climate change? Do you believe that greenhouse gasses can be absorbed by the environment without significant damage or not? Do you feel we should be minimizing our human byproducts and pollution?
You can interpret these questions with sufficient qualifiers such that eventually you will get almost every climate change denier or promoter to agree. For example, many climate change deniers are not against mitigation policies per se, rather they tend to be against specific policies because they do not believe that those policies are a cost-effective solution to the problem. For example, taxing carbon emissions at such a level that people simply cannot afford to travel except in absolute necessity will have the effect of reducing emissions, but is this even possible to implement in a democratic society where people need to travel to work? If you believe that climate change is caused by human intervention, perhaps you could target non-essential travel – for example, ban tourism. Superficially, this would create non-trivial reduction in emissions. However, such a policy would eliminate a significant source of income for many developing nations and seriously impede their ability to offer public services such as health care.
This is a significant unintended consequence. The policy would destroy the livelihoods of everyone in the tourism industry and of many third world nation service industries – is that what the ‘environmentalists’ want to happen? We posit that this is unlikely.
To make this point even more strongly, the Government of
Canada attended the Paris climate convention and signed the
Paris climate accords. Later that year, the Parliamentary Budget
Office (an agency that provides independent cost analyses of
parliamentary initiatives) issued a report that sought to bring
the commitments into perspective for the average citizen
[
In contrast, sustainability initiatives in California underwent
significant planning efforts, culminating in realistic
implementation plans [
Emotional Content: Emotionally charged content is prevalent across many of the articles, as evidenced by the author’s selection of adjectives and adverbs and by the positions taken by supporters and detractors within the accompanying comment sections. From the Twitter storms of President Trump and his interactions with Kim Jong-un to people issuing threats on social media platforms toward people who oppose their position on issues of the day, how do we get past all of that negativity and unwillingness to compromise to even get to the point of being able to agree upon a goal, let alone solutions? Are these behavior patterns evident even when performing RE for software intensive systems?
Time: The time needed to introduce and pass legislation in support of policy initiatives (e.g. reduce industrial CO2 emissions), and to see the effects of the policies (often measured in decades), is much longer than the average time a government holds power (four to six years in most democratic countries). This reality has led to two patterns: New governments try to reverse policies set by previous governments resulting in aborted efforts and significant waste and, in anticipation, current governments try to establish policies in such a way that they cannot be easily modified. What is lost if the original policy implementation was actually going to be effective? How does a new government undo a policy that has proven deleterious?
Using the MSM as a data source, rather than peer-reviewed
academic papers is certainly a ‘different’ research experience.
The prevalence of opinion, often strong opinion, without
substantiating evidence converts the quality evaluation process to
one of (1) how well is the article written (for me)? (2) does the
article align with my personal biases and (3) what do I perceive
to be the reputation of the author? When one includes the
comments in the analysis, it is easy to be affected by the
strength of the (often negative) positions held by the
commenters. This negativity bias can easily be passed on to the analysts
and this contagion is a known psychological phenomenon
[
How does the MSM affect stakeholder perceptions, opinions, and the hardening of both? There seems to be an acceleration and hardening of positions in mainstream media – is this something that RE might have to consider or be more cognizant of moving forward? For example, do you send your (politically) left-leaning team in when you have a (politically) leftleaning client? Such a proactive effort can amplify the biases (prejudices) but has the potential to lower the risk of miscommunication. We take special note of the seeming rise in ideological bias on the part of policy makers, thought leaders and the general public. This trend toward the adherence to a position or interpretation independent of rational analysis of the underlying facts could have far-reaching and unexpected effects. While we have observed evidence of ideological bias in policy, we must ask whether this trend will have an effect on RE for software artifacts. For example, will practitioners need to be more diligent in exploring stakeholder statements of their wants, exploring whether or not these wants can be evaluated as stated (in the transition from wants to needs during prioritization) or whether the statements must be further explored to identify ideological biases? When attempting to understand the risks and threats arising from this trend we are prompted to ask: How does this knowledge inform us about how they (policy makers, their constituents, and politicians) perceive circumstances and issues; what gains, risks and threats does this offer to RE practice? In this context, miscommunication challenges can potentially be greater than anticipated.
We were somewhat surprised, if not shocked, by the number of instances of open conflict in regulations and legislation. Perhaps we were naïve in assuming that the legal structures 3
Associates of the lead author actually held what could (charitably) be called a mini-intervention with him in an effort to determine what had caused him to become increasingly negative over the prior six months. would be more organized and better structured than they are, but we can’t help but wonder what it would be like if they were as relatively error-free as well-crafted software.
In this work, we investigated the use of the mainstream media as a data source for an analysis of public policy and related governmental initiatives. Our first research question asked whether we could identify challenges associated with defining, formulating and realizing public policies from this data source. We were able to identify challenges by analyzing the content, removing the commentary, then identifying the underlying “facts”. We were also able to identify how people perceived the challenges, which often were biased by ideology. Furthermore, we found that the challenges do have analogs in RE for software intensive systems, so there is potential that RE can help to proactively identify possible public policy challenges.
A metaphor that we find useful is that public policy is the algorithm for governing the operation of the machine that is society. As such it is easy to answer the next research question in the affirmative: how could RE techniques help mitigate the identified public policy challenges? We saw no evidence that RE techniques could not be successfully applied in this domain. Finally, we asked whether RE techniques could be used to proactively identify possible public policy challenges during formulation and before enactment? The final research question is not as easy to answer. Certainly, validation and verification techniques could be used to identify issues and mitigate risks if the participants had sufficient domain expertise and, dare we say it, wisdom.
We see sufficient evidence for there to be a role for RE in helping people at large to understand the technology that seems to overwhelm them, possible consequences and side effects.
While this work is another piece of evidence of the universal nature of problem patterns and critical thinking, it has also delivered significant context for future work. Each of the major points in Section V could readily become a research thrust: • Legislative contradictions • Same old problems • Identifying the “right” problem • Side effects • Magnitude of unintended consequences • Holistic perspectives • Affected domains • Biases in AI and data mining • Social perspectives • Emotional content • Time
Upon reflection, we must also ask whether the focus of this work is even something with which RE practitioners should be concerned. What are, and should be, the bounds of RE? Despite the fact we are pushing the bounds of RE, are we pushing too far into ethics and overreaching? Are we oblivious to the fact that there are many other people already attempting to address these issues?
Other research questions that are prompted by our experience, but farther afield from RE include: • Do opinion article writers become thought leaders? Are they good barometers of the populous and their emotions? • Can we use machine learning across the “wisdom of the crowd” as a means to validate what the pundits and politicians are saying? • Can we extract the core content of each document and perform formal semantic analysis to identify the biases in the presentation and to quantify the intensity of the bias? We could attempt to identify the ideology of the policy authors, reporters and commentators, but sense that this is far out of our traditional field. • Are things really “as bad” as the MSM would seem to want to have us believe? Is it possible to know the relative scale of the negative elements – were the reported negatives only a small proportion of the overall initiatives? Were the reported negatives only relatively rare occurrences in the greater context of society?
This work has proven to be a rich source of research questions and opportunities that provide ample pointers towards future work. First, we want to deepen our analysis using empirical methods and expand on this exploratory study with a quantitative analysis of evidence for said RE challenges. Second, we want to use that data for a scenario analysis, applying RE techniques to explore whether we can use this form of analysis to prevent some of the unintended consequences that played out in those scenarios. Third, we plan to detail a research agenda of other opportunities outside of software and systems engineering where requirements engineering techniques could have significant positive impact and improvement potential for the situations under analysis. Finally, we plan to perform an RE analysis of the goals of the GDPR, their feasibility, validation and verification techniques, and monitor for evidence of unintended consequences. solving:
A task [13] Checkland, Peter. "Systems thinking." Rethinking management information systems (1999): 45-56. [14] N. Chomsky, "What makes mainstream media mainstream." Z magazine 10.10 (1997): 17-23. [15] Results Codebook, https://drive.google.com/file/d/14ErlsN
OOMj79GnXknWvOxu1UtG3ZUB1_/view?usp=sharing [16] W. N. Dunn, Public policy analysis. Routledge, 2015. [17] Dyer, Ezra. ”That Tesla Battery Emissions Study Maming the Round? It’s Bunk.” Popular mechanics, June 2017. https://www.popularmechanics.com/cars/hybridelectric/news/a27039/tesla-battery-emissions-study-fake-news/ [18] J. Entine, "The myth of social investing: A critique of its practice and consequences for corporate social performance research." Organization & Environment 16.3 (2003): 352-368.