Creating software products is a complex endeavor requiring the cooperation of people with di erent skills/knowledge/thinking. Managers, developers, testers and technical writers have to work together to create and deliver software products. These roles might have di erent views, perceptions, knowledge even in the same project. In order to understand their commonalities, di erences and the evolution of their skills we run a survey that was lled in by 456 professionals working in software development projects. We have found among others that (1) Internet is one of the most useful source of information for professionals; (2) trial and error is perceived to be more e cient then formal training; (3) testing skills are among the most important ones; (4) there are little di erences in people's way of thinking compared by roles, by size of employing company or by experience levels; (5) at the same time, larger companies seem to be more e cient and their experts are better at monitoring and testing new technologies/methods; (6) interestingly, although most companies support the improvement of internal quality, most respondents have very limited knowledge or are not concerned of anti-patterns.
Meet Joe, a software developer working in a team of 4-7 members. His friends, Jennifer the manager and Ben the tester, both collaborate with 30+ people. Finally Alice, a technical writer, is again part of a team of 4-7 members.
All of them work at companies employing 1000+ people and have less than 2 years of experience. In their eyes formal training is a waste of time, they learn from the Internet and on-the-job. They believe that developer and tester mindsets are both important. Although their companies support them in creating quality products, they mostly don't know or don't care about internal quality. According to our survey they represent the largest groups of people in their eld, working in software development projects at companies present in Hungary in the year 2015.
Copyright c by the paper's authors. Copying permitted for private and academic purposes.
We surveyed individuals working in software development projects. We wished to understand how the knowledge of IT employees di ers having various roles (manager, developer, tester, technical writer), how they gain new knowledge, how they vary in thinking about their processes and anti-patterns in software development. This paper presents the results of the survey focusing on roles, experience levels and the size of the companies respondents were working for.
Our research questions are:
RQ1 How well known are the techniques of di erent elds? RQ2 How important are the di erent mindsets? RQ3 What are the main sources of new knowledge? RQ4 How useful are the several knowledge gaining methods in daily work? RQ5 How di erent is the way of thinking in the various roles?
RQ7 Are people motivated and supported to resolve anti-patterns? RQ8 How does the size of the company or team organization impact people's knowledge, thinking and perception of anti-patterns?
RQ9 How does experience level impact people's knowledge, thinking and perception of anti-patterns? This paper is organized as follows. In Section 2 we present earlier works related to anti-patterns in software development. Section 3 details the methodology used to create and run the survey. Section 4 shows the generic answers and details on each role group. Later we address how the same questions are related to the size of company in section 5 and experience level of respondents in section 6. Section 7 summarizes our results and observations. Finally, section 8 deals with the validity of our results. 2 2.1
2.2
In 2013 Yamashita et al. [16] conducted a survey nding that 32% of the respondents did not know about code smells, nor did they care. Those who were at least somewhat concerned about code smells indicated di culties with obtaining organizational support and tooling.
The \State of Testing 2015" survey [17] showed that the demand for testers, who can do more than \just testing", is increasing. 81:5% of the testers reported to learn their mastery mostly while doing their work, while only 17% on formal trainings.
The \Worldwide Software Testing Practices Report 2015-2016" [19] survey found that organizations use on the job trainings (72:9%), certi cations (51:9%) and formal training (46%) to improve the competency of their employees. This survey also found that Agile management techniques (Scrum, Extreme programming, Kanban) are being adopted more often (69:6%) in software development projects. 3
In our survey we investigated the knowledge and concerns of people working in software development projects. 3.1
Our main goal was to explore the thinking of software professionals working in di erent roles, to gain knowledge on how they align with industry-standard processes. The secondary goal was to explore what they know, how they learn, and how they are committed to internal quality. 3.2
To get comparable information from di erent elds involved in software development we used two control groups. We asked the rst control group { at least one person from each target group { to evaluate our questions. They were given the survey with the explicitly stated aim of validating the expressions/sentences. Once the reported issues were corrected we created a second control group. This time participants were asked to ll in the survey on the web form it will appear later in. This was done in order to validate the corrections of earlier mentioned issues and to discover potential problems/technical di culties with the layout of the survey. The results of the control groups were not included in the nal survey.
To reach as many people as possible we created our anonymous survey with Google Forms using a minimum number of open ended questions. To track the knowledge of respondents we used questions with several prede ned answers. To track the opinion of respondents we o ered scales with ve options. At some questions we asked for percentages of time spent with some activity. 3.3
We grouped the 47 survey questions (found in the appendix) into 6 sections: 1. \Generic information" established information, regarding the respondent's main role, task and size of their organization. 2. \Familiarity with di erent techniques" contained speci c questions related to the four main targeted role groups to understand the actual knowledge of participants. 3. \Gaining new knowledge" collected information on how and from where participants gather new knowledge to improve their existing skills. 4. \Process and methodology related questions" assessed how many participants follow the industry-standard methods in their work. 5. \Anti-patterns" contained questions on how the participants are committed on the internal quality of their work. 6. \Static analysis and traceability" contained questions on static analysis tools, reviews, traceability issues. 3.4
Exploiting our social networks we contacted IT people from several companies (performing software development) with o ces mainly in Hungary and asked them to ll in the survey (for example Ericsson, LogMeIn, NSN, SAP, NNG, Prezi, GE). We have also contacted several meetup1 groups to let us advertise our survey on their site: Test & Tea, Hungarian C++ Community, Budapest DevOps Meetup, Freelancers in Budapest. The survey was posted to the Hungarian IT professionals group at www.linkedin.com. From the public forums we used www.hup.hu2 and www.prog.hu3.
We have also contacted the Technical Writers group of facebook. 4
In total we received 456 responses from several professionals: 39 architects, 8 business operation supporters, 171 developers, 2 executive managers, 10 line managers, 3 manager of managers, 20 project managers, 2 self-employed, 28 team leaders, 28 technical writers and 145 testers.
To make the information processing easy we grouped the roles into four distinct groups: developers (210), testers (145), managers (71) and technical writers (28). At the end we decided to exclude responses from the self-employed respondents. Their answers could not be merged into the other groups as they might do in their daily work all of the tasks of each role group. At the same time we could not analyze their answers separately as that could have compromised their anonymity.
In order to be able to calculate statistics we mapped the \Not required { Required", \Never { Always", \Not concerned { Concerned" terms in the answers to the scale from one to ve points. 4.1
86% of the respondents work for multi-national companies (85% of developers, 89% of testers, 81% of managers, 96% of technical writers). All but one technical writers responded to work for a multi-national company.
63% of the respondents work for companies having 1000+ employees. The ratio of testers is the highest in 501-1000 employee companies (52%), while the ratio of developers is the highest (70%) in companies employing 10 or fewer people (Fig. 1(a)).
Tech. Writers Management Testing Development
32% of the respondents work together with more than 30 people in their main project (Fig. 1(b)). The second largest team size is 4-7. Most of the managers (47%) and testers (39%) work together with more than 30 people. Most of the developers (31%) work in projects of team size 4-7, just like technical writers.
51% of the respondents have less than 2 years of experience (29% have 3-5 years, 11% have 6-10 years, 7% have over 10 years of experience). We observed approximately the same ratio in all four groups (except that no technical writers reported to have 6-10 years of experience).
Figure 2 shows the tasks of people re ected to their role in 2014. Developers were developing systems (44% of all respondents), editing code (22%) and doing maintenance work (9%). Testers were testing (79%). Managers 1communities organized on www.meetup.com 2Hungarian Unix Portal 3a web portal claiming to be the largest developer and programmer community in Hungary itsgTen lttsvySeeeepdnomm iitgeeddnoC iiitttrcgaenudnnoomW ilggaaeepnnpoM itcaaeennnM ijtrsggaaePnnoM rscaeehR iittrvggaaeeennhnnnomM iittrrggaeeeehnunqRm ltyeepnoDm itrsvTeeew iliiiftttrrccgaaenupnnnooom irveeedoCw iiittrsandnoAm
W managed people (35%) and projects (20%). Technical writers wrote documentation (89%). Only developers did code reviews (1%) as main task, the environment was mostly managed by testers (3%).
As most common additional responsibilities we recorded writing documentation (48%), testing (47%) and code review (43%). Test review and code editing took 4th and 5th place (37%) overall.
The most common secondary tasks were: for developers code review (67%) and testing (30%), for testers test review (67%) and writing documentation (53%), for managers managing people (42%) and administration (38%), for technical writers administration (39%) and product \research" (35%). Both developer and tester mindsets are very important in software development projects. While testing techniques are well known, development techniques rank as the least known.
The top three known design patterns are (Fig. 3(a)): singleton (55%), iterator (52%) and factory (49%).
The top three known testing patterns are (Fig. 3(b)): function testing (89%), use-case testing (69%) and review (64%).
The top three management methodologies are (Fig. 3(c)): scrum (88%), agile (87%) and waterfall (82%).
The top three technical writer patterns are (Fig. 3(d)): user documentation (64%), system documentation (59%) and review (38%).
In each experience level the ratio of people knowing any given design pattern is similar (Fig. 21(a)).
Developers, testers and managers know approximately the same ratio of testing techniques, management techniques and technical writing techniques.
Technical writers know review, walk-through, inspection the most from testing techniques and scrum, agile and waterfall from management techniques. Technical writers have a balanced knowledge, more emphasis on analysis of audience, precise expressions proof-reading and less emphasis on user and system documentation.
Managers concentrate more on focus groups, documentation life-cycle management, and less on user testing and review.
Comparing all patterns we can see that the most known techniques are: Function testing (89%), Scrum (88%), User documentation (64%) and Singleton (55%).
Developer mindset was selected to be important (4-5 points) by all groups (93% developers, 61% testers, 65% managers and 46% technical writers). Testing mindset was selected to be important as well (4-5 points) by all groups (76% developers, 97% testers, 69% managers and 50% technical writers). Technical writer's mindset was selected to be important (4-5 points) mostly for technical writers (13% developers, 36% testers, 24% managers and 96% technical writers). Management mindset was selected to be important (4-5 points) mostly for managers (15% developers, 41% testers, 93% managers and 57% technical writers).
Altogether, the developer and tester mindsets were selected to be the most important in the software projects (Fig. 4(a)). This points to an interesting observation: testing mindset is reported to be important and testing techniques are well known, however, development techniques are the least known, but the mindset was still considered to be one of the most important. Management mindset is considered to be only the 3rd on the 0% 20% 40% 80%
100% Function testing Use-case testing
Review Boundary value anaysis
Inspection
Walk-through Exploratory testing
Code metrics
Coding standard Decision table testing
Error guessing
Branch testing Statement testing
Fault injection
Call graphs Control flow analysis
Path testing
Pairwise testing Fault attack with defect checklist
Cause-effect graph Classification tree method
None of the above
User documentation System documentation
Review User testing
Interview Documentation Life Cycle
Clear design Proofreading Focus groups
i18n
Survey Gathering specific vocabulary
Precise expressions None of the above
Analysis of audience Problem-Method-Solution
L10n Chain of new concepts Chronological structure
Camera-ready
S-V-O structure importance level, still, some techniques are known by 30% more respondents than the most known development technique. 4.3
The top three sources of new learning (Fig. 4(b)) were: Internet forums and blogs (82%), colleagues (79%) and books (65%). All 4 groups investigated show similar preferences. These results were repeated in the answer on what resources they have used in the previous year.
Some additional sources for gaining new knowledge (that were not included but remarked in the questionnaire): meetups, online courses, self study.
We found (Fig. 5) that all role groups came by approximately the same ratio of knowledge through formal training (24%). However, the maximum ratio were very di erent: some developers could gain 100% of their knowledge in this way, while for technical writers the maximum was only 50%.
On-the-job training was most useful for managers (41% on average) and least useful for developers (30% on average). In this case the maximum ratio reported was 90% for technical writers, and 100% for all others.
Self study is the main source of knowledge for developers (44% on average), while technical writers use it the least (31% on average).
Formal training On the job training Self study Trial end error 50% 40% 30% 20% 10% 0% 100 80 60 40
Tech. Writing Management Testing
Development Formal training On the job training Self study Trial end error
Internet forums and blogs
Collegues
Books
Training Company intranet
Conferences Research papers
Vendor sites 50 40 30 20
Trial and error is the source of 27-29% of knowledge for developers, testers and managers (on average). Some of them reported to gain 100% of their knowledge this way. Technical writers gain only 21% of their knowledge in this way (on average), and none of them reported to have gained more than 50%.
Technical writers can rely the least on both formal trainings and learning by trial and error. They might get the same amount of knowledge from these methods, but they can get at most half of their experience in these ways (on average).
Formal trainings are less useful than trial and error based learning, and less people could claim to have learned everything on these ways. 4.4
To be able to compare the di erent groups of people participating in software development projects we decided
to check how strong their process and methodology orientation is versus an ad-hoc and intuitive practice in their
daily work (we call this as \scienti c" thinking). We asked whether (
Results show (scores between 3 and 4) that at most companies it is somewhat important to work in a manner ful lling strict processes and methodologies in order to see from where and to where tasks and people are heading, to understand where and in what position the work is standing.
When we compared the main roles of the respondents based on their scienti c thinking/methods we observed
that respondents in development, testing and technical writing show similar values, while managers provided
distributed answers (Fig. 6 and Fig. 7). The average standard deviations for the process and methodology
related questions (Fig. 7) in descending order: Q28 (1; 14), Q23 (1; 13), Q24 (1; 12), Q25 (1; 11), Q30 (1; 1), Q35
(
Checking the correlation coe cients between the answers given by people having di erent roles revealed the following:
The highest correlation could be found between developers and testers: 0:93 on average.
Developers and architects way of thinking had the second largest correlation coe cient: 0:90.
4,5
4
3
3,5
2,5
t
n
e
m
e
g
a
n
a
m
e
v
i
t
u
c
e
x
E
s
r
e
g
a
n
a
m
f
o
g
n
i
g
a
n
a
M
t
r
o
p
sse spu
isn /no
uB itra
e
p
o
p
i
h
s
r
e
d
a
e
l
m
a
e
T
t
n
e
m
e
g
a
n
a
m
tc
e
j
o
r
P
t
tc
e
i
h
c
r
A
t
n
e
m
e
g
a
n
a
m
e
n
i
L
t
n
e
m
p
o
l
e
v
e
D
g
n
i
t
s
e
T
g
n
ii
t
r
w
l
a
c
i
n
h
c
e
T
The team leadership way of thinking is correlated with (
The architect way of thinking is correlated with: (
We also observed a correlation of 0:80 between technical writing and testing mindsets.
All other correlation coe cients were below 0:80. The process and methodology orientation of the management (including executive management, managing of managers, business operation/support, project and line management) has little in common with each other and with other roles. In the technical writer's thinking they are the closest to testing (0:80) and development (0:78).
Executive management Managing of managers Business operation/support Team leadership Project management Architect Line management Development Testing
Technical writing
Q22 Q23 Q24 Q25 Q26 Q28 Q30 Q31 Q32 Q33 Q34 Q35
Respondents reported (Fig. 8(a)) that in their company the newest technologies/methodologies are frequently monitored and evaluated (from never to always: 2%, 15%, 26%, 40%, 16%). Mostly managers and technical writers responded with con rmative values ( 70%), but even most of the developers and testers perceived that their organizations perform these tasks ( 50% con rmative answers).
We had similar distribution of answers for the question of how extensively new technologies are tested before introducing them into the organization's life (from never to always: 5%, 20%, 26%, 33%, 19%). We found that developers, managers and technical writers gave 4-5 marks in 50%, while testers in 60% of the cases (Fig. 8(b)). Technical writers found their tools best tested before introduction. 45% 40% 35% 30% 25% 20% 15% 10% 5% 0% 35% 30% 25% 20% 15% 10% 5% 0%
The answers for the question \how often testable hypotheses are established before work starts?" were in the middle of the frequency range (Fig. 9). In this case the di erent groups had very di erent perceptions. Developers gave the fewest high values and technical writers the most. Tech. Writers Management Testers Developers
When an activity is not done as speci ed respondents mostly follow a de ned process to improve it (Fig. 10). Developers rate their improvement processes the weakest, while technical writers the best.
When the outcome is defective despite all activities done as speci ed respondents mostly modify their processes. Again, developers gave the lowest values for the frequency of their process modi cation procedures, while technical writers gave the highest values.
Approximately half of the respondents in all groups reported the ability to detect when someone is idle for long and then follow a de ned process to modify or reassign activities. Respondents reported to have been idle 9-15% of their time in 2014 independently from their roles. The maximum idle ratio was almost twice longer for developers and testers, and almost one and half times longer for managers than for technical writers.
Approximately 40% of the developers, testers and managers reported high con rmative values (4-5 points) for being able to detect if someone is overloaded and then being able to follow a de ned process in order to modify or reassign activities. The average ratio of being overloaded in 2014 was 24% for developers, 28% for testers and 31% for managers and technical writers. The maximum reported ratio of being overloaded was very high in all groups.
Only 30% of developers are able to redesign their processes if they nd that a non-speci c activity is needed compared to the 45% of testers and managers.
In all groups, the respondents were able to detect easily what the next activity is in their processes being actually performed. High values (4-5 points) were given by 55% of the developers, 60% of the testers, 64% of the managers and 68% of the technical writers. In all groups only 15% of the respondents gave scores 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Developers
Testers below 3.
We observed the same ratio for determining who has to perform the next activity in the process.
Only 30% of the developers, testers and managers check the current state of a airs rigorously before making a change, compared to 45% of technical writers. Only 5% of the respondents reported that they always asses the current state of a airs before making a change.
When the results are not the ones expected 50% of the developers, testers and technical writers check how the change was done and what e ects it might had, compared to the 60% of managers. 4.4.1
When we look at how people in the di erent roles rate their processes and methodologies (Fig. 7) we get some interesting insights into how much and where their thinking (perception of their processes) di ers. Based on the average values of each role for each question (that fall outside the 3:2 - 4 range):
Executive Managers believe they are monitoring new technologies (4:5) and carefully testing them before
integration (
Managers of managers believe they set up hypotheses before work starts (4:3), but are least likely to check the reason if the result of a change is di erent from the expected (2:6).
Business operation/support believe they asses the current state rigorously (
Team leaders believe they are bad at nding out who has to carry out the next activity (2:75) and establishing
a testable hypotheses before work starts (
Project Managers nd it hard to identify idle (2:75) and overloaded persons (2:65). They also don't believe
they create testable hypotheses before starting the work (3:1), or assessing current state of a air with rigor
(
Architects generally give scores between 2:9 and 3:5. They don't believe to have an improvement process to follow when something goes wrong (2:7), or to assess the current state before making changes (2:7). They also don't believe they create good hypotheses before starting the work (2:8), or nd out why a non-speci c activity is needed (2:9), or telling if someone is overloaded (2:8).
Line managers believe they have good processes for telling who is idle (3:8), what the next activity is (3:7)
and who has to carry it out (3:78). They don't believe they are assessing the current state before a change
(2:9) or follow a process to improve (3:1).
Developers generally give scores between 3:1 and 3:4. They don't believe they assess the current state (2:87)
and establish a hypotheses (2:87) before starting the work. They also don't believe that, when something is
not done as speci ed or some extra activity is need, they follow a process to improve (
Testers generally give scores between 3:2 and 3:5. They don't believe they assess the current state (
Technical writers generally give scores between 3:5 and 4. They believe it is it clear what the next activity is (3:9), who has to carry it out (3:78) and when they defective out in spite of doing everything right they modify their processes (3:78). They least believe they can nd out why some non-speci c activity is need (3:28) or assess the current state of a air with rigor (3:32). 4.5
Although most companies support the improvement of internal quality, most respondents have never heard of or are not concerned about anti-patterns.
We have described anti-patterns as \an anti-pattern is a common response to a recurring problem that is usually ine ective and risks being highly counterproductive" in the survey.
35% of the respondents answered to have never heard of them, and 20% to have heard of anti-patterns but not sure what they are. 15% know them, but are not concerned. Only 25% reported trying to avoid them, and 2% reported a strong understanding. Anti-patterns are most understood by developers, and least by testers (Fig. 11).
When checked the question in more detail, we got that 51% of the architects tries to avoid them, 87% of business operations/support have never heard of them or are not sure what they are. 26% of the developers have never heard of them, 19% are not sure what they are, 19% are not concerned, 33% try to avoid them, but only 2% have strong knowledge and use tools to detect and remove them. Line, executive and manager's managers have balanced knowledge (half of them are familiar with anti patterns on some level, half of them not). 75% of project managers have never heard of them, are not sure what they are, or are not concerned. Only 12% of the testers know and try to avoid them and only 1% uses tools for detection and removal.
When asked how concerned respondents are about anti-patterns in their product, 31% of them reported to be not concerned and 30% to be mildly concerned. In all role groups at least 20% of the respondents were not concerned at all and only 5-15% were concerned (Fig. 11(b)). Developers (13%) and technical writers (10%) being the most concerned.
The result means that: at least 60% of the respondents in all groups are supported by his organization to improve the internal quality of their products (Fig. 12). The ratio is the best (65%) for technical writers. at least 40% of the respondents either have pre-planned sessions and work lists for internal quality improvements or correct such issues immediately when they notice them.
Such work is planned and done a formal activity On a regular basis When absolutely necessary Sometimes Seldom Never
We have allocated time for this kind of work in our processes When we have free time Tools are available In theory No (a) The abundance of working on existing products in order (b) The necessity of working on existing products to imto improve their internal quality. prove their internal quality supported by your organization. less than 6% have reported to have no organizational support for internal quality improvements. less than 7% have reported to have no process for internal quality improvements.
In 2014 most respondents produced low quality results in order to satisfy short term needs 1-5 times (35% 1-2 times, 29% 3-5 times). There were 68 respondents who did not need to give up on quality (Fig. 13), while 11% produced low quality 10+ times. The ratio of no compromises was best among technical writers (21%), followed by developers (18%), testers (13%) and managers (7%).
Our further analysis shows that most of the found issues are traced back to the early stages of processes and are controlled by static tools, manual code reviews and direct contact to customers.
According to respondents most issues can be traced back to code writing, concept/system design and requirement collection (Fig. 14). Regarding the role groups we had similar rates except that technical writers found task management as a source of problems principally and placed less emphasis on code writing.
Both technical writers and testers placed the most emphasis on the available documentation as the source of problem solving. Most organizations apply tools to statically check adherence to coding standards and to measure metrics (Fig. 15). At this point we observed a clear di erence in the roles: developers, testers and managers take static analysis tool supports by approximately the same percentage in their work, but technical writers reported to be less supported in checking coding standards and measuring metrics. They also reported the highest ratio of not being supported by static analysis tools.
We asked furthermore whether manual code reviews are used in internally developed products: 73% answered yes (81% of developers, 67% of testers, 74% of managers and only 39% of technical writers, Fig. 16).
The average time of manual code reviews took 51 minutes for testers and technical writers, 56 minutes for managers and 58 minutes for developers. The maximum time spent with manual reviews was 8 hours for managers and technical writers, 16 hours for developers, and testers could spend up to 24 hours.
By our measurements 40% of the respondents selected to have no direct contact with their users (Fig. 17). After direct email (51%) this option received the second most votes. 0% Data flow analysis Control flow analysis Our techniques are not tool supported Other tool support static analyses techniques Checking of metrics Checking of coding standards Some respondents mentioned that customer support and issue tracking tools are \the" direct contact to users.
The question \How do you judge if a speci cation is out-of-date?" was o ered to the respondents in order to describe the situation with their own words. 30% of the respondents gave any answer. 4% categorized as \do not care", 2% answered to check the version number of the appropriate document, 1:9% decided to verify the date of the last modi cation, 2:5% would ask for help to decide. 1% of the respondents answered that their processes make out of date documents impossible. 2% would compare it to the code or existing features. Other 1% of the respondents mentioned some mechanisms or tools that are able to check the validity of the speci cation before work starts. Rest of the responses either did not understand the question, or could not be categorized in larger groups. For example: working on prototype means documents are always outdated, have not happened yet, \by my standards", \too bad". 5
In this section we analyse the di erent mindsets through the size of the company.
We experienced that bigger companies have more career options, more experienced people, better processes, better quality validations and better on the job training instead of reliance on self-study. Bigger companies use their resources more e ciently, without overloading them more, without indirectly forcing them to produce lower quality.
In all companies with less than 1000 employees we found only 1 employee with 10+ years of experience, while 1000+ employee companies employ 6%.
As the size of companies grows, more job roles appear (1-10: 5; 11-50: 7; 51-150: 7; 151-1000: 8; 1000+: 9).
The larger the company is the more developer mindset is demonstrable. In companies with 1-10 employees three times more people selected 5 (most important) for the importance of the developer mindset than 1 (least important). In 1000+ companies the ratio is twenty three. The same is true for the testing mindset with multipliers in the range of 2-150. In the case of the management mindset the multiplier is 2-3 in all company size ranges. Technical writer mindsets are the most important in 51-150 employee companies, but on absolute scale they receive the most 5-s in 1000+ employee companies (10%). 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 19% 81% 31% 69% 25% 75% 61% 39%
No
Yes Phone contact Formal meetings held periodically Chat application (Skype, Messenger, etc) We have no direct contact to users
Direct Email Developers
There are various ways to consider experiences. Our one of the most surprising observation was that spending more time at the same working place improves the way of thinking only a little. 1-10 11-50
51-150 151-500 501 - 1000 1000+ Number of employees 1-10 11-50
We were interested in where the experienced employees are working. The respondents having 10+ years of experiences consist of 7% of all employees, 14% have 6-10 years, 26% have 3-5 years and 53% of the employees have less than or equal to two years of experiences. Figure 19 shows the distribution of experiences in various team sizes.
35% 30% 25% 20% 15% 10% 5% 0% 1-3 4-7 8-14 15-30 30+ 10+ year 6-10 year 3-5 year 0-2 year 4,5 4,3 4,1 3,9 3,7 3,5 3,3 3,1 2,9 2,7 2,5 Formal training On the job training Self-study
Trial and error Formal training On the job training Self-study Trial and error
The importance of the mindsets is similar in all experience ranges (Fig. 20(a)). We measured that technical writer mindset gets more important with experience, the management mindset drops back in the 10+ years experience group. The developer and tester mindsets did not change signi cantly with the experiences. (a) The average importance of mindsets by experience (b) Acquiring knowledge by experience groups (in percentgroups (in percentage). age).
In all experience groups the average amount of knowledge, used in work, acquired through on-the-job training/self-study/trial and error is approximately constant, while the average amount of knowledge gained through formal training drops from 24% to 16% at 10+ years of experience (Fig. 20(b)).
We observed as well that the knowledge of design patterns, testing and management techniques known does not depend on the respondents working experiences. However, technical writer techniques knowledge changes with working experience: the importance of user documentation, system documentation and reviews rises until 10 years of experience. After 10 years of experience the importance of user and system documentation no longer increases: reviews and user testing fall back. Proofreading shows an opposite trend: its usage drops back with experience, but after 10 years of experience it becomes the 3rd most known technique.
We examined the experience through the thinking/method related questions. We found that the answers for almost all questions were approximately the same in all experience groups. Until 6-10 years of experience the most improved properties were: understanding who has to carry out the next step (17% increase) and checking how the change was done when the result is not as expected (11% increase). Some properties even fall back: monitoring and evaluating the newest technologies/methodologies (18% drop), detecting if someone is idle for too long (11% drop) and learning why a non-speci c activity is needed (18% drop).
The biggest progression happens between 6 and 10 years of experience: monitoring and evaluating the newest techniques/methodologies (44% increase), extensive testing before introduction (31% increase), learning why a non-speci c activity is needed (30% increase).
The average amount of being idle drops from 12-14% to 4% when reaching 10+ years of experience. The average amount of being overloaded slowly grows from 25% to 31%.
In all experience groups the ratio of respondents using tools to detect and remove anti-patterns was under 2%. The ratio of respondents who know about anti-patterns and try to avoid them was between 20-30% in all experience groups.
From all experience range the employees traced back the issues to the same sources with the same ratio with one exception: only 1 respondents with 10+ years of experience selected user support as the source of problems. He also placed more emphasis on reviews than others with the same or less experience. 0% 20% 40% 60% 80%
100% (a) The distribution of known software design patterns by experience groups. 0% 10% 20% 30%
40% (b) Knowledge of design anti-patterns and experience.
After 10 years of experience all employees are working some time on internal quality improvements. The ratio of regularly improving internal quality was the highest in this group: 50%. At the same time 43% of them is improving internal quality in his free time, while only 30% of people with less experience reported the same.
With the amount of experience the average time spent at manual reviews rises from 40 to 75 minutes. 7
This paper reports on the ndings from a survey conducted on 456 respondents working in software development
projects. In order to better understand how people with di erent roles, experience levels or working in di erent
size communities di er we compared: (
We have seen that software is mostly developed in large companies, by people with little experience (mostly less than 2 years) in their role. According to our survey most respondents turn to the Internet, colleagues or books to learn new skills (RQ3). Formal training is shown as the least e ective form of gaining new knowledge, even trial and error is perceived to be more useful on average (RQ4). For all roles surveyed on the job training and self study were reported to bring the most bene ts.
Among the respondents both developer and tester mindsets are recognized to be very important in software development projects (RQ1). This result re ects, that as software systems grow, their testing also becomes a harder problem, that needs special attention. But while testing techniques are well known, development techniques rank as the least known ones (RQ2). This seems to indicate, that testing is bigger/more important, needs the participation/cooperation of more people, with diverse backgrounds.
All our questions related to scienti c thinking, received answers between 3-4 points on average. There was little di erence when we controlled against role (RQ5), experience level (RQ9) and size of the company (RQ8). Showing that people in di erent roles think alike and/or follow similar processes. The quality of thinking and processes in the case of developers and testers was the most similar: their average answers correlated with a 0:93 coe cient. Some researchers (e.g. [13], [14], [15]) have already noticed a kind of \convergence" between testing and development, our data seems to support.
With experience the usefulness of on the job training raises at the cost of self study. When estimating how scienti c their thinking/processes are managers give the most spread out answers.
We found that most companies support the improvement of internal quality, but most respondents have never heard of or are not concerned about anti-patterns (RQ6, RQ7). Most of the issues are traced back to the early stages of the process and are controlled by static tools, manual code reviews and direct contact to customers.
The responses show, that bigger companies have more career options, more experienced people, better processes, better quality validations and o er better on the job training instead of relying on self-study (RQ8). Bigger companies also use their resources more e ciently, without overloading people more, without forcing people to produce lower quality.
There are two ways to think about our results regarding time. Either our results show that with spending more time at the same place the methods/thinking improves only a little bit, or we are witnessing a slow degradation of methods/thinking since 10 years ago (RQ9). This question needs to be investigated further.
The biggest di erence in how people with 10+ years think better is related to how they monitor newest technologies/methods and how extensively they test them before their introduction. An interesting question can be asked: if formal training is perceived to be the least e ective, people learn the most from the Internet and colleagues during on-the-job training and self-study, the maximum of knowledge (perceived) can be reached in 2 years ... should large companies still expect university degrees or rather recruit people without degrees and train them in-house? 8
This study might su er from the usual threats to external validity. There might be limits to generalizing our results beyond our settings (we only contacted companies with o ces in Hungary).
The questions we have selected to measure how well the various techniques of the di erent roles are known, cover only a small part of the elds knowledge, and these techniques might not be used on a daily level.
We also can not claim to have eliminated cultural biases. In di erent countries, di erent domains and involving di erent forums we might get di erent results (e.g. we noticed too late that user experience related roles were left out).
When we noticed that the number of technical writers lling in our survey might not be enough, to make comparisons, we contacted the Technical Writers facebook group. As they might not be working at companies with o ces in Hungary, this slightly changes the target group of our survey. We believe, that the 28 technical writers who lled in the survey, did not change the results of other groups and provided a better representation of their own group. 9
We would like to thank all those people who o ered help distributing our survey. We would like to thank companies allowing their employees to ll in our survey (for example Ericsson, Nokia, LogMeIn, NSN, SAP, NNG, Prezi, GE). Our thanks goes also to the meetup groups allowing us to reach their members (Test & Tea, Hungarian C++ Community, Budapest DevOps Meetup, Freelancers in Budapest) and to all visitors of the Hungarian IT professionals group at www.linkedin.com, www.hup.hu and www.prog.hu who lled in our survey.
Special thanks goes to the leaders of the Technical Writers facebook group, by whom we were able to reach more technical writers.
Here are the survey questions. The layout below is simpli ed to meet space limitations. We have noted within /**/ comments the di erent types of responses expected if not listed here.
1. Are you working for a multi-national company? (A company present in several countries.) /* yes-no */ 2. How large is the company you are working for? (The number of employees working in your country.) (a) 1-10 employees (b) 11-50 employees (c) 51-150 employees (d) 151-500 employees (e) 501 - 1000 employees (f) 1000+ employees 3. How many people are you working with in your main project?
(a) 1-3 (b) 4-7 (c) 8-14 (d) 15-30 (e) 30+ 4. For how long have you been working in your current position?
(a) 0-2 year (b) 3-5 year (c) 6-10 year (d) 10+ year 5. What is your predominant role or responsibility within your organization?
(a) Development (b) Testing (c) Architect (d) Technical writing (e) Team leadership (f) Project management (g) Business operation/support (h) Executive management (i) Managing of managers (j) Line management (k) Self-employed
(a) Requirement gathering (b) Research (c) System development (d) Writing conceptual information (e) Code editing (f) Code review (g) Deployment (h) Testing (i) Test review (j) Writing documentation (k) Maintenance (l) Managing the environment (m) Managing people (n) Administration (o) Managing Projects (p) Sales 7. What other responsibilities did you have beside your main task in 2014?
(a) Requirement gathering (b) Research (c) System development (d) Writing conceptual information (e) Code editing (f) Code review (g) Deployment (h) Testing (i) Test review (j) Writing documentation (k) Maintenance (l) Managing the environment (m) Managing people (n) Administration (o) Managing Projects (p) Sales A.2
Familiarity with di erent techniques 8. Which of the following software design patterns are you familiar with?
(a) Builder (b) Factory (c) Singleton (d) Decorator (e) Composite (f) Proxy (g) Iterator (h) Chain of responsibility (i) State (j) Visitor (k) Strategy (l) Join (m) Lock (n) Message Design Pattern (o) Monitor (p) None of the above 9. Which of the following testing techniques are you familiar with?
(a) Function testing (b) Boundary value anaysis (c) Decision table testing (d) Pairwise testing (e) Classi cation tree method (f) Statement testing (g) Branch testing (h) Exploratory testing (i) Fault attack with defect checklist (j) Error guessing (k) Cause-e ect graph (l) Use-case testing (m) Path testing (n) Fault injection (o) Control ow analysis (p) Coding standard (q) Code metrics (r) Call graphs (s) Review (t) Walk-through (u) Inspection (v) None of the above 10. Which of the following techniques/methodologies are you familiar with?
(a) Sequential development (b) Waterfall (c) V-model (d) Spiral model (e) Extreme programming (f) Scrum (g) Kanban (h) Agile (i) Test Driven Development (j) Feature Driven Development (k) Acceptance Test Driven Development (l) Continuous Integration (m) Integration Centric Engineering (n) Lean Development (o) 6 Sigma (p) Pair programming (q) CMMI (r) Planning poker (s) Refactoring (t) None of the above 11. Which of the following technical writing techniques are you familiar with?
(a) Analysis of audience (b) Gathering speci c vocabulary (c) Precise expressions (d) Clear design (e) Chain of new concepts (f) Review (g) i18n (h) L10n (i) Survey (j) User documentation (k) System documentation (l) Documentation Life Cycle (m) Problem-Method-Solution (n) Chronological structure (o) User testing (p) Camera-ready (q) S-V-O structure (r) Proofreading (s) Interview (t) Focus groups (u) None of the above 12. In your opinion how important is to have a a developer's mindset for your work? /* marks between 1 and 5 */ 13. In your opinion how important is to have a tester's mindset for your work? /* marks between 1 and 5 */ 14. In your opinion how important is to have a technical writer's mindset for your work? /* marks between 1 and 5 */ 15. In your opinion how important is to have a management mindset for your work? /* marks between 1 and 5 */ A.3
Gaining new knowledge 16. What are your main sources of gaining new knowledge?
(a) Books (b) Research papers (c) Colleagues (d) Classes (e) Trainings (f) Vendor sites (g) Internet forums and blogs (h) Company intranet (i) Conferences (j) Other; 17. Which of the following resources did you use to learn last year?
(a) Books (b) Research papers (c) Colleagues (d) Classes (e) Trainings (f) Vendor sites (g) Internet forums and blogs (h) Company intranet (i) Conferences (j) Other; 18. How much of the knowledge you need in your work have you acquired through formal training? (Percentage between 0 and 100) 19. How much of the knowledge you need in your work have you acquired through job training? (Percentage between 0 and 100) 20. How much of the knowledge you need in your work have you acquired through self-study? (Percentage between 0 and 100) 21. How much of the knowledge you need in your work have you acquired through trial and error? (Percentage between 0 and 100) A.4
Process and methodology related questions 22. In our company we are monitoring and evaluating the newest technologies/methodologies. /* marks between 1 and 5 */ 23. When a new piece of technology/methodology is available we do extensive testing before introducing it into our processes. /* marks between 1 and 5 */ 24. When a new activity/artifact is de ned we establish sets of hypotheses that can be tested before work starts.
/* marks between 1 and 5 */ 25. When an activity is not done as speci ed we follow a de ned process to improve. /* marks between 1 and 5 */ 26. When we see a defective outcome despite all activity done as speci ed, we modify the processes. /* marks between 1 and 5 */ 27. In 2014 in my opinion I was idle ...% of my time: /* asking for percentage */ 28. As far as I can tell, when someone is idle for long, our team is able to detect the situation and follow a de ned process to modify or reassign activities. /* marks between 1 and 5 */ 29. In 2014 in my opinion I was overloaded ...% of my time: /* asking for percentage */ 30. As far as I can tell, when someone is overloaded for long, our team is able to detect the situation and follow a de ned process to modify or reassign activities. /* marks between 1 and 5 points */ 31. If we nd that a non-speci c activity is needed, we learn why it is needed and redesign our processes. /* marks between 1 and 5 points */ 32. In most cases in the processes we follow I nd it clear what the next activity is . /* marks between 1 and 5 points */ 33. I nd it clear who has to carry out the next activity in the processes we follow. /* marks between 1 and 5 points */ 34. When we plan to make a change we asses the current state of a air with scienti c rigor. /* marks between 1 and 5 points */ 35. When the result of a change is di erent from the expected we check how the change was done, what e ects it had and redesign the change if needed. /* marks between 1 and 5 points */ A.5 36. How familiar are you with design anti-patterns?
(a) I have never heard of them (b) I have heard of them, but I'm not sure what they are (c) I know of them, but I'm not very concerned of them appearing in my work (d) I know and try to avoid them (e) I have a strong understanding and frequently use tools to detect and remove anti-patterns 37. How concerned are you about the presence of anti-patterns in your products? /* marks between 1 and 5 points */ 38. How often do you work on existing products to improve their internal quality without changing their external behaviour?
(a) Never (b) Seldom (c) Sometimes (d) When absolutely necessary (e) On a regular basis (f) Such work is planned and done a formal activity. 39. Is working on existing products to improve their internal quality supported by your organization? (Only internal quality, without changing external behaviour)
(a) No (b) In theory (c) Tools are available (d) When we have free time (e) We have allocated time for this kind of work in our processes 40. If internal quality improvement is done, when is it done?
(a) We don't perform internal quality improvements (b) When there are issues, we correct it (c) When we notice a possibility to improve we take it immediately (d) We have pre-planned sessions and work lists for internal quality improvements 41. During 2014, how many times did you have to produce solutions you felt they were of low quality in order to satisfy short term needs?
(a) Never (b) 1-2 times (c) 3-5 times (d) 6-10 times (e) 10+ times A.6
Static analysis and traceability 42. Which tool supported static analysis techniques are used in your organization?
(a) Checking of static metrics (b) Checking of coding standards (c) Control ow analysis (d) Data ow analysis (e) Other tools supporting static analysis (f) Our techniques are not tool supported 43. Do you have manual code reviews for internally developed products? /* yes - no question */ 44. How long does a manual review take? (In minutes): /* expecting a number */