Towards a theory of software development expertise Baltes et al., ESEC/FSE’18
This is the last paper we’ll be looking at this year, so I’ve chosen something a little more reflective to leave you with (The Morning Paper will return on Monday 7th January, 2019). The question Baltes and Diehl tackle is this: “How do you get better as a software developer?” What does expert performance look like?
We present a first conceptual theory of software development expertise that is grounded in data from a mixed-methods survey with 335 software developers and in literature on expertise and expert performance…. [the theory] describes central properties of software development expertise and important factors influencing its formation.
In essence, ask a bunch of practitioners what they think, use a disciplined coding scheme to interpret the answers (a “grounded theory”), and then layer in what we know about expertise and expert performance in general. The end result is a “conceptual theory” that shows the various contributors to expert performance and the relationships between them. “Software Development” in the current work is synonymous with “programming.”
To make the paper come alive you need to engage with it a little: Does the theory developed by the authors make sense to you? What’s missing? How would you weight the various factors? How could you apply this on a personal level in 2019? How could this be applied in your team or organisation to raise the collective level of expertise next year?
Software developers can use our results to see which properties are distinctive for experts in their field, and which behaviors may lead to becoming a better software developer…. Employers can learn what typical reasons for demotivation among their employees are, and how they can build a work environment supporting the self-improvement of their staff.
A grounded theory
The first phase involved sending a questionnaire to users active on both GitHub and StackOverflow between Jan 2014 and October 2015. The questionnaire was sent to 1,000 individuals, and received 122 responses.
(Enlarge)
The grounded theory (GT) coding exercise was then used to generate a theory from the qualitative data:
… the process of coding assigns “summative, salient, essence-capturing” words or phrases to portions of the unstructured data. Those codes are iteratively and continuously compared, aggregrated, and structured into higher levels of abstractions, the categories and the concepts. This iterative process is called constant comparison.
(Aside: it strikes me that the body of work on grounded theory development might be very interesting to study from the perspective of domain-driven design and the building of a ubiquitous language.)
After much distillation, the model comes out looking like this:
The grounded theory describes software development expertise as a combination of a certain quantity and quality of knowledge and experience, both general and for a particular language. The work context, behavior, character traits, and skills influence the formation of expertise, which can be observed when experts write well-structured, readable, and maintainable source code.
You’ll know an expert programmer by the quality of the code that they write. Experts have good communication skills, both sharing their own knowledge and soliciting input from others. They are self-aware, understanding the kinds of mistakes they can make, and reflective. They are also fast (but not at the expense of quality).
Experience should be measured not just on its quantity (i.e., number of years in the role), but on its quality. For example, working on a variety of different code bases, shipping significant amounts of code to production, and working on shared code bases. The knowledge of an expert is T-shaped with depth in the programming language and domain at hand, and a broad knowledge of algorithms, data structures, and programming paradigms.
A preliminary conceptual theory
The next phase was to take the grounded theory and embed it within the existing literature on expertise and expert performance, for which the main resource used was ‘The Cambridge Handbook of Expertise and Expert Performance’.
This handbook is the first, and to the best of our knowledge most comprehensive, book summarizing scientific knowledge on expertise and expert performance.
The result of this process is a preliminary conceptual theory that looks like this:
Acquiring expertise is not exclusively a cognitive matter, personality and motivation influence behaviours that may or may not lead to improvements of expertise. The work context, including team members, managers, and customers, can also influence the behaviour of a developer, and this can also vary according to the type of task being undertaken.
Reaching true experts levels requires deliberate practice combined with monitoring, feedback, and self-reflection.
Deliberate practice
Having more experience with a task does not automatically lead to better performance. Research has shown that once an acceptable level of performance has been attained, additional “common” experience has only a negligible effect, in many domains the performance even decreases over time. The length of experience has been found to be only a weak correlate of job performance after the first two years.
Deliberate practice is required to become an expert: prolonged efforts to improve performance while continuously increasing the difficulty and centrality of development tasks.
…studies have shown that deliberate practice is necessary but not sufficient to achieve high levels of expert performance— individual differences also play an important role.
Monitoring, feedback, and self-reflection
Deliberate practice requires a way of monitoring performance, which could be e.g. from a teacher, coach, mentor, or peer: “the more channels of accurate and helpful feedback we have access to, the better we are likely to perform.“. Monitoring and self-reflection also influence motivation and consequently behaviour.
The full conceptual theory
For the third and final phase the authors sampled two additional programmer populations, active Java developers, and very experienced developers, with the goal of further elaborating and refining the categories and relationships in the theory.
The final resulting model looks like this:
(Enlarge)
The most frequently cited tasks that an expert should be good at were designing software architecture, writing source code, and analysing and understanding requirements. Within the software architecture task, understanding modularisation and decomposition were frequently mentioned.
In terms of personality traits, experts should be open minded and curious, be team players, and pay attention to detail. Patience and self-reflection were also cited. In terms of general skills, “problem solving” came top of the list under which analytical thinking, logical thinking, and abstraction/decomposition all feature. Another important skill is being to assess trade-offs.
Mentors should be guiding, patient, and open-minded. Participants were most motivated by mentors that posed challenging tasks.
To facilitate continuous development of their employee’s software development skills, (employees suggested that) employers should:
- Encourage learning (e.g. training courses, conference attendance, and access to a good analog or digital library)
- Encourage experimentation (e.g. through side projects and by building a work environment that is open to new ideas and technologies)
- Improve information exchange between development teams, departments, and even companies. E.g. lunch and learn sessions, rotation between teams, pairing, mentoring, and code reviews.
- Grant freedom (primarily in the form of less time pressure) to allow developers to invest in learning new technologies or skills.
In contrast, non-challenging or routine tasks result in demotivation. Other causes of performance decline over time are lack of a clear vision or direction, absence of reward for quality work, stress in the work environment, and bad management or team structure.
Your turn
How will you ensure that in 2019 you grow your expertise, and not simply add another year of (the same or similar) ‘experience’ ?
See you in January! Thanks, Adrian.