Search Close
Log inRegister

You are here: SE » ResearchHome » ErrorHome

Error prevention in software development

09 Aug 2023 - 17:03 | Version 27 |

In recent years, progress in quality assurance for software products has mainly been made in the area of testing (i.e. executing the software with the aim to cause failures which in turn are signs for defects) and static code analysis. Inspection techniques seek defects in software artefact or any other intermediate document directly. As it is, quality assurance departments in software organizations recognize their job mainly as "hunting for bugs". Of course, after finding such problems they still need to be resolved. It's a strange phenomenon that software engineering seems to accept software defects as to be removed after being created by professionals.

In contrast, our research on error prevention focuses on prediction defects from being inserted in the first place. We assume that
  • defects are actively being built into software, because programmers - being humans - make errors
  • making errors is not bad luck and therefore defects do not occur by chance and should not be treated by statistical means only
  • defect prevention, i.e. prevention or awareness of making human errors, is mainly a task of process observation and process adjustment
  • analysis of defects' root causes is a matter of empirical investigation done on site

In the first step we focus on the study of coding process, but generally design activities share similar properties. The research is currently guided by three ways of investigation.
  • Analysis of programmers' activity ("micro-process") to examine faulty behavior
  • Review of psychological based research on programming and human error
  • Classification of failures, defects, errors and misbeliefs in software development.

Some terminology

  • Failure (in German: Versagen): An event during software execution which can be classified as being inconsistent with the expected and intended behaviour. All quality criteria not being met (e.g. stability, usability, scalability etc.) are classified as failures, not only system crashes or wrong results.
  • Defect (in German: Defekt): A defect is a structural weakness of an artefact like code or design specifications. Failures may be caused by one or more defects but defects do not necessarily cause a system to fail. Opposed to failures a defect is a property or state, not an event.
  • Error (in German: Fehler): A human action causing a defect. Note that this characterization sees the software engineer as the main actor. We do not investigate human error while actually using a system. Errors are events just like failures.
In short, errors cause defects (immediately) and defects may cause failures (usually not immediately). To stress the similarity, errors can also be termed as "human failures". Consequently, we add the equivalent to the "human defect", the
  • Misbelief (in German: Irrtum): Root cause of the occurance of an error, usually in the engineers mind but including all kinds of slips and lapses, like typos or mix-ups. ("Misbelief" is not an official term but newly introduced here.)

Note that the research's subject is "*Error* prevention...". We focus on errors, not on defects. And we focus on suppoting the programmer to learn about her own errors to avoid them later.

In the past, lots of quality aware process improvements have been suggested including Root Cause Analysis, Orthogonal Defect Classification and similar. These focus on analysing defects which have been found during quality assurance with the result of suggestions for process improvements.

Additionally, we benefit from improvements of engineer's tools and techniques, for example

A special case is the Personal Software Process, part of which is concerned with defect analysis which indeed fulfils the four assumptions mentioned above but is proved as being too challenging to implement in software development departments.

Other like the Cleanroom Software Development are based on formal methods to prevent defects from the very beginning of development by reducing errors in the process of translating requirements to computer-executable code.

Capturing the micro-process of software development (activity analysis)

First step is to analyse which activities (in German: Tätigkeiten) are performed during programming (mostly coding; we will analyse design aspects later) to get a feeling for typical patterns of work and typical situations where code was written, which later was classified as defective. In short, with doing micro-process analysis we'll define the vocabulary to describe problematic coding situations. The long-term objective is to automatically detect those situations and warn the programmer just-in-time.

In the first step, we will concentrate on the task of programming. With ongoing investment in formalizing the design process and in using tools to support the construction of well-defined design documents while keeping them in sync with the code, the ideas presented here can be extended to design activities of software development as well.

The vocabulary for describing the micro-process of coding will be based on events occurring on some code part, like
  • Changing code parts
  • Browsing through code
  • Executing the program
  • Pausing for a while, etc.

These events will be grouped to typical episodes like
  • Trial-and-error cycles (which itself is a special kind of removing defects)
  • Copy code, paste it and change it (as a way of changing code fragments)
  • Interrupted work, etc

The aim is to log events and episodes and find statistically relevant relations to introduced defects, i.e. identifying typical patterns of defect insertion. For example, typical “anti-patterns” and indicators are
  • Being interrupted and not reentering the work where it was left
  • Changing a small part of code very often
  • Not changing copied code fragments correctly, etc

By carefully examining the cause it will be possible to predict errors and therefore prevent resulting defects. At least it will be possible for the developer to reinvestigate how she introduced the defect, i.e. performing some root cause analysis, which may result in some micro-process improvement.

For a more detailed discussion on micro-processes in software development, please see our special pages at MicroprocessHome.

Related work on analysing programmer's activity include:

Psychology of programming and human error

Human errors mark the subset of human actions which are faulty or wrong. Faulty actions simply have been weakly performed although the initial intention was correct. For example, in order to enter a room, one may try to push the door instead of pulling it. Wrong actions on the other hand have been performed well, but were unsuited for the situation they occurred in. For example, opening the door of a car while driving with high speed is a bad idea. Programmers usually perform lots of faulty or wrong activities.

Psychology provides some in-deep studies of human error with the aim to explain why they occur. Although explanation is not our focus, it may give some hints for error prediction. Additionally, errors have been classified, although it's been mainly done for actions (in German: Handlungen) performed by the human body, but human errors in reasoning have also been classified, especially in planning tasks. All this may provide us with good ground to re-phrase these psychological motivated descriptions for the special task of programming. For example, the error provoking effect on interruptions have been studied considerably. As another example, it has been observed that man only remembers 7 +- 2 items in short term memory which may limit the programming activity a lot. So far, only little research has systematically investigated human error and misbelief as a cause for defects in software development.

Amoung others, following references provide deeper understanding.

There's been a lot of psychological studies on the micro-process of coding, code comprehension, design, and others. Please see MicroprocessHome for details.

Classification of failures, defects, errors, and misbeliefs

Classification of defects has been provided by many of the efforts on preventing defects but no agreement on taxonomies exists. A classification will help us focus on important types of defects and errors and gives insight into mechanisms why defects occurred (i.e. which error was performed). The weakness of many classifications is that they do not clearly distinguish between defects and errors. Some even mix up failures and defects. We will provide an alternative DefectTaxonomy which suits our needs, i.e.
  • Raises the difference between failure, defect, error, and misbelief
  • Classifies defects due to the code change done to remove the defect as well as the causing error
  • Takes the phase of the development process into account in which the defect was introduced

The kinds of failure, i.e. bad computation, endless loops, or accessing protected memory are seldom of interest. A lot of effort has been made in classifying defects, although many did not separate defect and human error very well. As mentioned above, defects can be described by categorizing the code change which removes it, covering simple "off-by-one" loop condition defects as well as redesign of complete parts to resolve concurrency problems, for example.

Lots of related work can be presented here, like

Contact

In case of any question or suggestion please get into contact with SebastianJekutsch.

SWTIDSR

Sitemap

ABI
AGDepSys
AGGeom
AgMathLife
BeGenDiv
Cluster
CompMolBio
CompStatPhys
DiBiMath
DILIS
GeschichteDesComputers
HCC
Inf
Internal
IT
Main
Math
Mi
SE
System
Tec
Freie Universität Berlin, Institut für Informatik